2015.07.26 Exploring Clojure in the REPL

When I first learned Linux, my friend taught me a few commands. How to to list files, how to change directories and see your current directory, how to run things, how to ask a program what parameters it takes, and how to look at a file (ls, cd, pwd, ./foo or /usr/bin/foo or foo, foo --help, cat foo or more foo). After that I just... ran a lot of stuff. I eventually discovered 'man' -- but going through /bin and running ALL the commands was really educational.

Let's get some basic tools to do that with the Clojure REPL! The major things we want to do are find things (mostly functions), and then learn all about them individually.

Finding Things

So Clojure has namespaces, which are kinda like directories, using "." as a way to indicate nesting. Vars (and thereby named functions) are kinda like files. So the full "path" to a var looks like "clojure.string/split" where "clojure.string" is the namespace and "split" is the function.

First let's get a list of all the loaded namespaces. The "ns-all" gives us this, but we want a nice sorted printout, so we'll add a bit of fancy.

user=> (doseq [n (sort (map ns-name (all-ns)))] (println n))
; ... trimmed ...
clojure.core
clojure.core.protocols
clojure.inspector
; ... trimmed ...
clojure.repl
clojure.set
clojure.stacktrace
clojure.string
clojure.template
clojure.test
clojure.tools.cli
clojure.tools.nrepl
; ... trimmed ...
user


Use "dir" to look at all the exposed vars from a namespace

user=> (dir clojure.string)
blank?
capitalize
escape
join
lower-case
re-quote-replacement
replace
replace-first
reverse
split
split-lines
trim
trim-newline
triml
trimr
upper-case


Search for functions with "split" in their name. There is also "apropos" but it doesn't show namespaces -- "apropos-better" does, and "find-name" is an alias of that, so is better.

user=> (find-name "split")
(split-at split-with clojure.string/split clojure.string/split-lines net.cgrand.parsley.grammar/split-empty-prods)


You can also use a regex! Not sure how useful this is... but here we are getting all the functions that start with an "s" and in with a "t".

user=> (find-name #"^s.*t$") (set short sort sorted-set spit split-at struct clojure.set/select clojure.string/split clojure.test/set-test net.cgrand.sjacket/shift net.cgrand.sjacket/shift-right net.cgrand.sjacket/str-pt net.cgrand.sjacket/subedit)  Learning About Things Once you've found your function, you'll want to learn more about it and maybe give it a try. "doc" can be used to get the documentation associated with a function. This is pretty cool! It shows the full name of the function, its signature, and it's documentation. user=> (doc print) ------------------------- clojure.core/print ([& more]) Prints the object(s) to the output stream that is the current value of *out*. print and println produce output for human consumption.  Note that you can tab-complete user=> (doc print<tab> print print-ctor print-dup print-method print-simple print-str printf println println-str  If you don't know what you're looking for, you can also try find-doc. This will search both the name and the documentation itself for your string or regex. user=> (find-doc "split") ; ..... too long to include ;)  THE ULTIMATE: Get the source for a function! user=> (source print) (defn print "Prints the object(s) to the output stream that is the current value of *out*. print and println produce output for human consumption." {:added "1.0" :static true} [& more] (binding [*print-readably* nil] (apply pr more)))  Not everything is a function. Heck, sometimes you might not know what sort of thing you're looking at. But we can find out. user=> (type 5) java.lang.Long user=> (type 5.2) java.lang.Double user=> (type "hello") java.lang.String user=> (type 'hello) clojure.lang.Symbol user=> (type {:a 5, :b 6}) clojure.lang.PersistentArrayMap user=> (type [1 2 3]) clojure.lang.PersistentVector user=> (type type) clojure.core$type
user=> (type dir)
CompilerException java.lang.RuntimeException: Can't take value of a macro: #'clojure.repl/dir, compiling:(/tmp/form-init5987247545872948247.clj:1:1)
user=> (type clojure.string/split)
clojure.string$split user=> (defn say-hi [] (println "hi!")) #'user/say-hi user=> (type say-hi) user$say_hi


Getting GUI

Clojure comes with clojure.inspector, which gives some Swing GUI for exporing data structures.

user=> (use 'clojure.inspector)
user=> (inspect-tree {:a 1 :b 2 :c [1 2 3 {:d 4 :e 5 :f [6 7 8]}]})


NOTE: I had to do this to get jdk-1.7 swing apps to display while using xmonad (and other tiling window managers I guess). I now dropped this in my ~/.zshrc. Uhg.

export _JAVA_AWT_WM_NONREPARENTING=1


Doing Horrible Things

So let's have some fun. First, let's get all the string function names. "dir-fn" is like dir, but returns the results instead of printing them.

user=> (def string-funcs (clojure.repl/dir-fn 'clojure.string))


Let's turn them into their actual functions.

user=> (def real-string-funcs (map #(resolve (symbol (str "clojure.string/" %))) string-funcs))


Given a function, here is how to tell how many parameters it takes (well... it might take more or less than this, but this will tell us how many params for the first definition. Good enough).

user=> (defn arg-count [f] (count (first (:arglists (meta f)))))


Here is a handy thing that tells us if the given function can deal with a single arg.

user=> (defn single-arg? [f] (= 1 (arg-count f)))


Now let's filter our string functions down to the ones that just take one argument.

user=> (def one-arg-funcs (filter single-arg? real-string-funcs))


So. We if have a one-argument function in the clojure.string namespace, let's assume it takes a string and pass in a string to it :) . The "pr-str" bit is so that we can pretty-print this with escaped newlines and such later.

user=> (defn one-arg-result [f] (pr-str (apply f ["Hello\nworld\n"])))


Now let's try this on all the one-argument string functions, printing out the function name and the result pretty like.

user=> (map #(println (:name (meta %))  "\"Hello\\nworld\\n\" ->" (one-arg-result %)) one-arg-funcs)
blank? "Hello\nworld\n" -> false
capitalize "Hello\nworld\n" -> "Hello\nworld\n"
join "Hello\nworld\n" -> "Hello\nworld\n"
lower-case "Hello\nworld\n" -> "hello\nworld\n"
re-quote-replacement "Hello\nworld\n" -> "Hello\nworld\n"
reverse "Hello\nworld\n" -> "\ndlrow\nolleH"
split-lines "Hello\nworld\n" -> ["Hello" "world"]
trim "Hello\nworld\n" -> "Hello\nworld"
trim-newline "Hello\nworld\n" -> "Hello\nworld"
triml "Hello\nworld\n" -> "Hello\nworld\n"
trimr "Hello\nworld\n" -> "Hello\nworld"
upper-case "Hello\nworld\n" -> "HELLO\nWORLD\n"


Woo!

2015.06.04 Polyglot Bridges

Here are the slides for Polyglot Bridges (pdf) lightning talk that I gave today at RubyNation.

The most important bit:

# Shell
sudo apt-get install matplotlib
gem install rubypython
# pending a small patch for debian/ubuntu 64bit

# Ruby; even works in the REPL!
require 'rubypython'
RubyPython.start
plt = RubyPython.import('matplotlib.pyplot')

plt.plot([1,2,4,6,28,50,234])
plt.ylabel('some numbers');
plt.show


Or in other words: pull in this rubypython gem, magically get all the python libs callable from Ruby. In this case, you get all the nice charts and graphs from pyplot. I originally played with this in Perl6/Rakudo Inline::Python, where it also works fantastically.

I love the idea of one community NOT completely re-writing a library in their language when it can just be used "directly" instead.

Today I'm playing with Open Sound Control (OSC), which is more general-purpose than midi but kinda similar idea. I have an android app on my phone named Control which has a few existing UIs, in my case I'm playing with the multi-touch. I set it to have two touch inputs.

In my Overtone REPL, I first set it up to listen for OSC events, and dump out whatever events it sees:

(def server (osc-server 44100 "osc-clj"))
(osc-listen server (fn [msg] (println msg)) :debug)
; Turn off with: (osc-rm-listener server :debug)


I got both my laptop and phone on the wifi (which allows peer-to-peer communication) and told the Control app to connect to my laptop server (192.168.0.15 port 44100, in this case). Now when I touch the screen I get a bunch of messages about the generated OSC events. They look like:

{:src-port 45161, :src-host 192.168.0.25, :path /multi/1, :type-tag ff, :args (0.36825398 0.3961456)}


From this I see that the path I want to react to is "/multi/1" and "/multi/2". Dragging my fingers around, the args are the x and y coordinates normalized to (0..1). For now I'll just make it beep a bit when it gets an event. I'm hooking the first touch up so the X axis generates a frequency, and the second touch up so that the Y axis generates a frequency (just as a test), giving:

(osc-handle server "/multi/1" (fn [msg]
(let [
x (nth (:args msg) 0)
y (nth (:args msg) 1)]
(demo 0.1 (saw (+ 100 (* 100 x))))
)
))

(osc-handle server "/multi/2" (fn [msg]
(let [
x (nth (:args msg) 0)
y (nth (:args msg) 1)]
(demo 0.1 (saw (+ 100 (* 100 y))))
)
))


Each touch plays for 1/10th of a second, so if I drag around these overlap a bunch. In fact, if I drag around a lot then something gets backed up and the sound is delayed.

Good progress today -- I made a hack to get around that BUNCHES of midi issue. Needed that because when I would hit a single note on my jack-keyboard (virtual midi keyboard), I got a whole bunch of events. I'm guessing I was getting 64 of them.

I also got Overtone Vim Integration working. Well technically I already had it working (had fireplace.vim installed, etc), I just didn't know it. Now I can start up the REPL in one terminal and start up vim in another. In vim do ":Require" and it finds the running REPL and hooks into it. I also added "nnoremap <F1> :Eval (stop)<cr>" to my .vimrc, so that I can jam on F1 to make the noise stop.

Here is the file I've got now:

(ns noise.core)
(use 'overtone.live)
(use 'overtone.inst.piano)

(on-event [:midi :note-on]
(fn [e]
(let [note (:note e)
vel  (:velocity e)
device-name (:name (:device e))]
(if (= "VirMIDI [hw:1,0,1]" device-name)
(piano note vel)
()
)
)
)
::midi-keyboard-handler
)


So that takes midi events specifically from that device (I picked one of my 64 virtual devices at random, seems to work) piped to the piano instrument. Next I'm going to make a new instrument of my own and hook that in!

Today I'm watching Overtone and ClojureScript which is a coding session of someone setting up an web UI to play Overtone stuff. Building a UI like this reminds me of the one-string guitar that I got at an art festival the other day. Clearly home-made, including an energy drink as the echo chamber. And very awesome. Also fun to see someone iterate through their development. Lots of interesting things in there, most of them I can read more or less but doesn't mean I could write them. One thing I noticed was (:use ...) to pull in instrument libs. I'll try that.

The Cheat Sheet is interesting, but I don't know enough to actually use a lot of the things on there. Current mission is to get the instruments working.

Ah! I need to do (use 'overtone.inst.piano) not (:use overtone.inst.piano).

• Now (piano) plays a note!
• (odoc note) shows that it takes a bunch of params
• (piano 60), (piano 65), etc plays some nice piano notes
• I looked up named parameteters in clojure. (piano :note 65) does what I want
• Achievement unlocked ... I can now use all the intstruments on the cheat sheet!
• Now when I do "(piano :note 60) (piano :note 65) (piano :note 69)" (still in REPL) it plays all three notes at once. Need to learn how to sequence things.
• I think I'll go back to that workshop video
• At 13:40 they talk about sequencing things
• Play a piano note in 1 second (at (+ 1000 (now)) (piano))
• From their tutorial, I created:
• (defn loop-beats [time] (at (+ 500 time) (kick)) (at (+ 1000 time) (open-hat)) (apply-at (+ 1500 time) loop-beats (+ 1500 time) []))
• (loop-beats (now))
• (stop)
• I don't know exactly what that last [] is for. (odoc apply-at) says it is "argseq" but gives no explanation.
• In general this is "temorial recursion" -- delays a bit before calling back
• As far as I can tell this _does_ create new stack frames! Will eventually run out. I know this because I accidentally put the loop-beats recursion in parens which invoked it.
• Following the tutorial, I transformed it to use a metronome:
• (def metro (metronome 180))
• (defn loop-beats [m beat-num] (at (m (+ 0 beat-num)) (kick)) (at (m (+ 1 beat-num)) (open-hat)) (apply-at (m (+ 2 beat-num)) loop-beats m (+ 2 beat-num) []))
• Get it started: (loop-beats metro (metro))
• Change the temp live! (metro :bpm 200)
• Stop it (stop)
• Instruments take 'notes' (midi numbers), though some things take hz
• Symbolic notes can be turned into midi numbers via (note :a4)
• Can map a sequence of symbolic notes to midi numbers (map note [:e4 :d4 :c4])
• Now I can make a general purpose play-note-sequence function. I'll just make it play on the beat.

(defn play-song [metro beat-num notes]
(map-indexed (fn [index n]
(at (metro (+ index beat-num)) (piano :note (note n)))
) notes)
)


• (play-song metro (metro) [:e4 :d4 :c4 :d4 :e4 :e4 :e4 :a0 :d4 :d4 :d4 :a0 :e4 :e4 :e4 :a0])
• OK. Midi time! Looking at Overtone wiki MIDI page
• Fired up jack-keyboard (software midi keyboard) since I'm not at home
• Apparently software midi doesn't get immediately detected. "modprobe snd-virmidi" gave me something I can connect to in jack
• Now I have BUNCHES of midi connections when I do (midi-connected-devices) -- like 64 of them. Too many.

1 Comment.

test comment

-- awwaiid 2015-03-12 11:47 UTC

More...

META INFORMATION: This is the technical blog and wiki of Brock Wilcox (awwaiid). Entries focus on my current projects, interests, and sometimes life events. If you'd like you can check out the list of All Entries or the RSS Feed. I also have a LiveJournal syndication feed for LJ friends.

2015-08-26

2015-08-25

2015-08-19

2015-08-17

2015-08-13

2015-08-29

2015-08-28

2015-08-23

... more changes