Posted on Tuesday the 9^th of February, 2010

I think that SuperCollider is one of the most exciting music technologies currently available. It’s super powerful, Open Source, and has even got a separate server and language runtime giving everyone the opportunity to experiment with different approaches. I’m very excited to see where we go with it, and how I can utilise it with my Polynome project.

However, despite the fact that it comes shipping with a very nice live-coding enabled interface, I use Emacs, and well, I love using Emacs for editing text of all kinds. I therefore wanted to have the convenience of the live editing with the powerful text manipulation features that Emacs offers. Luckily there’s an Emacs mode for SuperCollider and I just finished hooking it all up. Unfortunately I couldn’t find a nice tutorial for doing it, but I did manage to piece together a process that finally worked for me. It wasn’t a fun time, so I’m documenting it here in the hope that someone else might benefit from my pain.

Caveat and Context

Now, I need to issue a major caveat: there’s lots of moving parts in this situation, and your parts might not even be the same as mine. So, just to be clear - this might not work for you, but hopefully it’ll get you a major part of the distance. The more similar our setups, the more likely this will help, so for clarity I’m using:

• Cocoa Emacs 23.1.50 Git 2010-02-09 (the latest nightly build at the time of writing)
• SuperCollider 3.3.1 (the Mac version)
• OS X 10.6

I’m also assuming that you have already installed Emacs and SuperCollider and that you have a basic working knowledge of Emacs configuration, and general UNIX hockery pokery¹.

Update your Path (UNIX-style)

First up open your shell config file (~/.zshrc in my case) with your favourite editor² and append /Applications/SuperCollider to your PATH. Check to see if it worked:

λ sclang -h
Usage:
   sclang [options] [file..] [-]

Options:
   -d <path>                      Set runtime directory
   -D                             Enter daemon mode (no input)
   -g <memory-growth>[km]         Set heap growth (default 256k)
   -h                             Display this message and exit
   -l <path>                      Set library configuration file
   -m <memory-space>[km]          Set initial heap size (default 2m)
   -r                             Call Main.run on startup
   -s                             Call Main.stop on shutdown
   -u <network-port-number>       Set UDP listening port (default 57120)

Update your Path (Cocoa-Emacs-style)

Ok, you can do a few things here, particularly because Emacs on OS X does very funky things with the PATH variable:

• In a fresh terminal update the OS X plist version of the PATH to match your newly updated version: defaults write $HOME/.MacOSX/environment PATH "$PATH" and restart your machine.
• In your emacs config explicitly create a PATH variable: (setq path "/Applications/SuperCollider:/rest/of/PATH")(setenv "PATH" path)
• Again, in your Emacs config, add SuperCollider to your exec-path: (push "/Applications/SuperCollider" exec-path)

Not all of these may be necessary; I did the the last two.

Creating a Fresh SCClassLibrary

Create a new place for a fresh copy of SCClassLibrary:

 mkdir ~/.sclang

Copy the SCClassLibrary from your SuperCollider install into this new place:

 cp -R /Applications/SuperCollider/SCClassLibrary ~/.sclang

Also, copy across the plugins directory:

 cp -R /Applications/SuperCollider/plugins ~/.sclang

Finally, copy across the scsynth binary (needed to boot the server and make actual sounds):

cp /Applications/SuperCollider/scsynth ~/.sclang

/Applications/SuperCollider/README\ SCLang\ OSX advises you to edit the startup method in ~/.sclang/SCClassLibrary/Platform/osx/OSXPlatform.sc. Open it up and find the startup method. Delete everything from startup { to the closing } and replace it with the following:

startup {
            if ( this.hasFeature( \emacs ) ) {
                            Document.implementationClass.startup;
            };
            this.loadStartupFiles;
}

Grab a copy of scel

Now we’re ready to power up Emacs with SuperCollider goodness. I found a copy of scel on github that contained a few issues that I ironed out in my own clone³. Feel free to grab it and add it to your own emacs config. That might be as simple as downloading the tarball, or if you manage your config with git (which, by the way, is a great idea) then you could just add it as a submodule. I happen to have based my config on Phil Hagelberg’s wonderful emacs starter kit. So this can be as simple as:

cd ~/.emacs.d
git submodule add git://github.com:samaaron/scel.git vendor/supercollider

Copy scel’s sc files to your new SCClassLibrary

For scel to work correctly it has a number of SC Class files that need to be compiled and loaded when sclang is started. In order to achieve this copy the sc files from scel’s sc directory into your new SCClassLibrary:

cp -R ~/.emacs.d/vendor/supercollider/sc ~/.sclang/SCClassLibrary/Common/Emacs

Configure Emacs

Now you need to teach emacs about the presence of scel. In your emacs config add the following:

(add-to-list 'load-path "~/.emacs.d/vendor/supercollider/el")
(require 'sclang)

Finally, you need to tell it where to find your SCClassLibrary and other bits and bobs. scel advises you to use M-x sclang-customize, however I just set the variables by hand:

(custom-set-variables
'(sclang-auto-scroll-post-buffer t)
'(sclang-eval-line-forward nil)
'(sclang-help-path (quote ("/Applications/SuperCollider/Help")))
'(sclang-runtime-directory "~/.sclang/"))

Start up the Emacs SC Workbench

OK, well done for making it so far. I’m praying for you that things went hunky-dory. At this stage I simply restarted Emacs and hit M-x sclang-start and BOOM entered the SC Workbench. If this seems to work, and you see two buffers, a workbench and a log, try typing "Hello from Emacs".postln; in the buffer and with the cursor over it hit M-x sclang-eval-line. You should see the message pop up in the log. Finally, try booting the server with M-x sclang-server-start. You should be good to go.

Go grab a cup of tea, you’ve earned it.

Posted on Wednesday the 23^rd of December, 2009

Introducing the IokeLexer

It seems I just can’t stop writing syntax highlighters for Ioke. I started with adding support for TextMate, and then basic support for Geshi. Now, as a present for its P release¹ I’ve added Pygments to the list:

Pygments IokeLexer

One of the things that has bugged me since Ioke’s birth is that its incubator, GitHub has not been able to do the honourable thing and highlight it appropriately. Instead, one of the most exciting languages around at the moment has been treated like dirty plain text.

Well, hopefully this can change soon. GitHub happens to use Pygments to highlight its source. Therefore I just need to throw my IokeLexer at the Pocoo chaps, hope they include it in the next version of Pygments and then wait for GitHub to upgrade. At least someone doing the hard work of writing the lexer won’t be the blocker any more…

Just to show its glory², here’s the introduction example from http://ioke.org

#!/usr/bin/ioke

Ioke = LanguageExperiment with(
  goal: :expressiveness,
  data: as(code),
  code: as(data),
  features: [
    :dynamic,
    :object_oriented,
    :prototype_based,
    :homoiconic,
    :macros
  ],
  runtimes:(JVM, CLR),
  inspirations: set(Io, Smalltalk, Ruby, Lisp)
)

hello = method("Every example needs a hello world!",
  name,
  "hello, #{name}!" println)

Ioke inspirations select(
  features include?(:object_oriented)
) each(x, hello(x name))

Lessons Learned

OK, so apart from a fairly high number of hours battling with Python regular expressions and Ioke syntax, here’s a summary of the lessons I learned from this experience:

• You can get pretty far with plain regexps.
• You can get a long way with regexps and stackable contexts.
• Writing approximately correct lexers with regexps can be fun.
• Writing accurate lexers with regexps is for masochists, seriously.
• Look ahead and behind regexp matchers are truly wonderful.
• Writing a regexp lexer really helps you to hone your regexp skills.
• Pygments is nice software.
• Python people can be a little curt yet are very helpful nevertheless.
• Nothing beats a real lexer for lexing.

Finally, there really should be a standard syntax for this kind of thing so when the next syntax highlighting engine comes around we don’t have to continue to re-invent the wheel.

Posted on Tuesday the 24^th of November, 2009

Figure 1. A tortoise hand shadow from Henry Bursill’s Hand Shadows To Be Thrown Upon The Wall.

Any discussion of communicative programming requires us to be able to appreciate the individual notions of programming and communication; in fact, we need to be able to consider these notions both separately and in conjunction. However, although there is clearly plentiful discussion within software communities of what constitutes programming, I don’t feel that enough attention is paid to the notion of communication. I believe this is because defining, understanding and discussing the notion of programming as a form of communication is actually really hard.

So, what do programmers do with hard problems? Divide and conquer! This post is therefore one such attempt to divide the concept of communication into four smaller, more specific concepts: perception, language, context and perspective¹.

In order to help me think and reason about these constituent concepts, particularly in relation to programming, I find it very useful to consider the analogy of hand shadows. Let’s walk through it…

Perception: The Shadow

With hand shadows, the objective is to cast a shadow of a particular artefact. For example, consider the example illustrated in Figure 1. The artefact that is perceived in this case is a tortoise. When we communicate, the goal can be seen to transfer information in such a way that the correct perceptions are made clear to your audience. If you want them to perceive a tortoise, then clearly you have to achieve the equivalent of casting a shadow of a tortoise.

Figure 2. A man with a frightful expression.

Language: The Hand Formation

Hand shadows are created by manipulating your hands into a specific shape. You form a three dimensional object which has a specific two dimensional plane which projects a shadow of a given form (which is then perceived as described above).

In this case, your hands can be seen as language. Note that the relationship between the language and the shadow isn’t always particularly obvious and clear. In the case of the tortoise in Figure 1 the hand shape is pretty similar to the shadow. However, consider the hand shape used to project the frightful expression in Figure 2, the relationship between hand and shadow isn’t so obvious in this case.

Clearly, different combinations of twists and overlaps produce different shapes just like different combinations of syntactic elements produce different sentences.

Shared Context: The Surface

The surface is the area onto which the shadow is projected. Usually this surface is a flat wall. However, this need not be the case; the shape and properties of the surface will affect the shadow’s form. Imagine the same shadow projected onto a wall, then a sphere, how about onto curtains which are fluttering in the wind?

The surface is the context of the communication; it is the element that is consistent for a given hand shadow show².

Individual Perspective: The Light Source

The fourth component of the analogy is probably the most subtle; it’s so evasive it’s not even visible in the illustrations. It’s the light source which generates the shadows.

Usually when we create hand shadows we use a fixed light source such as a ceiling-mounted projector. However, this also need not be the case. If our light source is a torch and we start to shine it from different locations, we see a potentially profound effect on the shadow produced given a constant hand formation language; the shadow will stretch and distort. Given the right hand formation or language we might even construct a man that winks if we are to move the light source along a particular path³.

In this analogy the light source can be seen as an individual’s perspective. Each individual that you’re communicating with will have his or her own unique perspective which will cast it’s own different shadow given the same hand formation and surface. Understanding that each individual has their own unique perspective which will cause them to have different perceptions of the same language and context is key to understanding Communicative Programming.

Communicative Programming

OK, so how does all this relate to programming? To me, one of the clearest messages I get when I spend time considering this analogy is that we shouldn’t just focus on the code we write. We need to understand that programming is really just another form of communication, albeit a much more formal and process oriented one. It’s fair to say that predominantly the audience of our communication has been the computer, but we’re starting to see a growing need to communicate with a wider audience that includes real people - even people that have no knowledge of formal notation of process.

We need to start off by exploring the domain of the project in order to determine our context. The context contains many things such as shared vocabulary, concepts, ways of expression. Another way to consider it is a set of sensible defaults - a base or foundation from which to start from. If we can get a good grasp of the context and understand the surface we’re projecting onto, it can reduce the amount of contortion and manipulation we have to do with our language to achieve the desired result. We can start to let the context do some of the work for us.

Also, once we have understood who our audience is we can then start to understand their perspectives. Having some kind of insight into this will drastically increase our chance of casting the right shadows with our code in the given context so that they we can start to generate a common perception.

If we manage to get a common perception, all kinds of magic takes place. We are much more capable of having lucid and accurate conversations about the project that all participants are more confident about with much reduced overhead. This is what I refer to as conceptual efficiency. This is the holy grail.

Posted on Wednesday the 11^th of November, 2009

As some of you may know, Susanna, my wife, has recently started a Ph.D. in Bioethics at Cambridge in the Centre For Family Research. I’m so wonderfully proud of her and it’s fantastic to see her applying her mind to something that’s both interesting and challenging for her, yet ultimately valuable for us all as a society in times ravaged by technological progress. However, it was very difficult to deal with the fact that we would be reduced to fleeting weekend visits; islands of togetherness in a silent ocean of isolation.

So very luckily, Innovation Factory the company I work for, are fantastically understanding about this and have agreed to let me work remotely one week a month from Cambridge. I really feel proud to have such caring colleagues to support me with this arrangement.

All of this means I spent last week living and working back in England from the (then remarkably pleasant) climes of Cambridge.

I have only ever briefly visited Cambridge before this stay, so only managed to get a glimpse of the condensed beauty it offers. Even compared to Amsterdam it is a tiny city, and like Amsterdam it offers wonderful architecture, a rich cultural history and an affinity for cycling. Although don’t get your hopes up too much - this is still strictly an English city and its cycling offerings are only relatively better than those available elsewhere in England - all of which pale in comparison to the facilities here in the Netherlands.

However, what I hadn’t experienced until last week’s visit were the people Cambridge attracts. Having trained as an academic, I have certainly met my fair share of interesting people endeavouring in a wide range of exciting studies. Cambridge has a wealth of such endeavours. Yet, what sets those I met in Cambridge apart is that they’re not only interesting, they’re interested. I have some strange and obscure ideas about the communication of process especially when I apply them to the context of music, yet I found no shortage of people readily wanting to listen and discuss them with me. I found this overwhelmingly powerful. Here there were people not only capable of working in their own field but also free and willing to explore other areas and able to combine and contrast concepts from wildly different contexts.

I’m really looking forward to my next visit and if anyone reading this happens to also be in Cambridge, I’d love to meet you for a coffee one morning.

Posted on Monday the 9^th of November, 2009

For those that have noticed the recent lull in postings, you will be pleased to hear that a normal service will shortly be resumed. I’ve been very busy working hard on a number of interesting projects that I’m looking forward to sharing with everyone.

The theme of this blog will also focus more on my research interests, namely matters concerned with Communicative Programming - seeing programming as a form of communication. Therefore I’ll be using this space to explore my ideas on concepts such as Domain Specific Languages, Polyglot Programming, Conceptual Efficiency, and important notions such as perspective, intention context and time.

You might also note that this blog no longer contains a comments form. This is partly due to technical reasons and mostly due to the huge personal benefits I gain by not having to deal with spam filters and all that paraphernalia. The technical aspects are down to me choosing a static site generator tool (Jekyll in this case) which allows me to completely bypass any web site authentication and interfaces, and focus purely on writing text in Markdown with Emacs and pushing updates with Git. Things like this really soothe my mind.

I’m really looking forward to getting some more of my ideas out and sharing them with you all. Although there is no comment form, I will be accepting comments via email and if I think they’re helpful and relevant will happily append them to the post they’re related to. Hopefully the small amount of effort of emailing a comment will help to increase each comment’s quality - a win for everyone!

See you back here soon…