Posted on Fri Nov 02 00:00:00 -0400 2012

As some of you will know, I’ve spent the last few years hacking furiously on something called Overtone which is a new language front-end to the sound synthesis server SuperCollider written in Clojure. We’ve make a large amount of progress with it and adoption is continuing to grow amongst the programming community. Here’s an example of it in action:

More recently I’ve joined the Raspberry Pi team for a few months to figure out how much I can port across to the small confines of the Pi with an eye on making something useful for engaging school children with a more modern computer science curriculum.

The first step was obviously to get SuperCollider running on the Pi. This unfortunately proved to be much more tricky than apt-get install supercollider. After a good while of running into dead ends, a remarbly kind chap called Stephan Lachowsky helped put the final pieces of the puzzle together - so many thanks to him. I’ll walk you through this final process so you too can get the crazy synth sounds out of the Pi. Hopfully future updates will render this post redundant, but until that time I hope it helps you out.

Initial Setup

First, let’s get some base dependencies installed:

• Get a terminal connection to your Pi (either by sshing into the Pi itself or by opening up the Terminal app).
• Update your packages: sudo apt-get update
• Install Jack 2: sudo apt-get install jackd2
• Install supercollider: sudo apt-get install supercollider

Install Jack2 v1.9.8

Under Linux, the latest versions of SuperCollider requires Jack to make sound. Unfortunately, the version of jackd2 that apt-get just installed is currently broken. If you try running it, you get a remarkably unhelpful Bus Error message as it dies due to struct alignment errors.

The solution: either wait until Jack or the compiler (whichever turns out to be the culprit) is fixed, or compile and install an older version which doesn’t suffer from the errors. It just so happens that version 1.9.8 works perfectly well. So, either download the source and compile it yourself, or follow these steps to set up a pre-compiled jackd2 binary available here.

• Move into your home directory: cd ~
• Fetch the Jack binary: wget http://sam.aaron.name/files/jack_for_pi_v1.9.8.tar.gz
• Unpack the binary: tar xf jack_for_pi_v1.9.8.tar.gz
• Update your PATH: export PATH=/home/pi/local/bin:${PATH}
• Ensure the library is visible: export LD_LIBRARY_PATH=/home/pi/local/lib
• Test that jackd points to the newly downloaded version: which jackd should point to /home/pi/local/bin/jackd

Start Jack and SuperCollider

Next we need to do a bunch of things:

• Start Jack in dummy mode - i.e. it doesn’t talk directly to any sound card but can be connected to from external apps.
• Start alsa_out which does talk directly to the sound card, and can be connected to Jack to receive audio. (This is necessary because the Pi’s soundcard doesn’t currently support mmap which is required for direct Jack -> sound card connectivity).
• Start scsynth, the SuperCollider server.
• Connect both left and right channels within SuperCollider to alsa_out.

This can all be achieved by copying the following to a file such as ~/bin/start-sc:

#!/bin/bash

jackd -m -p 32 -d dummy &
sleep 1
alsa_out -q1 2>&1 > /dev/null &
sleep 1
scsynth -u 4555 &
sleep 1
jack_connect SuperCollider:out_1 alsa_out:playback_1 &
jack_connect SuperCollider:out_2 alsa_out:playback_2 &

Now, give your file an execute bit: chmod u+x ~/bin/start-sc and start it up:

~/bin/start-sc

If you see the following, SuperCollider’s server, scsynth, should be running and listening for UDP packets on port 4555 (don’t worry about the Zeroconf warning):

jackdmp 1.9.8
Copyright 2001-2005 Paul Davis and others.
Copyright 2004-2011 Grame.
jackdmp comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions; see the file COPYING for details
JACK server starting in realtime mode with priority 10
JackDriver: client name is 'SuperCollider'
SC_AudioDriver: sample rate = 48000.000000, driver's block size = 1024
SuperCollider 3 server ready..
Zeroconf: failed to create client: Daemon not running

If you’re using Overtone, you can connect to it from a remote machine with:

(use 'overtone.core)
(connect-external-server "ip address of your pi" 4555)
(demo (sin-osc)) ;=> Plug in some headphones to hear a
                 ;   sine wave on the Pi's audio jack.

Have fun and happy hacking!

Posted on Mon Mar 29 00:00:00 -0400 2010

A good friend of mine, Michael Hunger, is planning on describing a number of differing implementations of Conway’s Game of Life in an upcoming keynote. In pursuit of this, he recently emailed me requesting an ‘idiomatic’ version in the exciting yet really rather experimental programming language Ioke.

This left me with an interesting question floating in my mind: “What on earth is idiomatic Ioke?”. In fact, how can the notion idiomatic even exist in any fledgling medium?

However, I was undeterred and resolved on attempting to try my best to build something that at least I was happy with. After much writing and re-writing, polishing and folding I finally settled on the the notation I’ll explain in this post. Please note, you’ll need the latest edge build of Ioke to interpret it.

As a quick sidenote, if you haven’t previously looked at the GoL, I encourage you to take a quick peak at the Wikipedia entry before you continue with this article - it’s fun and interesting stuff. Rather than rewriting the description here’s an excerpt from the article itself:

The universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells, each of which
is in one of two possible states, live or dead. Every cell interacts with its eight neighbors, which are the
cells that are directly horizontally, vertically, or diagonally adjacent. At each step in time, the following
transitions occur:

1. Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
2. Any live cell with more than three live neighbours dies, as if by overcrowding.
3. Any live cell with two or three live neighbours lives on to the next generation.
4. Any dead cell with exactly three live neighbours becomes a live cell.

The initial pattern constitutes the seed of the system. The first generation is created by applying the above
rules simultaneously to every cell in the seed — births and deaths happen simultaneously, and the discrete
moment at which this happens is sometimes called a tick (in other words, each generation is a pure function
of the one before). The rules continue to be applied repeatedly to create further generations.

I tackled the problem by considering the perspectives of the interface and implementation. For the interface I created a runner script, and for the implementation I created a GameOfLife object.

Game of Life Runner

A generic GoL runner might consist of the following three phases:

• Firstly, the initialisation of a life environment - this requires a grid size to be specified in terms of the number of rows and columns,
• Secondly for certain cells in the environment’s grid to be manually spawned - this is the seed of the pattern,
• Finally for a number of evolutions to be initiated with the possibility of displaying the state of the current evolution.

For this I created the following runner script:

#!/usr/bin/env ioke
;gol_runner.ik
use("gol")

;read in from the program arguments the number of evolutions to initiate
numTimes = System programArguments first toRational

;Phase 1: create the environment
life = GameOfLife mimic(rows: 5, columns: 6)

;Phase 2: Manually Spawn cells
[[1,0], [1,1], [1,2], [1,3], [1,4], [1,5]] each(coords,
  life grid spawnCell(*coords))

;Phase 3: Evolve the environment the requested number of times
;the current state which counts as the first evolution
life grid asText println

;initiate the following n-1 evolutions
(numTimes - 1) times(life evolve asText println)

Note that I’m also reading in the number of iterations to initiate from the program arguments. It might seem strange that I’m calling the method toRational on the first argument (which comes in as text), however this is currently the only way of converting text to an integer in Ioke at the moment. Ola has said that this is something he’s looking into improving.

Running the Runner

Now, if we assume the presence of a functioning GoL implementation within “gol.ik”, We can run this and see life evolving before our very eyes:

∴ /Users/sam/Development/gol
λ ./gol_runner.ik 8

+-------------+
|             |
| * * * * * * |
|             |
|             |
|             |
+-------------+

+-------------+
|   * * * *   |
|   * * * *   |
|   * * * *   |
|             |
|             |
+-------------+

+-------------+
|   *     *   |
| *         * |
|   *     *   |
|     * *     |
|             |
+-------------+

+-------------+
|             |
| * *     * * |
|   * * * *   |
|     * *     |
|             |
+-------------+

+-------------+
|             |
| * *     * * |
| *         * |
|   *     *   |
|             |
+-------------+

+-------------+
|             |
| * *     * * |
| *         * |
|             |
|             |
+-------------+

+-------------+
|             |
| * *     * * |
| * *     * * |
|             |
|             |
+-------------+

+-------------+
|             |
| * *     * * |
| * *     * * |
|             |
|             |
+-------------+

How lovely. I do hope that you appreciate ASCII-art as much as me! OK, so now for the actual implementation. For this I decomposed GoL into the following three objects: GameOfLife, Grid and CellData.

CellData

This is a very basic object which is essentially a nice package of useful cell information such as its liveliness, coordinates and number of live neighbours. Think of it as a basic struct of information. This object is generated by the Grid and used for the evolution process within the GameOfLife object.

GameOfLife

This is the environment, the containing object that is used by the runner. It describes the three phases described above, initialisation, spawning and evolving, each realised by a corresponding method. The source for this, which I will deconstruct below, is as follows:

GameOfLife = Origin mimic do(
  initialize = method(rows:, columns:,
    @grid = Grid withDimensions(rows, columns))

  spawnCell = method(row, col,
    grid spawnCell(row, col))

  evolve = method(
    nextGrid       = grid blankGrid
    survivingCells = grid filter(live?) filter(numLiveNeighbours in?(2..3))
    newCells       = grid reject(live?) filter(numLiveNeighbours == 3)

    (newCells + survivingCells) each(cellData,
      nextGrid spawnCell(cellData row, cellData col))

    @grid = nextGrid)
)

initialize

initialize = method(rows:, columns:,
  @grid = Grid withDimensions(rows, columns))

This is the standard Ioke initialisation hook method. This gets called when the containing object is mimiced (the only way in Ioke to create new objects). In this case we create a new Grid with the requested dimensions and assign it to a local cell called grid. Note that @grid is just shorthand for self grid.

spawnCell

spawnCell = method(row, col,
  grid spawnCell(row, col))

Here we essentially palm-off the logic to the grid object (explained below). I hope that one day Ioke will get the ability to delegate methods in a much more elegant fashion.

evolve

REXML could not parse this XML/HTML: 
<div>
  <pre><code class='ioke'>evolve = method(
  nextGrid       = grid blankGrid
  survivingCells = grid filter(live?) filter(numLiveNeighbours in?(2..3))
  newCells       = grid reject(live?) filter(numLiveNeighbours == 3)

(newCells + survivingCells) each(cellData, nextGrid spawnCell(cellData row, cellData col))

@grid = nextGrid)</code></pre> </div>

Here we essentially have all the logic for Conway’s Game of Life expressed in a very small number of succinct lines. We also get to see some of Ioke’s lovely sequence facilities in action.

Firstly, we create a new grid which is essentially the current grid, but with all the cells reset to the default state (empty). This is the grid we will populate for the next evolution; consider it a blank slate. Next we collect the set of cells which should survive from the current generation and also which cells will be spawned as a resultant of the correct number of live neighbour cells. As described above, the GoL rules state that a cell is spawned if it has exactly 3 live neighbours. Here we can see that Ioke makes expressing this remarkably elegant and readable:

newCells = grid reject(live?) filter(numLiveNeighbours == 3)

Here we take all the cells in the grid and reject the ones which are live (i.e. have live? set to a non-true value), we then filter again based on which cells have 3 live neighbours. How beautiful is that!

Finally, we combine the newly spawned cells and the surviving sells and we spawn a corresonding cell in the next grid which we then use to replace the current grid. Job done.

Grid

This is the grid of cells. Rather than using the standard initialize method hook I decided to write my own constructor-style method which has a slightly more intention-revealing name: withDimensions. This is implemented as follows:

withDimensions = method(rows, columns,
    with(
      state: Dict withDefault(0),
      numCols: columns,
      maxRowIdx: rows - 1,
      maxColIdx: columns - 1))

Here we initiate the new Grid object with some default cells representing dimensional information for the grid and also the state of the grid. This state is stored in a Dict which is a dictionary, or a hash in other languages. The state stores a value for each coordinate of the grid, where the value is either 0 for an empty cell and 1 for a live cell. The default for this dictionary is 0. The spawnCell method essentially sets this bit to 1 to indicate that the given cell is alive. The main reason for using the integers 0 and 1 to represent the liveliness of the cells is that they lend themselves very well for counting. This property is taken advantage of in countLiveNeighbouts:

countLiveNeighbours = method("Counts the number of live neighbours for a given set of cell coordinates",
    row, col,
    neighbourCoords = permutations((-1..1), (-1..1)) - [(0,0)]
    neighbourCoords inject(0, sum, (r_mod,c_mod), state[[row + r_mod, col + c_mod]] + sum))

This method essentially iterates through all the possible neighbour coordinates relative to the current cell and sums the number of live cells. Here we get a good feel for how Ioke allows functional style programming in a slightly more elegant manner than Ruby. Also note that Ioke allows you to pass in a documentation string as the first parameter to any method. This documentation string essentially acts as both an inline source comment and also is available to the runtime which makes it accessible from the repl and also by documentation generation tools such as DokGen.

One of the nicer Ioke features is its Rubyesque mixins. In this source I use the Sequenced mixin to give the Grid object a sequence interface:

mimic!(Mixins Sequenced)

This gives it the nice filter method for free which is used in the evolve method described above. To achieve this I just need to mimic the Sequenced mixin and implement a seq method. In this case Grid’s seq method piggybacks on the seq method of the list of all cells returned by allCells which is is a list of CellData objects:

seq = method(allCells seq)

Finally, there are a couple of methods which are used to display the current state. The main method for this used by the runner is asText which generates a nice pretty ASCII representation of a given grid.

Complete Source

OK, I realise that I haven’t covered everything in detail, but hopefully I’ve described enough to give you a taste of Ioke and left enough for you to work through and discover yourself. One thing to note is that this is probably the slowest GoL implementation (that wasn’t specifically written to be slow) that you’ll ever see. It’s so slow it’s painful. However, performance isn’t the main purpose of objective of Ioke - it’s an experiment in expressiveness. Hopefully this example illustrates this for you.

As a parting gift, here is the complete source of my GoL implementation. I’d love it if any of you would like to send me some feedback regarding this approach. This is my first GoL implementation, so I’m positive I have much to learn.

GameOfLife = Origin mimic do(
  initialize = method(rows:, columns:,
    @grid = Grid withDimensions(rows, columns))

  spawnCell = method(row, col,
    grid spawnCell(row, col))

  evolve = method(
    nextGrid       = grid blankGrid
    survivingCells = grid filter(live?) filter(numLiveNeighbours in?(2..3))
    newCells       = grid reject(live?) filter(numLiveNeighbours == 3)

    (newCells + survivingCells) each(cellData,
      nextGrid spawnCell(cellData row, cellData col))

    @grid = nextGrid)
)

GameOfLife Grid = Origin mimic do(
  mimic!(Mixins Sequenced)

  CellData = Origin mimic do(
    row = method(coords first)
    col = method(coords second))

  withDimensions = method(rows, columns,
    with(
      state: Dict withDefault(0),
      numCols: columns,
      maxRowIdx: rows - 1,
      maxColIdx: columns - 1))

  blankGrid = method("Resets the grid", with(state: Dict withDefault(0)))
  spawnCell = method("Animates a given cell", row, col, state[[row, col]] = 1)

  ;internal methods
  seq = method(allCells seq)
  permutations = method(a, b, a flatMap(i, b map(j, (i,j))))

  countLiveNeighbours = method("Counts the number of live neighbours for a given set of cell coordinates",
    row, col,
    neighbourCoords = permutations((-1..1), (-1..1)) - [(0,0)]
    neighbourCoords inject(0, sum, (r_mod,c_mod), state[[row + r_mod, col + c_mod]] + sum))

  allCellCoords = method("Generates a list of tuples representing the coordinates of all the cells",
    permutations(0..maxRowIdx, 0..maxColIdx))

  allCells = method("Generates a list of all the cells in the Grid with associated metadata",
    allCellCoords map((row, col),
      CellData with(coords: (row, col),
                    numLiveNeighbours: countLiveNeighbours(row, col),
                    live?: state[[row, col]] == 1)))

  ;for output purposes
  asList = method("Returns the list of cells as a list of 0s and 1s, with 1 representing a live cell",
    rowsOfCoords = allCellCoords seq sliced(numCols)
    cellList = []
    while(rowsOfCoords next?, cellList << (rowsOfCoords next map((row, col), state[[row,col]]))))

  asText = method("Returns the list of cells in a pretty ascii art representation",
    gridContents = asList map(map(i, if(i == 0, "  ", "* ")) join)
    "\n+-#{"--" * (numCols)}+\n| #{gridContents join("|\n| ")}|\n+-#{"--" * (numCols)}+")
)

Posted on Tue Feb 09 00:00:00 -0500 2010

Update: This post is badly out of date and represented a cludgy work-around that I discovered. I haven’t put any further effort into cleaning this up or weeding out the unecessary steps as all my time is spent hacking on Overtone these days. However, this post looks to provide a much simpler solution: http://www.gloryisasilentthing.com/glory/supercollider-emacs-scel-and-the-right-way/

I think that SuperCollider is one of the most exciting music technologies currently available. It’s super powerful, Open Source, and has even got a separate server and language runtime giving everyone the opportunity to experiment with different approaches. I’m very excited to see where we go with it, and how I can utilise it with my Polynome project.

However, despite the fact that it comes shipping with a very nice live-coding enabled interface, I use Emacs, and well, I love using Emacs for editing text of all kinds. I therefore wanted to have the convenience of the live editing with the powerful text manipulation features that Emacs offers. Luckily there’s an Emacs mode for SuperCollider and I just finished hooking it all up. Unfortunately I couldn’t find a nice tutorial for doing it, but I did manage to piece together a process that finally worked for me. It wasn’t a fun time, so I’m documenting it here in the hope that someone else might benefit from my pain.

Caveat and Context

Now, I need to issue a major caveat: there’s lots of moving parts in this situation, and your parts might not even be the same as mine. So, just to be clear - this might not work for you, but hopefully it’ll get you a major part of the distance. The more similar our setups, the more likely this will help, so for clarity I’m using:

• Cocoa Emacs 23.1.50 Git 2010-02-09 (the latest nightly build at the time of writing)
• SuperCollider 3.3.1 (the Mac version)
• OS X 10.6

I’m also assuming that you have already installed Emacs and SuperCollider and that you have a basic working knowledge of Emacs configuration, and general UNIX hockery pokery¹.

Update your Path (UNIX-style)

First up open your shell config file (~/.zshrc in my case) with your favourite editor² and append /Applications/SuperCollider to your PATH. Check to see if it worked:

λ sclang -h
Usage:
   sclang [options] [file..] [-]

Options:
   -d <path>                      Set runtime directory
   -D                             Enter daemon mode (no input)
   -g <memory-growth>[km]         Set heap growth (default 256k)
   -h                             Display this message and exit
   -l <path>                      Set library configuration file
   -m <memory-space>[km]          Set initial heap size (default 2m)
   -r                             Call Main.run on startup
   -s                             Call Main.stop on shutdown
   -u <network-port-number>       Set UDP listening port (default 57120)

Update your Path (Cocoa-Emacs-style)

Ok, you can do a few things here, particularly because Emacs on OS X does very funky things with the PATH variable:

• In a fresh terminal update the OS X plist version of the PATH to match your newly updated version: defaults write $HOME/.MacOSX/environment PATH "$PATH" and restart your machine.
• In your emacs config explicitly create a PATH variable: (setq path "/Applications/SuperCollider:/rest/of/PATH")(setenv "PATH" path)
• Again, in your Emacs config, add SuperCollider to your exec-path: (push "/Applications/SuperCollider" exec-path)

Not all of these may be necessary; I did the the last two.

Creating a Fresh SCClassLibrary

Create a new place for a fresh copy of SCClassLibrary:

 mkdir ~/.sclang

Copy the SCClassLibrary from your SuperCollider install into this new place:

 cp -R /Applications/SuperCollider/SCClassLibrary ~/.sclang

Also, copy across the plugins directory:

 cp -R /Applications/SuperCollider/plugins ~/.sclang

Finally, copy across the scsynth binary (needed to boot the server and make actual sounds):

cp /Applications/SuperCollider/scsynth ~/.sclang

/Applications/SuperCollider/README\ SCLang\ OSX advises you to edit the startup method in ~/.sclang/SCClassLibrary/Platform/osx/OSXPlatform.sc. Open it up and find the startup method. Delete everything from startup { to the closing } and replace it with the following:

startup {
            if ( this.hasFeature( \emacs ) ) {
                            Document.implementationClass.startup;
            };
            this.loadStartupFiles;
}

Grab a copy of scel

Now we’re ready to power up Emacs with SuperCollider goodness. I found a copy of scel on github that contained a few issues that I ironed out in my own clone³. Feel free to grab it and add it to your own emacs config. That might be as simple as downloading the tarball, or if you manage your config with git (which, by the way, is a great idea) then you could just add it as a submodule. I happen to have based my config on Phil Hagelberg’s wonderful emacs starter kit. So this can be as simple as:

cd ~/.emacs.d
git init #unless you already manage your .emacs.d with git
git submodule add git://github.com/samaaron/scel.git vendor/supercollider

Copy scel’s sc files to your new SCClassLibrary

For scel to work correctly it has a number of SC Class files that need to be compiled and loaded when sclang is started. In order to achieve this copy the sc files from scel’s sc directory into your new SCClassLibrary:

cp -R ~/.emacs.d/vendor/supercollider/sc ~/.sclang/SCClassLibrary/Common/Emacs

Configure Emacs

Now you need to teach emacs about the presence of scel. In your emacs config add the following:

(add-to-list 'load-path "~/.emacs.d/vendor/supercollider/el")
(require 'sclang)

Finally, you need to tell it where to find your SCClassLibrary and other bits and bobs. scel advises you to use M-x sclang-customize, however I just set the variables by hand:

(custom-set-variables
'(sclang-auto-scroll-post-buffer t)
'(sclang-eval-line-forward nil)
'(sclang-help-path (quote ("/Applications/SuperCollider/Help")))
'(sclang-runtime-directory "~/.sclang/"))

Start up the Emacs SC Workbench

OK, well done for making it so far. I’m praying for you that things went hunky-dory. At this stage I simply restarted Emacs and hit M-x sclang-start and BOOM entered the SC Workbench. If this seems to work, and you see two buffers, a workbench and a log, try typing "Hello from Emacs".postln; in the buffer and with the cursor over it hit M-x sclang-eval-line. You should see the message pop up in the log. Finally, try booting the server with M-x sclang-server-start. You should be good to go.

Go grab a cup of tea, you’ve earned it.

Posted on Wed Dec 23 00:00:00 -0500 2009

Introducing the IokeLexer

It seems I just can’t stop writing syntax highlighters for Ioke. I started with adding support for TextMate, and then basic support for Geshi. Now, as a present for its P release¹ I’ve added Pygments to the list:

Pygments IokeLexer

One of the things that has bugged me since Ioke’s birth is that its incubator, GitHub has not been able to do the honourable thing and highlight it appropriately. Instead, one of the most exciting languages around at the moment has been treated like dirty plain text.

Well, hopefully this can change soon. GitHub happens to use Pygments to highlight its source. Therefore I just need to throw my IokeLexer at the Pocoo chaps, hope they include it in the next version of Pygments and then wait for GitHub to upgrade. At least someone doing the hard work of writing the lexer won’t be the blocker any more…

Just to show its glory², here’s the introduction example from http://ioke.org

#!/usr/bin/ioke

Ioke = LanguageExperiment with(
  goal: :expressiveness,
  data: as(code),
  code: as(data),
  features: [
    :dynamic,
    :object_oriented,
    :prototype_based,
    :homoiconic,
    :macros
  ],
  runtimes:(JVM, CLR),
  inspirations: set(Io, Smalltalk, Ruby, Lisp)
)

hello = method("Every example needs a hello world!",
  name,
  "hello, #{name}!" println)

Ioke inspirations select(
  features include?(:object_oriented)
) each(x, hello(x name))

Lessons Learned

OK, so apart from a fairly high number of hours battling with Python regular expressions and Ioke syntax, here’s a summary of the lessons I learned from this experience:

• You can get pretty far with plain regexps.
• You can get a long way with regexps and stackable contexts.
• Writing approximately correct lexers with regexps can be fun.
• Writing accurate lexers with regexps is for masochists, seriously.
• Look ahead and behind regexp matchers are truly wonderful.
• Writing a regexp lexer really helps you to hone your regexp skills.
• Pygments is nice software.
• Python people can be a little curt yet are very helpful nevertheless.
• Nothing beats a real lexer for lexing.

Finally, there really should be a standard syntax for this kind of thing so when the next syntax highlighting engine comes around we don’t have to continue to re-invent the wheel.

Posted on Tue Nov 24 00:00:00 -0500 2009

Figure 1. A tortoise hand shadow from Henry Bursill’s Hand Shadows To Be Thrown Upon The Wall.

Any discussion of communicative programming requires us to be able to appreciate the individual notions of programming and communication; in fact, we need to be able to consider these notions both separately and in conjunction. However, although there is clearly plentiful discussion within software communities of what constitutes programming, I don’t feel that enough attention is paid to the notion of communication. I believe this is because defining, understanding and discussing the notion of programming as a form of communication is actually really hard.

So, what do programmers do with hard problems? Divide and conquer! This post is therefore one such attempt to divide the concept of communication into four smaller, more specific concepts: perception, language, context and perspective¹.

In order to help me think and reason about these constituent concepts, particularly in relation to programming, I find it very useful to consider the analogy of hand shadows. Let’s walk through it…

Perception: The Shadow

With hand shadows, the objective is to cast a shadow of a particular artefact. For example, consider the example illustrated in Figure 1. The artefact that is perceived in this case is a tortoise. When we communicate, the goal can be seen to transfer information in such a way that the correct perceptions are made clear to your audience. If you want them to perceive a tortoise, then clearly you have to achieve the equivalent of casting a shadow of a tortoise.

Figure 2. A man with a frightful expression.

Language: The Hand Formation

Hand shadows are created by manipulating your hands into a specific shape. You form a three dimensional object which has a specific two dimensional plane which projects a shadow of a given form (which is then perceived as described above).

In this case, your hands can be seen as language. Note that the relationship between the language and the shadow isn’t always particularly obvious and clear. In the case of the tortoise in Figure 1 the hand shape is pretty similar to the shadow. However, consider the hand shape used to project the frightful expression in Figure 2, the relationship between hand and shadow isn’t so obvious in this case.

Clearly, different combinations of twists and overlaps produce different shapes just like different combinations of syntactic elements produce different sentences.

Shared Context: The Surface

The surface is the area onto which the shadow is projected. Usually this surface is a flat wall. However, this need not be the case; the shape and properties of the surface will affect the shadow’s form. Imagine the same shadow projected onto a wall, then a sphere, how about onto curtains which are fluttering in the wind?

The surface is the context of the communication; it is the element that is consistent for a given hand shadow show².

Individual Perspective: The Light Source

The fourth component of the analogy is probably the most subtle; it’s so evasive it’s not even visible in the illustrations. It’s the light source which generates the shadows.

Usually when we create hand shadows we use a fixed light source such as a ceiling-mounted projector. However, this also need not be the case. If our light source is a torch and we start to shine it from different locations, we see a potentially profound effect on the shadow produced given a constant hand formation language; the shadow will stretch and distort. Given the right hand formation or language we might even construct a man that winks if we are to move the light source along a particular path³.

In this analogy the light source can be seen as an individual’s perspective. Each individual that you’re communicating with will have his or her own unique perspective which will cast it’s own different shadow given the same hand formation and surface. Understanding that each individual has their own unique perspective which will cause them to have different perceptions of the same language and context is key to understanding Communicative Programming.

Communicative Programming

OK, so how does all this relate to programming? To me, one of the clearest messages I get when I spend time considering this analogy is that we shouldn’t just focus on the code we write. We need to understand that programming is really just another form of communication, albeit a much more formal and process oriented one. It’s fair to say that predominantly the audience of our communication has been the computer, but we’re starting to see a growing need to communicate with a wider audience that includes real people - even people that have no knowledge of formal notation of process.

We need to start off by exploring the domain of the project in order to determine our context. The context contains many things such as shared vocabulary, concepts, ways of expression. Another way to consider it is a set of sensible defaults - a base or foundation from which to start from. If we can get a good grasp of the context and understand the surface we’re projecting onto, it can reduce the amount of contortion and manipulation we have to do with our language to achieve the desired result. We can start to let the context do some of the work for us.

Also, once we have understood who our audience is we can then start to understand their perspectives. Having some kind of insight into this will drastically increase our chance of casting the right shadows with our code in the given context so that they we can start to generate a common perception.

If we manage to get a common perception, all kinds of magic takes place. We are much more capable of having lucid and accurate conversations about the project that all participants are more confident about with much reduced overhead. This is what I refer to as conceptual efficiency. This is the holy grail.