Making time

Time, the ever baffling one directional mystery. A lot of it has been spent between the members of slub on ways to synchronise multiple machines to share a simple beat, sometimes attempting industrial strength solutions but somehow the longest standing approach we always come back to for our various ad-hoc software remains to be a single osc message. This is the kind of thing that seems to normally involve stressed pre-performance hacking, so after having to rewriting it for temporal recursion I thought I should get it down here for future reference!

The message is called “/sync” and contains two floating point values, the first the number of beats in a “bar” (which is legacy, we don’t use this now) and then the current beats per minute. The time the message is sent is considered to be the start of the beat. A sync message comes into my system via a daemon called syncup. All this really does is attach a timestamp to the sync message recording what the local time on my machine was when it arrived, and sends it on to fluxus. Shared timestamps would be better, but don’t make any sense without a shared clock, and they seem fragile to our demands. The daemon polls on a fairly tight loop (100ms) and the resulting timestamp seems accurate enough for our ears (fluxus runs on the frame refresh rate which is too variable for this job).

So now we have a new sync message which includes a timestamp for the beat start. The first thing the system does is to assume this is in the past, and that the current time has already moved ahead. There are 3 points of time involved:

From the sync time (in the past, on the left) and the bpm we can calculate the beat times into the future. We have a “logical time” which is initialised with the current time from the system clock, a safety margin added, and then gets “snapped” to the nearest beat. The safety margin is needed as the synth graph build and play messages coming from fluxus need to be early enough to get scheduled by fluxa’s synth engine to play with sample accuracy.

The beat snapping has to be able to move back in time as well as forwards, for tiny adjustments from the sync messages (as they never come in exactly when they are expected) otherwise we skip beats. The algorithm to do this is as follows:

(define (snap-time-to-sync time)
  (+ time (calc-offset time last-sync-time (* (/ 1 bpm) 60)))) 

(define (calc-offset time-now sync-time beat-dur)
  ;; find the difference in terms of tempo
  (let* ((diff (/ (- sync-time time-now) beat-dur))
         ;; get the fractional remainder (doesn't matter how
         ;; far in the past or future the synctime is)
         (fract (- diff (floor diff))))
    ;; do the snapping
    (if (< fract 0.5) 
        ;; need to jump forwards - convert back into seconds
        (* fract beat-dur) 
        ;; the beat is behind us, so go backwards
        (- (* (- 1 fract) beat-dur)))))

The last thing that is needed is a global sync offset, which I add at the start of the process, to the incoming message timestamps – this has to be tuned by ear, and accounts for the fact that the latency between the synth playing a note and the speakers moving air seems to vary between machines dependent on many uncertain factors – sound card parameters, battery vs ac power, sound system setup, colour of your backdrop etc.

Other than this we tend to keep the networking tech to a minimum and use our ears and scribbled drawn scores (sometimes made from stones) to share any other musical data.

scheme bricks 2

A new version of scheme bricks is under way, planned to be tested out with slub on the Mozilla Fest Party, then taken across the Atlantic for some more livecoding action in Mexico City! New things include blocks with depth – cosmetic for the moment, but I plan to prototype some new ideas based on this, separately zoom-able code blocks, and most importantly it’s a complete rewrite into functional R5RS Scheme for portability – it should now be relatively simple to get it on android via Nomadic which uses Tinyscheme.

Using the new temporal recursion, the code produced is much less monolithic. Massively tall structures resulting from plugging together sequences during long performances were a bit of an issue before, but splitting the code into a multitude of functions (which can be shrunk and put in the “background”) seems to be a far easier way of working so far.

Aniziz and Zizim

The online part of the borrowed scenery project is an experiment in geotagging plants and plant related locations via a website/app called Zizim (the compass) combined with a multiplayer online game called Aniziz (the soil) where you can interact with the plants people have found. Having spent the last couple of months developing them, they are now ready for more of an open beta phase. Another part of the project is the forum here for collecting any feedback and thoughts.

Your role is to strengthen the connection between the world of Aniziz and the plants of Ghent. The plants are broadcasting messages which can only be correctly tuned into by energising them with fungi, the more plants you energise the higher your score will be.

The latest addition are specially tagged items called “pataportals” you can create with the android app which create “wormholes” in the Aniziz world. Stepping into one causes you to get sent to another one – which could be thousands of miles away. Right now Ghent is connected with the Cornish town of Penryn via a wormhole on the sea shore:

Temporal recursion

Slub have a number of important livecoding transmissions coming up (including a performance at the Mozilla Festival!) so it’s time to work on fluxus/fluxa/scheme bricks. Here are some recording tests of a feature I’ve been wanting to use for a long time – temporal recursion.

These recordings were not changed by hand as they played, but started and left running in ‘generative audio’ mode in order to try and understand the technique. This method of sequencing is inspired by Impromptu which uses a similar idea. In fluxa it’s all based around a single new function: “in” which schedules a call to a function – which can be the current function (this is different to the existing ‘timed tasks’ in fluxus which are less precise for this kind of sequencing).

(define (tick time a)
    (play (+ time 3) (sample "ga.wav" (note (pick '(40 42 45) a))))
    (in time 0.22 tick (+ a 1)))

(in (time-now) 1 tick 0)

The “in” function takes the current time, the time to wait before the call, the function to call and it’s parameters. In the example above the argument “a” gets incremented each time, resulting in a sequence of notes being played. Recursion generally brings up thoughts of self similarity and fractal patterns – as in the graphical use of recursion in fluxus, but here it’s better to imagine a graph of function calls. Each function can branch to a arbitrary number of others, so limitations have to be put in place to stop the thing exploding with too many concurrent calls. What seems to happen (even with small function call graphs) is the appearance of high level time structures – state changes and shifts into different modes where different patterns of calls lock into sequence. You can hear this clearly in the second recording above which alters itself half way through.

I’ve also experimented with visualising the call graph, with limited success with this more complex example – the round nodes are functions, the boxes parameter changes and labels on the connections are the branch conditions:

(require fluxus-018/fluxa)

(searchpath "/home/dave/noiz/nm/")
(define n '(23 30))
(set-scale '(1 1 2 1))

(define (za time a b)
    (play (+ time 3) (mul (mul (adsr 0 0.1 1 1) (pick '(0.4 0.1 1 0.4 1) a)) 
        (sample "ga.wav" (note (- (modulo (- b a) 17) (* (pick n a) 2))))) -0.5)
    (when (zero? (modulo a 16)) (in time 0.3 zm a b))
    (if (eq? (modulo a 14) 12)
        (in time 1.0 zoom a (- b 1))
        (in time (- 0.5 (* (modulo a 13) 0.03)) za (+ a 1) (+ b 2))))

(define (zm time a b)
    (play (+ time 3) (mul (mul (adsr 0 0.1 1 1) (pick '(0.1 0.4 1) a))
        (sample "ga.wav" (note (+ (modulo b 5) (/ (pick n a) 2))))) 0.5)
    (if (> a 12)
        (in time 1.0 za b a)
        (in time (pick '(1.3 1.5 0.5) a) zm (+ a 1) b)))

(define (zoom time a b)
    (play (+ time 3) (mul (mul (adsr 0 0.1 1 1) (pick '(0.1 0.2 0.3) a)) 
        (sample "ga.wav" (note (+ (pick n a) (* 3 (modulo a (+ 1 (modulo b 5)))))))))
    (if (> a 16) 
        (in time 0.3 zm 0 (+ b 1))
        (in time (pick '(1.3 0.12) a) zoom (+ a 1) b)))

(in (time-now) 1 zoom 0 1)

Aniziz on iPad

Thanks to HTML5 canvas, my first game that works on the iPad. All I had to do was hook up the touch events to call the mouse handlers and it was pretty functional, although the game would be better dealing with touch events differently using more drag-drop approach. The game runs around 20fps compared to 60fps on my laptop, but it’s fairly playable on Safari.