More music from bikes (part 4)

The last few days has seen intense work on Kaffe Matthews’ Ghent Bicycle Opera which goes live in a few days, lots of new stuff on the git repo for Beagle Board/GPS powered sample playback.


(image from Timelab/Kaffe’s site)

The main problem with the version we tested in June was that all the samples needed to be preloaded, fixing us to <512Mb total sample data. The sample loading was also extremely slow, meaning it could take up to 10 minutes after starting up till the bikes were usable. This slowness turned out to be uncompressing the ogg samples into memory on the not so fast ARM processor. Seeing as they needed to be entirely uncompressed to play anyway, there was little point in compressing them (we have no shortage of storage memory on the SD cards). Now the zones (which link areas of the city to audio samples) are surrounded by a couple of hundred metres of loading/unloading area - crossing into this area causes the sample to start loading, exiting frees it from memory. We can now fit all samples for all the routes (collections of zones for different parts of the city) onto a single system - much more than the total memory on the boards. Other significant improvements include a more data-driven sample playback, where zones are given a naming convention that describe if the sample needs to loop, be panned to come from a specific compass direction (based on the calculated direction of the cyclist, from GPS data) of if they need to be a special mysterious type of zone - called a "ghost zone". Ghost zones are programmed to travel across the city over time - blending between a start and end shape/position, so in the morning they might be heard in different places to the afternoon. You'll be able to experience The swamp that was, a bicycle opera at Electrified III: The Responsive City, in Ghent from this weekend.

Aniziz (the soil)

The borrowed scenery game has been named ‘Aniziz’ – from the 12th century Lingua Ignota used by Hildegard of Bingen. The term means “the soil” and represents the game’s role in the wider project, for nurturing plants found in the city, as well as the mapping element.

It’s working much more like a proper multiplayer game now as it has player accounts, using four types of avatar from Theun which represent patabotanical tarot versions of the hermit, hierophant, magician and high priestess. Your avatar will be picked when you register, in the form of a ‘reading’ ritual – still to be designed.

The text rendering has been greatly improved – the support provided by HTML5 canvas is quite basic, so you need to do things like word wrapping yourself. Particle systems now provide more interactivity for the fungi and the game includes a chat mode for players using the same font as on the tagged city plants and the the android app. The thinking behind this is to create a situation where interpretation is encouraged over direct text communication, inspired (as in previous multiplayer online games I’ve worked on) by Tale of Tale’s Endless Forest.

The game will be made public closer to the event, but the code and assets for all these projects are now available on FoAM’s git repository.

Baltan Laboratories FaceSponge workshop

This is a very late report on a workshop on Facebook livecoding/hacking we gave at Baltan Laboratories in Eindhoven in May. We were invited us to run a workshop based on Naked on Pluto as part of their Tools Series:

The Tools Series is a series of Baltan Sessions that examines the complex and changing relationships artists and designers have with the technologies and tools they develop, modify or use to create, with an aim to explore social awareness around the tool choices they make as well as the (aesthetic) influences of these choices on the work they create.

During the Naked on Pluto project one of the key ways to confront the problems of centralised social networks turned out to be to encourage a deeper understanding of the processes and protocols of these sites.

So, like the previous workshop at CCCB, we centred this around a web application called FaceSponge, which we developed as a social programming interface giving quick access to the Facebook API and allowing participants to try out each other’s scripts. The other key issue was to find out people’s opinions, and so we collected answers on post-it’s to three questions for each area, which the participants later sorted for presentation to the public.

Social advertising

This workshop was perfectly timed with Facebook’s IPO, and as 82% of it’s revenue comes from advertising we started off by working on a simple spoof advert. We took one friend, and picked something they have ‘liked’ and wrote some code to promote it. This is what happens on social networks where a brand gets advertised to you because one of your friends follows or likes it. Being able to put a friend’s name in an advert is seen as an exciting future of advertising (or perhaps less so as the share price continues to drop).

function runme() {
    FB.api("/me/friends", function(friends) {
        var friend=friends.data[0];
        FB.api("/"+friend.id+"/likes", function(likes) {
            var like=likes.data[0];
            display(friend.name+" endorses "+like.name+" BUY SEVERAL TODAY!");
            FB.api("/"+like.id+"/picture?type=large", function(picture) {
                display_image(picture);
            });
        });
    });
}

Privacy

There are vast amounts of pictures available on facebook, and it was fun to write a script that presented them all back at in a chaotic manner without any other information. This also gave us a chance to show how the privacy on Facebook is imaginary, as the URL’s FB gives you for your friend’s pictures are public – regardless of anyone’s privacy settings.

// showing the holes in the walls                                               
// you think your photos are private?                                           
// these images are accessible without a login                                  
function runme() {
    FB.api('/me/friends', function(friends) {
        friends.data.forEach(function(friend) {
            FB.api('/'+friend.id+'/photos', function(f) {
                 if (f.data.length>0) {
                     var gallery=f.data[0];
                     // show the public url                                     
                     display(gallery.images[0].source);
                     // show the image                                          
                     display_image(gallery.images[0].source);
                 }
            });
        });
    });
}

Social pressures

The third area we were interested in exploring was the more subtle ways that social media are affecting communication methods. We came up with this strange script that collects the last things posted by your friends and puts them together without information on who posted them, or who they are for:

function runme() {
    FB.api('/me/friends', function(friends) {
        friends.data.forEach(function(friend) {
            FB.api('/'+friend.id+'/feed', function(feed) {
                if (feed.data && feed.data.length>0
                    && feed.data[0].message) {
                    display(feed.data[0].message);
                }
            });
        });
    });
}

We continued to play with and adapt these scripts in order to show more information. The mood was interesting as it flipped from serious to hilarity and then slight awkwardness at what we were dredging up. We followed each of these practical sessions by collecting feedback on thoughts and emotions for each section. Although this was a very demanding workshop (changing between coding, politics, funny juxtapositions of friend’s personal data and having to think about how it felt) we recorded a wide range of thoughts – from the dismissive, “doesn’t matter” to the outright enraged. Perhaps one of the most important aspects of this workshop was being able to expose these mechanisms to groups of people normally considered ‘users’.

Borrowed Scenery – spacesuits for plants and full screen scrolling in HTML5 canvas

Lots more of Theun’s new artwork which is coming on quickly over the last few days has been added to the game, the glass bubbles (more precisely cloches) are space suits for the plants from our world to live in, in their new patabotanical environment. Information about tagged plants is printed in a Voinych Manuscript inspired font.

I’ve been doing a bit more research into HTML5 canvas, turns out the the screen size doesn’t affect rendering framerate too much, and it now adapts to the browser size. Full screen scrolling is working – centring the view on the player. It wasn’t immediately obvious how this could be done without redrawing all the sprites each frame (which is way too slow), in this respect HTML5 development is really reminding me of 16bit era – hacking things like this to work quickly. It turns out it’s possible to use drawImage using the current canvas as it’s own input – so redrawing the whole thing with an offset:

function scroll(diff_x,diff_y) {
    // calculate the source and destination offsets - whether we offset
    // the source or destination rectangle depends on which direction 
    // we are travelling in 
    var sx=0;
    var dx=diff_x;
    var width=this.ctx.canvas.width-diff_x;
    if (diff_x<0) {
        sx=-diff_x;
        dx=0;
        width=this.ctx.canvas.width+diff_x;
    }
        
    var sy=0;
    var dy=diff_y;
    var height=this.ctx.canvas.height-diff_y;
    if (diff_y<0) {
        sy=-diff_y;
        dy=0;
        height=this.ctx.canvas.height+diff_y;
    }

    // remember to convert to ints, subpixel 
    // scrolling results in crazy artifacts
    this.ctx.drawImage(this.ctx.canvas,
                       ~~(sx),~~(sy),~~(width),~~(height),
                       ~~(dx),~~(dy),~~(width),~~(height));
}

Borrowed Scenery tendrils reach out

Some serious connecting work going on with borrowed scenery for joining physical and imaginary worlds together. A new Boskoi database is up and running, giving us a place to put all kinds of story elements and plants found in the city. Boskoi uses the Ushahidi platform, which provides an API the game is now using to pull all items tagged in the map (which can come from the Android app or web app) into the game where they can interact with players or other entities.

More of Theun’s artwork has gone in, including a magician tarot avatar on the left while the three plants in the screenshot above represent those tagged on the map below. Getting everything in the right place (map lat/long coordinates vs game location coordinates and then aligning the map) took a lot of time to get right!

Algorithmic fungi patterns

Central to the borrowed scenery game is an ecosystem of fungi that players will grow to feed the plants found by foragers in the city of Ghent using Boskoi. These fungi will work differently from the plants in Germination X, partly in response to some interesting game testing feedback – the fungi will only grow to the next stage when activated by a player, but the positions of new fungi will be algorithmically decided.

This calls on a kind of cellular automata, which requires a bit of prototyping to get the right values to make the kind of patterns I want. I mocked up something in clojure (so I can plug it into the game server easily) using ascii art for previewing the patterns resulting, the four stages of fungi are:

. : spore
o : young
O : adult
* : fruiting

Each fungi cell only goes from adult to fruiting if conditions are right (counting number of neighbours as in the game of life), after which 5 spores are created around it. After fruiting or adult stage the fungi dies and gets removed. This is a screenshot showing some different colonies which emerge starting with just two fungi spores:

                     .Ooo*.                o..O   
                     o*o ..                 *.O   
                      ..*.                 o*oo.  
                       ...                 .*OOoo 
                                            ..Oo  
                  O o                       ..*.  
                  O                            o  
              O       o                           
                oO                                
            .O                                    
           O*                                     
           oO..                                   
         o  o *.OO                                
             ..O                O O               
              ** o                O               
             .*..o              OooO              
             .o.*.             .oOO*O O           
               . .              ..O*O O.          
                                O*. O*o*.         
                                Oo OoO. .     

And here is the code:

(def *spore-count* 5)
(def *neighbour-distance* 15)
(def *max-neighbour* 60)
(def *min-neighbour* 0)

(defn make-cell-world [cells]
  {:cells cells})

(defn make-cell [x y]
  {:x x :y y
   :state "."})

(defn cell-dist [cell-a cell-b]
  (let [v {:x (- (:x cell-a) (:x cell-b))
           :y (- (:y cell-a) (:y cell-b))}]
    (Math/sqrt (+ (* (:x v) (:x v))
                  (* (:y v) (:y v))))))

(defn cell-process [cell neighbours]
  (if (< 50 (rand-int 100))
    (cond
     (= (:state cell) ".") (merge cell {:state "o"})
     (= (:state cell) "o") (merge cell {:state "O"})
     (= (:state cell) "O") (if (and (< (count neighbours) *max-neighbour*)
                                    (> (count neighbours) *min-neighbour*))
                             (merge cell {:state "*"})
                             (merge cell {:state "X"}))
     (= (:state cell) "*") (merge cell {:state "X"})
     :else cell)
    cell))

(defn cell-get-neighbours [world cell]
  (filter
   (fn [other]
     (< (cell-dist other cell) *neighbour-distance*))
   (:cells world)))

(defn cells-find [world x y]
  (filter
   (fn [cell] (and (= (:x cell) x)
                   (= (:y cell) y)))
   (:cells world)))

(defn cells-make-rnd-pos-list [cell]
  (repeatedly
   *spore-count*
   (fn [] {:x (+ (:x cell) (- (rand-int 3) 1))
           :y (+ (:y cell) (- (rand-int 3) 1))})))

(defn cells-spore [world]
  (merge world {:cells
                (concat
                 (reduce
                  (fn [r cell]
                    (if (= (:state cell) "*")
                      (reduce
                        (fn [r pos]
                          (if (empty? (cells-find world (:x pos) (:y pos)))
                            (cons (make-cell (:x pos) (:y pos)) r)
                            r))
                        r
                        (cells-make-rnd-pos-list cell))
                      r))
                  ()
                  (:cells world))
                 (:cells world))}))

(defn cells-death [world]
  (merge world {:cells
                (filter
                 (fn [cell]
                   (not (= (:state cell) "X")))
                 (:cells world))}))

(defn cells-run [world]
  (cells-death
   (cells-spore
    (merge world {:cells
                  (map
                   (fn [cell]
                     (cell-process cell (cell-get-neighbours world cell)))
                   (:cells world))}))))

(defn cell-world-print [world w h]
  (dotimes [sy h]
    (dotimes [sx w]
      (let [f (cells-find world sx sy)]
        (if (> (count f) 0)
          (print (:state (first f)))
          (print " "))))
    (print "\n")))

(defn cell-world-loop [world n]
  (println "-------------------------------------------------")
  (cell-world-print world 50 20)
  (when (> n 0)
    (recur (cells-run world) (- n 1))))


(defn -main []
  (cell-world-loop
   (make-cell-world (list
                     (make-cell 25 10)
                     (make-cell 27 10)))
   1000))

Live notation at the Arnolfini

I was honoured to take part in the live notation unit’s event at the Arnolfini on Friday, and to perform with Alex McLean and Hester Reeve in the evening.

Live notation is a project exploring connections between Live Art and Live Coding, both art forms revolving around performance, but with very different cultures and backgrounds.

The day started with workshops. The first one by Yuen Fong Ling played with the power structures inherent in Life Drawing. We tried breaking some conventions, instead of everyone drawing the same model – one scenario involved arranging the easels in a line where one person drew the model and everyone else copied the previous person in line. This ‘drawing machine’ resulted in an intriguing pictorial version of “Chinese whispers”. The second workshop involved programming choreography live via drawing and an overhead projector, firstly with workshop leader Kate Sicchio as the dancer, and then more and more livecoders joining in until the roles were reversed.

The performances consisted of a mix of live art and livecoding, and also served to demonstrate the breadth of approaches that these art forms represent – Wrongheaded performed a spectacular livecoding invasion of religious ritual, while Kate Sicchio followed beautiful instructions she’d received a couple of hours before interpreting Nicholas Poussin’s painting ‘The Triumph of David’ using brightly coloured silks. Thor Magnusson unleashed a sub bass rumbling agent driven visual approach to livecoding with a very considered minimal performance. As an audience member, I think livecoding needs a dose of cross fertilisation with related areas, especially if they are outside of the computer music sphere – we can think more about our roles, the situation and less about the mechanics. As a performer, I’m still processing (and waiting for photos) and will write a bit more on our performance in a few days.