Deershed is a music festival designed to accommodate families with lots of activities for children. Part of this year’s festival was a Machines Tent, including Lego robot building, Mechano constructions, 3D printing and computer games.
Slub’s daily routine in the Machines Tent started by setting up the Al Jazari gamepad livecoding installation, a couple of hours with Martyn Eggleton teaching Scratch programming on an amazing quad Raspberry Pi machine (screens/processors and keyboards all built into a welded cube).
At some point we would switch to Minecraft, trying some experiments livecoding the LAN game world using Martyn’s system to access the Minecraft API using Waterbear, a visual programming language using a similar blocks approach as Scratch and Scheme Bricks.
During the afternoons Alex and I could try some music livecoding experiments. This was a great environment for playful audience participatory performances, with families continually passing through the tent I could use a dancemat to trigger synths in fluxus while Alex livecoded music designed to encourage people to jump up and down.
One of the most interesting things for me was to be able to see how lots of children (who mostly didn’t know each other) collaborate and self organise themselves in a LAN game, there was quite a pattern to it with all the groups:
- Mess around with Minecraft as usual (make some blocks, start building a house).
- Find something built by someone else, destroy a few bricks.
- Snap out of the game to notice that the other kids are complaining.
- Realise that there are other people in the world – and they are sat around them!
- Attempt to fix the damage.
At this point other people would join in to help fix things, after which there would be some kind of understanding reached between them to respect each other’s creations. This has all really inspired me to work on Al Jazari 2 which combines a lot of these ideas.
Preparations for a busy summer, new Al Jazari installation gamepads on the production line:
This weekend Alex and I are off to the Deershed Festival in Yorkshire to bring slub technology to the younger generation. We’ll be livecoding algorave, teaching scratch programming on Raspberry Pis and running an Al Jazari installation in between. Then onwards to London for a Sonic Bike Lab with Kaffe Matthews where we’re going to investigate the future of sonic bike technology and theory – including possibly, bike sensor driven synthesis and on the road post-apocalyptic mesh networking.
At the end of August I’m participating in my local media arts festival – Fascinate in Falmouth, where I’ll be dispensing a dose of algorave and probably even more musical robot techno.
Some decent sized screenshots of al jazari and scheme bricks rendered with fluxus’s tiled frame dump command. This set includes some satisfyingly glitchy al jazari shots – not sure what was causing this, I initially assumed it was the orthographic projection, but the same artefacts occurred on the perspective first-person robot views, so it needs further investigation.
Our second offshore Algorave on the MS Stubnitz, during the ship’s final night in London. The crowd was pleasingly diverse with lots of people new to algorave and livecoding, and although behind the scenes we had some hitches due to the ship’s impending departure for France, the event was relaxed and went smoothly. Our performance was honoured by guest appearance from Elvi$ Ca$h as well as a re-compile of the Al Jazari rave bots. As one of those spending the night on the ship afterwards, I had to be careful not to have too much of a lie in!
Better, and considerably wider angle photos can be found here by Yoshizen.
A big part of the look of Minecraft comes from it’s Smooth lighting, which includes an illumination model called ambient occlusion. Ambient occlusion darkens areas of an object based on how obscured they are from a wide area light source, for example an entire sky area, as opposed to a point light source. This is often complicated to calculate in realtime, but with simplified geometric voxels it’s fairly fast to do, building on the code from the hollowing out optimisation I mentioned previously. For each point on a cube’s corners you can add up it’s surrounding empty cubes – the more empty space in the 8 voxels, the brighter the corner needs to be. In this way, we are assuming light is coming from all directions (hence ambient) and don’t need to take into account positions of lights etc.
Here is the raw ambient occlusion lighting in Al Jazari 2, rendered as unlit vertex colours on each visible cube:
Then we add standard direct lighting to the ambient occlusion:
And here is the final image with textures added:
There are a few artefacts in the lighting due to the cubes not all being the same size (in order to minimise the complexity of what is drawn), here is the world coloured based on the size of the octree leaf node – the brighter blue areas are bigger cubes:
Once the ambient occlusion is calculated we only need to recalculate it when surrounding voxels are changed. In practice the splitting/joining of the octree levels when blocks are created and destroyed seems to take care of this in most cases.
;; returns a list of samples to test (define (occ-samples pos) (list (vadd pos (vector -1 -1 -1)) (vadd pos (vector -1 -1 0)) (vadd pos (vector -1 0 -1)) (vadd pos (vector -1 0 0)) (vadd pos (vector 0 -1 -1)) (vadd pos (vector 0 -1 0)) (vadd pos (vector 0 0 -1)) (vadd pos (vector 0 0 0)))) ;; count up the empty voxels and calculate occlusion value (define (calc-occlusion o pos) (let* ((samples (occ-samples pos)) (coverage (/ (foldl (lambda (p r) (+ (if (octree-leaf-empty? (octree-ref o p)) 1 0) r)) 0 samples) (length samples)))) (min 1 (* 2 coverage)))) ;; helper to set a bunch of verts in one go (define (pdata-list-set! k l v) (for-each (lambda (i) (pdata-set! k i v)) l)) ... ;; uses the cube's position and size to set the vertex colour on each corner (with-primitive my-cube (translate pos) (pdata-list-set! "c" '(3 7 19) (calc-occlusion o (vadd pos (vector 0 0 0)))) (pdata-list-set! "c" '(10 14 21) (calc-occlusion o (vadd pos (vector size size size)))) (pdata-list-set! "c" '(2 4 23) (calc-occlusion o (vadd pos (vector size 0 0)))) (pdata-list-set! "c" '(9 15 17) (calc-occlusion o (vadd pos (vector 0 size size)))) (pdata-list-set! "c" '(0 8 18) (calc-occlusion o (vadd pos (vector 0 size 0)))) (pdata-list-set! "c" '(5 13 22) (calc-occlusion o (vadd pos (vector size 0 size)))) (pdata-list-set! "c" '(6 12 16) (calc-occlusion o (vadd pos (vector 0 0 size)))) (pdata-list-set! "c" '(1 11 20) (calc-occlusion o (vadd pos (vector size size 0)))))
Optimisation is a game where you write more code in order to do less. In Al Jazari 2 doing less means drawing less blocks. Contiguous blocks of the same type are already automatically collapsed into single larger ones with the Octree – but if we can figure out which blocks are completely surrounded by other blocks, we can save more time by not building or drawing them either.
Here is a large sphere – clipped by the world volume, showing a slice through the internal block structure:
The next version has all internal blocks removed, in this case shaving 10% off the total primitives built and drawn:
The gaps in the sphere from the clipping allow us to look inside at how the octree has optimised the structure. The gain is higher in a more normal Minecraft set up with a reasonably flat floor covering a large amount of blocks. Here is the code involved, built on top of a functional library I’m building up on to manipulate this kind of data. It maps over each Octree leaf checking all the blocks it touches on each of its six sides, taking into account that the size of the leaf block may be bigger than one.
(define (octree-check-edge f o pos size) (define (do-x x y) (cond ((eq? x -1) #f) ((octree-leaf-empty? (octree-ref o (vadd pos (f x y)))) #t) (else (do-x (- x 1) y)))) (define (do-y y) (cond ((eq? y -1) #f) ((do-x size y) #t) (else (do-y (- y 1))))) (do-y size)) (define (octree-is-visible? o pos size) (or (octree-check-edge (lambda (x y) (vector size x y)) o pos size) (octree-check-edge (lambda (x y) (vector -1 x y)) o pos size) (octree-check-edge (lambda (x y) (vector x size y)) o pos size) (octree-check-edge (lambda (x y) (vector x -1 y)) o pos size) (octree-check-edge (lambda (x y) (vector x y size)) o pos size) (octree-check-edge (lambda (x y) (vector x y -1)) o pos size))) (define (octree-calc-viz o) (octree-map (lambda (v pos size depth) (octree-leaf (octree-is-visible? o pos size) (octree-leaf-value v))) o))
The Al Jazari 2 bots currently have six basic actions – move forward/backwards, turn 90 degrees left or right, pick up the block underneath them or drop the block to the space they are currently sitting on. Given these instructions, how do we procedurally build pyramids (of any given size) like this in their minecraft-esque world?
1. A pyramid can be built as a series of plateaus layered on top of each other, the plateaus can be built from material mined from nearby:
2. A single plateau can be built as a series of single block wide ridges next to each other, mined from a series of trenches. This is an example ridge/trench of size 3:
3. We need a gap between the ridge and the trench in order to place the plateau in the correct place in the pyramid (also the bots can only climb a single block at a time, otherwise they get stuck).
So in order to build a complete pyramid, we write the code to build a ridge/trench of any size and figure out the steps in-between to get the robot into the right position for the next one. The simplest ridge/trench is a single block long, so lets try writing some code to do that:
The lambda, and bot-sequence etc are scheme code required to get the bot language working, we’re just interested in the contents of the “seq”. After running these instructions we’re in the right place for the next block. Note that the majority of the actions are involved with positioning the bot after doing it’s work. To place the next cube we copy the code and add some more ‘forward’s (as we have to travel a bit further going back and forth):
This is already getting pretty long – we could do with a way to do repetition, so I’ve added a ‘repeat’ form to the language which takes a count, a name bound to the current iteration number (like a ‘for’ loop) and a list of instructions. This is the complete ridge/trench definition for any size, including gaps of any size:
The majority of the code is the maths to get the bot picking up and placing blocks further and further apart including the gap parameter. When collapsed into a function and run we get this:
With the bot ending up in the same place as it started. In order to create a square plateau we call this function, move sideways and repeat, and then move back to where we started again when we’re done:
Going from a plateau function to a pyramid is even shorter, and involves moving inwards diagonally and building smaller plateaus each time. Of course it also mines out a negative pyramid at the same time:
Here is a time lapse of a massive 8×8 pyramid being built, the code ‘compiles’ to 3224 low level instructions:
So this is a kind of programming that encourages solving problems through composition of abstractions – from the low level instructions, simple loops, primitive building constructs, up to complete structures. I’m not sure why in educational languages such as Scratch this is somehow sidelined (interestingly it’s not in Logo, it’s predecessor). Whether this is due to the ubiquity of imperative programming that leads to a focus on manipulation of state, or this kind of programming being considered as too advanced – but for me it’s fundamental, and I’m pretty sure it wouldn’t be that challenging for the kids in the CodeClub I’m running either.
Onboard a robot, who is following a sequence of instructions (in the spirit of the original Al Jazari), to pick up, carry and drops blocks in order to re-organise it’s environment. Halfway through we switch to a robot which is running code that makes it player controllable, so we can look around. This is all more than slightly influenced by teaching Scratch at Code Club and feeling the need for something 3D for them to play with. The scheme bricks coding interface is next…
Some screenshots of the in-progress next generation Al Jazari livecoding world. This is a voxel rendered world, inspired in part by Minecraft but with an emphasis on coding robots in scheme bricks who construct artefacts from the materials around them. The robot language is still to be designed, but will probably resemble Scratch.
You can ‘jump on board’ the different robots (cycling through them with ‘space’) and program them with commands which include picking up or dropping individual blocks. The program above allows you to control the robot with the ‘w’, ‘a’, ‘s’, ‘d’ keys with ‘z’ to tunnel downwards, and ‘x’ to remove the block in front of the robot.
The world is built quite simply from an octree – which provides an optimised structure for rendering the 64x64x64 level cube in realtime. The view below shows the compression – large areas containing the same material (or empty space) can be represented by leaf nodes terminating the tree early without needing to store each of the 262,144 1x1x1 cubes. After each edit, the octree may fragment or collapse it’s tree (via setting new values in 3D space or a ‘compress’ operation). The scheme code can be found here.