On the Penelope project, our plan was to develop technologies that could be useful in constructing a swarm of robots which could be livecoded by using the pattern matrix – a general purpose tangible programming system based on the Raspberry Pi. In order to make communication possible remotely, radio is the most obvious approach to get up and running quickly (other options that are intriguing are infra-red and audible sound).
A commonly used and extremely cheap module for radio communication is the nrf24l01+, which uses the 2.4Ghz band (the same as wifi) and allows you to send messages between arbitrary numbers of transceivers. It provides a very flexible networking setup – you can address them all individually, or broadcast to many at one go (which doesn’t seem to be mentioned anywhere), or you can construct a mesh network, passing on messages to receivers outside of the range of the original sender.
There are a few examples online for getting these working with both Raspberry Pi, Arduino and bare-bones ATMel microprocessors as used in our prototyping system. However the tricky thing about radio communication is that it requires you to have both a transmitter and a receiver operating before you actually can tell if it works – it can be extremely difficult to know what is happening when you are developing both at the same time.
I found this quite a challenge – probably not helped by using different hardware at each end, as I needed to cobble together multiple examples to get anything to happen. Here are some things I picked up that might be handy if you are trying to get these radios to work:
- Before doing any sending or receiving, check that the SPI serial is set up properly by reading registers on the nrf24l01 and checking that they match the default settings in the data sheet.
- Then try writing to them and reading again to check that the writes actually take effect. Note that this only works when the radio is not sending or listening for transmissions! Not reading the documentation carefully cost me a few hours here.
- Make sure _all_ the registers match on both sides, there are obvious things like the frequency channel and data rate but also auto-acknowledge, crc and payload size all need to be exactly the same otherwise not even the interrupt will fire to let you know something has been received. Although it should have been obvious, most of my problems came down to this – using different example code and not knowing the defaults to begin with made it tricky to know for sure what was going on.
- Get the transmitter working first. There is a very useful function on the nrf24l01+ which allows you to check if a carrier signal is detected on a frequency without doing any other processing. There is an Arduino library scanner example program in the RF24 library which steps through all the frequencies checking for any carrier signals. You can easily see where your local wifi is with this, so you can pick an empty section of the frequency band to broadcast on and then watch for it’s carrier to be detected. Once I had seen that the Pi was actually sending *something* (you need to make it send as many messages as possible for this to work well) I could concentrate on why the receiver was rejecting or not tuned to it.
- There is a lot of stuff on the internet about power supply being a problem – I worried about this a lot and tried many different capacitors across the voltage pins, but in the end all my problems were software based. On the robots we’re using 3.7V Li-Ion batteries and a 3.3v regulator (a low-dropout one as the voltages are very close) plus 2.2uF capacitors to keep the supply smooth on both the 3.3v and 3.7v sides, which works fine even with the servos sharing the power.
One of the great unknowns following the first weavecoding project was the nature of tablet weave. Other than a few primitive attempts that didn’t work in all cases and lead us to further questions, modelling tablet weave fully was left as an undeciphered mystery. Tablet weave is a complex and particularly ancient form of weaving, while it’s simple to do with easily found materials, it produces a kind of double weave with twisting, and you can create crazy higher level 3D structure as it is free from the constraints of fixed loom technology.
The trick to start understanding this (I still have quite some way to go) came from only thinking about a single square tablet. If we follow the paths of each of the four threads while turning the square 90 degrees at a time we can see how tablet weaving is a combination of a weave (up and down movement) and a braid (left and right), as it twists the threads in relation to each other.
From this sketchy starting point it was possible to create two 3D objects to represent each twist, one for clockwise and another for anticlockwise. If you colour the separate threads appropriately and combine them together you get something like this:
While this looks fancy, it’s wrong. The threads may be in the correct form conceptually, but woven structure comes about as a relationship between the positioning of the threads and the tension applied to them. Many of the threads above should be pulled straight and push others out of the way to give a pattern that was actually straight stripes of colour, rather than chevrons. So how can we add tension?
One way to approach this problem would be to use a physical simulation of the kind usually applied to cloth simulation, and ‘relax’ the the threads to achieve a realistic result, using a stochastic approach to iteratively tighten them within collision constraints, until it ‘looked right’. The problem with this is that it wouldn’t lead to a deeper understanding of what is going on here. This in a way is related to a bigger issue with AI and machine learning, where techniques like artificial neural networks can be trained to solve problems well enough to be useful, for example in speech recognition – but do not provide any new knowledge about language or understanding of deeper scientific issues.
So if we want to understand some of the ‘thread logic’ of table weaving, we are can approach this in a more symbolic manner. Can we add additional straightened threads to our two twisted ones?
As with the twists, there need to be two forms of straightening – left or right twist to straightened threads, and then we need to get back from a straightened thread to a left or right twist.
Notice that some of these shapes connect, while others are incompatible. We can start with the original twisted weave above, and process it to pull the threads straight. In order to do this we need to know the past and future actions of the weaver, or the current twist in the context of those before and after it. This makes sense, as when weaving structure emerges fully a few wefts behind the current one you are weaving – only as the tension is applied to the fabric does it take form.
The rules to describe this turn out to be well represented as a diagram. The nodes are the 3D shapes required and the edges are the actions of the weaver (the special ‘floating’ state change interestingly depends on the action before the last one – memory does seem important in tablet weaving).
For example, we can ‘left twist’ repeatedly (the top right state) as the arrow points to itself. If we start going in the other direction we then need to pass through two straightening states to get to a full ‘right twist’. If we start going backwards and forwards in smaller numbers of turns then more complex things happen.
When we process the first weave with these rules, you can see some of the straightening effects. The tension on the threads means that some cover up others, e.g none of the yellow threads are now visible on the top of the fabric at all.
The structure is more visible here than on a real weaving as the threads are thinner that they would be for the resulting weave which would be more densely packed together (this is less realistic but helps to understand what is going on).
How do we know if any of this is correct? The only way to test this for sure is against real weave. We can try out different sequences of actions and see if the model matches. As indicated above, tablet weaving is a technique that comprises several categories of weaves – these define some specific types of structure we can test.
Type 1: Repeated twists and turn back
Most normal tablet weave consists of twisting repeatedly 90 degrees in the same direction and weaving a weft each time. In practice there is only so far you can go in the same direction before the unwoven warp threads behind the tablets get tangled up, so you need to change direction and go the other way until they are untangled, providing some symmetry to the pattern. The first example has all the tablet threads aligned in the same sequence – and we weave 8 turns one way and 8 turns back again. You can see in the middle when we change direction we create a short straightened ‘float’ section which causes the tension to pull the threads straight here.
One of the further mysteries that our first tablet weaving simulations couldn’t previously recreate were situations where the pattern on the back and the front of the weave were not opposite of each other. This is highly unusual in weaving, but this model seems to represent this correctly. Here the actions are the same as the first example – 8 one way and then the other, but the thread colours in the tablets are offset from one another so they are staggered and you get the diagonal patterns.
Type 2: Single faced double weave
Part of the complexity of tablet weaving is because it is a kind of double weave – there are two intertwined weave structures happening at the same time. If we repeat two wefts of 90 degrees one way followed by two more the other direction, the two weaves remain on the same side of the textile – which can be seen clearly if we colour them appropriately. This example keeps the white weave on the top side with the brown one on the lower side.
Type 3: Degenerate floats
The third type of weave is not really a weave but a breakdown of the process caused by only weaving single 90 degree turns backwards and forwards repeatedly. This means half of the threads are not incorporated into the weave and ‘float’ along the surface on both sides.
While the language to fully describe the tablet weaving has yet to be developed properly, you can have a go yourself with this model which is currently online here (takes a few moments to render at first).
This gets us closer to a working model of tablet weaving, and provides something we can start to use for a more advanced aims of the Penelope project. For example, can we use the pattern matrix to tangibly livecode tablet weaving? Does this make it possible to explore and explain this type of weaving?
If this kind of textile wasn’t complicated enough, people in ancient times combined multiple weaving techniques, for example tablet weaving and warp weighted weaving in the same piece of fabric. Creating a kind of ‘grand unified’ weaving model is an additional future challenge, so we can start to understand better the thought processes involved in these advanced techniques.
Next week the weaving codes project assembles in the Museum fÃ¼r AbgÃ¼sse Klassischer Bildwerke in Munich. We’ll be working for a few days surrounded by these characters to get us in a classical mood:
The museum is hosting Ellen’s ‘Textile Matrix’ exhibition, a crossbreeding of logical science, religion, crafts and visual arts. The word ‘matrix’ originates from the latin word for mother or uterus, but today is predominantly used in mathematics, science and technology. Her work, as much of the weavingcodes project, provides new perspectives on connections between modern digital technology and ancient weaving.
Alex, Ellen, Emma and I will be in residency in the museum from the 6th May and I’ll be reporting our activities here as usual. On the 9th there will be talks and slub will be performing a special livecoding gig. On the 10th we will be presenting the work we have been doing during the residency and inviting people to participate in a citizen science event, exploring mathematics, weaving, music and code – including the brand new pattern matrix tangible weavecoding device.
Compilers are perceived to be magical artifacts, carefully crafted
by the wizards, and unfathomable by the mere mortals.
This is from a great paper by Abdulaziz Ghuloum on how compilers are made. It, along with a timely email from Julian Rohrhuber inspired me to have a go at making a tiny one for compiling scheme to betablocker – the idea is that you only need a handful of scheme primitives before you are able to bootstrap all the other code together. Betablocker is no way powerful enough to go too far with this, but I think there may be some good uses for compiling bits of fluxus scheme code on Raspberry Pi and Android, maybe not to native binary but mixing an interpreter with a fast intermediate language for graphics processing would make it possible to compile object deformation inner loops i.e. (pdata-map!) and friends for per-vertex operations, but have it transparently the same language to the user/livecoder.
I’ve only dealt with fixnums, booleans, null lists and a handful of simple core procedures, but compiling something stupid like this:
(and (zero? (- (+ 12 4) (+ 43 (add1 34)))) (boolean? #f))
Results in this:
Via the slightly more conventional looking:
pshl 24 pshl 8 add pshl 86 pshl 68 pshl 2 add add sub pshl 0 equ jmpz label-1 pshl 4 jmp label-2 label-1: pshl 0 label-2: pshl 0 pshl 3 and pshl 3 equ jmpz label-3 pshl 4 jmp label-4 label-3: pshl 0 label-4: and jmpz label-5 pshl 4 jmp label-6 label-5: pshl 0 label-6:
A system for creating an abundance of useless software for tiny devices. Spork Factory evolves programs that run on Atmel processors – the same make as found on the Arduino, in this case the ATtiny85 – a Â£2.50 8 pin 8bit CPU. I’m currently simply using a piezo speaker as an output and evolving programs based on the frequency of the sound produced by flipping the pins up and down, so creating 2bit synths using the Fourier transform as the fitness function. With more hardware (input as well as output) perhaps we could evolve small robots, or even maybe cheap claytronics or programmable matter experiments.
This project reuses the previous genetic programming experiments (including jgap as its genetic algorithm framework), and is also inspired by Till Bovermann’s recent work with Betablocker in Supercollider for bytecode synthesis.
The programs generated don’t use the Atmel instruction set directly, but interpret a custom one derived from Betablocker for two reasons. Atmel processors separate their instruction memory from data (the Harvard architecture) which makes it difficult to modify code as it’s running (either uploading new evolved code or running self modifying instructions), the other is that using a simplified custom instruction set makes it easier for genetic algorithms to create all kinds of strange programs that will always run.
I’ve added an ‘OUT’ instruction, which pops the top of the stack and writes it to the pins on the ATtiny, so the first thing a program needs to do is generate and output some data. The second thing it needs to do is create an oscillator to create a tone, after that the fitness function grades the program on the amount of frequencies present in the sound, encouraging it to make richer noises.
Here are two example programs from a single run, first the ancestor, a simple oscillator which evolved after 4 or 5 generations:
out out nop nop dec nop nop nop out nop jmpz 254 nop nop nop dup
It’s simply outputting 0’s, then using the ‘dec’ to decrement the top of the stack to make a 255 which sets the rightmost bit to 1 (the one the speaker is attached to) and then loops with the ‘jmpz’ causing it to oscillate. This program produces this fft plot:
After 100 or so further generations, this descendant program emerges. The dec is replaced by ‘pshl 81’ which does the same job (pushes the literal value 81 onto the stack, setting our speaker bit to 1) but also uses a ‘dup’ (duplicate top of the stack) to shuffle the values around to make a more complex output signal with more frequencies present:
out out not nop pshl 81 pshi 149 out nop out nop dup psh 170 jmp 0
Some further experiments, and perhaps even sound samples soon…
The first UK focus group/game testing session for Germination X will take place at the “Loading” gaming cafe in Falmouth, Cornwall on the 28th March at 18.00!
I spent thursday afternoon in the tangible auditory interfaces studio at the Helsinki Media Lab with Till Bovermann and Erich Berger. Till has a report on our investigations of taking the Betablocker virtual machine into new noisemaking territories.
Here is an example of a betablocker program written without the help of the graphical programming interface, by poking heap memory directly to create a simple sawtooth oscillator:
// begin LFSaw u32 threadID = m.add_thread(23); m.poke(23, ORG); m.poke(24, PIP); m.poke(25, 6); m.poke(26, JMP); m.poke(27, 1); // end