New pattern matrix developments

A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.

A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.

We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:

Ferrite magnet test:

For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.

The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.

In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.

Viruscraft next steps

Following on from the first viruscraft workshop, we can now start planning the viruscraft game. The field of virology from genetics and interactions on the microscopic scale to the spread of disease and it’s effects on the ecosystem is huge, so we used the workshop primarily to identify the core things that are the most important to convey, and promising ways we can use to explain them. Getting high quality feedback so early on has allowed us to get a good sense of what is important with a diverse mix of people – the things that they picked up on (and just as importantly the things that they didn’t) saves a lot of time – and sharpens our focus right from the start.

1. Phylogeny

Phylogeny is the name given to a kind of family tree that shows the evolution and development of species over time. Ben’s work is concerned with how viruses can jump across species, so the concept of phylogeny is central to his work. There is also something very concrete and humbling about hearing the time scales involved here – nearest common ancestors of related species of fruit flies (his model organism for study) being tens of million years distant, while you need to go back 800 million years to find their common ancestor with us. These numbers are hard to grasp, but at the same time put things into perspective. Playing with and visualising long time spans will be an important aspect of this project.

2. Interaction of hosts and viruses

Viruses and their hosts are very different, hosts can be any creatures from bacteria, plants or animals while viruses seem little more than self replicating geometric shapes. Despite these differences, viruses and hosts have a huge effects on each other’s evolution – viruses need to spread and infect as many individuals as possible, but if they get ‘too good’ they kill off their hosts and they die off too. We’ve dealt with the co-evolution of hosts and diseases before in the red king sonification project, and again here the dynamics between competing organisms needs to be a central theme.

3. Shape matching/arms race

Our workshop participants found it surprising that one of the defining aspects of a viruses success is down to shape – e.g. whether you succumb to a cold or not is down to the ‘lock and key’ connection which needs to happen for the virus to attach to receptors on a cell’s external structure. These physical forms are the most promising area for viruscraft in terms of game mechanics – particularly if we are thinking about physical, tangible interfaces.

As an arms race situation between the host and the virus, our butterfly mimicry game for Cambridge university is a good reference for how a game mechanic can work in this context, as it accelerates evolution over the course of a minute or two as you play.

Given these themes we can now fill out our initial sketch a little with the new scientific information we learnt from the workshop as well as the feedback and ideas. One major addition is to add phylogeny as a kind of racing game in reverse, with time heading backwards so you can see the effects of your virus on a population, and also gives us a way to visualise your virus skipping between species. Like the Inca, we travel backwards into the future, giving us a view on biological history.

The primary game mechanic controlling the species jumping and virus success in general is combining shapes based on receptors on the host cells – this is very roughly shown above, the virus is currently attaching to the spherical receptors and infecting those host populations, if one of the players plugs in the shapes they are holding they will jump across to infect one of the other species too. The precise nature of this needs to be realistic enough that players get the idea that this is actually how things work rather than a metaphor, but schematic and abstract enough that it’s simple enough to understand within a few minutes of play. The other aspect of the shape matching is that the relatedness of host species should be reflected in the receptors in some sensible way (closely related ones should be similar) – so we need a procedural ‘receptor combination generator’ to make matching interesting, with zillions of possible combinations.

Another limitation is that of feasibility with regards to building a tangible interface. Luckily we have a few stepping stones that give us a range in terms of cost and risk. One of the activities from the workshop was building a large virus capsid out of bamboo – which led to a large scale modular origami climbing frame crossed with a dance mat as a grand vision, incorporating feedback consisting of haptics, lighting or projection mapping. In terms of practicalities (and budget) we need to get there a step at a time.

We can start by building a screen based system for manipulating the virus structure and shape matching, as it needs to be able to work in browsers anyway for accessibility – so this is a good place to begin. Once we have this working we can test the game properly and move on to building a smaller scale tangible interface, like in the sketch above – perhaps based on the pattern matrix technology, primarily designed for exhibitions and family groups to play with together.

The underlying model or simulation that is being manipulated is unusual for us in that it does not have to be a scientific research model, so we can design something that is realistic enough for educational purposes but no more. This is the next priority, and we can build on our viruscraft prototypes by having a model based on individuals navigating a mesh of connected nodes that alter to represent dynamic geographical changes (formation of land bridges and islands). The phylogeny chart can then be a separate visualisation of this same process (with references to futuristic driving games such as f-zero and wipeout). Time may run matched to the earth’s time line, so perhaps every day starts at the appearance of mammals and ends with the present day – the world map could match earth, or could be an alien planet. We can have external events to liven the game up, perhaps if you are doing too well your hosts could in impacted by climate change, asteroid strikes or tectonic shifts.