Viruscraft: tangible interface electronics

We needed to get a working prototype of our tangible interface running for the second Viruscraft workshop, so that we could have a complete system up and running from the custom hardware to the on-screen game world for people to test and give us feedback on how it worked.

You can read Amber’s full report on this workshop here.

A recap on how this is supposed to work – we need to plug different types of ligand (protrusions on the outside of viruses that allow them to attach to cells in order to infect them) into a large wooden model virus capsid (the body of the virus) that we are building for use in exhibitions. The choices of shape you pick affects which host species your virus can infect on a simulation that is being run in real time. The best ligand shapes to use will change over time, as the hosts adapt to your mutations.

We decided to start by using a simple conductivity system to detect which ligand shape is plugged in, with three sets of three pairs of conductive pads on each triangle face of the capsid that can be connected (or not) with strips of copper on the bases of the ligands. This means we can detect between 7 different shapes in total. It’s very cheap and simple, but a conductivity measurement like this will never be 100% accurate, which is why we are doing it in triplicate – the redundancy allows us some leeway if they are not plugged fully in, but this also results in a lot of connections we need test.

We started by using 6 digital inputs on a Raspberry Pi GPIO – just reading two sets of connectors on one triangle. Even with the triple redundancy, the connection across the pads is never going to be perfectly conductive, so in order to increase the sensitivity of the digital inputs (which need quite a bit of voltage to register as ‘on’ without extra circuitry) we added gigantic pull up resistors (22 Mohm). This means they default to on (+5V) but with very little current passing through – so a tiny amount of conductivity to ground will drop the voltage enough to register as ‘off’. This means they are so sensitive that you can trigger a change just by bridging the gap on the pads with your fingers – this is exactly the same way that MakeyMakeys work.

Based on our previous experiences with the self-etched PCBs we designed some neater ones that we could get manufactured so they would be consistent. In the meantime we have also recently got our hands on a CNC mill we can use for future PCB construction like this – but haven’t quite had time to get it set up for production yet.

The next step was to come up with a more scalable way to read the signals, by moving everything onto a ATmega328 microcontroller – which is much cheaper than a Pi and suitable to embed in a custom circuit. The first plan was to keep it simple for the workshop and use it’s 18 of its i/o pins to directly read just 2 triangles, so we could quickly get on with the job of connecting it to the game world. We didn’t have time to get a PCB specifically designed for this, so I used strip board for rapid construction, just a few days before the workshop. To speed things up a bit I tried using KiCAD to help with the layout design.

Although KiCAD is not really designed for strip board it works quite well if you stick to a few rules. You set the grid to 2.54mm (1/10 inch) and use the lower copper side to represent the strips – drawing each one as you need them and leaving spaces to indicate where you need to cut the copper tracks. The top side copper layer is used to plan where the jumper wires will be placed to connect across the tracks where needed. The idea is to minimise these wires to make it simpler to build – and KiCAD is good for checking the circuit is correct for you as you organise everything. We then printed each side out on paper for reference, and glued the bottom side to the board to indicate where to cut the tracks.

One advantage of prototyping in strip board rather than making PCBs is that it’s much easier to modify things in an adhoc manner. For example, halfway through building this we made the wooden parts of the capsid structure, and thought it would be a better demonstration to use 3 or more faces for the workshop if we could. At this point we could use a multiplexer chip to allow the microcontroller to select which triangle it is reading and share the 9 digital pins between all the triangles connected, in order to read each one in turn. This would mean we could use a single microcontroller to read 16 (or more) entire triangles (144 digital signals) rather than just 2 (18 signals) which is probably be enough in fact for the final design.

To do this we used a 74HC4067 multiplexer chip left over from the flotsam tangible programming project. These work like a general purpose digitally controlled switch, allowing you to connect a single pin to one of 16 connections and select them using 4 inputs as a binary address. We can now use this chip to select which face is connected to ground, meaning that the microcontroller would only be able to read the connections for that one face – even though they are all sharing the same microcontroller input pins.

This seemed to work well on the breadboard, but has a big problem that in the pre-workshop rush gave me a frustrating time to figure out precisely. When a selected face has a connection on the same position as another face which is currently unselected by the multiplexer, the ground connection loops back round the circuit to the unselected face too. If that unselected face also has a connection across the pads in a location that the selected face doesn’t, then the microcontroller will mistakenly read that as a connection signal as well as it will be grounded and effectively read both of them at the same time.

The problem is that this only happens in particular arrangements of ligand codes, which means the interference doesn’t always occur. In order to prevent this grounding problem where it ‘leaks out’ to the rest of the circuit the fix is to add diodes to each separate face connection, which only allow the current to move in one direction.

So this simple conductivity system turned out to not be quite so simple as hoped!

Emergency diodes…

Connecting this to the simulation, which is running in the browser (entering the world of the web) was actually much less of a problem than that of stray electrons. The microcontroller is connected to a Raspberry Pi via i2c serial for data – this setup is borrowed from the pattern matrix, which means the Raspberry Pi can read part the microcontroller’s memory. A python script reads the state of the ligands over i2c and overwrites a text file each time a change is detected. This text file is accessible to a web server that is also hosting the game. The code running on the browser (in this case on the workshop participant’s laptops, but eventually on an exhibition computer) reads this text file every second and updates the virus that is used for the simulation automatically when it is changed by a player.

In the workshop we used the Raspberry Pi to serve the game over a wifi hotspot that people could connect their devices to, and we could get participants to test the tangible interface by adding/changing ligands. As they did this it would mutate the virus running in all their simulation worlds so they could see what was happening on the physical virus – and the hosts would get infected by the new virus on each player’s world.

Building Viruscraft planets

The last months have been booked solid with production on various projects, so I’m very behind with blogging. This means that there are a few loose threads that I need to look back on and write about, one of which is the Viruscraft world.

This is a screenshot of the current ‘alpha version’ of viruscraft we tested with the custom tangible interface (more on that soon) during the second game testing workshop. You can read Amber’s report on this workshop here.

It took a while to develop this planet, we started with a more conventional system:

This was a planet modelled in blender, the idea being to create islands and regions for different host species to live. We automatically made zones for different species to live (sea and land mainly but also higher terrain is possible) and then populated this with host organisms. There are two problems with this approach, one is that it’s a static model – we could have a set of variations on planets but e can’t change the terrain. One of the initial obvious references we talked about to the Wellcome Trust was Minecraft – which allows players to modify a procedurally generated terrain. Another problem with this is that if we look at our initial mood board for the project, where we collect artistic references:

There are lots of interesting ideas here, including perhaps using geomtric virus shapes in an interesting way. There is a problem I’ve seen on many games projects where if there isn’t a strong artistic style at the start, a more photorealistic logic takes over and pushes things into less and less interesting territory. So we decided to try something a little stranger and more interesting – could we create worlds built from triangular sections of terrain in the same was as viruses are created from triangular proteins?

There is also a distinct advantage to a procedural technique we can do with modular world building because the shape of the land affects how viruses spread – the number and physical location of population clusters has a big affect on how difficult it is, as a virus – to spread and become successful. The more dynamic and controllable we can make the world, the more interesting the game will be.

We chose a shape called a ‘Pentakis icosidodecahedron‘ – basically an 80 sided sphere built from triangles. One mistake I made with this was to not read the mathematical fine print – it took quite a lot of swearing before I figured out that it’s not regular, some of the triangles are equilaterals and others are isosceles. This means we have to squash and stretch the components to fit precisely, otherwise there are gaps between them.

Once we can arrange the triangles in the right places we need to make sure the terrain edges match up. It turns out there are 8 possible arrangements – from all sea, all land to the 6 varieties of crossings that can occur between the edges of the triangles:

We can also control the probability of a triangle of containing land, and use a slider as a sea level setting. Roughly speaking, the more unified the planet – the less, bigger populations possible and the easier it is for your virus to spread. The more fragmented and diverse the populations, the harder it is.

We’ll be adding more controls here in the future, so you’ll be able to customise your planet quite a bit more. We then made some animated low-poly host creatures to populate the planet. These have receptors for the virus that match the different shaped ligands – they breed and evolve to gain resistance to your virus over time.

In order to place them correctly on the right habitat we created some test ‘flat worlds’ to make sure everything was lined up correctly. It works by calculating barycentric coordinates in each triangle based on the particular terrain section, so we can generate a random location and check if its land or sea.

Another possibility is to have different themed planets, an ice world, urban planet, etc – perhaps rather than creating them you could explore a galaxy planet by planet. One important idea that came from the feedback would be to use the ‘difficulty setting’ that we’ve figured out from the terrain composition to provide tutorial levels that you could work through in increasing difficulty.

Viruscraft tangible hardware prototyping: etching PCBs

While we experiment with new fabrication techniques in order to shorten supply chains (with a philosophy of collapse in mind), electronics is problematic. Components can be salvaged and reclaimed but a particular problem is printed circuit board manufacture.

Like many we have tended to outsource this work to China, where the costs allow us to do short-run prototyping with our tiny budgets. Are these lower costs simply due to scale and the Communist support of the Maker community there? Or more worryingly, is this due to reduced regulations in the areas of employment rights and environmental protection?

Viruscraft online game virus interface, and some initial physical receptor designs

Either way, outsourcing this kind of fabrication sometimes doesn’t make sense. For Viruscraft we need a finer understanding of the materials involved to design our tangible virus interface. We need to detect the ‘shapes’ of receptors attached to a physical virus capsid, and the first approach we want to try is using voltage detection of the kind used in things like the Makey Makey. If we give each virus receptor three bits of information we can represent seven different receptor shapes, which is plenty for our needs. We need some kind of connection point that can read this information – a printed circuit board with exposed conductive ‘touch pads’ embedded in the virus capsid seems a good way to proceed.

The end point: a finished prototype capsid section with connection point for a receptor

This is also a chance to experiment with PCB manufacture, which is pretty messy and involves very nasty chemicals. In ‘doing it ourselves’, the theory is we can learn ways to minimise environmental degradation which could impact our designs in future, even if we end up having to outsource their manufacture.

The general concept is that copper clad boards are etched with acid in patterns that control the flow of electricity when components are soldered to them. We will be using some (quite old) sodium persulfate crystals here, added to water to form an acid bath for the board, which will be immersed until the exposed areas of copper are dissolved. We wanted to try two approaches to creating a ‘resist’ that would prevent the etching, one using insulation tape (actually horse leg bandage tape – well, we are based in a rural setting!) – the other simply using permanent marker to draw the circuit.

First we design a outline and print it out at the correct scale, you can see this in the background in the photo below. As we are using triangular plugs/sockets, we need to be able to ‘read’ the receptor in any orientation, so 18 pads are needed in total.

Then we completely cover a copper clad board in insulation tape. One of the big lessons from this process is that etching large areas uses up more acid and takes longer – so you only want to expose the absolute minimum.

Board completely covered in insulation tape

We stuck the template to the board and used it to guide cuts that go all the way through the tape to expose the copper.

Cutting the insulation tape

We are trying two different methods and designs here – one using just the tape cut-out where we create 18 separated islands which we can drill though and solder to create raised touchpads, and another drawing a more complex circuit with a sharpie felt tip pen. The second approach provides a flat connection area, as no soldering is needed on the touch pads themselves – but we can still solder connections on the edges of the board (this becomes a bit clearer later).

Two approaches – tape and marker pen

Here we have dissolved the sodium persulfate crystals to create the acid bath – note we are doing this outside, as the process creates fumes you don’t really want to be breathing. We started the temperature at 50 degrees – you are supposed to maintain this but we didn’t have a way to do that. The etching took about an hour and a half, it would probably be more like 10 minutes at the proper temperature, but it doesn’t seem to matter if you are not in a hurry.

Etchant, bath and board ready for exposure

Afterwards we store the acid as it can be used again, the blueness indicates the amount of dissolved copper – when this gets too dark it stops working, then you can neutralise the acid by adding bicarbonate of soda and take it to the tip for safe disposal. Obviously the labelling is important – not just the fact it’s toxic but the date it was made, for future use – it should last a few months.

Safely labelled etchant after use

Here is the etched board after washing well in water and drying – no copper visible!

The etched board, after washing.

The next step is to remove all the resist – both the tape (and the glue that comes with it), and the marker pen. That requires a bit of alcohol and elbow grease.

Amber cleaning up the etched PCB

Here we can see the copper and mostly nice clean tracks – there was one place which required some work with a scalpel to break a remaining connection, but we didn’t expect it to be this good for our first attempt!

Cleaned up boards

Now the PCBs can be separated…

PCBs after cutting out

And soldered…

And finally – tested on a Raspberry Pi, using the GPIO pins for now to read the receptor type directly. On this PCB we have two of the three sets of connectors working – so six values in total. This redundancy means that the more that match, the more confident we can be that we have the correct reading.

Testing the connections on a Raspberry Pi

Things to try next are some tweaks to the Pi so it can read the receptor more sensitively as currently it needs nearly the full 3.3V to register on the digital pins. The Makey Makey seem to do this by using large pull up resistors to detect the presence of tiny voltages and filtering the noisy signal this creates in software.

It’s not clear at this point if we will be making the final connectors in this way, but it would be nice if we can (and get a bit better at it) – at least we are now able to quickly iterate their design ourselves.

Other options to be looked into include using a small CNC mill to drill out PCB tracks, and more exotically – the use of micro-organisms to eat copper.

Viruscraft: Genetic model connected to a tree visualisation

The genetic model we were working on previously has now been ported into a browser compatible form and connected to a new tree visualisation that displays the species that emerge as the host population adapts to a virus infection. It’s still a prototype with rough edges, but have a play with it here, some example pics:

tree7

This is one of the earlier attempts, which I like the look of, but later cleaned up versions are a bit clearer to read what is going on.

tree17

tree18

Firstly the genetic part is working in that the population evolves to get gradually better at coping with the virus infection (the fitness score increases) – it takes a bit too long at the moment, but it’s great to be able to see this working in realtime as it happens with new species branching off older ones.

One early observation is that this has the potential to show why diversity is beneficial. If you modify the virus (fitness function) at a point when there are lots of different species present, the chances are that a few of them will be resilient enough to the infection to expand into the new environmental niche, and things eventually continue as before. If you alter the virus when there is only a single really well evolved species that is a bit too good at coping with the existing virus – the chances are that you will cause the population to go extinct as it won’t be able to adapt. This is analogous to the situation with bananas: “To carry on growing the same genetic banana is stupid”.

A big chunk of the work here was actually spent optimising the code. It’s pretty amazing how developed browsers are for development – I’ve been using the profiler in the chromium browser to locate slowness and keep the frame rate as near 60 frames per second as possible. I just noticed taking the screenshot for this post the slowest single part seems to be the debug text rendering, strangely enough.

prof

Viruscraft: building a ‘reasonably accurate’ genetic game world simulation

The concept for the viruscraft game is to have a realtime genetic model or simulation of the host evolution which is adapting to the properties of a virus you are building (either on screen or via a tangible interface as part of an exhibit). This model needs to be realistic, but only up to a point – it can be more of a caricature of biology than a research model would need to be, as our intentions are educational rather than biological research.

Using our previous species prototype as a starting point, we have a network of connected locations that can be inhabited by organisms. These organisms can jump to neighbouring locations and be infected by others in the same place at the same time. Now we need to figure out how different species of these organisms could emerge over time that evolve immunity to a virus – so we can build up a family tree (phylogeny) similar to the ones we created for the egglab game but that is responding to the viruses that you create in realtime as you play. The evolution itself also has to happen fast enough that you can see effects of your actions ‘quickly enough’, but we’ll worry more about that later.

For a job like this we need to move back from fancy visualisations and graphics and try to get some fundamental aspects working, using standard tools like graphviz to understand what is going on to save time.

The first thing to do is to add a fixed length genetic string to each individual organism, this is currently 40 elements long and is made from biologically based A,T,C and G nucleotide symbols. We chose these so we can use biological analysis tools to test the system as we go along just like any other genetic process (more on that below). The organisms can also reproduce by spawning copies of themselves. When they do this they introduce random errors in the genetic code of their offspring which represents mutation.

Previously we were using a ‘SIRS’ model for virus infection (susceptible -> infected -> recovered -> susceptible), based on 4 global parameters that determined the probability of jumping from one state to the next. Using the genetics, the probability of infection is now different for every individual based on:

1. Is a virus infected individual in the current location?

2. If so, use our genetic code to determine the probability of catching it. Currently we use the ratio of A’s to T’s in the genetic string as a totally arbitrary place-holder ‘fitness function’, the lower the number the better. AAAAAAT is bad (fitness: 6) while TTTTTTA is good (fitness: 0.1666) – so we would expect the A’s to disappear over time and the T’s increase in the genetic strings. This number also determines the probability of dying from the disease and (inversely) the probability of gaining immunity to it.

3. A very small ‘background infection’ probability which overrides this, so the virus is always present at a low level and can’t die out.

The next thing we need is a life cycle for the organism – this needs to include the possibility of death and the disease model is now a ‘SIR’ one, as once recovered, individuals cannot go back to being susceptible again.

state

All the other non virus related probabilities in the simulation (spawning offspring, moving location, natural death) are currently globally set – to make sure we are seeing evolution based only on disease related behaviour for now.

This model as it is could form the foundation of a world level visualisation – seeing organisms running around from place to place catching and spreading your virus and evolving resistance to it. However this is only half the story we want to tell in the game, as it doesn’t include our time based ‘phylogenetic’ family tree view. For this, we still need to figure out how to group individuals into species so we can fully visualise the effects of your virus on the evolution of all the populations as a whole.

First we need to decide exactly what a species is – which turns out to be quite an arbitrary concept. The rather course approach that seems to work here is to say that two organisms represent two distinct species if more than a quarter of their genes are different between them.

We can now check each organism as it’s born – and compare its genome against a ‘blueprint’ one that represents the species that it’s parent belongs to. If it’s similar enough we add it to its parents species, if it’s too different we create a new species for it. This new species will have a copy of its genome as the ‘blueprint’ to compare all its descendants with. This should mean we can build up a set of related species over time.

If we run the simulation for 5000 time steps we can generate a phylogenetic family tree at the end, using the branch points between species to connect them. We are hiding species with only 1 member to make it simpler, and the population is started off with 12 unique individuals. Only one of which (species 10) is successful – all the later species are descendants of that one:

test

The numbers here are the ID, fitness and size of population for each species. The colours are an indication of population size. The fitness seems to increase towards the right (as the number drops) – which is what we’d expect if new species are emerging that cope better with the virus. You can imagine changing the virus will cause all this to shift dramatically. The “game mechanic” for viruscraft will all be about tinkering with the virus in different ways that changes the underlying fitness function of the host, and thus the evolution of the populations.

As we used standard biological symbols for our genetic code, we can also convert each species into an entry in a FASTA format text file. These are used by researchers to determine population structure from limited information contained in genetic samples:


> 1 0.75 6
TGCTCTTGCGTACTAGACTGTTGACATCTCCACCGGATAA
> 3 0.46153846153846156 5
TGGTTTTCTGCTGTGGGGATAACCTGCCACTCAGTGGTGA
> 5 0.6153846153846154 171
CACTATCGCTCATTGCACTGTCGTGGTTTTAGTAACGAGC
...

In the FASTA file in the example above, the numbers after the ‘>’ are just used as identifiers and are the same as the tree above. The second line is the blueprint genome for the species (its first individual). We can now visualise these with one of many online tools for biological analysis:

phylo_tree

This analysis is attempting to rebuild the first tree in a way, but it doesn’t have as much information to go on as it’s only looking at similarity of the genetic code. Also 40 bases is not really enough to do this accurately with such a high mutation rate – but I think it’s a good practice to keep information in such a way that it can be analysed like this.

Viruscraft next steps

Following on from the first viruscraft workshop, we can now start planning the viruscraft game. The field of virology from genetics and interactions on the microscopic scale to the spread of disease and it’s effects on the ecosystem is huge, so we used the workshop primarily to identify the core things that are the most important to convey, and promising ways we can use to explain them. Getting high quality feedback so early on has allowed us to get a good sense of what is important with a diverse mix of people – the things that they picked up on (and just as importantly the things that they didn’t) saves a lot of time – and sharpens our focus right from the start.

1. Phylogeny

Phylogeny is the name given to a kind of family tree that shows the evolution and development of species over time. Ben’s work is concerned with how viruses can jump across species, so the concept of phylogeny is central to his work. There is also something very concrete and humbling about hearing the time scales involved here – nearest common ancestors of related species of fruit flies (his model organism for study) being tens of million years distant, while you need to go back 800 million years to find their common ancestor with us. These numbers are hard to grasp, but at the same time put things into perspective. Playing with and visualising long time spans will be an important aspect of this project.

2. Interaction of hosts and viruses

Viruses and their hosts are very different, hosts can be any creatures from bacteria, plants or animals while viruses seem little more than self replicating geometric shapes. Despite these differences, viruses and hosts have a huge effects on each other’s evolution – viruses need to spread and infect as many individuals as possible, but if they get ‘too good’ they kill off their hosts and they die off too. We’ve dealt with the co-evolution of hosts and diseases before in the red king sonification project, and again here the dynamics between competing organisms needs to be a central theme.

3. Shape matching/arms race

Our workshop participants found it surprising that one of the defining aspects of a viruses success is down to shape – e.g. whether you succumb to a cold or not is down to the ‘lock and key’ connection which needs to happen for the virus to attach to receptors on a cell’s external structure. These physical forms are the most promising area for viruscraft in terms of game mechanics – particularly if we are thinking about physical, tangible interfaces.

As an arms race situation between the host and the virus, our butterfly mimicry game for Cambridge university is a good reference for how a game mechanic can work in this context, as it accelerates evolution over the course of a minute or two as you play.

Given these themes we can now fill out our initial sketch a little with the new scientific information we learnt from the workshop as well as the feedback and ideas. One major addition is to add phylogeny as a kind of racing game in reverse, with time heading backwards so you can see the effects of your virus on a population, and also gives us a way to visualise your virus skipping between species. Like the Inca, we travel backwards into the future, giving us a view on biological history.

The primary game mechanic controlling the species jumping and virus success in general is combining shapes based on receptors on the host cells – this is very roughly shown above, the virus is currently attaching to the spherical receptors and infecting those host populations, if one of the players plugs in the shapes they are holding they will jump across to infect one of the other species too. The precise nature of this needs to be realistic enough that players get the idea that this is actually how things work rather than a metaphor, but schematic and abstract enough that it’s simple enough to understand within a few minutes of play. The other aspect of the shape matching is that the relatedness of host species should be reflected in the receptors in some sensible way (closely related ones should be similar) – so we need a procedural ‘receptor combination generator’ to make matching interesting, with zillions of possible combinations.

Another limitation is that of feasibility with regards to building a tangible interface. Luckily we have a few stepping stones that give us a range in terms of cost and risk. One of the activities from the workshop was building a large virus capsid out of bamboo – which led to a large scale modular origami climbing frame crossed with a dance mat as a grand vision, incorporating feedback consisting of haptics, lighting or projection mapping. In terms of practicalities (and budget) we need to get there a step at a time.

We can start by building a screen based system for manipulating the virus structure and shape matching, as it needs to be able to work in browsers anyway for accessibility – so this is a good place to begin. Once we have this working we can test the game properly and move on to building a smaller scale tangible interface, like in the sketch above – perhaps based on the pattern matrix technology, primarily designed for exhibitions and family groups to play with together.

The underlying model or simulation that is being manipulated is unusual for us in that it does not have to be a scientific research model, so we can design something that is realistic enough for educational purposes but no more. This is the next priority, and we can build on our viruscraft prototypes by having a model based on individuals navigating a mesh of connected nodes that alter to represent dynamic geographical changes (formation of land bridges and islands). The phylogeny chart can then be a separate visualisation of this same process (with references to futuristic driving games such as f-zero and wipeout). Time may run matched to the earth’s time line, so perhaps every day starts at the appearance of mammals and ends with the present day – the world map could match earth, or could be an alien planet. We can have external events to liven the game up, perhaps if you are doing too well your hosts could in impacted by climate change, asteroid strikes or tectonic shifts.