Viruscraft: Genetic model connected to a tree visualisation

The genetic model we were working on previously has now been ported into a browser compatible form and connected to a new tree visualisation that displays the species that emerge as the host population adapts to a virus infection. It’s still a prototype with rough edges, but have a play with it here, some example pics:

tree7

This is one of the earlier attempts, which I like the look of, but later cleaned up versions are a bit clearer to read what is going on.

tree17

tree18

Firstly the genetic part is working in that the population evolves to get gradually better at coping with the virus infection (the fitness score increases) – it takes a bit too long at the moment, but it’s great to be able to see this working in realtime as it happens with new species branching off older ones.

One early observation is that this has the potential to show why diversity is beneficial. If you modify the virus (fitness function) at a point when there are lots of different species present, the chances are that a few of them will be resilient enough to the infection to expand into the new environmental niche, and things eventually continue as before. If you alter the virus when there is only a single really well evolved species that is a bit too good at coping with the existing virus – the chances are that you will cause the population to go extinct as it won’t be able to adapt. This is analogous to the situation with bananas: “To carry on growing the same genetic banana is stupid”.

A big chunk of the work here was actually spent optimising the code. It’s pretty amazing how developed browsers are for development – I’ve been using the profiler in the chromium browser to locate slowness and keep the frame rate as near 60 frames per second as possible. I just noticed taking the screenshot for this post the slowest single part seems to be the debug text rendering, strangely enough.

prof

Technology after Collapse

The philosophy underlying contemporary ‘seamless’ technology production seems to be one of endless energy, bountiful resources and waste being someone else’s problem. Naive working assumptions of some form are a requirement when designing for the future, but do we believe in these enough now to make them useful? Flashy ‘aspirational’ tech videos of ever thinning devices disappearing into the ‘cloud’ seem to be less common than they once were, so perhaps not.

Whenever a philosophy starts to look shaky, there are huge opportunities to try different ideas. For example, what happens if we instead use collapse as a working assumption for design? A sudden global societal collapse may be as unlikely as fusion power coming along to ‘solve’ climate change – but as a working assumption it shines a different light on usability – for a future that will mostly likely be somewhere between these two extremes. It’s also a view that is shared by some areas of research (particularly military) and I suspect by quite a few internal corporate future planning departments.

This research direction was triggered by Amber who found a paper called “Unplanned Obsolescence: Hardware and Software After Collapse”, this quote is from one of the papers it cites:

“In our present world of virtually unlimited resources, at least from the consumer’s perspective, acquiring the newest piece of technology is often considered a desirable lifestyle choice (e.g., for early adopters). A future of collapse might see a different picture…”

Sustainable Interaction Design: Obsolescence in a Future of Collapse and Resource Scarcity, Christian Remy, Elaine M. Huang

For this weeks seminar at the Institute for Music and Media in Düsseldorf, Julian Rohrhuber and I wanted to explore this area a little with the students. This is something that has been present in FoAM Kernow’s work for a long time – partly as we work closely with environmental researchers so it’s hard to avoid, but it’s also inherited from the history of the FoAM network.

excavate
We started by discussing the great Atari video game burial of 1983 and the comparative durability of the technology used at this time in history compared to that of today. This was an important aspect of the What Remains, so we looked at how we recycled old cartridges to store new software on for that project.

The specific scenario of collapse used in this type of research is one where all electronics production has ceased, all logistic and communication channels are destroyed, damaged or restricted. This could be due to a global crash, or an area becoming cut off from the rest of the world. Each component in our devices has a lifetime, most of them less than 10 years. While its possible that we could salvage components and repair our devices, there is also the issue of the knowledge required – much of it currently restricted to specialist silos of expertise.

“Social networks or institutions of people interested in computer repair could be invaluable for sourcing parts and maintaining skills needed to keep computing alive until devices and power are no longer scarce.”

Unplanned Obsolescence: Hardware and Software After Collapse, Esther Jang, Matthew Johnson, Edward Burnell, Kurtis Heimerl

tangle
Quote from Radix’s Adventures in Tech #3: Tanglebots, assemble! by Fiona Campbell-Howes

The starting point for our Tanglebots workshop for example, is an e-waste scrapheap we search through for things to take apart and reuse. Sustainability issues like this are often better presented implicitly, to set the scene – which then leads to a more interesting situation than starting with identical ‘kits’. You get to explore internal parts of cheap toys like dissecting natural organisms, and this process fits well with the haphazard reality of building technology, and working with family groups leads to knowledge sharing of the kind that would be needed to keep skills alive after collapse.

For the practical part of our seminar we wanted to focus on internet infrastructure, although originally developed to be resilient to war situations – the current scale involved, with undersea cables and centralised servers makes it one of the weakest aspects of our societies infrastructure to collapse, along with mobile technology:

“Long distance networking and information sharing becomes difficult with the decay of Internet infrastructure. Therefore, it may be crucial to establish communication channels, file sharing practices, and communities before the breakdown of mobile computing.”

Unplanned Obsolescence: Hardware and Software After Collapse

Some of the problems involved include software updates – as these become more challenging users become more prone to malware and viruses as they can’t be protected as effectively by the software authors. How would we manage software distribution post-collapse?

morse-dat
Unknown signal, a composite morse/data packet combination.

Global data transmissions are happening all the time on short wave radio. Certain frequencies bounce from the ionosphere or micro-meteorite trails to cover the globe, and data ‘modes’ are used rather than voice as they can travel further at lower power. There is even a slow-scan TV data mode which is used by the International Space Station to transmit images to amateur radio operators around the world – and regular contacts are made with people on the ground using shortwave frequencies.

iss

We spent the afternoon scanning the airwaves using the websdr.org network (no radio hardware required). You can choose your global location and frequency band (and by extension the time of day too – as transmissions travel further in the night time). The web interface allows you to record sound as wav files, which we analysed using Audacity.

In order to figure out what a particular transmission is, there are some clues you can use. The first is the frequency used, which you can look up via this chart as there are rules for what frequencies can be used for. We also found these example data recordings which we used to figure out the type of transmission data based on listening.

Here are some example recordings we made:

The first one was a bit of an enigma for a while – it sounds like really slow badly tuned music. We figured out it was probably an RTTY text messaging protocol called JT65 – for low power, long range short text communication:

jt65

The second one is still unknown, but there are a few clues. It’s a periodic burst of high speed data transmitted on a frequency band (6765 to 7000 kHz) that includes ‘FAX’ modes. It’s likely that this is some kind of weather map – but it doesn’t sound quite like any of the examples I’ve heard.

The third recording is a collection of simultaneous morse transmissions on similar frequencies. They all seem to be machine generated, a lot of repetition – probably some kind of beacons. I’ve tried to decipher them below but no hidden messages unfortunately!

decoded

This seminar was a good crash course for us all, and a way to get a handle on the sheer amount of activity on these not too well known global communication channels. A lot of these concepts are making their way into technology we are building, for example the penelope pattern matrix where we are using wood construction and modular circuits that we can adapt for different uses.

Future radio directions could be to set up a packet radio server to host some websites, even on a very tiny range (perhaps simply transmitted/received over audio) so we wouldn’t need a licence.

Additional reading material:

Pattern Matrix PCBs arrived & first tests

After triple checking the schematics and design files and ordering 80 PCBs (50 sensors and 30 i2c boards) there was an anxious wait for them to arrive and do some initial tests to find out if there were any mistakes. We now have enough boards to make two new pattern matrix devices, one 4X4 and one 5X5 – the plan is to evaluate the design and refine it for future builds.

img_20170628_105103

img_20170628_105226

img_20170628_110233

The picture below shows the first test boards populated and plugged it into the Pi – it’s much neater than the lego and breadboard prototype! The good news is that it seems to work so far, the only problem I’ve had is with the hall effect sensors, the pads are a tiny bit too close together for my skills. After a couple of tricky situations fixed with a de-soldering pump, I think I’ve come up with a strategy that works. I can bend the outer pins away from each other and solder the central pin first – then bend them back to finish the outer ones and being very careful not to bridge the pads.

img_20170628_173331

The blue jumpers on the square i2c boards allow you to program the device channel that the two expanders use – these could alternatively be hard soldered, but it’s good to have the option to reuse the parts or reconfigure a pattern matrix so we can add different sensors etc.

img_20170628_185854

For reference, the KiCad 3D viewer models look pretty close.

i2c-render

sensor-render

Viruscraft: building a ‘reasonably accurate’ genetic game world simulation

The concept for the viruscraft game is to have a realtime genetic model or simulation of the host evolution which is adapting to the properties of a virus you are building (either on screen or via a tangible interface as part of an exhibit). This model needs to be realistic, but only up to a point – it can be more of a caricature of biology than a research model would need to be, as our intentions are educational rather than biological research.

Using our previous species prototype as a starting point, we have a network of connected locations that can be inhabited by organisms. These organisms can jump to neighbouring locations and be infected by others in the same place at the same time. Now we need to figure out how different species of these organisms could emerge over time that evolve immunity to a virus – so we can build up a family tree (phylogeny) similar to the ones we created for the egglab game but that is responding to the viruses that you create in realtime as you play. The evolution itself also has to happen fast enough that you can see effects of your actions ‘quickly enough’, but we’ll worry more about that later.

For a job like this we need to move back from fancy visualisations and graphics and try to get some fundamental aspects working, using standard tools like graphviz to understand what is going on to save time.

The first thing to do is to add a fixed length genetic string to each individual organism, this is currently 40 elements long and is made from biologically based A,T,C and G nucleotide symbols. We chose these so we can use biological analysis tools to test the system as we go along just like any other genetic process (more on that below). The organisms can also reproduce by spawning copies of themselves. When they do this they introduce random errors in the genetic code of their offspring which represents mutation.

Previously we were using a ‘SIRS’ model for virus infection (susceptible -> infected -> recovered -> susceptible), based on 4 global parameters that determined the probability of jumping from one state to the next. Using the genetics, the probability of infection is now different for every individual based on:

1. Is a virus infected individual in the current location?

2. If so, use our genetic code to determine the probability of catching it. Currently we use the ratio of A’s to T’s in the genetic string as a totally arbitrary place-holder ‘fitness function’, the lower the number the better. AAAAAAT is bad (fitness: 6) while TTTTTTA is good (fitness: 0.1666) – so we would expect the A’s to disappear over time and the T’s increase in the genetic strings. This number also determines the probability of dying from the disease and (inversely) the probability of gaining immunity to it.

3. A very small ‘background infection’ probability which overrides this, so the virus is always present at a low level and can’t die out.

The next thing we need is a life cycle for the organism – this needs to include the possibility of death and the disease model is now a ‘SIR’ one, as once recovered, individuals cannot go back to being susceptible again.

state

All the other non virus related probabilities in the simulation (spawning offspring, moving location, natural death) are currently globally set – to make sure we are seeing evolution based only on disease related behaviour for now.

This model as it is could form the foundation of a world level visualisation – seeing organisms running around from place to place catching and spreading your virus and evolving resistance to it. However this is only half the story we want to tell in the game, as it doesn’t include our time based ‘phylogenetic’ family tree view. For this, we still need to figure out how to group individuals into species so we can fully visualise the effects of your virus on the evolution of all the populations as a whole.

First we need to decide exactly what a species is – which turns out to be quite an arbitrary concept. The rather course approach that seems to work here is to say that two organisms represent two distinct species if more than a quarter of their genes are different between them.

We can now check each organism as it’s born – and compare its genome against a ‘blueprint’ one that represents the species that it’s parent belongs to. If it’s similar enough we add it to its parents species, if it’s too different we create a new species for it. This new species will have a copy of its genome as the ‘blueprint’ to compare all its descendants with. This should mean we can build up a set of related species over time.

If we run the simulation for 5000 time steps we can generate a phylogenetic family tree at the end, using the branch points between species to connect them. We are hiding species with only 1 member to make it simpler, and the population is started off with 12 unique individuals. Only one of which (species 10) is successful – all the later species are descendants of that one:

test

The numbers here are the ID, fitness and size of population for each species. The colours are an indication of population size. The fitness seems to increase towards the right (as the number drops) – which is what we’d expect if new species are emerging that cope better with the virus. You can imagine changing the virus will cause all this to shift dramatically. The “game mechanic” for viruscraft will all be about tinkering with the virus in different ways that changes the underlying fitness function of the host, and thus the evolution of the populations.

As we used standard biological symbols for our genetic code, we can also convert each species into an entry in a FASTA format text file. These are used by researchers to determine population structure from limited information contained in genetic samples:


> 1 0.75 6
TGCTCTTGCGTACTAGACTGTTGACATCTCCACCGGATAA
> 3 0.46153846153846156 5
TGGTTTTCTGCTGTGGGGATAACCTGCCACTCAGTGGTGA
> 5 0.6153846153846154 171
CACTATCGCTCATTGCACTGTCGTGGTTTTAGTAACGAGC
...

In the FASTA file in the example above, the numbers after the ‘>’ are just used as identifiers and are the same as the tree above. The second line is the blueprint genome for the species (its first individual). We can now visualise these with one of many online tools for biological analysis:

phylo_tree

This analysis is attempting to rebuild the first tree in a way, but it doesn’t have as much information to go on as it’s only looking at similarity of the genetic code. Also 40 bases is not really enough to do this accurately with such a high mutation rate – but I think it’s a good practice to keep information in such a way that it can be analysed like this.

PCB design for pattern matrix 2

This is the pattern matrix 2 tangible sensor schematic, which is fairly simple – just 4 hall effect sensors and a capacitor to smooth out any noise on the power supply.

sensor-schematic

We need to make hundreds of these for the Penelope Project, and we can save some costs by using the built in pull up resistors in the MCP32017 to get a decent signal from the sensors. The difficulty with this PCB is arranging the sensors so they align with the magnets in the tangible programming block in the optimum manner. From tests with the prototype Lego rig, this isn’t actually too critical – but it’s set up so the lead length can be tweaked a bit at soldering time.

sensor-render

This took me about 20 variations to finally get right, but the circuit is just about simple enough that it can be made single sided – this is good because the top side will be partly exposed, while the lower side with all the copper traces can be protected. It’s good practice to have large areas of copper left connected to ground, partly as it’s a common connection needed all over the board, partly for stability but also it reduces the amount of chemicals required to etch the circuit – as only the parts around the traces need to be removed.

sensor-render2

The i2c expander board is a little more complicated. The design is made to be modular so we can stack up any number of these connected to the Raspberry Pi for different arrangements of sensors. Each board can deal with 8 sensor locations (each comprising 4 individual hall effect sensors). Their job is to convert the digital signals from each sensor into serial data (using the i2c protocol) so the Raspberry Pi can read them all just using 2 wires, plus 2 for power.

i2c-schematic

Each board can be configured to a separate i2c device address to tell it apart from the others using jumper connectors. This one had to be 2 sided, but I managed it without any ‘vias’ (holes to pass traces from one side of the board to another). I also added a power indicator LED as a last minute addition.

i2c

I’ve been learning the open source Kicad software to design these, which is now used by CERN for building the LHC, so it’s pretty fully featured! The idea is that you draw the schematic first, link each component with a physical ‘footprint’, then switch to the PCB design stage. Other software I’ve used in the past tries to route everything in one go for you (and can come up with some pretty strange and messy results). Kicad works in a semi-automatic manner – you need to draw each trace by hand, but it routes them around components and other traces, and suggests the shortest path for you. This is quite a lot better than a fully automatic approach as you have more control over the end result, and easily end up with a decent placement of all the parts.

i2c-render

This project is of course open hardware, and can be found on github here.

Pattern Matrix 2 haptic experiments

One of the potential future additions to our tangible programming hardware is haptic feedback, using sound/vibration to provide a extra channel of information through your fingers when programming with tangible blocks. We wanted to test this before designing the PCB hardware in case we could add it to the system simply – this was initiated by watching a reverse engineering video on youtube (I think by bigclivedotcom) where a technique was mentioned for sneakily reusing input lines (of which we have hundreds) as outputs when they are not being read, by using diodes to separate the signals.

img_20170602_164024

Piezo transducers are often used as cheap speakers, and they can also provide some touch feedback by using lower frequencies. We tested one to see if it interfered with the magnetic fields, and the diaphragm doesn’t seem to be ferrous metal (I’m not actually sure what it is made from) so it can be placed right over the sensors with no effect. A bit more Lego to attach it to the prototype sensor unit, and to see we can feel it through the tangible block when it’s touching the sensor.

The first attempt consisted of simply plugging a speaker into the i/o line from the MCP23017 port expander, switching between input and output on the Pi and adding a diode to prevent the output voltage getting to the sensor.

circuit1

The problem is all the circuits we’re using run on 3.3V, but the piezo speaker I’m using is rated up to 20V – so it’s just not strong enough to feel it. The Pi also provides 5V pins, so a second attempt to use a simple driver circuit (single transistor common emitter amplifier) to boost the voltage:

circuit2

You can hear this version in the video above – it’s playing a sound using a frequency crudely determined by the 4 bit value the block orientation represents whenever it changes. This is louder (loud enough at least to make a recording) but still not enough to feel easily, even when you put the piezo between the sensor and the block. So it seems the best way to get this to work properly will be via a separate circuit, not something we can slot into the existing input hardware. Another advantage of doing it separately is that we can treat it more like a multichannel audio setup with a dedicated processor that is not interrupted by the sensor reading. One solution I saw was to use the same H bridges as used to drive motors from microcontrollers as these take much higher voltages – this will be something to try later on.

New pattern matrix developments

A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.

A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.

We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:

Ferrite magnet test:

For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.

The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.

In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.

Viruscraft next steps

Following on from the first viruscraft workshop, we can now start planning the viruscraft game. The field of virology from genetics and interactions on the microscopic scale to the spread of disease and it’s effects on the ecosystem is huge, so we used the workshop primarily to identify the core things that are the most important to convey, and promising ways we can use to explain them. Getting high quality feedback so early on has allowed us to get a good sense of what is important with a diverse mix of people – the things that they picked up on (and just as importantly the things that they didn’t) saves a lot of time – and sharpens our focus right from the start.

1. Phylogeny

Phylogeny is the name given to a kind of family tree that shows the evolution and development of species over time. Ben’s work is concerned with how viruses can jump across species, so the concept of phylogeny is central to his work. There is also something very concrete and humbling about hearing the time scales involved here – nearest common ancestors of related species of fruit flies (his model organism for study) being tens of million years distant, while you need to go back 800 million years to find their common ancestor with us. These numbers are hard to grasp, but at the same time put things into perspective. Playing with and visualising long time spans will be an important aspect of this project.

2. Interaction of hosts and viruses

Viruses and their hosts are very different, hosts can be any creatures from bacteria, plants or animals while viruses seem little more than self replicating geometric shapes. Despite these differences, viruses and hosts have a huge effects on each other’s evolution – viruses need to spread and infect as many individuals as possible, but if they get ‘too good’ they kill off their hosts and they die off too. We’ve dealt with the co-evolution of hosts and diseases before in the red king sonification project, and again here the dynamics between competing organisms needs to be a central theme.

3. Shape matching/arms race

Our workshop participants found it surprising that one of the defining aspects of a viruses success is down to shape – e.g. whether you succumb to a cold or not is down to the ‘lock and key’ connection which needs to happen for the virus to attach to receptors on a cell’s external structure. These physical forms are the most promising area for viruscraft in terms of game mechanics – particularly if we are thinking about physical, tangible interfaces.

As an arms race situation between the host and the virus, our butterfly mimicry game for Cambridge university is a good reference for how a game mechanic can work in this context, as it accelerates evolution over the course of a minute or two as you play.

Given these themes we can now fill out our initial sketch a little with the new scientific information we learnt from the workshop as well as the feedback and ideas. One major addition is to add phylogeny as a kind of racing game in reverse, with time heading backwards so you can see the effects of your virus on a population, and also gives us a way to visualise your virus skipping between species. Like the Inca, we travel backwards into the future, giving us a view on biological history.

The primary game mechanic controlling the species jumping and virus success in general is combining shapes based on receptors on the host cells – this is very roughly shown above, the virus is currently attaching to the spherical receptors and infecting those host populations, if one of the players plugs in the shapes they are holding they will jump across to infect one of the other species too. The precise nature of this needs to be realistic enough that players get the idea that this is actually how things work rather than a metaphor, but schematic and abstract enough that it’s simple enough to understand within a few minutes of play. The other aspect of the shape matching is that the relatedness of host species should be reflected in the receptors in some sensible way (closely related ones should be similar) – so we need a procedural ‘receptor combination generator’ to make matching interesting, with zillions of possible combinations.

Another limitation is that of feasibility with regards to building a tangible interface. Luckily we have a few stepping stones that give us a range in terms of cost and risk. One of the activities from the workshop was building a large virus capsid out of bamboo – which led to a large scale modular origami climbing frame crossed with a dance mat as a grand vision, incorporating feedback consisting of haptics, lighting or projection mapping. In terms of practicalities (and budget) we need to get there a step at a time.

We can start by building a screen based system for manipulating the virus structure and shape matching, as it needs to be able to work in browsers anyway for accessibility – so this is a good place to begin. Once we have this working we can test the game properly and move on to building a smaller scale tangible interface, like in the sketch above – perhaps based on the pattern matrix technology, primarily designed for exhibitions and family groups to play with together.

The underlying model or simulation that is being manipulated is unusual for us in that it does not have to be a scientific research model, so we can design something that is realistic enough for educational purposes but no more. This is the next priority, and we can build on our viruscraft prototypes by having a model based on individuals navigating a mesh of connected nodes that alter to represent dynamic geographical changes (formation of land bridges and islands). The phylogeny chart can then be a separate visualisation of this same process (with references to futuristic driving games such as f-zero and wipeout). Time may run matched to the earth’s time line, so perhaps every day starts at the appearance of mammals and ends with the present day – the world map could match earth, or could be an alien planet. We can have external events to liven the game up, perhaps if you are doing too well your hosts could in impacted by climate change, asteroid strikes or tectonic shifts.

Debugging midi bytes with sonification

I’m currently working on some hardware for interfacing a Raspberry Pi 3 with MIDI. You can of course use a normal USB audio interface, and there is a ready made MIDI Hat module for the Pi already – but I only need MIDI out, and it shouldn’t be a problem to come up with something simple using the GPIO pins.


(Debugging: Wrong clock rate – but we have data! I ended up debugging by sonifying the MIDI bytes with a piezo speaker)

I followed the instructions here, expanded here and updated for Raspberry Pi 3 here – and using bits of all of them I managed to get it working in the end, so I thought it would be a good idea to document it here.

The main idea is that the Raspberry Pi already contains a serial interface we can reconfigure to send MIDI bytes, rather than bolting on lots of new hardware. All you need is to free it up from its default purpose which is to allow external terminal connection and as the bluetooth interface on the Pi3. We also need to slow it right down to 1980’s MIDI baud rate. You can then connect your MIDI cable to the Raspberry Pi’s ‘TX’ pin with a little bit of buffering hardware in between. This is needed to bring it up to 5V from the Pi’s 3.3V logic output (but as they need to be Schmitt triggers perhaps an additional function is to ‘hold’ the voltage for longer). I tried a normal TTL buffer and it didn’t work – but didn’t look into this too closely. As an aside, I found a UK made IC in my stockpile:

ukchip

On the Raspbian end of the equation we need a few tweaks to set up the Raspberry Pi. Edit /boot/cmdline.txt and remove the part that says console=serial0,115200, this stops the kernel listening to the serial device. Next edit /boot/config.txt to add these lines to the bottom:

dtoverlay=pi3-miniuart-bt # disable bluetooth
init_uart_clock=39062500
init_uart_baud=38400
dtparam=uart0_clkrate=48000000

This slows the serial (UART) clock down to the right rate relative to the Pi3 clock and disables bluetooth. Due to the OS switch to systemd on Linux a lot of the documentation is out of date, but I also ran:

sudo systemctl disable serial-getty@ttyAMA0.service

In order to turn off, and stop the console serial interface from starting at boot, I’m not sure if that was required. Then it’s a case of rebooting, compiling ttymidi (which required a small Makefile change to add -lpthread to the link line). This program sets up a MIDI device accessible by ALSA, so we can then install vkeyb (a virtual midi keyboard) and connect it to ttymidi in qjackcontrol. This is a pic of the testing setup with my trusty SU10 sampler.

miditest