Here’s an article on midimutant we did with Aphex Twin for MagPi Magazine, written by Sean McManus. Most of the work on this project recently has revolved around exploring custom hardware using old FM synth chips from games consoles, but with any luck there should be some more evolved DX7 sounds around here soon.
A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.
A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.
We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:
Ferrite magnet test:
For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.
The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.
In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.
I’m currently working on some hardware for interfacing a Raspberry Pi 3 with MIDI. You can of course use a normal USB audio interface, and there is a ready made MIDI Hat module for the Pi already – but I only need MIDI out, and it shouldn’t be a problem to come up with something simple using the GPIO pins.
(Debugging: Wrong clock rate – but we have data! I ended up debugging by sonifying the MIDI bytes with a piezo speaker)
I followed the instructions here, expanded here and updated for Raspberry Pi 3 here – and using bits of all of them I managed to get it working in the end, so I thought it would be a good idea to document it here.
The main idea is that the Raspberry Pi already contains a serial interface we can reconfigure to send MIDI bytes, rather than bolting on lots of new hardware. All you need is to free it up from its default purpose which is to allow external terminal connection and as the bluetooth interface on the Pi3. We also need to slow it right down to 1980’s MIDI baud rate. You can then connect your MIDI cable to the Raspberry Pi’s ‘TX’ pin with a little bit of buffering hardware in between. This is needed to bring it up to 5V from the Pi’s 3.3V logic output (but as they need to be Schmitt triggers perhaps an additional function is to ‘hold’ the voltage for longer). I tried a normal TTL buffer and it didn’t work – but didn’t look into this too closely. As an aside, I found a UK made IC in my stockpile:
On the Raspbian end of the equation we need a few tweaks to set up the Raspberry Pi. Edit /boot/cmdline.txt and remove the part that says console=serial0,115200, this stops the kernel listening to the serial device. Next edit /boot/config.txt to add these lines to the bottom:
dtoverlay=pi3-miniuart-bt # disable bluetooth
init_uart_clock=39062500
init_uart_baud=38400
dtparam=uart0_clkrate=48000000
This slows the serial (UART) clock down to the right rate relative to the Pi3 clock and disables bluetooth. Due to the OS switch to systemd on Linux a lot of the documentation is out of date, but I also ran:
sudo systemctl disable serial-getty@ttyAMA0.service
In order to turn off, and stop the console serial interface from starting at boot, I’m not sure if that was required. Then it’s a case of rebooting, compiling ttymidi (which required a small Makefile change to add -lpthread to the link line). This program sets up a MIDI device accessible by ALSA, so we can then install vkeyb (a virtual midi keyboard) and connect it to ttymidi in qjackcontrol. This is a pic of the testing setup with my trusty SU10 sampler.
Here is a bit of a writeup of the gubbins going into the sonic kayaks project. We only have a few weeks to go until the kayaks’ maiden voyages at the British Science Festival, so we are ramping things up, with a week of intense testing and production last week with Kirsty Kemp, Kaffe Matthews and Chris Yesson joining us at FoAM Kernow. You can read Amber’s report on the week here.
The heart of the system is the Raspberry Pi 2. This is connected to a USB GPS dongle, and running the sonic bike software we have used in many cities over the last couple of years. We have some crucial additions such as two water temperature sensors and a hydrophone. We have also switched all audio processing over to pure data, so we can do a lot more sound wise – such as sonify sensor data directly.
How to do this well has been a tricky part to get right. There is a trade off between constant irritating sound (in a wild environment this is more of a problem than a city, as we found out in the first workshop) and ‘overcooking’ the sound so it’s too complex to be able to tell what the sensors are actually reporting.
This is the current pd patch – I settled on cutting out the sound when there is no change in temperature, so you only hear anything when you are paddling through a temperature gradient. The pitch represents the current temperature, but it’s normalised to the running minimum and maximum the kayak has observed. This makes it much more sensitive, but it takes a few minutes to self calibrate at the start. Currently it ranges from 70 to 970 Hz, with a little frequency modulation at 90 Hz to make the lower end more audible.
Here it is on the water with our brand new multi-kayak compatible mounting system and 3D printed horn built in blender. The horrible sound right at the start is my rubbish phone.
In addition to this, we have the hydrophone, which is really the star of the show. Even with a preamp we’re having to boost it in pure data by 12 times to hear things, but what we do hear is both mysterious and revealing. It seems that boat sounds are really loud – you can hear engines for quite a long way, useful in expanding your kayak senses if they are behind you. We also heard snapping sounds from underwater creatures and further up the Penryn river you can hear chains clinking and there seems to be a general background sound that changes as you move around.
We still want to add a layer of additional sounds to this experience for the Swansea festival for people to search for out on the water. We are planning different areas so you can choose to paddle into or away from “sonic areas” comprising multiple GPS zones. We spent the last day with Kaffe testing some quick ideas out:
Looking at sea temperature and sensing the hidden underwater world, climate change is the big subject we keep coming back to, so we are looking for ways to approach this topic with our strange new instrument.
I’ve tried a lot of different ways of teaching children programming, starting a few years ago with primary school children in a classroom, then doing inset training days for teachers and finally private tutoring in homes. For the finale to the weavingcodes project we are trying a new approach, teaching families about code, robotics and thread by building “tanglebots”.
The concept is to combine programming with physical objects, concentrating on sensor input and movement as output. It’s important that we incorporate our weavingcodes research process, so deliberately setting goals we don’t yet know the answers to.
Knitty tangle #tanglebot pic.twitter.com/qrnsu2HIi1
— Weaving Codes (@kairotic) March 20, 2016
The weaving focus allows us to ground the workshop in loom technology and demonstrate the challenges of manipulating thread, with its enormous history of technological development. For the first Cornwall workshop, Ellen started us off with an introduction using FoAM Kernow’s Harris loom and the fundamentals of weaving. We were also joined by Janet and Jon from lovebytes who are helping us to run these events. When first talking about possible workshops with children, we’d discussed the impossibility of making a functional loom in a couple of hours with only broken toys and lego – and so the focus on tangling was suggested by Alex as a way to turn these difficulties to an advantage. Similarly we created a series of prizes for different categories such as “Most technical effort with least impressive result” – inspired by hebocon events.
The workshop format we used is also influenced by Paul Granjon’s wrekshops – wherever possible we’re recycling by pulling apart e-waste, making use of electronics, motors, gears and ideas from the surprising complexity of what’s inside things people are throwing away. This turned out have a powerful implicit message about recycling, parents I talked to had tried taking things apart to learn about them, but the next step – making use of the parts discovered as we were doing here, needs a bit more help to do.
Also as normal for FoAM projects was the importance of the food, in this case tangled by Amber and Francesca to both provide sustenance and inspiration with cardamom knots, spiralised courgetti and tangle fritters.
The groups ended up a bit lopsided, so in future we plan to pre-arrange them as we did on the machine wilderness workshop. In order to do that we need to ask for more information from participants beforehand such as family ages and backgrounds.
We tried using the small Pi touchscreens – these were a bit too fiddly to get away without a mouse, but are much less oppressive somehow than larger PC monitors – as they are so small, they became incorporated into the tanglebots themselves.
Crocodile clips were the best way to connect to random/plundered electronics as well as the lego motors. These removed the need for soldering (which we had set up anyway, but in a separate space).
A selection of other notes we made:
All our resources are being uploaded to the kairotic github repository so other people can make use of the materials.
As well as being supported by AHRC Digital Transformations, this project was part of British Science Week, supported by the British Science Association.
Mongoose 2000 version 2 is now being used in the Banded Mongoose Research Project Fieldsite on the Mweya Peninsula, in the Queen Elizabeth National Park, western Uganda.
We’ve added two new focal observations – where a single mongoose in a specific life stage is followed, and has it’s activity recorded for 20 minutes. These observations include different events that can happen (fighting or cooperating with other individuals etc). Nearly all the interfaces are shown below – the system includes adding new packs or individuals, data review and syncronisation with other tablets via the Raspberry Pi.
Some photos of Mónica Arias running her “Hungry Birds” butterfly catching experiment at the Muséum national d’Histoire naturelle in Paris.
The Museum’s internet capability was challenging, so we ran the game server on a Raspberry Pi with an adhoc wifi and provided the data collection ourselves. The project is concerned with analysing pattern recognition and behaviour in predators. We’re using ten different wing patterns (or morphs), and assigning one at random to be the toxic one, and looking at how long it takes people to learn which are edible.
We recently had the second inset training day in programming related activities at Truro school. Following on from the previous session I didn’t want to introduce too much new stuff, so we concentrated on going back over Sonic Pi and Minecraft/Python programming in the morning, then discussed a lot more about our future workshops in the afternoon. These will include children from most of the primary schools in the Truro area and take place during the next term. I also wanted to use the day to work on some specific ideas the teachers wanted to get going. One of these involved running a Kinect camera with the Pi, which we managed to get more or less working – reading depth data in Python for potentially plugging into Minecraft at a later date.
The big idea I wanted to get feedback about was the use of robotics and electronics in conjunction with code (the photo above is my desk while preparing the day before). I didn’t have any lesson plans for this, but based on some comments from the first workshop and from playing a bit with this during CodeClub teaching I felt it might be good with this age group, who already know quite a bit about screen based activities. To keep costs down (as well as building in issues like e-waste) we’re planning to make use of recycled junk to extract motors from toys and hack them to do different things. It seems this doesn’t cause too many issues from safety POV (everything will be low-voltage and we can check everything beforehand), even the use of soldering irons seems to be acceptable as they have supervision.
The advantage of using code to move real things (as shown by a long history) is that it directly connects programming with the world outside of the screen (where it most certainly now has great importance in our lives), and at the same time results in teamwork – as it’s not easy for a single person to make a robot while programming it. When we tried this at Troon Primary they self organised into a group with one person programming while another was building stuff in lego and a third provided excited communication between them and more or less managed the task. Other programming activities tend to be more individualistic – with the possible exception of networked Minecraft, which is important but a very different form of collaboration.
Last week I had the chance to help out the Raspberry Pi foundation at their Picademy in Exeter. It was good to meet up with Sam Aaron again to talk livecoding on Pis, and also see how they run these events. They are designed for local teachers to get more confident with computers, programming and electronics to the point where they can start designing their own teaching materials on the second day of the two day course. This is a model I’m intending to use for the second inset teacher training day I’m doing next week at Truro school – it’s pretty exciting to see the ideas that they have for activities for their pupils, and a good challenge to help find ways to bring them into existence in a day.
We also had the ending of Future Thinking for Social Living at the Miners Court summer party last week. We exhibited the map made during the workshops, made lots of tea, and had some fun with the pattern matrix in musical mode out in the garden – I adapted Alex’s music system we used with Ellen in Munich to run on Raspberry Pi so it didn’t require a laptop, or a screen at all – simply a speaker. It was interesting how quickly people got the idea, in many ways music is easier to explain than weaving as listening while coding is multi-sensory.
One of the three citizen science game projects we currently have running at Foam Kernow is a commission for Mónica Arias at the Muséum national d’Histoire naturelle in Paris, who works with this research group. She needed to use the Evolving butterflies game we made last year for the Royal Society Summer exhibition to help her research in pattern evolution and recognition in predators, and make it into a citizen science game. This is a standalone game for the moment, but the source is here.
We had several issues to address with this version. Firstly a lot more butterfly wing patterns were needed – still the heliconius butterfly species but different types. Mónica then needed to record all player actions determining toxic or edible patterns, so we added a database which required a server (the original is an educational game that runs only on the browser, using webgl). She also needed to run the game in an exhibition in the museum where internet access is problematic, so we worked on a system than could run on a Raspberry Pi to provide a self contained wifi network (similar to Mongoose 2000). This allows her to use whatever makes sense for the museum or a specific event – multiple tablets or PC with a touchscreen, all can be connected via wifi to display the game in a normal browser with all the data recorded on the Pi.
The other aspect of this was to provide her with an ‘admin’ page where she can control and tweak the gameplay as well as collect the data for analysis. This is important as once it’s on a Raspberry Pi I can’t change anything or support it as I could in a normal webserver – but changes can still be made on the spot in reaction to how people play. This also makes the game more useful as researchers can add their own butterfly patterns and change how the selection works, and use it for more experiments in the future.
