Sonic Kayaks: musical instruments for marine exploration

Here is a bit of a writeup of the gubbins going into the sonic kayaks project. We only have a few weeks to go until the kayaks’ maiden voyages at the British Science Festival, so we are ramping things up, with a week of intense testing and production last week with Kirsty Kemp, Kaffe Matthews and Chris Yesson joining us at FoAM Kernow. You can read Amber’s report on the week here.

IMG_20160727_173525

setup-annotated

The heart of the system is the Raspberry Pi 2. This is connected to a USB GPS dongle, and running the sonic bike software we have used in many cities over the last couple of years. We have some crucial additions such as two water temperature sensors and a hydrophone. We have also switched all audio processing over to pure data, so we can do a lot more sound wise – such as sonify sensor data directly.

How to do this well has been a tricky part to get right. There is a trade off between constant irritating sound (in a wild environment this is more of a problem than a city, as we found out in the first workshop) and ‘overcooking’ the sound so it’s too complex to be able to tell what the sensors are actually reporting.

watertempsine

This is the current pd patch – I settled on cutting out the sound when there is no change in temperature, so you only hear anything when you are paddling through a temperature gradient. The pitch represents the current temperature, but it’s normalised to the running minimum and maximum the kayak has observed. This makes it much more sensitive, but it takes a few minutes to self calibrate at the start. Currently it ranges from 70 to 970 Hz, with a little frequency modulation at 90 Hz to make the lower end more audible.

Here it is on the water with our brand new multi-kayak compatible mounting system and 3D printed horn built in blender. The horrible sound right at the start is my rubbish phone.

In addition to this, we have the hydrophone, which is really the star of the show. Even with a preamp we’re having to boost it in pure data by 12 times to hear things, but what we do hear is both mysterious and revealing. It seems that boat sounds are really loud – you can hear engines for quite a long way, useful in expanding your kayak senses if they are behind you. We also heard snapping sounds from underwater creatures and further up the Penryn river you can hear chains clinking and there seems to be a general background sound that changes as you move around.

We still want to add a layer of additional sounds to this experience for the Swansea festival for people to search for out on the water. We are planning different areas so you can choose to paddle into or away from “sonic areas” comprising multiple GPS zones. We spent the last day with Kaffe testing some quick ideas out:

Looking at sea temperature and sensing the hidden underwater world, climate change is the big subject we keep coming back to, so we are looking for ways to approach this topic with our strange new instrument.

A tanglebots workshop report

I’ve tried a lot of different ways of teaching children programming, starting a few years ago with primary school children in a classroom, then doing inset training days for teachers and finally private tutoring in homes. For the finale to the weavingcodes project we are trying a new approach, teaching families about code, robotics and thread by building “tanglebots”.

25956891515_f3d0ea3ec7

The concept is to combine programming with physical objects, concentrating on sensor input and movement as output. It’s important that we incorporate our weavingcodes research process, so deliberately setting goals we don’t yet know the answers to.

The weaving focus allows us to ground the workshop in loom technology and demonstrate the challenges of manipulating thread, with its enormous history of technological development. For the first Cornwall workshop, Ellen started us off with an introduction using FoAM Kernow’s Harris loom and the fundamentals of weaving. We were also joined by Janet and Jon from lovebytes who are helping us to run these events. When first talking about possible workshops with children, we’d discussed the impossibility of making a functional loom in a couple of hours with only broken toys and lego – and so the focus on tangling was suggested by Alex as a way to turn these difficulties to an advantage. Similarly we created a series of prizes for different categories such as “Most technical effort with least impressive result” – inspired by hebocon events.

25956891515_f3d0ea3ec7_2

25324352694_30f0a1a2a4_k2

The workshop format we used is also influenced by Paul Granjon’s wrekshops – wherever possible we’re recycling by pulling apart e-waste, making use of electronics, motors, gears and ideas from the surprising complexity of what’s inside things people are throwing away. This turned out have a powerful implicit message about recycling, parents I talked to had tried taking things apart to learn about them, but the next step – making use of the parts discovered as we were doing here, needs a bit more help to do.

Also as normal for FoAM projects was the importance of the food, in this case tangled by Amber and Francesca to both provide sustenance and inspiration with cardamom knots, spiralised courgetti and tangle fritters.

25931225896_007b088e5d_k2

The groups ended up a bit lopsided, so in future we plan to pre-arrange them as we did on the machine wilderness workshop. In order to do that we need to ask for more information from participants beforehand such as family ages and backgrounds.

We tried using the small Pi touchscreens – these were a bit too fiddly to get away without a mouse, but are much less oppressive somehow than larger PC monitors – as they are so small, they became incorporated into the tanglebots themselves.

Crocodile clips were the best way to connect to random/plundered electronics as well as the lego motors. These removed the need for soldering (which we had set up anyway, but in a separate space).

A selection of other notes we made:

  • Start with a manual tangling exercise (weaving with rope, tablets etc)
  • Lego has a strange all or nothing effect, once you start using it – everything has to work that way, avoiding it may lead to more creative options than including it
  • A first aid kit is needed for these sorts of things
  • The Pimoroni Explorer Hats are good but needed periodic resets in some cases – the motors seemed to get jammed, not sure if this is short circuits interrupting the i2c comms?
  • The Raspberry Pi docs are riddled with minor errors, e.g. the Scratch GPIO section on the explorer hats has a lot of sometimes confusing typos.

All our resources are being uploaded to the kairotic github repository so other people can make use of the materials.

As well as being supported by AHRC Digital Transformations, this project was part of British Science Week, supported by the British Science Association.

25956947035_a44aa6bdd9_k(1)

Mongoose 2000 version 2

Mongoose 2000 version 2 is now being used in the Banded Mongoose Research Project Fieldsite on the Mweya Peninsula, in the Queen Elizabeth National Park, western Uganda.

We’ve added two new focal observations – where a single mongoose in a specific life stage is followed, and has it’s activity recorded for 20 minutes. These observations include different events that can happen (fighting or cooperating with other individuals etc). Nearly all the interfaces are shown below – the system includes adding new packs or individuals, data review and syncronisation with other tablets via the Raspberry Pi.

interfaces-s

Hungry birds citizen science at the Paris Natural History Museum

Some photos of Mónica Arias running her “Hungry Birds” butterfly catching experiment at the Muséum national d’Histoire naturelle in Paris.

comp

The Museum’s internet capability was challenging, so we ran the game server on a Raspberry Pi with an adhoc wifi and provided the data collection ourselves. The project is concerned with analysing pattern recognition and behaviour in predators. We’re using ten different wing patterns (or morphs), and assigning one at random to be the toxic one, and looking at how long it takes people to learn which are edible.

hungrybirds2

Training teachers in coding, and thinking about e-waste/recycled robots

We recently had the second inset training day in programming related activities at Truro school. Following on from the previous session I didn’t want to introduce too much new stuff, so we concentrated on going back over Sonic Pi and Minecraft/Python programming in the morning, then discussed a lot more about our future workshops in the afternoon. These will include children from most of the primary schools in the Truro area and take place during the next term. I also wanted to use the day to work on some specific ideas the teachers wanted to get going. One of these involved running a Kinect camera with the Pi, which we managed to get more or less working – reading depth data in Python for potentially plugging into Minecraft at a later date.

IMG_20150617_143455

The big idea I wanted to get feedback about was the use of robotics and electronics in conjunction with code (the photo above is my desk while preparing the day before). I didn’t have any lesson plans for this, but based on some comments from the first workshop and from playing a bit with this during CodeClub teaching I felt it might be good with this age group, who already know quite a bit about screen based activities. To keep costs down (as well as building in issues like e-waste) we’re planning to make use of recycled junk to extract motors from toys and hack them to do different things. It seems this doesn’t cause too many issues from safety POV (everything will be low-voltage and we can check everything beforehand), even the use of soldering irons seems to be acceptable as they have supervision.

The advantage of using code to move real things (as shown by a long history) is that it directly connects programming with the world outside of the screen (where it most certainly now has great importance in our lives), and at the same time results in teamwork – as it’s not easy for a single person to make a robot while programming it. When we tried this at Troon Primary they self organised into a group with one person programming while another was building stuff in lego and a third provided excited communication between them and more or less managed the task. Other programming activities tend to be more individualistic – with the possible exception of networked Minecraft, which is important but a very different form of collaboration.

Picademy Exeter and Future Thinking for Social Living

Last week I had the chance to help out the Raspberry Pi foundation at their Picademy in Exeter. It was good to meet up with Sam Aaron again to talk livecoding on Pis, and also see how they run these events. They are designed for local teachers to get more confident with computers, programming and electronics to the point where they can start designing their own teaching materials on the second day of the two day course. This is a model I’m intending to use for the second inset teacher training day I’m doing next week at Truro school – it’s pretty exciting to see the ideas that they have for activities for their pupils, and a good challenge to help find ways to bring them into existence in a day.

IMG_20150605_141147

We also had the ending of Future Thinking for Social Living at the Miners Court summer party last week. We exhibited the map made during the workshops, made lots of tea, and had some fun with the pattern matrix in musical mode out in the garden – I adapted Alex’s music system we used with Ellen in Munich to run on Raspberry Pi so it didn’t require a laptop, or a screen at all – simply a speaker. It was interesting how quickly people got the idea, in many ways music is easier to explain than weaving as listening while coding is multi-sensory.

IMG_20150603_151747

Hungry birds 2 – the citizen science edition

One of the three citizen science game projects we currently have running at Foam Kernow is a commission for Mónica Arias at the Muséum national d’Histoire naturelle in Paris, who works with this research group. She needed to use the Evolving butterflies game we made last year for the Royal Society Summer exhibition to help her research in pattern evolution and recognition in predators, and make it into a citizen science game. This is a standalone game for the moment, but the source is here.

title

aie

We had several issues to address with this version. Firstly a lot more butterfly wing patterns were needed – still the heliconius butterfly species but different types. Mónica then needed to record all player actions determining toxic or edible patterns, so we added a database which required a server (the original is an educational game that runs only on the browser, using webgl). She also needed to run the game in an exhibition in the museum where internet access is problematic, so we worked on a system than could run on a Raspberry Pi to provide a self contained wifi network (similar to Mongoose 2000). This allows her to use whatever makes sense for the museum or a specific event – multiple tablets or PC with a touchscreen, all can be connected via wifi to display the game in a normal browser with all the data recorded on the Pi.

The other aspect of this was to provide her with an ‘admin’ page where she can control and tweak the gameplay as well as collect the data for analysis. This is important as once it’s on a Raspberry Pi I can’t change anything or support it as I could in a normal webserver – but changes can still be made on the spot in reaction to how people play. This also makes the game more useful as researchers can add their own butterfly patterns and change how the selection works, and use it for more experiments in the future.

admin

Pattern matrix – putting it together

Here is a member of staff at Miners Court trying some tangible weave coding in the midst of our crafts area – at the moment it’s simply displaying the weave structure on the simulated warp weighed loom with a single colour each for warp and weft threads, the next thing is to get ‘colour & weave’ patterns working.

DSC_1064

The pattern matrix is the second generation of tangible programming device from the weavecoding project. It’s been built as an open hardware project in collaboration with Falmouth University’s Makernow fablab, who have designed and built the chassis using many 3D printed parts and assembled the electronics using surface mount components (far beyond my stripboard skills).

Here you can see the aluminium framework supporting the AVR based row controller boards with the Raspberry Pi in the corner. The hall effect sensors detect magnetic fields – this picture was taken before any of the wiring was started.

IMG_20150408_105446

The row controllers are designed to read the sensor data and dispatch it to the Raspberry Pi using i2c serial communication running on their atmega328 processors. This design was arrived at after the experience of building flotsam which centralised all of the logic in the Raspberry Pi, resulting in lots of wiring required to collect the 128 bits of information and pass it to the GPIO port on the Pi. Using i2c has the advantage that you only need two wires to communicate everything, processing can be distributed and it can be far more modular and extendible in future. In fact we plan to try different sensors and configurations – so this is a great platform for experimenting with tangible programming.

This video shows the current operation of the sensors and row controllers, I’ve programmed the board with test code that displays the state of the magnetic field with the status LED, making sure that it can tell the orientation of the programming block:

The row controllers have a set of multiplexers that allow you to choose between 20 sensor inputs all routed to an analogue pin on the AVR. We’re just using digital here, but it means we can try totally different combinations of sensors without changing the rest of the hardware.

After getting the first couple of rows working and testing it with elderly people at our Miners Court residency there were a couple of issues. Firstly the magnets were really strong, and I worried about leaving it unattended with the programming blocks snapping together so violently (as we plan to use it in museum settings as well as at Miners Court). The other problem was that even with strong magnets, the placement of the blocks needed to be very precise. This is probably to do with the shape of the magnets, and the fact that the fields bend around them and reverse quite short distances from their edges.

To fix these bugs it was a fairly simple matter to take the blocks apart, remove 2 of the 3 magnets and add some rings to guide placement over the sensors properly:

IMG_20150418_114347

Future Thinking for Social Living: Weavecoding in assisted housing

Our work on weavecoding is now reaching out to other uses and projects. One is Future Thinking for Social Living, run by Magda Tyżlik-Carver and Fiona Hackney.

This research project aims to look at the relationship between wellbeing, home, making and technology and is centred on Miners Court, who provide assisted housing in Redruth in Cornwall. As well as a range of flats and accommodation, the residents have shared communal areas with a variety of activities throughout the week. Along with Christiane Berghoff, Robin Hawes and Lucie Hernandez we set up camp with a lot of materials for knitting, crochet and weaving as well as some Raspberry Pis and the all new pattern matrix tangible weavecoding device.

miners

The Future Thinking for Social Living project is set up to research how we can think more critically about home and community, and with particular focus on the future. From discussions with the staff at Miners Court – specific issues they are interested in are how to make better use of communal spaces, and how can they get more men involved with crafts and shared activities.

I’m also interested in how we can use these settings for artists residencies – how does working with people like this affect a design process, does working in such a place – and using it as way to start conversations (rather than being too much in ‘teacher mode’) affect the people living there positively? Also the weavecoding project provides some ideas in bridging gaps, both between technology and people – but also across gender gaps, mixing textiles with electronics for example.

miners2
Here is the new magnetic pattern matrix, running the 3D Raspberry Pi warp weighted loom simulation (more on this soon!) with a nice 4 shaft loom in the background.

On Monday and Tuesday we spent a long time talking, weaving, knitting and making cups of tea of course (and a bit of time debugging magnets on my part). I’ve found helping people weave with tablets on the inkle loom is a good way to get talking, as this seems new to even people who are experienced with crafts. It also appeals to people with mathematics or design background who normally are uninterested in knitting and other crafts, and seems gender neutral perhaps for the same reasons. It also helps to talk about the history of what we are weaving with, the fact that this is an ancient technique and yet there are so many surprises – I can’t really predict to them what will happen e.g. to the pattern when we change rotation direction, and this seems to be important.

What we have yet to do (but a few weeks to experiment yet) is bridge the technology gap. Many of them have an immediate reaction of distaste to computers, as most of them have them but report that they have become unusable or feel that they are not designed well with their needs in mind. Partly the situation of having some circuit boards getting tangled up in the more familiar materials and using the Raspberry Pi simulation to show what is happening on the loom next to it is a start. One interesting thing is that neither the Pi nor the AVR boards look enough like ‘a computer’ for it to stand out too much (which also part of the Pi’s role in the classroom) – this was more so after plugging it into their large TV and getting rid of the monitor. As it gradually gets into a working state, I’d like to first try using it to demonstrate well known weaves – e.g. plain, twill and satin.

Working in this environment on the pattern matrix between weaving with different people has already had an effect on it’s design process. One initial observation resulted in reducing the magnet strength – I hadn’t even considered before that having them snap together too forcefully would be a problem for some people. Such things are obvious in these kinds of settings.

Easter Python/Minecraft programming day at dbsCode

Thursday saw our second dbsCode Easter programming taster, and like last year we focused on minecraft programming with our procedural architecture api.

The main change this time was that for the 20 11-16 year old participants we doubled our teachers to 4 (Glen Pike, Francesca Sargent and Matthew Dodkins and me), plus a couple of interested parents helped us out too. This meant that the day was much more relaxed and we noticed they were engaged with the programming for a much higher proportion of the time. Another factor was that we went straight into coding, as none of them needed introduction to minecraft this year. I think one of the biggest strengths of this kind of learning is that they are able to easily switch between playful interaction (jumping into each other’s worlds, building stuff the normal way) and programming. This means there is low pressure which I think makes it more of a self driven activity, as well as making a long (4 hour) workshop possible.

Here are some screenshots of their creations – this was a melon palace created in a world that had somehow become sliced apart:

minecraftDay4

Inside the melon palace, the waterfall pulsed with a while loop and sleeps that altered the water source blocks.

minecraftDay5

As last year there was a lot of mixing of activities, using code to create big shapes and then editing them manually for the finer details:

minecraftDay3

IMG_20150402_142138

Here, a huge block of water inexplicably cuts through the scenery:

IMG_20150402_143112

And two houses, that became merged together and then filled with bookshelves and other homely items:

IMG_20150402_143000