Penelopean robotics (part 2)

Penelopean robotics are about rebuilding technology in the woven cosmos. You can read more about the theory in part 1, but roughly our aims are to:

  1. Embody Penelopean technological practice – they should be easily undone (taken apart) so they can be understandable, self documenting and repairable.
  2. They are not automated looms, but must eventually be capable of weaving in some form, maybe by interacting with ancient Greek weaving technology.
  3. Capable of embodying elements of ancient Greek poetry and dance, perhaps via livecoding.
  4. Be constructed in the spirit of Simone Giertz rather than Boston dynamics

Some potential further directions we have discussed are swarm robotics and the use of May poles as a stepping stone between dancing, braiding and weaving.

This is a story of one Penelopean robot.

The original concept of the weaving robot as described in the project was an ‘arachnoid mechanism’. We wanted to stick with this general direction and begin by building a six legged insectoid. After a bit of acting out of the movements required, and looking on youtube for similar robots it seemed that it was possible to do this with 3 servos, two for the front and back which rotate horizontally and one in the middle which rotates vertically.

Weaving crops up in plenty of medical, space and industrial situations – the strength and lightness of woven structures has lots of advantages over other techniques used in engineering, and in the spirit of the project it seems appropriate to try tablet weaving as a way to construct the main framework for the robot. For more info on how to do tablet weaving yourself, see this previous post and this one.

Double weave can be used for this as it allows you to weave tubes and pockets without any cutting or stitching, so they have very strong joins. Here is a sketch of the initial design:

Three joined tubes can hold the micro servos in place, and I later added another smaller one in between for a cylindrical lithium ion battery from Adafruit that was the perfect shape for this.

The process of weaving was a case of starting with a normal section, followed by beginning a double weave split via two weft threads, and checking it every couple of wefts to fit the first servo. When it’s big enough to tightly fit you close the first tube and immediately start a second, for the battery – then the middle servo (which fits at right angles to the others so is slightly smaller) and finally the last servo, with a normal section of tablet weave to close it up. This didn’t take very long, but to make more I could easily count wefts based on the first one and do a lot more quickly – for example, manufacturing swarms of these robots.

As a comparison, this process would be much more difficult and slow via 3D printing – as you can’t generally change anything on the fly, and the design needs to be completed before printing, and the softness turns out to be important.

Although the weave is strong in itself, it still needs a skeleton for the robot to stand up. I used 2mm wire for both this job as well as the legs. One piece of wire is used for the top (which loops around
the battery) and another bent double for the lower side. This scaffolding means that the middle legs will be able to push against something to raise the other legs to walk. Then all you need to do is tie loops of thread around each servo to hold it in place along with the scaffold. This may be unnecessary with a bit more development.

The PCB for the microcontroller and radio (more on that in a later post) can also be tucked under the top wire so it’s locked out of the way of the legs on top.


Christian Faubel showing us the unique nature of analogue computation and what happens when you apply this to robotics at the Penelope robotics workshop in Munich.

At the time of of the Penelope robotics workshop I could get the robot to shuffle a bit in generally the right direction but it wasn’t very efficient or convincing. It took a little servo sequencer programmed by the pattern matrix (in a similar manner to how it sequences music) to experiment with it and work it out properly. This takes a short list of symbols representing angles (from -90 to +90 degrees) which loop.

This is the working code for walking on the pattern matrix – more descriptive tokens will be made in future! These patterns are sent over radio to the robot whenever they change.

The general idea is that if you have 6 legs you can keep 3 feet on the ground while moving the others, which is much more stable than if you need to balance. The front and back servos job are to drive the robot forward and have the same movement as each other at the same time. The role of the middle legs are to lift the side of the robot up in alternate phase so only the ones pushing ‘back’ are touching the ground.

This meant we could use the pattern matrix to do things like switch direction by inverting the middle row above, which controls the central servo. This lifts the robot’s sides in the inverse phase, which means its feet (following the same pattern) move it in the other direction.

You can see a selection of different patterns of movement on this video:

The best aspect of the woven structure was how strong it was – it could throw itself around all day without problems and the flexibility helps too with the movements required. I could also stuff the robot into the bottom of a bag and it would be fine the next day.

There will be more info here on how to build them – for now the code is all here, the electronics is based on this prototype board – watch this space for more of the details on the radio enabled tangible robot livecoding.

Stackable hexagon prototype boards

We are working on a lot of hardware projects at the moment as we are interested in how to to rebuild technology from various alternative starting points. It seems most “off the shelf” hardware has converged on increasingly inaccessible and conservative forms, but luckily (and probably not due to entirely unrelated reasons) at the same time there has been an explosion in the availability, community documentation and potential of open hardware and DIY micro controllers, modules and components.

Right now we are simultaneously working on Viruscraft – building a tangible programming interface inspired by and for constructing virus geometry, Penelope where we are making more tangible interfaces and robotics inspired by ancient weaving tech (watch this space), Midimutant – re-purposing obsolete sound technology (more on this soon also) along with Sonic Kayaks – live sonification of underwater sensor data.

We start these projects with a more or less standard approach, prototyping and testing components using breadboards connected to Raspberry Pis or Arduinos, seeing if we can get them to work and investigating how it all performs. The next step would generally be to design a series of custom PCBs and eventually build a finished product.

The problem for us is that there seems to be a gap missing when we want to test something that needs to work in a relatively robust way (for a workshop, event or performance) but we do not have the final design settled yet, and need still need a lot of flexibility. Breadboards are problematic if you need to move things around a lot or depend on it for an workshop event. Strip board is very nearly good enough – except that you end up doing a lot of work rebuilding the same circuit many times, and it’s a very time consuming process. On the other hand, it’s wasteful ordering a custom PCB as you may well need to discard most of them.

In order to try and make this process a bit easier, we have designed a modular hexagonal ‘perma-proto’ PCB that can be ordered in large quantities and shared and reused for entirely different projects and uses.

The requirements were:

  • Make it small – both so it can be used when space is a constraint and so it can be cheap to manufacture, so we can have lots of them.
  • Include a small freeform prototying area – with a built in power bus.
  • It should have a second stackable breakout PCB version for more prototyping space.
  • Work with atmega328 or 168 – which are familiar Arduino compatible 8 bit microcontrollers with loads of library code.
  • Use a ‘through hole’ chip so we can use a socket for swapping them around, removing them after accidentally destroying them etc!
  • Expose all ports on pin headers in a logical way.
  • Include an ISP header so we can program it directly.
  • Include 4 pin power and i2c so it’s compatible with the rest of our recent hardware.

The open hardware KiCad files and gerber production files we are using are all here.

This is the main microcontroller board and stackable “hat” which has more space for adding components.

Viruscraft: tangible interface electronics

We needed to get a working prototype of our tangible interface running for the second Viruscraft workshop, so that we could have a complete system up and running from the custom hardware to the on-screen game world for people to test and give us feedback on how it worked.

You can read Amber’s full report on this workshop here.

A recap on how this is supposed to work – we need to plug different types of ligand (protrusions on the outside of viruses that allow them to attach to cells in order to infect them) into a large wooden model virus capsid (the body of the virus) that we are building for use in exhibitions. The choices of shape you pick affects which host species your virus can infect on a simulation that is being run in real time. The best ligand shapes to use will change over time, as the hosts adapt to your mutations.

We decided to start by using a simple conductivity system to detect which ligand shape is plugged in, with three sets of three pairs of conductive pads on each triangle face of the capsid that can be connected (or not) with strips of copper on the bases of the ligands. This means we can detect between 7 different shapes in total. It’s very cheap and simple, but a conductivity measurement like this will never be 100% accurate, which is why we are doing it in triplicate – the redundancy allows us some leeway if they are not plugged fully in, but this also results in a lot of connections we need test.

We started by using 6 digital inputs on a Raspberry Pi GPIO – just reading two sets of connectors on one triangle. Even with the triple redundancy, the connection across the pads is never going to be perfectly conductive, so in order to increase the sensitivity of the digital inputs (which need quite a bit of voltage to register as ‘on’ without extra circuitry) we added gigantic pull up resistors (22 Mohm). This means they default to on (+5V) but with very little current passing through – so a tiny amount of conductivity to ground will drop the voltage enough to register as ‘off’. This means they are so sensitive that you can trigger a change just by bridging the gap on the pads with your fingers – this is exactly the same way that MakeyMakeys work.

Based on our previous experiences with the self-etched PCBs we designed some neater ones that we could get manufactured so they would be consistent. In the meantime we have also recently got our hands on a CNC mill we can use for future PCB construction like this – but haven’t quite had time to get it set up for production yet.

The next step was to come up with a more scalable way to read the signals, by moving everything onto a ATmega328 microcontroller – which is much cheaper than a Pi and suitable to embed in a custom circuit. The first plan was to keep it simple for the workshop and use it’s 18 of its i/o pins to directly read just 2 triangles, so we could quickly get on with the job of connecting it to the game world. We didn’t have time to get a PCB specifically designed for this, so I used strip board for rapid construction, just a few days before the workshop. To speed things up a bit I tried using KiCAD to help with the layout design.

Although KiCAD is not really designed for strip board it works quite well if you stick to a few rules. You set the grid to 2.54mm (1/10 inch) and use the lower copper side to represent the strips – drawing each one as you need them and leaving spaces to indicate where you need to cut the copper tracks. The top side copper layer is used to plan where the jumper wires will be placed to connect across the tracks where needed. The idea is to minimise these wires to make it simpler to build – and KiCAD is good for checking the circuit is correct for you as you organise everything. We then printed each side out on paper for reference, and glued the bottom side to the board to indicate where to cut the tracks.

One advantage of prototyping in strip board rather than making PCBs is that it’s much easier to modify things in an adhoc manner. For example, halfway through building this we made the wooden parts of the capsid structure, and thought it would be a better demonstration to use 3 or more faces for the workshop if we could. At this point we could use a multiplexer chip to allow the microcontroller to select which triangle it is reading and share the 9 digital pins between all the triangles connected, in order to read each one in turn. This would mean we could use a single microcontroller to read 16 (or more) entire triangles (144 digital signals) rather than just 2 (18 signals) which is probably be enough in fact for the final design.

To do this we used a 74HC4067 multiplexer chip left over from the flotsam tangible programming project. These work like a general purpose digitally controlled switch, allowing you to connect a single pin to one of 16 connections and select them using 4 inputs as a binary address. We can now use this chip to select which face is connected to ground, meaning that the microcontroller would only be able to read the connections for that one face – even though they are all sharing the same microcontroller input pins.

This seemed to work well on the breadboard, but has a big problem that in the pre-workshop rush gave me a frustrating time to figure out precisely. When a selected face has a connection on the same position as another face which is currently unselected by the multiplexer, the ground connection loops back round the circuit to the unselected face too. If that unselected face also has a connection across the pads in a location that the selected face doesn’t, then the microcontroller will mistakenly read that as a connection signal as well as it will be grounded and effectively read both of them at the same time.

The problem is that this only happens in particular arrangements of ligand codes, which means the interference doesn’t always occur. In order to prevent this grounding problem where it ‘leaks out’ to the rest of the circuit the fix is to add diodes to each separate face connection, which only allow the current to move in one direction.

So this simple conductivity system turned out to not be quite so simple as hoped!

Emergency diodes…

Connecting this to the simulation, which is running in the browser (entering the world of the web) was actually much less of a problem than that of stray electrons. The microcontroller is connected to a Raspberry Pi via i2c serial for data – this setup is borrowed from the pattern matrix, which means the Raspberry Pi can read part the microcontroller’s memory. A python script reads the state of the ligands over i2c and overwrites a text file each time a change is detected. This text file is accessible to a web server that is also hosting the game. The code running on the browser (in this case on the workshop participant’s laptops, but eventually on an exhibition computer) reads this text file every second and updates the virus that is used for the simulation automatically when it is changed by a player.

In the workshop we used the Raspberry Pi to serve the game over a wifi hotspot that people could connect their devices to, and we could get participants to test the tangible interface by adding/changing ligands. As they did this it would mutate the virus running in all their simulation worlds so they could see what was happening on the physical virus – and the hosts would get infected by the new virus on each player’s world.

Penelopean robotics theory and the woven cosmos (part 1)

The Penelope project is concerned with many things, pattern structures in ancient poetry, comparisons of Andean and Greek mathematics, and the role of liveness in thought processes – to name just a few. We can also add robotics to this list.

A weaver in action is often mistaken for a robot – perhaps it’s the repetitive actions that mask the accompanying thought processes from our eyes. Similarly, a livecoding performer typing code into a laptop can face a problematic interpretation that they are grandstanding their expertise, rather than simply attempting to contribute to a party atmosphere at an Algorave. These activities often seem to be misunderstood in a similar manner but in different directions, perhaps partly due to the fact they both employ technology that is either greatly over or under-played by the ubiquity of their end results.

The Penelope project is concerned with using these technological misunderstandings in order to search for deeper assumptions we may have. One case we’ve discussed for a while has been to combine the situations above, if a weaver livecodes a robot which is operating an ancient loom – what is the response to this?

One way to design these situations well is to use an ‘artistic conceit’, a implicit fictitious assumption that provides a framework to fit things together so they make sense. For example, working with the Quipu database, we used the central idea that the Inca were attempting to communicate with us to naively employ cryptanalysis, visualisation and sonification in an attempt to understand their untranslatable knotted thread codes.

An important reference for the Penelope project for me is Carl Sagan. In his influential series ‘Cosmos’, weaving is mentioned many times as a central ordering principle of time and space for the Ionian thinkers of ancient Greece, who initially wrote down many of the fundamental scientific ideas he is introducing. He embarks on a thought experiment considering what life would be like today without the destruction of the library of Alexandria – which contained much of their discoveries and ancient knowledge from all over the world, now lost forever. His conjecture is that this single event may have set us back as much 500 years in scientific thought, so it’s just possible that without this event occurring, on a different timeline to ours – around about now, the first space ships would be returning from the surveying planets orbiting other stars.

It it this kind of imagination and suspension of disbelief that can give us a new perspective on our current predicaments – and perhaps a more humble position with which to question that our ways of thinking are inevitable, obvious and about as good as they can be.

For example, if the invention of general purpose computation and robotics had occurred in a society much earlier – one founded on the woven cosmos, what could it have looked like, and how would it have worked? What might it tell us about different ways of doing things?

Building Viruscraft planets

The last months have been booked solid with production on various projects, so I’m very behind with blogging. This means that there are a few loose threads that I need to look back on and write about, one of which is the Viruscraft world.

This is a screenshot of the current ‘alpha version’ of viruscraft we tested with the custom tangible interface (more on that soon) during the second game testing workshop. You can read Amber’s report on this workshop here.

It took a while to develop this planet, we started with a more conventional system:

This was a planet modelled in blender, the idea being to create islands and regions for different host species to live. We automatically made zones for different species to live (sea and land mainly but also higher terrain is possible) and then populated this with host organisms. There are two problems with this approach, one is that it’s a static model – we could have a set of variations on planets but e can’t change the terrain. One of the initial obvious references we talked about to the Wellcome Trust was Minecraft – which allows players to modify a procedurally generated terrain. Another problem with this is that if we look at our initial mood board for the project, where we collect artistic references:

There are lots of interesting ideas here, including perhaps using geomtric virus shapes in an interesting way. There is a problem I’ve seen on many games projects where if there isn’t a strong artistic style at the start, a more photorealistic logic takes over and pushes things into less and less interesting territory. So we decided to try something a little stranger and more interesting – could we create worlds built from triangular sections of terrain in the same was as viruses are created from triangular proteins?

There is also a distinct advantage to a procedural technique we can do with modular world building because the shape of the land affects how viruses spread – the number and physical location of population clusters has a big affect on how difficult it is, as a virus – to spread and become successful. The more dynamic and controllable we can make the world, the more interesting the game will be.

We chose a shape called a ‘Pentakis icosidodecahedron‘ – basically an 80 sided sphere built from triangles. One mistake I made with this was to not read the mathematical fine print – it took quite a lot of swearing before I figured out that it’s not regular, some of the triangles are equilaterals and others are isosceles. This means we have to squash and stretch the components to fit precisely, otherwise there are gaps between them.

Once we can arrange the triangles in the right places we need to make sure the terrain edges match up. It turns out there are 8 possible arrangements – from all sea, all land to the 6 varieties of crossings that can occur between the edges of the triangles:

We can also control the probability of a triangle of containing land, and use a slider as a sea level setting. Roughly speaking, the more unified the planet – the less, bigger populations possible and the easier it is for your virus to spread. The more fragmented and diverse the populations, the harder it is.

We’ll be adding more controls here in the future, so you’ll be able to customise your planet quite a bit more. We then made some animated low-poly host creatures to populate the planet. These have receptors for the virus that match the different shaped ligands – they breed and evolve to gain resistance to your virus over time.

In order to place them correctly on the right habitat we created some test ‘flat worlds’ to make sure everything was lined up correctly. It works by calculating barycentric coordinates in each triangle based on the particular terrain section, so we can generate a random location and check if its land or sea.

Another possibility is to have different themed planets, an ice world, urban planet, etc – perhaps rather than creating them you could explore a galaxy planet by planet. One important idea that came from the feedback would be to use the ‘difficulty setting’ that we’ve figured out from the terrain composition to provide tutorial levels that you could work through in increasing difficulty.

Viruscraft tangible hardware prototyping: etching PCBs

While we experiment with new fabrication techniques in order to shorten supply chains (with a philosophy of collapse in mind), electronics is problematic. Components can be salvaged and reclaimed but a particular problem is printed circuit board manufacture.

Like many we have tended to outsource this work to China, where the costs allow us to do short-run prototyping with our tiny budgets. Are these lower costs simply due to scale and the Communist support of the Maker community there? Or more worryingly, is this due to reduced regulations in the areas of employment rights and environmental protection?

Viruscraft online game virus interface, and some initial physical receptor designs

Either way, outsourcing this kind of fabrication sometimes doesn’t make sense. For Viruscraft we need a finer understanding of the materials involved to design our tangible virus interface. We need to detect the ‘shapes’ of receptors attached to a physical virus capsid, and the first approach we want to try is using voltage detection of the kind used in things like the Makey Makey. If we give each virus receptor three bits of information we can represent seven different receptor shapes, which is plenty for our needs. We need some kind of connection point that can read this information – a printed circuit board with exposed conductive ‘touch pads’ embedded in the virus capsid seems a good way to proceed.

The end point: a finished prototype capsid section with connection point for a receptor

This is also a chance to experiment with PCB manufacture, which is pretty messy and involves very nasty chemicals. In ‘doing it ourselves’, the theory is we can learn ways to minimise environmental degradation which could impact our designs in future, even if we end up having to outsource their manufacture.

The general concept is that copper clad boards are etched with acid in patterns that control the flow of electricity when components are soldered to them. We will be using some (quite old) sodium persulfate crystals here, added to water to form an acid bath for the board, which will be immersed until the exposed areas of copper are dissolved. We wanted to try two approaches to creating a ‘resist’ that would prevent the etching, one using insulation tape (actually horse leg bandage tape – well, we are based in a rural setting!) – the other simply using permanent marker to draw the circuit.

First we design a outline and print it out at the correct scale, you can see this in the background in the photo below. As we are using triangular plugs/sockets, we need to be able to ‘read’ the receptor in any orientation, so 18 pads are needed in total.

Then we completely cover a copper clad board in insulation tape. One of the big lessons from this process is that etching large areas uses up more acid and takes longer – so you only want to expose the absolute minimum.

Board completely covered in insulation tape

We stuck the template to the board and used it to guide cuts that go all the way through the tape to expose the copper.

Cutting the insulation tape

We are trying two different methods and designs here – one using just the tape cut-out where we create 18 separated islands which we can drill though and solder to create raised touchpads, and another drawing a more complex circuit with a sharpie felt tip pen. The second approach provides a flat connection area, as no soldering is needed on the touch pads themselves – but we can still solder connections on the edges of the board (this becomes a bit clearer later).

Two approaches – tape and marker pen

Here we have dissolved the sodium persulfate crystals to create the acid bath – note we are doing this outside, as the process creates fumes you don’t really want to be breathing. We started the temperature at 50 degrees – you are supposed to maintain this but we didn’t have a way to do that. The etching took about an hour and a half, it would probably be more like 10 minutes at the proper temperature, but it doesn’t seem to matter if you are not in a hurry.

Etchant, bath and board ready for exposure

Afterwards we store the acid as it can be used again, the blueness indicates the amount of dissolved copper – when this gets too dark it stops working, then you can neutralise the acid by adding bicarbonate of soda and take it to the tip for safe disposal. Obviously the labelling is important – not just the fact it’s toxic but the date it was made, for future use – it should last a few months.

Safely labelled etchant after use

Here is the etched board after washing well in water and drying – no copper visible!

The etched board, after washing.

The next step is to remove all the resist – both the tape (and the glue that comes with it), and the marker pen. That requires a bit of alcohol and elbow grease.

Amber cleaning up the etched PCB

Here we can see the copper and mostly nice clean tracks – there was one place which required some work with a scalpel to break a remaining connection, but we didn’t expect it to be this good for our first attempt!

Cleaned up boards

Now the PCBs can be separated…

PCBs after cutting out

And soldered…

And finally – tested on a Raspberry Pi, using the GPIO pins for now to read the receptor type directly. On this PCB we have two of the three sets of connectors working – so six values in total. This redundancy means that the more that match, the more confident we can be that we have the correct reading.

Testing the connections on a Raspberry Pi

Things to try next are some tweaks to the Pi so it can read the receptor more sensitively as currently it needs nearly the full 3.3V to register on the digital pins. The Makey Makey seem to do this by using large pull up resistors to detect the presence of tiny voltages and filtering the noisy signal this creates in software.

It’s not clear at this point if we will be making the final connectors in this way, but it would be nice if we can (and get a bit better at it) – at least we are now able to quickly iterate their design ourselves.

Other options to be looked into include using a small CNC mill to drill out PCB tracks, and more exotically – the use of micro-organisms to eat copper.

Midimutant in MagPi Magazine

Here’s an article on midimutant we did with Aphex Twin for MagPi Magazine, written by Sean McManus. Most of the work on this project recently has revolved around exploring custom hardware using old FM synth chips from games consoles, but with any luck there should be some more evolved DX7 sounds around here soon.

Further attempts at untangling tablet weave

One of the great unknowns following the first weavecoding project was the nature of tablet weave. Other than a few primitive attempts that didn’t work in all cases and lead us to further questions, modelling tablet weave fully was left as an undeciphered mystery. Tablet weave is a complex and particularly ancient form of weaving, while it’s simple to do with easily found materials, it produces a kind of double weave with twisting, and you can create crazy higher level 3D structure as it is free from the constraints of fixed loom technology.

pic

The trick to start understanding this (I still have quite some way to go) came from only thinking about a single square tablet. If we follow the paths of each of the four threads while turning the square 90 degrees at a time we can see how tablet weaving is a combination of a weave (up and down movement) and a braid (left and right), as it twists the threads in relation to each other.

scribble

From this sketchy starting point it was possible to create two 3D objects to represent each twist, one for clockwise and another for anticlockwise. If you colour the separate threads appropriately and combine them together you get something like this:

pre-tension

While this looks fancy, it’s wrong. The threads may be in the correct form conceptually, but woven structure comes about as a relationship between the positioning of the threads and the tension applied to them. Many of the threads above should be pulled straight and push others out of the way to give a pattern that was actually straight stripes of colour, rather than chevrons. So how can we add tension?

One way to approach this problem would be to use a physical simulation of the kind usually applied to cloth simulation, and ‘relax’ the the threads to achieve a realistic result, using a stochastic approach to iteratively tighten them within collision constraints, until it ‘looked right’. The problem with this is that it wouldn’t lead to a deeper understanding of what is going on here. This in a way is related to a bigger issue with AI and machine learning, where techniques like artificial neural networks can be trained to solve problems well enough to be useful, for example in speech recognition – but do not provide any new knowledge about language or understanding of deeper scientific issues.

So if we want to understand some of the ‘thread logic’ of table weaving, we are can approach this in a more symbolic manner. Can we add additional straightened threads to our two twisted ones?

As with the twists, there need to be two forms of straightening – left or right twist to straightened threads, and then we need to get back from a straightened thread to a left or right twist.

primitives

Notice that some of these shapes connect, while others are incompatible. We can start with the original twisted weave above, and process it to pull the threads straight. In order to do this we need to know the past and future actions of the weaver, or the current twist in the context of those before and after it. This makes sense, as when weaving structure emerges fully a few wefts behind the current one you are weaving – only as the tension is applied to the fabric does it take form.

The rules to describe this turn out to be well represented as a diagram. The nodes are the 3D shapes required and the edges are the actions of the weaver (the special ‘floating’ state change interestingly depends on the action before the last one – memory does seem important in tablet weaving).

tension

For example, we can ‘left twist’ repeatedly (the top right state) as the arrow points to itself. If we start going in the other direction we then need to pass through two straightening states to get to a full ‘right twist’. If we start going backwards and forwards in smaller numbers of turns then more complex things happen.

When we process the first weave with these rules, you can see some of the straightening effects. The tension on the threads means that some cover up others, e.g none of the yellow threads are now visible on the top of the fabric at all.

post-tension

The structure is more visible here than on a real weaving as the threads are thinner that they would be for the resulting weave which would be more densely packed together (this is less realistic but helps to understand what is going on).

How do we know if any of this is correct? The only way to test this for sure is against real weave. We can try out different sequences of actions and see if the model matches. As indicated above, tablet weaving is a technique that comprises several categories of weaves – these define some specific types of structure we can test.

Type 1: Repeated twists and turn back

Most normal tablet weave consists of twisting repeatedly 90 degrees in the same direction and weaving a weft each time. In practice there is only so far you can go in the same direction before the unwoven warp threads behind the tablets get tangled up, so you need to change direction and go the other way until they are untangled, providing some symmetry to the pattern. The first example has all the tablet threads aligned in the same sequence – and we weave 8 turns one way and 8 turns back again. You can see in the middle when we change direction we create a short straightened ‘float’ section which causes the tension to pull the threads straight here.

aligned-comp

One of the further mysteries that our first tablet weaving simulations couldn’t previously recreate were situations where the pattern on the back and the front of the weave were not opposite of each other. This is highly unusual in weaving, but this model seems to represent this correctly. Here the actions are the same as the first example – 8 one way and then the other, but the thread colours in the tablets are offset from one another so they are staggered and you get the diagonal patterns.

rotate-comp

Type 2: Single faced double weave

Part of the complexity of tablet weaving is because it is a kind of double weave – there are two intertwined weave structures happening at the same time. If we repeat two wefts of 90 degrees one way followed by two more the other direction, the two weaves remain on the same side of the textile – which can be seen clearly if we colour them appropriately. This example keeps the white weave on the top side with the brown one on the lower side.

doublefaced-comp

Type 3: Degenerate floats

The third type of weave is not really a weave but a breakdown of the process caused by only weaving single 90 degree turns backwards and forwards repeatedly. This means half of the threads are not incorporated into the weave and ‘float’ along the surface on both sides.

float-comp

While the language to fully describe the tablet weaving has yet to be developed properly, you can have a go yourself with this model which is currently online here (takes a few moments to render at first).

This gets us closer to a working model of tablet weaving, and provides something we can start to use for a more advanced aims of the Penelope project. For example, can we use the pattern matrix to tangibly livecode tablet weaving? Does this make it possible to explore and explain this type of weaving?

If this kind of textile wasn’t complicated enough, people in ancient times combined multiple weaving techniques, for example tablet weaving and warp weighted weaving in the same piece of fabric. Creating a kind of ‘grand unified’ weaving model is an additional future challenge, so we can start to understand better the thought processes involved in these advanced techniques.

Futurecrafting in Cornwall with FoAM Earth

The future is in short supply at the moment, particularly in a country with so many things changing – it is a constant surprise that there are so few ideas about where we could be headed. At the smaller scales too, at FoAM Kernow, in common with many organisations there is a feeling we tend to be so fixated on the current problems (do we need to invoice for that project, have we replied to this important email, have we reached an agreement on the next steps for this other project), that we miss the long view. Why are we doing this? What are we doing it for? Do we know if this is the right approach? These seem difficult to answer when stuck in short term thinking.

One of the central parts of FoAM’s research for some time has been future preparedness. Through a series of projects Maja and Nik have been building up a collection of methods that allow an individual or organisation to picture itself in a future under a range of conditions called scenarios. Scenarios are designed to be extremes, or caricatures of the possible – if you can describe them well enough, you can envisage how you or your organisation would adapt to them. The goal is to reach a point where change is something that can be reasoned about in advance, rather than reacted to as it happens.

Our initial question was (as FoAM Kernow): “What do we want to be doing? ” broken down a bit more specifically into these points:

  1. What would be choose to do?
  2. What makes a FoAM Kernow project?
  3. What are our selection criteria & success criteria?
  4. How to say no?
  5. How to back out of projects where contributions are less relevant?
  6. What is our role as collaborators?
  7. What is the nature of our collaborations?

After this we went through a series of exercises to build up a comprehensive list of things both internal and external that could cause changes to our organisation. These were quite diverse, but we seemed to focus on external things a bit more than internal ones e.g.: inaccessibility of lab equipment, the rise in nostalgia as a form of solutionism, and that more people are active in politics than there used to be.

The aim is to figure out two significant ‘critical uncertainties’ (interestingly, these are not necessarily the most essential ones, but the most uncertain) which we can use to build our scenarios from. We ended up with these:

The way are living is DOOMED! ⟷ The status quo is FINE!Continuous learning & exploring the new ⟷ doing what you know. same old.

The first uncertainty defines the amount to which our society has the capacity to deal with the many big problems currently facing it. The second one we took to have meaning both internally and externally – society being more or less progressive and open to new ideas, as well as the requirement that we would need to learn new things, or whether our existing skill set is sufficient.

The next thing we did was to draw them as two axes and split them into four scenarios which represent the extremes of these uncertainties. We were then able to describe the world and our place in it by focusing on each one in turn, importantly giving them names which really helped give them character. This part was really fun, as they really are caricatures, and resulted in quite a lot of laughter – even (or especially) when describing what might be quite uncomfortable futures.

Poundland [the way we live is fine, we only do what we know]

This is a world where everything works out fine without any new knowledge needed. It’s hopeful in that we dodge some bullets, but a lot of top down control needs to be exerted for this world to operate, so it’s restrictive in a conservative, traditional way.

“Stasis reigns in Poundland. While class inequality is as bad as it always was, there isn’t much questioning authority. Most people defer decisions to the powers that be. The ruling class is conservative, praising traditional values. Critical education has made way for edutainment and propaganda. Hard Brexit happens seamlessly, the borders are locked down, society surveilled. There are a lot less people in Poundland than there were in the UK. Climate change is not as bad as people thought in the 2010s. A new ice age has arrived and we’re coping. With appropriate urgency, solutions can be found to maintain reasonably comfortable lifestyles.

On the surface, FoAM is doing everything right, yet has not much visible power. We receive government funding to start a new university for applied sciences and technologies, focusing on mitigating effects of the Ice Age. Beneath this benign face, we are guerrilla teachers. Using workshops and other forms of adult education, we prepare our students to think for themselves. With the university as a front, we are hiding in plain sight.”

To a certain extent this seems to be a world that is close to the vision of groups like those behind Vote Leave – a future where the UK is a “Singapore on the shores of Europe”, deregulated to the point where we can kickstart our economy and closeted workforce into higher productivity. It’s a bit like the future described in this book (reading of which was a bit of a follow up for this scenario):

Britannia Unchained: Global Lessons for Growth and Prosperity is a political book written by several British Conservative Party MPs.
Britannia Unchained: Global Lessons for Growth and Prosperity is a political book written by several British Conservative Party MPs.

The problems in society identified in this book seem pretty good – but the unshakable belief in the free market pushes issues like human rights, health, quality of life and the environment into irrelevancies with only passing mentions – the assumption (as in this scenario) that these things will ‘somehow just be fixed’, so prosperity can continue.

Silicon Wharf [the way we live is fine, with a thirst for knowledge]

Silicon Wharf is a progressive, shiny future where everything is solvable by big investment in new knowledge. It’s a world of expensive institutional buildings with odd angled walls, carpet tiles and people wearing lanyards, it’s a very EU future vision – and seems mildly nostalgic and quaint, perhaps because it is a world reliant on abundance of many things.

“The Silicon Wharf is ruled by technocrats (both in government and corporate sector), with rationality and logic as the dominant virtues. It is a node in a globalised network of excellence, where the “best minds” work on technological solutions to climate change. There is a lot of funding for (blue sky) research, as long as the results are presented as innovative products and services that can be scaled up to produce social and/or environmental impact (e.g. carbon sequestration into diamonds for the luxury market). There are mega infrastructure projects under way, championed by China. Brexit has been cancelled and Cornwall has become a key player in the Europe of Regions. The world is increasingly open and there is much cross- border exchange (of knowledge, people, stuff).

The FoAM network has expanded to hundreds of studios and has formed its own NoE (Network of Excellence). The network works on long-term technology research and development and disruptive education. We focus on technologies that are relevant, accessible, tangible and open source. The knowledge we gain through R&D we pass on to anyone who is interested, in workshops and other p2p educational formats. We teach people to make things that can solve their immediate issues, and teach them to teach each other. We get to engage in interesting translation and dissemination activities, which take us out to different contexts, regions and cultures. We’re therefore continuously learning and expanding our horizons.”

Life in this world seems quite easy but somehow dangerous too – just imagine the amount of administration we’d be doing, there are lots of spreadsheets here! The general openness to new learning would make this an exciting place though, with lots of travel possible, due to new innovation excellence in transportation technology.

IRK (Independent Republic of Kernow) [the way we live is doomed, with a thirst for knowledge]

This is in some ways a collapse scenario, nothing has worked out but there is still a thirst for new ways – and because acceptance that the old ways are broken are unavoidable, there are opportunities to attempt much bigger alterations and experiments.

“The Independent Republic of Kernow is a good place to live in highly uncertain times. Urban centres have become danger zones of poverty, disease and violence, so the predominantly rural IRK receives many urban emigrants and refugees. While there is a low life-expectancy in IRK, life is filled with buzzing experimentation and excitement. People dare to take risks, knowing that their survival depends on learning from failure. There are different political and economic experiments (including UBI, a new tithe system, etc.). As the NHS collapsed, new forms of Medical Citizen Science are on the rise. A plethora of infrastructure projects appear and disappear. Hybrid land-sea infrastructure is the most experimented with, as IRK has become a seasonal island. New Dawn traders deploy their sail ships to bring in goods from further afield, but most of the produce is locally sourced (short-chain economies, including initiatives like New Lina, End of The World Garden…). There isn’t much leisure or time to rest, so burnout has become a plague. Some burnt-out people created Circles of Stability, where they tell each other fables of bygone stable lifestyles. While these circles often promote hopeless nostalgia, most of the world has moved to new myths: biomimetic stories of humans becoming social-insects.

FoAM has moved into the Eden Project. We are issuing our own currency (FoAMcoin) and are part of several trade networks (incl. the well established Feral Business Network). The main dome is a productive garden where we grow food and medicine for our collaborators and wider network (in collaboration with other horticulturalists and farmers in the area). What we can’t use ourselves, we barter for other goods. One of the domes is converted into the warehouse for New Dawn traders, another is devoted to agritech experiments and another is focusing on our communication infrastructure projects, such as waterproof mesh networks. We’re trading produce, but primarily we exchange skills. We aim to cultivate a skill-ecosystem that includes a wide range of people with diverse knowledge, talents and skills. We travel more, using slow modes of transport. Some of us stay in Eden, while others opt for journeyer lifestyles with no fixed studio and no desk (work). FoAM thrives in this trans-local republic, and even though we know we might die soon, we’ll die happy and fulfilled.”

So, appropriately IRK is the opposite of Poundland, life has the potential to be good but generally short – whereas in Poundland stability was rigidly enforced and life seemed hard. When we were discussing this scenario there was a lot of laughter and general surprise that this would be the best one in a lot of ways, even though everything is doomed.

Swamp.ac.uk [the way we live is doomed, we only do what we know]

The last scenario is, by all rights the worst one possible – there is no desire or drive for learning new things, so everything is irretrievably broken. However, one of the significant things that happens when futurecrafting is finding that what you thought were the worst case scenarios have features of them that are still interesting and desirable. This is the scenario of escapism, VR and games – it also has the best parties. As scenarios are extremes, they are not going to happen – but they do give indications of situations that we might recognise in future so we can connect with these desirable outcomes, even if they are a result of an overall situation we might not wish for.

“The way humans live is doomed, and we have accepted it. It’s too late to change anything anyway. We’re living in a Trump-inspired world, in what is left of the UK. There is no funding for research, no electric car subsidies, but regional funding and (corrupt) local philanthropy is growing. It’s all about who you know. Universities are small, focusing on keeping the local population occupied (some dedicated to spreading ‘the word’ of the regional philanthropic magnates). Brexit is still in (perpetual) negotiation, leading to additional economic, social and political paralysis and fragmentation. A sense of powerlessness and inequality permeates everything, leading to a simmering, low level civil war. In their despair people turn to escapism and nostalgia. VR games and end of the world parties are plentiful, with people crowding together ‘to go out with a bang’.

FoAM has realised that the only way to survive in this world is to build a stable institution ourselves, to become one of the power players (in loose alignment with local rulers). We abandoned our individual passions; instead, all our energy goes into collective endeavours. Our institute is known for developing high quality and transparent technologies. Our biggest department focuses on techniques and technologies for maintenance and repair of crumbling infrastructure for everyday life. Our other line of work is less known in the mainstream, but hugely popular with the escapist underground; we organise exorbitant crypto-parties and citizen science game-fests (VirusCraft became hugely popular, funding most of our non-profit programmes). Accesslab is our public outreach programme, helping people make sense of the nonsense they’re faced with every day. Seemingly dealing with short term issues, we’re actually teaching (subversive) long-term thinking.”

It seems that the only hope in this world would be to somehow nurture a shift to a more IRK like situation, attempting to show in small ways some way out of the powerlessness. The significance of AccessLab in this world is interesting, as it would be increasingly important here to connect a science starved of funding with the reality of the situation to keep it relevant, as well as making it useful for providing potential new avenues.

Of course we live in neither the worst or best of all possible worlds, but thinking of the extremes helps us to be prepared for moves in the middle ground; combinations of these futures are more likely. We can now use these scenarios to plan projects, for each potential new thread of investigation we can consider how many of these scenarios would it work under, and the more the better.

We’ve had quite a lot of success over the past couple of years with really popular projects we’ve nurtured into existence ourselves, which seem to strike a chord and get a lot of attention – Sonic Kayaks, AccessLab and midimutant for example. These scenarios are a new tool for us to continuously gauge the relevance of what we are doing, and provide hints for how to make an idea useful regardless of how the future pans out.

How to design a tangible programming language – Pattern Matrix at Algomech (part 2)

Once we acknowledge that weaving and programming are part of the same technological timeline, we can begin to look at the history of weaving as a eight thousand year long tale of human relationship with digital technologies – and use this long view to research new approaches to software engineering, a field with a much less developed history and many interesting problems to solve.

Using augmented reality to display dynamic information on a tangible programming language.
Using augmented reality to display dynamic information on a tangible programming language.

(Follows Part 1 here.)

One of our threads of investigation is using the pattern matrix as a general purpose tangible programming system – one that we can use for controlling swarms of robots, programming different types of weaving systems and describing complex processes, such as live musical systems.

The magnetic system on the new pattern matrix consists of four hall effect sensors on every location you can place a block. There are four unique ways you can arrange the magnets – which means four types of programming block are possible. As we want to reuse these physical blocks for various uses and programming languages, we decided to use abstract shapes to denote the block types to begin with. Each of the four blocks can be rotated and flipped to give 32 total possible orientations, or symbol ‘tokens’:

All combinations of token orientation with four magnets
All combinations of token orientation with four magnets

However, only 16 of these orientations are actually unique. We can only determine flip orientation on the circular block, and only rotation on the rectangular and triangular ones – where flipping them makes no difference to the magnets. The square block is a kind of special one, as we can tell both rotation and flip orientation, so it can represent eight tokens in total all by itself:

All unique tokens and orientations possible. With mid-grey shapes the flip (which side is up) is irrelevant.
All unique tokens and orientations possible. With mid-grey shapes the flip (which side is up) is irrelevant.

It’s important to note at this point that the parallels with tablet weaving are no coincidence: rotating and flipping arrangements of four binary elements for this magnetic system are the same actions as those performed when weaving using tablets. Weaving in the pattern matrix is more than a subject, it’s built directly into it’s mode of operation.

Next we need to test the applicability of this tangible programming system for wider uses. The other cultural phenomena the Penelope project is involved in is livecoding – so is it possible to use the pattern matrix to introduce a weaving centred programming technology in a very different context, not to describe weaving but to generate music in a performance such as an Algorave? This is something that Ellen first pioneered at our weavecoding performance at The Museum of casts of classical sculptures in Munich, but the new pattern matrix has better capabilities for a general purpose programming language.

Having 16 states of four blocks is indeed limiting for a language, but not too limiting. Some types of programming language, such as a string rewriting system like a Lindenmayer system are particularly well suited to this. They are also surprisingly Turing complete languages, able to represent any other programming language in existence, given enough space and time.

Here is a quick example of how this works in text form – a string rewriting system is simply a list of search-replace actions that are carried out in a consistent order. The original example, used to model the growth pattern of algae – consists of a starting string: “A” and two replacement rules, replace “A” with “AB” and replace “B” with “A”. If we run these two rules over and over on the same bit of text we gradually ‘grow’ a pattern like this:


n = 0 : A
n = 1 : AB
n = 2 : ABA
n = 3 : ABAAB
n = 4 : ABAABABA
n = 5 : ABAABABAABAAB
n = 6 : ABAABABAABAABABAABABA
n = 7 : ABAABABAABAABABAABABAABAABABAABAAB

On the pattern matrix we use four of the rows to represent four different rules that replicate in this manner (each made of 5 possible symbols, as it’s a 5×5 grid), which we run 8 times on the starting string (A) to create a musical sequence. Four of the tokens represent these rules (A,B,C and D), the remaining tokens represent musical actions – changes in pitch, rests and sound triggers. There is massive variety of potential patterns, you can control the amount of recursion by the number of rule reference tokens you use – to control the resulting length of the sequences, and thus the complexity of the music. Interestingly we also need a ‘no operation’ (NOP) instruction that does nothing – as in low level assembler languages. We need this as a way to be able to shift timing in the musical sequence by one instruction.

A musical language in 16 instructions.
A musical language in 16 instructions.

With a tangible programming language like this it’s also very important to consider how you categorise instructions by shape – as you can quickly switch between similar operations by simply rotating or flipping tokens, while switching between different shapes takes longer (as you need to pick up a new block) so should represent bigger changes if possible.

Four rules are plenty for generating hugely complex sequences, so we can use the fifth bottom row to control global parameters like scale, synchronisation options (for our slub collaborative sync protocol) or switch between more banks of sounds for greater variety.

Slub performance including the pattern matrix at the Brighton British Science Festival Algorave
Slub performance including the pattern matrix at the Brighton British Science Festival Algorave

The first time we tried this out was at the British Science Festival Algorave in Brighton. A projection was set up with a camera to show the pattern matrix being used, and while technically everything went fine (other than some syncing difficulties), it highlighted a key problem with tangible programming languages. With no dynamic feedback other than the state of the blocks on the pattern matrix, it’s very difficult to tell what is happening during a performance, it’s hard to understand what musically is resulting from the changes you are making.

In order to find a way around this we designed an augmented reality ‘layer’ to place over the pattern matrix, which gives feedback on the currently triggered notes and the paths between the recursive string production rules. We use fluxus and it’s AR feature, which was written by Gabor Papp – which is based on the ARToolkit library. We use a printed out marker to find the plane and camera scale of the centre of the pattern matrix in the image from a USB camera. Once this is done the marker can be removed (as neither the camera or pattern matrix moves) and we can use millimetres as units and place objects over the block locations in 3D space. When the sensors detect a change we can display this new information, also updating the current position in the sequence playback to give feedback on the current sound playing.

Pattern matrix livecoding as part of slub performance at the Algomech Algorave
Pattern matrix livecoding as part of slub performance at the Algomech Algorave. Pix thanks to Dan Hett

As an initial trial the AR improved things when first tried out at the Algomech Algorave in Sheffield, it makes the pattern matrix easier to understand and perform with – and also provides some feedback for the audience in a projection. In a last minute change we switched from Latin characters to Linear A, an undeciphered ancient Greek script – a reference to Flavia’s work on the Penelope project. This is actually preferable to Latin characters as the musical language represents meaning in a way that that actual glyph used is irrelevant – it’s better if it can’t be ‘read’ or confused with another meaning by anyone (still alive).

So it seems that AR could be one of the items in our toolbox for further tangible programming experiments. Perhaps we can better explain the structural changes caused by livecoding the weaving notation for the warp weighted loom by having a dynamic weave structure ‘floating’ on top of the tokens, alongside the loom simulation. This could also be of use for describing tablet weaving actions with these blocks, which would need to be more abstract than the binary weaving notation.

Another area to explore is the design of the blocks themselves, moving away from the abstract shapes, we can design them for specific purposes. Similar to our work on viruscraft, where we have more closely explored the correspondence between physical form (receptors and structural protein arrangements) and tangible interfaces, it seems that these shapes may be worth considering more closely now the sensor matrix is working well.