Penelopean robotics (part 2)

Penelopean robotics are about rebuilding technology in the woven cosmos. You can read more about the theory in part 1, but roughly our aims are to:

  1. Embody Penelopean technological practice – they should be easily undone (taken apart) so they can be understandable, self documenting and repairable.
  2. They are not automated looms, but must eventually be capable of weaving in some form, maybe by interacting with ancient Greek weaving technology.
  3. Capable of embodying elements of ancient Greek poetry and dance, perhaps via livecoding.
  4. Be constructed in the spirit of Simone Giertz rather than Boston dynamics

Some potential further directions we have discussed are swarm robotics and the use of May poles as a stepping stone between dancing, braiding and weaving.

This is a story of one Penelopean robot.

The original concept of the weaving robot as described in the project was an ‘arachnoid mechanism’. We wanted to stick with this general direction and begin by building a six legged insectoid. After a bit of acting out of the movements required, and looking on youtube for similar robots it seemed that it was possible to do this with 3 servos, two for the front and back which rotate horizontally and one in the middle which rotates vertically.

Weaving crops up in plenty of medical, space and industrial situations – the strength and lightness of woven structures has lots of advantages over other techniques used in engineering, and in the spirit of the project it seems appropriate to try tablet weaving as a way to construct the main framework for the robot. For more info on how to do tablet weaving yourself, see this previous post and this one.

Double weave can be used for this as it allows you to weave tubes and pockets without any cutting or stitching, so they have very strong joins. Here is a sketch of the initial design:

Three joined tubes can hold the micro servos in place, and I later added another smaller one in between for a cylindrical lithium ion battery from Adafruit that was the perfect shape for this.

The process of weaving was a case of starting with a normal section, followed by beginning a double weave split via two weft threads, and checking it every couple of wefts to fit the first servo. When it’s big enough to tightly fit you close the first tube and immediately start a second, for the battery – then the middle servo (which fits at right angles to the others so is slightly smaller) and finally the last servo, with a normal section of tablet weave to close it up. This didn’t take very long, but to make more I could easily count wefts based on the first one and do a lot more quickly – for example, manufacturing swarms of these robots.

As a comparison, this process would be much more difficult and slow via 3D printing – as you can’t generally change anything on the fly, and the design needs to be completed before printing, and the softness turns out to be important.

Although the weave is strong in itself, it still needs a skeleton for the robot to stand up. I used 2mm wire for both this job as well as the legs. One piece of wire is used for the top (which loops around
the battery) and another bent double for the lower side. This scaffolding means that the middle legs will be able to push against something to raise the other legs to walk. Then all you need to do is tie loops of thread around each servo to hold it in place along with the scaffold. This may be unnecessary with a bit more development.

The PCB for the microcontroller and radio (more on that in a later post) can also be tucked under the top wire so it’s locked out of the way of the legs on top.


Christian Faubel showing us the unique nature of analogue computation and what happens when you apply this to robotics at the Penelope robotics workshop in Munich.

At the time of of the Penelope robotics workshop I could get the robot to shuffle a bit in generally the right direction but it wasn’t very efficient or convincing. It took a little servo sequencer programmed by the pattern matrix (in a similar manner to how it sequences music) to experiment with it and work it out properly. This takes a short list of symbols representing angles (from -90 to +90 degrees) which loop.

This is the working code for walking on the pattern matrix – more descriptive tokens will be made in future! These patterns are sent over radio to the robot whenever they change.

The general idea is that if you have 6 legs you can keep 3 feet on the ground while moving the others, which is much more stable than if you need to balance. The front and back servos job are to drive the robot forward and have the same movement as each other at the same time. The role of the middle legs are to lift the side of the robot up in alternate phase so only the ones pushing ‘back’ are touching the ground.

This meant we could use the pattern matrix to do things like switch direction by inverting the middle row above, which controls the central servo. This lifts the robot’s sides in the inverse phase, which means its feet (following the same pattern) move it in the other direction.

You can see a selection of different patterns of movement on this video:

The best aspect of the woven structure was how strong it was – it could throw itself around all day without problems and the flexibility helps too with the movements required. I could also stuff the robot into the bottom of a bag and it would be fine the next day.

There will be more info here on how to build them – for now the code is all here, the electronics is based on this prototype board – watch this space for more of the details on the radio enabled tangible robot livecoding.

Penelopean robotics theory and the woven cosmos (part 1)

The Penelope project is concerned with many things, pattern structures in ancient poetry, comparisons of Andean and Greek mathematics, and the role of liveness in thought processes – to name just a few. We can also add robotics to this list.

A weaver in action is often mistaken for a robot – perhaps it’s the repetitive actions that mask the accompanying thought processes from our eyes. Similarly, a livecoding performer typing code into a laptop can face a problematic interpretation that they are grandstanding their expertise, rather than simply attempting to contribute to a party atmosphere at an Algorave. These activities often seem to be misunderstood in a similar manner but in different directions, perhaps partly due to the fact they both employ technology that is either greatly over or under-played by the ubiquity of their end results.

The Penelope project is concerned with using these technological misunderstandings in order to search for deeper assumptions we may have. One case we’ve discussed for a while has been to combine the situations above, if a weaver livecodes a robot which is operating an ancient loom – what is the response to this?

One way to design these situations well is to use an ‘artistic conceit’, a implicit fictitious assumption that provides a framework to fit things together so they make sense. For example, working with the Quipu database, we used the central idea that the Inca were attempting to communicate with us to naively employ cryptanalysis, visualisation and sonification in an attempt to understand their untranslatable knotted thread codes.

An important reference for the Penelope project for me is Carl Sagan. In his influential series ‘Cosmos’, weaving is mentioned many times as a central ordering principle of time and space for the Ionian thinkers of ancient Greece, who initially wrote down many of the fundamental scientific ideas he is introducing. He embarks on a thought experiment considering what life would be like today without the destruction of the library of Alexandria – which contained much of their discoveries and ancient knowledge from all over the world, now lost forever. His conjecture is that this single event may have set us back as much 500 years in scientific thought, so it’s just possible that without this event occurring, on a different timeline to ours – around about now, the first space ships would be returning from the surveying planets orbiting other stars.

It it this kind of imagination and suspension of disbelief that can give us a new perspective on our current predicaments – and perhaps a more humble position with which to question that our ways of thinking are inevitable, obvious and about as good as they can be.

For example, if the invention of general purpose computation and robotics had occurred in a society much earlier – one founded on the woven cosmos, what could it have looked like, and how would it have worked? What might it tell us about different ways of doing things?

Further attempts at untangling tablet weave

One of the great unknowns following the first weavecoding project was the nature of tablet weave. Other than a few primitive attempts that didn’t work in all cases and lead us to further questions, modelling tablet weave fully was left as an undeciphered mystery. Tablet weave is a complex and particularly ancient form of weaving, while it’s simple to do with easily found materials, it produces a kind of double weave with twisting, and you can create crazy higher level 3D structure as it is free from the constraints of fixed loom technology.

pic

The trick to start understanding this (I still have quite some way to go) came from only thinking about a single square tablet. If we follow the paths of each of the four threads while turning the square 90 degrees at a time we can see how tablet weaving is a combination of a weave (up and down movement) and a braid (left and right), as it twists the threads in relation to each other.

scribble

From this sketchy starting point it was possible to create two 3D objects to represent each twist, one for clockwise and another for anticlockwise. If you colour the separate threads appropriately and combine them together you get something like this:

pre-tension

While this looks fancy, it’s wrong. The threads may be in the correct form conceptually, but woven structure comes about as a relationship between the positioning of the threads and the tension applied to them. Many of the threads above should be pulled straight and push others out of the way to give a pattern that was actually straight stripes of colour, rather than chevrons. So how can we add tension?

One way to approach this problem would be to use a physical simulation of the kind usually applied to cloth simulation, and ‘relax’ the the threads to achieve a realistic result, using a stochastic approach to iteratively tighten them within collision constraints, until it ‘looked right’. The problem with this is that it wouldn’t lead to a deeper understanding of what is going on here. This in a way is related to a bigger issue with AI and machine learning, where techniques like artificial neural networks can be trained to solve problems well enough to be useful, for example in speech recognition – but do not provide any new knowledge about language or understanding of deeper scientific issues.

So if we want to understand some of the ‘thread logic’ of table weaving, we are can approach this in a more symbolic manner. Can we add additional straightened threads to our two twisted ones?

As with the twists, there need to be two forms of straightening – left or right twist to straightened threads, and then we need to get back from a straightened thread to a left or right twist.

primitives

Notice that some of these shapes connect, while others are incompatible. We can start with the original twisted weave above, and process it to pull the threads straight. In order to do this we need to know the past and future actions of the weaver, or the current twist in the context of those before and after it. This makes sense, as when weaving structure emerges fully a few wefts behind the current one you are weaving – only as the tension is applied to the fabric does it take form.

The rules to describe this turn out to be well represented as a diagram. The nodes are the 3D shapes required and the edges are the actions of the weaver (the special ‘floating’ state change interestingly depends on the action before the last one – memory does seem important in tablet weaving).

tension

For example, we can ‘left twist’ repeatedly (the top right state) as the arrow points to itself. If we start going in the other direction we then need to pass through two straightening states to get to a full ‘right twist’. If we start going backwards and forwards in smaller numbers of turns then more complex things happen.

When we process the first weave with these rules, you can see some of the straightening effects. The tension on the threads means that some cover up others, e.g none of the yellow threads are now visible on the top of the fabric at all.

post-tension

The structure is more visible here than on a real weaving as the threads are thinner that they would be for the resulting weave which would be more densely packed together (this is less realistic but helps to understand what is going on).

How do we know if any of this is correct? The only way to test this for sure is against real weave. We can try out different sequences of actions and see if the model matches. As indicated above, tablet weaving is a technique that comprises several categories of weaves – these define some specific types of structure we can test.

Type 1: Repeated twists and turn back

Most normal tablet weave consists of twisting repeatedly 90 degrees in the same direction and weaving a weft each time. In practice there is only so far you can go in the same direction before the unwoven warp threads behind the tablets get tangled up, so you need to change direction and go the other way until they are untangled, providing some symmetry to the pattern. The first example has all the tablet threads aligned in the same sequence – and we weave 8 turns one way and 8 turns back again. You can see in the middle when we change direction we create a short straightened ‘float’ section which causes the tension to pull the threads straight here.

aligned-comp

One of the further mysteries that our first tablet weaving simulations couldn’t previously recreate were situations where the pattern on the back and the front of the weave were not opposite of each other. This is highly unusual in weaving, but this model seems to represent this correctly. Here the actions are the same as the first example – 8 one way and then the other, but the thread colours in the tablets are offset from one another so they are staggered and you get the diagonal patterns.

rotate-comp

Type 2: Single faced double weave

Part of the complexity of tablet weaving is because it is a kind of double weave – there are two intertwined weave structures happening at the same time. If we repeat two wefts of 90 degrees one way followed by two more the other direction, the two weaves remain on the same side of the textile – which can be seen clearly if we colour them appropriately. This example keeps the white weave on the top side with the brown one on the lower side.

doublefaced-comp

Type 3: Degenerate floats

The third type of weave is not really a weave but a breakdown of the process caused by only weaving single 90 degree turns backwards and forwards repeatedly. This means half of the threads are not incorporated into the weave and ‘float’ along the surface on both sides.

float-comp

While the language to fully describe the tablet weaving has yet to be developed properly, you can have a go yourself with this model which is currently online here (takes a few moments to render at first).

This gets us closer to a working model of tablet weaving, and provides something we can start to use for a more advanced aims of the Penelope project. For example, can we use the pattern matrix to tangibly livecode tablet weaving? Does this make it possible to explore and explain this type of weaving?

If this kind of textile wasn’t complicated enough, people in ancient times combined multiple weaving techniques, for example tablet weaving and warp weighted weaving in the same piece of fabric. Creating a kind of ‘grand unified’ weaving model is an additional future challenge, so we can start to understand better the thought processes involved in these advanced techniques.

How to design a tangible programming language – Pattern Matrix at Algomech (part 2)

Once we acknowledge that weaving and programming are part of the same technological timeline, we can begin to look at the history of weaving as a eight thousand year long tale of human relationship with digital technologies – and use this long view to research new approaches to software engineering, a field with a much less developed history and many interesting problems to solve.

Using augmented reality to display dynamic information on a tangible programming language.
Using augmented reality to display dynamic information on a tangible programming language.

(Follows Part 1 here.)

One of our threads of investigation is using the pattern matrix as a general purpose tangible programming system – one that we can use for controlling swarms of robots, programming different types of weaving systems and describing complex processes, such as live musical systems.

The magnetic system on the new pattern matrix consists of four hall effect sensors on every location you can place a block. There are four unique ways you can arrange the magnets – which means four types of programming block are possible. As we want to reuse these physical blocks for various uses and programming languages, we decided to use abstract shapes to denote the block types to begin with. Each of the four blocks can be rotated and flipped to give 32 total possible orientations, or symbol ‘tokens’:

All combinations of token orientation with four magnets
All combinations of token orientation with four magnets

However, only 16 of these orientations are actually unique. We can only determine flip orientation on the circular block, and only rotation on the rectangular and triangular ones – where flipping them makes no difference to the magnets. The square block is a kind of special one, as we can tell both rotation and flip orientation, so it can represent eight tokens in total all by itself:

All unique tokens and orientations possible. With mid-grey shapes the flip (which side is up) is irrelevant.
All unique tokens and orientations possible. With mid-grey shapes the flip (which side is up) is irrelevant.

It’s important to note at this point that the parallels with tablet weaving are no coincidence: rotating and flipping arrangements of four binary elements for this magnetic system are the same actions as those performed when weaving using tablets. Weaving in the pattern matrix is more than a subject, it’s built directly into it’s mode of operation.

Next we need to test the applicability of this tangible programming system for wider uses. The other cultural phenomena the Penelope project is involved in is livecoding – so is it possible to use the pattern matrix to introduce a weaving centred programming technology in a very different context, not to describe weaving but to generate music in a performance such as an Algorave? This is something that Ellen first pioneered at our weavecoding performance at The Museum of casts of classical sculptures in Munich, but the new pattern matrix has better capabilities for a general purpose programming language.

Having 16 states of four blocks is indeed limiting for a language, but not too limiting. Some types of programming language, such as a string rewriting system like a Lindenmayer system are particularly well suited to this. They are also surprisingly Turing complete languages, able to represent any other programming language in existence, given enough space and time.

Here is a quick example of how this works in text form – a string rewriting system is simply a list of search-replace actions that are carried out in a consistent order. The original example, used to model the growth pattern of algae – consists of a starting string: “A” and two replacement rules, replace “A” with “AB” and replace “B” with “A”. If we run these two rules over and over on the same bit of text we gradually ‘grow’ a pattern like this:


n = 0 : A
n = 1 : AB
n = 2 : ABA
n = 3 : ABAAB
n = 4 : ABAABABA
n = 5 : ABAABABAABAAB
n = 6 : ABAABABAABAABABAABABA
n = 7 : ABAABABAABAABABAABABAABAABABAABAAB

On the pattern matrix we use four of the rows to represent four different rules that replicate in this manner (each made of 5 possible symbols, as it’s a 5×5 grid), which we run 8 times on the starting string (A) to create a musical sequence. Four of the tokens represent these rules (A,B,C and D), the remaining tokens represent musical actions – changes in pitch, rests and sound triggers. There is massive variety of potential patterns, you can control the amount of recursion by the number of rule reference tokens you use – to control the resulting length of the sequences, and thus the complexity of the music. Interestingly we also need a ‘no operation’ (NOP) instruction that does nothing – as in low level assembler languages. We need this as a way to be able to shift timing in the musical sequence by one instruction.

A musical language in 16 instructions.
A musical language in 16 instructions.

With a tangible programming language like this it’s also very important to consider how you categorise instructions by shape – as you can quickly switch between similar operations by simply rotating or flipping tokens, while switching between different shapes takes longer (as you need to pick up a new block) so should represent bigger changes if possible.

Four rules are plenty for generating hugely complex sequences, so we can use the fifth bottom row to control global parameters like scale, synchronisation options (for our slub collaborative sync protocol) or switch between more banks of sounds for greater variety.

Slub performance including the pattern matrix at the Brighton British Science Festival Algorave
Slub performance including the pattern matrix at the Brighton British Science Festival Algorave

The first time we tried this out was at the British Science Festival Algorave in Brighton. A projection was set up with a camera to show the pattern matrix being used, and while technically everything went fine (other than some syncing difficulties), it highlighted a key problem with tangible programming languages. With no dynamic feedback other than the state of the blocks on the pattern matrix, it’s very difficult to tell what is happening during a performance, it’s hard to understand what musically is resulting from the changes you are making.

In order to find a way around this we designed an augmented reality ‘layer’ to place over the pattern matrix, which gives feedback on the currently triggered notes and the paths between the recursive string production rules. We use fluxus and it’s AR feature, which was written by Gabor Papp – which is based on the ARToolkit library. We use a printed out marker to find the plane and camera scale of the centre of the pattern matrix in the image from a USB camera. Once this is done the marker can be removed (as neither the camera or pattern matrix moves) and we can use millimetres as units and place objects over the block locations in 3D space. When the sensors detect a change we can display this new information, also updating the current position in the sequence playback to give feedback on the current sound playing.

Pattern matrix livecoding as part of slub performance at the Algomech Algorave
Pattern matrix livecoding as part of slub performance at the Algomech Algorave. Pix thanks to Dan Hett

As an initial trial the AR improved things when first tried out at the Algomech Algorave in Sheffield, it makes the pattern matrix easier to understand and perform with – and also provides some feedback for the audience in a projection. In a last minute change we switched from Latin characters to Linear A, an undeciphered ancient Greek script – a reference to Flavia’s work on the Penelope project. This is actually preferable to Latin characters as the musical language represents meaning in a way that that actual glyph used is irrelevant – it’s better if it can’t be ‘read’ or confused with another meaning by anyone (still alive).

So it seems that AR could be one of the items in our toolbox for further tangible programming experiments. Perhaps we can better explain the structural changes caused by livecoding the weaving notation for the warp weighted loom by having a dynamic weave structure ‘floating’ on top of the tokens, alongside the loom simulation. This could also be of use for describing tablet weaving actions with these blocks, which would need to be more abstract than the binary weaving notation.

Another area to explore is the design of the blocks themselves, moving away from the abstract shapes, we can design them for specific purposes. Similar to our work on viruscraft, where we have more closely explored the correspondence between physical form (receptors and structural protein arrangements) and tangible interfaces, it seems that these shapes may be worth considering more closely now the sensor matrix is working well.

Pattern Matrix at Algomech (part 1)

I’m writing this on the train with a slightly sleep deprived brain fizzing and popping from thoughts, ideas and conversations from this year’s Algomech festival in Sheffield. The Penelope project took a significant role in the festival, with the group’s participation in the Unmaking Symposium, the exhibition and also testing our latest weavecoding technology at the Algorave. I’ll be writing more on the algorave in a subsequent post.

During the symposium we discussed the critical, liberating and potentially dangerous aspects of Unmaking in a wide variety of contexts – from reverse engineering knitwear and classical Greek dance to discovering the untapped abilities of classical musical instruments when human limitations become a secondary consideration. The symposium also provided us with a good opportunity to take stock of our own group’s current directions and thinking in regard to the Penelope project.

Pix thanks to
Pix thanks to James Vanderhoven

comp

We also had our own corner of the Algomech exhibition, which included Jacquard woven experiments, the Quipu sonification and visualisation and the first public trials of the new pattern matrix V2.

exhib

As is our usual practice, we used this exhibition to get essential feedback on our new design for the pattern matrix, as well as the interpretation of what we are doing – being there in person talking to people allows you to very quickly determine what works, and adapt the focus based on the responses you get.

We discussed the long view on digital technology, the role of weavers in foundations of western mathematics and simply the practicalities of the technology we are developing – the constant stream of visitors represented a wide range of different ages and backgrounds. These aspects of the pattern matrix seemed significant (in no particular order):

  • The construction technique was immediately interesting to most people, specifically the material, beech wood spalting and open frame construction.
  • Related to this, there seems little association with the pattern matrix as a ‘device’ in the sense of a ‘gizmo’ – which is interesting as the Raspberry Pi and other PCBs are clearly visible. We’ve noticed this at some level with the previous version but with the wood construction this effect is much more pronounced.
  • It is seen as being game like, e.g. a “70’s educational toy”. There is an expectation that it is something to be ‘played’, and similarly its potential as a musical sequencer is a common observation.
  • The understandability of the magnet sensing seems a key ingredient. There is no other particularly hidden magic like computer vision or RFID involved, and polarity and digital arrangements of magnets are easily explained and experienced by holding the tokens together.
  • Having some extra circuit boards and wood cut parts to hand, originally intended as backups – were great for people who wanted to know more about what was going on. In future we should also have the token block parts to show as well.
  • The different shaped blocks were immediately appealing – they seemed to invite experimentation more than alternating the binary tokens by flipping them. To follow this up we need to investigate ways to use different shapes to configure thread colour at the same time as structure in a better way than we are now. The black/white sides could define structure while the shape could correspond to the colour for the specific thread. It also indicates that using shaped tokens as instructions for tablet weaving is worth experimenting with quite soon.
  • Younger children focused on the blocks alone (and of course tried all sorts of things no one else did, like stacking them) but slightly older children worked out they were having an effect on the weaving process and generally could patiently work themselves it out without any explanation required.
  • Having Anni Albers’ ‘On Weaving’ book next to pattern matrix helped with older visitors, perhaps representing a more conventional and authoritative source of information to introduce the concepts of notation, structure and pattern in weaving.
Pix thanks to James Vanderhoven
Pix thanks to James Vanderhoven
The 8-way tangible colour switching instruction
An 8-way tangible colour switching instruction

Physical vs digital – a false dichotomy

More general concepts that came up in conversation included a common theme during Algomech, exploring the inescapably fuzzy boundaries of concepts such as digital, physical and analogue. The myth of the “real world” being analogue and the “virtual world” being digital is a troublesome one to a weaver.

The ‘anti-device’ effect of the pattern matrix has the potential to explore this conundrum, as it represents a seemingly acceptable demonstration of the physical nature of the digital, and that forms of digital technology have inhabited the world of the reassuringly physical for many millennia of human invention.

Check your supply chains

One aspect of the pattern matrix I picked on for the symposium which came up in the exhibition as well was the fact that the beech wood came from a single tree in Cornwall courtesy of Aaron Moore – and used this as an example of our design philosophy of taking on the myth of collapse rather than the myth of infinite abundance.

img_20171107_145340

Feedback from Penelope team members

Having a 5X5 grid initially was thought to be excessive, as most ancient weaves can be expressed with a 4X4 matrix. This larger capability turns out to be more important than we first thought, as it means you can demonstrate the importance of odd and even numbers in the mathematics of weaving.

There is a problem with the single colour change block due to a faulty use of the code from the old version. This has always been a somewhat temporary feature so we should sort this out properly (e.g. using other shapes for colour across the matrix) before we use it next.

We can also try using an augmented reality approach to show the weave structure directly over the grid, so it’s easier to see how the token block changes relate to specific crossings. This could be displayed alongside the current warp weighted loom rather than as a replacement for it.

PCB design for pattern matrix 2

This is the pattern matrix 2 tangible sensor schematic, which is fairly simple – just 4 hall effect sensors and a capacitor to smooth out any noise on the power supply.

sensor-schematic

We need to make hundreds of these for the Penelope Project, and we can save some costs by using the built in pull up resistors in the MCP32017 to get a decent signal from the sensors. The difficulty with this PCB is arranging the sensors so they align with the magnets in the tangible programming block in the optimum manner. From tests with the prototype Lego rig, this isn’t actually too critical – but it’s set up so the lead length can be tweaked a bit at soldering time.

sensor-render

This took me about 20 variations to finally get right, but the circuit is just about simple enough that it can be made single sided – this is good because the top side will be partly exposed, while the lower side with all the copper traces can be protected. It’s good practice to have large areas of copper left connected to ground, partly as it’s a common connection needed all over the board, partly for stability but also it reduces the amount of chemicals required to etch the circuit – as only the parts around the traces need to be removed.

sensor-render2

The i2c expander board is a little more complicated. The design is made to be modular so we can stack up any number of these connected to the Raspberry Pi for different arrangements of sensors. Each board can deal with 8 sensor locations (each comprising 4 individual hall effect sensors). Their job is to convert the digital signals from each sensor into serial data (using the i2c protocol) so the Raspberry Pi can read them all just using 2 wires, plus 2 for power.

i2c-schematic

Each board can be configured to a separate i2c device address to tell it apart from the others using jumper connectors. This one had to be 2 sided, but I managed it without any ‘vias’ (holes to pass traces from one side of the board to another). I also added a power indicator LED as a last minute addition.

i2c

I’ve been learning the open source Kicad software to design these, which is now used by CERN for building the LHC, so it’s pretty fully featured! The idea is that you draw the schematic first, link each component with a physical ‘footprint’, then switch to the PCB design stage. Other software I’ve used in the past tries to route everything in one go for you (and can come up with some pretty strange and messy results). Kicad works in a semi-automatic manner – you need to draw each trace by hand, but it routes them around components and other traces, and suggests the shortest path for you. This is quite a lot better than a fully automatic approach as you have more control over the end result, and easily end up with a decent placement of all the parts.

i2c-render

This project is of course open hardware, and can be found on github here.

Pattern Matrix 2 haptic experiments

One of the potential future additions to our tangible programming hardware is haptic feedback, using sound/vibration to provide a extra channel of information through your fingers when programming with tangible blocks. We wanted to test this before designing the PCB hardware in case we could add it to the system simply – this was initiated by watching a reverse engineering video on youtube (I think by bigclivedotcom) where a technique was mentioned for sneakily reusing input lines (of which we have hundreds) as outputs when they are not being read, by using diodes to separate the signals.

img_20170602_164024

Piezo transducers are often used as cheap speakers, and they can also provide some touch feedback by using lower frequencies. We tested one to see if it interfered with the magnetic fields, and the diaphragm doesn’t seem to be ferrous metal (I’m not actually sure what it is made from) so it can be placed right over the sensors with no effect. A bit more Lego to attach it to the prototype sensor unit, and to see we can feel it through the tangible block when it’s touching the sensor.

The first attempt consisted of simply plugging a speaker into the i/o line from the MCP23017 port expander, switching between input and output on the Pi and adding a diode to prevent the output voltage getting to the sensor.

circuit1

The problem is all the circuits we’re using run on 3.3V, but the piezo speaker I’m using is rated up to 20V – so it’s just not strong enough to feel it. The Pi also provides 5V pins, so a second attempt to use a simple driver circuit (single transistor common emitter amplifier) to boost the voltage:

circuit2

You can hear this version in the video above – it’s playing a sound using a frequency crudely determined by the 4 bit value the block orientation represents whenever it changes. This is louder (loud enough at least to make a recording) but still not enough to feel easily, even when you put the piezo between the sensor and the block. So it seems the best way to get this to work properly will be via a separate circuit, not something we can slot into the existing input hardware. Another advantage of doing it separately is that we can treat it more like a multichannel audio setup with a dedicated processor that is not interrupted by the sensor reading. One solution I saw was to use the same H bridges as used to drive motors from microcontrollers as these take much higher voltages – this will be something to try later on.

New pattern matrix developments

A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.

A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.

We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:

Ferrite magnet test:

For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.

The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.

In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.

Pixelquipu installed at the Open Data Institute

Pixelquipu Inca Harddrives installed at the Open Data Institute (Weaving Codes/coding with knots, with Julian Rohrhuber at the Institut Fuer Musik und Medien) Part of their Thinking out Loud exhibition. Julian also built a sonification installation to play the quipu at different times of the day.

pixelquipu

Here’s a closeup:

bit

A tanglebots workshop report

I’ve tried a lot of different ways of teaching children programming, starting a few years ago with primary school children in a classroom, then doing inset training days for teachers and finally private tutoring in homes. For the finale to the weavingcodes project we are trying a new approach, teaching families about code, robotics and thread by building “tanglebots”.

25956891515_f3d0ea3ec7

The concept is to combine programming with physical objects, concentrating on sensor input and movement as output. It’s important that we incorporate our weavingcodes research process, so deliberately setting goals we don’t yet know the answers to.

The weaving focus allows us to ground the workshop in loom technology and demonstrate the challenges of manipulating thread, with its enormous history of technological development. For the first Cornwall workshop, Ellen started us off with an introduction using FoAM Kernow’s Harris loom and the fundamentals of weaving. We were also joined by Janet and Jon from lovebytes who are helping us to run these events. When first talking about possible workshops with children, we’d discussed the impossibility of making a functional loom in a couple of hours with only broken toys and lego – and so the focus on tangling was suggested by Alex as a way to turn these difficulties to an advantage. Similarly we created a series of prizes for different categories such as “Most technical effort with least impressive result” – inspired by hebocon events.

25956891515_f3d0ea3ec7_2

25324352694_30f0a1a2a4_k2

The workshop format we used is also influenced by Paul Granjon’s wrekshops – wherever possible we’re recycling by pulling apart e-waste, making use of electronics, motors, gears and ideas from the surprising complexity of what’s inside things people are throwing away. This turned out have a powerful implicit message about recycling, parents I talked to had tried taking things apart to learn about them, but the next step – making use of the parts discovered as we were doing here, needs a bit more help to do.

Also as normal for FoAM projects was the importance of the food, in this case tangled by Amber and Francesca to both provide sustenance and inspiration with cardamom knots, spiralised courgetti and tangle fritters.

25931225896_007b088e5d_k2

The groups ended up a bit lopsided, so in future we plan to pre-arrange them as we did on the machine wilderness workshop. In order to do that we need to ask for more information from participants beforehand such as family ages and backgrounds.

We tried using the small Pi touchscreens – these were a bit too fiddly to get away without a mouse, but are much less oppressive somehow than larger PC monitors – as they are so small, they became incorporated into the tanglebots themselves.

Crocodile clips were the best way to connect to random/plundered electronics as well as the lego motors. These removed the need for soldering (which we had set up anyway, but in a separate space).

A selection of other notes we made:

  • Start with a manual tangling exercise (weaving with rope, tablets etc)
  • Lego has a strange all or nothing effect, once you start using it – everything has to work that way, avoiding it may lead to more creative options than including it
  • A first aid kit is needed for these sorts of things
  • The Pimoroni Explorer Hats are good but needed periodic resets in some cases – the motors seemed to get jammed, not sure if this is short circuits interrupting the i2c comms?
  • The Raspberry Pi docs are riddled with minor errors, e.g. the Scratch GPIO section on the explorer hats has a lot of sometimes confusing typos.

All our resources are being uploaded to the kairotic github repository so other people can make use of the materials.

As well as being supported by AHRC Digital Transformations, this project was part of British Science Week, supported by the British Science Association.

25956947035_a44aa6bdd9_k(1)