Pattern Matrix at Algomech (part 1)

I’m writing this on the train with a slightly sleep deprived brain fizzing and popping from thoughts, ideas and conversations from this year’s Algomech festival in Sheffield. The Penelope project took a significant role in the festival, with the group’s participation in the Unmaking Symposium, the exhibition and also testing our latest weavecoding technology at the Algorave. I’ll be writing more on the algorave in a subsequent post.

During the symposium we discussed the critical, liberating and potentially dangerous aspects of Unmaking in a wide variety of contexts – from reverse engineering knitwear and classical Greek dance to discovering the untapped abilities of classical musical instruments when human limitations become a secondary consideration. The symposium also provided us with a good opportunity to take stock of our own group’s current directions and thinking in regard to the Penelope project.

We also had our own corner of the Algomech exhibition, which included Jacquard woven experiments, the Quipu sonification and visualisation and the first public trials of the new pattern matrix V2.

exhib

As is our usual practice, we used this exhibition to get essential feedback on our new design for the pattern matrix, as well as the interpretation of what we are doing – being there in person talking to people allows you to very quickly determine what works, and adapt the focus based on the responses you get.

We discussed the long view on digital technology, the role of weavers in foundations of western mathematics and simply the practicalities of the technology we are developing – the constant stream of visitors represented a wide range of different ages and backgrounds. These aspects of the pattern matrix seemed significant (in no particular order):

  • The construction technique was immediately interesting to most people, specifically the material, beech wood spalting and open frame construction.
  • Related to this, there seems little association with the pattern matrix as a ‘device’ in the sense of a ‘gizmo’ – which is interesting as the Raspberry Pi and other PCBs are clearly visible. We’ve noticed this at some level with the previous version but with the wood construction this effect is much more pronounced.
  • It is seen as being game like, e.g. a “70’s educational toy”. There is an expectation that it is something to be ‘played’, and similarly its potential as a musical sequencer is a common observation.
  • The understandability of the magnet sensing seems a key ingredient. There is no other particularly hidden magic like computer vision or RFID involved, and polarity and digital arrangements of magnets are easily explained and experienced by holding the tokens together.
  • Having some extra circuit boards and wood cut parts to hand, originally intended as backups – were great for people who wanted to know more about what was going on. In future we should also have the token block parts to show as well.
  • The different shaped blocks were immediately appealing – they seemed to invite experimentation more than alternating the binary tokens by flipping them. To follow this up we need to investigate ways to use different shapes to configure thread colour at the same time as structure in a better way than we are now. The black/white sides could define structure while the shape could correspond to the colour for the specific thread. It also indicates that using shaped tokens as instructions for tablet weaving is worth experimenting with quite soon.
  • Younger children focused on the blocks alone (and of course tried all sorts of things no one else did, like stacking them) but slightly older children worked out they were having an effect on the weaving process and generally could patiently work themselves it out without any explanation required.
  • Having Anni Albers’ ‘On Weaving’ book next to pattern matrix helped with older visitors, perhaps representing a more conventional and authoritative source of information to introduce the concepts of notation, structure and pattern in weaving.
The 8-way tangible colour switching instruction
An 8-way tangible colour switching instruction

Physical vs digital – a false dichotomy

More general concepts that came up in conversation included a common theme during Algomech, exploring the inescapably fuzzy boundaries of concepts such as digital, physical and analogue. The myth of the “real world” being analogue and the “virtual world” being digital is a troublesome one to a weaver.

The ‘anti-device’ effect of the pattern matrix has the potential to explore this conundrum, as it represents a seemingly acceptable demonstration of the physical nature of the digital, and that forms of digital technology have inhabited the world of the reassuringly physical for many millennia of human invention.

Check your supply chains

One aspect of the pattern matrix I picked on for the symposium which came up in the exhibition as well was the fact that the beech wood came from a single tree in Cornwall courtesy of Aaron Moore – and used this as an example of our design philosophy of taking on the myth of collapse rather than the myth of infinite abundance.

Feedback from Penelope team members

Having a 5X5 grid initially was thought to be excessive, as most ancient weaves can be expressed with a 4X4 matrix. This larger capability turns out to be more important than we first thought, as it means you can demonstrate the importance of odd and even numbers in the mathematics of weaving.

There is a problem with the single colour change block due to a faulty use of the code from the old version. This has always been a somewhat temporary feature so we should sort this out properly (e.g. using other shapes for colour across the matrix) before we use it next.

We can also try using an augmented reality approach to show the weave structure directly over the grid, so it’s easier to see how the token block changes relate to specific crossings. This could be displayed alongside the current warp weighted loom rather than as a replacement for it.

PCB design for pattern matrix 2

This is the pattern matrix 2 tangible sensor schematic, which is fairly simple – just 4 hall effect sensors and a capacitor to smooth out any noise on the power supply.

sensor-schematic

We need to make hundreds of these for the Penelope Project, and we can save some costs by using the built in pull up resistors in the MCP32017 to get a decent signal from the sensors. The difficulty with this PCB is arranging the sensors so they align with the magnets in the tangible programming block in the optimum manner. From tests with the prototype Lego rig, this isn’t actually too critical – but it’s set up so the lead length can be tweaked a bit at soldering time.

sensor-render

This took me about 20 variations to finally get right, but the circuit is just about simple enough that it can be made single sided – this is good because the top side will be partly exposed, while the lower side with all the copper traces can be protected. It’s good practice to have large areas of copper left connected to ground, partly as it’s a common connection needed all over the board, partly for stability but also it reduces the amount of chemicals required to etch the circuit – as only the parts around the traces need to be removed.

sensor-render2

The i2c expander board is a little more complicated. The design is made to be modular so we can stack up any number of these connected to the Raspberry Pi for different arrangements of sensors. Each board can deal with 8 sensor locations (each comprising 4 individual hall effect sensors). Their job is to convert the digital signals from each sensor into serial data (using the i2c protocol) so the Raspberry Pi can read them all just using 2 wires, plus 2 for power.

i2c-schematic

Each board can be configured to a separate i2c device address to tell it apart from the others using jumper connectors. This one had to be 2 sided, but I managed it without any ‘vias’ (holes to pass traces from one side of the board to another). I also added a power indicator LED as a last minute addition.

i2c

I’ve been learning the open source Kicad software to design these, which is now used by CERN for building the LHC, so it’s pretty fully featured! The idea is that you draw the schematic first, link each component with a physical ‘footprint’, then switch to the PCB design stage. Other software I’ve used in the past tries to route everything in one go for you (and can come up with some pretty strange and messy results). Kicad works in a semi-automatic manner – you need to draw each trace by hand, but it routes them around components and other traces, and suggests the shortest path for you. This is quite a lot better than a fully automatic approach as you have more control over the end result, and easily end up with a decent placement of all the parts.

i2c-render

This project is of course open hardware, and can be found on github here.

Pattern Matrix 2 haptic experiments

One of the potential future additions to our tangible programming hardware is haptic feedback, using sound/vibration to provide a extra channel of information through your fingers when programming with tangible blocks. We wanted to test this before designing the PCB hardware in case we could add it to the system simply – this was initiated by watching a reverse engineering video on youtube (I think by bigclivedotcom) where a technique was mentioned for sneakily reusing input lines (of which we have hundreds) as outputs when they are not being read, by using diodes to separate the signals.

img_20170602_164024

Piezo transducers are often used as cheap speakers, and they can also provide some touch feedback by using lower frequencies. We tested one to see if it interfered with the magnetic fields, and the diaphragm doesn’t seem to be ferrous metal (I’m not actually sure what it is made from) so it can be placed right over the sensors with no effect. A bit more Lego to attach it to the prototype sensor unit, and to see we can feel it through the tangible block when it’s touching the sensor.

The first attempt consisted of simply plugging a speaker into the i/o line from the MCP23017 port expander, switching between input and output on the Pi and adding a diode to prevent the output voltage getting to the sensor.

circuit1

The problem is all the circuits we’re using run on 3.3V, but the piezo speaker I’m using is rated up to 20V – so it’s just not strong enough to feel it. The Pi also provides 5V pins, so a second attempt to use a simple driver circuit (single transistor common emitter amplifier) to boost the voltage:

circuit2

You can hear this version in the video above – it’s playing a sound using a frequency crudely determined by the 4 bit value the block orientation represents whenever it changes. This is louder (loud enough at least to make a recording) but still not enough to feel easily, even when you put the piezo between the sensor and the block. So it seems the best way to get this to work properly will be via a separate circuit, not something we can slot into the existing input hardware. Another advantage of doing it separately is that we can treat it more like a multichannel audio setup with a dedicated processor that is not interrupted by the sensor reading. One solution I saw was to use the same H bridges as used to drive motors from microcontrollers as these take much higher voltages – this will be something to try later on.

New pattern matrix developments

A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.

A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.

We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:

Ferrite magnet test:

For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.

The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.

In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.

A tanglebots workshop report

I’ve tried a lot of different ways of teaching children programming, starting a few years ago with primary school children in a classroom, then doing inset training days for teachers and finally private tutoring in homes. For the finale to the weavingcodes project we are trying a new approach, teaching families about code, robotics and thread by building “tanglebots”.

25956891515_f3d0ea3ec7

The concept is to combine programming with physical objects, concentrating on sensor input and movement as output. It’s important that we incorporate our weavingcodes research process, so deliberately setting goals we don’t yet know the answers to.

The weaving focus allows us to ground the workshop in loom technology and demonstrate the challenges of manipulating thread, with its enormous history of technological development. For the first Cornwall workshop, Ellen started us off with an introduction using FoAM Kernow’s Harris loom and the fundamentals of weaving. We were also joined by Janet and Jon from lovebytes who are helping us to run these events. When first talking about possible workshops with children, we’d discussed the impossibility of making a functional loom in a couple of hours with only broken toys and lego – and so the focus on tangling was suggested by Alex as a way to turn these difficulties to an advantage. Similarly we created a series of prizes for different categories such as “Most technical effort with least impressive result” – inspired by hebocon events.

25956891515_f3d0ea3ec7_2

25324352694_30f0a1a2a4_k2

The workshop format we used is also influenced by Paul Granjon’s wrekshops – wherever possible we’re recycling by pulling apart e-waste, making use of electronics, motors, gears and ideas from the surprising complexity of what’s inside things people are throwing away. This turned out have a powerful implicit message about recycling, parents I talked to had tried taking things apart to learn about them, but the next step – making use of the parts discovered as we were doing here, needs a bit more help to do.

Also as normal for FoAM projects was the importance of the food, in this case tangled by Amber and Francesca to both provide sustenance and inspiration with cardamom knots, spiralised courgetti and tangle fritters.

25931225896_007b088e5d_k2

The groups ended up a bit lopsided, so in future we plan to pre-arrange them as we did on the machine wilderness workshop. In order to do that we need to ask for more information from participants beforehand such as family ages and backgrounds.

We tried using the small Pi touchscreens – these were a bit too fiddly to get away without a mouse, but are much less oppressive somehow than larger PC monitors – as they are so small, they became incorporated into the tanglebots themselves.

Crocodile clips were the best way to connect to random/plundered electronics as well as the lego motors. These removed the need for soldering (which we had set up anyway, but in a separate space).

A selection of other notes we made:

  • Start with a manual tangling exercise (weaving with rope, tablets etc)
  • Lego has a strange all or nothing effect, once you start using it – everything has to work that way, avoiding it may lead to more creative options than including it
  • A first aid kit is needed for these sorts of things
  • The Pimoroni Explorer Hats are good but needed periodic resets in some cases – the motors seemed to get jammed, not sure if this is short circuits interrupting the i2c comms?
  • The Raspberry Pi docs are riddled with minor errors, e.g. the Scratch GPIO section on the explorer hats has a lot of sometimes confusing typos.

All our resources are being uploaded to the kairotic github repository so other people can make use of the materials.

As well as being supported by AHRC Digital Transformations, this project was part of British Science Week, supported by the British Science Association.

25956947035_a44aa6bdd9_k(1)

Tanglebots workshop preparation

It’s workshop time again at Foam Kernow. We’re running a Sonic Kayak development open hacklab with Kaffe Matthews (more on this soon) and a series of tanglebots workshops which will be the finale to the weavingcodes project.

Instead of using my cobbled together homemade interface board, we’re using the pimoroni explorer hat (pro). This comes with some nice features, especially a built in breadboard but also 8 touch buttons, 4 LEDs and two motor drivers. The only downside is that it uses the same power source as the Pi for the motors, so you need to be a little careful as it can reset the Pi if the power draw is too great.

2hat

We have a good stock of recycled e-waste robotic toys we’re going to be using to build with (along with some secondhand lego mindstorms):

toys

Also lots of recycled building material from the amazing Cornwall Scrap Store.

material

In order to keep the workshop balanced between programming and building, and fun for all age groups, we want to use Scratch – rather than getting bogged down with python or similar. In a big improvement to previous versions of the Pi OS, the recent raspbian version (jessie) supports lots of extension hardware including the explorer hat. Things like firing the built in LEDs work ‘out of the box’ like this:

2scratch-led

While the two motor controllers (with speed control!) work like this:

2scratch-motor

The touch buttons were a bit harder to get working as they are not supported by default, so I had to write a scratch driver to do this which you can find here. Once the driver script is running (which launches automatically from the desktop icon), it communicates to scratch over the network and registers the 8 touch buttons as sensors. This means they can be accessed in scratch like so:

2scratch-touch

How to warp a tablet loom (/neolithic digital computing device)

Tablet looms have some interesting properties. Firstly, they are very
very old – our neolithic ancestors invented them. Secondly they
are quite straightforward to make and weave but form an extremely
complex structure that incorporates both weaving and braiding (and one I
haven’t managed to simulate correctly yet) – they are also the only form
of weaving that has never been mechanised.

I’ve learned to warp tablets very much by trial and error, so I expect
there are many improvements possible (please let me know), but as I had
to warp a load of tablet looms for the weavecoding workshop in Düsseldorf last
week, I thought I’d document the process here.

The first thing you need to do is make the tablets themselves from
fairly stiff card. You need squares of about 5cm, and holes punched out
from the corners. Beer mats or playing cards are good, I’m just using recycled card here. It saves a bit of time later if you can get the holes lined up when
the tablets are stacked together – but I’ve never managed to do this
very well. A good number of cards to start with are 8 or 10, fewer in
number are easier to manipulate and use less yarn – if you don’t have
much to spare.

IMG_20160122_120729

The second step is to prepare the warp yarn. You need four separate
balls or cones of wool – it’s easiest to start with four different
colours, although you can make more patterns with double faced weave
(two colours opposite each other on the cards). Fluffy knitting wool
works fine but can catch and be annoying sometimes, cotton is better. In
order to help prevent the yarn getting tangled together which is
probably the biggest problem with this job – it’s a good idea to set
this up so the yarn passes through the back of a chair with the yarn on
the floor like this – it restricts the distance the balls roll as you
pull the yarn.

IMG_20160122_120905

You need two sticks you can easily loop the warp over, I’m using a cut
broom stick clamped to the chair here. The distance between them determines
how long the final woven band will be, a metre or so is good.

Next you need to thread each of the four warp threads through the
corners of the tablets – each thread needs it’s own corner so be careful
not to mix them up. If the holes line up you can do them all at once,
otherwise it’s one by one.

IMG_20160122_121329

Here are the tablets with all the corners threaded.

IMG_20160122_122040

Now we tie the threads together looped over the warp stick furthest from
the yarn, this knot is temporary.

IMG_20160122_122324

Hook the other end of the warp over the stick on the other side, and
leave one of the tablets half way along. Loop it back again leaving
another tablet – go backwards and forwards repeating.

IMG_20160122_122645

I can never manage to keep the tablets in order, and usually end up with a mess like this – don’t panic if you get a similar result!

IMG_20160122_123113

When you have done all of the tablets, quickly check that every warp
pass has a card associated with it and then cut the first knot and tie
the last warp threads to the first.

IMG_20160122_123216

This gives you a continuous warp, which is good for adjusting the
tension. Group all the cards together and arrange them so the colours
are aligned – rotate them until the same colours are at the top and the bottom.

IMG_20160122_123849

This is also a good point to check the twist of the tablets so they alternate in
terms of the direction the threads are coming from. This is quite
difficult to explain with text but these images may help. It’s basically a bit easier if they are consistent when you start weaving, then you can see how changing this alters the patterns as you go.

IMG_20160122_123910

IMG_20160122_123927

You can actually reorder and flip the tablets at any time, even after
starting weaving – so none of this is critical. It’s handy to equalise
the tension between the warp threads at this point though, so grab all
the cards in alignment, put some tension on the warp and drag them up
and down the warp – if the loops at either end are not too close this
should get the lengths about the same.

Once this is done, tie all the tablets together to preserve their order
and alignment.

IMG_20160122_124223

Then tie loops of strong cord around both ends of the warp.

IMG_20160122_124426

Then you’re done – you can pull the ends off the sticks and start tablet
weaving!

IMG_20160122_124528

As I needed to transport the tablet looms I wrapped the warps (keeping the upper/lower threads separated) around cardboard tubes to keep the setup from getting tangled up. This seemed to work well:

IMG_20160122_130602

If you find one of the warp threads is too long and is causing
a tablet to droop when the warp is under tension, you can pull it tight
and tie it back temporarily, after weaving a few wefts it will hold in
place.

The two most important things I’ve learned about weaving – the older the
technique the more forgiving it is to mistakes, and you can never have
too many sticks.

A cryptoweaving experiment

Archaeologists can read a woven artifact created thousands of years ago, and from its structure determine the actions performed in the right order by the weaver who created it. They can then recreate the weaving, following in their ancestor’s ‘footsteps’ exactly.

This is possible because a woven artifact encodes time digitally, weft by weft. In most other forms of human endeavor, reverse engineering is still possible (e.g. in a car or a cake) but instructions are not encoded in the object’s fundamental structure – they need to be inferred by experiment or indirect means. Similarly, a text does not quite represent its writing process in a time encoded manner, but the end result. Interestingly, one possible self describing artifact could be a musical performance.

Looked at this way, any woven pattern can be seen as a digital record of movement performed by the weaver. We can create the pattern with a notation that describes this series of actions (a handweaver following a lift plan), or move in the other direction like the archaeologist, recording a given notation from an existing weave.

example
A weaving and its executable code equivalent.

One of the potentials of weaving I’m most interested in is being able to demonstrate fundamentals of software in threads – partly to make the physical nature of computation self evident, but also as a way of designing new ways of learning and understanding what computers are.

If we take the code required to make the pattern in the weaving above:

(twist 3 4 5 14 15 16)
(weave-forward 3)
(twist 4 15)
(weave-forward 1)
(twist 4 8 11 15)

(repeat 2
 (weave-back 4)
 (twist 8 11)
 (weave-forward 2)
 (twist 9 10)
 (weave-forward 2)
 (twist 9 10)
 (weave-back 2)
 (twist 9 10)
 (weave-back 2)
 (twist 8 11)
 (weave-forward 4))

We can “compile” it into a binary form which describes each instruction – the exact process for this is irrelevant, but here it is anyway – an 8 bit encoding, packing instructions and data together:

8bit instruction encoding:

Action  Direction  Count/Tablet ID (5 bit number)
0 1         2              3 4 5 6 7 

Action types
weave:    01 (1)
rotate:   10 (2)
twist:    11 (3)

Direction
forward: 0
backward: 1

If we compile the code notation above with this binary system, we can then read the binary as a series of tablet weaving card flip rotations (I’m using 20 tablets, so we can fit in two instructions per weft):

0 1 6 7 10 11 15
0 1 5 7 10 11 14 15 16
0 1 4 5 6 7 10 11 13
1 6 7 10 11 15
0 1 5 7 11 17
0 1 5 10 11 14
0 1 4 6 7 10 11 14 15 16 17
0 1 2 3 4 5 6 7 11 12 15
0 1 4 10 11 14 16
1 6 10 11 14 17
0 1 4 6 11 16
0 1 4 7 10 11 14 16
1 2 6 10 11 14 17
0 1 4 6 11 12 16
0 1 4 7 10 11 14 16
1 5

If we actually try weaving this (by advancing two turns forward/backward at a time) we get this mess:

close

The point is that (assuming we’ve made no mistakes) this weave represents *exactly* the same information as the pattern does – you could extract the program from the messy encoded weave, follow it and recreate the original pattern exactly.

The messy pattern represents both an executable, as well as a compressed form of the weave – taking up less space than the original pattern, but looking a lot worse. Possibly this is a clue too, as it contains a higher density of information – higher entropy, and therefore closer to randomness than the pattern.

Pixel Quipu

The graphviz visualisations we’ve been using for quipu have quite a few limitations, as they tend to make very large images, and there is limited control over how they are drawn. It would be better to be able to have more of an overview of the data, also rendering the knots in the right positions with the pendants being the right length.

Meet the pixelquipu!

ur018

These are drawn using a python script which reads the Harvard Quipu Database and renders quipu structure using the correct colours. The knots are shown as a single pixel attached to the pendant, with a colour code of red as single knot, green for a long knot and blue as a figure of eight knot (yellow is unknown or missing). The value of the knot sets the brightness of the pixel. The colour variations for the pendants are working, but no difference between twisted and alternating colours, also no twist direction is visualised yet.

hp017

Another advantage of this form of rendering is that we can draw data entropy within the quipu in order to provide a different view of how the data is structured, as a attempt to uncover hidden complexity. This is done hierarchically so a pendant’s entropy is that of its data plus all the sub-pendants, which seemed most appropriate given the non-linear form that the data takes.

ur037

e-ur037

We can now look at some quipus in more detail – what was the purpose of the red and grey striped pendants in the quipu below? They contain no knots, are they markers of some kind? This also seems to be a quipu where the knots do not follow the decimal coding pattern that we understand, they are mostly long knots of various values.

ur051

There also seems to be data stored in different kinds of structure in the same quipu – the collection of sub-pendants below in the left side presumably group data in a more hierarchical manner than the right side, which seems much more linear – and also a colour change emphasises this.

ur015

Read left to right, this long quipu below seems very much like you’d expect binary data to look – some kind of header information or preamble, followed by a repeating structure with local variation. The twelve groups of eight grey pendants seem redundant – were these meant to be filled in later? Did they represent something important without containing any knots? We will probably never know.

UR1176

The original thinking of the pixelquipu was to attempt to fit all the quipus on a single page for viewing, as it represents them with the absolute minimum pixels required. Here are both pendant colour and entropy shown for all 247 quipu we have the data for:

all

entropy-local