Pattern Matrix at Algomech (part 1)

I’m writing this on the train with a slightly sleep deprived brain fizzing and popping from thoughts, ideas and conversations from this year’s Algomech festival in Sheffield. The Penelope project took a significant role in the festival, with the group’s participation in the Unmaking Symposium, the exhibition and also testing our latest weavecoding technology at the Algorave. I’ll be writing more on the algorave in a subsequent post.

During the symposium we discussed the critical, liberating and potentially dangerous aspects of Unmaking in a wide variety of contexts – from reverse engineering knitwear and classical Greek dance to discovering the untapped abilities of classical musical instruments when human limitations become a secondary consideration. The symposium also provided us with a good opportunity to take stock of our own group’s current directions and thinking in regard to the Penelope project.

Pix thanks to
Pix thanks to James Vanderhoven

comp

We also had our own corner of the Algomech exhibition, which included Jacquard woven experiments, the Quipu sonification and visualisation and the first public trials of the new pattern matrix V2.

exhib

As is our usual practice, we used this exhibition to get essential feedback on our new design for the pattern matrix, as well as the interpretation of what we are doing – being there in person talking to people allows you to very quickly determine what works, and adapt the focus based on the responses you get.

We discussed the long view on digital technology, the role of weavers in foundations of western mathematics and simply the practicalities of the technology we are developing – the constant stream of visitors represented a wide range of different ages and backgrounds. These aspects of the pattern matrix seemed significant (in no particular order):

  • The construction technique was immediately interesting to most people, specifically the material, beech wood spalting and open frame construction.
  • Related to this, there seems little association with the pattern matrix as a ‘device’ in the sense of a ‘gizmo’ – which is interesting as the Raspberry Pi and other PCBs are clearly visible. We’ve noticed this at some level with the previous version but with the wood construction this effect is much more pronounced.
  • It is seen as being game like, e.g. a “70’s educational toy”. There is an expectation that it is something to be ‘played’, and similarly its potential as a musical sequencer is a common observation.
  • The understandability of the magnet sensing seems a key ingredient. There is no other particularly hidden magic like computer vision or RFID involved, and polarity and digital arrangements of magnets are easily explained and experienced by holding the tokens together.
  • Having some extra circuit boards and wood cut parts to hand, originally intended as backups – were great for people who wanted to know more about what was going on. In future we should also have the token block parts to show as well.
  • The different shaped blocks were immediately appealing – they seemed to invite experimentation more than alternating the binary tokens by flipping them. To follow this up we need to investigate ways to use different shapes to configure thread colour at the same time as structure in a better way than we are now. The black/white sides could define structure while the shape could correspond to the colour for the specific thread. It also indicates that using shaped tokens as instructions for tablet weaving is worth experimenting with quite soon.
  • Younger children focused on the blocks alone (and of course tried all sorts of things no one else did, like stacking them) but slightly older children worked out they were having an effect on the weaving process and generally could patiently work themselves it out without any explanation required.
  • Having Anni Albers’ ‘On Weaving’ book next to pattern matrix helped with older visitors, perhaps representing a more conventional and authoritative source of information to introduce the concepts of notation, structure and pattern in weaving.
Pix thanks to James Vanderhoven
Pix thanks to James Vanderhoven
The 8-way tangible colour switching instruction
An 8-way tangible colour switching instruction

Physical vs digital – a false dichotomy

More general concepts that came up in conversation included a common theme during Algomech, exploring the inescapably fuzzy boundaries of concepts such as digital, physical and analogue. The myth of the “real world” being analogue and the “virtual world” being digital is a troublesome one to a weaver.

The ‘anti-device’ effect of the pattern matrix has the potential to explore this conundrum, as it represents a seemingly acceptable demonstration of the physical nature of the digital, and that forms of digital technology have inhabited the world of the reassuringly physical for many millennia of human invention.

Check your supply chains

One aspect of the pattern matrix I picked on for the symposium which came up in the exhibition as well was the fact that the beech wood came from a single tree in Cornwall courtesy of Aaron Moore – and used this as an example of our design philosophy of taking on the myth of collapse rather than the myth of infinite abundance.

img_20171107_145340

Feedback from Penelope team members

Having a 5X5 grid initially was thought to be excessive, as most ancient weaves can be expressed with a 4X4 matrix. This larger capability turns out to be more important than we first thought, as it means you can demonstrate the importance of odd and even numbers in the mathematics of weaving.

There is a problem with the single colour change block due to a faulty use of the code from the old version. This has always been a somewhat temporary feature so we should sort this out properly (e.g. using other shapes for colour across the matrix) before we use it next.

We can also try using an augmented reality approach to show the weave structure directly over the grid, so it’s easier to see how the token block changes relate to specific crossings. This could be displayed alongside the current warp weighted loom rather than as a replacement for it.

New pattern matrix developments

A few weeks ago we kicked off the new Penelope project, and while in Munich one of our first jobs was to deliver the prototype pattern matrix to the Museum of Casts of Classical sculpture for exhibition over the summer as part of our Penelopean lab. Our next mission in Cornwall is to design new tangible programming hardware so we can start manufacturing a small run of alternative versions with more sensors to try new experiments with them. Some of them will be used for public exhibition, others for the researchers to use in talks and seminars, others for musical livecoding performances.

A big focus for us is the materials and physical design, on the one hand like everything FoAM Kernow builds it needs to be open source and appropriate technology (so easily explainable and built by others) and on the other it needs to be sympathetic to it’s context in the museum, displayed alongside looms and technology that are thousands of years old. This has resonance with the Al Jazari livecoding installation in the Alhambra in 2008, where a juxtaposition of modern and ancient curiously worked. As part of this we want to switch from materials like aluminium and plastic to wood construction – employing similar building techniques to the looms themselves, but more along the lines of inspiration to inform alternative technological choices rather than simple mimicry.

We’re also trying out simpler electronics designs – firstly switching to slightly cheaper hall effect sensors (SS411P from SS411A, previously) and testing different kinds of magnets – which turns out to be the more tricky part to get right. Here is a rare earth magnet test:

Ferrite magnet test:

For environmental and cost reasons ferrite magnets would be much better to use, and they are strong enough to be picked up by the sensors in a useful range – however presumably in order to increase their ‘stickiness’ it appears that ferrite magnets are often magnetised in complex ways, with both poles being present on the same (active) side, and much reduced on the other. This means we can’t use them in the same way, they flip the field on and off with the same orientation and don’t do anything on the other. We’re still searching for a fix for this, but currently the best we can do is reduce the rare earth magnet thickness to 1.5 mm from 3mm used in the prototype.

The prototype was useful for demonstrating that we can use digital signals rather than needing analogue sensors which it was built to allow if the hall effect sensors were not good enough – so a big development is removing the microcontrollers we needed before and replacing them with port expander ICs (MCP23017). These even use the same serial communication we were using (i2c) to talk to the Raspberry Pi so it’s a straight swap.

In order to test the new system all together as well as new magnet combinations and spacing we built a prototype with lego to hold the sensors in the right position, and provide the base for the tangible programming block to rest or be rotated on. This is important to do for the design of the PCB before it goes for production – as we can’t change the sensor position afterwards, more on that part soon.

A tanglebots workshop report

I’ve tried a lot of different ways of teaching children programming, starting a few years ago with primary school children in a classroom, then doing inset training days for teachers and finally private tutoring in homes. For the finale to the weavingcodes project we are trying a new approach, teaching families about code, robotics and thread by building “tanglebots”.

25956891515_f3d0ea3ec7

The concept is to combine programming with physical objects, concentrating on sensor input and movement as output. It’s important that we incorporate our weavingcodes research process, so deliberately setting goals we don’t yet know the answers to.

The weaving focus allows us to ground the workshop in loom technology and demonstrate the challenges of manipulating thread, with its enormous history of technological development. For the first Cornwall workshop, Ellen started us off with an introduction using FoAM Kernow’s Harris loom and the fundamentals of weaving. We were also joined by Janet and Jon from lovebytes who are helping us to run these events. When first talking about possible workshops with children, we’d discussed the impossibility of making a functional loom in a couple of hours with only broken toys and lego – and so the focus on tangling was suggested by Alex as a way to turn these difficulties to an advantage. Similarly we created a series of prizes for different categories such as “Most technical effort with least impressive result” – inspired by hebocon events.

25956891515_f3d0ea3ec7_2

25324352694_30f0a1a2a4_k2

The workshop format we used is also influenced by Paul Granjon’s wrekshops – wherever possible we’re recycling by pulling apart e-waste, making use of electronics, motors, gears and ideas from the surprising complexity of what’s inside things people are throwing away. This turned out have a powerful implicit message about recycling, parents I talked to had tried taking things apart to learn about them, but the next step – making use of the parts discovered as we were doing here, needs a bit more help to do.

Also as normal for FoAM projects was the importance of the food, in this case tangled by Amber and Francesca to both provide sustenance and inspiration with cardamom knots, spiralised courgetti and tangle fritters.

25931225896_007b088e5d_k2

The groups ended up a bit lopsided, so in future we plan to pre-arrange them as we did on the machine wilderness workshop. In order to do that we need to ask for more information from participants beforehand such as family ages and backgrounds.

We tried using the small Pi touchscreens – these were a bit too fiddly to get away without a mouse, but are much less oppressive somehow than larger PC monitors – as they are so small, they became incorporated into the tanglebots themselves.

Crocodile clips were the best way to connect to random/plundered electronics as well as the lego motors. These removed the need for soldering (which we had set up anyway, but in a separate space).

A selection of other notes we made:

  • Start with a manual tangling exercise (weaving with rope, tablets etc)
  • Lego has a strange all or nothing effect, once you start using it – everything has to work that way, avoiding it may lead to more creative options than including it
  • A first aid kit is needed for these sorts of things
  • The Pimoroni Explorer Hats are good but needed periodic resets in some cases – the motors seemed to get jammed, not sure if this is short circuits interrupting the i2c comms?
  • The Raspberry Pi docs are riddled with minor errors, e.g. the Scratch GPIO section on the explorer hats has a lot of sometimes confusing typos.

All our resources are being uploaded to the kairotic github repository so other people can make use of the materials.

As well as being supported by AHRC Digital Transformations, this project was part of British Science Week, supported by the British Science Association.

25956947035_a44aa6bdd9_k(1)

A cryptoweaving experiment

Archaeologists can read a woven artifact created thousands of years ago, and from its structure determine the actions performed in the right order by the weaver who created it. They can then recreate the weaving, following in their ancestor’s ‘footsteps’ exactly.

This is possible because a woven artifact encodes time digitally, weft by weft. In most other forms of human endeavor, reverse engineering is still possible (e.g. in a car or a cake) but instructions are not encoded in the object’s fundamental structure – they need to be inferred by experiment or indirect means. Similarly, a text does not quite represent its writing process in a time encoded manner, but the end result. Interestingly, one possible self describing artifact could be a musical performance.

Looked at this way, any woven pattern can be seen as a digital record of movement performed by the weaver. We can create the pattern with a notation that describes this series of actions (a handweaver following a lift plan), or move in the other direction like the archaeologist, recording a given notation from an existing weave.

example
A weaving and its executable code equivalent.

One of the potentials of weaving I’m most interested in is being able to demonstrate fundamentals of software in threads – partly to make the physical nature of computation self evident, but also as a way of designing new ways of learning and understanding what computers are.

If we take the code required to make the pattern in the weaving above:

(twist 3 4 5 14 15 16)
(weave-forward 3)
(twist 4 15)
(weave-forward 1)
(twist 4 8 11 15)

(repeat 2
 (weave-back 4)
 (twist 8 11)
 (weave-forward 2)
 (twist 9 10)
 (weave-forward 2)
 (twist 9 10)
 (weave-back 2)
 (twist 9 10)
 (weave-back 2)
 (twist 8 11)
 (weave-forward 4))

We can “compile” it into a binary form which describes each instruction – the exact process for this is irrelevant, but here it is anyway – an 8 bit encoding, packing instructions and data together:

8bit instruction encoding:

Action  Direction  Count/Tablet ID (5 bit number)
0 1         2              3 4 5 6 7 

Action types
weave:    01 (1)
rotate:   10 (2)
twist:    11 (3)

Direction
forward: 0
backward: 1

If we compile the code notation above with this binary system, we can then read the binary as a series of tablet weaving card flip rotations (I’m using 20 tablets, so we can fit in two instructions per weft):

0 1 6 7 10 11 15
0 1 5 7 10 11 14 15 16
0 1 4 5 6 7 10 11 13
1 6 7 10 11 15
0 1 5 7 11 17
0 1 5 10 11 14
0 1 4 6 7 10 11 14 15 16 17
0 1 2 3 4 5 6 7 11 12 15
0 1 4 10 11 14 16
1 6 10 11 14 17
0 1 4 6 11 16
0 1 4 7 10 11 14 16
1 2 6 10 11 14 17
0 1 4 6 11 12 16
0 1 4 7 10 11 14 16
1 5

If we actually try weaving this (by advancing two turns forward/backward at a time) we get this mess:

close

The point is that (assuming we’ve made no mistakes) this weave represents *exactly* the same information as the pattern does – you could extract the program from the messy encoded weave, follow it and recreate the original pattern exactly.

The messy pattern represents both an executable, as well as a compressed form of the weave – taking up less space than the original pattern, but looking a lot worse. Possibly this is a clue too, as it contains a higher density of information – higher entropy, and therefore closer to randomness than the pattern.

Procedural weave rendering

We’ve been working on new approaches to 3D rendering ancient weaves, using Alex’s new behavioural language (which describes a weave from the perspective of a single thread) as the description for our modelling. This new approach allows us to build a fabric out of a single geometric shape, where warp and weft are part of the same thread.

toothpaste-mix

This is mix of tabby and 2:2 twill, created by this code:

warp 12 24 ++ [TurnIn] ++ threadWeftBy'' Odd (rot 3) ([Over,Under]) 12 12 ++ threadWeftBy'' Odd (rot 3) ([Over,Over,Under,Under]) 12 12

I’m still learning this language, but more on that soon. This line produces an large list of instructions the weave renderer uses to build it’s model, turning the thread and shifting it up and down as it crosses itself.

In the video in his last post Alex describes using this to mix two separate weaving techniques together, which is one of our main reasons for developing this language – existing weave simulations cannot replicate the weaving technology of the ancient Greeks who for example, combined tablet and warp weighted weaving in the same fabric.

The second problem with weave simulations is shown by the following screenshot from a popular existing system:

wxsg2b

Fabrics modelled in this way are considered to infinitely repeating sections with chopped off threads. There is no consideration for the selvedge at the edge of the fabric – which as we’ve shown in our past research is almost like a completely separate weave system of it’s own, and rarely considered by notation systems or modelling (and often left to the weaver to ‘livecode’). Here is a different view of the same fabric:

toothpaste-edge

We can also now introduce other changes to the yarn structure, for example modifying the width using a sine wave.

toothpaste-yarnwidth

I still have a few glitches to fix as you can see above, but here is a video of the development process from the first script, getting the polygons lined up, fixing the turning, adding over/under, reading Alex’s code and finally lining everything up.

“The mystery of the drawdown”

Double weave has intrigued me since first figuring out how it works with tablets – it shows how weaving is a 3D process, and is an example of shape making from code. It’s the starting point for more advanced methods for creating strong woven composite materials and structures. I’ve been reading this document by Paul O’Connor to understand the process for a 4 shaft loom. He includes this critique on current methods for visualising weaving:

The traditional method for creating the drawdown shows all the warp and all the weft threads in one layer. What is needed is a way to make two drawdowns, one for the top cloth layer and the other for the bottom cloth layer. Most of the commercially available computer weave programs display the drawdown in this same fashion, as a single layer.

This is a draft plan (the traditional notation technique) for double weave which contains the drawdown he’s talking about:

double

Drawdowns are designed for thinking about weaving as 2D pattern, and like any notation system they abstract a situation so we can reason about it, with the trade-off that they make it impossible to understand different aspects. This is a big problem we have in programming too – and is the reason we need to have many levels and hundreds of different languages to describe problems. I’ve noticed that if people (or organisations) stick with one too long, the specialisation starts to hinder ability to even recognise certain issues. It’s interesting to see such a clear analogy here.

In order to see what’s really happening with double weave, we need to switch to a different notation and look at the structure more closely:

double2

Hopefully you can see that all of the black threads are sitting on top of all the white threads, and forms it’s own plain or tabby weave fabric. The same is true for the white threads underneath. Below is the same structure rendered in the weavecoding dyadic device – this shows the heddles that need raising to create the sheds in the lift plan, and you can see a repeating zigzag pattern. As an aside, I’ve noticed that there seems to be some kind of underlying categorisation of lift plan patterns which I’ve not found mentioned anywhere yet – something else to look into at some point.

double-d

Weavecoding Munich

Ellen’s exhibition in Munich was always going to be a pivotal event in the weavecoding project – one of the first opportunities to expose our work to a large audience. The Museum of casts of classical sculptures was the perfect context for the mythical aspects of weaving, overlooked by Penelope and friends with her subversive woven/unwoven work, we could explore the connections between livecoding and weaving.

IMG_8477 2

Practically we focused on developing the tangible weavecoding exhibit for events later in the week, as well as discussing the many languages we have developed so far for different looms and weaving techniques. One of our discoveries is that none of the models or languages we have created seem sufficient in themselves – weaving could be far too big to be able to be described or solved from a single perspective. We’ve tried approaches describing weave structures from the actions of the weaver, setup of the loom and structure of the fabric – perhaps the most promising is to explor the story of weaving from the perspective of the thread itself.

IMG_20150510_062737

IMG_20150508_153211

One of the distinctive things about weaving in antiquity is how multiple technologies were combined to form a single piece of fabric, weaving in different directions, weft becoming warp, use of tablets vs warp weighted weaving. To explain this via the path of a single conceptual thread crossing through itself may make this possible to describe in a more flexible, declarative and abstracted manner than having to explain each method separately as if in it’s own world.

IMG_20150508_152045

IMG_20150509_100421

The pattern matrix has now been made into good shape for explaining the relationship between colour and structure in pattern formation. For the first time we also used all 4 sensors per block on the bottom row which meant we could use a special “colour” block that the system recognises from the normal warp/weft ones and use it’s rotation to choose between 8 preset colour settings. This was quite a breakthrough as it had all been theoretical before.

IMG_20150508_194220

Adding this more complex use of the magnetic patterns meant that Alex could set up the matrix as a tangible interface for his tidal livecoding software meaning Ellen could join us for a collaborative slub weavecoding performance on the Saturday evening. The prospect of performing together was something we have talked about since the very beginning of the project, so it was great to finally reach this point. The reverb in the museum was vast, meaning that we had to play the space a lot, and provide ‘music for looking at sculptures by':

Future Thinking for Social Living: Weavecoding in assisted housing

Our work on weavecoding is now reaching out to other uses and projects. One is Future Thinking for Social Living, run by Magda Ty┼╝lik-Carver and Fiona Hackney.

This research project aims to look at the relationship between wellbeing, home, making and technology and is centred on Miners Court, who provide assisted housing in Redruth in Cornwall. As well as a range of flats and accommodation, the residents have shared communal areas with a variety of activities throughout the week. Along with Christiane Berghoff, Robin Hawes and Lucie Hernandez we set up camp with a lot of materials for knitting, crochet and weaving as well as some Raspberry Pis and the all new pattern matrix tangible weavecoding device.

miners

The Future Thinking for Social Living project is set up to research how we can think more critically about home and community, and with particular focus on the future. From discussions with the staff at Miners Court – specific issues they are interested in are how to make better use of communal spaces, and how can they get more men involved with crafts and shared activities.

I’m also interested in how we can use these settings for artists residencies – how does working with people like this affect a design process, does working in such a place – and using it as way to start conversations (rather than being too much in ‘teacher mode’) affect the people living there positively? Also the weavecoding project provides some ideas in bridging gaps, both between technology and people – but also across gender gaps, mixing textiles with electronics for example.

miners2
Here is the new magnetic pattern matrix, running the 3D Raspberry Pi warp weighted loom simulation (more on this soon!) with a nice 4 shaft loom in the background.

On Monday and Tuesday we spent a long time talking, weaving, knitting and making cups of tea of course (and a bit of time debugging magnets on my part). I’ve found helping people weave with tablets on the inkle loom is a good way to get talking, as this seems new to even people who are experienced with crafts. It also appeals to people with mathematics or design background who normally are uninterested in knitting and other crafts, and seems gender neutral perhaps for the same reasons. It also helps to talk about the history of what we are weaving with, the fact that this is an ancient technique and yet there are so many surprises – I can’t really predict to them what will happen e.g. to the pattern when we change rotation direction, and this seems to be important.

What we have yet to do (but a few weeks to experiment yet) is bridge the technology gap. Many of them have an immediate reaction of distaste to computers, as most of them have them but report that they have become unusable or feel that they are not designed well with their needs in mind. Partly the situation of having some circuit boards getting tangled up in the more familiar materials and using the Raspberry Pi simulation to show what is happening on the loom next to it is a start. One interesting thing is that neither the Pi nor the AVR boards look enough like ‘a computer’ for it to stand out too much (which also part of the Pi’s role in the classroom) – this was more so after plugging it into their large TV and getting rid of the monitor. As it gradually gets into a working state, I’d like to first try using it to demonstrate well known weaves – e.g. plain, twill and satin.

Working in this environment on the pattern matrix between weaving with different people has already had an effect on it’s design process. One initial observation resulted in reducing the magnet strength – I hadn’t even considered before that having them snap together too forcefully would be a problem for some people. Such things are obvious in these kinds of settings.

Loose threads from weavecoding

Midway through the weavecoding project and our researches have thrown up a whole load of topics that either don’t quite fit into our framework, or we simply won’t have time to pursue properly. Here are some of the tangents I’ve collected so far.

Coding with knots: Khipu

One of the cultures I’m increasingly interested in are the Incas. Their empire flourished to up to 37 million people, without the need of money or a written language. We know that some numeric information was stored using Khipu, a knot based recording system which was used in combination with black and white stones to read and calculate. Two thirds of the quipus we have are un-translated, and do not fit into the known numeric coding system – what information do they hold?

quipu

Harvard University provides a Khipu Database Project with many surviving examples documented – I’m hoping to run a workshop soon to look through some of this data in a variety of ways.

Tablet weaving NAND gates

Image2
Diagram thanks to Phiala’s String Page – the only place I’ve seen tablet weaving explained properly.

There are logic gates in tablet weaving logic. I haven’t fully figured this out yet, but I noticed modelling tablet weaving that you end up basically mapping the combinations of the weaving actions (such as turn direction) and colour as truth tables.

Top face colour based on top left/top right hole yarn in a single card and turn direction (clockwise/counter clockwise)

TL Yarn : TR Yarn : Turn : Top face colour
--------------------------------------------
Black   : Black   : CCW  : Black
Black   : Black   : CW   : Black
Black   : White   : CCW  : Black
Black   : White   : CW   : White
White   : Black   : CCW  : White
White   : Black   : CW   : Black
White   : White   : CCW  : White
White   : White   : CW   : White

Things get stranger when you include twist and combinations of actions with multiple cards. Would it be possible to compile high level programming languages into weaving instructions for carrying out computation? Perhaps this is what the untranslatable quipus are about?

Nintendo made a knitting machine

We could really do with some of these, unfortunately they never went beyond prototype stage.

nintendo

Asemic writing

Asemic writing is a post-literate written form with no semantic content. Miles Visman programs procedural asemic languages and hand weaves them. I think this may be an important connection to livecoding at some point.

aw001