Pattern Matrix at Algomech (part 1)

I’m writing this on the train with a slightly sleep deprived brain fizzing and popping from thoughts, ideas and conversations from this year’s Algomech festival in Sheffield. The Penelope project took a significant role in the festival, with the group’s participation in the Unmaking Symposium, the exhibition and also testing our latest weavecoding technology at the Algorave. I’ll be writing more on the algorave in a subsequent post.

During the symposium we discussed the critical, liberating and potentially dangerous aspects of Unmaking in a wide variety of contexts – from reverse engineering knitwear and classical Greek dance to discovering the untapped abilities of classical musical instruments when human limitations become a secondary consideration. The symposium also provided us with a good opportunity to take stock of our own group’s current directions and thinking in regard to the Penelope project.

We also had our own corner of the Algomech exhibition, which included Jacquard woven experiments, the Quipu sonification and visualisation and the first public trials of the new pattern matrix V2.

exhib

As is our usual practice, we used this exhibition to get essential feedback on our new design for the pattern matrix, as well as the interpretation of what we are doing – being there in person talking to people allows you to very quickly determine what works, and adapt the focus based on the responses you get.

We discussed the long view on digital technology, the role of weavers in foundations of western mathematics and simply the practicalities of the technology we are developing – the constant stream of visitors represented a wide range of different ages and backgrounds. These aspects of the pattern matrix seemed significant (in no particular order):

  • The construction technique was immediately interesting to most people, specifically the material, beech wood spalting and open frame construction.
  • Related to this, there seems little association with the pattern matrix as a ‘device’ in the sense of a ‘gizmo’ – which is interesting as the Raspberry Pi and other PCBs are clearly visible. We’ve noticed this at some level with the previous version but with the wood construction this effect is much more pronounced.
  • It is seen as being game like, e.g. a “70’s educational toy”. There is an expectation that it is something to be ‘played’, and similarly its potential as a musical sequencer is a common observation.
  • The understandability of the magnet sensing seems a key ingredient. There is no other particularly hidden magic like computer vision or RFID involved, and polarity and digital arrangements of magnets are easily explained and experienced by holding the tokens together.
  • Having some extra circuit boards and wood cut parts to hand, originally intended as backups – were great for people who wanted to know more about what was going on. In future we should also have the token block parts to show as well.
  • The different shaped blocks were immediately appealing – they seemed to invite experimentation more than alternating the binary tokens by flipping them. To follow this up we need to investigate ways to use different shapes to configure thread colour at the same time as structure in a better way than we are now. The black/white sides could define structure while the shape could correspond to the colour for the specific thread. It also indicates that using shaped tokens as instructions for tablet weaving is worth experimenting with quite soon.
  • Younger children focused on the blocks alone (and of course tried all sorts of things no one else did, like stacking them) but slightly older children worked out they were having an effect on the weaving process and generally could patiently work themselves it out without any explanation required.
  • Having Anni Albers’ ‘On Weaving’ book next to pattern matrix helped with older visitors, perhaps representing a more conventional and authoritative source of information to introduce the concepts of notation, structure and pattern in weaving.
The 8-way tangible colour switching instruction
An 8-way tangible colour switching instruction

Physical vs digital – a false dichotomy

More general concepts that came up in conversation included a common theme during Algomech, exploring the inescapably fuzzy boundaries of concepts such as digital, physical and analogue. The myth of the “real world” being analogue and the “virtual world” being digital is a troublesome one to a weaver.

The ‘anti-device’ effect of the pattern matrix has the potential to explore this conundrum, as it represents a seemingly acceptable demonstration of the physical nature of the digital, and that forms of digital technology have inhabited the world of the reassuringly physical for many millennia of human invention.

Check your supply chains

One aspect of the pattern matrix I picked on for the symposium which came up in the exhibition as well was the fact that the beech wood came from a single tree in Cornwall courtesy of Aaron Moore – and used this as an example of our design philosophy of taking on the myth of collapse rather than the myth of infinite abundance.

Feedback from Penelope team members

Having a 5X5 grid initially was thought to be excessive, as most ancient weaves can be expressed with a 4X4 matrix. This larger capability turns out to be more important than we first thought, as it means you can demonstrate the importance of odd and even numbers in the mathematics of weaving.

There is a problem with the single colour change block due to a faulty use of the code from the old version. This has always been a somewhat temporary feature so we should sort this out properly (e.g. using other shapes for colour across the matrix) before we use it next.

We can also try using an augmented reality approach to show the weave structure directly over the grid, so it’s easier to see how the token block changes relate to specific crossings. This could be displayed alongside the current warp weighted loom rather than as a replacement for it.

Pixel Quipu

The graphviz visualisations we’ve been using for quipu have quite a few limitations, as they tend to make very large images, and there is limited control over how they are drawn. It would be better to be able to have more of an overview of the data, also rendering the knots in the right positions with the pendants being the right length.

Meet the pixelquipu!

ur018

These are drawn using a python script which reads the Harvard Quipu Database and renders quipu structure using the correct colours. The knots are shown as a single pixel attached to the pendant, with a colour code of red as single knot, green for a long knot and blue as a figure of eight knot (yellow is unknown or missing). The value of the knot sets the brightness of the pixel. The colour variations for the pendants are working, but no difference between twisted and alternating colours, also no twist direction is visualised yet.

hp017

Another advantage of this form of rendering is that we can draw data entropy within the quipu in order to provide a different view of how the data is structured, as a attempt to uncover hidden complexity. This is done hierarchically so a pendant’s entropy is that of its data plus all the sub-pendants, which seemed most appropriate given the non-linear form that the data takes.

ur037

e-ur037

We can now look at some quipus in more detail – what was the purpose of the red and grey striped pendants in the quipu below? They contain no knots, are they markers of some kind? This also seems to be a quipu where the knots do not follow the decimal coding pattern that we understand, they are mostly long knots of various values.

ur051

There also seems to be data stored in different kinds of structure in the same quipu – the collection of sub-pendants below in the left side presumably group data in a more hierarchical manner than the right side, which seems much more linear – and also a colour change emphasises this.

ur015

Read left to right, this long quipu below seems very much like you’d expect binary data to look – some kind of header information or preamble, followed by a repeating structure with local variation. The twelve groups of eight grey pendants seem redundant – were these meant to be filled in later? Did they represent something important without containing any knots? We will probably never know.

UR1176

The original thinking of the pixelquipu was to attempt to fit all the quipus on a single page for viewing, as it represents them with the absolute minimum pixels required. Here are both pendant colour and entropy shown for all 247 quipu we have the data for:

all

entropy-local

Quipu: further experiments in Düsseldorf

A report on further experimentation with Julian Rohrhuber and his students at the Institute for Music and Media in Düsseldorf during our coding with weaves and knots remote seminar this week.

skype

As we have so little idea what the Inca are telling us in their Quipu, it seems appropriate to add a cryptanalysis approach to our toolkit of inquiry. One of the first things that a cryptanalyst will do when inspecting an unknown system is to visualise it’s entropy in order to get a handle on any structures or patterns in the underlying information. This concept comes from Claude Shannon’s work on information theory in the 40’s, where he proved that information obeys fundamental laws of physics. The concept that information and “cyberspace” may not be as intangible and otherworldly as we might believe (in fact is grounded in physical reality along with everything else) is one of the recurring themes of the weavingcodes project.

Shannon’s innovation was to separate the concepts of data quantity from information value, and he claims that information is equivalent to surprise – the more surprising a piece of data is, the more information it contains. Conversely a piece of information which we expect to hear by definition doesn’t really tell us very much. The potential for some data to be surprising (or more specifically it’s potential to reduce our uncertainty) can be measured statistically, with a quantity he called entropy, as it is analogous to states in thermodynamic systems.

slide

Shannon defined a generalised communication system, which is handy to give us a way of reasoning about our situation in relation to the Inca. Our main unknown is the source of the messages they are sending us, are they accounting information, calendars or stories? We know a bit more about the transmitters of the messages, the khipukamayuq – the knot makers and quipu keepers. At the time Shannon was working on information theory, he was part of the start of the movement away from analogue, continuous signals and towards digital signals – with advantages that they are highly resistant to noise and can be carried further and combined together to increase bandwidth. Quipu are also mainly comprised of digital information – the type of a knot, the number of turns it’s comprised of or the twist direction of a thread are all discreet (either one thing or another) and therefore highly robust to material decay or decomposition. We can still ‘read’ them confidently after 500 years or more without the digital signal they represent being degraded too badly, if only we could understand it. At the same time, none of us working on this have access to a real quipu, so our receivers are the archaeologists and historians who study them, and compile archives such as the Harvard Quipu Archive we are using.

Although entropy is a very simplistic approach mathematically, it’s main use is to give us a tool for measuring the comparative information carrying potential of data which we have no idea about. Here are all the quipu in the Harvard database in order of average entropy bits they contain (only listing every other quipu ID):

entropy-per-quipu

This graph is calculated by making lists of all the discreet data of the same type, e.g. knot value, type, tying direction, pendant colours and ply direction (ignoring lengths and knot positions as these are continuous) – then calculating Shannon entropy on histograms for each one and adding them together.

We can also compare different types of information against one another, for example the main data we currently understand has some specific meaning are the knot values, partly derived from the knot type (long, single or figure of eight), which represent a decimal notation. If we compare the entropy of these we can expect them to have roughly similar average amounts of information:

entropy-values-types

The meanings of colours, ply and structure are largely unknown. Here are the knot values compared with the colours:

entropy-values-colours

And this is pendant ply direction compared with knot values for each quipu:

entropy-values-ply

At this point the most useful aspect of this work is to give us some outliers to inspect visually and sonically – more on that soon.

Coding with knots: Inca Quipu

This week I’m teaching at IMM Düsseldorf with Julian Rohrhuber which has given me a chance to follow up a bit on Inca Quipu coding with knots, a dangling thread from the weavecoding project. Quipu are how the Incas organised their society, as they had no written texts or money – things like exchanges (for example from their extensive store houses) were recorded via knots. Researchers have been able to decode the basic numeric system they used, but 20% of the quipu seem to follow a different set of rules, along with extra information encoded via thread material, twist direction, colour and other knot differences. I’ve written a python program for converting the Khipu Database Project excel charts into graphviz files for visualising:

quipu5

The knots are described in ascii art, with S and Z relating to the ply and knot ‘handedness’ direction they are tied in:

O : a single knot 
O/O : two single knots tied in S direction (it's rotated 90 degrees :)
(\\\\) : a long knot of value '4' tied in the Z direction
/8 : end (figure of 8) knot tied S direction

The pendant nodes also have labels describing their ply direction and the side the attach on, so “S R” is S ply & recto attached.

The hardest part of this has been a bit of more recent media archeology to figure out the colour values, I’ve had to cross reference the original Ascher-Ascher Quipu Databooks published in 1978 which contain their own colour system which more or less maps to the NBS-ISCC Munsell colour chart originally proposed in 1898. Luckily that site provides hex colour values – hopefully they are vaguely accurate, the current lookup table is here:

colour_lookup = {
    "W": "#777777",
    "SR":  "#BF2233",
    "MB" : "#673923",
    "GG" : "#575E4E",
    "KB" : "#35170C",
    "AB" : "#A86540",
    "HB" : "#5A3D30",
    "RL" : "#AA6651",
    "BG" : "#4A545C",
    "PG" : "#8D917A",
    "B" : "#7D512D",
    "0B" : "#64400F",
    "RM" : "#AB343A",
    "PR" : "#490005",
    "FR" : "#7F180D",
    "DB" : "#4D220E",
    "YB" : "#BB8B54",
    "MG" : "#817066",
    "GA" : "#503D33"
}

Loose threads from weavecoding

Midway through the weavecoding project and our researches have thrown up a whole load of topics that either don’t quite fit into our framework, or we simply won’t have time to pursue properly. Here are some of the tangents I’ve collected so far.

Coding with knots: Khipu

One of the cultures I’m increasingly interested in are the Incas. Their empire flourished to up to 37 million people, without the need of money or a written language. We know that some numeric information was stored using Khipu, a knot based recording system which was used in combination with black and white stones to read and calculate. Two thirds of the quipus we have are un-translated, and do not fit into the known numeric coding system – what information do they hold?

quipu

Harvard University provides a Khipu Database Project with many surviving examples documented – I’m hoping to run a workshop soon to look through some of this data in a variety of ways.

Tablet weaving NAND gates

Image2
Diagram thanks to Phiala’s String Page – the only place I’ve seen tablet weaving explained properly.

There are logic gates in tablet weaving logic. I haven’t fully figured this out yet, but I noticed modelling tablet weaving that you end up basically mapping the combinations of the weaving actions (such as turn direction) and colour as truth tables.

Top face colour based on top left/top right hole yarn in a single card and turn direction (clockwise/counter clockwise)

TL Yarn : TR Yarn : Turn : Top face colour
--------------------------------------------
Black   : Black   : CCW  : Black
Black   : Black   : CW   : Black
Black   : White   : CCW  : Black
Black   : White   : CW   : White
White   : Black   : CCW  : White
White   : Black   : CW   : Black
White   : White   : CCW  : White
White   : White   : CW   : White

Things get stranger when you include twist and combinations of actions with multiple cards. Would it be possible to compile high level programming languages into weaving instructions for carrying out computation? Perhaps this is what the untranslatable quipus are about?

Nintendo made a knitting machine

We could really do with some of these, unfortunately they never went beyond prototype stage.

nintendo

Asemic writing

Asemic writing is a post-literate written form with no semantic content. Miles Visman programs procedural asemic languages and hand weaves them. I think this may be an important connection to livecoding at some point.

aw001