Learning about thinking and weaving in Leeds and Sheffield

The second week of intense work on weavingcodes/codingweaves took place in the north of England, and began with a talk at the New Mechanics Institute show and tell meeting in Leeds. This was a good place to pick up from the previous week in Denmark as it included a talk on pixel art from the early days to contemporary forms that had many similarities with our slides.

IMG_20141020_182924

The first half of the week was spent in Leeds University at the Interdisciplinary Centre for Scientific Research in Music where we met Tim Ingold who’s books (e.g. Making: Anthropology, Archaeology, Art and Architecture) have been a big influence on the project. One of the things we discussed with Tim was the cyberspace myth – the idea that computing is intangible and exists in some “other world” (or in a “cloud” somewhere) which is increasingly problematic. The need to highlight the physical basis of computation is one that this project is well placed to address – and if we consider weaving to be a form of computation, then it may be able to inspire approaches to programming that have at least three thousand years of history behind them. Tim also discussed knowledge as movement, his understanding of tacit knowledge – that which is otherwise unwritten or unspoken. Tacit knowledge is fluid and exists underneath a counterpart, called ‘articulated knowledge’ which is fixed by definitions. Culturally we have a bias towards articulated knowledge, and the idea of an unspoken knowledge is something which I realise I am trying to deal with in my teaching activities, but I’ve never been able to suitably describe it.

On day 2 we met with Leslie Downes – a retired professional weaver, who’s woven structures have gone into space, been used in jet engines and bullet proof vests.

Alex and Ellen talking with Leslie Downes IMG_20141021_094720

There are many places where woven structures are superior to other kinds of manufacturing, but at the same time there is a lack of knowledge about weaving technology in engineering in general. Industrial fabric weaving has focused on increasing speed and production output while the kind of looms needed for materials like carbon fibre need to focus on precision. Leslie talked of occasions where he had to return to using hand looms to try out prototypes and how simulations of fibre interactions in weaving exist, but are unnecessary and deficient compared with prototyping with the actual materials. In weaving, as in computing, older tends to mean less restricted, as well as slower. New developments add to, rather than eclipse older ones.

The main feeling I had from Leslie was that his approach to technology came from a combination of an ability to reason about material based on a deep understanding of the capabilities of the looms themselves, how they can be adapted for each job (e.g. a project where he had to cut down the middle of a huge loom to remove a metre or so) mixed with experience in working with people to understand their needs. This generalist approach to knowledge is so lacking in society that it becomes highly regarded.

For the second part of the week we moved to Sheffield, and had breakfast with Luigina Ciolfi researcher in Human-Centred Computing. Lui’s input was both valuable and timely, as she pointed out that one of the key elements of interest is that nature of our collaboration, rather than any products we may or may not come up with. As such we need to be careful that we take the time to document our process and decision making.

After breakfast we headed into the woods – to a temporary base in The J G Graves Woodland Discovery Centre to shift into plotting and planning mode.

In the woods looking at the kinds of weaves used in antiquity

Emma Cocker joined us again, and helped to restrengthen the kairotic and poetic roots of our thinking and we were visited by the lovebytes crew to discuss the use of weaving and livecoding for teaching children. I set up the flotsam tangible programming prototype for it’s first ever use by anyone other than me. This involved a bit of code review/pair programming with Ellen in order to fix bugs in the weave generating algorithm.

Ellen programming tangibly

We face challenges with our collaboration in both directions between our practices. On the one hand we have normal software decisions, is what we make going to be online or offline? Do we prioritise flexibility or accessibility for our choices in platforms and languages? How does what we do connect best with Ellen’s existing programming experience?

What we have decided is to work on a series of prototypes rather than focus on a monolithic development. The task then it so make sure we use common language for concepts across the project (perhaps using the concepts in the ancient greek as a grounding influence). Similarly, using protocols and formats to share common things between experiments – and fitting with existing programs and archives already in use where practical.

In the other direction, much of our discussion revolves around clarification of weaving practicalities and explanations – both of weaving in general and concepts and styles in use in antiquity. Both Alex and I need to start weaving if we are going to have a subtle enough understanding of the material we are dealing with. To this effect Alex already has a commission for a warp weighted loom in the works and I need to warp up the Harris loom.

We also have some fairly concrete projects to think about over the coming months, for example providing a good live-codable explanation of weaving to replace Ellen’s previous windows program as well as looking into the possibilities of using tangible programming in an art installation.

Future plans

Flotsam: A prototype screenless livecoding language

Two languages are working with Flotsam, the new name for the prototype screenless tangible programming language I’ve been building (which comes from the fact it’s largely made from driftwood). It’s somehow already been featured on the Adafruit blog!

The circuit seems to be fully debugged now, with short circuits fixed – which took a little while and more than a little frustration 🙂 The Raspberry Pi python code is currently on the weavingcodes repository (more on this project on the kairotic site), and the first language is a declarative style L system for describing weave structure and pattern with yarn width and colour. The LEDs indicate that the evaluation happens simultaneously, as this is a functional language. The blocks represent blue and pink yarn in two widths, with rules to produce the warp/weft sequence based on the rows the blocks are positioned on:

The weaving simulation is written in pygame (which I’ve been using lately for teaching), and is deliberately designed to make alternative weave structures than those possible with Jacquard looms by including yarn properties. The version in the video is plain weave, but more complex structures can be defined as below – in the same way as Alex’s gibber software:

star

This is a completely different language for building shapes in Minecraft, and is an imperative, stack based language for driving a turtle in 3D space. Eventually (when I’ve manufactured a few more programming blocks) it will be possible to change Minecraft block materials and react to player actions. The LEDs indicate here the more sequential evaluation of this Forth like language:

All that’s needed to switch languages is to redraw the symbols with chalk and run a different script. It won’t be truly screenless until I write a musical language for it, which is obviously coming soon…

IMG_20141016_142021

Learning to read, notate and compute textiles in Aarhus

Setting off from Copenhagen, the weaving codes tour continued as Emma Cocker, Alex McLean, Ellen Harlizius-Klück and I sped across the Danish countryside on the train. We were heading for Aarhus to spend some time working with people at the other end of the technology spectrum – The Center for Participatory IT in Aarhus University.

We were invited by Geoff Cox to run a workshop over two days, and given the nature of the faculty and time working together being at a premium on this project, we decided to to run the workshop more as invitation to join in our work (and therefore learn a bit about how artistic research is done), rather than having specific material to cover.

Workshop participants working in Aarhus Lots of pieces of woven fabric

Alex and I needed to learn from Ellen how to ‘read’ a textile sample and notate it’s structure, a kind of reverse engineering process. Day one consisted of sharing out different types of weave and carefully figuring out the crossing points of warp and weft. The first challenge is to attempt to find the smallest repeat of the pattern, then record on graph paper a cross where the warp threads show over the weft threads. The big surprise is that the weave does not obviously relate to the visible pattern. It can also be hard to determine from a small sample which direction is warp and which is weft, so you can choose this arbitrarily if need be.

Alex recording warp crossings on graph paper

We could then compare different weaves, and also different notation styles that people used – little shorthand ways of describing large areas of plain weave for example. Much of the fabric came from Ellen’s Pepita Virus exhibition, and contained many different sizes of the “dog tooth” pattern. By comparing the notation we had all recorded for our different samples, we could tell that to scale up a pattern it’s not the case that you can also scale up the weave – the structure needs to change completely between 4 or 5 different types.

Example fabric and notation Example fabric and notation

Day two consisted of testing Alex’s Javascript loom/computer, by entering the weave structure and colour sequences and checking if the resulting patterns matched. By livecoding, we could also play with the patterns without the need to physically weave them, in order to gain a better insight into how changes affected the resulting pattern – and a better overall understanding of the weaving process.

Ellen pointing at code Two workshop participants programming patterns

To finish up, I talked about the patterns inherent in computation, based on my previous Z80 explorations to show how the pattern creating origins of computation are still present, and the role of programming languages as ways for people to come to shared understanding via notation rather than simply telling a machine what to do.

Unravelling technology in Copenhagen

Last week the weavingcodes/codingweaves project started with a trip to Denmark, our first stop was the Centre for Textiles Research in Copenhagen where we presented the project and gathered as much feedback as possible right at the beginning. The CTR was introduced to us by Eva Andersson Strand, and is an interdisciplinary centre which focuses on the relationships between textiles, environment and society.

IMG_20141006_150755

This long-view perspective of technology is critical for us, as we are dealing with a combination of thinking in the moment via livecoding and a history of technology dating back to the neolithic. This is a warp weighted loom, the focus of much of Ellen Harlizius-Klück’s research and the technology we are going to be using for the project.

IMG_20141007_114259959

Weights like this are widespread in the archaeological record for many cultures around the world, with the earliest ones around 5000 BC. Similarly – a post-it note including a handy cuneiform translation:

IMG_20141007_165613

Alex talked about livecoding as a backwards step, removing the interface – thinking about it as an unravelling of technology. His introduction to Algorave led to many connections later when Giovanni Fanfani described the abstract rhythmic patterns of Homeric rhapsodic poetry. These were performed by citizens, in a collaborative and somewhat improvised manner – the structures they form musically and in language are potentially of interest as they seem to echo the logic of weaving pattern.

IMG_20141006_143209

Ellen described her research into tacit knowledge of ancient Greek society – how weaving provided thinking styles and ordering concepts for the earliest forms of mathematics and science which is the basis for much of the weavingcodes project. One additional theme that has come up fairly consistently is cryptography – Flavia Carraro’s description of ‘The Grid in the decipherment of the Linear B writing system: a “paper-­‐loom”?’ was another addition to this area.

Emma Cocker talked about Peneolopeian time – constant weaving and unravelling as a subversive act, and the concept of the kairos, as a timely action – the name given to the point at which the weft is made when the warp ‘shed’ is provided, as well as a part of the warp weighted loom. Her input was to provide a broader view to our explorations (as coders, weavers and archaeologists all tend to get caught up in technical minutiae from time to time). From our discussions it was apparent that one of the strongest connections between livecoding and ‘weaving as thought’ is a subversion of a form of work that is considered by the dominant culture as entirely utilitarian.

Thinking outside of the screen (#1)

I’m starting a new exploratory project to build a screen-less programming language based on two needs:

  • A difficulty with teaching kids programming in my CodeClub where they become lost ‘in the screen’. It’s a challenge (for any of us really but for children particularly) to disengage and think differently – e.g. to draw a diagram to work something out or work as part of a team.
  • A problem with performing livecoding where a screen represents a spectacle, or even worse – a ‘school blackboard’ that as an audience we expect ourselves to have to understand.

I’ve mentioned this recently to a few people and it seems to resonate, particularly in regard to a certain mismatch of children’s ability to manipulate physical objects against their fluid touchscreen usage. So, with my mind on the ‘pictures under glass’ rant and taking betablocker as a starting point (and weaving code as one additional project this might link with), I’m building some prototype hardware to provide the Raspberry Pi with a kind of external physical memory that could comprise symbols made from carved wood or 3D printed shapes – while still describing the behaviour of real software. I also want to avoid computer vision for a more understandable ‘pluggable’ approach with less slightly faulty ‘magic’ going on.

Before getting too theoretical I wanted to build some stuff – a flexible prototype for figuring out what this sort of programming could be. The Raspberry Pi has 17 configurable I/O pins on it’s GPIO interface, so I can use 5 of them as an address lookup (for 32 memory locations to start with, expandable later) and 8 bits as input for code or data values at these locations.

The smart thing would be to use objects that identify themselves with a signal, using serial communication down a single wire with a standard protocol. The problem with this is that it would make potential ‘symbol objects’ themselves fairly complicated and costly – and I’d like to make it easy and cheap to make loads of them. For this reason I’m starting with a parallel approach where I can just solder across pins on a plug to form a simple 8 bit ID, and restrict the complexity to the reading hardware.

Testing the 74HC4067 16-channel analog mult iplexer/demultiplexer
Testing the 74HC4067 16-channel multiplexer/demultiplexer

I’ve got hold of a bunch of 74HC4067 multiplexers which allow you to select one signal from 16 inputs (or the other way around), using 4 bits – and stacking them up, one for each 8 bits X 16 memory locations. This was the furthest I could go without surface mount ICs (well out of my wonky soldering abilities).

Building the board, (with narrow 24 pin IC holders sliced down the middle). The input comes in via a common bus down the centre of the board.
Building the board, (with narrow IC holders hacked by slicing them down the middle). The input comes in via a common bus down the centre of the board.
Solder practice
Solder practice
Testing the first 4 bits on the breadboard
Testing the first 4 bits on the breadboard

Now 4 bits are working it’s harder to test with an LED – so next up is getting the Raspberry Pi attached.

Neural Network livecoding and retrofitting ZX Spectrum hardware

An experimental, and quite angry neural network livecoding synth (with an audio ‘weave’ visualisation) for the ZX Spectrum: source code and TZX file (for emulators). It’s a bit hard to make out in the video, but you can move around the 48 neurons and modify their synapses and trigger levels. There are two clock inputs and the audio output is the purple neuron at the bottom right. It allows recurrent loops as a form of memory, and some quite strange things are possible. The keys are:

  • w,d: move diagonally north west <-> south east
  • s,e: move diagonally south west <-> north east
  • t,y,g,h: toggle incoming synapse connections for the current neuron
  • space: change the ‘threshold’ of the current neuron (bit shifts left)

This audio should load up on a real ZX Spectrum:

One of the nice things about tech like this is that it’s easily hackable – this is a modification to the video output better explained here, but you can get a standard analogue video signal by connecting the internal feed directly to the plug and detaching the TV signal de-modulator with a tiny bit of soldering. Look at all those discrete components!

IMG_20140809_125815

Computation is weaving

With my mind on our upcoming AHRC weave/code project (and seeing as Alex has already started writing code) I thought I’d have a go at visualising how computers work in relation to pattern manipulation. These screenshots are from a ZX Spectrum where I’ve modified some library assembler code for higher level arithmetic to display the contents of 7 CPU z80 registers graphically between each instruction – time runs from top to bottom.

Most processors don’t actually have circuits for mathematics, they simply implement ‘add’ along with bitwise instructions for ‘and’, ‘or’, ‘not’, ‘xor’ and a handful of instructions for shifting the bits left and right. This is true even with modern CPU’s where the arithmetic instructions for multiply, divide etc are built with hidden ‘microcode’ routines. For this reason the underlying operation of a computer has more to do with patterns than it does with concepts such as language or even numbers as we normally think of them.

The simplest (and shortest) are multiply in 8 bits. In this function, the ‘a’ register contains one number and the ‘h’ register contains the other – at the end the ‘a’ register contains the result. In the first screenshot the numbers are fairly simple so it’s possible to see what’s going on (ie. in 1*1 the ‘a’ and ‘h’ registers both contain 00000001)
mul8

170 in 8 bits looks like ‘10101010’ so easy to see – here are some different ways of reaching the same answer:
mul8-170

16bit multiply operates over 2 registers – the first value is stored in ‘h’ and ‘l’ and the other is on the stack, but is loaded into ‘d’ and ‘e’ after a few instructions:
mul16

43690 is ‘1010101010101010’ so in the first run here we multiply it by one as a test:
mul16-2

Some 16 bit divides – these take a longer time to calculate, so a whole page for all the instructions involved, and I have no idea how this works:
div16

65535 is the largest value we can store, divide by itself to end up with 1:
div16-2

div16-3

The code for all this is here.

Splinterfields: Mathematickal Arts #3

A third and final update on the Mathematickal Arts Workshop – following the previous outline, some more detail on the process of going from software to textile using plain weave.

First create a program and choose the colours:

Axiom: O
Rule: O => : O O :
Repeat 5 generations

: = Green
0 = Orange

Then “compile” the code to expand the pattern:

1= O
2= : O O :
3= : : O O : : O O : :
4= : : : O O : : O O : : : : O O : : O O : : :
5= : : : : O O : : O O : : : : O O : : O O : : : : : : O O : : O O : : : : O O : : O O : : : :

The resulting weave can be previewed in ASCII as described earlier. This was useful to discover the range of patterns possible. When done, install the compiled code on the loom as a warp thread colour sequence.

Then you follow the same sequence for the weft threads, choosing the colour based on the code generated. In this way you become a mere part of the process yourself.

This is another, more complex program with 2 rules. This expands rather quicker than the last one, so only three generations are required:

Axiom: O
Rule 1: O => O : O :
Rule 2: : => : O :
Run for 3 generations

This technique draws comparisons with Jacquard looms, but obviously it’s far simpler as the weave itself is the same, we are only changing between 2 colours (and a human is very much required to do the weaving in this case). However, one of the activities I would have tried with more time available would have been reverse engineering Jacquard woven fabric – to attempt to decode the rules used.

During the workshop it was also suggested that a woven quine may be possible – where the pattern somehow contains the instructions for it’s own manufacture.

Splinterfields: Mathematickal Arts #2

For my contribution to the Mathematickal Arts workshop, I wanted to explore weaving, specifically plain weave. This is the simplest form of weaving, but when combined with sequences of colour it can produce many different types of pattern.

Some of these patterns when combined with muted colours, have in the past been used as a type of camouflage – and are classified into District Checks for use in hunting in Lowland Scotland. This is, I guess, a kind of less prestigious form of Tartan.

I started off by trying to understand how the patterns emerge, beginning with the basic structure:

The threads running top to bottom are the warp, those running across are the weft. If we consider the top most thread as visible, we can figure out the colours of this small section of weave. A few lines of scheme calculate and print the colours of an arbitrarily sized weave, by using lists of warp and weft as input.

; return warp or weft, dependant on the position
(define (stitch x y warp weft)
  (if (eq? (modulo x 2)
           (modulo y 2))
      warp weft))

; prints out a weaving
(define (weave warp weft)
  (for ((x (in-range 0 (length weft))))
       (for ((y (in-range 0 (length warp))))
            (display (stitch x y 
                             (list-ref warp y)
                             (list-ref weft x))))
       (newline)))

I’ve been visualising the weaves with single characters representing colours for ascii previewing, here are some examples:

(weave '(O O O O O O O) '(: : : : : : : : :))
=>
 O : O : O : O
 : O : O : O :
 O : O : O : O
 : O : O : O :
 O : O : O : O
 : O : O : O :
 O : O : O : O
 : O : O : O :
 O : O : O : O

(weave '(O O : : O O : : O O) '(O : : O O : : O O :))
 =>
 : O : : : O : : : O
 O : : : O : : : O :
 O O O : O O O : O O
 O O : O O O : O O O
 : O : : : O : : : O
 O : : : O : : : O :
 O O O : O O O : O O
 O O : O O O : O O O
 : O : : : O : : : O

This looked quite promising as ascii art, but I didn’t really know how it would translate into a textile. I also wanted to look into ways of generating new patterns algorithmically, using formal grammars – this was actually one of the simpler parts of the project. The idea is that you begin with an axiom, or starting state, and do a search replace on it repeatedly following one or more simple rules:

Axiom: O
Rule: O -> :O:O

Generation 1: O
Generation 2: :O:O
Generation 3: ::O:O::O:O 
Generation 4: :::O:O::O:O:::O:O::O:O

We can then generate quite complex patterns for the warp and the weft from very small instructions, next I’ll show what happens when some of these are translated into real woven fabric…