Future Thinking for Social Living: Weavecoding in assisted housing

Our work on weavecoding is now reaching out to other uses and projects. One is Future Thinking for Social Living, run by Magda Tyżlik-Carver and Fiona Hackney.

This research project aims to look at the relationship between wellbeing, home, making and technology and is centred on Miners Court, who provide assisted housing in Redruth in Cornwall. As well as a range of flats and accommodation, the residents have shared communal areas with a variety of activities throughout the week. Along with Christiane Berghoff, Robin Hawes and Lucie Hernandez we set up camp with a lot of materials for knitting, crochet and weaving as well as some Raspberry Pis and the all new pattern matrix tangible weavecoding device.

miners

The Future Thinking for Social Living project is set up to research how we can think more critically about home and community, and with particular focus on the future. From discussions with the staff at Miners Court – specific issues they are interested in are how to make better use of communal spaces, and how can they get more men involved with crafts and shared activities.

I’m also interested in how we can use these settings for artists residencies – how does working with people like this affect a design process, does working in such a place – and using it as way to start conversations (rather than being too much in ‘teacher mode’) affect the people living there positively? Also the weavecoding project provides some ideas in bridging gaps, both between technology and people – but also across gender gaps, mixing textiles with electronics for example.

miners2
Here is the new magnetic pattern matrix, running the 3D Raspberry Pi warp weighted loom simulation (more on this soon!) with a nice 4 shaft loom in the background.

On Monday and Tuesday we spent a long time talking, weaving, knitting and making cups of tea of course (and a bit of time debugging magnets on my part). I’ve found helping people weave with tablets on the inkle loom is a good way to get talking, as this seems new to even people who are experienced with crafts. It also appeals to people with mathematics or design background who normally are uninterested in knitting and other crafts, and seems gender neutral perhaps for the same reasons. It also helps to talk about the history of what we are weaving with, the fact that this is an ancient technique and yet there are so many surprises – I can’t really predict to them what will happen e.g. to the pattern when we change rotation direction, and this seems to be important.

What we have yet to do (but a few weeks to experiment yet) is bridge the technology gap. Many of them have an immediate reaction of distaste to computers, as most of them have them but report that they have become unusable or feel that they are not designed well with their needs in mind. Partly the situation of having some circuit boards getting tangled up in the more familiar materials and using the Raspberry Pi simulation to show what is happening on the loom next to it is a start. One interesting thing is that neither the Pi nor the AVR boards look enough like ‘a computer’ for it to stand out too much (which also part of the Pi’s role in the classroom) – this was more so after plugging it into their large TV and getting rid of the monitor. As it gradually gets into a working state, I’d like to first try using it to demonstrate well known weaves – e.g. plain, twill and satin.

Working in this environment on the pattern matrix between weaving with different people has already had an effect on it’s design process. One initial observation resulted in reducing the magnet strength – I hadn’t even considered before that having them snap together too forcefully would be a problem for some people. Such things are obvious in these kinds of settings.

Screenless music livecoding

Programming music with flotsam – for the first time, it’s truly screen-less livecoding. All the synthesis is done on the Raspberry Pi too (raspbian release in the works). One of the surprising things I find with tangible programming is the enforced scarcity of tokens, having to move them around provides a situation that is good to play around with, in contrast to being able to simply type more stuff into a text document.

The programming language is pretty simple and similar to the yarn sequence language from the weavecoding project. The board layout consist of 4 rows of 8 possible tokens. Each row represents a single l-system rule:

Rule A: o o o o o o o o
Rule B: o o o o o o o o
Rule C: o o o o o o o o
Rule D: o o o o o o o o

The tokens themselves consist of 13 possible values:

a,b,c,d : The 'note on' triggers for 4 synth patches
. : Rest one beat
+,- : Change current pitch
<,> : Change current synth patch set
A,B,C,D : 'Jump to' rule (can be self-referential)
No-token: Ends the current rule

The setup currently runs to a maximum depth of 8 generations – so a rule referring to itself expands 8 times. A single rule ‘A’ such as ‘+ b A - c A ‘ expands to this sequence (the first 100 symbols of it anyway):

+b+b+b+b+b+b+b+b+b+b-c-c+b-c-c+b+b-c-c+b-c-c+b+b+b-c-c+b-c-c+b+b-c-c+b-c-c+b+b+b+b-c-c+b-c-c+b+b-c-c

I’m still working on how best to encode musical structures this way, as it needs a bit more expression – something to try is running them in parallel so you can have different sequences at the same time. With a bit more tweaking (and with upcoming hardware improvements) the eventual plan is to use this on some kid’s programming teaching projects.

New tangible weavecoding device – pattern matrix

We’re starting construction of version 2 of the flotsam tangible programming device, specialised to weaving – and henceforth known as the ‘pattern matrix’. This will be tested during May at our upcoming performance/workshop/residency at Munich’s Museum für Abgüsse Klassischer Bildwerke (Museum of Casts of Classical Sculpture) with the Coding weaves project, and then for later use in Cornwall (more on that part soon).

1

The first thing we are exploring is removing the need for physical plugs – although I like them a lot, they are problematic for people as it takes time to learn how to align the blocks in the current prototype. In order to get around this, and maintain the cheapness of the programming blocks themselves we’re looking at using magnetism to represent information. We can use blocks with no connections, painted white and black on different sides and detect their orientation and position via a magnet in the centre.

Initially this idea came from thinking about reed switches with Francesca, and playing with mobile phone magnetometers on the UAV project led to us investigating Hall effect sensors (the building blocks of magnetometers). We had a bit of a testing workshop with Andy from the Falmouth University makernow fablab who are helping with construction of this project.

halleffect

Hall effect sensors allow us to detect the polarity of nearby magnetic fields – and seem to be restricted enough in range that they can be very precise. Even with fairly weak magnets we found we could put the sensors right next to each other (see above) and still determine the difference between two opposed or aligned fields.

For the warp/weft weave pattern structure we only need 1 bit of information to be detected, but for future extensibility for the yarn colour programming setup it’s important to be able to read more (4 bits are encoded in the flotsam blocks).

Our plan is to try putting 4 sensors in a square which adds an intriguing possibility of rotating the blocks to change their meaning, as well as flipping them. The great thing is that this gets very close to tablet weaving in terms of the notation and the actions required. We can also represent all 16 states with only 4 blocks – if negative is 0 and positive is 1, and we read the code as binary clockwise from top left:

Starting state [0,1,5,6]
- -   + -   + -   - +
- -   - -   - +   - +

Rotate clockwise [0,2,10,12]
- -   - +   - +   - - 
- -   - -   + -   + +

Horizontal flip [15,11,10,12]
+ +   + +   - +   - - 
+ +   + -   + -   + +

Rotate counter-clockwise [15,13,5,6]
+ +   + -   + -   - + 
+ +   + +   - +   - +

Vertical flip [0,4,5,6]
- -   - -   + -   - + 
- -   - +   - +   - +

Here is Andy’s design for the PCB we’ll use under each of the 25 board locations:

hallboard

Tangible livecoding tests in the wild, and material as type in programming

Last week I took the flotsam tangible livecoding system to my programming tutoring lesson for some first tests with the real experts. To provide some background, we started a while back with Raspberry Pi, messing around with the Minecraft API and python and we’ve recently moved on to laptops and pygame. I arrived with the system set up for the Minecraft building language, and we gave it 10 minutes or so before resuming normal activities – although it did get a bit more use during natural breaks in the lesson. Here’s a recent pic of the weaving l-system setup:

tangibleweaves

First impressions were that it was immediately playful, most of the blocks were eagerly removed and examined before we’d turned the thing on. One problem with this was that the chalk symbols rubbed off very quickly! A similarity with the Scratch programming language was also picked up on fairly soon.

A big issue was getting the connectors the right way round – this is not easy as the blocks are circular with little indication of which way is ‘up’. This could be fixed by altering the shape or cutting a little notch – learning how to manipulate them is part of the point, but right now it’s too difficult. This will also be a big problem in livecoding performance situations.

Using Minecraft, the screen was still a distraction. The connection between what’s happening in the 3D world and with the physical blocks is not obvious enough – even with the LED indicators. Also the mouse and keyboard need to be plugged in so we can move the camera around and see stuff, leading to a few too many things going on. I’m not sure how this can be solved regarding Minecraft, but indicates that a musical approach (with no screen or any other peripherals needed other than speakers) is the way to go.

Overall there is huge potential to think much more about the touch/feel of the blocks. Lots of these problems can be solved by using different shapes for different symbols (removing the need for chalk) with a clear orientation. Using different materials to provide textures and ‘feel’ in order to represent different sounds, or in terms of weaving, using the actual yarn material to represent itself is a huge area to explore.

materials

In the photo above, I’m using thick and thin blue yarn wrapped around the blocks that represent them, and tinfoil for l-system rule symbols (X, Y and Z) – this is a kind of material based type indication. Interestingly the yarn feels very different even though it looks the same in the photo. In use this results in much less checking of the block when you pick them up, as your fingers tell you what they are.

dremelling

Learning about thinking and weaving in Leeds and Sheffield

The second week of intense work on weavingcodes/codingweaves took place in the north of England, and began with a talk at the New Mechanics Institute show and tell meeting in Leeds. This was a good place to pick up from the previous week in Denmark as it included a talk on pixel art from the early days to contemporary forms that had many similarities with our slides.

IMG_20141020_182924

The first half of the week was spent in Leeds University at the Interdisciplinary Centre for Scientific Research in Music where we met Tim Ingold who’s books (e.g. Making: Anthropology, Archaeology, Art and Architecture) have been a big influence on the project. One of the things we discussed with Tim was the cyberspace myth – the idea that computing is intangible and exists in some “other world” (or in a “cloud” somewhere) which is increasingly problematic. The need to highlight the physical basis of computation is one that this project is well placed to address – and if we consider weaving to be a form of computation, then it may be able to inspire approaches to programming that have at least three thousand years of history behind them. Tim also discussed knowledge as movement, his understanding of tacit knowledge – that which is otherwise unwritten or unspoken. Tacit knowledge is fluid and exists underneath a counterpart, called ‘articulated knowledge’ which is fixed by definitions. Culturally we have a bias towards articulated knowledge, and the idea of an unspoken knowledge is something which I realise I am trying to deal with in my teaching activities, but I’ve never been able to suitably describe it.

On day 2 we met with Leslie Downes – a retired professional weaver, who’s woven structures have gone into space, been used in jet engines and bullet proof vests.

Alex and Ellen talking with Leslie Downes IMG_20141021_094720

There are many places where woven structures are superior to other kinds of manufacturing, but at the same time there is a lack of knowledge about weaving technology in engineering in general. Industrial fabric weaving has focused on increasing speed and production output while the kind of looms needed for materials like carbon fibre need to focus on precision. Leslie talked of occasions where he had to return to using hand looms to try out prototypes and how simulations of fibre interactions in weaving exist, but are unnecessary and deficient compared with prototyping with the actual materials. In weaving, as in computing, older tends to mean less restricted, as well as slower. New developments add to, rather than eclipse older ones.

The main feeling I had from Leslie was that his approach to technology came from a combination of an ability to reason about material based on a deep understanding of the capabilities of the looms themselves, how they can be adapted for each job (e.g. a project where he had to cut down the middle of a huge loom to remove a metre or so) mixed with experience in working with people to understand their needs. This generalist approach to knowledge is so lacking in society that it becomes highly regarded.

For the second part of the week we moved to Sheffield, and had breakfast with Luigina Ciolfi researcher in Human-Centred Computing. Lui’s input was both valuable and timely, as she pointed out that one of the key elements of interest is that nature of our collaboration, rather than any products we may or may not come up with. As such we need to be careful that we take the time to document our process and decision making.

After breakfast we headed into the woods – to a temporary base in The J G Graves Woodland Discovery Centre to shift into plotting and planning mode.

In the woods looking at the kinds of weaves used in antiquity

Emma Cocker joined us again, and helped to restrengthen the kairotic and poetic roots of our thinking and we were visited by the lovebytes crew to discuss the use of weaving and livecoding for teaching children. I set up the flotsam tangible programming prototype for it’s first ever use by anyone other than me. This involved a bit of code review/pair programming with Ellen in order to fix bugs in the weave generating algorithm.

Ellen programming tangibly

We face challenges with our collaboration in both directions between our practices. On the one hand we have normal software decisions, is what we make going to be online or offline? Do we prioritise flexibility or accessibility for our choices in platforms and languages? How does what we do connect best with Ellen’s existing programming experience?

What we have decided is to work on a series of prototypes rather than focus on a monolithic development. The task then it so make sure we use common language for concepts across the project (perhaps using the concepts in the ancient greek as a grounding influence). Similarly, using protocols and formats to share common things between experiments – and fitting with existing programs and archives already in use where practical.

In the other direction, much of our discussion revolves around clarification of weaving practicalities and explanations – both of weaving in general and concepts and styles in use in antiquity. Both Alex and I need to start weaving if we are going to have a subtle enough understanding of the material we are dealing with. To this effect Alex already has a commission for a warp weighted loom in the works and I need to warp up the Harris loom.

We also have some fairly concrete projects to think about over the coming months, for example providing a good live-codable explanation of weaving to replace Ellen’s previous windows program as well as looking into the possibilities of using tangible programming in an art installation.

Future plans

Flotsam: A prototype screenless livecoding language

Two languages are working with Flotsam, the new name for the prototype screenless tangible programming language I’ve been building (which comes from the fact it’s largely made from driftwood). It’s somehow already been featured on the Adafruit blog!

The circuit seems to be fully debugged now, with short circuits fixed – which took a little while and more than a little frustration 🙂 The Raspberry Pi python code is currently on the weavingcodes repository (more on this project on the kairotic site), and the first language is a declarative style L system for describing weave structure and pattern with yarn width and colour. The LEDs indicate that the evaluation happens simultaneously, as this is a functional language. The blocks represent blue and pink yarn in two widths, with rules to produce the warp/weft sequence based on the rows the blocks are positioned on:

The weaving simulation is written in pygame (which I’ve been using lately for teaching), and is deliberately designed to make alternative weave structures than those possible with Jacquard looms by including yarn properties. The version in the video is plain weave, but more complex structures can be defined as below – in the same way as Alex’s gibber software:

star

This is a completely different language for building shapes in Minecraft, and is an imperative, stack based language for driving a turtle in 3D space. Eventually (when I’ve manufactured a few more programming blocks) it will be possible to change Minecraft block materials and react to player actions. The LEDs indicate here the more sequential evaluation of this Forth like language:

All that’s needed to switch languages is to redraw the symbols with chalk and run a different script. It won’t be truly screenless until I write a musical language for it, which is obviously coming soon…

IMG_20141016_142021

A screenless programming language: putting it all together

More background on this project here. This is the finished Raspberry Pi interface board, most of the circuitry here is an LED driver to light up one of the 32 programming blocks as they are being scanned, or in any pattern required depending on the needs of the particular programming language in use. The small IC is an old 74LS02 NOR gate I’m misusing as a NOT gate (which I couldn’t find in my supplies). I hard wired one of the gate’s inputs to ground so the other one gets inverted. I need this in order to switch between the two 16 way multiplexers to get 32 outputs, by alternating their enable pins.

IMG_20141002_084650

This time I remembered to include decoupling capacitors (thanks to Julian). Here is the completed wiring for the block input plugs, with everything connected up. I’m new to this scale of building, so really pretty happy it seems to be working:

IMG_20140927_155501

The same thing, the right way up with a single programming block plugged in, and the Raspberry Pi in the foreground. The first time it’s starting to look like something:

IMG_20140927_162218

I’m also starting to document the project, as it will be released as open source hardware, here’s an unfinished schematic. This is attempting to be complete, but it’s not really as complex as it looks – I’ll work on a block diagram that makes it easier to describe:

unfinished-circuit

Screenless programming language: assembling the plug panel

Never seen so many VGA ports, ironic perhaps. It’s great that it’s transparent, but I’m regretting the material choice for the panel – acrylic is horrible to work with and leaves behind lots of microplastics pollution I have to carefully collect up. Lesson learnt for next time, lots more soldering to do now – then more testing with the Raspberry Pi. Following on from the previous post, the instruction blocks will plug in to these 9 pin plugs.

IMG_20140924_150824

IMG_20140924_165531