We have a brand new citizen science project starting with the wild crickets research group at Exeter University! These researchers are examining how evolution works with insects in their natural environment, rather than in lab conditions. In order to do this they have hundreds of CCTV cameras set up recording the burrows of field crickets, resulting in many hundreds of hours of footage. This footage needs to be watched in order to determine the various events that make up the life story of the insects. Each individual it turns out has quite distinct characteristics, and we thought it would be fun to open up this process and make it into a citizen science project – partly to get some help and speed up the job, but also the vast quantity of material (hundreds of thousands of hours in total) has it’s own appeal – and it would be great to be able to use it for a creative project like this.
Here is an example of the footage, rather sped up – check out the frog and the sudden switch to daylight mode:
This is the first interface sketch – my plan is to focus on the individual insects and visualise the information coming together by displaying their characteristics, along with which players are their ‘biggest fans’, i.e the people who’ve put the most time into tagging them:
The video tagging interface itself is the focus of the first prototype I’m working on at the moment. I’ve got a database set up for storing relationships between crickets, their movies and events that people create based on a combination of django and popcorn.js. Below is the first attempt, the buttons add new events as the movie plays that get recorded on the database, and displayed on the timeline bar at the bottom. Currently all players can see all the events globally, so that’s one of the first things to figure out how to handle.
9,000 players, 20,000 games played and 400,000 tested egg patterns later we have over 30 generations complete on most of our artificial egg populations. The overall average egg difficulty has risen from about 0.4 seconds at the start to 2.5 seconds.
Thank you to everyone who contributed their time to playing the game! We spawned 4 brand new populations last week, and we’ll continue running the game for a while yet.
In the meantime, I’ve started working on ways to visualise the 500Mb of pattern generating code that we’ve evolved so far – here are all the eggs for one of the 20 populations, each row is a generation of 127 eggs starting at the top and ordered in fitness score from left to right:
This tree is perhaps more useful. The ancestor egg at the top is the first generation and you can see how mutations happen and successful variants get selected.
We’ve released our latest citizen science camouflage game Egglab! I’ve been reporting on this for a while here so it’s great to have it released in time for Easter – we’ve had coverage in the Economist, which is helping us recruit egg hunters and 165,000 eggs have been tested so far over the last 3 days. At time of writing we’ve turned over 13 generations starting with random pattern programs and evolving them with small mutations, testing them 5 times with different players and picking the best 50% each time.
Here is an image of some of the first generation of eggs:
And this shows how they’ve developed 13 generations later with the help of many thousands of players:
We can also click on an individual egg and see how it’s evolved over time:
And we see how on average the time taken to find eggs is changing:
Technically this project involves distributed pattern generation on people’s browsers using HTML5 Canvas, making it scalable. Load balancing what is done on the server over three machines and a Facebook enabled subgame – which I’ll use another blog post to explain.
I’m putting the final pieces together for the release of the all new Project Nightjar game (due in the run up to Easter, of course!) and the automatic pattern generation has been a focus right up to this stage. The challenge I like most about citizen science is that along with all the ‘normal’ game design creative restrictions (is it fun? will it work on the browser?) you also have to satisfy the fairly whopping constraints of the science itself, determining which decisions impact on the observations you are making – and being sure that they will be robust to peer review in the context of publication – I never had to worry about that with PlayStation games 🙂
With this game, similar to the last two, we want to analyse people’s ability to recognise types of pattern in a background image. Crucially, this is a completely different perception process from recognition of a learned pattern (a ‘search image’), so we don’t want to be generating the same exact egg each time from the same description – we don’t want people to ‘learn’ them. This also makes sense in the natural context of course, in that an individual bird’s eggs will not be identical, due to there being many many additional non-deterministic processes happening that create the pattern.
We’ve had tens of thousands of people spotting nightjars and donating a bit of their time to sensory ecology research. The results of this (of course it’s still on-going, along with the new nest spotting game) is a 20Mb database with hundreds of thousands of clicks recorded. One of the things we were interested in was seeing what people were mistaking for the birds – so I had a go at visualising all the clicks over the images (these are all parts of the cropped image – as it really doesn’t compress well):
Then, looking through the results – I saw a strange artefact:
My first thought was that someone had tried cheating with a script, but I can hardly imagine that anyone would go to the bother and it’s only in one image. Perhaps some form of bot or scraping software agent – I thought that browser click automation was done by directly interpreting the web page? Perhaps it’s a fall back for HTML5 canvas elements?
It turns out it’s a single player (playing as a monkey, age 16 to 35 who had played before) – so easy enough to filter away, but in doing that I noticed the click order was not as regular as it looked, and it goes a bit wobbly in the middle:
We’ve released a new game for Project Nightjar called Where is that nest? This is an adaptation of Where is that nightjar?, but the variety of species of birds is greater, some of the nests are much harder to find than the birds were so we added two levels – and a hall of fame high score table!
It’s a perception test to see how good you are at spotting the camouflaged birds – a great use of the photos the researchers are collecting in the field, and we can also use the data as an experiment by comparing our timing when searching for birds with different predator perception, Monkeys – who see the same colours we do, or Mongeese – who being dichromats can’t differentiate between red and green.
We also had a great chance to test the game very thoroughly at the Science in the Square event in Falmouth last week, set up by Exeter University to promote science to the public. We had a touchscreen computer set up that people could use, and had a large range of people hunting for nightjars (4155 attempted spotting “clicks” in total!).
Time to announce a new a new project with the Sensory Ecology and Evolution group at Exeter University. We’re going to be working on games and experimental online work to bring their research into the evolution of camouflage and animal perception to new audiences, particularly focused on these stealthy characters, the Fiery-necked Nightjar:
The group’s previous work includes all kinds of experiments on animal perception, this one is a favourite of mine: