The UAV toolkit & appropriate technology

The UAV toolkit’s second project phase is now complete, the first development sprint at the start of the year was a bit of research into what we could use an average phone’s sensors for, resulting in a proof of concept remote sensing android app that allowed you to visually program different scripts which we then tested on some drones, a radio controlled plane and a kite.

photo-1444743618-504214

This time we had a specific focus on environmental agencies, working with Katie Threadgill at the Westcountry Rivers Trust has meant we’ve had to think about how this could be used by real people in an actual setting (farm advisors working with local farmers). Making something cheap, open source and easy to use, yet open ended has been the focus – and we are now looking at providing WRT with a complete toolkit which would comprise a drone (for good weather) a kite (for bad weather/no flight licences required) and an android phone so they don’t need to worry about destroying their own if something goes wrong. Katie has produced this excellent guide on how the app works.

The idea of appropriate technology has become an important philosophy for projects we are developing at Foam Kernow, in conjunction with unlikely connections in livecoding and our wider arts practice. For example the Sonic Bike project – where from the start we restricted the technology so that no ‘cloud’ network connections are required and all the data and hardware required has to fit on the bike – with no data “leaking” out.

photo-1444742698-352249

With the UAV toolkit the open endedness of providing a visual programming system that works on a touchscreen results in an application that is flexible enough to be used in ways and places we can’t predict. For example in crisis situations, where power, networking or hardware is not available to set up remote sensing devices when you need them most. With the UAV toolkit we are working towards a self contained system, and what I’ve found interesting is how many interface and programming ‘guidelines’ I have to bend to make this possible – open endedness is very much against the grain of contemporary software design philosophy.

The “app ecosystem” is ultimately concerned with elevator pitches – to do one thing, and boil it down to the least actions possible to achieve it. This is not a problem in itself, but the assumption that this is the only philosophy worth consideration is wrong. One experience that comes to mind recently is having to make and upload banner images of an exact size to the Play Store before it would allow me to release an important fix needed for Mongoose 2000, which is only intended to ever have 5 or 6 users.

photo-1444743271-137365

For the UAV toolkit, our future plans include stitching together photos captured on the phone and producing a single large map without the need to use any other software on a laptop. There are also interesting possibilities regarding distributed networking with bluetooth and similar radio systems – for example sending code to different phones is needed, as currently there is no way to distribute scripts amongst users. This could also be a way of creating distributed processing – controlling one phone in a remote location with another via code sent by adhoc wifi or SMS for example.

Mongoose 2000 version 2

Mongoose 2000 version 2 is now being used in the Banded Mongoose Research Project Fieldsite on the Mweya Peninsula, in the Queen Elizabeth National Park, western Uganda.

We’ve added two new focal observations – where a single mongoose in a specific life stage is followed, and has it’s activity recorded for 20 minutes. These observations include different events that can happen (fighting or cooperating with other individuals etc). Nearly all the interfaces are shown below – the system includes adding new packs or individuals, data review and syncronisation with other tablets via the Raspberry Pi.

interfaces-s

Mongoose 2000 in the wilds of Uganda

One of our most ambitious projects: Mongoose 2000, is now up and running after 6 months of testing. This is a Raspberry Pi and Android tablet system to synchronise and store masses of data for a long running behavioral experiment recording the activities of packs of mongooses in the field site in Uganda. They broad aims of the project are to study mongoose behaviour in order to understand the evolution of society.

use

The timespan that we are working with is long, and the location – while not as remote as it could be (there is some internet access and power) required some consideration – so this was a project where we really needed to employ an appropriate technology approach which is manifested in various ways:

Open source software: means that Foam Kernow are not a bottle neck to continued development, as we do not have exclusive control over the source code (which is released into the commons). New developers can be found if required (for whatever reason) who do not need to start from nothing – this gives the research team more control and future proofing.

Use of commodity hardware: it’s likely that the hardware in use will become obsolete in this timeframe. The lifespan of android software should mean it can be installed on compatible devices for a long time, and the team can make use of advances in sensor or battery technology. The raspberry pi we are using is already an older model now, but as it’s a standard linux setup we can easily move it to other machines in the future. Currently it acts as an ‘appliance’ which just needs to be turned on, but we can add a web interface to control it from the tablet – or eventually replace it with a peer to peer syncronisation system.

The Ugandans working on the project have an healthy DIY relationship with technology, they expect to be able to repair or modify things themselves, and I’d like to figure out ways we can work with this more. The UAV toolkit project provides some indication of what can be done with programming these kinds of devices in the field. Part of the decisions on the hardware (and the design of the software, e.g. using a scheme interpreter) were to use devices that were open to a more end-user programming approach in the future.

use2

use3

Syncronising a tablet with observations previously recorded on the Raspberry Pi:

use4

Hindi translations in Symbai

A couple of screenshots of the hindi version of Symbai – our solar powered Raspberry Pi/Android anthropological research tool. As is usual we’re still having a few issues with the unicode but it’s nearly there. We’ve been working on this software for the last few months, making sure the data (including photos and audio recordings of verbal agreements) synchronise properly across multiple devices.

Screenshot_2014-09-15-10-45-14

Screenshot_2014-09-15-10-45-45

Mongoose 2000: Group composition

I’ve recently been building the Mongoose 2000 “group composition” tool that the researchers will use for recording information about a whole pack of mongooses (and synchronise data via a Raspberry Pi providing a local wifi node) in their field site in Uganda. As I wrote a bit about before, one of the interesting things about this project is that the interface design has to focus on long term speed and flexibility over immediate ease of use. In this way it seems appropriate that it’s moving in the direction of a musical interface rather than a normal touch screen interface. The different colours in the mongoose selectors show which individuals are present and which have data recorded from them already, the screenshot below is the section where they record relationships between the adult females (at the top) and adult males that may be guarding – or pestering them (below). At the same time, they need to be able to record events that may be occurring with the pack as a whole – in this case an interaction with another pack of mongeese.

Screenshot_2014-06-26-11-34-40

Solar powered computation

This is a Raspberry Pi running a Racket Servlet (with the Mongoose 2000 data sync server) serving a webpage to the tablet. The neat thing is that it’s running on solar power – a day of gloomy winter Cornish light (not even outside) charging the on board battery results in over an hour of running time. This includes the ad-hoc wifi transmitter, which is presumably the main power drain as the CPU usage is negligible.

This system, which I’ll be developing more in the coming months, will be heading to rural India as part of an exciting anthropological research project with Shakti Lamba involving the Aakash tablet and some interesting ubiquitous livecoding experiments.

IMG_20140223_111740

Mongoose 2000

A screen shot from the Mongoose 2000 project, we now have most of the ‘pup focal’ interfaces working and syncing their data via the Raspberry Pi. This is the interface for recording a pup aggression event – including the identity of the aggressive mongoose and some information on the cause and severity. Each mongoose has a code, and we’re using sliding toggle button interfaces for quickly picking them – these can be filtered to restrict them to adults, pups, males or females where required.

pupaggr

The interface was written using “starwisp” – my system for building android applications in Scheme. The Mongoose 2000 app has lots of reusable interfaces, so it’s mostly constructed from fragments. There are no specialised database tables, so I can simply add or modify the widgets here and the data automagically appears in the Raspberry Pi export, which makes it very fast to build. I’ve abstracted the mongoose button grid selectors and tristate buttons (yes/no/maybe) as they are used in a lot of places. Here is the entire definition of the fragment for the interface above, the code includes everything for creating and recording the database entity for this event and all the android callbacks it needs to respond to external events.

(fragment
   "ev-pupaggr"
   
   ;; define the interface layout first
   (linear-layout
    (make-id "") 'vertical fillwrap pf-col
    (list
     (mtitle "title" "Event: Pup aggression")
     (build-grid-selector "pf-pupaggr-partner" 
                          "single" "Aggressive mongoose")
     (linear-layout
      (make-id "") 'horizontal 
      (layout 'fill-parent 100 '1 'left 0) trans-col
      (list
       (vert
        (mtext "" "Fighting over")
        (spinner (make-id "pf-pupaggr-over") 
                 (list "Food" "Escort" "Nothing" "Other") fillwrap
                 (lambda (v)
                   (entity-add-value! "over" "varchar" v) '())))
       (vert
        (mtext "" "Level")
        (spinner (make-id "pf-pupaggr-level") 
                 (list "Block" "Snap" "Chase" "Push" "Fight") fillwrap
                 (lambda (v)
                   (entity-add-value! "level" "varchar" v) '())))
       (tri-state "pf-pupaggr-in" "Initiate?" "initiate")
       (tri-state "pf-pupaggr-win" "Win?" "win")))
     (spacer 20)
     (horiz
      (mbutton "pf-pupaggr-done" "Done"
        (lambda ()
          (entity-add-value! "parent" "varchar" 
            (get-current 'pup-focal-id ""))
            (entity-record-values db "stream" "pup-focal-pupaggr")
            (list (replace-fragment (get-id "event-holder") 
                                    "events"))))
      (mbutton "pf-pupaggr-cancel" "Cancel"
        (lambda ()
          (list (replace-fragment (get-id "event-holder") 
                                  "events")))))))

   ;; define the fragment's event callbacks 
   (lambda (fragment arg) ;; on create, return layout for building
     (activity-layout fragment))

   ;; on start - update contents from the db
   (lambda (fragment arg)  
     (entity-reset!)
     (list
      (populate-grid-selector
       "pf-pupaggr-partner" "single" ;; select single mongoose
       (db-mongooses-by-pack) #t     ;; from the whole pack 
       (lambda (individual)          ;; <- called when selected
         (entity-add-value! "id-with" "varchar" 
           (ktv-get individual "unique_id")) 
         (list)))
      ))

   (lambda (fragment) '()) ;; on stop
   (lambda (fragment) '()) ;; on resume
   (lambda (fragment) '()) ;; on pause
   (lambda (fragment) '())) ;; on destroy