hannah dee

Thinking and learning about play

I’ve just finished a MOOC (massive open online course) on play, with Futurelearn and the University of Sheffield: Exploring Play. Ideas about play have been coming up quite a bit in my work in the last few years – both in teaching (gamification, exploration) and in research (particularly in the research I’ve been doing into kids and coding). But I didn’t really know much about theoretical or practical ideas of play, particularly not outside of computing, so I signed up for a MOOC to take the broader look.

I found that earlier on in this course, the readings about play types enriched my conception of what play could be. Thinking about play in terms of taxonomies of play (rough and tumble play, imaginative play, etc.) has helped me break down what we mean when we say play, and I have found it useful to think about the things children do in terms of these taxonomies. I even found myself wondering how many different kinds of play particular activities or equipment affording, and wondering if I could alter activities to include more variety in play type. There’s an implicit assumption here, which is that play is good, and lots of types of play is better than one type of play. This has clear implications for the kinds of work we do with schoolkids (and to a lesser extent with uni students); a lot of our activities have time for exploratory play (“what does this do”). Thinking about play types leads us to try and incorporate other types of play e.g. creative play (“what can i make this do”, “what can I make with this”), mastery play (“can I get better at playing with this”), communication play (“can I use this to communicate?”). This brings more variety into the activity which may well end up with deeper learning.

The course was very broad, which I liked – I did it to get the big picture, and for that it really succeeded. We looked at cross-cultural play, online play, the spaces we play in, historical attitudes to play, disability and play…

Thinking about how those with disabilities can access play turned a lot of my ideas upside down: thinking in terms of play as activities with particular values leads to a normative understanding of play. The taxonomies provide a rich conception of what play could be, but they don’t dictate what it should be. Reading case studies about play and disability showed me that this normative conception (play should be educational, for example), doesn’t have to hold. Play doesn’t have to be something that lets children rehearse ideas for the real world. It can be something for itself – catching a ball repeatedly, fidget spinners, and other repetitive actions can all be playful in some way. These could be seen as mastery play, developing close motor skills, or they could just be play. Allowing people the time, space, and equipment to explore play with whatever actions they are able is something we need to do for our children and for ourselves.

In all I enjoyed the MOOC a lot – I think it will take a while for the ideas to settle in my mind as we touched briefly on a lot of different topics, but I also think that some of the things I’ve learned will be put in to action in my teaching and outreach activities pretty soon.

BCSWomen AI Accelerator

BCSWomen Chair Sarah Burnett has had a fab idea, which is to hold a series of webinars that talk about AI and how it is changing the world. In BCSWomen we do a lot of stuff about the women, and a lot of stuff to support women, but we also do a lot of stuff that is useful for tech people in general. The AI Accelerator falls into this category; the idea is that tech is changing and AI is driving that change, so we’re going to try and provide a background and overview of AI to help people get to grips with this. Once I heard the idea I had to put my hand up for a talk, and I grabbed the first slot general intro talk – “What is AI?“. The other speaker in the session was Andrew Anderson of Celaton, who talked about the business side of AI. If you want to join in follow @bcswomen on twitter and I’m sure they’ll tweet about the next one soon.

the talk

As ever I went a bit over the top on the talk prep, but managed to come up with a theme and 45 slides with a bunch of reveals/animations that I thought covered some key concepts quite well. (Yes 45 slides for 20 minutes is a bit much but hey, I rehearsed the timings down to a tee and it was OK.) The live webinar had a few issues with audio, so I re-recorded my talk as a stand-alone youtube presentation; it’s not as good as the original outing (as a bit of time had passed and I hadn’t rehearsed as much) but I think it still works OK. If you want to watch it, here it is:

You can find the slides online here: AI Accelerator talk slides. I am 99% certain that all the images I used were either free for reuse, or created by me, but if it turns out I’ve used a copyright image let me know and I’ll replace it.

the reasoning behind the talk

I’ve been “doing” AI since I first went to uni in 1993, and what people mean when they say AI has changed massively over this time. Things that I read about as science fiction are now everyday, and a lot of this is down to advances in machine learning (ML). So when I started working on the talk I actually asked myself “What do people really mean, when they say AI?”; it turns out that a lot of the time they’re actually talking ML. There are a lot of other questions that need to be raised (if not answered) – the difference between weak AI and strong AI, the concept of embodiment, the way in which some things which we think of as hard (e.g. chess) turned out to be quite easy, and some things we thought would be easy (e.g. vision) turned out to be quite hard. Hopefully in the talk I covered enough of this stuff to introduce the questions.

I decided that for a tech talk there needed to be a bit of tech in it too though, which is why I spent the second half breaking down a bit what we mean by machine learning, and introducing some different subtypes of machine learning. I expect that if you work in the area there’s nothing much new in the talk, but hopefully it gives an overview, and also gives enough depth for people to learn something from it.

so what about the cute robots?

I wanted a visual example for my slides on ML and particularly classification, so I created a robot image, then edited it about a bit to get 16 different variants (different buttons, different numbers of legs, aerials, arm positions). I then wrote a short program to switch the colours around so I got twice as many (just switching the blue and the red channels gives some cyan robots and some yellow robots).

If you want to use them in talks or whatever, feel free. You can get all 32 of the robots, here, along with the python program that switches colours and the gimp file (.xcf) if you want to edit them yourself.

BMVA workshop: plants in computer vision

On Wednesday I hosted my first ever British Machine Vision Association (BMVA) one-day workshop. The BMVA are the organisation which drives forwards computer vision in the UK, and they run a series of one-day technical meetings, usually in London, which are often very informative. In order to run one, you have to first propose it, and then the organisation work with you to pull together dates, program, bookings and so on. If you work in computer vision and haven’t been to one yet, you’re missing out.

I won’t write an overview of the whole day – that’s already been done very well by Geraint from GARNet the Arabidopsis research network. So if you want a really nice blow by blow account pop over to the GARNet blog.

We had some posters, and some talks, and some demos, and around 55 attendees. The quality was good – one of the best plant imaging workshops I have been to, with no dud talks. I think London is an attractive venue, the meetings are cheap (£30 for non-members, £10 for members), and both of these factors contributed. But I suspect the real reason we had such a strong meeting was that we’re becoming quite a strong field.

The questions and challenges that come up will be familiar to people who work in other applied imaging fields, like medical imaging :

  • should we use machine learning? (answer: probably)
  • can we trust expert judgments? (answer: maybe… but not unconditionally!)
  • we need to share data – how can we share data? what data can we share?
  • if we can’t automatically acquire measurements that people understand, can we acquire proxy measurements (things which aren’t the things that people are used to measuring, but which can serve the same purpose)?
  • can deep learning really do everything?
  • if we’re generating thousands of images a day, we have to be fully automatic. this means initialisation stages have to be eliminated somehow.

One of the presenters – Milan Sulc, from the Centre for Machine Perception in Prague – wanted to demo his plant identification app. Unfortunately, we discover that all of the plants at the BCS are plastic. Milan disappears to a nearby florists to get some real plants, at which point, the receptionist arrives with an orchid. Which also turns out to be plastic. The lesson here? Always remember to bring a spare plant.

This workshop was part funded by my EPSRC first grant, number EP/LO17253/1, which enabled me to bring two keynotes to the event and that was another real bonus for me. Hanno Scharr from Jülich and Sotos Tsaftaris from Edinburgh are both guys who I’ve wanted to chat with for some time, and they both gave frankly excellent presentations. It was also very good to catch up with Tony Pridmore and the rest of the Nottingham group; it’s been a while since I made a conference in computer vision / plant science, as I had a diary clash over IAMPS this year.

We’re hoping to put together a special issue of Plant Methods on the meeting.

First paper from first grant!

We’ve had our first journal paper published from my EPSRC first grant. It gives a comprehensive review of work into the automated image analysis of plants – well, one particular type of plant, Arabidopsis Thaliana. It’s by Jonathan Bell and myself, and it represents a lot of reading, talking and thinking about computer vision and plants. We also make some suggestions which we hope can help inform future work in this area. You can read the full paper here, if you’re interested in computer vision and plant science.

The first grant as a whole is looking at time-lapse photography of plants and aims to build sort-of 3d models representing growth. It’s coming to an end now so we’re wrapping up the details and publishing the work we’ve done. This means keen readers of this blog1 can expect quite a few more posts relating to the first grant soon: we’re going to release a dataset, a schools workshop, and we’ll be submitting another journal paper talking about the science rather than the background.

1Yes, both of you

HCC 2016: Human Centred Computing Summer School

Last week I was invited to present at the first Human Centred Cognition summer school, near Bremen in Germany. Summer schools are a key part of the postgraduate training experience, and involve gathering together experts to deliver graduate level training (lectures, tutorials and workshops) on a particular theme. I’ve been to summer schools before as a participant, but never as faculty. We’re at a crossroads in AI at the moment: there’s been a conflict between “good old fashioned AI” (based upon logic and the symbol manipulation paradigm) and non-symbolic or sub-symbolic AI (neural networks, probabilistic models, emergent systems) for as long as I have known, but right now with deep machine learning in the ascendant it’s come to a head again. The summer school provided us participants with a great opportunity to think about this and to talk about it with real experts.

Typically, a summer school is held somewhere out of the way, to encourage participants to talk to each other and to network. This was no different, so we gathered in a small hotel in a small village about 20 miles south of Bremen. The theme of this one was human centred computing – cognitive computing, AI, vision, and machine learning. The students were properly international (the furthest travelled students were from Australia, followed by Brazil) and were mostly following PhD or Masters programs; more than half were computing students of one kind or another, but a few psychologists, design students and architects came along too.

The view from my room

The bit I did was a workshop on OpenCV, and I decided to make it a hands-on workshop where students would get to code their own simple vision system. With the benefit of hindsight this was a bit ambitious, particularly as a BYOD (Bring Your Own Device) workshop. OpenCV is available for all main platforms, but it’s a pain to install, particularly on macs. I spent a lot of time on the content (you can find that here: http://github.com/handee/opencv-gettingstarted) and not so much time on thinking about infrastructure or thinking about installation. I think about half of the group got to the end of the tutorial, and another 5 or 10 managed to get someway along, but we lost a few to installation problems, and I suspect these were some of the less technical students. I used jupyter notebooks to create the course, which allow you to intersperse code with text about the code, and I think this may have created an extra layer of difficulty in installation, rather than a simplification of practicalities. If I were to do it again I’d either try and have a virtual machine that students could use or I’d run it in a room where I knew the software. I certainly won’t be running it again in a situation where we’re reliant on hotel wifi…

The class trying to download and install OpenCV

The school timetable involved a set of long talks – all an hour or more – mixed between keynote and lecture. The speakers were all experts in their field, and taken together the talks truly provided a masterclass in artificial intelligence, machine learning, the philosophical underpinnings of logic and robotics, intelligent robotics, vision, perception and attention. I really enjoyed sitting in the talks – some of the profs just came for their own sessions, but I chose to attend pretty much the whole school (I skipped the session immediately before mine, for a read through of my notes, and I missed a tutorial as I’d been sat down for too long and needed a little exercise).

It’s hard to pick a highlight as there were so many good talks. Daniel Levin from Vanderbilt (in Tennessee) gave an overview of research in attention and change blindness, showing how psychologists are homing in on the nature of selective attention and attentional blindness. I’ve shown Levin’s videos in my lectures before so it was a real treat to get to see him in person and have a chat. Stella Yu from Berkeley gave the mainstream computer vision perspective, from spectral clustering to deep machine learning. Ulrich Furbach from Koblenz presented a more traditional logical approach to AI, touching on computability and key topics like commonsense reasoning, and how to automate background knowledge.

One of my favourite presenters was David Vernon from Skövde. He provided a philosophical overview of cognitive systems and cognitive architectures: if we’re going to build AI we have a whole bunch of bits which have to fit together; we can either take our inspiration from human intelligence or we can make it up completely, but either way we need to think at the systems level. His talk gave us a clear overview of the space of architectures, and how they relate to each other. He was also very funny, and not afraid to make bold statements about where we are with respect to solving the problem of AI: “We’re not at relativistic physics. We’re not even Newtonian. We’re in the dark ages here“.

Prof Vernon giving us an overview of the conceptual space of cognitive architectures

When he stood up I thought to myself “he looks familiar” and it turns out I actually bought a copy of his book a couple of weeks ago. Guess I’d better read it now.

Kristian Kersting of TU Dortumund, and Michael Beetz of Bremen both presented work that bridges the gap between symbolic and subsymbolic reasoning; Kristian talked about logics and learning, through Markov Logic Networks and the like. Michael described a project in which they’d worked on getting a robot to understand recipes from wikihow, which involved learning concepts like “next to” and “behind“. Both these talks gave us examples of real-world AI that can solve the kinds of problems that traditional AI has had difficulty with; systems that bridge the gap between the symbolic and the subsymbolic. I particularly liked Prof. Beetz’s idea of bootstrapping robot learning using VR: by coding up a model of the workspace that the robot is going to be in, it’s possible to get people to act out the robot’s motions in a virtual world enabling the robot to learn from examples without real-world training.

Prof Kersting reminding us how we solve problems in AI

Each night the students gave a poster display showing their own work, apart from the one night we went into the big city for a conference dinner. Unfortunately the VR demo had real issues with the hotel wifi.

The venue for our conference dinner, a converted windmill in the centre of old Bremen

In all a very good week, which has really got me thinking. Big thanks to Mehul Bhatt of Bremen for inviting me out there. I’d certainly like to contribute to more events like this, if it’s possible; it was a real luxury to spend a full week with such bright students and such inspiring faculty. I alternated between going “Wow, this is so interesting, Isn’t it great that we’re making so much progress!” and “Oh no there is so much left to do!“.

Hotel Bootshaus from the river

The Aberystwyth Image Analysis Workshop AIAW

Last week (on Friday) we held the Aberystwyth Image Analysis workshop. I think it was the 3rd, or maybe the 4th one of these I’ve organised. The aim is to have some informal talks and posters centred around the theme of image analysis (including image processing, computer vision, and other image-related stuff) from across Aberystwyth. To encourage participation from people whether they’ve got results or not we have 10 minute slots for problem statements, short talks, work in progress and so on, and we have 20 minute slots for longer pieces of work. This year there were 4 departments represented in talks: Computer science, Maths, Physics and IBERS (biology), and we had speakers who were PhD students, post docs, lecturers and senior lecturers (no profs this year, boo!).

The range of topics covered was as usual very broad – images are used in research all the time, and it’s really useful to get together and see the kinds of techniques people are using. In Physics, they’re working on tightly and precisely calibrated cameras and instruments, using images as a form of measurement. In Maths the images are fitted to models and used to test theories about (for example) foam. In computer science people are working on cancer research using images, plant and crop growth modelling, and video summarisation (to name but a few of the topics). And the IBERS talk this year came from Lizzy Donkin, a PhD student who’s working on slugs.

Lizzy and I have been trying to track slugs so that she can model how they forage – she spoke for 10 minutes on the problem of slugs and imagery, and I spoke for 10 minutes on preliminary slug tracking results. Here’s a screenshot of my favourite slide, showing the wide range of shape deformations that a single slug can undergo.

Building a lightstage

A Lightstage is a system which lets you completely control illumination in a particular space, and capture images from multiple views. They’re used for high resolution graphics capture and computer vision, and they’re fairly rare. I don’t think there are many non-commercial ones in the UK, and they’re research kit (which means you can’t really just go out and buy one, you’ve got to actually build it). Usually, Lightstages are used for facial feature capture, but I’m kinda interested to use them with plants. With the support of the National Plant Phenomics Centre, here in Aberystwyth, and and Aberystwyth University Research Fund grant (URF) I’ve been slowly putting one together.

The key ingredient of a Lightstage is a frame which can hold the lights and the cameras equidistant from the target object. We’ve gone for a geodesic dome construction. Here’s a time-lapse video of Geoff from Geodomes.com building ours (a 2 metre 3v dome made out of rigid struts covered in non-reflectant paint). He has a bit of help from Alassane Seck, who did a PhD here in Aberystwyth on Lightstage imaging.

Once we’d got the dome, the next job was to think about mounting lights on the dome. There are a couple of different approaches we can take, but the essential features are that some of the lights are polarised and some of the cameras also have polarising filters. This means we can separate out specular reflections (light that bounces straight off) and diffuse reflections (light that interacts more with the surface of the object). Pete Scully‘s been working on the light placement, doing a lot of work in simulation. Here’s an early simulated placement: dots are lights, boxes are cameras.

The dome was housed in the Physical Sciences building but it’s recently moved. This puts us in a room which is actually light-tight, a key consideration for reducing interference in the controlled lighting situation. Here’s an arty shot of the top of the dome in its new home.

Since the move of room (very recently) things have really picked up. We’ve got a light-proof space, and we’ve got an intern from France (Robin Dousse) working with us too. Andy Starr‘s been working on the electronics and construction from the outset, and during breaks in teaching has really driven the project forwards. Here’s a shot of Robin, Pete and Andy by the dome:

Robin’s been working on programming our LED controllers. We’ve a fairly complicated master-slave controller system, as running 100 or so ultra-bright lights is not trivial. We’re aiming for a pair (one polarised, one not) at each vertex. Here’s a 12 second video of some flashing LEDs. It’s going to look a lot more impressive than this once it’s actually going, but hey, this is actual lights lighting up on our actual dome so I am very pleased.

We’ve now also, finally, got cameras on the dome. We’re not 100% certain about positioning, but we’re getting there. Andy’s been working on the camera triggers. Soon we’ll have something which flashes, takes pictures, and gives us the data we want.