hannah dee

BMVA workshop: plants in computer vision

On Wednesday I hosted my first ever British Machine Vision Association (BMVA) one-day workshop. The BMVA are the organisation which drives forwards computer vision in the UK, and they run a series of one-day technical meetings, usually in London, which are often very informative. In order to run one, you have to first propose it, and then the organisation work with you to pull together dates, program, bookings and so on. If you work in computer vision and haven’t been to one yet, you’re missing out.

I won’t write an overview of the whole day – that’s already been done very well by Geraint from GARNet the Arabidopsis research network. So if you want a really nice blow by blow account pop over to the GARNet blog.

We had some posters, and some talks, and some demos, and around 55 attendees. The quality was good – one of the best plant imaging workshops I have been to, with no dud talks. I think London is an attractive venue, the meetings are cheap (£30 for non-members, £10 for members), and both of these factors contributed. But I suspect the real reason we had such a strong meeting was that we’re becoming quite a strong field.

The questions and challenges that come up will be familiar to people who work in other applied imaging fields, like medical imaging :

  • should we use machine learning? (answer: probably)
  • can we trust expert judgments? (answer: maybe… but not unconditionally!)
  • we need to share data – how can we share data? what data can we share?
  • if we can’t automatically acquire measurements that people understand, can we acquire proxy measurements (things which aren’t the things that people are used to measuring, but which can serve the same purpose)?
  • can deep learning really do everything?
  • if we’re generating thousands of images a day, we have to be fully automatic. this means initialisation stages have to be eliminated somehow.

One of the presenters – Milan Sulc, from the Centre for Machine Perception in Prague – wanted to demo his plant identification app. Unfortunately, we discover that all of the plants at the BCS are plastic. Milan disappears to a nearby florists to get some real plants, at which point, the receptionist arrives with an orchid. Which also turns out to be plastic. The lesson here? Always remember to bring a spare plant.

This workshop was part funded by my EPSRC first grant, number EP/LO17253/1, which enabled me to bring two keynotes to the event and that was another real bonus for me. Hanno Scharr from Jülich and Sotos Tsaftaris from Edinburgh are both guys who I’ve wanted to chat with for some time, and they both gave frankly excellent presentations. It was also very good to catch up with Tony Pridmore and the rest of the Nottingham group; it’s been a while since I made a conference in computer vision / plant science, as I had a diary clash over IAMPS this year.

We’re hoping to put together a special issue of Plant Methods on the meeting.

First paper from first grant!

We’ve had our first journal paper published from my EPSRC first grant. It gives a comprehensive review of work into the automated image analysis of plants – well, one particular type of plant, Arabidopsis Thaliana. It’s by Jonathan Bell and myself, and it represents a lot of reading, talking and thinking about computer vision and plants. We also make some suggestions which we hope can help inform future work in this area. You can read the full paper here, if you’re interested in computer vision and plant science.

The first grant as a whole is looking at time-lapse photography of plants and aims to build sort-of 3d models representing growth. It’s coming to an end now so we’re wrapping up the details and publishing the work we’ve done. This means keen readers of this blog1 can expect quite a few more posts relating to the first grant soon: we’re going to release a dataset, a schools workshop, and we’ll be submitting another journal paper talking about the science rather than the background.

1Yes, both of you

HCC 2016: Human Centred Computing Summer School

Last week I was invited to present at the first Human Centred Cognition summer school, near Bremen in Germany. Summer schools are a key part of the postgraduate training experience, and involve gathering together experts to deliver graduate level training (lectures, tutorials and workshops) on a particular theme. I’ve been to summer schools before as a participant, but never as faculty. We’re at a crossroads in AI at the moment: there’s been a conflict between “good old fashioned AI” (based upon logic and the symbol manipulation paradigm) and non-symbolic or sub-symbolic AI (neural networks, probabilistic models, emergent systems) for as long as I have known, but right now with deep machine learning in the ascendant it’s come to a head again. The summer school provided us participants with a great opportunity to think about this and to talk about it with real experts.

Typically, a summer school is held somewhere out of the way, to encourage participants to talk to each other and to network. This was no different, so we gathered in a small hotel in a small village about 20 miles south of Bremen. The theme of this one was human centred computing – cognitive computing, AI, vision, and machine learning. The students were properly international (the furthest travelled students were from Australia, followed by Brazil) and were mostly following PhD or Masters programs; more than half were computing students of one kind or another, but a few psychologists, design students and architects came along too.

The view from my room

The bit I did was a workshop on OpenCV, and I decided to make it a hands-on workshop where students would get to code their own simple vision system. With the benefit of hindsight this was a bit ambitious, particularly as a BYOD (Bring Your Own Device) workshop. OpenCV is available for all main platforms, but it’s a pain to install, particularly on macs. I spent a lot of time on the content (you can find that here: http://github.com/handee/opencv-gettingstarted) and not so much time on thinking about infrastructure or thinking about installation. I think about half of the group got to the end of the tutorial, and another 5 or 10 managed to get someway along, but we lost a few to installation problems, and I suspect these were some of the less technical students. I used jupyter notebooks to create the course, which allow you to intersperse code with text about the code, and I think this may have created an extra layer of difficulty in installation, rather than a simplification of practicalities. If I were to do it again I’d either try and have a virtual machine that students could use or I’d run it in a room where I knew the software. I certainly won’t be running it again in a situation where we’re reliant on hotel wifi…

The class trying to download and install OpenCV

The school timetable involved a set of long talks – all an hour or more – mixed between keynote and lecture. The speakers were all experts in their field, and taken together the talks truly provided a masterclass in artificial intelligence, machine learning, the philosophical underpinnings of logic and robotics, intelligent robotics, vision, perception and attention. I really enjoyed sitting in the talks – some of the profs just came for their own sessions, but I chose to attend pretty much the whole school (I skipped the session immediately before mine, for a read through of my notes, and I missed a tutorial as I’d been sat down for too long and needed a little exercise).

It’s hard to pick a highlight as there were so many good talks. Daniel Levin from Vanderbilt (in Tennessee) gave an overview of research in attention and change blindness, showing how psychologists are homing in on the nature of selective attention and attentional blindness. I’ve shown Levin’s videos in my lectures before so it was a real treat to get to see him in person and have a chat. Stella Yu from Berkeley gave the mainstream computer vision perspective, from spectral clustering to deep machine learning. Ulrich Furbach from Koblenz presented a more traditional logical approach to AI, touching on computability and key topics like commonsense reasoning, and how to automate background knowledge.

One of my favourite presenters was David Vernon from Skövde. He provided a philosophical overview of cognitive systems and cognitive architectures: if we’re going to build AI we have a whole bunch of bits which have to fit together; we can either take our inspiration from human intelligence or we can make it up completely, but either way we need to think at the systems level. His talk gave us a clear overview of the space of architectures, and how they relate to each other. He was also very funny, and not afraid to make bold statements about where we are with respect to solving the problem of AI: “We’re not at relativistic physics. We’re not even Newtonian. We’re in the dark ages here“.

Prof Vernon giving us an overview of the conceptual space of cognitive architectures

When he stood up I thought to myself “he looks familiar” and it turns out I actually bought a copy of his book a couple of weeks ago. Guess I’d better read it now.

Kristian Kersting of TU Dortumund, and Michael Beetz of Bremen both presented work that bridges the gap between symbolic and subsymbolic reasoning; Kristian talked about logics and learning, through Markov Logic Networks and the like. Michael described a project in which they’d worked on getting a robot to understand recipes from wikihow, which involved learning concepts like “next to” and “behind“. Both these talks gave us examples of real-world AI that can solve the kinds of problems that traditional AI has had difficulty with; systems that bridge the gap between the symbolic and the subsymbolic. I particularly liked Prof. Beetz’s idea of bootstrapping robot learning using VR: by coding up a model of the workspace that the robot is going to be in, it’s possible to get people to act out the robot’s motions in a virtual world enabling the robot to learn from examples without real-world training.

Prof Kersting reminding us how we solve problems in AI

Each night the students gave a poster display showing their own work, apart from the one night we went into the big city for a conference dinner. Unfortunately the VR demo had real issues with the hotel wifi.

The venue for our conference dinner, a converted windmill in the centre of old Bremen

In all a very good week, which has really got me thinking. Big thanks to Mehul Bhatt of Bremen for inviting me out there. I’d certainly like to contribute to more events like this, if it’s possible; it was a real luxury to spend a full week with such bright students and such inspiring faculty. I alternated between going “Wow, this is so interesting, Isn’t it great that we’re making so much progress!” and “Oh no there is so much left to do!“.

Hotel Bootshaus from the river

The Aberystwyth Image Analysis Workshop AIAW

Last week (on Friday) we held the Aberystwyth Image Analysis workshop. I think it was the 3rd, or maybe the 4th one of these I’ve organised. The aim is to have some informal talks and posters centred around the theme of image analysis (including image processing, computer vision, and other image-related stuff) from across Aberystwyth. To encourage participation from people whether they’ve got results or not we have 10 minute slots for problem statements, short talks, work in progress and so on, and we have 20 minute slots for longer pieces of work. This year there were 4 departments represented in talks: Computer science, Maths, Physics and IBERS (biology), and we had speakers who were PhD students, post docs, lecturers and senior lecturers (no profs this year, boo!).

The range of topics covered was as usual very broad – images are used in research all the time, and it’s really useful to get together and see the kinds of techniques people are using. In Physics, they’re working on tightly and precisely calibrated cameras and instruments, using images as a form of measurement. In Maths the images are fitted to models and used to test theories about (for example) foam. In computer science people are working on cancer research using images, plant and crop growth modelling, and video summarisation (to name but a few of the topics). And the IBERS talk this year came from Lizzy Donkin, a PhD student who’s working on slugs.

Lizzy and I have been trying to track slugs so that she can model how they forage – she spoke for 10 minutes on the problem of slugs and imagery, and I spoke for 10 minutes on preliminary slug tracking results. Here’s a screenshot of my favourite slide, showing the wide range of shape deformations that a single slug can undergo.

Building a lightstage

A Lightstage is a system which lets you completely control illumination in a particular space, and capture images from multiple views. They’re used for high resolution graphics capture and computer vision, and they’re fairly rare. I don’t think there are many non-commercial ones in the UK, and they’re research kit (which means you can’t really just go out and buy one, you’ve got to actually build it). Usually, Lightstages are used for facial feature capture, but I’m kinda interested to use them with plants. With the support of the National Plant Phenomics Centre, here in Aberystwyth, and and Aberystwyth University Research Fund grant (URF) I’ve been slowly putting one together.

The key ingredient of a Lightstage is a frame which can hold the lights and the cameras equidistant from the target object. We’ve gone for a geodesic dome construction. Here’s a time-lapse video of Geoff from Geodomes.com building ours (a 2 metre 3v dome made out of rigid struts covered in non-reflectant paint). He has a bit of help from Alassane Seck, who did a PhD here in Aberystwyth on Lightstage imaging.

Once we’d got the dome, the next job was to think about mounting lights on the dome. There are a couple of different approaches we can take, but the essential features are that some of the lights are polarised and some of the cameras also have polarising filters. This means we can separate out specular reflections (light that bounces straight off) and diffuse reflections (light that interacts more with the surface of the object). Pete Scully‘s been working on the light placement, doing a lot of work in simulation. Here’s an early simulated placement: dots are lights, boxes are cameras.

The dome was housed in the Physical Sciences building but it’s recently moved. This puts us in a room which is actually light-tight, a key consideration for reducing interference in the controlled lighting situation. Here’s an arty shot of the top of the dome in its new home.

Since the move of room (very recently) things have really picked up. We’ve got a light-proof space, and we’ve got an intern from France (Robin Dousse) working with us too. Andy Starr‘s been working on the electronics and construction from the outset, and during breaks in teaching has really driven the project forwards. Here’s a shot of Robin, Pete and Andy by the dome:

Robin’s been working on programming our LED controllers. We’ve a fairly complicated master-slave controller system, as running 100 or so ultra-bright lights is not trivial. We’re aiming for a pair (one polarised, one not) at each vertex. Here’s a 12 second video of some flashing LEDs. It’s going to look a lot more impressive than this once it’s actually going, but hey, this is actual lights lighting up on our actual dome so I am very pleased.

We’ve now also, finally, got cameras on the dome. We’re not 100% certain about positioning, but we’re getting there. Andy’s been working on the camera triggers. Soon we’ll have something which flashes, takes pictures, and gives us the data we want.