User avatar
Hove
Posts: 1202
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Motion detection using RaspiCam

Fri May 16, 2014 8:50 am

I need to detect short term horizontal motion for my quadcopter. I'm currently doing that with integrating the accelerometer which works to some extent, but I wondered whether sticking a downward facing RaspiCam under my quad could be used to track motion combined with direction? I'm not using this for long term path tracking - GPS provides a much better solution not relying on integration of the result, just purely to detect (and ultimately correct) movement detected short term. I'm sure it can be done using sequential photographs checking to match up differences, but this sounds like a complex, image processing heavy task unless the GPU is used.

Any pointers to anything vaguely related would be hugely appreciated.

Thanks,

Andy
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

User avatar
Hove
Posts: 1202
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Motion detection using RaspiCam

Fri May 16, 2014 9:28 am

OK, I remember where I'd seen it before - the Halloween magic mirror where the skeleton picture tracks the movement of people past it's camera. Time to dig a little further.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

mikerr
Posts: 2706
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Motion detection using RaspiCam

Fri May 16, 2014 9:54 am

That's not going to work well, as visual motion detecting generally relies on a static scene, with objects moving in it.

On a quadcoptor, the whole scene is moving !
Android app - Raspi Card Imager - download and image SD cards - No PC required !

User avatar
Hove
Posts: 1202
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Motion detection using RaspiCam

Fri May 16, 2014 10:17 am

My mention of the skull was a bad comparison.

It has been done commercially: http://copter.ardupilot.com/wiki/optical-flow-sensor/ and http://pixhawk.org/modules/px4flow

But the latter is > £100, and then it dawned on me the RPi might be able to do it by itself:

A little more thought suggests the following method:
  • take a downward facing RaspiCam, taking snaps as fast as possible
  • align the last two images for pitch / roll / altitude
  • apply high contast B&W filter
  • do image pattern matching to work out orientation, direction of movement etc
  • do as much of the above using the GPU allowing the CPU to manage the quad flight control
  • feed the result back into the quad code instead of the integrated accelerometer readings leaving the accelerometer just doing angles.
I realize this is a much harder project than the Python quad - I guess I'm just asking the forum for advice on whether it's completely impossible, or guidance to how to achieve and of the above steps.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20274
Joined: Sat Jul 30, 2011 7:41 pm

Re: Motion detection using RaspiCam

Fri May 16, 2014 11:03 am

The course motion estimation code recently added to the firmware and a modified raspivid could probably do what you want. You would need to process the CME data to get a general direction of movement, but the amount of data is greatly reduce from having to do it yourself so should be possible on the ARM. This data comes for free from the H264 encoder giving the entire ARM CPU over for processing the results.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
Hove
Posts: 1202
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Motion detection using RaspiCam

Fri May 16, 2014 11:24 am

Thank James, virtually everything you said is new to me, but that's OK, it gives me a direction to dig into which is what I'm looking for.

For others reading this thread, what I'm after is virtually the same as how an optical mouse works, but lighting is ambient - in fact previous revisions of one of the devices I listed used an optical mouse sensor! Don't know if that helps open up other solutions?
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

dmichel76
Posts: 5
Joined: Wed Jan 08, 2014 4:56 pm

Re: Motion detection using RaspiCam

Sat May 24, 2014 6:25 pm

Here is the link to the initial blog post regarding what jamesh mentioned:

http://www.raspberrypi.org/vectors-from ... stimation/

It would be very interesting to use this work to turn the raspicam into a flow detector

User avatar
Hove
Posts: 1202
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Motion detection using RaspiCam

Sat May 24, 2014 6:32 pm

Thanks, that's hugely helpful.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

elhigu
Posts: 1
Joined: Wed Dec 03, 2014 7:55 am

Re: Motion detection using RaspiCam

Wed Dec 03, 2014 8:06 am

Hi,

I had some problems my self interpreting data from motion vectors according to guide http://www.raspberrypi.org/vectors-from ... estimation (it was great help though, but actually didn't describe accurately how vectors could be read from stream).

So I wrote program, which read motion vector data from raspivid and streams it over websocket to be interpreted (https://github.com/elhigu/wsstreamprocessor). Browser reads vector data from websocket and plots motion vector information to screen so that color is selected by vector angle and intensity by vector length.

Main findings about raspivid -x format is described in README.md:
Motion vectors are coming from raspicam in following format:

{
signed char dx; // values seems to be around +- 80
signed char dy; // values seems to be around +-80
signed short sad; // values seems to be around 0..512 little endian
}
Numer of motion vectors can be calculated from your video resolution

var vectorsPerLine = Math.floor(imageWidth/16)+1
var vectorLines = Math.floor(imageHeight/16)+1
So for 1920x1080 FullHD video there are 121x68 vectors. First vector in frame is from bottom-left corner.

e.g vectors for 65x31 video stream (5x2 vectors) would be:

v6 v7 v8 v9 v10

v1 v2 v3 v4 v5
And frame size would be 40 bytes / frame.

User avatar
Hove
Posts: 1202
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: Motion detection using RaspiCam

Wed Dec 03, 2014 1:58 pm

That's, that gives a really useful insight - for the moment, I'm not using the RaspiCam for motion, but this has definitely brought the option back to life again, thanks.
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

Return to “Automation, sensing and robotics”

Who is online

Users browsing this forum: CharlyDelta and 17 guests