Facial recognition: OpenCV on the camera board

I’ve been talking to Pierre Raufast for a little while now about his efforts to get OpenCV ported smoothly to the Raspberry Pi camera board (which is available from the usual suspects: head to the links under “Buy a Pi” at the top right). OpenCV is an open-source library for real-time image processing, and is used in applications like gesture mapping, motion tracking – and facial recognition.

You might recognise Pierre’s name from a project we featured here back in April. The Magic Mirror recognises people looking into it, and talks to them accordingly. Back then he was using a webcam; now he’s got his hands on a camera board. The Raspberry Pi camera board is a better fit for the device, with a much higher framerate and better resolution.

In most webcams, OpenCV works natively. But because the Pi camera board isn’t a USB device, some work needed to be done to get it up and running. Pierre has done a bang-up job, and has made a seven-part tutorial available to get you up to speed. When you’re done, you’ll be able to see results like this:

Thanks Pierre – we can’t wait to see what you do next!

21 comments

Avatar

This is great work, well done!

Avatar

cool cool cool. I have been wanting to get some face recognition going. However, I will use a slightly different approach than doing it directly on the PI. Right now what I have working is a stream from the the pi, which is then captured by another PC and then interpretted with OpenCV. This has some nice advantages that you have a full desktop with a GPU, so you can use the GPU libraries (decode the stream on the GPU). Also a full CPU with all the debugging goodness. Possibly you can even use 2 rasberry PIs to do a makeshift stereo cam. Your tut is great because I really don’t have much experience with OpenCV. Latency is a question for sure, but the GPU should handle a HD stream with some success.

Avatar

Very nice!

Avatar

Hello,

what interface/API does the camera use? Not standard v4l2?

cheerio,

Rene Dudfield
pygame core dev

Avatar

You have two choices :
–MMAL (custom API by Broadcom)
— OpenMAX (“industry standard” , with Broadcom specifics and badly documented)

The first is said to be much easier to use.

Check our forums , people want to use pyGame with the cam and fail :( – its not a V4L2 device)
The foundation is currently working on a V4L2 “shim” , but perhaps you’re faster ;-)

Avatar

There’s some facial detection support in the OpenMAX IL, probably implemented inside the GPU, so it could provide some decent speed ups. Documentation is inside firmware repository or here: http://home.nouwen.name/RaspberryPi/documentation/ilcomponents/prop.html

Look for “Face detection” and “Face detection output”. I never tested this as I don’t have the cam, and I don’t know if it works.

Avatar

MMAL has some nice looking face detection stuff in it:
https://github.com/raspberrypi/userland/blob/master/interface/mmal/mmal_parameters_camera.h#L404

I wonder if it works?

Avatar

I’m trying to build libfacerec as part of the tutorial, and htet he following:


pi@rpi /usr/src/libfacerec $ cmake .
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/src/libfacerec
pi@rpi /usr/src/libfacerec $ make
[ 16%] Building CXX object CMakeFiles/opencv_facerec.dir/src/colormap.cpp.o
In file included from /usr/src/libfacerec/src/colormap.cpp:18:0:
/usr/src/libfacerec/include/facerec.hpp:73:41: error: expected class-name before ‘{’ token
make[2]: *** [CMakeFiles/opencv_facerec.dir/src/colormap.cpp.o] Error 1
make[1]: *** [CMakeFiles/opencv_facerec.dir/all] Error 2
make: *** [all] Error 2

Avatar

Answering my own post! This problem does not occur if you use the source for version 0.04 as available at https://github.com/bytefish/libfacerec/zipball/v0.04

Avatar

Oh, very nice.

Still waiting for the camera …..

Avatar

Interesting approach to actually take raspicam and bind it with openCV.

I actually did the opposite and called the standard raspicam in bin/ and used a C program with OpenCV bindings to call the Haar-like features function. The plan is for Raspbmc to detect when a user leaves the room and pause the video. This way, the user can still use raspicam as standard (original binaries, and thus any improvements, are preserved)

Avatar

How hard is it to recognize a different object that is pre-programmed such as a block, or even a logo?

Avatar

Or a Rhino….

Avatar

Nice to see you well again, liz.

Avatar

If the PI would just have a “little” more horsepower or an OpenCL capable GPU.

It’s weak ARM11 hardly keeps up with the OpenCV requirements. Would be great if raspberry V2 could have at least a Cortex-A9 or A15 chip…

Avatar

There is no need to rebuild the complete userland (step 2 in the tutorial) since all libraries are already available under /opt/vc/lib and also most needed include files are available under /opt/vc/include.
What is missing are the interface/mmal include files.
See my comment under steps 2 and 3 of the tutorial.
Any chance the mmal include files and raspicam sources can be included in the Raspbian distribution?

Avatar

Hey! I just impletemented this and everyone works as expected!
However, this implementation with OpenCV does not seem to make use of the raspberry pi’s GPU. Doesn’t the GPU handle matrix operations much faster? Is there a way to use the GPU with OpenCV? Cheers!

Avatar

There’s some work going on here at Pi Towers to get it working with the GPU. We’ll be writing more about it when it’s a bit further down the line.

Avatar

Any news ? I know that not much time has passed but I would really like to use the GPU with OpenCV. Right now I’m very far away from real time or near real time or even 1fps with some decent tracking code. Meaning my Pi won’t be really useful for any robotics yet… :(

Avatar

excuse me i want ask you question how to draw rectangular in face detection using simple cv

Avatar

Hello!
I’m a newbie and I’m interested in face recognition using the opencv libraries on my raspberry pi.
I’ve tried using the python “facedetect.py” example contained in the opencv-2.4.9
It works ok …but I would like to try a quicker solution with a compiled language, let’say C++.
Could you please indicate me if there’ s something on the net.
At the moment I’m using a logitech usb webcam, but I’m waiting for an already ordered Pi Camera.
Thank You in advance
Ciao
Antonio

Leave a Comment

Comments are closed