lagurus
Posts: 48
Joined: Wed Aug 07, 2013 8:02 am

mmal face detection

Wed Oct 09, 2013 12:23 pm

In mmal_parameters_camera.h are several definitions like

Code: Select all

MMAL_PARAMETER_FACE_TRACK
MMAL_PARAMETER_FACE_TRACK_RESULTS
...
It looks like some sort of face detection functions.

Has any sense to play with these functions and check if works?
Have someone any experience with them?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26434
Joined: Sat Jul 30, 2011 7:41 pm

Re: mmal face detection

Wed Oct 09, 2013 1:40 pm

lagurus wrote:In mmal_parameters_camera.h are several definitions like

Code: Select all

MMAL_PARAMETER_FACE_TRACK
MMAL_PARAMETER_FACE_TRACK_RESULTS
...
It looks like some sort of face detection functions.

Has any sense to play with these functions and check if works?
Have someone any experience with them?
I've never tried them on the Raspi. Might be worht someone having a go. If I have time I might try to incorporate it in to raspistill - no promises!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

lagurus
Posts: 48
Joined: Wed Aug 07, 2013 8:02 am

Re: mmal face detection

Thu Oct 17, 2013 4:04 pm

I've never tried them on the Raspi. Might be worht someone having a go. If I have time I might try to incorporate it in to raspistill - no promises!
I modified RaspiCamControl.c and added following code

Code: Select all

int raspicamcontrol_set_facedetection( MMAL_COMPONENT_T *camera )
{
    int nReturn = 0;

    MMAL_PARAMETER_FACE_TRACK_T rec_face_detection = {{MMAL_PARAMETER_FACE_TRACK, sizeof(MMAL_PARAMETER_FACE_TRACK_T)}};

    rec_face_detection.mode             = MMAL_PARAM_FACE_DETECT_ON;
    rec_face_detection.maxRegions       = 0;        // maximum number of regions to detect, 0 for unlimite
    rec_face_detection.frames           = 0;        // number of frames to apply this setting for,0 for unlimite
    rec_face_detection.quality          = 50;        // hint for algorithmic complexity, range is 0-100

    MMAL_STATUS_T statusFaceTrack = mmal_port_parameter_set( camera->control, &rec_face_detection.hdr );
    fprintf(stderr, "statusFaceTrack = %d\n", statusFaceTrack );

    return nReturn;
}
and added function call into function raspicamcontrol_set_all_parameters

But when I run program, it returns some error.

Code: Select all

mmal: mmal_vc_port_parameter_set: failed to set port parameter 64:0:EINVAL
statusFaceTrack = 3
It looks like invalid arguments problem, but I don't see anything wrong. Do you have some idea?

Thanks

User avatar
jacksonliam
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm

Re: mmal face detection

Sat Oct 19, 2013 8:16 am

lagurus wrote:
I've never tried them on the Raspi. Might be worht someone having a go. If I have time I might try to incorporate it in to raspistill - no promises!
I modified RaspiCamControl.c and added following code

Code: Select all

int raspicamcontrol_set_facedetection( MMAL_COMPONENT_T *camera )
{
    int nReturn = 0;

    MMAL_PARAMETER_FACE_TRACK_T rec_face_detection = {{MMAL_PARAMETER_FACE_TRACK, sizeof(MMAL_PARAMETER_FACE_TRACK_T)}};

    rec_face_detection.mode             = MMAL_PARAM_FACE_DETECT_ON;
    rec_face_detection.maxRegions       = 0;        // maximum number of regions to detect, 0 for unlimite
    rec_face_detection.frames           = 0;        // number of frames to apply this setting for,0 for unlimite
    rec_face_detection.quality          = 50;        // hint for algorithmic complexity, range is 0-100

    MMAL_STATUS_T statusFaceTrack = mmal_port_parameter_set( camera->control, &rec_face_detection.hdr );
    fprintf(stderr, "statusFaceTrack = %d\n", statusFaceTrack );

    return nReturn;
}
and added function call into function raspicamcontrol_set_all_parameters

But when I run program, it returns some error.

Code: Select all

mmal: mmal_vc_port_parameter_set: failed to set port parameter 64:0:EINVAL
statusFaceTrack = 3
It looks like invalid arguments problem, but I don't see anything wrong. Do you have some idea?

Thanks
Weird, I tried this on the 17th too! (and got the same results)
Though I couldn't find info about what quality, frames and maxReigions were, where did you find that out?

I used it like this:

Code: Select all

int raspicamcontrol_facetracking_on(MMAL_COMPONENT_T *camera)
{

   MMAL_PARAMETER_FACE_TRACK_T ft_mode = {{MMAL_PARAMETER_FACE_TRACK,sizeof(ft_mode)}, MMAL_PARAM_FACE_DETECT_ON, 0, 0, 50};

   if (!camera)
      return 1;

   return mmal_status_to_int(mmal_port_parameter_set(camera->control, &ft_mode.hdr));
}
I also tried it on all the camera->output ports (It just changes the number after 64: in the EINVAL message). I tried it on the encoder and null-sink ports, but got a function not implemented error.
I've yet to try it on the preview.

I also tried lots of different values for maxregions, frames and quality.

I also tried this:

Code: Select all

int raspicamcontrol_set_facetracking(MMAL_COMPONENT_T *camera, int facetrack)
{
   if (!camera)
      return 1;

   return mmal_status_to_int(mmal_port_parameter_set_boolean(camera->control, MMAL_PARAMETER_DRAW_BOX_FACES_AND_FOCUS, facetrack));
}
The command returns successfully, but nothing appears to happen. Perhaps facetracking needs to be 'ON' first. I haven't tried it on the preview yet but have tried on all the camera ports.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26434
Joined: Sat Jul 30, 2011 7:41 pm

Re: mmal face detection

Sat Oct 19, 2013 8:19 am

It needs to be on the camera control port I think. It's quite possible the face detection code isn't available for the Raspi. I think it is provided by a third party library, so would be a cost option.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
jacksonliam
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm

Re: mmal face detection

Thu Oct 24, 2013 6:29 pm

jamesh wrote:It needs to be on the camera control port I think. It's quite possible the face detection code isn't available for the Raspi. I think it is provided by a third party library, so would be a cost option.
Probably that then, Its a shame because it would've been very cool for demos of the awesomeness of the Pi and such! I'm thinking people following robots, cool presentations where it crops the face and zooms in, Crowd counting, adding AR to the video (e.g. putting hats on people), a webcam-pi on gimbal keeping a face in frame.

Some of that could possibly be done with offline openCV processing of individual frames, but not as well as a proper real-time face tracking solution.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26434
Joined: Sat Jul 30, 2011 7:41 pm

Re: mmal face detection

Thu Oct 24, 2013 7:38 pm

jacksonliam wrote:
jamesh wrote:It needs to be on the camera control port I think. It's quite possible the face detection code isn't available for the Raspi. I think it is provided by a third party library, so would be a cost option.
Probably that then, Its a shame because it would've been very cool for demos of the awesomeness of the Pi and such! I'm thinking people following robots, cool presentations where it crops the face and zooms in, Crowd counting, adding AR to the video (e.g. putting hats on people), a webcam-pi on gimbal keeping a face in frame.

Some of that could possibly be done with offline openCV processing of individual frames, but not as well as a proper real-time face tracking solution.
I'll ask around, but pretty sure the codebase doesn't yet have an inhouse face tracker. Note that I think the 3rd party one could track max 6 faces before it slowed down the system too much to get 30fps previews.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

jvcleave
Posts: 41
Joined: Thu May 24, 2012 10:27 pm

Re: mmal face detection

Mon Oct 28, 2013 9:50 pm

jamesh did you ever get an definite answer on this? In messing with an OpenMax implementation and I am able to successfully use OMX_SetConfig to not "not disabled"

Code: Select all

OMX_ERRORTYPE error = OMX_ErrorNone;
OMX_PARAM_CAMERADISABLEALGORITHMTYPE controlType;
OMX_INIT_STRUCTURE(controlType);
controlType.eAlgorithm = OMX_CameraDisableAlgorithmFacetracking;
controlType.bDisabled = OMX_FALSE;
error = OMX_SetConfig(camera, OMX_IndexParamCameraDisableAlgorithm, &controlType);
	
if(error== OMX_ErrorNone) 
{
	//is set
}
OMX_GetConfig fails but it also fails with OMX_CameraDisableAlgorithmImageEffects where I am able to set it to disabled and I know that is working

User avatar
Gavinmc42
Posts: 4508
Joined: Wed Aug 28, 2013 3:31 am

Re: mmal face detection

Tue Apr 25, 2017 8:55 am

Found the mmal face tracking firmware code while looking for something else.

This is old post and there does not seem much recent stuff.
Is this a dead duck or still unhatched egg?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8722
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: mmal face detection

Tue Apr 25, 2017 10:16 am

There are a number of optional processing stages that could be built into the firmware based on customer requirements and licencing agreements.

Face tracking, face recognition, antishake, face beautification, scene detection, and red eye reduction were all third party algorithms requiring licencing. They have not been licenced for the Pi, so aren't shipped.
High dynamic range was a Broadcom algorithm but required some very tight constraints on the sensor. The Pi camera modules don't really meet the constraints, hence it isn't shipped.
Temporal denoise was too slow to be that practical, and was more a proof of concept for shifting things into hardware for VC5. Again not shipped with Pi.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”