anson2004
Posts: 8
Joined: Wed Aug 07, 2013 6:16 am
Location: Finland

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Mon Aug 12, 2013 3:57 pm

Hello,

I add one cvSaveImage function in video_buffer_callback like this to save images from video:
//img = cvLoadImage("test.jpg",CV_LOAD_IMAGE_COLOR);
mmal_buffer_header_mem_lock(buffer);
memcpy(userdata->image->imageData, buffer->data, userdata->video_width * userdata->video_height); // ????
mmal_buffer_header_mem_unlock(buffer);
//printf("img = %d w=%d, h=%d\n", img, img->width, img->height);
cvSaveImage("foo.png",userdata->image);

But when I use timer in the code to get video run 20 min. Most of time I would get error: segmentation fault. Then the code stop.
I assume there is something wrong in memory allocation and release. I do not really understand the whole code.
Could someone help me to point out which statement(s) may cause the momery problem and if possible, how to solve the problem?

Anson

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22684
Joined: Sat Jul 30, 2011 7:41 pm

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Mon Aug 12, 2013 4:16 pm

I haven't looked at that code, but IIRC the buffer isn't necessarily the size of the whole image - it could just be a stripe from the image. So you may be copying much more than the allocated buffer, hence seg fault. You need to collate all the buffers in to a big buffer (of the size of the image) until you get an end of frame flag, at which point your overall buffer will hold the entire image. Then you save it.

One gotcha - if its a YUV image, then each Y,U,V plane must be padded to 16 width or height - For 1080p, 1920 is already divisible by 16, but 1080 isnt, so you round up to the next which is 1088. RGB you just pad the whole image.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

anson2004
Posts: 8
Joined: Wed Aug 07, 2013 6:16 am
Location: Finland

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Mon Aug 12, 2013 7:45 pm

Thanks, while this code ( I modified) is working at least more than 10 min, then in some point after 10 min. It crash (seg fault). So I am not sure what cause the problem. I assume it may the memory release. I will try to check buffer size though I did not understand the source code.

anson2004
Posts: 8
Joined: Wed Aug 07, 2013 6:16 am
Location: Finland

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Thu Aug 15, 2013 7:49 am

Thanks for jamesh's answer and tasanakorn's code. The error line is exactly memcpy() in callback function when I use gdb to test. But the size of buffer is right . The problem is when you jump out of the while loop. You need to release the port and connection.etc in case of new frame coming in callback function. Here is the code I add in the end. Please tell me if I miss something since I am new in RPI and camera board. BTW, I notice there are some frames lost from view of my screen. But I am not sure if it exactly loss frame when I capture images in 10 fps ????


// some release
vcos_semaphore_delete(&userdata.complete_semaphore);
// mmal_status_to_int(status); // ???
// Disable all our ports that are not handled by connections
if (camera_video_port && camera_video_port->is_enabled)
mmal_port_disable(camera_video_port);
//check_disable_port(camera_video_port); //

mmal_connection_destroy(camera_preview_connection); //mmal_connection_destroy(state.preview_connection);
/* Disable components */
//
if ( preview )
// mmal_component_disable(preview);
mmal_component_destroy(preview);

if (camera) {
mmal_component_disable(camera);
mmal_component_destroy(camera);
}


Anson

samarth_i
Posts: 2
Joined: Thu Sep 05, 2013 4:24 pm

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Thu Sep 05, 2013 4:50 pm

Hi, thanks for your work and for sharing it with us!

I am currently trying to write a wrapper for your code to use the new OpenCV C++ API. And also to get color images, not just grayscale.

The crux of my code is:

Code: Select all

int w = userdata->video_width, h = userdata->video_height
char * pointer = (char *) (buffer -> data);
cv::Mat y = cv::Mat(w, h, CV_8UC1, pointer);
pointer += w*h;
cv::Mat u = cv::Mat(w/2, h/2, CV_8UC1, pointer);
pointer += w*h/4;
cv::Mat v = cv::Mat(w/2, h/2, CV_UC1, pointer);
This, I believe is the standard way to decode a contiguous I420 buffer into separate Y, U and V matrices.

However, I am observing (I think) some form of padding. Please see the attached image - the Y image is quite good, but the entire U is blank and V seems to be shifted down. Does anybody know how to deal with this or if the I420 format specifies some padding requirements that I don't know of?

Thanks for the help in advance!
screenshot1.png
Screenshot of Y, U and V images
screenshot1.png (46.94 KiB) Viewed 3474 times

cosmicboy
Posts: 17
Joined: Mon Sep 03, 2012 9:25 am

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Fri Nov 22, 2013 10:06 pm

Hello, I did download the repository and compiled it, but how do I start the software, e.g. the face recognition example?
I guess I'm too stupid...
Any example would be highly appreciated.

Thanks

dragondark
Posts: 1
Joined: Fri Feb 07, 2014 1:24 am

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Fri Feb 07, 2014 1:39 am

peepo wrote:thanks for your prompt response,

afaict 'works for me' using:
mmal_video_record > test2.h264

looking forward to your further comments and patches

how to set length of time to record?

Peepo, where did you find the 'mmal_video_record' to set the file? I've been going over the code forward and backwards trying to find it. If its not in the code, could you breifly describe what you did to set the output to a file? Also, did you find a way to set lenght of record time?


Thank you in advance for any replies!!

D

bendzo
Posts: 1
Joined: Fri Feb 07, 2014 11:01 am

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Fri Feb 07, 2014 11:23 am

highway900 wrote:So I have been doing some Augmented reality on the Pi with opencv and aruco.

Playing with the res gets me about 20fps marker detection.

Will post some code really soon.
Hello,
I'm currently working with aruco in university project.
We only need marker detection and it would be very nice to have such fast detection.
Can You please share some code for marker detection?

Thank You very much!
Jacob


dtavares07
Posts: 1
Joined: Fri May 01, 2015 7:33 pm

Re: Experiment with Camera Module : MMAL, OpenCV, Overlay

Fri May 01, 2015 7:39 pm

Hi.

Can someone tell how do I compile a examples.

I managed to install everything but I cant compile any example with: gcc -o foo foo.c

Thanks

Return to “Camera board”