sodnpoo
Posts: 2
Joined: Sat Apr 05, 2014 6:32 am

camera with opencv motion detection and recording

Sat Apr 05, 2014 6:36 am

I've pushed some code for the raspberry pi and it's camera - based on tasanakorn's work - up to github that uses some very simple OpenCV to do background subtraction and then detect movement.

http://www.sodnpoo.com/posts.xml/raspbe ... ording.xml

lagurus
Posts: 48
Joined: Wed Aug 07, 2013 8:02 am

Re: camera with opencv motion detection and recording

Fri Apr 25, 2014 1:46 pm

Nice work, you have inspired me.

Just some note - I wasn't able to achieve 1920x1080x30fps for video, maximum were only 22fps without any opencv processing (with
opencv processing only 17fps).
You can check test version: https://github.com/lagurus/rpi-mmal-ope ... etect-test

I was able to achieve 30fps for video only if I used MMAL_ENCODING_OPAQUE, but in this case is not possible
to achieve any data for opencv processing :(

Off topic:
Does anybody know, how to use MMAL_ENCODING_OPAQUE with MMAL_ENCODING_MJPEG or MMAL_ENCODING_JPEG?
I want to use callback on video port and then put data directly to encoder (without mmal port connections). For MMAL_ENCODING_H264 it works, for MMAL_ENCODING_MJPEG or MMAL_ENCODING_JPEG it leads to application crash and camera stop working till next reboot.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8465
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: camera with opencv motion detection and recording

Fri Apr 25, 2014 3:36 pm

lagurus wrote:Off topic:
Does anybody know, how to use MMAL_ENCODING_OPAQUE with MMAL_ENCODING_MJPEG or MMAL_ENCODING_JPEG?
I want to use callback on video port and then put data directly to encoder (without mmal port connections). For MMAL_ENCODING_H264 it works, for MMAL_ENCODING_MJPEG or MMAL_ENCODING_JPEG it leads to application crash and camera stop working till next reboot.
Sorry, I'm slightly confused by your comment. You're setting both camera output[1] and video_encode input[0] to encoding MMAL_ENCODING_OPAQUE but not using a connection, and that is causing a crash? Before or after the first buffer is actually passed across?
I'm assuming if you use a mmal_connection it does work.
The only thing I can think of is that the format isn't defined correctly and a buffer is being rejected. Is there anything in dmesg or vcdbg log msg? Or is there a simple test case I can run?

MJPEG encode does require more memory in the GPU than H264, so that might be the other reason.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

lagurus
Posts: 48
Joined: Wed Aug 07, 2013 8:02 am

Re: camera with opencv motion detection and recording

Fri Apr 25, 2014 9:28 pm

Sorry, I'm slightly confused by your comment. You're setting both camera output[1] and video_encode input[0] to encoding MMAL_ENCODING_OPAQUE but not using a connection, and that is causing a crash? Before or after the first buffer is actually passed across?
I'm assuming if you use a mmal_connection it does work.
The only thing I can think of is that the format isn't defined correctly and a buffer is being rejected. Is there anything in dmesg or vcdbg log msg? Or is there a simple test case I can run?

MJPEG encode does require more memory in the GPU than H264, so that might be the other reason.
If it is not clear try to look on simple demo code:

https://github.com/lagurus/rpi-mmal-ope ... etect-h264 - h264 - work
https://github.com/lagurus/rpi-mmal-ope ... tect-mjpeg - mjpeg - no frames encoded -> need to kill app -> then not able to access camera anymore.

Both sources differs only in encoder format MMAL_ENCODING_H264 x MMAL_ENCODING_MJPEG

vcdbg log msg contain:

Code: Select all

mmal: mmal_vll_load: could not load VLL 'videnc.vll':
It can be somehow connected with available GPU memory because for 1280x720 it works. I have 196MB reserved for GPU.

But on the other hand if is used as "source" MMAL_ENCODING_I420 (not MMAL_ENCODING_OPAQUE) it works without any problem.

__posman
Posts: 1
Joined: Sun Jan 04, 2015 4:01 am

Re: camera with opencv motion detection and recording

Sun Jan 04, 2015 4:04 am

Hi

I'm trying to compile the code but I'm getting an error

Code: Select all

[100%] Building C object CMakeFiles/mmal_opencv_modect.dir/opencv_modect.c.o
In file included from /usr/include/opencv2/video/video.hpp:47:0,
                 from /home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.c:14:
/usr/include/opencv2/video/background_segm.hpp:48:1: error: unknown type name ‘namespace’
/usr/include/opencv2/video/background_segm.hpp:49:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
CMakeFiles/mmal_opencv_modect.dir/build.make:54: recipe for target 'CMakeFiles/mmal_opencv_modect.dir/opencv_modect.c.o' failed
make[2]: *** [CMakeFiles/mmal_opencv_modect.dir/opencv_modect.c.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/mmal_opencv_modect.dir/all' failed
make[1]: *** [CMakeFiles/mmal_opencv_modect.dir/all] Error 2
Makefile:72: recipe for target 'all' failed
make: *** [all] Error 2
I installed the prerequisites (cmake, libopencv-dev) and compiled the required library (/opt/vc/src/hello_pi/libs/vgfont)

andreionutz
Posts: 2
Joined: Mon Jan 05, 2015 2:56 pm

Re: camera with opencv motion detection and recording

Mon Jan 05, 2015 3:08 pm

it's because the code is treated as c instead of c++. i've just solved this by changing the file extension to .cpp and thus forcing the c++ compiler to jump in.
now i have a conversion error, and more than that, it appears as the openCv library is not properly included: 'cvResize' was not declared in this scope. does anyone have a solution for this?

andreionutz
Posts: 2
Joined: Mon Jan 05, 2015 2:56 pm

Re: camera with opencv motion detection and recording

Tue Jan 06, 2015 2:29 pm

and then for the openCv functions I had to add this:

Code: Select all

#include <opencv2/imgproc/imgproc_c.h>
... and then i switched to https://github.com/orgicus/PiCapture because i just needed to get started somewhere.
but i'm still interested to find out when this works again and looking forward to give it a try.

powenko
Posts: 4
Joined: Thu Apr 03, 2014 11:52 pm

Re: camera with opencv motion detection and recording

Fri Sep 18, 2015 9:34 pm

I fixed the c problems follow below steps.

$. mv opencv_modect.c opencv_modect.cpp
$ nano CMakeList.txt

Change
add_executable(mmal_opencv_modect opencv_modect.c)

To
add_executable(mmal_opencv_modect opencv_modect.cpp)


$ make


but, II still have below error.

[100%] Building CXX object CMakeFiles/mmal_opencv_modect.dir/opencv_modect.cpp.o
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp: In function ‘void camera_video_buffer_callback(MMAL_PORT_T*, MMAL_BUFFER_HEADER_T*)’:
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:145:43: error: invalid conversion from ‘uint8_t* {aka unsigned char*}’ to ‘char*’ [-fpermissive]
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp: In function ‘int raspicamcontrol_set_rotation(MMAL_COMPONENT_T*, int)’:
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:318:95: error: ‘mmal_port_parameter_set_int32’ was not declared in this scope
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp: In function ‘int setup_camera(PORT_USERDATA*)’:
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:350:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:351:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:352:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:353:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:354:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:355:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:356:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:357:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:358:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:359:13: error: expected primary-expression before ‘.’ token
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:410:148: error: ‘mmal_port_pool_create’ was not declared in this scope
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:430:85: error: ‘mmal_port_parameter_set_boolean’ was not declared in this scope
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp: In function ‘int setup_encoder(PORT_USERDATA*)’:
/home/pi/src/raspberrypi/rpi-mmal-opencv-modetect-master/opencv_modect.cpp:504:152: error: ‘mmal_port_pool_create’ was not declared in this scope
CMakeFiles/mmal_opencv_modect.dir/build.make:54: recipe for target 'CMakeFiles/mmal_opencv_modect.dir/opencv_modect.cpp.o' failed
make[2]: *** [CMakeFiles/mmal_opencv_modect.dir/opencv_modect.cpp.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/mmal_opencv_modect.dir/all' failed
make[1]: *** [CMakeFiles/mmal_opencv_modect.dir/all] Error 2
Makefile:72: recipe for target 'all' failed
make: *** [all] Error 2

danicymru54
Posts: 11
Joined: Fri Jan 20, 2017 12:57 pm
Contact: Website

Re: camera with opencv motion detection and recording

Mon Nov 20, 2017 3:57 pm

If anyone is interested I have done some work on this subject. Using a webcam and pizero I have developed a security system which records the motion and pushes to dropbox. This manages to get between 27 and 30 fps (frames per second) video. You can also control the settings remotely using dropbox. https://danicymru.wordpress.com/2017/10 ... nd-opencv/. Let me know if you find anything interesting.

Return to “Camera board”