Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 1:01 pm

Yep, I get select errors even with the code matching the webcam framerate.

Thanks for the tip on gstreamer, I'll take a look at that. So I guess I'll have to settle for super-fast videos played back at a slower rate, not too much of an issue. I might look into wrapping it in some container so I can get the framerate down.

I guess I am just limited by the speed that my webcam can server images over USB. Out of interest, when the CSI camera module is released, would this help me in anyway? My aim being to record 5fps 640x480 (or higher).

Cheers for the help fbutler and jamesh :)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27419
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 2:26 pm

I've run the camera down to 2fps encoded to H264 without any problem on the Raspi camera (I get fast playback in OMXPlayer as well!). Not sure why 1fps didnt work though..should do! I'd like to try even lower fps (timelapse direct to H264)eventually. The current timelapse mode saves stills at whatever gaps you want (down to a minimum about 2fps) from which you can generate a video should you want, but direct to H264 would be better.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 2:37 pm

Sounds good. Would the CSI camera be capable of 640x480 (preferably higher) at 5fps, recorded on the Pi? I am almost certainly being limited by either the USB bandwidth or the webcam at the moment. I've done some more testing and the benchmark figures I am getting per frame grab is around 300ms at 640x480. This can peak at over 1000ms though! I can get a reasonably decent 2FPS 640x480 video out of this. Since then I've cut the video down to 320x240 and the time to grab a frame goes down to around 10ms (with the occasional peak to around 80ms).

Cheers
Last edited by Tw1stid on Sat Apr 27, 2013 3:10 pm, edited 1 time in total.

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 3:06 pm

I am almost certainly being limited by either the USB bandwidth or the webcam at the moment.
Have you done a recent sudo apt-get update && sudo apt-get upgrade recently and run rpi-update? There was a recent USB fix that improved webcam performance for a lot of people

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 4:08 pm

That's it! I'd never run rpi-update before....and now its recording at 25fps 640x480 :). Thanks for the suggestion, feel a bit stupid for not doing it earlier...can't believe its given me around 10x increase in USB speed from the webcam.

Now I'm going to try some other resolutions, then investigate how I might place a timestamp onto each frame so you can see when it was recorded. Any suggestions welcome :lol: but I'm gonna have a hunt around google and see if I can find anything.

Thanks again for the help!

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 5:53 pm

Tw1stid wrote:That's it! I'd never run rpi-update before....and now its recording at 25fps 640x480 :). Thanks for the suggestion, feel a bit stupid for not doing it earlier...can't believe its given me around 10x increase in USB speed from the webcam.
That's great news. :-)
Tw1stid wrote:Now I'm going to try some other resolutions, then investigate how I might place a timestamp onto each frame so you can see when it was recorded. Any suggestions welcome :lol: but I'm gonna have a hunt around google and see if I can find anything.
Gstreamer is your friend if you want to overlay stuff. The OMX gstreamer elements only work with the latest Gstreamer 1.0 version which is not yet available via apt-get on Raspian. However if you want them then defiant has kindly packaged them as described in this post:
http://www.raspberrypi.org/phpBB3/viewt ... 63#p293663
so if you add that line to your /etc/apt/sources.list: file you should than be able to install them using apt-get install.
The set of gstreamer plug-ins that I have installed are:
  • gst-libav
    gst-plugins-bad
    gst-plugins-good
    gstreamer-core
    gst-omx
    gst-plugins-base
    gst-plugins-ugly orc
Each set of plugins contains elements that can be used in a gstreamer pipeline. Once you have them installed do a:

gst-inspect-1.0 | grep overlay

to see the available overlay elements.

You can use gst-inspect-1.0 <element name> to get more details on any individual elements. The one's that you will probably be interested in are: clockoverlay, timeoverlay, and textoverlay. There's a gstreamer cheat sheet here which shows some examples of their usage:

http://wiki.oz9aec.net/index.php/Gstreamer_cheat_sheet

I haven't tried them out with h264 based pipelines using the gst-omx plugins but they certainly worked when I used v4lsrc with UYVY based pipelines. So probably worth having a look at them. The GStreamer documentation is available at: http://gstreamer.freedesktop.org/documentation/

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 6:29 pm

Sounds perfect for what I need, will investigate, thanks!

Also noticed that setting the fps on the camera now works properly after rpi-update :D

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Sat Apr 27, 2013 7:04 pm

Tw1stid wrote:Sounds perfect for what I need, will investigate, thanks!

Also noticed that setting the fps on the camera now works properly after rpi-update :D
Ahhh, interesting. I was somewhat puzzled :? by it as I wasn't having the same issue.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27419
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware H.264 Encode of Webcam

Sun Apr 28, 2013 3:34 am

Tw1stid wrote:Sounds good. Would the CSI camera be capable of 640x480 (preferably higher) at 5fps, recorded on the Pi? I am almost certainly being limited by either the USB bandwidth or the webcam at the moment. I've done some more testing and the benchmark figures I am getting per frame grab is around 300ms at 640x480. This can peak at over 1000ms though! I can get a reasonably decent 2FPS 640x480 video out of this. Since then I've cut the video down to 320x240 and the time to grab a frame goes down to around 10ms (with the occasional peak to around 80ms).

Cheers
The CSI camera is a 5mp stills device, which is also capable of up to 1080p30 recording. You can certainly record any size up to that from 2-30fps (In the future we will get the 60 and 90fps modes working in 640x480). All encoding is H264 up to about 30Mbits/s, so pretty good HD quality.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Mon Apr 29, 2013 6:42 am

Sounds promising!

I'll see how the webcam goes. It's currently sat recording 960x544 video at 5 FPS and seems to be working great. Low light performance isn't so good though and the image isn't very focused. I put that down to it being a webcam which is designed to focus on faces up-close and in a well lit environment. I'd be interested to hear if the CSI Camera Module has decent low light performance or adjustable focus?

Cheers

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27419
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware H.264 Encode of Webcam

Mon Apr 29, 2013 6:52 am

Tw1stid wrote:Sounds promising!

I'll see how the webcam goes. It's currently sat recording 960x544 video at 5 FPS and seems to be working great. Low light performance isn't so good though and the image isn't very focused. I put that down to it being a webcam which is designed to focus on faces up-close and in a well lit environment. I'd be interested to hear if the CSI Camera Module has decent low light performance or adjustable focus?

Cheers
Fixed focus, but much better low light performance than a webcam I would think.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Sun May 12, 2013 9:35 am

Tw1stid wrote:Its probably also worth mentioning that when I set the webcam to 5 fps using v4l2-ctl --set-parm 5 then I just get select timeout errors
I think I've found the issue. I've been testing using a 3.8.6 kernel. When I reverted back to a 3.6.11 kernel this morning I also got the select timeout errors. In the code try changing "tv.tv_sec = 2;" to "tv.tv_sec = 10;". That solved the issue for me. Unfortunately I'm now getting a green screen when capturing in 3.6.11, while capturing using 3.8.6 works. I'll have to investigate the reason for this later.

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Sun May 12, 2013 9:56 am

Hi,

Those select timeouts stopped when I did rpi-update. :D

It's been running successfully now for a while, recording 1 hour videos at 960x544, 5FPS. Then pops them in an mkv container and copies them to my NAS. Runs for a few days at a time before crashing, just added loads of printfs to try to workout where it's falling over.

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Sun May 12, 2013 5:24 pm

Tw1stid wrote:Hi,

Those select timeouts stopped when I did rpi-update. :D

It's been running successfully now for a while, recording 1 hour videos at 960x544, 5FPS. Then pops them in an mkv container and copies them to my NAS. Runs for a few days at a time before crashing, just added loads of printfs to try to workout where it's falling over.
If you run the program using the gdb debugger you can use a "bt" command to backtrace to where in the code the program crashed.

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Sun May 12, 2013 5:50 pm

fbutler wrote:
Tw1stid wrote:Hi,

Those select timeouts stopped when I did rpi-update. :D

It's been running successfully now for a while, recording 1 hour videos at 960x544, 5FPS. Then pops them in an mkv container and copies them to my NAS. Runs for a few days at a time before crashing, just added loads of printfs to try to workout where it's falling over.
If you run the program using the gdb debugger you can use a "bt" command to backtrace to where in the code the program crashed.
Ahh that's good to know, thanks. Unfortunately I don't think it's actually crashing - at least it's not terminating, it just appears to stop. Just trying to work out which function is blocking but it's a slow process when it only happens every couple of days....

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Mon May 13, 2013 6:52 pm

fbutler wrote:
Tw1stid wrote:Hi,

Those select timeouts stopped when I did rpi-update. :D

It's been running successfully now for a while, recording 1 hour videos at 960x544, 5FPS. Then pops them in an mkv container and copies them to my NAS. Runs for a few days at a time before crashing, just added loads of printfs to try to workout where it's falling over.
If you run the program using the gdb debugger you can use a "bt" command to backtrace to where in the code the program crashed.
Thanks for this tip! It's just stopped again and it seems its deadlocked inside ilclient_get_input_buffer. Just wondered if you'd seen this before? Full backtrace is below. As a test I've changed the final argument of ilclient_get_input_buffer to 0. I think thats a flag for whether or not the call blocks (decent documentation seems to be impossible to find!). I suspect changing it 0 will mean it simply continually fails after a few days but I'll give it a try.
#0 0xb6224700 in sem_wait@@GLIBC_2.4 () from /lib/arm-linux-gnueabihf/libpthread.so.0
#1 0xb6247f9c in vcos_semaphore_wait (sem=0xb625536c)
at /home/dc4/userland/build/inc/interface/vcos/vcos_platform.h:257
#2 _vcos_thread_sem_wait () at /home/dc4/userland/build/inc/interface/vcos/vcos_platform.h:603
#3 vcos_generic_event_flags_get (flags=0x4fcf48, bitmask=1, op=5, suspend=4294967295,
retrieved_bits=0xbec245cc)
at /home/dc4/userland/interface/vcos/generic/vcos_generic_event_flags.c:223
#4 0x0000da20 in vcos_event_flags_get (flags=0x4fcf48, requested_events=1, op=5, suspend=4294967295,
retrieved_events=0xbec245cc)
at /opt/vc/include/interface/vcos/generic/vcos_generic_event_flags.h:118
#5 0x00010fb0 in ilclient_get_input_buffer (comp=0x4fcf30, portIndex=200, block=1) at ilclient.c:1352
#6 0x0000b338 in process_image (p=0xb5439000, size=1044480, frame_number=1618)
at v4l2capturetoH264.c:271
#7 0x0000c350 in read_frame (frame_number=1618) at v4l2capturetoH264.c:555
#8 0x0000c6c4 in mainloop () at v4l2capturetoH264.c:624
#9 0x0000d900 in main (argc=1, argv=0xbec248c4) at v4l2capturetoH264.c:1004

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Tue May 14, 2013 4:59 am

Tw1stid wrote:Thanks for this tip! It's just stopped again and it seems its deadlocked inside ilclient_get_input_buffer. Just wondered if you'd seen this before? Full backtrace is below. As a test I've changed the final argument of ilclient_get_input_buffer to 0. I think thats a flag for whether or not the call blocks (decent documentation seems to be impossible to find!). I suspect changing it 0 will mean it simply continually fails after a few days but I'll give it a try.
Unfortunately I haven't seen that behaviour before. You are right about finding documentation, it's not easy :( . Have you seen this documentation?: http://www.khronos.org/files/openmax_il_spec_1_0.pdf
I'll have a look at the code again tonight and see if I can spot anything. It would also be worth looking at the original hello_encode example, which the code is based on, in case I've inadvertently changed something in my code to cause the issue. If you figure out what the issue is please post an update. I'll let you know if I find anything when I look at the code again.

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Tue May 14, 2013 6:45 am

fbutler wrote:Unfortunately I haven't seen that behaviour before. You are right about finding documentation, it's not easy :( . Have you seen this documentation?: http://www.khronos.org/files/openmax_il_spec_1_0.pdf
I'll have a look at the code again tonight and see if I can spot anything. It would also be worth looking at the original hello_encode example, which the code is based on, in case I've inadvertently changed something in my code to cause the issue. If you figure out what the issue is please post an update. I'll let you know if I find anything when I look at the code again.
I haven't seen that documentation, looks like it might be quite helpful, thanks :)

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Tue May 14, 2013 8:23 am

Tw1stid wrote:I haven't seen that documentation, looks like it might be quite helpful, thanks :)
It would also be worth looking at the ilclient source code which can be found in: /opt/vc/src/hello_pi/libs/ilclient

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Thu May 16, 2013 1:03 pm

Something else worth having a look at: http://www.raspberrypi.org/phpBB3/viewt ... 82#p350182

I haven't had a chance to try it out yet.

Tw1stid
Posts: 20
Joined: Sat Apr 13, 2013 2:27 pm

Re: Hardware H.264 Encode of Webcam

Thu May 16, 2013 1:06 pm

Looks helpful, thanks. Got hold of the CSI camera yesterday so am going to take a look at porting the code to use that at some point in the future.

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: Hardware H.264 Encode of Webcam

Thu May 16, 2013 1:24 pm

Tw1stid wrote:Looks helpful, thanks. Got hold of the CSI camera yesterday so am going to take a look at porting the code to use that at some point in the future.
Yeah, one of mine arrived this morning so I'll start playing with it tonight.

looeyc
Posts: 13
Joined: Sat May 11, 2013 6:46 am

Re: Hardware H.264 Encode of Webcam

Thu May 30, 2013 2:57 am

Hi guys,
I am also working on a similiar subject with you. I capture 320x240 images from a USB camera, and want to perform H.264 encoding before RTMP sending. The problem is the poor fps, which is about 5 fps with 80% CPU usage :| . I am using libx264 to perform the H.264 encoding. I downloaded the source tar ball of libx264 from www.videolan.org, and cross compile it with "--disable-asm" option (compiling fails without this option). I think the reason for the poor fps is the encoding process takes too much CPU cycles. On the other way, RPi does not have any hardware H.264 encoder, dose it? So software is the only choise. Could you please give me some suggestions? :?:

Without "--disable-asm" option, the compilation of libx264 just fails with errors, because the CPU of RPi is a ARMv6 processor and there are some non-ARMv6 asm instructions in libx264. Do you or someone else have any idea to optimize libx264 for RPi? :?:

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 27419
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware H.264 Encode of Webcam

Thu May 30, 2013 5:36 am

The Raspi has a HW H264 encoder capable of 1080p30 at 30MBits/s - there are a number of posts in the forum on how to use it using OpenMAX.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

looeyc
Posts: 13
Joined: Sat May 11, 2013 6:46 am

Re: Hardware H.264 Encode of Webcam

Sat Jun 01, 2013 3:04 pm

jamesh wrote:The Raspi has a HW H264 encoder capable of 1080p30 at 30MBits/s - there are a number of posts in the forum on how to use it using OpenMAX.
Hi Jamesh,
Thanks for your help! I am encoding H.264 with openmax at 25fps now! :lol:
I am so susuprised to be capable of reaching such a high performance! :o
Before your advice, I a lot of introductions about the RPi, and there is only the H.264 decoder mentioned, so I doubted there would be no encoder. Your reply save me a lot of time and energy, thank you again! :D
Best regards!

Return to “Graphics, sound and multimedia”