jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22724
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Tue Dec 03, 2013 6:48 pm

I wouldn't expect many more options than you get with the raspistill/vid apps - the underlying GPU code is still the same - but it's good to get some sort of standard GUI.

No point in asking me about any issues you may have - I had nothing to do with the v4l driver!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Tue Dec 03, 2013 7:17 pm

jbeale, use the overlay preview from the v4l tool. Use guvcview -o (note the -o) only to control the settings.

Using the preview built into guvcview will be slow.

User avatar
jbeale
Posts: 3439
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Tue Dec 03, 2013 7:37 pm

I don't know if there is any plan for continuing work on the V4L2 driver, but just in case... is there any chance of getting, for example /dev/video0 and /dev/video1 where you can get two different resolutions at once? There are several use cases: low-res for realtime transport over network and/or CPU-intensive processing steps, and high-res for local display and archiving. When you have only one video device and switch between resolutions, you have a significant latency while the camera restarts and autoexposure get reset, etc. meanwhile the thing you wanted to record may be gone.

Also, thanks towolf; using

Code: Select all

v4l2-ctl --overlay=1 # enable viewfinder
guvcview -o
it works smoothly and uses nearly no CPU time. However then you don't have the option to capture and save a still JPEG from the application.

User avatar
g7ruh
Posts: 67
Joined: Mon Apr 23, 2012 9:49 am
Location: Blackfield UK

Re: Official V4L2 driver

Tue Dec 03, 2013 8:49 pm

jbeale wrote:
fernandovg wrote:How to check it was installed ok?

If you don't see any error messages and find a valid JPEG file "test1.jpg" in your current directory, then the driver is installed and working.
I am working with a fresh install of Rasbian, updated software and firmware.

I get this error "v4l2-ctl: error while loading shared libraries: libv4l2.so.0: cannot open shared object file: No such file or directory"

the fix (or a fix) is to

Code: Select all

sudo apt-get install libv4l-dev
then I get the test1.jpg :)

Roger

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Tue Dec 03, 2013 9:02 pm

Use

Code: Select all

configure --prefix=/usr

dynamitemedia
Posts: 30
Joined: Sun Dec 01, 2013 9:12 pm

Re: Official V4L2 driver

Tue Dec 03, 2013 9:14 pm

will this get rid of the choppy video issues? i tried the rpi-update and that didnt help, using the c290 webcam. which is odd cause it works fine in Gucview but nothing else.

i am going to test with a whole new install and this update and report back... and i just bought a beaglebone last night to use with my cam, gonna stink if this fixes it ! lol

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5268
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Official V4L2 driver

Tue Dec 03, 2013 9:22 pm

dynamitemedia wrote:will this get rid of the choppy video issues? i tried the rpi-update and that didnt help, using the c290 webcam. which is odd cause it works fine in Gucview but nothing else.
This is only for the official Pi camera. It won't affect USB cameras.

dynamitemedia
Posts: 30
Joined: Sun Dec 01, 2013 9:12 pm

Re: Official V4L2 driver

Tue Dec 03, 2013 9:26 pm

ok thanks!

i have 4 logitech's sitting here is why i haven't gotten a camera module yet, plus wasn't sure the cam module recorded audio

unit_vector
Posts: 3
Joined: Tue Dec 03, 2013 10:34 pm

Re: Official V4L2 driver

Tue Dec 03, 2013 11:19 pm

After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5268
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Official V4L2 driver

Wed Dec 04, 2013 12:50 am

unit_vector wrote:After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.
Do raspistill/raspivid currently work?

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Wed Dec 04, 2013 10:12 am

Very promising first test results. Looks good.

I was able to stream 10fps at 640x480 with motion. That is quite useable.

I am using palette 8 (V4L2_PIX_FMT_YUV420 : 8 'YU12').
The only other palette, which I got working was 6.

It would be great is mjpeg or jpeg would work as then motion would not need to do the conversion:
[1] Test palette JPEG (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Device or resource busy
[1] Config palette index 3 (JPEG) doesn't work.
[1] Supported palettes:
[1] 0: YU12 (4:2:0, packed YUV)
[1] 1: YUYV (4:2:2, packed, YUYV)
[1] 2: BGR3 (RGB24 (BE))
[1] 3: JPEG (JPEG)
[1] 4: H264 (H264)
[1] Selected palette YU12
CPU seems to be the limiting factor and I had to set jpeg quality to 90% and then I have 75% cpu load (motion takes 100% when I try with 100% quality).
I have 128M memory for GPU and CPU is overclocked to 900 MHz. And I have disabled motion's motion detection (because I use zoneminder).

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Wed Dec 04, 2013 11:07 am

I increased fps from 10 to 15 and now I have 95 % CPU usage, but it has worked over an hour, so looks reliable.
I am getting these message (dmesg) now and then:
[ 3074.023234] bcm2835-v4l2: error 0 waiting for frame completion
[ 3077.183462] bcm2835_v4l2: port_parameter_get:result:0 component:0x40 port:3 parameter:13
[ 3251.487127] bcm2835-v4l2: error 0 waiting for frame completion
[ 4041.164435] bcm2835_v4l2: port_parameter_get:result:0 component:0x40 port:2 parameter:13
[ 6494.448669] bcm2835-v4l2: error 0 waiting for frame completion
[ 6498.198919] bcm2835_v4l2: port_parameter_get:result:0 component:0x40 port:2 parameter:13

Are they ok?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6895
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Wed Dec 04, 2013 11:57 am

As one of those behind this driver, I'll stick my head above the parapet!

Based on the comments so far, it sounds like we've got most stuff right - there are so many apps that could use V4L2 that it was tricky to find a representative set. It sounds like Motion and guvcview are the most useful ones that people have been using so far.

Issues I've seen raised:
- JPEG compression quality should be 1-100, not 0-100. Easy fix.
- Extra controls to mimic those exposed with raspistill. Do-able if they are available through the V4L2 API. The initial aims were to get the basic framework running with some parameters. Adding extras should be relatively trivial (extra array values in controls.c). AWB modes (we don't do full colour temperature control - sorry), and manual exposure should be simple so I'll give it a shot.
- MJPEG support. Two approaches that I could take on the GPU. I need to check which is the most appropriate but should be possible.
- Crash on quitting guvcview. This looks to be due to framerate control or lack of it. I started looking at framerate, but V4L2 doesn't make it trivial (it is a time per frame value, but also set through a slightly weird route). I'll look again at adding fps.
- Two output devices for differing resolutions. Sorry, that one is unlikely to fly if we try to do it in a generic way. We may be able to juggle it so that you could get the raw images from one device, H264 encoded data as a second, and JPEG as a third device. There would be restrictions on resolution between the first two (they would need to be the same), and enforcing that through V4L2 may be tricky.

For info, the overlay is done solely on the GPU, and almost totally in hardware, so there should be no significant impact on system loading (extra SDRAM bandwidth, but that's about it).

Please be aware that this has dropped back to being a background project on the Broadcom side, so extra work will be dependent on finding time. Whilst I'd love to do Pi stuff out of office hours, a small girl generally demands Daddy time, so it's unlikely to happen.

Dave S
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

mikerr
Posts: 2768
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Official V4L2 driver

Wed Dec 04, 2013 12:44 pm

I've got this working now in motion and just via v4l2-ctl,
but Is there device support in this ( /dev/video ) ?

I'm looking to use it with python/openCV and that seems to want device access,
unless anyone knows another python access method?

[edit] I'm an idiot, seems raspistill had frozen when I was testing the driver.
Last edited by mikerr on Thu Dec 05, 2013 9:48 am, edited 1 time in total.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Wed Dec 04, 2013 12:51 pm

mikerr wrote:I've got this working now in motion and just via v4l2-ctl,
but Is there device support in this ( /dev/video ) ?

I'm looking to use it with python/openCV and that seems to want device access,
unless anyone knows another python access method?
How do you use motion without /dev/video0 ?
My tests were done using the normal /dev/video0, so it definitely works.

$ grep videodev motion.conf
videodevice /dev/video0

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 1:18 pm

6by9 wrote:- Extra controls to mimic those exposed with raspistill. Do-able if they are available through the V4L2 API. The initial aims were to get the basic framework running with some parameters. Adding extras should be relatively trivial (extra array values in controls.c). AWB modes (we don't do full colour temperature control - sorry), and manual exposure should be simple so I'll give it a shot.
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.

Can you say what the timestamps correspond to? What generates them? How precise are they?

And then eventually more resolution modes including binned full frame (with information about what is scaled when if at all).

That would make me very happy.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6895
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Wed Dec 04, 2013 1:59 pm

Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.
I'm restricted by what V4L2 can expose and control. We do have a good relationship with one of the V4L2 maintainers, so can and do ask questions when necessary. I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Can you say what the timestamps correspond to? What generates them? How precise are they?
All our buffers should have V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC set in the flags field. Refer to the V4L2 docs and you'll see the definition "The buffer timestamp has been taken from the CLOCK_MONOTONIC clock. To access the same clock outside V4L2, use clock_gettime(2) "
Each frame off the GPU is timestamped by the GPU when the frame start was received by the CSI2 receiver, which is pretty much synonymous with end of exposure for the first pixel of the frame. For accuracy, we get the kernel time and the GPU time at start_streaming. For each frame we then take (frame GPU timestamp) - (start GPU timestamp) + (start kernel timestamp) to end up with as accurate a number as we can for V4L2. Resolution is usecs.
And then eventually more resolution modes including binned full frame (with information about what is scaled when if at all).
Well those are common between Rapsistill/vid and the V4L2 driver. If James has the time to sort them in the GPU code, then the V4L2 driver will automatically use them.


If people want features, PLEASE check whether V4L2 can support them first, and ideally provide the references into the docs for the V4L2 controls that are of interest. We're not going to start adding or requesting features to the V4L2 framework, so if it doesn't support it, we won't support it.
Thanks.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 2:48 pm

6by9 wrote:
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.
I'm restricted by what V4L2 can expose and control. We do have a good relationship with one of the V4L2 maintainers, so can and do ask questions when necessary. I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
This is the respective commit by JamesH to raspivid: https://github.com/raspberrypi/userland/commit/12446df9

Looks like MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER is the parameter.

I think it can be switched on at all times? it's only a minor amount of bytes added to the bitstream.
Can you say what the timestamps correspond to? What generates them? How precise are they?
All our buffers should have V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC set in the flags field. Refer to the V4L2 docs and you'll see the definition "The buffer timestamp has been taken from the CLOCK_MONOTONIC clock. To access the same clock outside V4L2, use clock_gettime(2) "
Each frame off the GPU is timestamped by the GPU when the frame start was received by the CSI2 receiver, which is pretty much synonymous with end of exposure for the first pixel of the frame. For accuracy, we get the kernel time and the GPU time at start_streaming. For each frame we then take (frame GPU timestamp) - (start GPU timestamp) + (start kernel timestamp) to end up with as accurate a number as we can for V4L2. Resolution is usecs.
Sweet. Sounds good.
And then eventually more resolution modes including binned full frame (with information about what is scaled when if at all).
Well those are common between Rapsistill/vid and the V4L2 driver. If James has the time to sort them in the GPU code, then the V4L2 driver will automatically use them.
Cool thanks.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 2:57 pm

6by9 wrote:I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Ah you meant on the V4L side. I think so, yes. But if this control rally needs exposing I don't know.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6895
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Wed Dec 04, 2013 3:00 pm

towolf wrote:
6by9 wrote:
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.
I'm restricted by what V4L2 can expose and control. We do have a good relationship with one of the V4L2 maintainers, so can and do ask questions when necessary. I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
This is the respective commit by JamesH to raspivid: https://github.com/raspberrypi/userland/commit/12446df9

Looks like MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER is the parameter.
It's MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER in the MMAL world. It is what from the V4L2 world that should map on to it that I need to know, hence quoting a V4L2 control enum. You should be able build all the V4L2 docs from the normal kernel tree by doing "make htmldocs". In there are the list of all the V4L2 enums that are defined with descriptions.
towolf wrote:I think it can be switched on at all times? it's only a minor amount of bytes added to the bitstream.
It can't be turned on at all times. Many (admittedly badly written) container writers will fall over if they get a new set of buffer headers in the middle of a stream - I know that from experience!
If it can't be controlled from the V4L2 world, then I'm afraid we can't implement it.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5268
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Official V4L2 driver

Wed Dec 04, 2013 3:04 pm

towolf wrote:
6by9 wrote:I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Ah you meant on the V4L side. I think so, yes. But if this control rally needs exposing I don't know.
I think V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER is the right control.

I think it should be optional. There's a standard V4L2 control for this (so other people think making it optional is the right thing to do).
The majority of use cases don't require it, and for low bitrate videos (e.g. security cam with little changing most of the time) duplicating SPS/PPS every frame is likely to be a significant fraction of bitrate.

unit_vector
Posts: 3
Joined: Tue Dec 03, 2013 10:34 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 4:19 pm

dom wrote:
unit_vector wrote:After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.
Do raspistill/raspivid currently work?
I'm not sure why I didn't think to try that, but I bet raspistill wouldn't have worked either at the time. Trying raspistill now, after letting it sit unused overnight and checking the connections, both raspistill and modprobe worked as expected and everything seems fine. It was possibly a loose cable connection or static buildup or something. Thanks for the reply!

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 4:40 pm

dom wrote:
towolf wrote:
6by9 wrote:I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Ah you meant on the V4L side. I think so, yes. But if this control rally needs exposing I don't know.
I think V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER is the right control.

I think it should be optional. There's a standard V4L2 control for this (so other people think making it optional is the right thing to do).
The majority of use cases don't require it, and for low bitrate videos (e.g. security cam with little changing most of the time) duplicating SPS/PPS every frame is likely to be a significant fraction of bitrate.
Ok, I went and asked the committer. If the guy from CC responds I will paste here too.
[email protected][...]o.com wrote:While initially intended for MPEG 1/2/4 sequence headers, it makes
perfect sense to me to extend this to H264 as well. The documentation
would have to be extended to include H264, though.

Note: the latest spec is always available here:

http://hverkuil.home.xs4all.nl/spec/media.html

It's rebuilt daily.

I've CC-ed Kamil D[...], who is more of an expert on H264 than I am.
Kamil, do you see any reason why the V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER
can't be extended to cover H264?

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 5:08 pm

k.d[...][email protected][...]g.com> wrote:I think that the control you mentioned should be a good fit.
Judging by the documentation and your needs you should use it. Changing the
documentation not to list specific codecs would be a great idea.

The following presentation may help you while writing the v4l2 codec driver.
It is a presentation I gave on LinuxCon Europe 2012.
https://events.linuxfoundation.org/imag ... debski.pdf

In addition, you may find these example/test applications handy.
http://git.infradead.org/users/kmpark/public-apps

User avatar
CopterRichie
Posts: 131
Joined: Tue Mar 26, 2013 3:14 am
Location: Los Angeles CA.

Re: Official V4L2 driver

Wed Dec 04, 2013 5:22 pm

Will this driver eventually end up in the repository requiring nothing more than apt-get install?

Return to “Camera board”