Page 2 of 22

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 6:48 pm
by jamesh
I wouldn't expect many more options than you get with the raspistill/vid apps - the underlying GPU code is still the same - but it's good to get some sort of standard GUI.

No point in asking me about any issues you may have - I had nothing to do with the v4l driver!

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 7:17 pm
by towolf
jbeale, use the overlay preview from the v4l tool. Use guvcview -o (note the -o) only to control the settings.

Using the preview built into guvcview will be slow.

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 7:37 pm
by jbeale
I don't know if there is any plan for continuing work on the V4L2 driver, but just in case... is there any chance of getting, for example /dev/video0 and /dev/video1 where you can get two different resolutions at once? There are several use cases: low-res for realtime transport over network and/or CPU-intensive processing steps, and high-res for local display and archiving. When you have only one video device and switch between resolutions, you have a significant latency while the camera restarts and autoexposure get reset, etc. meanwhile the thing you wanted to record may be gone.

Also, thanks towolf; using

Code: Select all

v4l2-ctl --overlay=1 # enable viewfinder
guvcview -o
it works smoothly and uses nearly no CPU time. However then you don't have the option to capture and save a still JPEG from the application.

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 8:49 pm
by g7ruh
jbeale wrote:
fernandovg wrote:How to check it was installed ok?

If you don't see any error messages and find a valid JPEG file "test1.jpg" in your current directory, then the driver is installed and working.
I am working with a fresh install of Rasbian, updated software and firmware.

I get this error "v4l2-ctl: error while loading shared libraries: libv4l2.so.0: cannot open shared object file: No such file or directory"

the fix (or a fix) is to

Code: Select all

sudo apt-get install libv4l-dev
then I get the test1.jpg :)

Roger

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 9:02 pm
by towolf
Use

Code: Select all

configure --prefix=/usr

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 9:14 pm
by dynamitemedia
will this get rid of the choppy video issues? i tried the rpi-update and that didnt help, using the c290 webcam. which is odd cause it works fine in Gucview but nothing else.

i am going to test with a whole new install and this update and report back... and i just bought a beaglebone last night to use with my cam, gonna stink if this fixes it ! lol

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 9:22 pm
by dom
dynamitemedia wrote:will this get rid of the choppy video issues? i tried the rpi-update and that didnt help, using the c290 webcam. which is odd cause it works fine in Gucview but nothing else.
This is only for the official Pi camera. It won't affect USB cameras.

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 9:26 pm
by dynamitemedia
ok thanks!

i have 4 logitech's sitting here is why i haven't gotten a camera module yet, plus wasn't sure the cam module recorded audio

Re: Official V4L2 driver

Posted: Tue Dec 03, 2013 11:19 pm
by unit_vector
After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 12:50 am
by dom
unit_vector wrote:After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.
Do raspistill/raspivid currently work?

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 10:12 am
by shuckle
Very promising first test results. Looks good.

I was able to stream 10fps at 640x480 with motion. That is quite useable.

I am using palette 8 (V4L2_PIX_FMT_YUV420 : 8 'YU12').
The only other palette, which I got working was 6.

It would be great is mjpeg or jpeg would work as then motion would not need to do the conversion:
[1] Test palette JPEG (640x480)
[1] Error setting pixel format VIDIOC_S_FMT: Device or resource busy
[1] Config palette index 3 (JPEG) doesn't work.
[1] Supported palettes:
[1] 0: YU12 (4:2:0, packed YUV)
[1] 1: YUYV (4:2:2, packed, YUYV)
[1] 2: BGR3 (RGB24 (BE))
[1] 3: JPEG (JPEG)
[1] 4: H264 (H264)
[1] Selected palette YU12
CPU seems to be the limiting factor and I had to set jpeg quality to 90% and then I have 75% cpu load (motion takes 100% when I try with 100% quality).
I have 128M memory for GPU and CPU is overclocked to 900 MHz. And I have disabled motion's motion detection (because I use zoneminder).

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 11:07 am
by shuckle
I increased fps from 10 to 15 and now I have 95 % CPU usage, but it has worked over an hour, so looks reliable.
I am getting these message (dmesg) now and then:
[ 3074.023234] bcm2835-v4l2: error 0 waiting for frame completion
[ 3077.183462] bcm2835_v4l2: port_parameter_get:result:0 component:0x40 port:3 parameter:13
[ 3251.487127] bcm2835-v4l2: error 0 waiting for frame completion
[ 4041.164435] bcm2835_v4l2: port_parameter_get:result:0 component:0x40 port:2 parameter:13
[ 6494.448669] bcm2835-v4l2: error 0 waiting for frame completion
[ 6498.198919] bcm2835_v4l2: port_parameter_get:result:0 component:0x40 port:2 parameter:13

Are they ok?

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 11:57 am
by 6by9
As one of those behind this driver, I'll stick my head above the parapet!

Based on the comments so far, it sounds like we've got most stuff right - there are so many apps that could use V4L2 that it was tricky to find a representative set. It sounds like Motion and guvcview are the most useful ones that people have been using so far.

Issues I've seen raised:
- JPEG compression quality should be 1-100, not 0-100. Easy fix.
- Extra controls to mimic those exposed with raspistill. Do-able if they are available through the V4L2 API. The initial aims were to get the basic framework running with some parameters. Adding extras should be relatively trivial (extra array values in controls.c). AWB modes (we don't do full colour temperature control - sorry), and manual exposure should be simple so I'll give it a shot.
- MJPEG support. Two approaches that I could take on the GPU. I need to check which is the most appropriate but should be possible.
- Crash on quitting guvcview. This looks to be due to framerate control or lack of it. I started looking at framerate, but V4L2 doesn't make it trivial (it is a time per frame value, but also set through a slightly weird route). I'll look again at adding fps.
- Two output devices for differing resolutions. Sorry, that one is unlikely to fly if we try to do it in a generic way. We may be able to juggle it so that you could get the raw images from one device, H264 encoded data as a second, and JPEG as a third device. There would be restrictions on resolution between the first two (they would need to be the same), and enforcing that through V4L2 may be tricky.

For info, the overlay is done solely on the GPU, and almost totally in hardware, so there should be no significant impact on system loading (extra SDRAM bandwidth, but that's about it).

Please be aware that this has dropped back to being a background project on the Broadcom side, so extra work will be dependent on finding time. Whilst I'd love to do Pi stuff out of office hours, a small girl generally demands Daddy time, so it's unlikely to happen.

Dave S

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 12:44 pm
by mikerr
I've got this working now in motion and just via v4l2-ctl,
but Is there device support in this ( /dev/video ) ?

I'm looking to use it with python/openCV and that seems to want device access,
unless anyone knows another python access method?

[edit] I'm an idiot, seems raspistill had frozen when I was testing the driver.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 12:51 pm
by shuckle
mikerr wrote:I've got this working now in motion and just via v4l2-ctl,
but Is there device support in this ( /dev/video ) ?

I'm looking to use it with python/openCV and that seems to want device access,
unless anyone knows another python access method?
How do you use motion without /dev/video0 ?
My tests were done using the normal /dev/video0, so it definitely works.

$ grep videodev motion.conf
videodevice /dev/video0

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 1:18 pm
by towolf
6by9 wrote:- Extra controls to mimic those exposed with raspistill. Do-able if they are available through the V4L2 API. The initial aims were to get the basic framework running with some parameters. Adding extras should be relatively trivial (extra array values in controls.c). AWB modes (we don't do full colour temperature control - sorry), and manual exposure should be simple so I'll give it a shot.
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.

Can you say what the timestamps correspond to? What generates them? How precise are they?

And then eventually more resolution modes including binned full frame (with information about what is scaled when if at all).

That would make me very happy.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 1:59 pm
by 6by9
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.
I'm restricted by what V4L2 can expose and control. We do have a good relationship with one of the V4L2 maintainers, so can and do ask questions when necessary. I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Can you say what the timestamps correspond to? What generates them? How precise are they?
All our buffers should have V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC set in the flags field. Refer to the V4L2 docs and you'll see the definition "The buffer timestamp has been taken from the CLOCK_MONOTONIC clock. To access the same clock outside V4L2, use clock_gettime(2) "
Each frame off the GPU is timestamped by the GPU when the frame start was received by the CSI2 receiver, which is pretty much synonymous with end of exposure for the first pixel of the frame. For accuracy, we get the kernel time and the GPU time at start_streaming. For each frame we then take (frame GPU timestamp) - (start GPU timestamp) + (start kernel timestamp) to end up with as accurate a number as we can for V4L2. Resolution is usecs.
And then eventually more resolution modes including binned full frame (with information about what is scaled when if at all).
Well those are common between Rapsistill/vid and the V4L2 driver. If James has the time to sort them in the GPU code, then the V4L2 driver will automatically use them.


If people want features, PLEASE check whether V4L2 can support them first, and ideally provide the references into the docs for the V4L2 controls that are of interest. We're not going to start adding or requesting features to the V4L2 framework, so if it doesn't support it, we won't support it.
Thanks.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 2:48 pm
by towolf
6by9 wrote:
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.
I'm restricted by what V4L2 can expose and control. We do have a good relationship with one of the V4L2 maintainers, so can and do ask questions when necessary. I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
This is the respective commit by JamesH to raspivid: https://github.com/raspberrypi/userland/commit/12446df9

Looks like MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER is the parameter.

I think it can be switched on at all times? it's only a minor amount of bytes added to the bitstream.
Can you say what the timestamps correspond to? What generates them? How precise are they?
All our buffers should have V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC set in the flags field. Refer to the V4L2 docs and you'll see the definition "The buffer timestamp has been taken from the CLOCK_MONOTONIC clock. To access the same clock outside V4L2, use clock_gettime(2) "
Each frame off the GPU is timestamped by the GPU when the frame start was received by the CSI2 receiver, which is pretty much synonymous with end of exposure for the first pixel of the frame. For accuracy, we get the kernel time and the GPU time at start_streaming. For each frame we then take (frame GPU timestamp) - (start GPU timestamp) + (start kernel timestamp) to end up with as accurate a number as we can for V4L2. Resolution is usecs.
Sweet. Sounds good.
And then eventually more resolution modes including binned full frame (with information about what is scaled when if at all).
Well those are common between Rapsistill/vid and the V4L2 driver. If James has the time to sort them in the GPU code, then the V4L2 driver will automatically use them.
Cool thanks.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 2:57 pm
by towolf
6by9 wrote:I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Ah you meant on the V4L side. I think so, yes. But if this control rally needs exposing I don't know.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 3:00 pm
by 6by9
towolf wrote:
6by9 wrote:
Definitely inline pps/sps headers as in raspivid if possible. This is essential for streaming live. Or any other H264 encoding options. QP instead of bitrate would be appreciated as well.
I'm restricted by what V4L2 can expose and control. We do have a good relationship with one of the V4L2 maintainers, so can and do ask questions when necessary. I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
This is the respective commit by JamesH to raspivid: https://github.com/raspberrypi/userland/commit/12446df9

Looks like MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER is the parameter.
It's MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER in the MMAL world. It is what from the V4L2 world that should map on to it that I need to know, hence quoting a V4L2 control enum. You should be able build all the V4L2 docs from the normal kernel tree by doing "make htmldocs". In there are the list of all the V4L2 enums that are defined with descriptions.
towolf wrote:I think it can be switched on at all times? it's only a minor amount of bytes added to the bitstream.
It can't be turned on at all times. Many (admittedly badly written) container writers will fall over if they get a new set of buffer headers in the middle of a stream - I know that from experience!
If it can't be controlled from the V4L2 world, then I'm afraid we can't implement it.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 3:04 pm
by dom
towolf wrote:
6by9 wrote:I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Ah you meant on the V4L side. I think so, yes. But if this control rally needs exposing I don't know.
I think V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER is the right control.

I think it should be optional. There's a standard V4L2 control for this (so other people think making it optional is the right thing to do).
The majority of use cases don't require it, and for low bitrate videos (e.g. security cam with little changing most of the time) duplicating SPS/PPS every frame is likely to be a significant fraction of bitrate.

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 4:19 pm
by unit_vector
dom wrote:
unit_vector wrote:After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.
Do raspistill/raspivid currently work?
I'm not sure why I didn't think to try that, but I bet raspistill wouldn't have worked either at the time. Trying raspistill now, after letting it sit unused overnight and checking the connections, both raspistill and modprobe worked as expected and everything seems fine. It was possibly a loose cable connection or static buildup or something. Thanks for the reply!

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 4:40 pm
by towolf
dom wrote:
towolf wrote:
6by9 wrote:I will have a look for PPS/SPS controls in there. V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER looks plausible - do you agree?
Ah you meant on the V4L side. I think so, yes. But if this control rally needs exposing I don't know.
I think V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER is the right control.

I think it should be optional. There's a standard V4L2 control for this (so other people think making it optional is the right thing to do).
The majority of use cases don't require it, and for low bitrate videos (e.g. security cam with little changing most of the time) duplicating SPS/PPS every frame is likely to be a significant fraction of bitrate.
Ok, I went and asked the committer. If the guy from CC responds I will paste here too.
[email protected][...]o.com wrote:While initially intended for MPEG 1/2/4 sequence headers, it makes
perfect sense to me to extend this to H264 as well. The documentation
would have to be extended to include H264, though.

Note: the latest spec is always available here:

http://hverkuil.home.xs4all.nl/spec/media.html

It's rebuilt daily.

I've CC-ed Kamil D[...], who is more of an expert on H264 than I am.
Kamil, do you see any reason why the V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER
can't be extended to cover H264?

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 5:08 pm
by towolf
k.d[...][email protected][...]g.com> wrote:I think that the control you mentioned should be a good fit.
Judging by the documentation and your needs you should use it. Changing the
documentation not to list specific codecs would be a great idea.

The following presentation may help you while writing the v4l2 codec driver.
It is a presentation I gave on LinuxCon Europe 2012.
https://events.linuxfoundation.org/imag ... debski.pdf

In addition, you may find these example/test applications handy.
http://git.infradead.org/users/kmpark/public-apps

Re: Official V4L2 driver

Posted: Wed Dec 04, 2013 5:22 pm
by CopterRichie
Will this driver eventually end up in the repository requiring nothing more than apt-get install?