dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5268
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Official V4L2 driver

Wed Dec 04, 2013 5:28 pm

CopterRichie wrote:Will this driver eventually end up in the repository requiring nothing more than apt-get install?
Yes, consider this a beta. The next sdcard image will include the kernel updates, and a recent version of v4l-utils installed.
We'll also make these available with apt-get install/uprade.
No doubt there will be a blog post when this happens.

unit_vector
Posts: 3
Joined: Tue Dec 03, 2013 10:34 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 6:03 pm

unit_vector wrote:
dom wrote:
unit_vector wrote:After installing everything, when I type "sudo modprobe bcm2835-v4l2" the operation doesn't complete - doesn't return to the prompt until I ctrl-C. Everything seemed to go OK up to this point. Any suggestions? The only complicating factor might be that I previously had the 3rd party userspace v4l drivers installed and working, then I uninstalled them before trying this.
Do raspistill/raspivid currently work?
I'm not sure why I didn't think to try that, but I bet raspistill wouldn't have worked either at the time. Trying raspistill now, after letting it sit unused overnight and checking the connections, both raspistill and modprobe worked as expected and everything seems fine. It was possibly a loose cable connection or static buildup or something. Thanks for the reply!
After further testing it seems that calling modprobe does not work for me unless I call raspistill first. If I try calling modprobe first and cancel it, even raspistill will not work. Are others able to get this to work by calling modprobe right after booting before using raspistill?

Assuming I can get that to work, what would be the best/recommended way to enable V4L for the camera on startup?

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Wed Dec 04, 2013 6:22 pm

6by9 wrote:- Two output devices for differing resolutions. Sorry, that one is unlikely to fly if we try to do it in a generic way. We may be able to juggle it so that you could get the raw images from one device, H264 encoded data as a second, and JPEG as a third device. There would be restrictions on resolution between the first two (they would need to be the same), and enforcing that through V4L2 may be tricky. [...]
Dave S
First, thank you very much for your work on this! Getting the standard /dev/video0 device for the R-Pi camera is a very nice step forward for the system. Hopefully we'll be able to get applications like 'motion' able to accept JPEG or even MJPEG from /dev/video0, as well as what works now which is YU12, BGR3, RGB3.

It sounds like two video devices are at least not out of the question, which is hopeful. By "raw images" I presume you mean uncompressed YU12, and not pre-bayer really raw data(?) Regarding the resolution issue, I gather that many webcams offer only a few pre-set resolutions anyway. If you have video0 and video1 which must match resolution, maybe you simply don't accept changes to resolution of video1, and just use video0 as the one with settable resolution and that sets both. Would the three devices you mention correspond to the internal MMAL structure having Preview, Video, and Still output ports?

From the work done by 'waveform80' on his Pure Python camera interface I gather you can connect either a H.264 encoder or a JPEG encoder to the "video" output port? Although the still output port apparently has (at least) more time-domain noise filtering so it is better optimized for still image quality, than the video port.

At any rate the use-case I'm most interested in is this one:
  1. A. Real-time processing of YU12 or RGB3 frames at low resolution with at least 10 fps on the ARM, and
    B. Save select high-resolution JPEGS with lower frame rate, but near-zero latency depending on the results of step A.
Right now I can do either (A) or (B) separately, but not both at the same time. If there's a way to do both with the current interface options, I don't know how. I understand the JPEG resolution might be limited to 1920x1080 (or 1920 x 1088?) due to camera limitations in the video mode.

User avatar
0xFF
Posts: 202
Joined: Tue Nov 20, 2012 7:52 pm
Location: Poland

Re: Official V4L2 driver

Wed Dec 04, 2013 9:03 pm

Great news!
Santa came early this year ;-)
I didn't try code yet, but I would like to thank you for your effort.

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver - initialization bug?

Wed Dec 04, 2013 10:03 pm

I did exactly what g7ruh did, and got exactly the same error he did ( v4l2-ctl: error while loading shared libraries: libv4l2.so.0: cannot open shared object file: No such file or directory ). http://www.raspberrypi.org/phpBB3/viewt ... 28#p463481

So the full V4L2 install process from a "clean slate" Raspbian install (after rpi-update) seems to be this:

Code: Select all

sudo apt-get install autoconf gettext libtool libjpeg62-dev libv4l-dev
git clone git://git.linuxtv.org/v4l-utils.git
cd v4l-utils
autoreconf -vfi
./configure
make
sudo make install
So far, so good with no errors. Unfortunately, now when I do 'sudo modprobe bcm2835-v4l2' the camera board LED flashes briefly and goes out, and the system freezes. Control-C restores the command prompt, but the camera module is not active (no /dev/video0 device), moreover, raspistill no longer works either, and freezes such that Control-C does not work. From another window 'ps aux' shows: [raspistill] <defunct> and it connot be killed even with 'sudo kill -9 [pid]'. Time for a reboot.

After reboot, I can do

Code: Select all

raspistill -o test.jpg
sudo modprobe bcm2835-v4l2
v4l2-ctl --overlay=1
without error. So I have confirmed unit_vector's experience that 'raspistill' must be run before the modprobe call, for it to work.

UPDATE: At first I thought that the camera LED setting in /boot/config.txt file made the difference, but it seems it does not. Looks to me now like there is some camera device initialization which must done to the camera by 'raspistill' or 'raspivid' (or other MMAL application like motion-mmal) before doing the modprobe bcm2835-v4l2 step. This initialized state survives through a reboot (!) but NOT through a power cycle.

UPDATE2: On another R-Pi board I was able to add 'bcm2835-v4l2' to /etc/modules so it is automatically loaded at boot, and that worked (probably because I hadn't power-cycled it since using raspistill). However on this (fresh-install, power-cycled) R-Pi, that setup causes the device to permanently hang during boot:

Code: Select all

[info] Loading kernel module snd-bcm2835.
[info] Loading kernel module bcm2835-v4l2.
[   14.740230] media: Linux media interface: v0.10
[   14.783922] Linux video capture interface: v2.00
[   15.833319] bcm2835_v4l2: error 0 waiting for sync completion
Fortunately I was able to use the NOOBS "rescue system" login to disable (rename) my /etc/modules file and recover from this dead-end:

Code: Select all

recovery login: root
password: raspberry
# mkdir /mnt2
# mount /dev/mmcblk0p6 /mnt2
# cd /mnt2/etc
# mv modules modules_old_nogood
(...and reboot)

seumas
Posts: 4
Joined: Wed Dec 04, 2013 11:08 pm

Re: Official V4L2 driver

Wed Dec 04, 2013 11:32 pm

Hello,

Since you were asking for results from various V4L software I figured I'd share this:

Code: Select all

[email protected]:~# ffmpeg -f v4l2 -i /dev/video0 -s 640x480 -c x264  /pi/test.x264
ffmpeg version N-58743-g8dd4f5e Copyright (c) 2000-2013 the FFmpeg developers
  built on Dec  4 2013 16:20:48 with gcc 4.6 (Debian 4.6.3-14+rpi1)
  configuration: --enable-shared --enable-gpl --prefix=/usr/local --enable-nonfree --enable-libx264 --enable-version3 --disable-mmx --cc=arm-linux-gnueabihf-gcc --cxx=arm-linux-gnueabihf-g++
  libavutil      52. 56.100 / 52. 56.100
  libavcodec     55. 45.100 / 55. 45.100
  libavformat    55. 22.100 / 55. 22.100
  libavdevice    55.  5.102 / 55.  5.102
  libavfilter     3. 91.100 /  3. 91.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
[video4linux2,v4l2 @ 0x679d50] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
/dev/video0: Inappropriate ioctl for device
The "inappropriate ioctl for device" error pops up even if I trim the command down to "ffmpeg -f v4l2 -i /dev/video0".

Motion streaming works as expected. The above ffmpeg was built today from a git clone via a distcc server running a cross-compiler.

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Thu Dec 05, 2013 12:22 am

seumas wrote:[video4linux2,v4l2 @ 0x679d50] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
/dev/video0: Inappropriate ioctl for device
For what it's worth, that error was reported with a different device (Easycap DC60 card) and driver (stk1160) as a possible FFmpeg bug in this thread,
http://www.mail-archive.com/ffmpeg-trac ... 13274.html
http://www.mail-archive.com/ffmpeg-trac ... 13315.html
So I think ffmpeg or stk1160 driver has init wrong. After I set
driver=v4l2:width=720:height=576:norm=PAL:outfmt=uyvy

then it worked.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Thu Dec 05, 2013 7:09 am

I have not run raspistill and I can do modprobe without issues.
I put it into rc.local and it has worked ever since. I had the camera and motion running the whole nigh without issues.
Good work indeed! Thanks a lot. (Now I just need to check how I can better quality in low light).

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Thu Dec 05, 2013 3:02 pm

shuckle wrote:I have not run raspistill and I can do modprobe without issues.
I put it into rc.local and it has worked ever since. I had the camera and motion running the whole nigh without issues.
It's natural to assume a reboot clears everything, but that was not my experience. Maybe I have a hardware fault, but I found that only if your actually remove power from the device, then reapply, then you try to do the v4l2 modprobe before (raspistill/raspivid/other MMAL app), the device will lock up. Be careful because if you make it part of your bootup you can be locked out of your machine, as I was.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Official V4L2 driver

Thu Dec 05, 2013 3:38 pm

I just happened to move it today to a new place with new power source, so it definitely was cold. And it still worked. :)
I could try with a new camera board but not today.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22747
Joined: Sat Jul 30, 2011 7:41 pm

Re: Official V4L2 driver

Thu Dec 05, 2013 3:53 pm

I think, having read some of the comments here, that it's worth reiterating that this driver doesn't provide any more underlying features than raspistill or vid, since its uses the same (or very similar) ISP code on the GPU. So don't expect to suddenly get long exposures, or HW MJPEG out of the system, or whatever feature you are currently missing from the demo apps.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

ethanol100
Posts: 583
Joined: Wed Oct 02, 2013 12:28 pm

Re: Official V4L2 driver

Thu Dec 05, 2013 4:20 pm

I was hoping for one new feature: I would like to record video and audio.

I thought it would be simple just to capture the "H264" stream and audio from an external usb soundcard.
But I have no luck.

Is there any program, that can use the "H264" stream and capture audio at the same time putting both into one container?

I have tried:

ffmeg/avconv: avconv -f video4linux2 -list_formats all -i /dev/video0

Code: Select all

[video4linux2 @ 0x178d460] R :   yuv420p :    4:2:0, packed YUV :
[video4linux2 @ 0x178d460] R :   yuyv422 :  4:2:2, packed, YUYV :
[video4linux2 @ 0x178d460] R :     bgr24 :           RGB24 (BE) :
[video4linux2 @ 0x178d460] R : Unsupported :                 JPEG :
[video4linux2 @ 0x178d460] R : Unsupported :                 H264 :
So it does not recognize the H264 and I have found no way to use it in avconv

cvlc: Can stream and can save h264 to disk

Code: Select all

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=4
v4l2-ctl --set-ctrl video_bitrate=1000000
cvlc -v v4l2:///dev/video0:chroma="H264":width=1920:height=1080:fps=30 --sout="#rtp{sdp=rtsp://:8554/live}"
cvlc -v v4l2:///dev/video0:width=1920:height=1080:fps=30  :input-slave:"alsa://hw:1,0" --sout  file/avi:stream.avi
but the input-slave is not working the output file only contains the "H264" stream and no audio.
And using cvlc -v alsa://hw:1,0 with :input-slave:v4l2 creates only audio stream...

streamer/xawtv: both can not handle "H264". But work with YUV2 with strange color red/blue exchanged.

So which software could I use to capture H264 and alsa?

There is one feature we get from the v4l2 driver, 8 addidtional pixel in height. And I have no luck to set it to 1080 pixel.

Ps. often the driver has some problems: Trying to process the stream with avconv or ffmpeg, often the module reports:

Code: Select all

bcm2835_v4l2: error 0 waiting for sync completion
vidioc_s_fmt_vid_cap: failed to setup mmal components: -62
and only a reboot fixes this, removing the module and reinsert will fail with a timeout error message. And the camera is blocked until reboot.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6901
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Dec 05, 2013 5:02 pm

jbeale wrote:First, thank you very much for your work on this! Getting the standard /dev/video0 device for the R-Pi camera is a very nice step forward for the system. Hopefully we'll be able to get applications like 'motion' able to accept JPEG or even MJPEG from /dev/video0, as well as what works now which is YU12, BGR3, RGB3.
I'm looking at 'motion' accepting JPEG at the moment (just don't tell my boss!). Something is going wrong in setting the buffer sizes. Now need to track down what.
jbeale wrote:It sounds like two video devices are at least not out of the question, which is hopeful. By "raw images" I presume you mean uncompressed YU12, and not pre-bayer really raw data(?) Regarding the resolution issue, I gather that many webcams offer only a few pre-set resolutions anyway. If you have video0 and video1 which must match resolution, maybe you simply don't accept changes to resolution of video1, and just use video0 as the one with settable resolution and that sets both. Would the three devices you mention correspond to the internal MMAL structure having Preview, Video, and Still output ports?
Yes, by raw I mean YUV domain, not Bayer. There may be scope to add Bayer in, but that is even further down the priority list.
I'm still learning what is and isn't possible through V4L2 and how it hangs together. It may be possible to set video1 to not accept resolution changes, but we still need to setup buffer size changes. And what do we do if video1 is streaming and someone changes the resolution on video0? Those are the kind of issues that will trip us up.

Yes, the three ports would correspond to the 3 MMAL output ports of the camera component, although with tunnels set up on the GPU to add in the encoding when requested. If you read through the current driver, you'll see that
- output[0] tunnels to video_render for the overlay.
- output[1] is used for YU12/YUYV/RGB up to 1280x720, or H264 via a tunnel to video_encode.
- output[2] is used for YU12/YUYV/RGB above1280x720, or JPEG via a tunnel to image_encode.

If we split it out to 3 devices, then we'd do an internal splitter off output[0] to feed the overlay or the app as required.
I'm currently thinking that if we do this split then doesn't the app need to know which device to use when. So going this route loses some of the generic nature of just open the device, even if we do expand the feature set. Another issue for discussion.
jbeale wrote:From the work done by 'waveform80' on his Pure Python camera interface I gather you can connect either a H.264 encoder or a JPEG encoder to the "video" output port? Although the still output port apparently has (at least) more time-domain noise filtering so it is better optimized for still image quality, than the video port.
You could connect an H264 or JPEG encoder to any of the camera output ports.
You are correct that camera output[2] triggers a load of extra processing when producing the image, and if the JPEG encoder is connected there you should also get a full EXIF header with thumbnail. This path is limited in overall framerate though.
Similarly connecting the H264 encoder to camera output[1] has various performance tweaks to push more on to the hardware.
jbeale wrote:At any rate the use-case I'm most interested in is this one:
  1. A. Real-time processing of YU12 or RGB3 frames at low resolution with at least 10 fps on the ARM, and
    B. Save select high-resolution JPEGS with lower frame rate, but near-zero latency depending on the results of step A.
Right now I can do either (A) or (B) separately, but not both at the same time. If there's a way to do both with the current interface options, I don't know how. I understand the JPEG resolution might be limited to 1920x1080 (or 1920 x 1088?) due to camera limitations in the video mode.
This will again rely on creating multiple /dev/videoN devices, but it is all possible.
The GPU with the correct sensor setup receive 3-4MPix from the sensor, rescale to 1080P30 and H264 encode, and then produce a 3-4MPix JPEG in parallel (about 1-2fps), without dropping frames on the H264 stream. We even have parameters to control exactly which frame it is that you wish to encode (generally up to 4 frames before the current one). Getting YUV frames to the ARM is a little more involved than H264 (bigger frames), but 10fps should certainly be possible.
With the sensor on RPi, it is only reading 1920x1080 off the sensor, so there would be no point in producing a JPEG at a higher resolution than that - upscaling is always nasty!

One step at a time though - we will be investigating multiple devices, and we'll see what falls out of that.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6901
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Thu Dec 05, 2013 5:17 pm

ethanol100 wrote:streamer/xawtv: both can not handle "H264". But work with YUV2 with strange color red/blue exchanged.
Red/blue swapped will be it expecting YVYU instead of YUYV. I'll double check which order YUV2 is meant to be and that we get our bytes in the right order.
If I've got it wrong, then it is a simple tweak to bcm2835-camera.c to amend the table entry
{
.name = "4:2:2, packed, YUYV",
.fourcc = V4L2_PIX_FMT_YUYV,
.mmal = MMAL_ENCODING_YUYV,
.depth = 16,
.mmal_component = MMAL_COMPONENT_CAMERA,
},
to select MMAL_ENCODING_YVYU instead. (And, yes, UYVY and VYUY all exist too). It could either be our driver or the app which is getting them swapped - it is very easy to do.
ethanol100 wrote:There is one feature we get from the v4l2 driver, 8 addidtional pixel in height. And I have no luck to set it to 1080 pixel.
Something else already on our todo list. The hardware requires that buffers are a multiple of 32 on width and 16 on height. So the V4L2 buffer size has to accommodate that. There is a V4L2 clipping parameter that could be used to set the desired number of active pixels within that buffer. I haven't worked out if it valid to go backwards from setting a clipping rectangle and getting that to update the buffer size parameters. It doesn't seem great, but otherwise puts the onus on the app to know the restrictions and do the right thing, and that seems worse.

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Thu Dec 05, 2013 5:45 pm

6by9 wrote: I'm still learning what is and isn't possible through V4L2 and how it hangs together. It may be possible to set video1 to not accept resolution changes, but we still need to setup buffer size changes. And what do we do if video1 is streaming and someone changes the resolution on video0? Those are the kind of issues that will trip us up.
I'm no V4L2 expert, quite the contrary in fact. So far I've been using 'motion' but only really glanced at the codebase. My first impulse if there is a resolution-change on video0 while video1 is already in use, would be to simply not honor that request and issue some kind of appropriate error message. But apparently the way (single-video-device) webcams work now is that if you use v4l2-ctl to request an unsupported resolution, the driver modifies your request to the nearest resolution it does support. Also, you return -EBUSY if device is in use.

Below quote from a 2007 LWN.net article Video4Linux2 part 5b: format negotiation :
When the application wants to change the hardware's format for real, it does a VIDIOC_S_FMT call [....]
Unlike VIDIOC_TRY_FMT, this call cannot be made at arbitrary times. If the hardware is currently operating, or if it has streaming buffers allocated (a topic for yet another future installment), changing the format could lead to no end of mayhem. Consider what happens, for example, if the new format is larger than the buffers which are currently in use. So the driver should always ensure that the hardware is idle and fail the request (with -EBUSY) if not.
By the way, I think v4l2-ctl -V is supposed to report the current settings, but it does not seem to report resolution in JPEG mode. I also notice 8 pixels are cropped from the full image height (1944 -> 1936) when it does work (in YUYV format), I guess that is just the divisible-by-16 requirement.

Code: Select all

[email protected]:~$ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3
[email protected]:~$ v4l2-ctl -V
Format Video Capture:
        Width/Height  : 0/0
        Pixel Format  : 'JPEG'
        Field         : None
        Bytes per Line: 0
        Size Image    : 102400
        Colorspace    : SRGB
[email protected]:~$ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=1
[email protected]:~$ v4l2-ctl -V
Format Video Capture:
        Width/Height  : 2592/1936
        Pixel Format  : 'YUYV'
        Field         : None
        Bytes per Line: 5184
        Size Image    : 10036224
        Colorspace    : Broadcast NTSC/PAL (SMPTE170M/ITU601)
At any rate I'm very glad that these issues are at least under consideration, and thanks again for your efforts.

User avatar
pi3g
Authorised Reseller
Authorised Reseller
Posts: 89
Joined: Mon Nov 05, 2012 9:58 pm
Location: Germany
Contact: Website

Can't find bcm2835-v4l2

Thu Dec 05, 2013 9:43 pm

I have followed all the steps outlined in these posts, especially jbeale's nice summary, on a stock NOOBS+Raspbian with all updates installed. Apart from libreoffice there are no other special packages on this system. The whole compilation & install process yields no errors ...

Code: Select all

modprobe bcm2835-v4l2
fails:

FATAL: Module bcm2835-v4l2 not found.

I've tried reboot, etc. Thought briefly that me not having enabled the camera before compilation had been the problem, but no ... a make uninstall, followed by the complete procedure doesn't change a thing.

I've also tried searching for the module, and can't locate it anywhere on my SD.

Some lines which MAY be relevant from autoreconf -vfi:

Code: Select all

autoreconf: configure.ac: not using Gettext
autoreconf: running: libtoolize --copy --force
Do you have any suggestions for me? It's much appreciated, as I want to include motion in our new camera kit for christmas :-).

Image
picockpit.com - tools to make your life with the Pi a little bit easier

ethanol100
Posts: 583
Joined: Wed Oct 02, 2013 12:28 pm

Re: Official V4L2 driver

Thu Dec 05, 2013 9:47 pm

The kernel is only updated using rpi-update. So you need to install rpi-update and run it to get the latest firmware and the v4l2 module.

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Thu Dec 05, 2013 10:07 pm

ethanol100 wrote:often the driver has some problems: Trying to process the stream with avconv or ffmpeg, often the module reports:

Code: Select all

bcm2835_v4l2: error 0 waiting for sync completion
vidioc_s_fmt_vid_cap: failed to setup mmal components: -62
and only a reboot fixes this, removing the module and reinsert will fail with a timeout error message. And the camera is blocked until reboot.
Very interesting. "bcm2835_v4l2: error 0 waiting for sync completion" is what I consistently got (even with reboot) until I power-cycled and then ran raspistill before attempting modprobe bcm2835-v4l2, only after that did it work.

So both 'unit_vector' and I had that consistently, fixed by power-cycle, reboot and running raspistill. Meanwhile 'ethanol100' sees this "often", fixed by reboot only. And 'shuckle' and I presume the driver author, '6by9' do not see it at all. Could be a subtle initialization problem, maybe a logic timing race condition that varies between devices, or even with temperature? Or just some variation in the software setup and other processes running on our various boards.

@pi3g: You do want to be running the latest kernel (sudo rpi-update) which is currently

Code: Select all

[email protected]:~ $ uname -a
Linux raspberrypi 3.10.21+ #604 PREEMPT Mon Dec 2 20:17:09 GMT 2013 armv6l GNU/Linux

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6901
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Official V4L2 driver

Fri Dec 06, 2013 8:35 am

jbeale wrote:Very interesting. "bcm2835_v4l2: error 0 waiting for sync completion" is what I consistently got (even with reboot) until I power-cycled and then ran raspistill before attempting modprobe bcm2835-v4l2, only after that did it work.
I know there are a few error handling cases that aren't perfect, but I haven't seen that specific one. It is probably due to ffmpeg and avconv using V4L2 in subtly different way to other apps. Not necessarily wrong, just different.
We can investigate, but please remember this is a spare time project for me now, so it may be a little while.

Generally on any failure (particularly the current issue with 'motion' and JPEG that I'm looking at), "rmmod bcm2835-v4l2" does unload the driver cleanly and frees the camera on the GPU. There may be issues that can't be cleared that way, but it works for me. "modprobe bcm2835-v4l2" after to reload the driver, and the pair of commands are quicker than a reboot :)

As a little background, Raspistill/yuv/vid and this driver are all talking to the GPU over the same IPC mechanism (VCHIQ). The V4L2 driver is a different as we can't use our normal userspace MMAL library as the interface, so it's been rerolled in a more direct mode. There are minor differences between the two which have caused a few issues during development, and there are likely to be more lurking.

The V4L2 driver opens the sensor driver when it is loaded to confirm that the sensor is connected. It then powers it all back down again to allow others to access it. When you ask it to stream, it all gets powered back up again and starts grabbing images as requested. Stop streaming and we power it all back down again. Things can change whilst it is all powered down (you could have unplugged the sensor for a start - just don't at risk of blowing up the Pi or sensor!), so the reopen could fail. It's quite possible that the handling for that is missing, but that's only a guess at the moment.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Sat Dec 07, 2013 3:34 pm

Finally Arch Linux Arm for Rpi uploaded the latest kernel and I can test.

I find the latency improved compared to piping raspivid into Gstreamer (it ranges from 150 to 200 msec over two wireless links). Preserving the V4l2 timestamps via Gstreamer over RTP seems to work as well. There’s always a slight deviation from 30fps.

Here’s a gst-launch-1.0 RTP pipeline I’m testing with:

Sender:

Code: Select all

gst-launch-1.0 -e v4l2src do-timestamp=true ! video/x-h264,width=640,height=480,framerate=30/1 ! h264parse ! rtph264pay config-interval=1 ! gdppay ! udpsink host=192.168.178.20 port=5000
Receiver:

Code: Select all

gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! fpsdisplaysink sync=false text-overlay=false
h264parse is needed to insert sequence headers. Here’s how I filmed the clock to derive screen-to-screen latencies

User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Sat Dec 07, 2013 3:43 pm

towolf wrote:I find the latency improved compared to piping raspivid into Gstreamer (it ranges from 150 to 200 msec over two wireless links). Preserving the V4l2 timestamps via Gstreamer over RTP seems to work as well. There’s always a slight deviation from 30fps.
Great work! and a good portion of that latency could be contributed by the wireless links.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Official V4L2 driver

Sat Dec 07, 2013 3:51 pm

For some reason this doesn’t work. I wanted to try out switching to night mode during streaming:

Code: Select all

[email protected]:~# v4l2-ctl -c auto_exposure=0
[email protected]:~# v4l2-ctl -c auto_exposure=1
[email protected]:~# v4l2-ctl -c auto_exposure=2
VIDIOC_S_EXT_CTRLS: failed: Invalid argument
Error setting MPEG controls: Invalid argument
[email protected]:~# v4l2-ctl -c auto_exposure=3
VIDIOC_S_EXT_CTRLS: failed: Invalid argument
Error setting MPEG controls: Invalid argument

EDIT:
Maybe this more informative help text explains why:

Code: Select all

User Controls

                     brightness (int)    : min=0 max=100 step=1 default=50 value=50 flags=slider
                       contrast (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                     saturation (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                horizontal_flip (bool)   : default=0 value=0
                  vertical_flip (bool)   : default=0 value=0
                      sharpness (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                  color_effects (menu)   : min=0 max=15 default=0 value=0
                                0: None
                                1: Black & White
                                2: Sepia
                                3: Negative
                                4: Emboss
                                5: Sketch
                                6: Sky Blue
                                7: Grass Green
                                8: Skin Whiten
                                9: Vivid
                                10: Aqua
                                11: Art Freeze
                                12: Silhouette
                                13: Solarization
                                14: Antique
                                15: Set Cb/Cr
                         rotate (int)    : min=0 max=360 step=90 default=0 value=0
             color_effects_cbcr (int)    : min=0 max=65535 step=1 default=32896 value=32896

MPEG Encoder Controls

             video_bitrate_mode (menu)   : min=0 max=1 default=0 value=0 flags=update
                                0: Variable Bitrate
                                1: Constant Bitrate
                  video_bitrate (int)    : min=25000 max=25000000 step=25000 default=10000000 value=10000000

Camera Controls

                  auto_exposure (menu)   : min=0 max=3 default=0 value=0
                                0: Auto Mode
                                1: Manual Mode
             auto_exposure_bias (intmenu): min=0 max=16 default=8 value=8
                                0: -24 (0xffffffffffffffe8)
                                1: -21 (0xffffffffffffffeb)
                                2: -18 (0xffffffffffffffee)
                                3: -15 (0xfffffffffffffff1)
                                4: -12 (0xfffffffffffffff4)
                                5: -9 (0xfffffffffffffff7)
                                6: -6 (0xfffffffffffffffa)
                                7: -3 (0xfffffffffffffffd)
                                8: 0 (0x0)
                                9: 3 (0x3)
                                10: 6 (0x6)
                                11: 9 (0x9)
                                12: 12 (0xc)
                                13: 15 (0xf)
                                14: 18 (0x12)
                                15: 21 (0x15)
                                16: 24 (0x18)
      white_balance_auto_preset (menu)   : min=0 max=9 default=1 value=1
                                1: Auto
                                2: Incandescent
                                3: Fluorescent
                                4: Fluorescent H
                                5: Horizon
                                6: Daylight
                                7: Flash
                                8: Cloudy
                                9: Shade
            image_stabilization (bool)   : default=0 value=0
                iso_sensitivity (intmenu): min=0 max=4 default=0 value=0
                                0: 0 (0x0)
                                1: 100 (0x64)
                                2: 200 (0xc8)
                                3: 400 (0x190)
                                4: 800 (0x320)
         exposure_metering_mode (menu)   : min=0 max=2 default=0 value=0
                                0: Average
                                1: Center Weighted
                                2: Spot

JPEG Compression Controls

            compression_quality (int)    : min=0 max=100 step=1 default=30 value=30


User avatar
jbeale
Posts: 3441
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Official V4L2 driver

Sun Dec 08, 2013 12:36 am

Just for my own reference, here's what I've got now in /etc/rc.local to run motion at startup (at the end, just before 'exit 0'). For some reason I get rather overexposed images unless I shift the AE bias down (default = 8). I don't see that using raspistill, only V4L2 via /dev/video0.

Code: Select all

# do-nothing camera cmd to intialize MMAL system
/usr/bin/raspistill -t 500

# install V4L2 driver
sudo modprobe bcm2835-v4l2

# shift AGC setting to not overexpose
/usr/local/bin/v4l2-ctl --set-ctrl=auto_exposure_bias=2

cd /home/pi/motion
nohup /home/pi/motion/motion_mm -c /home/pi/motion/cmotion.conf &

oldnpastit
Posts: 34
Joined: Wed Dec 04, 2013 7:57 am

Waiting for sync completion....

Sun Dec 08, 2013 6:55 pm

A couple of wild guesses at the "waiting for sync completion" problem:

Guess 1: could it be related to overclocking the pi?

My settings are set to the default, no overclocking, and I don't see the problem; could someone with the problem post up their overclocking settings (if any).

Guess 2: could it be related to the camera board version? Mine says "Raspberry Pi Camera Rev 1.3".

For reference: my kernel version is 3.10.18+. The board is powered from a regular USB wall-wart, with an 802.11N wifi dongle plugged and nothing else going on (no HDMI, but I've had HDMI connected and not seen the problem).

Thanks.

tauceti
Posts: 3
Joined: Tue Dec 10, 2013 12:01 am

Re: Official V4L2 driver

Tue Dec 10, 2013 12:04 am

Noob Question:

What is the difference between the official V4L2 and the UV4L
http://www.linux-projects.org/modules/news/

Which one should I use?

Return to “Camera board”