Official V4L2 camera driver

[RpiName]

Noob Question:

What is the difference between the official V4L2 and the UV4L
http://www.linux-projects.org/modules/news/

Which one should I use?

I'd go with UV4L. It's an userspace driver, so it won't crash your machine in case of problems. And it's also more performant if you give the process a real time scheduling policy.

[gsh]

I'd go with the one supported by Raspberry Pi...

[towolf]

Noob Question:

What is the difference between the official V4L2 and the UV4L
http://www.linux-projects.org/modules/news/

Which one should I use?

I'd go with UV4L. It's an userspace driver, so it won't crash your machine in case of problems. And it's also more performant if you give the process a real time scheduling policy.

Did you declare your conflict of interest?

[gsh]

Yeah,

I didn't declare mine either!

Is it really slower? Has anyone actually done any tests to confirm that?

Gordon

[oldnpastit]

It would be easier to comment on uv4l if the source code was available.

[jamesh]

It would be easier to comment on uv4l if the source code was available.

Well, not if you were just interested in performance and capability testing - what you have already should be sufficient for that.

[asb]

I've built a v4l-utils 1.0 deb for armhf wheezy (Raspbian) and added it to our foundation repos (archive.raspberrypi.org) so you should no longer need to manually build it. Just

Code: Select all

sudo apt-get update && sudo apt-get install v4l-utils

Please report back.

Also, version of the firmware in the repos is now new enough for you not to use rpi-update, if you prefer to apt-get update && apt-get upgrade.

[m_rx]

I've tried to "apt-get update + upgrade" twice and bricked 2 sdcard( image)s:
i don't even get the color-square (not to speak of the raspberry) when rebooting!!

it seems to break when trying to do something on the emergency image in /boot.

also, running hexxeh's rpi-update afterwards doesn't restore the sdcards bootability

[mcornils]

[attachment=0]screenshot-linphone-bria-small.png[/attachment]Hello,

first of all a big thank you to all those who made this (kernel) driver possible. This is an application report for linphone (the SIP softphone). We are using the command line client and version 3.5.2 on official Raspbian, connecting to our company sip phone client "Bria 3". Here, while H.263 video works reasonably well on a Logitech USB webcam, using the "official" camera gives a mostly-green picture (I will try to attach a screenshot of this behaviour). What kind of logs would be helpful for you regarding this?
Strangely enough, when I use the self-compiled H.264 (x264-based) plugin for linphone, the picture is not green but shows the real deal. However, since it is in no way hardware-accelerated, the performance on H.264, either with Logitech or with the official Broadcom cam board, is unacceptable for us.

How can we assist in getting this to work? We are mostly application-level software engineers and our video-format-kernel-fu is limited (at best).

Best Regards,
-Malte

** Log highlights: **

ortp-message-Setting video size 352x288
ortp-message-linphone_call_start_video_stream lc rotation:0

ortp-message-Using permissive algorithm
ortp-message-Limiting bitrate of video encoder to 486000 bits/s
ortp-message-parsing QCIF=2;CIF=2;VGA=2;CIF4=2;I=1;J=1;T=1
ortp-error-no such method on filter MSV4L2Capture, fid=16390 method index=0
ortp-message-Priority used: 99
ortp-message-Audio MSTicker setpriority() failed: Permission denied, nevermind.
ortp-message-alsa_open_r: opening default:1 at 8000Hz, bits=16, stereo=0
ortp-message-Driver is bm2835 mmal
ortp-message-v4l2: trying 176x144
ortp-message-v4lv2: YUV420P choosen
ortp-message-Size of webcam delivered pictures is 176x144
ortp-message-Setting sent vsize=176x144, fps=14.985000
ortp-message-ms_filter_link: MSV4L2Capture:0x13bf6d0,0-->MSPixConv:0x13ce978,0
ortp-message-ms_filter_link: MSPixConv:0x13ce978,0-->MSSizeConv:0x13db448,0
ortp-message-ms_filter_link: MSSizeConv:0x13db448,0-->MSTee:0x13ce8e0,0
ortp-message-ms_filter_link: MSTee:0x13ce8e0,0-->MSH263Enc:0x13d8408,0
ortp-message-ms_filter_link: MSH263Enc:0x13d8408,0-->MSRtpSend:0x13c0bd8,0
ortp-message-Codec bitrate set to 432120
ortp-message-Codec size set to w=176/h=144
ortp-message-msv4l2_thread starting
ortp-message-qmin=3 qmax=31
ortp-message-Call 0x13c1588: moving from state LinphoneCallConnected to LinphoneCallStreamsRunning

[6by9]

Just a little update for those still reading this thread. I've just sent a set of patches to be merged, including some of the features/fixes requested. Some could do with a little more testing, but it seemed an opportune time to get them out there.

- Add manual shutter speed control ("v4l2-ctl --set_ctrl=auto_exposure=1" for manual control, and then "v4l2-ctl --set_ctrl=exposure_time_absolute=<value desired in 100usecs>") , and correct EV values (V4L2 wants 1/1000ths and we weren't advertising this).
- Correct JPEG Q-factor range to 1-100, not 0-100.
- Fix the driver lockup if start_streaming failed. This was the reason that all set-fmt-video calls would then fail.
- Fix ISO controls. These were asking for incorrect values from the GPU.
- Support flicker avoidance - "v4l2-ctl --set_ctrl=power_line_frequency=[0-3]" for [off|50Hz|60Hz|auto].
- Add frame rate control - "v4l2-ctl --set-parm=<fps>"
- A couple of fixes to come closer to passing the conformance tests (I think it is one failure now).
- Support inline H264 headers as requested by towolf. This needs a GPU firmware update (about to be released) to work cleanly, but works anyway if you set the pixelformat as H264 before enabling it. "v4l2-ctl --set-ctrl=repeat_sequence_header=1"
- Add timestamping to JPEGs. This isn't as accurate as for other frames, but JPEG capture generally doesn't need that accuracy. I'll look at fixing the GPU firmware at some point.
- Fix an issue when reducing the JPEG capture resolution. This was what was initially tripping up Motion when generating JPEGs. Having fixed that I appear to be seeing something else weird with Motion and JPEG where it seg faults when decoding the JPEG. Something strange appears to be happening as the JPEG buffers produced are much smaller than I'd expect (they'll still have full EXIF and a thumbnail in there), and it may explain why it seg faults if the JPEG is corrupt. Starting the overlay before running motion produces more sensibly sized buffers, but I've broken my image and get illegal instructions in weird places now

These will hopefully be released in the next day or so (when popcornmix gets a chance). Sorry it's taken so long, but the day job takes precedence.

[6by9]

first of all a big thank you to all those who made this (kernel) driver possible. This is an application report for linphone (the SIP softphone). We are using the command line client and version 3.5.2 on official Raspbian, connecting to our company sip phone client "Bria 3". Here, while H.263 video works reasonably well on a Logitech USB webcam, using the "official" camera gives a mostly-green picture (I will try to attach a screenshot of this behaviour). What kind of logs would be helpful for you regarding this?
Strangely enough, when I use the self-compiled H.264 (x264-based) plugin for linphone, the picture is not green but shows the real deal. However, since it is in no way hardware-accelerated, the performance on H.264, either with Logitech or with the official Broadcom cam board, is unacceptable for us.

How can we assist in getting this to work? We are mostly application-level software engineers and our video-format-kernel-fu is limited (at best).

Best Regards,
-Malte

Hi Malte.
Well that has given us a fair amount of information to start with, so thanks.
Mainly green normally means the buffer is all 0, although your green is a little more vibrant than I'd expect from YUV(0,0,0), but that may just be the conversion to PNG.

First thing I'd ask you to do is confirm that your camera works with raspistill, or v4l2-ctl and turning on the overlay (v4l2-ctl --overlay=1). That just confirms that it isn't something silly in the hardware.
Second is to confirm the timestamps and sizes of the frames that you are getting back from the V4L2 driver.

I'm a little confused by your log, as part of it says "Setting video size 352x288", and then later it says "Size of webcam delivered pictures is 176x144". I suspect that 176x144 is what you're actually asking for, in which case we have a small issue at the moment that our hardware requires the buffer to be a multiple of 32 horizontally, and 16 vertically. 176 isn't a multiple of 32, so you'll actually get 192x144. It shouldn't all be green though. If you could try 352x288 then that may give a different result.

Otherwise, if there is a simple test setup that we can try, then I can try and find an hour or so to have a look through what is happening. Ping me a private message if there are any specifics that you can provide.

If we get the chance to develop this driver more, then we are intending to provide multiple subdevices (see the messages between jbeale and I), and we should be able to provide one stream of YUV buffers for the local display, and a ready encoded H264 version for you to send over to far end. Sadly that's going to be a little while away, but may be something to look forward to.

Dave

[shuckle]

Thank you very much. I hope you can tackle that mjpeg issue also soon.

[jamesh]

Just a little update for those still reading this thread. I've just sent a set of patches to be merged, including some of the features/fixes requested. Some could do with a little more testing, but it seemed an opportune time to get them out there.

- Add manual shutter speed control ("v4l2-ctl --set_ctrl=auto_exposure=1" for manual control, and then "v4l2-ctl --set_ctrl=exposure_time_absolute=<value desired in 100usecs>") , and correct EV values (V4L2 wants 1/1000ths and we weren't advertising this).
- Correct JPEG Q-factor range to 1-100, not 0-100.
- Fix the driver lockup if start_streaming failed. This was the reason that all set-fmt-video calls would then fail.
- Fix ISO controls. These were asking for incorrect values from the GPU.
- Support flicker avoidance - "v4l2-ctl --set_ctrl=power_line_frequency=[0-3]" for [off|50Hz|60Hz|auto].
- Add frame rate control - "v4l2-ctl --set-parm=<fps>"
- A couple of fixes to come closer to passing the conformance tests (I think it is one failure now).
- Support inline H264 headers as requested by towolf. This needs a GPU firmware update (about to be released) to work cleanly, but works anyway if you set the pixelformat as H264 before enabling it. "v4l2-ctl --set-ctrl=repeat_sequence_header=1"
- Add timestamping to JPEGs. This isn't as accurate as for other frames, but JPEG capture generally doesn't need that accuracy. I'll look at fixing the GPU firmware at some point.
- Fix an issue when reducing the JPEG capture resolution. This was what was initially tripping up Motion when generating JPEGs. Having fixed that I appear to be seeing something else weird with Motion and JPEG where it seg faults when decoding the JPEG. Something strange appears to be happening as the JPEG buffers produced are much smaller than I'd expect (they'll still have full EXIF and a thumbnail in there), and it may explain why it seg faults if the JPEG is corrupt. Starting the overlay before running motion produces more sensibly sized buffers, but I've broken my image and get illegal instructions in weird places now

These will hopefully be released in the next day or so (when popcornmix gets a chance). Sorry it's taken so long, but the day job takes precedence.

Hi David,

what firmware change was needed for inline headers - I thought that was added to the Raspi tree a couple of months ago? It certainly works with raspivid.

James

[dom]

Just a little update for those still reading this thread. I've just sent a set of patches to be merged, including some of the features/fixes requested. Some could do with a little more testing, but it seemed an opportune time to get them out there.

The patches are in latest rpi-update. Please update and test.

[6by9]

Hi David,

what firmware change was needed for inline headers - I thought that was added to the Raspi tree a couple of months ago? It certainly works with raspivid.

James

As per the comment.
The GPU code in video_encode previously only allowed that parameter to be set if the encoding format was already set as H264. That is an annoying limitation as you could set the parameter to true and then decide on the encoding format to H264. The GPU change just always allows that parameter to be set independent of codec. I checked with the codec guys first that the codec would be happy with that under all situations, and they assured me it would.
It went in on 9th Dec IIRC, and Dom is sorting a release at the moment, so the two changes should go hand in hand.

[jamesh]

Hi David,

what firmware change was needed for inline headers - I thought that was added to the Raspi tree a couple of months ago? It certainly works with raspivid.

James

As per the comment.
The GPU code in video_encode previously only allowed that parameter to be set if the encoding format was already set as H264. That is an annoying limitation as you could set the parameter to true and then decide on the encoding format to H264. The GPU change just always allows that parameter to be set independent of codec. I checked with the codec guys first that the codec would be happy with that under all situations, and they assured me it would.
It went in on 9th Dec IIRC, and Dom is sorting a release at the moment, so the two changes should go hand in hand.

Ah, that explains why it works in Raspivid. Ta.

[6by9]

Thank you very much. I hope you can tackle that mjpeg issue also soon.

It's on my list of things to look at, however our MJPEG encoder hasn't been used for a LONG time, so has probably bit-rotted. We'll have to see how bad it is and make a call from there. I know it took a moderate amount of work from Dom to get the MJPEG decoder back up and running, and he tends to have more time for Pi stuff than me.

[towolf]

Just a little update for those still reading this thread. I've just sent a set of patches to be merged, including some of the features/fixes requested. Some could do with a little more testing, but it seemed an opportune time to get them out there.

Thanks a lot. All of your additions and fixes are highly appreciated. Will test this weekend.

[jamesh]

Thank you very much. I hope you can tackle that mjpeg issue also soon.

It's on my list of things to look at, however our MJPEG encoder hasn't been used for a LONG time, so has probably bit-rotted. We'll have to see how bad it is and make a call from there. I know it took a moderate amount of work from Dom to get the MJPEG decoder back up and running, and he tends to have more time for Pi stuff than me.

I got the MJPEG encoder to build, but it didn't work. Might take a look over next couple of days.

[shuckle]

Thank you very much. I hope you can tackle that mjpeg issue also soon.

It's on my list of things to look at, however our MJPEG encoder hasn't been used for a LONG time, so has probably bit-rotted. We'll have to see how bad it is and make a call from there. I know it took a moderate amount of work from Dom to get the MJPEG decoder back up and running, and he tends to have more time for Pi stuff than me.

I got the MJPEG encoder to build, but it didn't work. Might take a look over next couple of days.

Very promising progress. Thanks a lot!

[6by9]

I got the MJPEG encoder to build, but it didn't work. Might take a look over next couple of days.

Very promising progress. Thanks a lot!

Good news for those after MJPEG - I have something working that is producing a nice stream of JPEGs. VLC doesn't like it (just displays the first frame, but does correctly get the clip duration as 10s long), but omxplayer is quite happy. v4l2-ctl claims to be getting 30fps at 1920x1080 from it too.
I'll get my GPU changes pushed and let those who handle releasing the firmware know, and likewise send them the patch for the V4L2 kernel driver.
I should say that I haven't added any mechanism for selecting MJPG-A or MJPG-B standards, so you're effectively just getting a stream of JPEGs with no EXIF at the moment. Seeing as I can't see a way to select this via V4L2, that may just be the way it has to remain - sorry. I don't know if it makes any significant difference anyway - I'm happy to be educated by someone with more information.

And you all thought Santa was due tomorrow night Happy Christmas!

[shuckle]

Excellent!
Just let me know when I can test it.

[dom]

Excellent!
Just let me know when I can test it.

Updated. Run rpi-update and give it a go.

[6by9]

Updated. Run rpi-update and give it a go.

Thanks Dom. I wasn't sure if you were working today as you weren't in the office.
If spending time at the in-laws gets too terrible, then I do have a Pi with me and will see what I can find out about VLC not liking the stream (I've also had a comment that Chrome didn't like it either). The MJPEG-A or MJPEG-B flavours may fair better.

[jamesh]

Did you add the option to raspivid to select H264 or MJPEG? Cannot look at the moment! I can add it tomorrow if it's working OK.