MarkDarcy
Posts: 44
Joined: Thu Sep 20, 2018 8:23 am

Behaviour of file handles returned by V4L2 has changed across OS releases.

Fri May 14, 2021 4:16 am

Hi,

I apologise if this is not the right forum to post this but there didn't seem to be anywhere else it seemed to fit. Please reposition as necessary.

I recently noticed the behaviour of file handles returned via V4L2 has changed. I say "recently"; however, it is not a recent kernel the change appears to have occurred in. The reason is that I don't run with the latest kernel versions is for stability purposes; I always run with versions a year or two old that are being patched less frequently (if at all).

My V4L2-based USB camera application has, until recently, been able to recover gracefully from the user unplugging the camera while it is streaming. Specifically, if the camera was unplugged when the application was blocked in this call:

Code: Select all

ioctl(nfdCamera, VIDIOC_DQBUF, pVidBuffer)

ioctl() would return -1 with errno as EIO, ENODEV, or EBADF. The system being used was:

Code: Select all

$ uname -a
Linux raspberrypi 4.14.50+ #1122 Tue Jun 19 12:21:21 BST 2018 armv6l GNU/Linux

$ lsb_release -irdc
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.4 (stretch)
Release:        9.4
Codename:       stretch

This system was being used on both Pi Zero's and Pi 3B+s.

Implementation of support for the Pi 4 caused a system upgrade to:

Code: Select all

$ uname -a
Linux raspberrypi 5.4.51-v7l+ #1333 SMP Mon Aug 10 16:51:40 BST 2020 armv7l GNU/Linux

$ lsb_release -irdc
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

With this system, if I unplug the camera during streaming then the call to ioctl(VIDIOC_DQBUF) just hangs. If I then interrupt the program with CTRL+C the call returns with EINTR. This situation is reproducible nearly 100% of the time.

Just to make sure, I booted the Pi Zero on the buster system and the Pi Zero also exhibited the problem. Returning the Pi Zero to the stretch system returned error reporting on the file handle to its original behaviour.

I did try an intermediary system just to see if I could pinpoint a little better where the difference in behaviour was introduced. This system:

Code: Select all

$ uname -a
Linux raspberry-1 4.19.66+ #1253 Thu Aug 15 11:37:30 BST 2019 armv6l GNU/Linux

$ lsb_release -irdc
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.11 (stretch)
Release:        9.11
Codename:       stretch

doesn't raise errors during ioctl(VIDIOC_DQBUF) about 50% of the times tested and 50% of the time it did.

It appears from a listing of the /dev directory that the change in behaviour occurred when the new V4L2 devices for interfacing with the GPU were introduced (/dev/video10 and above). Before these devices were introduced, ioctl(VIDIOC_DQBUF) raised errors as expected on device unplug, after these devices were introduced, ioctl(VIDIOC_DQBUF) just hangs until the program is interrupted.

Please could somebody confirm if this change in behaviour was by design? If it was (as opposed to a regression), what is the new method for detecting when camera devices have been unplugged during capture and forcing the blocked ioctl(VIDIOC_DQBUF) call to gracefully exit?

Thanks in advance.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11247
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Behaviour of file handles returned by V4L2 has changed across OS releases.

Fri May 14, 2021 6:20 am

All the /dev/videoN nodes are independent. /dev/video10-16 being added is more likely coincidence with changing kernel release (they were added in 4.19).

More likely things have been fixed in the mainline Linux kernel over time. Most USB webcams conform to the USB Video Class (UVC) spec, so use the uvcvideo driver. This is taken from the mainline kernel with no modifications.
I do recall some threads fixing UVC for race conditions on camera disconnect.

4.14 is ancient. 5.4 is largely dropped. 5.10 is the currently supported kernel for the Pi. 5.12 is the current latest mainline kernel.
Please test on 5.10 (available from the Raspberr Pi OS repos), and I'd suggest also comparing against an x86 box running a similar kernel (a standard Ubuntu 21.04 live image should get you a 5.10 kernel).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

MarkDarcy
Posts: 44
Joined: Thu Sep 20, 2018 8:23 am

Re: Behaviour of file handles returned by V4L2 has changed across OS releases.

Tue May 18, 2021 6:50 am

Hi,

Upgraded the Pi OS to the latest 5.10 build from the repositories:

Code: Select all

$ uname -a
Linux raspberrypi 5.10.17-v7l+ #1414 SMP Fri Apr 30 13:20:47 BST 2021 armv7l GNU/Linux

$ lsb_release -irdc
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

Observed the same behaviour as the 5.4 build so the problem doesn't appear to have been "fixed" in this latest build.

With regard to testing this application on i386, I don't know how practical that's going to be seeing as a whole raft of Pi-specific code has to be stripped from the code to even get it to compile.

As a result, it may be some while before I can report back on what a normal PC does.

You must test the V4L2 driver code with some kind of test program at your end during development, right?
I know it's an inconvenience, but would it be possible to confirm this behaviour with such a test program with a Pi at your end?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11247
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Behaviour of file handles returned by V4L2 has changed across OS releases.

Tue May 18, 2021 8:15 am

MarkDarcy wrote:
Tue May 18, 2021 6:50 am
Upgraded the Pi OS to the latest 5.10 build from the repositories:

Observed the same behaviour as the 5.4 build so the problem doesn't appear to have been "fixed" in this latest build.
uvcvideo was refactored in 2017/18 to clean up the handling of disconnect whilst open. https://lore.kernel.org/linux-media/201 ... xtens.net/ leading to https://github.com/torvalds/linux/commi ... 673a1b9ceb (first added in 5.0, but backported to 4.19)
The behaviour may have changed due to that, but it sequences the disconnect correctly.

You're relying on DQBUF blocking? Personally I'd switch to the non-blocking mode, and poll/select on the fd first. I suspect that the fd is flagging an exception, but that you don't get to see that in blocking mode.
https://www.kernel.org/doc/html/latest/ ... -qbuf.html
and https://www.kernel.org/doc/html/latest/ ... ure.c.html or https://github.com/6by9/yavta/blob/master/yavta.c#L2427 for examples of using non-blocking mode with select.
MarkDarcy wrote:With regard to testing this application on i386, I don't know how practical that's going to be seeing as a whole raft of Pi-specific code has to be stripped from the code to even get it to compile.

As a result, it may be some while before I can report back on what a normal PC does.

You must test the V4L2 driver code with some kind of test program at your end during development, right?
I know it's an inconvenience, but would it be possible to confirm this behaviour with such a test program with a Pi at your end?
The uvcvideo driver is unmodified from mainline Linux, therefore we don't do any additional testing on it.

There is v4l2-compliance under https://git.linuxtv.org/v4l-utils.git/t ... compliance, but it won't test unplugging of a camera as that isn't something that can be automated in a test environment. A high proportion of V4L2 devices aren't on hot-pluggable interfaces either.

I don't have an external USB webcam to hand to test with.

Mainline maintainers:

Code: Select all

$ ./scripts/get_maintainer.pl drivers/media/usb/uvc/uvc_video.c 
Laurent Pinchart <laurent.pinchart@ideasonboard.com> (maintainer:USB VIDEO CLASS)
Mauro Carvalho Chehab <mchehab@kernel.org> (maintainer:MEDIA INPUT INFRASTRUCTURE (V4L/DVB))
linux-media@vger.kernel.org (open list:USB VIDEO CLASS)
linux-kernel@vger.kernel.org (open list)
so an email to Laurent and linux-media (skip Mauro and linux-kernel) may be in order.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

MarkDarcy
Posts: 44
Joined: Thu Sep 20, 2018 8:23 am

Re: Behaviour of file handles returned by V4L2 has changed across OS releases.

Tue May 18, 2021 11:12 am

6by9 wrote:
Tue May 18, 2021 8:15 am
You're relying on DQBUF blocking? Personally I'd switch to the non-blocking mode, and poll/select on the fd first. I suspect that the fd is flagging an exception, but that you don't get to see that in blocking mode.
I do indeed poll the descriptor periodically to check if the device is still there when not capturing which is how I detect if the camera has been unplugged at other times.

The original reason for choosing blocking mode was that I did not want to be pre-empted while waiting in a select() loop which may result in a delay in returning DMA buffers to V4L2 so causing the camera to drop frames. There are other higher priority tasks that are running in different threads during capture that could cause the capture thread to be rescheduled. I did consider non-blocking mode as there were other considerations for which it would have been favourable. In the end, though, I decided against it. The problem with being pre-empted is that it is impossible to predict when, and for how long, it may happen. Thus, blocking mode was considered the safer option.

Thanks for the suggestion, though. I think I may code it up anyway and just see what happens. And thanks for the contact information for the maintainers.

Return to “Camera board”