User avatar
HermannSW
Posts: 3450
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Why does v4l2src change FOV starting at 41fps?

Mon Mar 26, 2018 10:18 pm

Running this command on Raspberry

Code: Select all

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480, framerate=41/1 ! jpegenc !  rtpjpegpay !  udpsink host=192.168.178.30 port=5200

and this command on laptop

Code: Select all

gst-launch -v udpsrc port=5200 !  application/x-rtp, encoding-name=JPEG,payload=26 !  rtpjpegdepay !  jpegdec ! autovideosink

streams Raspberry (v2) camera to laptop.

In case of framerate 40/1 or lower, I see this on laptop:
Image


Starting with 41/1 and higher FOV changed and this is shown on laptop:
Image


Can this behavior be controlled, either all framerates with 1st or all framerates with 2nd FOV?
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

ethanol100
Posts: 662
Joined: Wed Oct 02, 2013 12:28 pm

Re: Why does v4l2src change FOV starting at 41fps?

Mon Mar 26, 2018 10:28 pm

picamera documentation says mode 7 is only valid for 40<fps. The isp will select mode 4 for FPS<=40.

User avatar
HermannSW
Posts: 3450
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Why does v4l2src change FOV starting at 41fps?

Mon Mar 26, 2018 11:38 pm

Thanks, that explains it.

And is correct wrt 40fps side of mode 7 of v2 camera:

Code: Select all

7	640x480	4:3	40 < fps <= 90		x	 	Partial	2x2

But the other end is wrong, with Stretch version before 2018-03-13 this is correct !
(see this thread)

Code: Select all

7	640x480	4:3	40 < fps <= 120		x	 	Partial	2x2

For Stretch starting with 2018-03-13 this is correct:
(see this thread)

Code: Select all

7	640x480	4:3	40 < fps <= 180		x	 	Partial	2x2

In theory 200 is correct here, but above 185fps you get a lot of frame skips.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

User avatar
HermannSW
Posts: 3450
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Why does v4l2src change FOV starting at 41fps?

Wed Mar 28, 2018 7:36 am

Last night was the first time I was able to capture with 200fps without frame skips. But in that video there was a scene without change.

Another recording at 200fps was interesting, 200fps with frame skips in recording, but outside of generated .avi video. AND with timeoverlay buffer-time that gives millisecond resolution timestamps to the frames!

Code: Select all

$ raspivid -md 7 -w 640 -h 480 -t 17000 -fps 200 -pts t.pts -o - | gst-launch-1.0 -v fdsrc ! video/x-h264,width=640,height=480,framerate=200/1 ! h264parse ! avdec_h264 ! timeoverlay time-mode="buffer-time" ! avimux ! filesink location=t.avi

This command is problematic a bit, since I increased raspivid recoding time to 17s and only get a 3s of .avi video out, but it generated the video nicely illustrating the 0.005s delta time between successive frames:
Image


Next frame:
Image


Next frame:
Image


v4l2src is capping framerate at 90fps and is no option for 180fps/200fps videos.
In the evening I will try rpicamsrc:
https://github.com/thaytan/gst-rpicamsrc

I talked to Jan and he said that he took the maximal framerates from Raspberry at the time he did that work years ago.
So I verified that rpicamsrc is currently capping at 90fps as well, but since I had to build it myself, it should be possible to increase maximal framerate to 200fps for v2 camera mode 7 in order to match @6by9's raspivid changes with latest 2018-03-13-raspbian Stretch.


Here is analysis of timestamps generated by raspivid showing that even slightly more than 200fps were generated (less than 5000μs frame delta time).

This is for the run without any frame skips:

Code: Select all

pi@raspberrypi3Bplus:~/gst-rpicamsrc $ ../ptsanalyze t.pts 0
creating tstamps.csv
634 frames were captured at 200fps
frame delta time[us] distribution
      1 4978
      1 4980
      6 4982
     11 4983
     91 4984
    264 4985
    199 4986
     45 4987
      9 4988
      1 4989
      1 4990
      1 4991
      1 4993
after skip frame indices (middle column)

This is for the video shown above with frame skips:

Code: Select all

pi@raspberrypi3Bplus:~/gst-rpicamsrc $ ../ptsanalyze t.pts 0
creating tstamps.csv
614 frames were captured at 200fps
frame delta time[us] distribution
      1 4979
      4 4982
      8 4983
     57 4984
    267 4985
    184 4986
     21 4987
      4 4988
      2 4989
      1 4991
      3 9969
     10 9970
     27 9971
      5 9972
      9 14956
      1 19940
      2 19941
      2 19942
      1 24926
      1 39882
      1 1146621
after skip frame indices (middle column)
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 10277
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Why does v4l2src change FOV starting at 41fps?

Wed Mar 28, 2018 8:31 am

https://github.com/raspberrypi/linux/issues/1774 but it's just been a very low priority to deal with it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
HermannSW
Posts: 3450
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Why does v4l2src change FOV starting at 41fps?

Wed Mar 28, 2018 2:31 pm

Thanks @6by9, no hurry.

Based on our talk yesterday Jan increased max fps for rpicamsrc with this github change:
https://github.com/thaytan/gst-rpicamsr ... a180be5386

Although increasing maximum from 90fps to 1000fps is higher than the (current) 202fps restriction of Raspberry GPU, capturing videos with 200fps now works fine with rpicamsrc. In the past I had several times the impression that rpicamsrc is more performant than v4l2src, and its features sufficed for me. Using rpicamsrc is simply done by replacing "v4l2src device=/dev/video0" by "rpicamsrc" in "gst-launch-1.0" command line.

To use rpicamsrc:
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

Return to “Camera board”