jmmec
Posts: 26
Joined: Thu Dec 31, 2015 11:13 pm

raspivid: mode 1 vs mode 2/3 (v2 camera)

Thu Mar 16, 2017 2:32 am

Hello,

I have several Pi2 devices, each with a v2 camera, located at a remote cabin to capture wildlife, especially the kind that walks on two legs and vandalizes and steals property.

I'm terribly stumped about the field of view differences using raspivid.

Note: The video captures below were at 1920x1080 but have been scaled down to 1280x720.

1. This is the field of view that I really like! Below is mode 2 (but mode 3 is similar) where these modes seem significantly sharper than other modes, and they give the field of view that I'm after.

Code: Select all

$ raspivid -t xxx -md 2 -fps 25 -o raspivid_md2.h264
With the following resulting video:

http://medlock.info/raspivid_md2.mp4

Although I specified 25fps, I understand the actual capture is only 15fps (max), and this is obvious by how fast the hiker is walking when it is played at 25fps. I have to slow it down to appear normal; this isn't optimal or desireable if it can somehow be avoided.

2. If mode 1 is selected, then I can get the 25fps (30fps max):

Code: Select all

$ raspivid -t xxx -md 1 -fps 25 -o raspivid_md1.h264
But notice the awfully unsharp and narrow field of view compared to the video above. Is this expected, or am I doing something wrong?

http://medlock.info/raspivid_md1.mp4

I would happily use mode 1, but would like the field of view to look like mode 2. Is this possible?

I've tried some of the other modes (like 4, 5, ...) but the video quality doesn't seem nearly as good as mode 2 or mode 3. Does "binning" degrade the image quality?

Thank you very much for any help or suggestions!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7289
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: raspivid: mode 1 vs mode 2/3 (v2 camera)

Thu Mar 16, 2017 12:02 pm

Have you read https://picamera.readthedocs.io/en/late ... nsor-modes which gives a description and example images of the different modes.

If you're resizing to 1280x720 (I'm assuming with raspivid rather than afterwards, otherwise those clips aren't a useful comparison), then inherently taking the cropped 1080P image and resizing will lose some definition as there is a greater degree of interpolation going on.
Yes binning loses some image quality as you are again reducing the number of pixels being processed, but as there is a finite pixel readout speed on the sensor then that is the only way to get higher frame rates.

You have to choose whether you want absolutely pristine image quality, frame rate, or FOV. There isn't a way to get everything all at once.

You don't say what you're doing with the images, but using raspivid you can add "-pts timestamps.txt" to the command, and then "mkvmerge -o video.mkv --timecodes 0:timecodes.txt video.h264" to create an mkv file abiding by the timestamps generated. That way it will play back at the correct rate. You should be able to do the same with other tools if you can work out the correct runes to do so.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jmmec
Posts: 26
Joined: Thu Dec 31, 2015 11:13 pm

Re: raspivid: mode 1 vs mode 2/3 (v2 camera)

Thu Mar 16, 2017 11:09 pm

Thanks very much!

I should have been clearer about how I captured the mode 1 and mode 2 videos:

1. I didn't specify the height / width resolution in my usage of raspivid. I re-ran the raspivid commands (from my original post) to get the "-v" output, and it says that it is capturing both videos (mode 1 and mode 2) at 1920x1080 resolution. It also says 25fps for mode 2, but I think it is just repeating the command line arguments since max is 15fps.

2. I took both 1920x1080 resolution videos and used Shotcut to downsample them to 1280x720 to reduce the file sizes. I'm using an LTE wireless connection and wanted to save a bit of data transfer costs.

Regarding the two video captures in my original post:

I was surprised by the dramatically different fields of view between mode 1 and mode 2 since I thought both were being captured at 1920x1080 resolution as reported by raspivid? I've read the excellent PiCamera documentation, but still don't quite understand why there is such an extreme difference between the two modes? I'd prefer to use mode 1 (at 25fps) if I could get the same field of view as when using mode 2 or 3 (at 15fps), but I get something entirely different in mode 1. If "it is what it is" and I'm not doing something wrong, then I'll use mode 2 @ 15fps since it is quite good.

Appreciate the reference to the MKVToolNix package: I didn't know about the MKV format, or the set of tools, and have played around with it this afternoon and it helped "smooth" out the playback of the 15fps video captured in mode 2. I'll definitely explore this toolset further.

Regards

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7289
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: raspivid: mode 1 vs mode 2/3 (v2 camera)

Fri Mar 17, 2017 10:24 am

jmmec wrote:I should have been clearer about how I captured the mode 1 and mode 2 videos:

1. I didn't specify the height / width resolution in my usage of raspivid. I re-ran the raspivid commands (from my original post) to get the "-v" output, and it says that it is capturing both videos (mode 1 and mode 2) at 1920x1080 resolution. It also says 25fps for mode 2, but I think it is just repeating the command line arguments since max is 15fps.

2. I took both 1920x1080 resolution videos and used Shotcut to downsample them to 1280x720 to reduce the file sizes. I'm using an LTE wireless connection and wanted to save a bit of data transfer costs.
Unfortunately that means that I can't tell whether any lack of sharpness is due to the image processing or the resize in Shotcut. :(
jmmec wrote:Regarding the two video captures in my original post:

I was surprised by the dramatically different fields of view between mode 1 and mode 2 since I thought both were being captured at 1920x1080 resolution as reported by raspivid? I've read the excellent PiCamera documentation, but still don't quite understand why there is such an extreme difference between the two modes? I'd prefer to use mode 1 (at 25fps) if I could get the same field of view as when using mode 2 or 3 (at 15fps), but I get something entirely different in mode 1. If "it is what it is" and I'm not doing something wrong, then I'll use mode 2 @ 15fps since it is quite good.
Fundamentally you have an analogue to digital converter inside the sensor that has a maximum sample rate, so there is an absolute maximum number of pixels per second that can be read out.(1)
3280x2464x15 = 119,854,080 pixels/s, so that is the current limit(2). 120MPix/s is also the conservative limit for the image processing pipe.

Binning combines 4 pixels (arranged as 2x2) of the same colour into one conversion, but takes 3280x2464 down to 1640x1232 which is below 1080P so no use here.

Now admittedly 1920x1080x30 = 62,208,000 pixels/s, so there would be scope for increasing the resolution and field of view of mode 1 whilst still achieving 30fps. I suspect that those writing the driver were given a register set by Sony which read out a centre-cropped [email protected] and that is what you have as mode 1. It would be possible to tweak it up to a crop of 2592x1458 @ 30fps (113,374,080 pixels/sec), but didn't (it'd be approx the same FOV as the current mode 6). Changing it now would upset anyone who has carefully set their camera up on a scene using mode 1. Mode 3 is effectively redundant on the V2 camera, so that could be repurposed with minimal upset, but I'm not desperate to go playing with camera register sets again.
Sorry, this is all set up in the closed source firmware so no option for general tinkering.

(1) There is also a restriction that the CSI2 bus can "only" run at 1Gbit/s per lane, and we have 2 lanes. Not a problem here, but it is on other sources with more than 10bits/pixel.
(2) The datasheet does list a max pixel frame rate of 280MPix/s = 8MPix @ 35fps, but I suspect that requires 4 CSI-2 lanes which we don't have on Pi. It would certainly be exceeding the ISP throughput though, and almost no other part of the hardware will cope with that resolution/framerate either.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jmmec
Posts: 26
Joined: Thu Dec 31, 2015 11:13 pm

Re: raspivid: mode 1 vs mode 2/3 (v2 camera)

Sat Mar 18, 2017 4:26 pm

6by9 wrote:Unfortunately that means that I can't tell whether any lack of sharpness is due to the image processing or the resize in Shotcut. :(
Sure, I had to re-record, but here is the straight h264 output from raspivid for mode 1 and mode 2; I'm still playing with the focus on the v2 camera:
[email protected]:/media/nfs $ raspivid -t 10000 -w 1920 -h 1080 -md 1 -fps 25 -o raspi_md1_1920x1080.h264

http://medlock.info/raspi_md1_1920x1080.h264
And:
[email protected]:/media/nfs $ raspivid -t 10000 -w 1920 -h 1080 -md 2 -fps 15 -o raspi_md2_1920x1080.h264

http://medlock.info/raspi_md2_1920x1080.h264
Thank you very much for the additional clarification. If mode 1 (or a repurposed mode 3) is ever reworked to match or exceed the resolution / FoV of mode 2, then that would be awesome, and you'd probably make a lot of people extremely happy. (Or at least one person.)

Regards

Return to “Camera board”