[SOLVED: FW bug] How does rotation work with V4L2?


7 posts
by Malvineous » Tue Mar 21, 2017 3:33 am
Hi all,

I'm trying to rotate my camera output but I can't quite figure out how the rotation interacts with the binning and camera modes.

When I do this:
Code: Select all
v4l2-ctl -v width=1296,height=972,pixelformat=H264

Then it works fine, I get the 2x2 binned mode with full FOV, and no rotation: (note text above barcode is visible)

Image

But if I want this same view rotated 90 degrees, if I do this:

Code: Select all
v4l2-ctl -v width=1296,height=972,pixelformat=H264 --set-ctrl=rotate=90

Then I get a zoomed/cropped view, so it looks like I lose the 2x2 binning: (no text above barcode)

Image

So if I swap the width and height, thinking this will get me the original camera mode:

Code: Select all
v4l2-ctl -v width=972,height=1296,pixelformat=H264 --set-ctrl=rotate=90

Then I get a smeary purple mess all over the screen:

Image

So what is the correct way for getting the 1296x972 / 2x2 binned mode with 90 degree rotation?
Last edited by Malvineous on Wed Mar 22, 2017 1:13 am, edited 1 time in total.
Posts: 28
Joined: Wed Mar 07, 2012 10:31 am
by ethanol100 » Tue Mar 21, 2017 10:57 am
Does it work if you use 1280 and 960 instead of 1296 and 972 for your last command? If I remember correctly there where some limitation on some sizes being multiples of 16.
Posts: 446
Joined: Wed Oct 02, 2013 12:28 pm
by Malvineous » Tue Mar 21, 2017 11:23 am
No difference with 1280x960 or 960x1280 - same cropped or purple smear. It's not picking the correct camera mode from the available selection, and I'm not sure how to make this happen.

For the record the resolutions aren't made up, they are taken from this post where they explained how the new modes were added, and what resolutions to enter in order to select the right mode.
Posts: 28
Joined: Wed Mar 07, 2012 10:31 am
by ethanol100 » Tue Mar 21, 2017 12:06 pm
The reduced view while rotating is normal. How do you want to get your required resolution from the sensor? If you use the rotated V1 camera, the sensor has a width of 1944px and height 2592px. You now want to get a image of ratio 4:3 this means, you can only take the full width of 1944px as your "long" size and will only be able to get 1944px/4*3=1458 px of height of the image. This means you need to cut away ~22% of the image on both sides, which explains your images. This is the reduced FOV you get. For higher frame rate the sensor needs to choose the binned mode resulting in 972px x 729px which then are upscaled to your requested resolution. You can see from your images, that the height in your first image is exactly the width of your second one.

The only problem I see is with your third image, this should give you the right image. I don't know why it is not working.
Posts: 446
Joined: Wed Oct 02, 2013 12:28 pm
by 6by9 » Tue Mar 21, 2017 12:33 pm
The width/height you specify is always that of the final output buffers.

Taking your 2nd command, you've asked for an output that is 4:3, so that means you need a 3:4 image off the sensor. There are no 3:4 modes, so it will normally choose 1296x972, crop it to 730x972 to give 3:4, and then rotate and upscale it to give you 1296x972 out. Not great for FOV or image quality.

Your third command is correct to get the full original frame but rotated 90.
However what you've tripped over is a firmware bug :shock: The codec takes a specific weird image format as input, and that has a maximum height image that it can handle without switching to a different mode. That used to at 1344, but a change has been made that appears to have dropped that down to 1264.
Using raspivid:
Code: Select all
raspivid -w 972 -h 1264 -rot -o file/h264

encodes fine.
Code: Select all
raspivid -w 972 -h 1266 -rot -o file/h264

fails.
(1266 requires the next 16x16 macroblock, so is actually 1280 high).
I've done a very quick firmware mod to reduce the threshold down from 1344 to 1264 and both your V4L2 commands and the raspivid command work fine. I'll double check the numbers and push a patch to be released when Dom next makes a release (normally about once a week to be picked up with rpi-update - treat rpi-update with caution!).

And thanks for the report :D We can't test everything and these things do occasionally go wrong.
The change that broke this was either July or November 2015, so you can tell how few people hit this use case.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3814
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.
by Malvineous » Wed Mar 22, 2017 1:12 am
Excellent, many thanks! I almost put this in as an issue on GitHub but thought I'd better ask here first in case I was doing something silly :-) I'm happy to wait until the next firmware release, that's not a problem. I'm amazed you were able to identify and fix it so quickly!
Posts: 28
Joined: Wed Mar 07, 2012 10:31 am
by 6by9 » Wed Mar 22, 2017 9:44 am
Firmware release happened just before you posted last night :D https://github.com/raspberrypi/firmware ... d34c092d95

Normal warnings of don't use rpi-update on an important machine, and you can probably get away with just updating the firmware if you're worried about the jump to the 4.9 kernel ("sudo SKIP_KERNEL=1 rpi-update").

I must admit that when I started reading your first post I suspected it would be a PEBKAC issue, but always double check to avoid looking silly when wrong :)
Seeing the images, the symptoms were distinctive enough that I had a fair guess, and also knew of the other commit and that it might have had an impact. After that it was relatively trivial, although it did have a knock-on implication which took longer to resolve.

Thanks again.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 3814
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.