sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Thu Jul 12, 2018 1:23 pm

result : https://i.imgur.com/YgGMNrf.gifv Image

to reproduce:

Code: Select all

raspivid -w 2592 -h 1944 -t 500 -md 2 -ISO 100 -br 60 -cd MJPEG -o vid_02.mjpg
raspistill -o still_02.png
what i've observed thus far:
- raspiraw, raspistill and v4l all share similar field of view
- raspivid output seems to stretch a bit -- oddly the difference at the left side is not as profound as the difference at the right side.

question:
which one is the better presentation of the native resolution (e.g. 1-to-1 mapping between the sensor and the pixel)?

KnarfB
Posts: 187
Joined: Wed Dec 14, 2016 10:47 am
Location: Germany

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Sun Jul 15, 2018 8:12 pm

JPEG compresses MCUs aka "macroblocks". A macroblock typically comprises of 16x16 pixels. But, 1944 is not a multiple of 16. It seems the the video cuts 8 pixel columns off at the right hand side. The macroblock size is related to color space conversion, subsampling and the size of the 8x8 DCT transform internally used in JPEG.
Some modern compression schemes like H.264 allow for unaligned picture sizes. But, availability of this feature is up to the implementor, try it out.
hth
KnarfB

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Mon Jul 16, 2018 3:29 am

@KnarfB: thanks for your analysis

to summarize:
- pi camera uses yuv420 (ref: https://picamera.readthedocs.io/en/rele ... yuv-format)
- with this chroma subsampling the block would be 16*16 (ref: https://en.wikipedia.org/wiki/JPEG#Block_splitting)
- the height (1944) is not divisible by 16, so the pipeline needs to either pad the bottom or crop the image
no problem up to this point, but 2592 is divisible by 16 - it doesn't require cropping.

extrapolating from the summary above, we conclude that the pipeline crops instead of pads.
- from 1944 vertical, only 1936 are retained
- although the horizontal side is not cropped, the image is stretched while maintaining 4:3 ratio
- 1936 * 4 /3 = 2581.33 horizontal - rounded down to 2581
- the 2581*1936 is stretched to 2592*1944, filling in the gap of 11*8

on 2nd thought, there's an argument against "JPEG compresses MCUs" as the culprit.
using modified raspicam, the output of video port (from video_buffer_callback) is used.
by tapping at this point, the image shouldn't be compressed yet.
however the output from raspicam resembles the output of raspivid.

unless there's another insight, seems like still port is the way to go.

===============================================================
a bit oot:
- strobe works in all resolutions with raspiraw (ril.rawcam with the still port)
- strobe works in vga & 1280*960 resolution but not in 5mp resolution with raspicam (ril.camera with the video port)
- - - the frex registers would be reset/ignored.

KnarfB
Posts: 187
Joined: Wed Dec 14, 2016 10:47 am
Location: Germany

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Wed Jul 18, 2018 11:00 am

sorry, I mixed up width and height. KnarfB

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5372
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Wed Jul 18, 2018 2:17 pm

Both apps are going to be using the same set of register values into the sensor, therefore the raw input image should be the same.
The ISP can crop/resize in almost arbitrary amounts. I'm not aware of any deliberate differences being applied in video mode, unless you enable video stabilisation. IIRC Nothing actually passes through the ISP as "native resolution" - the resize block is always active, although the effect it will have is going to be minimal if input and output resolutions are the same.

It is true that (M)JPEG and H264 both work on a 16x16 macroblock. The requirement is normally that the last active row/column is duplicated into the unused rows/columns to avoid wasting bits on encoding rubbish, particularly as doing so can produce artefacts in the visible area of the image.
Encoding to MJPEG on the Pi is more involved than H264 or JPEG with format conversions, so it's possible that something ends up a couple of pixels off. That would normally be evident in the encoded size of the frame though.

Your best bet would be to compare raspividyuv and raspiyuv in capturing the raw YUV frames out of the camera to isolate whether it is the camera or the encoder that appears to be the issue. I'm afraid I haven't got time at the moment to investigate that.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Thu Jul 19, 2018 10:59 am

@6by9: thanks for your suggestion.

sadly, encountered the same problem mentioned in this thread: viewtopic.php?t=189757
will try to download the source and compile

-------------- edit 19-jul 2018 19:15 (gmt+7) ----------------------
same problem with the latest source; but realized that -t 0 is the problem.

setting a short duration works.
* though it intermittently prompts
mmal: Failed to write buffer data (0 from 15178752)- aborting

last frame of raspiyuv would always shift ; a colleague suggests that most streams come from the preview port, only the last one from the still port.
will check the code and the results later

-------------- edit 19-jul 2018 20:08 (gmt+7) ----------------------
very confused; apart from the last frame shift in the preview, observed more issues:
1. stored video of raspividyuv is actually a stretch from a cropped image https://i.imgur.com/p2ipySx.png
2. the 1st raspiyuv output has the color channels flipped https://i.imgur.com/N9iOCBU.jpg
3. both of above have vertical tearing issue -- this is the 1st time i've ever encountered a vertical tear
after observing these oddities, restarted the pi and tried to capture a raspiyuv.
the result is as expected: https://i.imgur.com/YrXcOzV.jpg

raspiraw would reset all registers.
perhaps raspividyuv & raspiyuv don't try to do the same -- and combinations of mismatched registers causing the oddities observed above?
it's a likely explanation for #2 above; i.e. flip/mirror register is messed up
unfortunately there's no orientation indicator in the 1st set to verify/refute this assumption.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5372
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Thu Jul 19, 2018 1:53 pm

Please make a new post rather than editing. Editing doesn't flag the post as unread, so people (ie me) may miss that updates.

I hadn't seen any reports of issues in raspiyuv and raspividyuv previously as it was raised in a slightly odd forum that I don't normally follow.

Looking at RaspiStillYUV, it uses the stills port exclusively for captures. The callback code is a little quirky in some of the conditionals, but looks valid.
RaspiVidYUV uses the video capture port exclusively for captures. The callback code looks fine.

OK, one bug found in RaspiVidYuv. It checks that the stills port is disabled, and disables it if it isn't already disabled. The stills port will always be disabled, and it should be checking the video port instead. That explains the "mmal: Failed to write buffer data (0 from 15178752)- aborting" on shutdown as the file can get closed before it should be. Bug fix in https://github.com/raspberrypi/userland/pull/477

I've only got a V2.1 camera to hand and can't see an issue there.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5372
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Thu Jul 19, 2018 2:09 pm

sanjaya wrote:
Thu Jul 19, 2018 10:59 am
sadly, encountered the same problem mentioned in this thread: viewtopic.php?t=189757
will try to download the source and compile
Diagnosed and comment made. I'll make a bug fix for that later
sanjaya wrote:-------------- edit 19-jul 2018 20:08 (gmt+7) ----------------------
very confused; apart from the last frame shift in the preview, observed more issues:
1. stored video of raspividyuv is actually a stretch from a cropped image https://i.imgur.com/p2ipySx.png
2. the 1st raspiyuv output has the color channels flipped https://i.imgur.com/N9iOCBU.jpg
3. both of above have vertical tearing issue -- this is the 1st time i've ever encountered a vertical tear
after observing these oddities, restarted the pi and tried to capture a raspiyuv.
the result is as expected: https://i.imgur.com/YrXcOzV.jpg
PLEASE stop editing old posts. Do it too much and I'll drop off the thread.

1. Did you forget to add the -md 2 parameter to raspividyuv? Without it it would be using the cropped 1080P mode. (Hmm, support for -md is missing from raspi(still)yuv)

2. What are you using to view the image data? It's raw YUV4:2:0 as three planes, yet you posted a PNG and a JPEG. If you're using x86 Linux then I can thoroughly Vooya for viewing raw images. It's available for Windows and MacOS, but I've not tried it, and it costs money on those platforms.

3. Tear, or you're not using the correct height? 1944 is not a multiple of 16, so the buffer will be aligned up to 1952. You don't state what resolution you're using, but I'm certainly not seeing those issues.
sanjaya wrote:raspiraw would reset all registers.
perhaps raspividyuv & raspiyuv don't try to do the same -- and combinations of mismatched registers causing the oddities observed above?
it's a likely explanation for #2 above; i.e. flip/mirror register is messed up
unfortunately there's no orientation indicator in the 1st set to verify/refute this assumption.
??!?!
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Thu Jul 19, 2018 3:25 pm

6by9 wrote:
Thu Jul 19, 2018 2:09 pm
PLEASE stop editing old posts. Do it too much and I'll drop off the thread.
noted. different forum has different rules. apologize for not familiarizing with the rules here.
6by9 wrote:
Thu Jul 19, 2018 2:09 pm
1. Did you forget to add the -md 2 parameter to raspividyuv? Without it it would be using the cropped 1080P mode. (Hmm, support for -md is missing from raspi(still)yuv)
you're right. my mistake. will try it tomorrow.
6by9 wrote:
Thu Jul 19, 2018 2:09 pm
2. What are you using to view the image data? It's raw YUV4:2:0 as three planes, yet you posted a PNG and a JPEG. If you're using x86 Linux then I can thoroughly Vooya for viewing raw images. It's available for Windows and MacOS, but I've not tried it, and it costs money on those platforms.
created a small opencv function to save into png.
i don't understand the logic in imgur, 50% of the png uploaded would be converted to jpg

uploaded the raws into dropbox. https://www.dropbox.com/sh/slqyb2em0kpr ... bE1Pa?dl=0
6by9 wrote:
Thu Jul 19, 2018 2:09 pm
3. Tear, or you're not using the correct height? 1944 is not a multiple of 16, so the buffer will be aligned up to 1952.
probably not.
all were taken with the same command: raspiyuv -rgb -o
under the assumption that registers would be reset every time to default (5mp resolution)
6by9 wrote:
Thu Jul 19, 2018 2:09 pm
You don't state what resolution you're using
as per the topic the resolution is 2592*1944
6by9 wrote:
Thu Jul 19, 2018 2:09 pm
sanjaya wrote:raspiraw would reset all registers.
perhaps raspividyuv & raspiyuv don't try to do the same -- and combinations of mismatched registers causing the oddities observed above?
it's a likely explanation for #2 above; i.e. flip/mirror register is messed up
unfortunately there's no orientation indicator in the 1st set to verify/refute this assumption.
??!?!
just trying to rationalize the odd result of the 19_12 snap.
to reiterate, the 20_03 snap was taken after a cold restart.

ethanol100
Posts: 530
Joined: Wed Oct 02, 2013 12:28 pm

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Thu Jul 19, 2018 5:09 pm

I only have a v2.1 camera to hand, but out of couriosity...

I see a horizontal stretch, too.

Code: Select all

raspiyuv -w 3280 -h 2464 -t 5000 -v -ag 1.0 -dg 1.0 -ss 31806 -awb off -awbg 1.640625,1.609375 -o raspiyuv2.yuv
raspividyuv -t 5000 -v -w 3280 -h 2464 -md 2 -fps 1 -ag 1.0 -dg 1.0 -ss 31806 -awb off -awbg 1.640625,1.609375 -o raspividyuv.yuv
ffmpeg -f rawvideo -video_size 3296x2464 -i raspiyuv2.yuv -pix_fmt yuv420p still%04d.png
ffmpeg -f rawvideo -video_size 3296x2464 -i raspividyuv.yuv -pix_fmt yuv420p vid%04d.png


If I squeeze the image of raspividyuv by 8 pixel, they overlap nicely. Somehow the image of video seems to be zoomed out by 8 pixel on the right edge.

I put the files into Google drive, ffmpeg exports uncompressed pngs the file size is quite big....

Is this the difference one sees in raspistill, if it switching from preview to displaying the captured image, too?
I vaguely remember that there was something like this regarding preview und captured image.

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 2:55 am

@ethanol100: thank you very much for participating.
my working environment vibrates too much for doing these kinds of precision tests.
sanjaya wrote:
Thu Jul 19, 2018 10:59 am
raspiraw would reset all registers.
perhaps raspividyuv & raspiyuv don't try to do the same -- and combinations of mismatched registers causing the oddities observed above?
it's a likely explanation for #2 above; i.e. flip/mirror register is messed up
unfortunately there's no orientation indicator in the 1st set to verify/refute this assumption.
although it's out of topic, got curious about the "vertical" tear issue.
unable to reproduce the issue thus far, but
- the 1st 171px are filled in with junks
- those junks shift the pixels, producing the effect that appears to be vertical tear.
these refute my assumption above.
the color flip is not caused by unexpected flip/mirror register messing up bayer order, but due to pixel shifts.

by checking each channel junks size is deduced to be 512 bytes (171*3 = 513)
modifying the opencv code to ignore the junks produce a proper image.

not sure what i did that causes persistent 512 bytes junk header in both raspividyuv and raspiyuv, but this shouldn't be a problem for average use cases.
* yes the 512 bytes junk header exists in every frame of raspividyuv from that particular test.

KnarfB
Posts: 187
Joined: Wed Dec 14, 2016 10:47 am
Location: Germany

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 6:02 am

Wouldn't it make sense to activate the sensors test pattern feature by setting ov5647 reg 0x503D?
Then, your measurments will be independent of optical targets and pixel peeping is much easier.
My v1.3 cam module is broken, so I cannot test it myself. hth KnarfB

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 7:30 am

some test pattern samples have been uploaded @https://www.dropbox.com/sh/rxvdd360765t ... Z2pYa?dl=0

Code: Select all

./raspiraw -hd -md 2 -r "503D,80" -o 503D_80.raw
./raspiraw -hd -md 2 -r "503D,81" -o 503D_81.raw
./raspiraw -hd -md 2 -r "503D,C0" -o 503D_C0.raw
./raspiraw -hd -md 2 -r "503D,C1" -o 503D_C1.raw
don't know how to analyze this though.
* the results differ from the documentation

KnarfB
Posts: 187
Joined: Wed Dec 14, 2016 10:47 am
Location: Germany

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 9:39 am

Thats a surprise. I was thinking of the checkerboard patterns where one can count out the pixels and check whats missing and/or what was interpolated. According to the docs, 503D,81 should do that, strange.
Did you try more combinations 82, 83, ... there may be a typo in the docs.

The idea was to side inject i2c commands also while raspivid is running and compare the results. Works for my v2 (IMX219) module:
i2cset -y 0 0x10 0x06 0x00 0x00 0x04 i

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5372
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 10:10 am

Minor bug found in the firmware - conspiracy theories can stop.
Part of the video stabilisation code was active, so it was ending up horizontally resizing 2589 pixels up to 2592 on the output side. Video stabilisation code is never active on the stills pipeline, therefore that resize wasn't happening there.
I'll push a fix for the next rpi-update release.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 10:18 am

@6by9: great. thanks.

@KnarfB uploaded 3 more test patterns; as per your suspicion, the doc is wrong -- 92 is what we want.

sanjaya
Posts: 14
Joined: Thu Jul 05, 2018 11:00 am

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 11:42 am

just for the record.
sanjaya wrote:
Thu Jul 19, 2018 10:59 am
last frame of raspiyuv would always shift
https://www.dropbox.com/s/u956j2zs5s1e7 ... t.avi?dl=0

Code: Select all

raspiyuv -rgb -o /dev/shm/gone 

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5372
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: either raspistill or raspivid is not in native resolution at 2592*1944 (ov5647)

Fri Jul 20, 2018 1:09 pm

sanjaya wrote:
Fri Jul 20, 2018 11:42 am
just for the record.
sanjaya wrote:
Thu Jul 19, 2018 10:59 am
last frame of raspiyuv would always shift
https://www.dropbox.com/s/u956j2zs5s1e7 ... t.avi?dl=0

Code: Select all

raspiyuv -rgb -o /dev/shm/gone 
The preview display shifts - probably the same issue. The preview pipe would be hit by the bug I referenced, whilst the stills pipe won't be. The preview frame for the stills capture (sometimes referred to as a snapshot frame) comes from the stills pipe.
If your storage is slightly slow (eg NFS mount), then after the capture whilst it is saving the image you'll often find preview has resumed whilst Linux is busy saving data. There the image should have shifted back again.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 10 guests