User avatar
jbeale
Posts: 3641
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 3:53 am

I want to take some full-resolution stills using the new Pi "High Quality" camera faster than raspistill will do it (as far as I know). That leaves me with raspividyuv. I found that I am able to record uncompressed RGB this way (at least, after increasing GPU memory to 512 MB):

Code: Select all

# Write an uncompressed RGB video file to RAM, at the full resolution of the High-Quality camera:
raspividyuv -w 4056 -h 3040 -rgb -o /dev/shm/testvid.rgb -t 2000 -fps 4

# On separate machine, use ImageMagick to convert RGB video file to a set of still images
# Note that the X dimension must be increased to an even multiple of 32, so 4056 becomes 4064
sudo apt install imagemagick
convert -depth 8 -size 4064x3040 rgb:testvid.rgb test%04d.jpg
# ================
So far, so good. However RGB files are so large that I quickly run out of space recording to /dev/shm after just a few frames.
I'm OK with lower chrominance resolution than luminance resolution, so YUV seems like a better option.
I can (I think) record a YUV file, doing for example:
raspividyuv -w 4056 -h 3040 -o /dev/shm/testvid.yuv -t 2000 -fps 6

But try as I might, I have been unable to convert an uncompressed YUV video file to a sequence of stills in a similar way as with RGB. Does anyone know how this is done in ImageMagick? I've tried the below command, which produces an interesting result, but it is not yet quite right:

convert -depth 8 -sampling-factor 4:2:0 -interlace plane -identify -size 4064x3040 rgb:testvid.yuv frame%04d.jpg
YUV-convert_05.jpg
YUV-convert_05.jpg (103.39 KiB) Viewed 418 times

User avatar
HermannSW
Posts: 2513
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 7:31 am

There is an easy way to convert YUV to .pgm (portable grey map):
viewtopic.php?f=43&t=274981&p=1666830#p1666224

Raw Bayer rgb gets converted to real image by demosaicing in GPU.
I don't know what the similar process for YUV is called, but suspect that it is in Raspberry GPU as well.

My tool i420toh264 (based on hello_pi encode.c sample) converts raspividyuv stream to .h264, but keeps only Y plane and sets all U and V plane values to 128. At least you get GPU encoded grey .h264 video (which was all that I needed):
"Simple modifying pipeline"
https://github.com/Hermann-SW2/userland ... g-pipeline

Converting raspividyuv output to .h264 can be done with ffmpeg as well, but without GPU:
https://github.com/Hermann-SW2/userland ... g-airplane
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

ethanol100
Posts: 634
Joined: Wed Oct 02, 2013 12:28 pm

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 8:03 am

This should be quite similar, I think

Code: Select all

convert -size 4064x3040  -sampling-factor 4:2:0 -depth 8 yuv:test.yuv -colorspace RGB out%04d.png
should work.

(I personally would use ffmpeg for the conversion.)

User avatar
jbeale
Posts: 3641
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 3:38 pm

ethanol100 wrote:
Thu May 28, 2020 8:03 am
This should be quite similar, I think

Code: Select all

convert -size 4064x3040  -sampling-factor 4:2:0 -depth 8 yuv:test.yuv -colorspace RGB out%04d.png
should work.
(I personally would use ffmpeg for the conversion.)
Thank you! I confirmed this does work at lower resolutions, eg 640x480. For the full 12MP frame, it appears my Linux Mint machine is just unable to perform this operation- it chugs away slowly for minutes (in RGB it took seconds) at 100% CPU but only 9% memory and then says

Code: Select all

$ convert -size 4064x3040  -sampling-factor 4:2:0 -depth 8 yuv:testvid.yuv -colorspace RGB out%04d.png
convert-im6.q16: DistributedPixelCache '127.0.0.1' @ error/distribute-cache.c/ConnectPixelCacheServer/244.
convert-im6.q16: cache resources exhausted `testvid.yuv' @ error/cache.c/OpenPixelCache/3984.
convert-im6.q16: memory allocation failed `testvid.yuv' @ error/yuv.c/ReadYUVImage/419.
convert-im6.q16: no images defined `out%04d.png' @ error/convert.c/ConvertImageCommand/3258.
I guess my next task is to find out how to use ffmpeg for this conversion.

EDIT: Success! I had to increase ImageMagick's default memory limits using sudo nano /etc/ImageMagick-6/policy.xml
It's pretty slow though. I wonder if ffmpeg can also do this, if it would be faster. Without recompiling IM from source, on my Linux box I'm stuck with the -Q16 version that uses 16-bit-per-pixel-per-colorplane intermediate storage, and 32-bpp HDRI floating-point computations. Which as good as the Pi HQ camera is, may be a little bit more per-pixel computation than the source material justifies.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8661
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 5:44 pm

Converting from YUV to RGB is a matrix multiplication operation on every pixel, therefore it is a fair amount of number crunching. Use of NEON or other SIMD would make it faster.
JPEGs are generally YUV natively, so JPEG compression should be faster, and if you set the quality to 100 then there should be minimal loss of detail.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ethanol100
Posts: 634
Joined: Wed Oct 02, 2013 12:28 pm

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 6:45 pm

For ffmpeg it is quite easy.

Code: Select all

ffmpeg -pixel_format yuv420p -video_size  4064x3040 -i test.yuv out%04d.png
This will create uncompressed png where each png will be 23MB, yet it is reasonably fast on the pi.

I had to increase the gpu_mem to 256 to be able to get raspividyuv to work properly.

For convert it works, if you first split the file:

Code: Select all

split -b $((4064*3040*3/4)) -d test.yuv
for i in x*; do convert -size 4064x3040 -sampling-factor 4:2:0 -depth 8 yuv:$i -colorspace RGB $i.png; done
This is much slower and takes about a minute per picture.

User avatar
jbeale
Posts: 3641
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 7:38 pm

Thank you; indeed ffmpeg does work as you say. (It turns out my low-end 1GB Pi isn't the right choice for this task, but that's a separate thing.)

ethanol100
Posts: 634
Joined: Wed Oct 02, 2013 12:28 pm

Re: HQ cam: convert raspividyuv output to still images ?

Thu May 28, 2020 8:29 pm

I used it on a 2GB Pi4.

Code: Select all

ffmeg -pixel_format yuv420p -video_size 4064x3040 -i test.yuv -filter:v "crop=4056:3040:0:0" -qscale:v 1 -qmin 1 -qmax 1 test%04d.jpg
This will also crop the image to the real size. It took less then 14 seconds for 14 frames, but, yes, it took 972MB of RAM. The saved jpg images are about 4.6MB per frame(Quality of output is about 97).

Return to “Camera board”