User avatar
jbeale
Posts: 3641
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

HQ cam: rapid raspistill captures?

Thu May 28, 2020 8:00 pm

I have now figured out how to get fairly rapid (eg. 4 fps) full-resolution still frames from the HQ camera, using raspividyuv as described here:
viewtopic.php?f=43&t=275363#p1668119 I believe these are strictly speaking video frames from the imaging pipeline's video port, and not from the separate stills port.

Just to be sure I'm not overlooking anything, can anyone confirm that < 2 fps is the limit for raspistill? For example, if I set it up in "signal" mode and also use the "burst" flag, with fixed exposure such as:

raspistill -t 0 --burst --signal -hf -vf -awbg 3.46,1.52 -ss 2357 -w 4056 -h 3040 -o /dev/shm/s%04d.jpg &

and then execute kill -s SIGUSR1 <pid>; sleep 0.6; kill -s SIGUSR1 <pid> then I (usually) get two JPEG images in /dev/shm
but if I instead do sleep 0.55 between the two signals, then I get only one image, leading me to conclude that 0.6 seconds is about the minimum usable interval between images using raspistill. Unless there's any settings I missed? This has been discussed in the past with the v1 and v2 cameras where it was hinted that faster times were possible in those cases. I also tried raspiyuv with uncompressed YUV output in case the JPEG compression was limiting, but got the same result (0.55 seconds is too short).

As mentioned, I know that I can use raspividyuv for even faster captures, but handling the uncompressed YUV is more cumbersome and limits the total number of shots due to RAM space, and the extra processing in raspistill does seem to give nicer looking images, so I'm just checking.

User avatar
HermannSW
Posts: 2522
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: HQ cam: rapid raspistill captures?

Fri May 29, 2020 6:42 am

Even with 0.6s I get only one frame captured in /dev/shm.
With 3 sigkills I get two, but it remains 2 for 4 sigkills, with >1s between
I did this on 4GB Pi4B with latest Raspbian:

Code: Select all

🍓 raspistill -t 0 --burst --signal -hf -vf -awbg 3.46,1.52 -ss 2357 -w 4056 -h 3040 -o /dev/shm/s%04d.jpg &
[1] 6670
🍓 pid=$!
🍓 
🍓 dt=0.6
🍓 kill -s SIGUSR1 $pid; sleep $dt; kill -s SIGUSR1 $pid; sleep $dt; kill -s SIGUSR1 $pid; sleep $dt; kill -s SIGUSR1 $pid
🍓 kill $pid
🍓 stat /dev/shm/s* | grep Change
Change: 2020-05-29 08:36:19.532591995 +0200
Change: 2020-05-29 08:36:20.712581837 +0200
[1]+  Terminated              raspistill -t 0 --burst --signal -hf -vf -awbg 3.46,1.52 -ss 2357 -w 4056 -h 3040 -o /dev/shm/s%04d.jpg
🍓 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

ethanol100
Posts: 636
Joined: Wed Oct 02, 2013 12:28 pm

Re: HQ cam: rapid raspistill captures?

Fri May 29, 2020 8:27 am

You can use timelapse=0 to get as many images as possible:

Code: Select all

raspistill -o test.jpg
strings test.jpg | grep gain

Code: Select all

dev=-1 mlux=-1 exp=62985 ag=477 focus=255 gain_r=3.292 gain_b=1.632 greenness=0 ccm=8520,-3718,-702,-514,6172,-1558,336,-1874,5640,0,0,0 md=0 tg=263 263 oth=263 235 b=0 f=263 263 fi=0 ISP Build Date: May 21 2020, 11:51:37 VC_BUILD_ID_VERSION: 2550bfc9d3e30e49382d22435da0c1ca23937b54 (clean) VC_BUILD_ID_USER: dom VC_BUILD_ID_BRANCH: bcm2711_2
Then I run the fast capture with:

Code: Select all

raspistill -bm -st -ag 1.87 -dg 1 -ss 62985 -md 3 -awb off -ex off -awbg 3.292,1.632 -o test%0 4d.jpg -set -v -t 10000 -tl 0
Generating the following file list:

Code: Select all

pi@testlab001:/dev/shm $ ls -g --time-style=full-iso test0*
-rw-r--r-- 1 pi 6525946 2020-05-29 10:19:57.547015405 +0200 test0000.jpg
-rw-r--r-- 1 pi 6515854 2020-05-29 10:19:59.146981434 +0200 test0001.jpg
-rw-r--r-- 1 pi 6525425 2020-05-29 10:19:59.756968484 +0200 test0002.jpg
-rw-r--r-- 1 pi 6521785 2020-05-29 10:20:00.386955111 +0200 test0003.jpg
-rw-r--r-- 1 pi 6521282 2020-05-29 10:20:00.996942162 +0200 test0004.jpg
-rw-r--r-- 1 pi 6525188 2020-05-29 10:20:01.566930064 +0200 test0005.jpg
-rw-r--r-- 1 pi 6523761 2020-05-29 10:20:02.266915207 +0200 test0006.jpg
-rw-r--r-- 1 pi 6525463 2020-05-29 10:20:02.976900139 +0200 test0007.jpg
-rw-r--r-- 1 pi 6530176 2020-05-29 10:20:03.576887406 +0200 test0008.jpg
-rw-r--r-- 1 pi 6531898 2020-05-29 10:20:04.176874674 +0200 test0009.jpg
-rw-r--r-- 1 pi 6532647 2020-05-29 10:20:04.786861731 +0200 test0010.jpg
-rw-r--r-- 1 pi 6521960 2020-05-29 10:20:05.406848576 +0200 test0011.jpg
-rw-r--r-- 1 pi 6533626 2020-05-29 10:20:06.016835634 +0200 test0012.jpg
-rw-r--r-- 1 pi 6520836 2020-05-29 10:20:06.626822694 +0200 test0013.jpg
-rw-r--r-- 1 pi 6523944 2020-05-29 10:20:07.246809541 +0200 test0014.jpg
-rw-r--r-- 1 pi 6528961 2020-05-29 10:20:07.866796390 +0200 test0015.jpg
which achivesless then 2 frames per second. I don't think raspistill can be quicker.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8667
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HQ cam: rapid raspistill captures?

Fri May 29, 2020 11:49 am

There are a couple of places that raspistill is sub-optimal, but it was originally only a demo app.
- It uses as single 64kB output buffer from the image_encode component as that is the default. 64kB is pretty tiny, and results in a large number of round trips for that single buffer. Increasing the size reduces the number of round trips, and increasing the number increases the amount that can be done in parallel.
- If you sacrifice more memory, then you should be able to keep the sensor streaming rather than stopping and starting. It's part of the CAMERA_CONFIG parameter, but I don't remember exactly where off the top of my head. SImplest approach is that taken with at line 855 of saying the max video res is full res.

Beyond that would take a little benchmarking to see what's going on. The sensor supposedly goes up to 10fps at full res IIRC. The ISP should do 120MPix/s fairly happily. JPEG is higher than the ISP rate. So overall it should be able to do better than 2fps.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
jbeale
Posts: 3641
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: HQ cam: rapid raspistill captures?

Fri May 29, 2020 8:12 pm

Thank you 6by9, that suggests there is some hope for better performance. I see that
status = mmal_component_create(MMAL_COMPONENT_DEFAULT_IMAGE_ENCODER, &encoder);
shows up in https://github.com/raspberrypi/userland ... ll.c#L1035
but is there any place controlling the size of the output buffer, whether it is 64kB or otherwise?

ethanol100
Posts: 636
Joined: Wed Oct 02, 2013 12:28 pm

Re: HQ cam: rapid raspistill captures?

Fri May 29, 2020 8:25 pm

I think you can have a look at RaspiVid.c, there the output buffer for the case of not being H264 is set.

I had no luck trying to change it, but I did not had time to really try...

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8667
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: HQ cam: rapid raspistill captures?

Fri May 29, 2020 8:49 pm

https://github.com/raspberrypi/userland ... ll.c#L1059
And 4 lines later it sets the number of buffers.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
jbeale
Posts: 3641
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: HQ cam: rapid raspistill captures?

Sun May 31, 2020 5:02 am

6by9 wrote:
Fri May 29, 2020 8:49 pm
https://github.com/raspberrypi/userland ... ll.c#L1059
And 4 lines later it sets the number of buffers.
FWIW: My quick-and-dirty test says the stock RaspiStill.c code gives me: encoder_out buffer size = 81920, encoder_out buffer num = 1 by default.
As an experiment I recompiled RaspiStill.c with the buffer size 10x larger and the buffer number 3x larger (meaning 3 buffers of size 819200). If the speed is sensitive to these values, I would expect to see a difference, but this change gave no improvement to output timing, using the ethanol100 style test:

Code: Select all

$ cd /dev/shm
$ raspistill -bm -st -ag 1.87 -dg 1 -ss 2000 -md 3 -awb off -ex off -awbg 3.292,1.632 -o test%04d.jpg -set -v -t 8000 -tl 0
$ ls -g --time-style=full-iso test0*

-rw-r--r-- 1 pi 1665512 2020-05-30 21:53:22.210262752 -0700 test0000.jpg
-rw-r--r-- 1 pi 1665942 2020-05-30 21:53:23.820207915 -0700 test0001.jpg
-rw-r--r-- 1 pi 1667381 2020-05-30 21:53:24.430187167 -0700 test0002.jpg
-rw-r--r-- 1 pi 1667031 2020-05-30 21:53:25.030166771 -0700 test0003.jpg
-rw-r--r-- 1 pi 1664278 2020-05-30 21:53:25.640146061 -0700 test0004.jpg
-rw-r--r-- 1 pi 1666666 2020-05-30 21:53:26.240125697 -0700 test0005.jpg
-rw-r--r-- 1 pi 1661776 2020-05-30 21:53:26.850105013 -0700 test0006.jpg
-rw-r--r-- 1 pi 1666546 2020-05-30 21:53:27.470084007 -0700 test0007.jpg
-rw-r--r-- 1 pi 1666480 2020-05-30 21:53:28.080063353 -0700 test0008.jpg
-rw-r--r-- 1 pi 1669539 2020-05-30 21:53:28.700042384 -0700 test0009.jpg
-rw-r--r-- 1 pi 1665162 2020-05-30 21:53:29.310021764 -0700 test0010.jpg
-rw-r--r-- 1 pi 1663129 2020-05-30 21:53:29.900001834 -0700 test0011.jpg

Return to “Camera board”