How can we get high framerate raws on 4b board?

[chillie]

Hi! I'm using raspiraw tool scripts to capture raws.
I found that I could capture high framerate raws on my pi zero w, say >500fps with 640x128_s. But none of the scripts could achieve > 300fps, actually most <200fps, on my 4b board. Does anyone know the reason...

[HermannSW]

Which raspiraw did you clone?

6by9 did merge my high framerate work when it was doing v1 camera only:
https://github.com/6by9/raspiraw/blob/m ... /640x128_s
See the code for "--height":
https://github.com/6by9/raspiraw/blob/m ... aw.c#L1577

This code does handle "--height" for v1 as well as v2 camera:
https://github.com/Hermann-SW/fork-rasp ... aw.c#L1026

You can find the v1 and v2 tools in separate directories in this code drop:
http://stamm-wilbrandt.de/en/forum/rasp ... _26_18.zip

There are only the 640xH tools needed to determine this diagram:



I played with raspiraw and HQ camera. Regarding high framerate HQ camera cannot even compete with v1 camera, so I did not do more. See this posting on failure to even get 300fps with HQ camera:
viewtopic.php?f=43&t=109137&hilit=raspi ... 0#p1666035

I did enable raspiraw to capture long exposures with HQ camera, up to 239s length -- but that is the opposite of high framerate:
viewtopic.php?f=43&t=109137&hilit=raspi ... 0#p1666035
239s long exposure allows to capture green laser beam from the side with raspiraw:
viewtopic.php?f=43&t=273358&p=1702906#p1702906

[chillie]

Thanks for your info. I test on v1 and v2 with 6by9's raspiraw. They both worked fine (of course v2 needs changing the reg addr to fit ---height) on my zero w board but failed on my 4b board. I just tried your code drop and here's the output (I used the 6by9's camera_i2c to expose camera address):

Code: Select all

pi@raspberrypi:~/raspiraw.drop.5_26_18/v1 $ ./640x64 1200
removing /dev/shm/out.*.raw
capturing frames for 1200ms with 665fps requested
186 frames were captured at 166fps
frame delta time[us] distribution
      1 -469586627
      1
      3 0
      1 1503
      3 4508
      3 4509
      2 6008
      9 6010
     76 6011
     19 6012
      3 7513
     46 7514
      6 7515
      2 7517
      1 9016
      8 9017
      2 9018
after skip frame indices (middle column)
6011,3,468400922
4509,4,468405431
4508,5,468409939
6011,6,468415950
4509,7,468420459
6011,8,468426470
4508,9,468430978
4509,10,468435487
6011,11,468441498
4508,12,468446006
6011,13,468452017
7514,14,468459531
7514,15,468467045
6011,16,468473056
9018,17,468482074
6010,18,468488084
7514,19,468495598
6012,20,468501610
6011,21,468507621
7513,22,468515134
6012,23,468521146
6011,24,468527157
7517,25,468534674
6008,26,468540682
6011,27,468546693
7515,28,468554208
6010,29,468560218
7514,30,468567732
6012,31,468573744
9017,32,468582761
6010,33,468588771
6012,34,468594783
7514,35,468602297
6011,36,468608308
7514,37,468615822
6011,38,468621833
6011,39,468627844
7514,40,468635358
6011,41,468641369
6011,42,468647380
7514,43,468654894
6012,44,468660906
6011,45,468666917
7514,46,468674431
6011,47,468680442
9017,48,468689459
6011,49,468695470
7514,50,468702984
6011,51,468708995
6011,52,468715006
7515,53,468722521
6010,54,468728531
7514,55,468736045
6012,56,468742057
6010,57,468748067
7515,58,468755582
6011,59,468761593
6011,60,468767604
7514,61,468775118
6011,62,468781129
9017,63,468790146
6011,64,468796157
7514,65,468803671
6011,66,468809682
6011,67,468815693
7514,68,468823207
6011,69,468829218
7514,70,468836732
6012,71,468842744
6011,72,468848755
7514,73,468856269
6011,74,468862280
6011,75,468868291
7514,76,468875805
6011,77,468881816
6011,78,468887827
9017,79,468896844
7514,80,468904358
6011,81,468910369
6012,82,468916381
7514,83,468923895
6010,84,468929905
6012,85,468935917
7514,86,468943431
6011,87,468949442
7514,88,468956956
6011,89,468962967
6011,90,468968978
7514,91,468976492
6011,92,468982503
6012,93,468988515
9017,94,468997532
7513,95,469005045
6011,96,469011056
6012,97,469017068
7514,98,469024582
6011,99,469030593
7514,100,469038107
6011,101,469044118
6011,102,469050129
7514,103,469057643
6011,104,469063654
6011,105,469069665
7514,106,469077179
6011,107,469083190
6012,108,469089202
7514,109,469096716
9017,110,469105733
6011,111,469111744
6011,112,469117755
7514,113,469125269
6011,114,469131280
6011,115,469137291
7514,116,469144805
6011,117,469150816
7514,118,469158330
6011,119,469164341
6012,120,469170353
7514,121,469177867
6011,122,469183878
6011,123,469189889
7514,124,469197403
9017,125,469206420
6011,126,469212431
6011,127,469218442
7514,128,469225956
6011,129,469231967
6012,130,469237979
7513,131,469245492
6011,132,469251503
6012,133,469257515
7514,134,469265029
6011,135,469271040
7514,136,469278554
6011,137,469284565
6011,138,469290576
7514,139,469298090
6011,140,469304101
9017,141,469313118
6011,142,469319129
7514,143,469326643
6011,144,469332654
6012,145,469338666
7514,146,469346180
6011,147,469352191
7514,148,469359705
6011,149,469365716
6011,150,469371727
7514,151,469379241
6011,152,469385252
6011,153,469391263
7517,154,469398780
6008,155,469404788
9018,156,469413806
6010,157,469419816
7515,158,469427331
6010,159,469433341
6012,160,469439353
7514,161,469446867
6011,162,469452878
7514,163,469460392
6011,164,469466403
6011,165,469472414
7514,166,469479928
6012,167,469485940
6010,168,469491950
7515,169,469499465
6011,170,469505476
6011,171,469511487
9016,172,469520503
7515,173,469528018
6011,174,469534029
6011,175,469540040
7514,176,469547554
6011,177,469553565
6011,178,469559576
7514,179,469567090
6011,180,469573101
7514,181,469580615
6012,182,469586627
49% frame skips

The same thing happens with 6by9's repository once I try on my 4b board.
btw I do have an HQ camera, but I don't have the software reference manual for imx477 sensor and have no idea how to hack it, so I didn't test high-speed photography on that.

[HermannSW]

So you are testing with v1 camera.
Did you ran rpi-update recently?
What is output of "uname -a" on your Pi4B?

[chillie]

So you are testing with v1 camera.
Did you ran rpi-update recently?
What is output of "uname -a" on your Pi4B?

I didn't run rpi-update recently.

Code: Select all

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.4.51-v7l+ #1326 SMP Fri Jul 17 10:51:18 BST 2020 armv7l GNU/Linux

[HermannSW]

I just asked to see which kernel you are running from.
On my 4GB Pi4B I still run 4.19 kernel, but I recently read that upgrade to 5.4 kernel was done with normal update.
I need to free up space on that Pi, in order to update and upgrade.
Will test raspiraw on that Pi4B then, v1 camera is already connected.

I think I know the problem already, with 5.4 kernel i2c device numbering changed to i2c-10 from i2c-0 before.

P.S:
The code to fix this for all tools is in this posting already:
viewtopic.php?f=43&t=109137&hilit=raspi ... 0#p1666035

Code: Select all

r=`uname -r | head --bytes 1`
if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi

And in each tool line do "... -y $i2c ..." to select 5.4 kernel i2c-10 device.

[chillie]

I just asked to see which kernel you are running from.
On my 4GB Pi4B I still run 4.19 kernel, but I recently read that upgrade to 5.4 kernel was done with normal update.
I need to free up space on that Pi, in order to update and upgrade.
Will test raspiraw on that Pi4B then, v1 camera is already connected.

I think I know the problem already, with 5.4 kernel i2c device numbering changed to i2c-10 from i2c-0 before.

P.S:
The code to fix this for all tools is in this posting already:
viewtopic.php?f=43&t=109137&hilit=raspi ... 0#p1666035

Code: Select all

r=`uname -r | head --bytes 1`
if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi

And in each tool line do "... -y $i2c ..." to select 5.4 kernel i2c-10 device.

I inserted the code snippet to the tools, say 640x64. It did improve the framerate(around 330fps) but still half of the expected(doc of raspiraw said it could reach 665fps on 2b/3b, or on 4.19 kernel). And for 640x64_s, it's still around 150fps and nothing changed. I'm wondering if I need to change sth else to let raspiraw run on 5.19 kernel?

[HermannSW]

OK, so 5.4 kernel update did break raspiraw.

Please use this camera_i2c for now, it knows how to deal with 5.4 i2c-10:
https://github.com/Hermann-SW/raspiraw/ ... camera_i2c

I attached corrected 640x64 tool for now, extract and use that:

640x64.zip

640x64.zip

(665 Bytes) Downloaded 5 times

I used it after executing camera_i2c first, and captured 2000ms.
That resulted in only slightly more than 300 frames (165fps) and not the previously possible 665fps.

I did convert frame 100 to png using "raw2ogg2anim x 100 100 1".
At least the frame processing is not broken, screw driver can be seen:

out.0100.ppm.d.png

out.0100.ppm.d.png (62.88 KiB) Viewed 296 times

I will investigate where the frame drop comes from, but most likely by 5.4 kernel update ignoring v1 camera.
The new raspistill "--focus" feature was implemented for v2 and HQ camera only, shows always 0 for v1 camera preview.

[HermannSW]

1)
The 0 frame deltas and the big negative delta come from last 4 "frames" captured:

Code: Select all

🍓 tail -6 tstamps.csv 
5228,330,2550267933
10456,331,2550278389
-2550278389,332,0
0,333,0
0,334,0
0,335,0
🍓 

Code: Select all

🍓 ls -lst /dev/shm/ | head -7
total 62228
  0 -rw-r--r-- 1 pi pi      0 Aug  2 22:25 out.0332.raw
  0 -rw-r--r-- 1 pi pi      0 Aug  2 22:25 out.0333.raw
  0 -rw-r--r-- 1 pi pi      0 Aug  2 22:25 out.0334.raw
  0 -rw-r--r-- 1 pi pi      0 Aug  2 22:25 out.0335.raw
188 -rw-r--r-- 1 pi pi 192000 Aug  2 22:25 out.0331.raw
188 -rw-r--r-- 1 pi pi 192000 Aug  2 22:25 out.0330.raw
🍓 

2)
Not everything got broken, 640x240 can still capture at 191fps with v1 camera on Pi4B:

Code: Select all

🍓 ./640x240 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 191fps requested
335 frames were captured at 191fps
frame delta time[us] distribution
      1 -2550278389
      1 
      3 0
     28 5227
    230 5228
     23 5229
     22 10455
     26 10456
      1 15683
after skip frame indices (middle column)
...

out.0102.ppm.d.png

out.0102.ppm.d.png (250.14 KiB) Viewed 266 times

[HermannSW]

There seems to be a 300fps raspiraw barrier with 5.4 kernel and Pi4B.

I did the same change for the tools as for 640x64 tool.
1) Add these lines before the first "echo" line:

Code: Select all

r=`uname -r | head --bytes 1`
if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi

2) Insert " -y $i2c" directly after "raspiraw" line in script.

Requested 282fps achieved:

Code: Select all

🍓 ./640x160 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 282fps requested
384 frames were captured at 282fps
frame delta time[us] distribution
      1 -9223372036854775808
      1 
      1 0
      1 3530
     52 3534
    145 3535
      4 3536

Requested 312fps fail:

Code: Select all

🍓 ./640x144 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 312fps requested
380 frames were captured at 156fps
frame delta time[us] distribution
      1 -9223372036854775808
      1 
      1 0
      8 3195
    127 3196
...

This is 640x144 frame captured at 288fps.
There some abnormal horizontal black lines ...

out.0103.ppm.d.png

out.0103.ppm.d.png (161.56 KiB) Viewed 238 times

I will ask in main raspiraw thread whether this is known issue.

[chillie]

Thanks a lot! I think maybe libcamera is a better choice for 5.4 kernel? I realize that there are some differences in mode registers between 5.4 kernel drivers and raspiraw sensor mode headers. I can refer to the v1 and v2 datasheet to figure out why there are differences.

[HermannSW]

I doubt that there are differences for v1 and v2 camera, since everything works up to 300fps for v1 camera, and up to 233fps for v2 camera:
viewtopic.php?f=43&t=109137&p=1706643#p1706643

I doubt that libcamera can do any high framerate capturing, it is more like raspivid than raspiraw, and always raspiraw was needed tor high framerates. Also note that libcamera has a xrandr scaling bug -- it cannot deal correctly with xrandr settings that are needed by display (I use 1360x768 19" and 1024x600 9" HDMI displays, and they need xrandr to work correctly under X11):
viewtopic.php?f=43&t=273018&p=1702786#p1692390