rayzrocket
Posts: 3
Joined: Tue Jul 30, 2019 4:57 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Aug 15, 2019 11:08 pm

I can't wait to get some C code running using the CSI-2 raspicamV2. For now my skills limit me to basic Python.
My Python3 code with CV2 frame capture on a USB camera and Numpy array manipulation running in Thonny IDE....
Track algo of Minimum sum:
2ms / 25 pixel wide area
9ms / 90 pixel wide area
12ms / 100 pixel wide area
15ms / 115 pixel wide area
20ms / 140 pixel wide area

Placing text on the frame = 2 to 3 ms
Read 25 bytes on Pi serial input @ 100kbaud = 3ms
Parse SBUS protocol into useful values = <1ms
Send out 25 bytes = 3ms
cv2.imshow (show the image on the screen) = <5ms
I sent you email of my algo and code since it's a bit large for the forum.

User avatar
HermannSW
Posts: 3138
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Aug 16, 2019 12:38 pm

Found this tweet from @MachinePix:
Removing unripe tomatoes with an optical sorter.
https://twitter.com/MachinePix/status/9 ... 5057209344

Not sure what optical sensors are used, nor have any information on the framerate in case it is a/are camera(s).
This is just small clip from slow motion part of the tweet's video, raspiraw should be able to provide support for the same functionality:
Image
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

lak4cyut
Posts: 6
Joined: Wed Aug 21, 2019 6:27 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Aug 21, 2019 9:30 am

6by9 wrote:
Thu May 30, 2019 9:46 am
The main ones:
- Licence. Any kernel driver almost has to be GPLv2 and therefore open source. Check with your sensor supplier that they are happy for the register set to be released. Copying raspiraw means that the register set can be hidden in your userspace app which can be under any licence you fancy (including closed). It's all a little silly as it is trivial to put an I2C analyser on the lines to the camera module, run your app, and capture the relevant commands.
- Framework. V4L2 is the Linux standard APIs and is therefore portable to other platforms. MMAL is specific to the Pi.
- Support. I'm less inclined to fix any issues discovered in rawcam than in V4L2. V4L2 is seen as the way forward.
rcasiodu wrote:In the first method, can you give me an example of drive file? Could I just modify the ov5647.c(/drivers/media/i2c/ov5647.c) to other sensor?
/drivers/media/i2c/ov5647.c is a relatively basic driver, but would work. imx258.c is a little more comprehensive, but lacks the dt/fwnode configuration and regulator control (not essential if you set the power control GPIO some other way). I believe ov5640.c is a reasonable example of that.
rcasiodu wrote:How to use the v4l2 driver to get raw data from sensor? Is it possible to transfer raw10/raw12 to yuv data?(use arm core?)
You use the V4L2 API to get raw frames out.
"v4l2-ctl --stream--map=3 --stream-to=foo.raw --stream-count=100" would be a simple existing tool.
https://linuxtv.org/downloads/v4l-dvb-a ... ure.c.html is the standard example for grabbing frames. Do what you want in process_image(). You want to be using the MMAP method, not read (userptr isn't supported).
Hi 6by9,
Thanks for your information.
I built a custom kernel to enable bcm2835-unicam and ov5647 kernel modules and add ov5647 dt overlay, try to capture camera image via V4L2 official interface. (Of course I bought a Raspberry Pi official camera module - OV5647 version.)
And use v4l2-ctl to capture image.

Code: Select all

v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1
But the captured image is abnormal (overlay & distortion) (I put the image below).
Would you like to provide me some hint about how to debug this problem? Or do you know what caused it.
It drive me crazy...

Thanks for the help.
frame2.jpg
frame2.jpg (79.79 KiB) Viewed 4764 times
frame.jpg
frame.jpg (122.32 KiB) Viewed 4764 times
Detail info:
Kernel base version: 4.19.64
Raspberry Pi: 3 model B v1.2
Camera Module: Raspbery Pi (OV5647) Rev 1.3

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Aug 22, 2019 10:34 am

lak4cyut wrote:
Wed Aug 21, 2019 9:30 am
Hi 6by9,
Thanks for your information.
I built a custom kernel to enable bcm2835-unicam and ov5647 kernel modules and add ov5647 dt overlay, try to capture camera image via V4L2 official interface. (Of course I bought a Raspberry Pi official camera module - OV5647 version.)
And use v4l2-ctl to capture image.

Code: Select all

v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1
But the captured image is abnormal (overlay & distortion) (I put the image below).
Would you like to provide me some hint about how to debug this problem? Or do you know what caused it.
It drive me crazy...

Thanks for the help.
Duplicate of https://www.raspberrypi.org/forums/view ... 2#p1522892, therefore I'm not responding here (it's not relevant to rawcam).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
HermannSW
Posts: 3138
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Aug 22, 2019 11:29 pm

@6by9 I want to play with my new 4GB Pi4B and raspiraw.
v2 camera works with raspivid on Pi4B.
I did clone latest raspiraw from your repo, because that knows about Pi4B.
But camera shows up as /dev/i2c-1 and not /dev/i2c-0.
I did change camera_i2c last line in 4B section to "i2cdetect -y 1" because of that.
But it shows no connected device, just double dashes everywhere.
I added "-y 1" to raspiraw call in tools/640x128, but the command fails as well.

Is this a Buster does not like raspiraw thing?
What do I need to change/do differently in order to get raspiraw working on Pi4B?
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

User avatar
HermannSW
Posts: 3138
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Aug 23, 2019 7:05 am

Sorry, my fault, I only enabled I2C in raspi-config and missed to add "dtparam=i2c_vc=on" to "/boot/config.txt".
Now raspiraw works, but I get two super huge frame skips when running 640x128 tool, investigating:

Code: Select all

pi@raspberrypi4B:~/raspiraw/c $ 640x128 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 349fps requested
495 frames were captured at 351fps
frame delta time[us] distribution
      1 -9223372036854775808
      1 
      3 0
      1 2839
      1 2841
      3 2842
    339 2843
    124 2844
      1 2848
      1 5686
      1 5687
      1 14218
      1 19903
      1 22746
      1 22747
      2 25589
      2 25590
      3 28432
      1 28433
      1 36963
      2 51179
      2 54022
      1 105201
      1 9223372036723459902
after skip frame indices (middle column)
5686,2,129328465
19903,3,129348368
54022,7,129482001
51179,8,129533180
105201,9,129638381
51179,11,129717992
54022,12,129772014
36963,16,129888588
14218,17,129902806
5687,139,130317921
9223372036723459902,491,-9223372036854775808
2% frame skips
pi@raspberrypi4B:~/raspiraw/c $

"head" looks fine:

Code: Select all

pi@raspberrypi4B:~/raspiraw/c $ head -3 tstamps.csv 
,1,129322779
5686,2,129328465
19903,3,129348368
pi@raspberrypi4B:~/raspiraw/c $

But "tail" looks ugly:

Code: Select all

pi@raspberrypi4B:~/raspiraw/c $ tail -6 tstamps.csv 
2844,490,131315906
9223372036723459902,491,-9223372036854775808
-9223372036854775808,492,0
0,493,0
0,494,0
0,495,0
pi@raspberrypi4B:~/raspiraw/c $ 



This is all to get raspiraw running:

Code: Select all

pi@raspberrypi4B:~ $ cat /etc/modules-load.d/modules.conf 
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

i2c-dev
pi@raspberrypi4B:~ $ 

Code: Select all

pi@raspberrypi4B:~/raspiraw/c $ grep i2c /boot/config.txt 
dtparam=i2c_arm=on
dtparam=i2c_vc=on
pi@raspberrypi4B:~/raspiraw/c $ 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

User avatar
HermannSW
Posts: 3138
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Aug 29, 2019 3:25 pm

Pi4B raspiraw from this branch showed problems:
https://github.com/6by9/raspiraw

So I used this repo for testing how long high framerate video becomes possible with Pi4B 2GB /dev/shm filesystem:
https://github.com/Hermann-SW/fork-raspiraw

The frame drops are higher on Pi4B than on the other PIs, I have no explanation.
I reduced the request framerate so long that finally 0% frame skips (integer division, means less than 1%) are reported.

For 640x75 tool framerate had to be reduced to 856fps (from 1007fps):

Code: Select all

...
2337,30156,12983691033
2337,30500,12984094153
0% frame skips
pi@raspberrypi4B:~/fork-raspiraw/c $ df /dev/shm
Filesystem     1K-blocks    Used Available Use% Mounted on
tmpfs            1973500 1953472     20028  99% /dev/shm
pi@raspberrypi4B:~/fork-raspiraw/c $ 
36 seconds of 640x75 video fit into /dev/shm:

Code: Select all

pi@raspberrypi4B:~/fork-raspiraw/c $ ./640x75 36000
removing /dev/shm/out.*.raw
capturing frames for 36000ms with 850fps requested
30523 frames were captured at 856fps
frame delta time[us] distribution
      1 
      1 1162
      1 1163
      7 1164
     23 1165
    550 1166
   1117 1167
  14349 1168
  12640 1169
   1116 1170
    509 1171
     14 1172
      7 1173
      1 1174
      3 2334
      6 2335
     62 2336
     69 2337
     11 2338
      1 4672
      4 4673b
      8 4674
     17 5842
      3 5843
      1 7010
      2 7011
after skip frame indices (middle column)
7011,2,12948131142
2335,384,12948578664
5842,450,12948660456
2337,552,12948780809
2336,625,12948867275
2336,895,12949183929
7010,1313,12949678189
2337,1666,12950091826
2336,1910,12950378101
2337,1923,12950394459
2336,1995,12950479756
4673,2175,12950693585
2337,2177,12950697091
2336,2680,12951285997
2336,2693,12951302355
5843,3038,12951710150
2336,3364,12952092238
2337,3464,12952210253
4674,3901,12952724377
2337,3978,12952815518
2336,4048,12952898478
2337,4235,12953118150
2337,4579,12953521270
2337,4650,12953605400
2336,4734,12953704719
2337,4749,12953723415
5842,4761,12953742110
2337,5089,12954126535
2336,5417,12954510960
2337,5519,12954631312
5842,5623,12954757506
2337,5859,12955034432
2337,6013,12955215545
2337,6014,12955217882
2337,6029,12955236578
4673,6484,12955771733
2336,6629,12955942329
2336,6696,12956021786
2336,6782,12956123441
2337,7144,12956547594
5842,7345,12956787129
2336,7466,12956929682
2337,7915,12957455491
2337,8087,12957657637
2336,8153,12957735923
4672,8208,12957803694
2336,8427,12958060756
2336,8685,12958363388
2336,8837,12958542164
2337,8942,12958666021
4674,9070,12958819090
2336,9197,12958968653
2337,9455,12959271287
2335,9520,12959348404
2337,9712,12959573918
5842,9930,12959833317
2336,9966,12959876550
2338,10116,12960052990
2336,10117,12960055326
2336,10200,12960153477
2337,10221,12960179183
2337,10479,12960481815
5842,10789,12960848713
2336,10992,12961087080
2336,11250,12961389712
2337,11508,12961692346
2336,11570,12961765958
4674,11653,12961866446
2337,11762,12961994977
2337,12020,12962297610
2336,12167,12962470543
2336,12253,12962572199
2337,12276,12962600242
5842,12513,12962881842
2336,12530,12962902874
2336,12615,12963003362
2338,12848,12963276784
2336,12934,12963378440
2336,13044,12963508139
5842,13373,12963897238
2337,13533,12964085362
2337,13556,12964113405
2337,13557,12964115742
2336,13615,12964184680
2336,13812,12964416036
4674,14235,12964913803
2337,14236,12964916140
2336,14299,12964990921
2337,14324,12965021301
2336,14582,12965323933
2336,14986,12965797162
5842,15096,12965930367
2336,15182,12966032023
2338,15583,12966501747
2336,15668,12966602234
2337,15697,12966637288
5842,15955,12966943426
2337,16127,12967145570
2336,16299,12967347715
2336,16350,12967408475
2337,16642,12967750835
5842,16814,12967956485
2335,17034,12968214715
2337,17154,12968356100
2334,17412,12968658732
2338,17634,12968919301
2337,17635,12968921638
2337,17668,12968961366
5843,17675,12968974219
2336,17714,12969020956
2337,17921,12969263997
2336,18179,12969566629
2338,18314,12969725542
2336,18350,12969768775
2336,18399,12969827197
2337,18434,12969869262
5843,18533,12969989614
2336,18688,12970171894
2336,19082,12970633438
2337,19204,12970777159
4673,19396,12971005009
2337,19459,12971079791
2337,19717,12971382425
2336,19765,12971439679
2337,19974,12971685056
4674,20258,12972020406
2336,20450,12972245919
2337,20487,12972290321
2336,20745,12972592953
5842,21122,12973038139
2336,21132,12973050992
2334,21257,12973198218
2337,21774,12973803484
2337,21775,12973805821
2336,21818,12973857232
4674,21982,12974052366
2335,22027,12974106115
2336,22503,12974663473
2337,22543,12974711380
2334,22801,12975014012
5842,22844,12975068930
2337,22882,12975114501
2338,23098,12975368058
2336,23184,12975469714
2337,23311,12975619277
2337,23655,12976022398
5842,23704,12976084326
2338,23780,12976174299
2337,23822,12976224543
2335,23865,12976275954
2337,24424,12976930295
2336,24553,12977082195
5842,24566,12977102059
2337,24591,12977132439
2337,24935,12977535560
2336,25236,12977888436
2336,25365,12978040336
4674,25427,12978116287
2337,25706,12978443457
2337,25878,12978645602
2336,25919,12978694677
5842,26290,12979132851
2337,26476,12979351354
2337,26603,12979500918
2337,26992,12979956619
5842,27152,12980148247
2337,27287,12980307159
2337,27418,12980461396
2337,27762,12980864516
2338,27934,12981066662
2336,27973,12981113399
4674,28013,12981163643
2335,28533,12981772413
2336,28658,12981919640
2337,28704,12981974558
7011,28875,12982180208
2337,29217,12982580991
2338,29253,12982624225
2338,29254,12982626563
2336,29338,12982725881
2336,29386,12982783135
2337,29730,12983186256
4673,29733,12983193266
2338,29935,12983430466
2336,30021,12983532121
2337,30156,12983691033
2337,30500,12984094153
0% frame skips
pi@raspberrypi4B:~/fork-raspiraw/c $ df /dev/shm
Filesystem     1K-blocks    Used Available Use% Mounted on
tmpfs            1973500 1953472     20028  99% /dev/shm
pi@raspberrypi4B:~/fork-raspiraw/c $ 
This is the tool:

Code: Select all

pi@raspberrypi4B:~/fork-raspiraw/c $ cat 640x75
#!/bin/bash

if [ "$1" = "" ]; then echo "format: `basename $0` ms"; exit; fi

echo "removing /dev/shm/out.*.raw"
rm -f /dev/shm/out.*.raw

fps=850
echo "capturing frames for ${1}ms with ${fps}fps requested"
raspiraw -md 7 -t $1 -ts tstamps.csv -hd0 hd0.32k --height 75 --voinc 01 --fps $fps -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null

us=`cut -f1 -d, tstamps.csv | sort -n | uniq -c | sort -n | tail -1 | cut -b9-`
l=`ls -l /dev/shm/out.*.raw | wc --lines`
echo "$l frames were captured at $((1000000 / $us))fps" 

echo "frame delta time[us] distribution"
cut -f1 -d, tstamps.csv | sort -n | uniq -c

echo "after skip frame indices (middle column)"
grep "^[2-9]" tstamps.csv

skips=`grep "^[2-9]" tstamps.csv | wc --lines | cut -f1 -d\ `
stamps=`wc --lines tstamps.csv | cut -f1 -d\ `
per=`expr \( 100 \* $skips \) / \( $skips + $stamps \)`
echo "$per% frame skips"
pi@raspberrypi4B:~/fork-raspiraw/c $ 

Reduction of framerate to 615fps (from 667fps) was necessary to get 0% frame skip reporting:

Code: Select all

...
3233,19469,12567260770
3231,19479,12567278549
0% frame skips
pi@raspberrypi4B:~/fork-raspiraw/c $ df /dev/shm
Filesystem     1K-blocks    Used Available Use% Mounted on
tmpfs            1973500 1961200     12300 100% /dev/shm
pi@raspberrypi4B:~/fork-raspiraw/c $ 
In total 32 seconds of 640x128 video can be captured into /dev/shm:

Code: Select all

raspberrypi4B:~/fork-raspiraw/c $ ./640x128 32000
removing /dev/shm/out.*.raw
capturing frames for 32000ms with 615fps requested
19612 frames were captured at 618fps
frame delta time[us] distribution
      1 
      6 1611
     13 1612
     10 1613
     19 1614
     78 1615
  11904 1616
   7372 1617
     30 1618
     16 1619
      8 1620
     15 1621
      3 1622
      1 3229
      6 3231
     49 3232
     50 3233
      1 4848
     14 4849
      9 6465
      7 6466
after skip frame indices (middle column)
6465,494,12536299075
3233,637,12536531834
3233,669,12536585175
3233,678,12536601339
3232,692,12536625584
3233,694,12536630433
3232,695,12536633665
4849,1112,12537310927
3233,1128,12537338406
3232,1181,12537425690
3233,1182,12537428923
3233,1183,12537432156
3232,1186,12537438621
3232,1684,12538245193
6465,1731,12538326011
3232,2169,12539035601
3233,2174,12539045300
3233,2177,12539051766
6465,2353,12539341096
3232,2529,12539627194
3232,2653,12539829242
3233,2654,12539832475
3233,2655,12539835708
3231,2668,12539858336
4849,2973,12540354564
3233,3115,12540585707
3232,3145,12540635814
3233,3456,12541140123
4848,3596,12541369648
3233,3636,12541435921
3233,3637,12541439154
3233,3638,12541442387
3232,3656,12541473097
3233,4096,12542185920
3232,4126,12542236027
3232,4128,12542240876
3232,4132,12542248958
3232,4150,12542279669
4849,4214,12542386349
3233,4565,12542955314
3233,4626,12543055531
3233,4643,12543084625
4849,4836,12543399817
3232,5109,12543842705
3233,5110,12543845938
3232,5137,12543891196
4849,5459,12544414902
3233,5570,12544595937
3232,5599,12544644428
3233,5600,12544647661
3233,5629,12544696153
6465,6080,12545429987
3231,6124,12545502723
3233,6587,12546252723
3233,6588,12546255956
3232,6604,12546283434
3231,6619,12546309295
6465,6701,12546446687
3233,7071,12547046364
3233,7072,12547049597
3233,7074,12547054446
3232,7095,12547090006
3232,7110,12547115868
3233,7218,12547292053
4849,7320,12547460155
3232,7589,12547896578
3232,7604,12547922440
4849,7944,12548475240
3232,8055,12548656275
3233,8056,12548659508
3233,8082,12548703150
3233,8083,12548706383
3232,8096,12548729012
3233,8548,12549461231
6465,8563,12549490325
3232,8590,12549535583
6466,9186,12550503793
3233,9529,12551059827
3232,9531,12551064676
3233,9582,12551148728
6466,9807,12551517261
3233,9937,12551729006
3233,10018,12551861550
3233,10020,12551866399
3232,10059,12551931054
3233,10073,12551955300
4849,10427,12552530729
3232,10554,12552737626
3232,10568,12552761872
6466,11051,12553547431
3232,11063,12553568443
3232,11547,12554352386
6466,11675,12554564131
3232,12042,12555158958
3229,12055,12555181587
4849,12298,12555577599
3233,12525,12555946134
3233,12550,12555988160
4849,12921,12556591067
3233,13016,12556746240
3232,13021,12556755938
3233,13044,12556794732
3232,13508,12557546346
3231,13541,12557601302
4849,13542,12557606151
3232,14024,12558386862
6465,14166,12558621236
3232,14519,12559193434
3232,14531,12559214447
4849,14789,12559634704
3233,15014,12560000006
3233,15026,12560021019
4849,15412,12560648172
3232,15509,12560806578
3232,15521,12560827590
3233,16006,12561613150
3232,16018,12561634162
6465,16034,12561664873
6466,16659,12562679958
3232,16971,12563185884
3233,17007,12563245691
6466,17281,12563693426
3231,17502,12564052261
4849,17905,12564706894
3232,17987,12564841054
3232,17997,12564858834
3232,18452,12565595902
3232,18470,12565626613
3232,18482,12565647626
3232,18492,12565665406
6465,18526,12565725211
3232,18971,12566446116
3233,18975,12566454198
3232,18985,12566471978
6466,19148,12566740296
3233,19469,12567260770
3231,19479,12567278549
0% frame skips
pi@raspberrypi4B:~/fork-raspiraw/c $ df /dev/shm
Filesystem     1K-blocks    Used Available Use% Mounted on
tmpfs            1973500 1961200     12300 100% /dev/shm
pi@raspberrypi4B:~/fork-raspiraw/c $ 
This is the tool:

Code: Select all

pi@raspberrypi4B:~/fork-raspiraw/c $ cat 640x128 
#!/bin/bash

if [ "$1" = "" ]; then echo "format: `basename $0` ms"; exit; fi

echo "removing /dev/shm/out.*.raw"
rm -f /dev/shm/out.*.raw

fps=615
echo "capturing frames for ${1}ms with ${fps}fps requested"
raspiraw -md 7 -t $1 -ts tstamps.csv -hd0 hd0.32k --height 128 --top 0 --vinc 17 --fps $fps -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null

us=`cut -f1 -d, tstamps.csv | sort -n | uniq -c | sort -n | tail -1 | cut -b9-`
l=`ls -l /dev/shm/out.*.raw | wc --lines`
echo "$l frames were captured at $((1000000 / $us))fps" 

echo "frame delta time[us] distribution"
cut -f1 -d, tstamps.csv | sort -n | uniq -c

echo "after skip frame indices (middle column)"
grep "^[2-9]" tstamps.csv

skips=`grep "^[2-9]" tstamps.csv | wc --lines | cut -f1 -d\ `
stamps=`wc --lines tstamps.csv | cut -f1 -d\ `
per=`expr \( 100 \* $skips \) / \( $skips + $stamps \)`
echo "$per% frame skips"
pi@raspberrypi4B:~/fork-raspiraw/c $ 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

User avatar
HermannSW
Posts: 3138
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 04, 2019 8:25 am

I wanted to watch the updating phase of a max7219 8-digit tube 7-segment display.
This was the first time I was able to use 640x64 tool to record 665fps video with v1 camera not for speed demonstration purpose.

The animations were extracted from two videos, from a phase without frame skips (2% frame skips in total for the videos).
The "8." on left is fixed, gets cleared first when printing a new value, and is directly written after printing.
So "8." displayed fully shows a real value on the right, otherwise value on right is in transition.

It takes less than 3/665=4.5ms to print all 8 digits (clear space left of number):
Image

For 1/100th second clock example you can see updates every 6th or 7th frame:
Image

https://github.com/Hermann-SW/DigitLedDisplay

What was remarkable is this:
For taking 665fps raspiraw video in the past I had to provide bright lighting to the scene.
Here scene around display was dark, with NO additional lighting.
The display tubes were bright enough to display nicely with only 1.5ms shutter time at 665fps.
And that was not even full brightness possible, was only 10 of 1-15 range.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

LinuxFanatic777
Posts: 13
Joined: Fri Nov 15, 2013 9:35 am

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 09, 2020 8:53 am

I'm using Lattice Crosslink to convert my image sensor stream to MIPI CSI-2 D-PHY. Is it possible to capture such stream using raspiraw without I2C access? I do not need to configure any of parameters, just catch this stream 25 fps 640x480. I2C interface is not available on my board. Another boards with CSI-2 is able to receive stream without I2C being accessed, but how to do the same on Raspbery Pi 3 Model B board?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 09, 2020 9:31 am

LinuxFanatic777 wrote:
Thu Jan 09, 2020 8:53 am
I'm using Lattice Crosslink to convert my image sensor stream to MIPI CSI-2 D-PHY. Is it possible to capture such stream using raspiraw without I2C access? I do not need to configure any of parameters, just catch this stream 25 fps 640x480. I2C interface is not available on my board. Another boards with CSI-2 is able to receive stream without I2C being accessed, but how to do the same on Raspbery Pi 3 Model B board?
Yes. rapsiraw is using i2c to detect which of the CSI2 devices that it knows about is connected, and then to configure those devices as most don't spontaneously stream data.

If you only want to support a single device, then modify it to remove the probe and setup lumps, and hardcode your 640x480 resolution, and the format of this stream, and the resulting buffers will contain your data.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

LinuxFanatic777
Posts: 13
Joined: Fri Nov 15, 2013 9:35 am

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 09, 2020 1:20 pm

6by9 wrote:
Thu Jan 09, 2020 9:31 am
Yes. rapsiraw is using i2c to detect which of the CSI2 devices that it knows about is connected, and then to configure those devices as most don't spontaneously stream data. If you only want to support a single device, then modify it to remove the probe and setup lumps, and hardcode your 640x480 resolution, and the format of this stream, and the resulting buffers will contain your data
Thank you for this tip. However, after this step which is just to evaluate receiving possibility, then I need to take hardware H264 compression in action. Do raspivid have ability to receive and compress without I2C? Raspivid sources available, I suppose you familiar with these sources too... :roll:

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 09, 2020 1:50 pm

LinuxFanatic777 wrote:
Thu Jan 09, 2020 1:20 pm
Thank you for this tip. However, after this step which is just to evaluate receiving possibility, then I need to take hardware H264 compression in action. Do raspivid have ability to receive and compress without I2C? Raspivid sources available, I suppose you familiar with these sources too... :roll:
Raspivid/still will only work with the supported camera modules as it is using the GPU firmware camera stack.
For reference, Raspivid source is at https://github.com/raspberrypi/userland ... RaspiVid.c

You don't say what format your device is generating.
If Bayer, then the ISP is the only hardware block that can consume it. You'll also need to look at how to control all the parameters for converting it to a useful format.
If YUV then most CSI2 devices produce one of the 4 flavours of YUYV (YUV422). This is accepted by some of the hardware blocks (mainly ISP again), but not all (eg not the Hardware Video Scalar for display). The permutations of RGB, likewise, are acceptable to some hardware blocks but not all.

The vc.ril.video_encode component will accept many RGB and YUV formats and internally converts them. Depending on whether your source is Bayer or YUV/RGB you want to be looking at either callback() (for RGB/YUV) or yuv_callback (for Bayer) within raspiraw.

Taking callback() as the example, you'll see there is a clause if (dev->isp_ip) that then passes the buffer to the ISP for processing. You want to duplicate all the associated setup for that and send the data to a vc.ril.video_encode component as well. You'll need to set up the encoder's output format appropriately with desired codec, bitrate, resolution, etc, and you'll want a new callback which can save the encoded data.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

LinuxFanatic777
Posts: 13
Joined: Fri Nov 15, 2013 9:35 am

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 10, 2020 4:21 pm

6by9 wrote:
Thu Jan 09, 2020 1:50 pm
You don't say what format your device is generating
Currently RGB 24-bit. I can change this to any more suitable 16-bit format, including YUYV. My dream is to capture 1 frame, don't care about color correctness
6by9 wrote:
Thu Jan 09, 2020 1:50 pm
Taking callback() as the example, you'll see there is a clause if (dev->isp_ip) that then passes the buffer to the ISP for processing. You want to duplicate all the associated setup for that and send the data to a vc.ril.video_encode component as well. You'll need to set up the encoder's output format appropriately with desired codec, bitrate, resolution, etc, and you'll want a new callback which can save the encoded data
I understand the following. Some MMAL unit (whichever it means) sends data further to it's sinks. Some of them passing data to the encoder module, the other sinks is final point with raw bytes data for raspiraw. There what I got:

Code: Select all

./raspiraw -f 25 -w 640 -h 480
Using i2C device /dev/i2c-0
sensor_crosslink
RaspiRaw: Encoding 36314742
No AWB
mmal: Set pack to 0, unpack to 0
mmal: Timing 6/2, 2/6/0, 0/0
mmal: Create pool of 6 buffers of size 614400
mmal: Create pool of 6 buffers of size 614400
...
mmal: Buffer 0xdbe520 returned, filled 0, timestamp 0, flags 0000
mmal: Buffer 0xdbe6f8 returned, filled 0, timestamp 0, flags 0000
...
Buffers are empty. When I switch Crosslink board power off then nothing change - same filled 0

I'm curious, is there any signs that Crosslink sending correct frames? Just how to prove that Raspberry processor receiving anything? May be my frames over CSI-2 is completely wrong, or wrong timings, or no frames at all? Is there error codes for CSI-2 stream? Is it correct to send CSI-2 frames before MMAL got ready? Total mess in my head

Not sure whether raspberry see my stream at all. How to debug that using MMAL, is there any counters or status codes? Debug complexity level: mission impossible :( :cry:

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 10, 2020 4:58 pm

LinuxFanatic777 wrote:
Fri Jan 10, 2020 4:21 pm
6by9 wrote:
Thu Jan 09, 2020 1:50 pm
You don't say what format your device is generating
Currently RGB 24-bit. I can change this to any more suitable 16-bit format, including YUYV. My dream is to capture 1 frame, don't care about color correctness
RGB is fine, although there isn't a defined CSI2 data type value for RGB formats.
LinuxFanatic777 wrote:
6by9 wrote:
Thu Jan 09, 2020 1:50 pm
Taking callback() as the example, you'll see there is a clause if (dev->isp_ip) that then passes the buffer to the ISP for processing. You want to duplicate all the associated setup for that and send the data to a vc.ril.video_encode component as well. You'll need to set up the encoder's output format appropriately with desired codec, bitrate, resolution, etc, and you'll want a new callback which can save the encoded data
I understand the following. Some MMAL unit (whichever it means) sends data further to it's sinks. Some of them passing data to the encoder module, the other sinks is final point with raw bytes data for raspiraw. There what I got:

Code: Select all

./raspiraw -f 25 -w 640 -h 480
Using i2C device /dev/i2c-0
sensor_crosslink
RaspiRaw: Encoding 36314742
No AWB
mmal: Set pack to 0, unpack to 0
mmal: Timing 6/2, 2/6/0, 0/0
mmal: Create pool of 6 buffers of size 614400
mmal: Create pool of 6 buffers of size 614400
...
mmal: Buffer 0xdbe520 returned, filled 0, timestamp 0, flags 0000
mmal: Buffer 0xdbe6f8 returned, filled 0, timestamp 0, flags 0000
...
Buffers are empty. When I switch Crosslink board power off then nothing change - same filled 0

I'm curious, is there any signs that Crosslink sending correct frames? Just how to prove that Raspberry processor receiving anything? May be my frames over CSI-2 is completely wrong, or wrong timings, or no frames at all? Is there error codes for CSI-2 stream? Is it correct to send CSI-2 frames before MMAL got ready? Total mess in my head

Not sure whether raspberry see my stream at all. How to debug that using MMAL, is there any counters or status codes? Debug complexity level: mission impossible :( :cry:
Please start a new thread if you want more comprehensive debug of your particular application.

You can get some more debug from the component by running "vcgencmd set_logging level=0xc0" before running raspiraw, and "sudo vcdbg log msg" afterwards. It will log if CRC or other errors are found in the CSI2 data.
You'll also get the log line "unicam_int_callback: instance=%d status = 0x%x, fi_req=%d, fi_ready=%d" for every interrupt that fires. status is the main value of use, and is a bitmask. 0x08 is frame start, 0x10 is frame end, 0x20 are line counts (you should get 4 of those per frame), 0x01 is CRC error. Other values are more obscure events. Most genuine errors will be reported in text anyway.

"I'm curious, is there any signs that Crosslink sending correct frames? Just how to prove that Raspberry processor receiving anything?" See the logging.
"Is there error codes for CSI-2 stream?" Oh yes, lots!
"Is it correct to send CSI-2 frames before MMAL got ready?" That's fine, although you may get a partial frame if a frame end is seen when the frame start was before the peripheral was enabled.

The MMAL component has partially been superceded by the open source V4L2 driver - https://github.com/raspberrypi/linux/bl ... 5-unicam.c.
To work with that driver you'd need to create a dummy sensor subdevice driver that reported back the supported pixel formats and resolution, but otherwise did nothing. Doing so shouldn't be more than 30mins work, although getting device tree to load it may be different.
With the CSI2 receiver code being open you can add additonal logging in there should it be needed.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

LinuxFanatic777
Posts: 13
Joined: Fri Nov 15, 2013 9:35 am

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 10, 2020 8:04 pm

6by9 wrote:
Fri Jan 10, 2020 4:58 pm
Please start a new thread if you want more comprehensive debug of your particular application
Thank you, I will create a new thread. You already gave me at lot of info how to debug, this will help me greatly! I think of creation raspiraw_noi2c fork, hope you will merge it into your mainline, let's make a noi2c option to raspiraw ^_^

devmonkey
Posts: 16
Joined: Tue Jul 05, 2016 7:38 pm

Raspiraw buffer size bug / line padding

Wed Jan 22, 2020 6:17 pm

Hi,

I've altered raspiraw for my own purposes to take a bayer 3280x480 strip from the middle of the IMX219 sensor. I noticed that the buffers allocated and returned were too large, I also tested this with the OV5497 sensor and the same is true for full resolution.

I have managed to decode the data by treating the extra bytes as padding equally appended to the end of each pixel row, but I think there must be a bug in how the MMAL rawcam component is determining buffer_size_recommended in the port format commit.

Currently the raspiraw takes whatever width and height and VCOS_ALIGN_UP(X, 16) to nearest 16 bytes. These are then passed to the port commit presumably with the bit depth. Anyway with 3280 width I get an extra 28 bytes per row, I believe rawcam for 3280x480xRAW10 must be calculating the following (complete guess as I don't have the source code):

Code: Select all

buffer_size_recommended = VCOS_ALIGN_UP(3280*5/4, 32) * VCOS_ALIGN_UP(480, 32)
Somehow this is also making it into the code that writes a CSI-2 packet body for each line into the buffer, presumably the write stride is set to VCOS_ALIGN_UP(3280*5/4, 32) as well.

This means for widths that are not multiples of both 16 and 32 you will get padding bytes after each row which is why it doesn't manifest for 640x480 which most people on this thread are using. Maybe this was already a known feature (say the underlying receiver only deals with 32 byte reads) and is already dealt with somewhere else in the tool chain, I noticed it as I'm punting the raw buffer bytes out over a socket to another application on a PC that is decoding the pictures.

I've fixed this in my receiving application by calculating the total number of bytes expected

Code: Select all

bytesExpected = width*height*5/4
bytesReceived = buffer->length
extraBytes = bytesReceived-bytesExpected
extraBytesPerRow = extraBytes/height
I then skip extraBytesPerRow at the end of each pixel row.

To get the centred strip of 480 pixels I altered these camera IMX219 registers (since -top is not coded for the v2.1 camera):

Code: Select all

      {0x0164, 0x00},	//x_addr_start
      {0x0165, 0x00},	//x_addr_start
      {0x0166, 0x0c},	//x_addr_end
      {0x0167, 0xcf},	//x_addr_end
      {0x0168, 0x05},	//y_addr_start	1400
      {0x0169, 0x78},	//y_addr_start
      {0x016a, 0x07},	//y_addr_end	1880
      {0x016b, 0x58},	//y_addr_end
      {0x016c, 0x0c},	//x-size	3280
      {0x016d, 0xd0},	//x-size
      {0x016e, 0x01},	//y-size	480	
      {0x016f, 0xe0},	//y-size
Also worth noting that without doing this I got a dark band of pixels at the top of the image, I think these may have been the reference black pixels or at least pixels you aren't supposed to read.

User avatar
HermannSW
Posts: 3138
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Jan 22, 2020 11:23 pm

v1 and v2 cameras align width/height to 32/16, you can see this in source code fore eg raspivid:
https://github.com/6by9/userland/blob/m ... id.c#L1629

I implemented "-top" for raspraw, but only for v1 camera (ov5647 sensor):
https://github.com/6by9/raspiraw/blob/m ... aw.c#L1587
You can see here how to separate the code for v1 and v2 camera:
https://github.com/Hermann-SW/fork-rasp ... aw.c#L1043

You could fork 6by9's raspiraw branch, add your "-top" code for v2 camera (imx219 sensor), commit&push and do a pull request to 6by9's branch.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

devmonkey
Posts: 16
Joined: Tue Jul 05, 2016 7:38 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 23, 2020 8:54 am

HermannSW wrote:
Wed Jan 22, 2020 11:23 pm
v1 and v2 cameras align width/height to 32/16, you can see this in source code fore eg raspivid:
https://github.com/6by9/userland/blob/m ... id.c#L1629
I see raspivid does 32/16 but raspiraw is doing 16/16 on the rawcam output then later 32/16 on the ISP I think. I'm a bit confused tbh, what is the ISP doing in raspiraw, is this for the YUV part it looks to me like callback() is taking buffers directly from rawcam? Is there a more minimal example that just grabs raw bayer data from the camera? I have a vague recollection of MMAL sinks forcing their input formats on MMAL sources they are linked to, maybe that is what is going on.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 23, 2020 3:06 pm

I think you haven't grasped the difference between width and height in the port format, and crop.width and crop.height.
width and height define the overall buffer geometry, sometimes referred to the stride and frame height. Use mmal_encoding_width_to_stride to convert width in pixels to stride in bytes.
crop.width and crop.height define the active area of the buffer.

Various hardware blocks have different restrictions on strides and/or frame heights. Unicam (rawcam) requires 16 byte alignment, whilst the ISP requires 32 byte alignment. You therefore have to take the greater of the two.
Yes, the ISP is there to convert to YUV (or RGB) as the majority of formats that can be received over CSI2 are incompatible with the display pipeline.
I think you may be right that the current implementation does have a bug in that it mixes and matches. The ISP component will sanity check the input buffers and complain should they be too small for the defined image by returning the buffers without creating any output. If we'd used a MMAL_CONNECTION_T to connect the two components then it would have raised an error, but instead we're replicating buffer headers within the app so that they can be sent to multiple destinations simultaneously, and the source buffer is only returned to the output port when all consumers have finished with the buffer.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

devmonkey
Posts: 16
Joined: Tue Jul 05, 2016 7:38 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Jan 23, 2020 11:50 pm

Thanks for the response, I understand the difference between crop and frame but I didn't know about mmal_encoding_width_to_stride, but now I've looked further through your code I see you use it for reading pixels out of the buffer, my bad for not reading all of the example app.

Just so I understand this is how rawcam is configured:

Code: Select all

	output->format->es->video.crop.width = sensor_mode->width;
	output->format->es->video.crop.height = sensor_mode->height;
	output->format->es->video.width = VCOS_ALIGN_UP(sensor_mode->width, 16);
	output->format->es->video.height = VCOS_ALIGN_UP(sensor_mode->height, 16);
	output->format->encoding = encoding;
	status = mmal_port_format_commit(output);
	output->buffer_size = output->buffer_size_recommended;
So effectively on the format commit rawcam is doing:

Code: Select all

	buffer_size_recommended = mmal_encoding_width_to_stride(format->encoding,  format->es->video.width) * format->es->video.height;
And mmal_encoding_width_to_stride is aligning to 32 bytes, i.e. 3280 / RAW10 -> VCOS_ALIGN_UP((3280*5)/4, 32)? This is where the 28 bytes of padding per row come from (4100->4128bytes).

What confused me was that with 640 width there was no padding but this is because 640*5/4 = 800 = 25 * 32.

I've now ripped out all the ISP stuff and now just receive the bayer buffers.

Another question, if I want to tune the exposure time on the fly can I do this by just writing to the exposure registers over i2c as and when I want to, without having to stop and restart the sensor?

Thanks again for your help.

devmonkey
Posts: 16
Joined: Tue Jul 05, 2016 7:38 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 24, 2020 8:59 am

Please ignore the exposure question, I've just tried it and it works.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 24, 2020 9:06 am

devmonkey wrote:
Fri Jan 24, 2020 8:59 am
Please ignore the exposure question, I've just tried it and it works.
Yes, you can send an exposure (or analogue gain change at any time.
Be aware that exposure changes generally get applied 2 frames later, and gain changes 1 frame later. If you read the PiCamera docs on how rolling shutter sensors work then it is fairly obvious that they can't immediately reset to new values, and you don't want them changing mid frame.
As the settings are split over multiple registers you may want to consider using group hold features of the sensors. These allow you to apply a set of changes atomically (it holds off actioning the changes until you release the group hold).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

devmonkey
Posts: 16
Joined: Tue Jul 05, 2016 7:38 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 24, 2020 10:36 am

My application is detecting the location of the peak of the gaussian profile of a red laser line crossing the raw sensor (no lens, all filters removed other than CFA). I'm extracting just the red pixels from the bayer data. If the detected gaussian is clipped (has a flat top) it means the pixels are saturated and integration time (exposure) needs to be reduced, otherwise increased, with some hysteresis. My processing pipeline discards frames which were over or under exposed by this definition so it doesn't really matter if there is a delay in the sensor reacting. The sensor/laser frame of reference is static during measurement so rolling shutter is not an issue.

Will look at the group write feature, currently my exposure values are all <0xFF so only the lower register needs to be written. I set the analogue gain to 1, will have to play with this and measure the noise level when changing it.

Do you know if these sensors apply any lens shading correction to boost the gain of pixels on the periphery? Also do you know if it is possible to disable defect pixel correction?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9839
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 24, 2020 11:32 am

devmonkey wrote:
Fri Jan 24, 2020 10:36 am
Do you know if these sensors apply any lens shading correction to boost the gain of pixels on the periphery?
I don't believe they normally do. There are some potential effects from Chief Ray Angle (CRA), but that's physics rather than additional processing that can be controlled.
devmonkey wrote:Also do you know if it is possible to disable defect pixel correction?
DPC is normally a function of the ISP, not of the sensor. The exception is where they have integrated an ISP into the sensor itself, so it's producing YUV or RGB images instead of Bayer (neither of the two Pi camera modules do this).
We generally treat raw Bayer sensors as dumb things that send out the raw pixels as they are read off the sensor. The processing available in the ISP is generally superior to that available in the sensor.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

devmonkey
Posts: 16
Joined: Tue Jul 05, 2016 7:38 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Jan 24, 2020 12:18 pm

The IMX219 datasheet has a section on defective pixel tables held in NVM on the sensor, I don't know if any of these have been set, presumably not as it would have required the factory to individually calibrate each sensor which would have been expensive. If I wanted to dump the NVM do you have any pointers?

According to https://arxiv.org/pdf/1911.13295.pdf the microlens array does cause cross talk proportional to pixel distance from the centre when the ray incidence is normal to the sensor. I'm hoping this isn't really a problem for my application as, although flat field, is treating each pixel row independently and it knows that the laser profile is gaussian so can detect and discard both cross talk and vignetting effects, which will show up as deviations (skew) from a true gaussian. Yet to prove this to myself though ;-)

Return to “Camera board”