PierreG
Posts: 1
Joined: Fri Dec 01, 2017 11:34 am

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Dec 07, 2017 1:08 pm

Hi everybody!
I'm thinking about interfacing another sony imx sensor (224) with Rpi3.
What is the requirements to get raw image with raspiraw?
1) Design/use a sensor board
2) Connect i2c/clock/CSI between sensor board and rpi
3) Define the sensor registers and mode in *_modes.h file
4) run raspiraw, debug and pray

Did someone try to connect another sensor than imx219 and ov5647?

Thanks a lot!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4604
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Dec 07, 2017 4:32 pm

AlexEts wrote:
Thu Dec 07, 2017 5:46 am
6by9 wrote:
Wed Nov 29, 2017 1:01 pm
....
To get 8 bit data through you need to either set the sensor to produce only 8 bits (and reset rawcam etc to expect the new data type value and packing), or you could run 10 bit and then set expsoure time and/or analogue gain to result in only the bottom 8 bits being used (likely to give worse image quality as all the analogue stages are then closer to the noise floor).
I tried to go along the 1st way by setting following IMX219 registers: 0x0155 COMP_ENABLE_A(compression 10 to 8 mode), 0x018C 0x018D CSI_DATA_FORMAT_A

Code: Select all

 
{0x0155, 0x01}
{0x018C, 0x08}
{0x018D, 0x08}
+
rx_cfg.unpack = MMAL_CAMERA_RX_CONFIG_UNPACK_NONE;
rx_cfg.pack = MMAL_CAMERA_RX_CONFIG_PACK_NONE;
output->format->encoding = MMAL_ENCODING_BAYER_SBGGR8

But no luck. Output data is little bit messy and it looks like pixels were shifted. I suspect that it is due to different output method used by the sensor for 8 bit mode
0x0155 enables DPCM 10 to 8 compression which is not what you want - see Annex E "Data Compression for RAW Data Types" in the CSI2 spec for details of what that is doing. The hardware supports unpacking that back out to 10bpp data, but that doesn't really help you.
I'm expecting there to be a register that switches to sending just the 8 most significant bits of the 10 bit value - 0x018C and 0x018D do look plausible, but don't enable compression with 0x0155. You need to alter the CSI data type to be 0x2A to match the sensor for it to work.
Whilst I'd expect that to work, it doesn't for me :(
AlexEts wrote:Is there any chance 8 bit mode will be correctly supported by the rawcam? ignoring 5th byte of RGRGrgrgr sequence will be perfect, as well as clipping last last 2 bits instead of top 2 of 10 bit data? (Sorry for the naive question, not sure how the vc.ril.rawcam component handles the data internally)
rawcam is just programming up the hardware (it doesn't touch the pixel data itself) and it's the hardware that isn't shifting the data up and down as I was expecting.
Likelihood of changes in the hardware? Sadly zero.
AlexEts wrote:I trying to process rawcam's output in OpenGL as fast as I can, but 10->8 bit conversion on CPU side takes tooo long which slows down overall performance.
rawcam in 16 bit mode seems to have the same clipping problem. I think 10 bit data get clipped to last 8 bits and then get promoted to 16 bit. At least this is what I see from the raw output.
Playing with exposure and gain is not an option for me as quality of the image will be poor(((
Last time I looked the unpacking to 16 bit mode was correct, although it is leaving the 10 bits of data in the least significant bits of the 16 bit word.
Testing again does show that I have a bug in my version of dcraw for raw16, but the actual data is good. It's LSB first, then MSB, which is not how an ARM will interpret it if cast as a uint16. I've pushed a fix to my version of dcraw (must split that off as a new repo).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4604
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Dec 07, 2017 4:49 pm

I've moved the modified version of dcraw out of my hacky RPiTest repo into https://github.com/6by9/dcraw
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Dec 07, 2017 5:48 pm

HermannSW wrote:
Thu Dec 07, 2017 11:42 am
Gavinmc42 wrote:
Thu Dec 07, 2017 1:56 am
I noticed in the video the rest of the image going from grey to black.
Is auto contrast/white balance or something on?
What is causing this?
I don't know, it is not an effect of animated .gif creation, because you can see that on the youtube video as well.

I was told as child not to take photos direction of sun, maybe this can be an explanation (flame == sun)?

I took another 600fps video, intentionally "into another sun" :D
In Germany we have 50Hz power network, wich should have a repeating pattern all 600/(50*2)=6 frames in 600fps video.
And it has!
I modified the conversion script to only take the last 24 frames, and to create a video with display framerate of 1fps for easy inspection:
(I did capture with "-t 120" only 120ms of video, and got 71 frames at 600fps. Then I did set index=48 in modified conversion script)
Image

Image
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Dec 07, 2017 7:02 pm

Back at our house I connected Pi 2B to HDMI monitor.
After I installed "mplayer" ...
https://github.com/notro/fbtft/wiki/Fra ... se#mplayer

... I was able to watch the generated "bulk.600fps.1ps.ogg" video on HDMI monitor looping by this command:

Code: Select all

$ mplayer -loop 0 -nolirc -vo fbdev2:/dev/fb0  bulb.600fps.1ps.ogg

I tid take two more "into the sun" 600fps videos, animated .gif from video consists of 24 frames with display speed 1fps.

The halogen lamp does not indicate any changes ...
Image

... but the halogen bulbs at room ceiling 4m away show the same 6 frame periodicity because of 50Hz power netwok in Germany: Left to the bulbs is my head. In this video the same overall brightness changes that Gavinmc42 asked about for the candle light video happen. But the frequency of that effect is here much higher.
Image
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Dec 07, 2017 9:02 pm

I got new gear motors that should do 2000rpm.
I calculated that less than 20 frames per motor axis rotation are needed at 600fps video.
Here you see animated .gif generated from video, this time at 2fps:
Image

I did record with "-t 100" and got 56 frames.
The number of frames for 1 rotation of gear motor axis can be conted as between 16 and 17.
Motor was powered with 12.03V. The rpm (rotations per minute) range matches:

Code: Select all

$ bc -ql
1/(17/(56/0.1))*60
1976.47058823529411783260
1/(16/(56/0.1))*60
2100.00000000000000063000

Measurement could be done much more precise.
Running raspiraw with "2>/dev/shm/err" captures all frame timestamps:

Code: Select all

...
mal: Buffer 0x7a12a0 returned, filled 51200, timestamp 8943100358, flags 0084
mmal: Buffer 0x7a0d18 returned, filled 51200, timestamp 8943102009, flags 0004
mmal: Buffer 0x7a0ef0 returned, filled 51200, timestamp 8943102009, flags 0084
mmal: Buffer 0x7a10c8 returned, filled 51200, timestamp 8943103660, flags 0004
mmal: Buffer 0x7a12a0 returned, filled 51200, timestamp 8943103660, flags 0084
mmal: Buffer 0x7a0d18 returned, filled 51200, timestamp 8943105311, flags 0004
...

It is important for such application to store standard error output "err" on ramdisk as well.

Btw, less than 60 frames for 0.1s is caused by skipped frames, timestamp deltas show 605fps:

Code: Select all

$ bc -ql
1000000/(8943102009-8943100358)
605.69351907934585099939
1000000/(8943103660-8943102009)
605.69351907934585099939
1000000/(8943105311-8943103660)
605.69351907934585099939

Then just step through the converted frames, find two just one rotation apart, lookup both frame's microsecond timestamps t0 and t1, and get rpm as 60,000,000/(t1-t0).

Image
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Dec 08, 2017 5:29 pm

rpdom wrote:
Sun Dec 03, 2017 10:34 am
You might get better performance if you just dump the output of raspiraw into a black hole. (I don't have a suitable setup, so I can't check this), unless you really need that output.
My previous response that I tested that and it makes no difference was based on modes with 180fps or 360fps. For these modes the statement remains true.

But for >=600fps modes it definitely makes a difference. The standard output does only contain 1 line and can be redirected to /dev/null. Standard error output can be directed to /dev/null as well if you do not need frame microsecond timestamps. If timestamps are needed, redirecting to /dev/shm/err is better.

In 460fps range only with err output to SD card:

Code: Select all

$ rm /dev/shm/out*raw
$ time ( raspiraw -hd -md 9 -t 1000 -sr 1 -o /dev/shm/out.%04d.raw 2>err >/dev/null )

real	0m1.091s
user	0m0.090s
sys	0m0.470s
$ ll /dev/shm/out*raw | wc --lines
463
$ 

In 580-590fps range with "2>/dev/shm/err":

Code: Select all

$ rm /dev/shm/out*raw
$ time ( raspiraw -hd -md 9 -t 1000 -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/shm/err >/dev/null )

real	0m1.082s
user	0m0.140s
sys	0m0.480s
$ ll /dev/shm/out*raw | wc --lines
589
$ 

600fps range with "2>/dev/null":

Code: Select all

$ rm /dev/shm/out*raw
$ time ( raspiraw -hd -md 9 -t 1000 -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null )

real	0m1.079s
user	0m0.110s
sys	0m0.490s
$ ll /dev/shm/out*raw | wc --lines
600
$
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Dec 11, 2017 12:21 am

6by9 wrote:
Mon Dec 04, 2017 10:28 am
The main thing on the firmware is supporting a sensible set of modes. FOV is more important to most people than very high speed capture - far too many memories of all the complaints early on when video was the cropped 1080P mode and stills were the full 5MPix. I'm afraid you are the exception, which is part of the reason we're trying to open up the drivers where possible, but all settings are then your responsiblity!
Agreed, good news is that there will be no new modes besides 1-7, high framerate options will just modify those existing. Later after lot of testing by you, me and others maybe new modes that make sense may arise.
6by9 wrote:
Wed Dec 06, 2017 11:39 am
Fixed and rawcam branch updated. My bad, I was stating what it sounded like and that needed to be the first thing to check.
The default format was VGA raw10, resulting in a 384000 byte buffer. The framework would increase that should the buffer size requirements increase, but would never decrease it. raspiraw didn't copy the values from buffer_size_recommended across, therefore you ended up with a minimum of 384000.
Thank you for fixing, I verified that the fix is good for all temporary new modes.
6by9 wrote:
Mon Dec 04, 2017 10:28 am
All your new modes should be possible from the sensor, but 480fps may be pushing the GPU in the current configuration - suck it and see time. If you wanted to create a pull request then I would review and merge.
I am just learning how to use github. I learned already that I have to create my own fork of your userland for doing a pull request, and did that. I added raspiraw documentation (new raspiraw folder with just README.md), and many new options. Will test more, and check and extend documentation.

These are the new command line options I added -- they allow to run all high framerate modes I ever created sofar:

Code: Select all

-r, --regs	: Change regs of current mode
-hi, --hinc	: Set horizontal odd/even inc
-vi, --vinc	: Set vertical odd/even inc
-f, --fps	: Set framerate
-w, --width	: Set width
-h, --height	: Set height
-v, --vts	: Set min_vts
-l, --line	: Set line_time_ns
-hd0, --header0	: Write the BRCM header to output file 0
-0, --zero	: Write empty output files


Cool is that eg. "--fps" works for all modes, here you can see speeding up 1296x720 mode5 from 30fps to 49fps:

Code: Select all

$ rm /dev/shm/out.*.raw
$ raspiraw -md 5 -t 1000 -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null
Using i2C device /dev/i2c-0
$ ls -l /dev/shm/out.*.raw | wc --lines
30
$ rm /dev/shm/out.*.raw
$ raspiraw -md 5 -t 1000 --fps 49 -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null
Using i2C device /dev/i2c-0
$ ls -l /dev/shm/out.*.raw | wc --lines
49
$ 

And here is 600fps 640x128 stretched ("-vinc 3D" replaces 35) temporary mode 9 with just command line options!

Code: Select all

$ rm /dev/shm/out.*.raw
$ raspiraw -md 7 -t 1000 -h 64 -v 65 -l 10000 --vinc 3D --fps 600 -r "380A,0040;3802,78;3806,05FF" -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null
Using i2C device /dev/i2c-0
$ ls -l /dev/shm/out.*.raw | wc --lines
604
$ 
This mode is my current favorite mode, 600fps is super high framerate, while 640x128 is not too small framesize.

"git diff" works well, will see how "git add", "git commit" and "git push" will work soon on Hermann-SW userland repo.
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4604
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Dec 11, 2017 7:33 am

PierreG wrote:
Thu Dec 07, 2017 1:08 pm
Hi everybody!
I'm thinking about interfacing another sony imx sensor (224) with Rpi3.
What is the requirements to get raw image with raspiraw?
1) Design/use a sensor board
2) Connect i2c/clock/CSI between sensor board and rpi
3) Define the sensor registers and mode in *_modes.h file
4) run raspiraw, debug and pray

Did someone try to connect another sensor than imx219 and ov5647?
Sorry, missed this one before.

You're right on what's required. The only bit I think you've missed is adding the include and sensor entry into raspiraw.c. You may need to tweak termination settings depending on how your register set configures csi low to high speed transitions.
Getting the register set right is half the battle, although I seem to recall imx224 has a few drivers kicking around if you search a bit.
Do remember that the csi clock and data lanes are pretty high speed (100-500MHz typically) and are meant to be impedance matched.

I know one other has had an fpga solution interfaced to raspiraw.
There are settings in raspiraw for adv7282m (analogue video to csi bridge), but I never got them to work reliably. I now know they keep the clock lane in high speed mode, so I might revisit that as hopefully it will be about a two line change.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4604
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Dec 11, 2017 10:52 am

Raspiraw app is now split out from userland to https://github.com/6by9/raspiraw.
It should have the required source, camera_i2c script, buildme (I probably should have done a makefile - never mind), and a copy of rpi3-gpiovirtbuf. One bonus is it builds faster, however it's not set up for cross compiling.

I've added deprecration notices to my userland rawcam branch, but won't delete it yet as HermannSW is learning git against it :)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4604
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Dec 11, 2017 1:07 pm

6by9 wrote:
Mon Dec 11, 2017 7:33 am
There are settings in raspiraw for adv7282m (analogue video to csi bridge), but I never got them to work reliably. I now know they keep the clock lane in high speed mode, so I might revisit that as hopefully it will be about a two line change.
ADV7282M setup now fixed on https://github.com/6by9/raspiraw (although it seems to have the potential for memory corruption - adding an extra few lines of padding to the buffer seems to fix it).

And I forgot that the TC358743 HDMI to CSI2 bridge chip is also working with the same setup, either via https://github.com/6by9/userland/tree/hdmi_in_hack, or UV4L is also using rawcam to talk to it
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Dec 11, 2017 10:38 pm

I saw bulb I used last week in secondary living place and did a test picture. Then I realized that I had two bulbs in photo taken, from reading lamp and from bulb on room ceiling. It took quite some time to adjust the camera position in order to show both bulbs in top quarter of 640x480 frame. That way they would show up in stretched 640x128 600fps video together. Since they share same power network I expected both to show the 600/(2*50)=6 frames repeat pattern together (because we have 50Hz power line frequency in Germany). And they do show synchronous pattern repeating every 6 frames.

This is how the video was taken with new raspiraw command line options:

Code: Select all

$ rm /dev/shm/out.*.raw
$ raspiraw -md 7 -t 1000 -hd0 -h 64 -v 65 -l 10000 --vinc 3D --fps 600 -r "380A,0040;3802,78;3806,05FF" -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null
Using i2C device /dev/i2c-0
$ ls -l /dev/shm/out.*.raw | wc --lines
604
$

And using modified raw2ogg2anim script shown below this animated .gif playing with 1fps was created:
Image

For comparison the 600fps animated .gif from same bulb created last week (6 frames take 10ms) :
Image

Only the top part of the script was modified, and "index" in "gst-launch-1.0" was changed to match the start of loop frames. As you can see "out.0000.raw" needs to be prepended to each frame taken because it holds the BCRM header needed for decoding with dcraw. Size of the header is 32768, size of the raw frames without header is 51200. Even when storing in ramdisk, using normal "-hd" option would skip some frames and drop overall framerate to 580-590fps range. With "-hd0" option used we get more than 600fps reliably:

Code: Select all

$ cat raw2ogg2anim 
#!/bin/bash
echo "copying /dev/shm/out.????.raw files"
# cp /dev/shm/out.????.raw .
for((i=300; i<324; ++i))
do
  cat /dev/shm/out.0000.raw /dev/shm/out.0$i.raw > out.0$i.raw
done
echo "dcraw each .raw file (to .ppm)"
for f in *.raw
do
  dcraw $f
  echo -en "$f     \r"
done
echo
echo ".ppm -> .ppm.d"
for f in *.ppm
do
  double $f > $f.d 
  echo -en "$f     \r"
done
echo
echo ".ppm.d -> .ppm.d.png"
for f in *.ppm.d
do
  pnmtopng $f > $f.png 
  echo -en "$f     \r"
done
echo
echo "now creating $1.ogg"
gst-launch-1.0 multifilesrc location="out.%04d.ppm.d.png" index=300 caps="image/png,framerate=\(fraction\)1/1" ! pngdec ! videorate ! videoconvert ! videorate ! theoraenc ! oggmux ! filesink location="$1.ogg"
echo "now creating $1.anim.gif"
gifenc.sh $1.ogg $1.anim.gif
echo "done"
$ 
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Dec 12, 2017 1:43 am

6by9 wrote:
Mon Dec 11, 2017 10:52 am
Raspiraw app is now split out from userland to https://github.com/6by9/raspiraw.
It should have the required source, camera_i2c script, buildme (I probably should have done a makefile - never mind), and a copy of rpi3-gpiovirtbuf.
Thank you for that. raspiraw is such a cool piece of software -- definitely deserving its own repository!
6by9 wrote:
Mon Dec 11, 2017 10:52 am
One bonus is it builds faster, however it's not set up for cross compiling.
That's true, under userland I did compile changes with "make -C ~/userland/build/raspberry/release raspiraw/fast" in less than 10 seconds on a Pi 2B. On same Pi 2B "./buildme" takes just 3 seconds.
6by9 wrote:
Mon Dec 11, 2017 10:52 am
I've added deprecration notices to my userland rawcam branch, but won't delete it yet as HermannSW is learning git against it :)
You can delete the repository, I did so with my userland fork from you.

I forked https://github.com/6by9/raspiraw, cloned it and brought my changes there.
Then I did 5 adds, my first commit and finally my first git push.

Before I will do pull request I will have to correct typos in README.md documentation where I added all the new high framerate options and examples and thoughts on ramdisk usage for high framerate raspiraw video capture. And I need to implement the "--timestps" option, until now it is only documentation. All other new options are implemented and work. You can lookup new documentation here
https://github.com/Hermann-SW/raspiraw

And clone that repo if you want to play with the new options now.
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Dec 12, 2017 9:17 pm

I just implemented the easy part of "--tstamps" option, recording timestamps for each callback call into a singly-linked list and storing into file. After storing timestamps only (big numbers because of microsecond resolution) I thought that storing deltas as well might be interesting for human reader of timstamp file. I ended up with writing "delta,index,timestamp" per line because that allows to easily determine frame indices where frame skips occured.

As shown before these are the 3 commands I used for a single capture:

Code: Select all

$ rm /dev/shm/out*raw
$ raspiraw -md 7 -t 1000 -ts -hd0 -h 64 -v 65 -l 10000 --vinc 3D --fps 600 -r "380A,0040;3802,78;3806,05FF" -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null
Using i2C device /dev/i2c-0
$ ls -l /dev/shm/out.*.raw | wc --lines
    607
$ 

This means that 605 frames got captured, /dev/shm/out.0000.raw contains BCRM header (because of -hd0) and /dev/shm/out.-001.raw contains the timestamps (because of -ts).

Looking at the file I realized that I do not need to write any analysis code in raspiraw.c, "cut", "sort" and "uniq" are friends here.

Here is a first run without frame skips:

Code: Select all

$ cat /dev/shm/out.-001.raw | cut -f1 -d, | sort -n | uniq -c
      1 
      3 1648
      2 1649
     61 1650
    532 1651
      1 1652
      3 1653
      2 1654
$ 

As you can see we have a really small range of deltas (1648us-1654us) with majority exactly in center 1651us, which corresponds to 605.7fps:

Code: Select all

$ echo "1000000/1651" | bc -ql
605.69351907934585099939
$ 

Here is another run with frameskips (despite storing everything into /dev/shm ramdisk):

Code: Select all

$ cat /dev/shm/out.-001.raw | cut -f1 -d, | sort -n | uniq -c
      1 
     65 1650
    531 1651
      4 1652
      2 3302
$ 

Having delta column as 1st has the benefit that identifying frame indices of skips is easy:

Code: Select all

$ grep "^3" /dev/shm/out.-001.raw
3302,2,14461261624
3302,91,14461410205
$ 
So we have frame skips before frame 2 and before frame 91. But frames 91-605 do not have any frame skip.

I did already git push the "--tstamps" changes and updated its documentation.

Have I said that I really like raspiraw? ;)
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 13, 2017 1:12 am

New command line option "-hd0" for storing BCRM header only once pays.

My favorite mode 640x128 stretched maximal framerate was 605fps.
Now I tested with "-hd0" and reached 665fps for same 640x128 stretched!

This is 8s recording:

Code: Select all

$ rm /dev/shm/out*raw
$ raspiraw -md 7 -t 8000 -ts -hd0 -h 64 -v 65 -l 10000 --vinc 3D --fps 660 -r "380A,0040;3802,78;3806,0603" -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null
Using i2C device /dev/i2c-0
$ ls -l /dev/shm/out*raw | wc --lines
5317
$

Analysis shows that only 6 frame skips happen during the 8s phase!
Mean frame delta is 1503us ± 5us, corresponding to 1,000,000/1503 = 665.3fps:

Code: Select all

$ cat /dev/shm/out.-001.raw | cut -f1 -d, | sort -n | uniq -c
      1 
     22 1498
    251 1499
    312 1500
    274 1501
   1061 1502
   2388 1503
    277 1504
    282 1505
    309 1506
    132 1507
      5 3005
      1 3006
$ 

Here are the frame skips:

Code: Select all

$ grep ^3 /dev/shm/out.-001.raw 
3005,2,27911765057
3005,84,27911889785
3006,1089,27913401545
3005,1691,27914307698
3005,1823,27914507563
3005,2292,27915213852
$ 

So we got frames 2292-5315 without frame skips.

I created animated .gif from 65 frames (2635-2699) worth less than 0.1s real time, playing with 19fps or 35 times slower than real:
Image


I still have no explanation why it seems my fast moving finger is transparent:
Image


Here is the same scene taken with "raspistill -w 640 -h 480 -o test.jpg" initially for calibration:
Image


I used Lego pieces of many colors to see what effect the 665fps framerate has on colors.
As can be seen the video is very pale/colorless compared to the raspistill image.
It seems that 1.503ms is not enough time to sum up light.
For this framesize time has come to play with other raspiraw settings and see the effects on frames captured


P.S:
Stepped back to 640x240 stretched video (same scene, again lighted with smartphone flash, no moving finger):

Code: Select all

$ raspiraw -md 7 -t 8000 -ts -hd0 -h 120 -v 121 -l 10000 --vinc 3D --fps 360 -r "380A,0078;3802,78;3806,063B" -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null

No frame skip in 8s, played with 10fps 36 times slower than real.
Mean frame delta is 2773us ± 4us, corresponding to 1,000,000/2773 = 360.6fps:
Image


P.P.S:
Ouch!
Had that before, same command line with just "-vinc 1F" instead of "-vinc 3D", and the colors come back!

No frame skip in 8s, played with 10fps 36 times slower than real.
Mean frame delta is 2773us ± 3us, corresponding to 1,000,000/2773 = 360.6fps:
Image
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

HermannSW
Posts: 330
Joined: Fri Jul 22, 2016 9:09 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 13, 2017 8:41 am

Hi 6by9,

as requested by you I created pull requst for the new command line options:
https://github.com/6by9/raspiraw/pull/1

Please let me know if I need to change anything before you can accept the request.

I plan to create new tools directory for sample scripts later
  • gstreamer pipeline for conversion of captured frame set into .ogg video
  • gifenc.sh for conversion of .ogg video into animated .gif
  • sample script for doing capture into /dev/shm ramdisk with frame delta analysis (rm ...; raspiraw ...; ls -l ...; cut -f1 ...)

Btw, I plan to capture missing register sets for imx219 sensor modes 2-7 after Christmas ...
--> Raspberry camera / gstreamer / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/6by9/raspiraw      https://github.com/Hermann-SW/raspiraw

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 16 guests