ayilm1
Posts: 4
Joined: Wed May 20, 2020 2:33 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed May 20, 2020 11:17 am

Thanks so much for the prompt reply.
6by9 wrote:
Wed May 20, 2020 11:02 am
My recommendation is to use the bcm2835-unicam driver instead of MMAL rawcam as at least then you can add additional logging into the driver.

Having just discussed it with a colleague, it should be possible to create a dummy sensor driver that provides the relevant hooks expected by the bcm2835-unicam driver, and reads a width, height, and pixel format from device tree. That would be usable by anyone adding an uncontrolled CSI2 source to the Pi. I have other things to look at today, but will see where I get to over implementing it.
This sounds like a much more promising approach. I'll take a look at unicam right away. If I can't get it working on my own by the time you get to it, at least I'll have some base knowledge to know what's going on and hopefully still be useful.
6by9 wrote:
Wed May 20, 2020 11:02 am
There's already a driver for ov7251 from mainline in the 5.4 release, so you in theory only need to build it and create a device tree overlay for it.
(There is the potential for the clock configuration to differ between what the driver is expecting vs that implemented on your board - life can get a little tricky then if you don't have a full datasheet for the sensor)
To be honest, my thought process in procuring the ov7251 was just to have something that was close enough from a bus width and resolution perspective to my FPGA, which I could use as a differential reference when bringing up the FPGA hardware for the first time. There's nothing worse than putting everything together without having unit tested against known good system blocks, to find that nothing works and you don't know whether it's the implementation, the layout, or the software that's to blame. As such, I was hoping that I could treat the OV7251 as if it it were as dumb as my FPGA from a configurability perspective - no I2C register reconfig, no device specific drivers. Just a dumb source where I do the bare minimum to the CSI RX peripheral config (width. height. datatype as you said above) to pull data off the thing. Because that's what the software's going to be doing with the FPGA too. This way I can prove that the software works and just swap the OV7251 for the FPGA, change the height and datatype to YUV422-8, and everything should work the same. Of course, if the OV7251 becomes too much of a walk up the garden path to get working, I'll likely just bite the bullet and test with the FPGA directly.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8404
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Wed May 20, 2020 11:23 am

ayilm1 wrote:
Wed May 20, 2020 11:17 am
To be honest, my thought process in procuring the ov7251 was just to have something that was close enough from a bus width and resolution perspective to my FPGA, which I could use as a differential reference when bringing up the FPGA hardware for the first time. There's nothing worse than putting everything together without having unit tested against known good system blocks, to find that nothing works and you don't know whether it's the implementation, the layout, or the software that's to blame. As such, I was hoping that I could treat the OV7251 as if it it were as dumb as my FPGA from a configurability perspective - no I2C register reconfig, no device specific drivers. Just a dumb source where I do the bare minimum to the CSI RX peripheral config (width. height. datatype as you said above) to pull data off the thing. Because that's what the software's going to be doing with the FPGA too. This way I can prove that the software works and just swap the OV7251 for the FPGA, change the height and datatype to YUV422-8, and everything should work the same. Of course, if the OV7251 becomes too much of a walk up the garden path to get working, I'll likely just bite the bullet and test with the FPGA directly.
Very few sensors will just stream data when you power them up. Most need a moderate number of I2C registers set to suitable values to configure any cropping or binning to be done on the sensor, and to configure the PLL as there is no one correct oscillator frequency to feed into the device. Defaults don't necessarily match with something that can actually work.
You can see from the mainline driver the lists that get sent dependent on the mode required. Without a datasheet these are basically black magic even if you've had many years of experience working on image sensors.

I see Arducam sell the OV7251 - I might see if I can obtain one and get it working.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ayilm1
Posts: 4
Joined: Wed May 20, 2020 2:33 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed May 20, 2020 11:42 am

6by9 wrote:
Wed May 20, 2020 11:23 am
Very few sensors will just stream data when you power them up. Most need a moderate number of I2C registers set to suitable values to configure any cropping or binning to be done on the sensor, and to configure the PLL as there is no one correct oscillator frequency to feed into the device. Defaults don't necessarily match with something that can actually work.
Yep, already experienced this with it. I've also got the Arducam variant that you've found below. There was no data on the MIPI I/F with just power applied. However after their UL driver https://www.arducam.com/docs/cameras-fo ... -examples/ is used, even after closing the the application, the camera stays configured and continues streaming data (albeit with terminations in the peripheral turned off. so the probed waveform changes by a few hundred mV after it closes). It doesn't reset the registers or shut down the camera. So I figured for testing, I could run their userland driver to initialise the camera, and then hand the camera over to rawcam/unicam. I know this is janky, but like I said, my aim is just to prove out that the software side will be capable of interfacing to my uncontrolled FPGA source.
6by9 wrote:
Wed May 20, 2020 11:23 am
I see Arducam sell the OV7251 - I might see if I can obtain one and get it working.
I really don't want you to got to this trouble and expense just for me. Like I said, for me the point of the OV7251 was just to have known working hardware to simplify debugging and it's not really part of the overall goal of my project. My finished device is for the time being, going to be uncontrolled.

ayilm1
Posts: 4
Joined: Wed May 20, 2020 2:33 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed May 20, 2020 4:47 pm

So I looked through the unicam/V4L2 route and as far as I've been able to ascertain, the amount of work involved in creating a dummy I2C device that can attach to the CSI endpoint and deliver the config required is going to be far beyond my abilities. Looking at both the OV7251 and TC358743 kernel drivers, although it should be fairly straightforward to make a kernel module that hooks into a dummy I2C device, I'm going to be quickly out of my depth when it comes to the V4L2 side of things.

Looking back through the posts in this thread, I came across grimepoch's posts again. I missed it the first time reading through them, but he/she also had an I2C-less setup. My setup is actually pretty much identical to his/hers, down to using the same FPGA (MachXO2). The difference is however that I'm not using the stock Lattice reference design. I went through their HDL and there's actually a huge number of bugs in it, some of which likely contributed directly to grimepoch's setup failing (if you're reading this, look into the data_type signal in the packetheader module. You're going to find an inconsistency compared to 'dt' going into parallel2byte. Also pay close attention to the SoF/EoF short packets generated. Pay special attention to any zero-length pulses you might see on the 'data_type' signal). Anyway the long and short of it is that I believe I fixed all of Lattice's bugs and I now have logic that (in simulation) obeys the packet data type and produces HS data consistent with what the pattern generator is outputting (I also scrapped their pattern generator and wrote my own).

The point of all this is, grimepoch seemed to get his/her setup limping by just "commenting out the I2C calls". I assume what is meant by I2C calls is just everything between the start of main() and where the MMAL components and ancillaries start getting initialised? If so, I guess this is basically what I already started doing, plus hard-setting some of the MMAL configs as these would obviously cause problems if left uninitialised. I'm just going around in circles at this point I think.

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Wed May 20, 2020 5:42 pm

6by9 wrote:
Mon May 11, 2020 3:31 pm
Done - https://github.com/6by9/raspiraw/commit ... a5bccd171a
Untested though, particularly on the exposure/gain/flip/vts setup.
Just tried raspiraw with HQ camera on 4GB Pi4B, does not work.

Here HQ camera is shown (1a):

Code: Select all

🍓 ./camera_i2c 
setting GPIO for board revsion: c03111
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- 64 -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         
🍓

But raspiraw does not detect any camera:

Code: Select all

🍓 ./raspiraw -md 4 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw
Using i2C device /dev/i2c-0
RaspiRaw: Probing sensor ov5647 on addr 36
RaspiRaw: Probing sensor imx219 on addr 10
RaspiRaw: Probing sensor adv7282 on addr 21
RaspiRaw: Probing sensor imx477 on addr 1A
RaspiRaw: No sensor found. Aborting
🍓 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

raun0
Posts: 1
Joined: Thu May 21, 2020 8:15 am

Re: Raw sensor access / CSI-2 receiver peripheral

Thu May 21, 2020 8:21 am

^ Same here. I tested with v2.1 camera and that works just fine. But I have same problem with HQ camera also on both ( Pi 4 and Pi 3B ).
Last edited by raun0 on Thu May 21, 2020 5:21 pm, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8404
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu May 21, 2020 3:35 pm

HermannSW wrote:
Wed May 20, 2020 5:42 pm
6by9 wrote:
Mon May 11, 2020 3:31 pm
Done - https://github.com/6by9/raspiraw/commit ... a5bccd171a
Untested though, particularly on the exposure/gain/flip/vts setup.
Just tried raspiraw with HQ camera on 4GB Pi4B, does not work.

Here HQ camera is shown (1a):

Code: Select all

🍓 ./camera_i2c 
setting GPIO for board revsion: c03111
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- 64 -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         
🍓

But raspiraw does not detect any camera:

Code: Select all

🍓 ./raspiraw -md 4 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw
Using i2C device /dev/i2c-0
RaspiRaw: Probing sensor ov5647 on addr 36
RaspiRaw: Probing sensor imx219 on addr 10
RaspiRaw: Probing sensor adv7282 on addr 21
RaspiRaw: Probing sensor imx477 on addr 1A
RaspiRaw: No sensor found. Aborting
🍓 
Was
Untested though, particularly on the exposure/gain/flip/vts setup.
unclear?
You had the source code. You had the reference. Therefore there was nothing stopping you fixing it yourselves. If you expect everything spoon-fed to you, then raspiraw is not the program for you.

I've done it now, but I'm not making any further modifications to it. If you get other things working then create a PR for it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri May 22, 2020 9:15 am

6by9 wrote:
Thu May 21, 2020 3:35 pm
Was
Untested though, particularly on the exposure/gain/flip/vts setup.
unclear?
You had the source code. You had the reference. Therefore there was nothing stopping you fixing it yourselves. If you expect everything spoon-fed to you, then raspiraw is not the program for you.
While I may have fixed the missing structs for imx477, I could not have fixed (currently) the additional register settings for all 4 modes you did (you have access to the datasheet under NDA, we not):
https://github.com/6by9/raspiraw/commit ... 48fbe09e38
I did a lot of high framerate additions in the past, and will try with HQ camera as well now.
I've done it now, but I'm not making any further modifications to it. If you get other things working then create a PR for it.
Thank you !
raspiraw is working now with HQ camera.
And I made my fork from your repo up to date so that creating PR is possible.

There are some changes for raspiraw compared to last time I used it:
  • now preview window gets displayed during capturing👍
  • captured images are flipped horizontally and vertically, adding "-vf -hf" was all that was needed
After syncing and "./buildme"
https://github.com/6by9/raspiraw
https://github.com/6by9/dcraw

Code: Select all

🍓 export PATH=~/dcraw:~/raspiraw:~/raspiraw/tools:$PATH
🍓 cd ~/raspiraw
🍓 camera_i2c 
setting GPIO for board revsion: c03111
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- 64 -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         
🍓

Then in a newly created subdirectory with this very 1st script:

Code: Select all

🍓 cd xq1
🍓 cat 1012x760 
#!/bin/bash

if [ "$1" = "" ]; then echo "format: `basename $0` ms"; exit; fi

echo "removing /dev/shm/out.*.raw"
rm -f /dev/shm/out.*.raw

if [ "$2" = "" ]; then fps=120; else fps=$2; fi
echo "capturing frames for ${1}ms with ${fps}fps requested"
raspiraw -md 3 -vf -hf -t $1 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null

us=`cut -f1 -d, tstamps.csv | sort -n | uniq -c | sort -n | tail -1 | cut -b9-`
l=`ls -l /dev/shm/out.*.raw | wc --lines`
echo "$l frames were captured at $((1000000 / $us))fps" 

echo "frame delta time[us] distribution"
cut -f1 -d, tstamps.csv | sort -n | uniq -c

D=`echo "0123456789" | sed "s/\`echo $((1000000/$fps)) | cut -b1\`//"`

echo "after skip frame indices (middle column)"
grep "^[$D]" tstamps.csv | sed "s/^/> /"

skips=`grep "^[$D]" tstamps.csv | wc --lines | cut -f1 -d\ `
stamps=`wc --lines tstamps.csv | cut -f1 -d\ `
per=`expr \( 100 \* $skips \) / \( $skips + $stamps \)`
echo "$per% frame skips"
🍓 

With no framerate requested in script, majority framerate reported is 197fps !?!?!
The first line glitch is caused by last four frames having timestamp "0" -- that can be fixed.
I need to bring in latest changes for ptsanalyze reporting average framerate in addition into this script as well.
For now, 323 frames in 2s, is average framerate 1000000/((4294695070-4292708587)/318) = 160.08fps:

Code: Select all

🍓 ./1012x760 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 120fps requested
323 frames were captured at 197fps
frame delta time[us] distribution
      1 -4294695070
      1 
      3 0
      1 5036
      1 5046
      1 5047
      1 5051
      1 5052
     65 5054
    164 5055
      5 5056
      1 5057
      1 5059
      1 5063
      1 5074
      1 10105
      1 10107
      3 10108
     59 10109
      9 10110
      1 10113
      1 10118
after skip frame indices (middle column)
...
🍓 

For now I want to see a single frame only (start and end frame 100):

Code: Select all

🍓 raw2ogg2anim xq1 100 100 1
removing old auxiliary files
copying /dev/shm/out.????.raw files
100     
dcraw each .raw file (to .ppm)
out.0100.raw     
.ppm -> .ppm.d
out.0100.ppm     
.ppm.d -> .ppm.d.png
out.0100.ppm.d     
now creating xq1.ogg
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.000503451
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
now creating xq1.anim.gif
[theora @ 0x248e530] 7 bits left in packet 82
[theora @ 0x24c1680] 7 bits left in packet 82
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF757875
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF767C77
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF787876
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF787A77
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF787E77
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF7B837C
[Parsed_palettegen_2 @ 0x26d2790] Dupped color: FF7C867E
[theora @ 0x17f9620] 7 bits left in packet 82
[theora @ 0x1877140] 7 bits left in packet 82
done
🍓 

Coversion of generated .png to .jpg for fitting attachment size of this forum (result is 24KB):

Code: Select all

🍓 pngtopnm out.0100.ppm.d.png | pnmtojpeg > out.0100.ppm.d.png.jpg
🍓 

There is some jitter in lower left part of the frame, but a good 1st frame captured at 197fps framerate with HQ camera:
out.0100.ppm.d.png.jpg
out.0100.ppm.d.png.jpg
out.0100.ppm.d.png.jpg (22.31 KiB) Viewed 474 times
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8404
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Fri May 22, 2020 9:26 am

HermannSW wrote:
Fri May 22, 2020 9:15 am
6by9 wrote:
Thu May 21, 2020 3:35 pm
Was
Untested though, particularly on the exposure/gain/flip/vts setup.
unclear?
You had the source code. You had the reference. Therefore there was nothing stopping you fixing it yourselves. If you expect everything spoon-fed to you, then raspiraw is not the program for you.
While I may have fixed the missing structs for imx477, I could not have fixed (currently) the additional register settings for all 4 modes you did (you have access to the datasheet under NDA, we not):
https://github.com/6by9/raspiraw/commit ... 48fbe09e38
I did a lot of high framerate additions in the past, and will try with HQ camera as well now.
None of the changes required access to the datasheet, just referencing the Linux kernel driver.
- The ID was wrong (early prototypes reported it differently)
- The register set was missing the command to start streaming
- analogue gain, exposure time, VTS, and flips weren't present in the register sets, so the command line parser was unable to set them (I hate that bit of code)
- initialisation didn't ensure the sensor was stopped before reconfiguring, therefore doing ctrl-c would mean that running again failed.

None of those were rocket science, and the fact that mcuicuc had already posted that he'd needed to alter the ID register would have got you started. Failing detection really was an obvious one.

Anyway, knock yourself out now.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RoboRob
Posts: 33
Joined: Tue Nov 05, 2013 8:43 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Fri May 22, 2020 9:33 am

How do I start using raspiraw for the HQ camera? I read up to page 5 of this thread (took me 1 hour) but it is too much to read through 24 thread pages of postings.

Is there a simple procedure to get me started with raspiraw i.c.w. the HQ camera?

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri May 22, 2020 9:50 am

RoboRob wrote:
Fri May 22, 2020 9:33 am
Is there a simple procedure to get me started with raspiraw i.c.w. the HQ camera?
See the steps I detailed in my previous posting in this thread.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 8:59 am

I did install my 2nd Pi4B freshly recently and did rpi-update for 5.4 kernel and libcamera experiments.
With "dtoverlay=imx477" commented out in "/boot/config.txt" and after reboot raspi(still|vid) do work, so camera was enabled, as well as i2c.
Anyway raspiraw's camera_i2c did not work, because of missing /dev/i2c-0.
I looked up in Robert's blog posting that "dtparam=i2c_vc=on" was missing in /boot/confg.txt:
https://blog.robertelder.org/recording- ... pi-camera/
After reboot now /dev/i2c-* devices are present:

Code: Select all

[email protected]:~ $ uname -a
Linux raspberrypi4B2 5.4.42-v7l+ #1319 SMP Wed May 20 14:12:03 BST 2020 armv7l GNU/Linux
[email protected]:~ $ ls -l /dev/i2c*
crw-rw---- 1 root i2c 89,  0 May 23 10:06 /dev/i2c-0
crw-rw---- 1 root i2c 89, 10 May 23 10:06 /dev/i2c-10
crw-rw---- 1 root i2c 89, 11 May 23 10:06 /dev/i2c-11
[email protected]:~ $ 

But unmodified camera_i2c shows no detected devices (unlike under Buster 4.x kernel without rpi-update):

Code: Select all

[email protected]:~ $ export PATH=~/dcraw/:~/raspiraw:~/raspiraw/tools:$PATH
[email protected]:~ $ cd raspiraw/
[email protected]:~/raspiraw $ camera_i2c 
setting GPIO for board revsion: b03111
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         
[email protected]:~/raspiraw $ 

It turned out that camera is shown on i2c-10 and i2c-11.

Code: Select all

$ i2cdetect -y 10 | grep 1a
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
$ i2cdetect -y 11 | grep 1a
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
$

@6by9:
Below fixes made camera_i2c and 1012x760 tool work again.
Is this the correct way to deal with kernel version differences?

Code: Select all

$ git diff
diff --git a/camera_i2c b/camera_i2c
...
 #raspi-gpio set 32 1
-i2cdetect -y 0
+r=`uname -r | head --bytes 1`
+if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi
+i2cdetect -y $i2c
 ;;
 
 *)
$

Code: Select all

$ diff 1012x760.old 1012x760
7a8,10
> r=`uname -r | head --bytes 1`
> if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi
> 
10c13
< raspiraw -md 3 -vf -hf -t $1 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null
---
> raspiraw -md 3 -y $i2c -vf -hf -t $1 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null
$

P.S:
raspiraw with HQ camera captured nicely on 4.x kernel.
Capturing on i2c-10 or i2c-11 results in completely white frames on 5.4 kernel.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

RoboRob
Posts: 33
Joined: Tue Nov 05, 2013 8:43 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 10:29 am

HermannSW wrote:
Fri May 22, 2020 9:50 am
RoboRob wrote:
Fri May 22, 2020 9:33 am
Is there a simple procedure to get me started with raspiraw i.c.w. the HQ camera?
See the steps I detailed in my previous posting in this thread.
Thanks for your response. I get the following error on the buildme of dcraw though:

Code: Select all

dcraw.c:77:10: fatal error: jasper/jasper.h: Bestand of map bestaat niet
 #include <jasper/jasper.h> /* Decode Red camera movies */
Nevertheless raspiraw seems to work though :D !

Code: Select all

camera_i2c
setting GPIO for board revsion: c03112
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
/var/www/html/raspiraw/camera_i2c: 111: /var/www/html/raspiraw/camera_i2c: ./rpi3-gpiovirtbuf: not found
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- 64 -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
I do get an error/warning when running camera_i2c though (see output above). That was easily fixed by removing the ./ in front of the rpi3-gpivirbuf command in this script so it runs also from other locations...

Running the 1012x760 script produces:

Code: Select all

./1012x760 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 120fps requested
ls: kan geen toegang krijgen tot '/dev/shm/out.*.raw': Bestand of map bestaat niet
./1012x760: regel 14: 1000000 / : syntaxfout: operator verwacht (het onjuiste symbool is "/ ")
frame delta time[us] distribution
after skip frame indices (middle column)
expr: deling door nul
% frame skips
it seems like tstamp.csv is not properly created (file is empty after running the 1012x760 script) and a division by zero is then producing an error...
Last edited by RoboRob on Sat May 23, 2020 11:28 am, edited 1 time in total.

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 11:26 am

RoboRob wrote:
Sat May 23, 2020 10:29 am
HermannSW wrote:
Fri May 22, 2020 9:50 am
RoboRob wrote:
Fri May 22, 2020 9:33 am
Is there a simple procedure to get me started with raspiraw i.c.w. the HQ camera?
See the steps I detailed in my previous posting in this thread.
Thanks for your response. I get the following error on the buildme of dcraw though:

Code: Select all

dcraw.c:77:10: fatal error: jasper/jasper.h: Bestand of map bestaat niet
 #include <jasper/jasper.h> /* Decode Red camera movies */
You missed to follow 6by9's instructions on dcraw README.me:

Code: Select all

sudo apt-get install libjasper-dev libjpeg8-dev gettext liblcms2-dev
./buildme
Running the 1012x760 script produces:

Code: Select all

./1012x760 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 120fps requested
ls: kan geen toegang krijgen tot '/dev/shm/out.*.raw': Bestand of map bestaat niet
./1012x760: regel 14: 1000000 / : syntaxfout: operator verwacht (het onjuiste symbool is "/ ")
frame delta time[us] distribution
after skip frame indices (middle column)
expr: deling door nul
% frame skips
it seems like tstamp.csv is not properly created (file is empty after running the 1012x760 script...
Something is wrong with your setup.
In such cases I copy out the raspiraw line from the tool, replace $1 with 2000 and remove the stdout and stderr redirections to /dev/null:

Code: Select all

raspiraw -md 3 -vf -hf -t 2000 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw
What is the output for that command for you?


I just created 1st Raspberry HQ camera raspiraw 1012x760 slowmo (captured with 150fps average framerate).
Played at 5fps, 30× slower than real; spinning 1€ coin, scene lighted with 5000lm led from above.
I had to change the filters line of gifenc.sh for 1012 width:

Code: Select all

🍓 git diff gifenc.sh
diff --git a/tools/gifenc.sh b/tools/gifenc.sh
...
 #filters="fps=15,scale=320:-1:flags=lanczos"
-filters="fps=25,scale=640:-1:flags=lanczos"
+filters="fps=5,scale=1012:-1:flags=lanczos"
 
 ffmpeg -v warning -i $1 -vf "$filters,palettegen" -y $palette
 ffmpeg -v warning -i $1 -i $palette -lavfi "$filters [x]; [x][1:v] paletteuse" -y $2
🍓 
Image
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

RoboRob
Posts: 33
Joined: Tue Nov 05, 2013 8:43 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 11:38 am

HermannSW wrote:
Sat May 23, 2020 11:26 am
RoboRob wrote:
Sat May 23, 2020 10:29 am
HermannSW wrote:
Fri May 22, 2020 9:50 am

See the steps I detailed in my previous posting in this thread.
Thanks for your response. I get the following error on the buildme of dcraw though:

Code: Select all

dcraw.c:77:10: fatal error: jasper/jasper.h: Bestand of map bestaat niet
 #include <jasper/jasper.h> /* Decode Red camera movies */
You missed to follow 6by9's instructions on dcraw README.me:

Code: Select all

sudo apt-get install libjasper-dev libjpeg8-dev gettext liblcms2-dev
./buildme
Got it! Compiles like a charm now :-) !
Running the 1012x760 script produces:

Code: Select all

./1012x760 2000
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 120fps requested
ls: kan geen toegang krijgen tot '/dev/shm/out.*.raw': Bestand of map bestaat niet
./1012x760: regel 14: 1000000 / : syntaxfout: operator verwacht (het onjuiste symbool is "/ ")
frame delta time[us] distribution
after skip frame indices (middle column)
expr: deling door nul
% frame skips
it seems like tstamp.csv is not properly created (file is empty after running the 1012x760 script...
Something is wrong with your setup.
In such cases I copy out the raspiraw line from the tool, replace $1 with 2000 and remove the stdout and stderr redirections to /dev/null:

Code: Select all

raspiraw -md 3 -vf -hf -t 2000 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw
What is the output for that command for you?

Code: Select all

raspiraw -md 3 -vf -hf -t 2000 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw
Using i2C device /dev/i2c-0
RaspiRaw: Probing sensor ov5647 on addr 36
RaspiRaw: Probing sensor imx219 on addr 10
RaspiRaw: Probing sensor adv7282 on addr 21
RaspiRaw: Probing sensor imx477 on addr 1A
RaspiRaw: No sensor found. Aborting
Using -y 10:

Code: Select all

raspiraw -md 3 -y 10 -vf -hf -t 2000 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw
Using i2C device /dev/i2c-10
RaspiRaw: Probing sensor ov5647 on addr 36
RaspiRaw: Probing sensor imx219 on addr 10
RaspiRaw: Probing sensor adv7282 on addr 21
RaspiRaw: Probing sensor imx477 on addr 1A
RaspiRaw: Found sensor imx477 at address 1A
RaspiRaw: Encoding 41414270
No AWB
then it sort of hangs... until I press <CTRL> C

UPDATE: well, removing dtoverlay=imx477 from /boot/config.txt and another power cycle seems to fix it... It works now...

RoboRob
Posts: 33
Joined: Tue Nov 05, 2013 8:43 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 12:41 pm

Soo, how do I take just dump the unprocessed data of one picture with raspiraw and how can I control the parameters for shutter speed, analog gain, digital gain, etc.

My end goal is to use the HQ Camera icw a telescope and retrieve as unprocessed images of the sensor as possible (so taken at unity gain, noise reduction off, etc.). All the processing will be done afterwards to ensure the weak signals of nebula's and galaxies don't get messed-up in the camera processing...

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 2:31 pm

RoboRob wrote:
Sat May 23, 2020 11:38 am
UPDATE: well, removing dtoverlay=imx477 from /boot/config.txt and another power cycle seems to fix it... It works now...
Wow, so you have it work with 5.4 kernel?
As reported I only get white frames with 5.4 kernel and "-y 10".
Have you converted a raw frame captured and looked what you get?
RoboRob wrote:
Sat May 23, 2020 12:41 pm
Soo, how do I take just dump the unprocessed data of one picture with raspiraw and how can I control the parameters for shutter speed, analog gain, digital gain, etc.
Just add the settings you want to raspiraw line in 1012x760 script:

Code: Select all

🍓 raspiraw

raspiraw Camera App 0.0.3

-?, --help	: This help information
-md, --mode	: Set sensor mode <mode>
-hf, --hflip	: Set horizontal flip
-vf, --vflip	: Set vertical flip
-e, --ss	: Set the sensor exposure time (not calibrated units)
-g, --gain	: Set the sensor gain code (not calibrated units)
-o, --output	: Set the output filename
-hd, --header	: Write the BRCM header to the output file
-t, --timeout	: Time (in ms) before shutting down (if not specified, set to 5s)
-sr, --saverate	: Save every Nth frame
-b, --bitdepth	: Set output raw bit depth (8, 10, 12 or 16, if not specified, set to sensor native)
-c, --cameranum	: Set camera number to use (0=CAM0, 1=CAM1).
-eus, --expus	: Set the sensor exposure time in micro seconds.
-y, --i2c	: Set the I2C bus to use.
-awbg, --awbgains	: Set the AWB gains to use.
-r, --regs	: Change (current mode) regs
-hi, --hinc	: Set horizontal odd/even inc reg
-vi, --vinc	: Set vertical odd/even inc reg
-f, --fps	: Set framerate regs
-w, --width	: Set current mode width
-h, --height	: Set current mode height
-lt, --left	: Set current mode left
-tp, --top	: Set current mode top
-hd0, --header0	: Sets filename to write the BRCM header to
-hdg, --headerg	: Sets filename to write the .pgm header to
-ts, --tstamps	: Sets filename to write timestamps to
-emp, --empty	: Write empty output files
-m, --metadata	: Decode register metadata
-awb, --awb	: Use a simple grey-world AWB algorithm
-n, --nopreview	: Do not send the stream to the display
-P, --processing	: Pass images into an image processing function
-p, --preview	: Preview window settings <'x,y,w,h'>
-fs, --fullscreen	: Fullscreen preview mode
-op, --opacity	: Preview window opacity (0-255)
-PY, --processing_yuv	: Pass processed YUV images into an image processing function
-oY, --output_yuv	: Set the output filename for YUV data
🍓 

The spinning coin in my previous post is a bit blurry. I did another recording, with setting shutter time to 200µs ("--expus 200") in 1012x760 raspiraw command line. First I turned on the 5000lm led, then started 4s recording and then snipped the 1€ coin:

Code: Select all

🍓 ./1012x760 4000

Conversion of frames 430-670 (frame 430 just before coin enters scene) took 9:22min -- perhaps time to introduce parallel execution to raw2ogg2anim tool (that tool requires netpbm as well as gesreamer to be installed):

Code: Select all

🍓 time ( raw2ogg2anim y 430 670 10 )
...
    Last message repeated 2 times
done

real	9m22.609s
user	8m1.633s
sys	0m19.996s
🍓

This recording was better, average framerate 165fps:

Code: Select all

🍓 head -430 tstamps.csv | tail -1
10109,430,2267005228
🍓 head -670 tstamps.csv | tail -1
5055,670,2268460971
🍓 bc -ql
(2268460971-2267005228)/240
6065.59583333333333333333
1000000/((2268460971-2267005228)/240)
164.86426518966603308413

Coin is much sharper with 200µs exposure, slowmo plays at 10fps this time, 16.5× slower than real:
Image


Since most of the scene keeps unchanged, the animated .gif sizes are low:

Code: Select all

$ du -sk spinning_coin.1*
1368	spinning_coin.150fps.anim.gif
7292	spinning_coin.165fps.anim.gif
$

Next is investigation what is responsible for the left bottom modifications.
And other HQ modes.
[email protected] on average might resuult in [email protected] average?!?


P.S:
Details on frame delta distribution just in 241 frames of animation:

Code: Select all

🍓 head -670 tstamps.csv | tail -240 | cut -f1 -d, | sort | uniq -c
      1 10107
      5 10108
     29 10109
      8 10110
      1 10111
      1 10112
      1 10117
      1 15164
      1 5015
      1 5047
      1 5048
      2 5049
      1 5052
      3 5053
     54 5054
    119 5055
      6 5056
      1 5057
      2 5061
      1 5062
      1 5094
🍓 

A picture says more than 1000 words, frame delta display for the 240 frames (Y-axis [µs]):
165fps.frame_deltas.png
165fps.frame_deltas.png
165fps.frame_deltas.png (29.05 KiB) Viewed 255 times
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8404
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 4:23 pm

HermannSW wrote:
Sat May 23, 2020 8:59 am
It turned out that camera is shown on i2c-10 and i2c-11.

Code: Select all

$ i2cdetect -y 10 | grep 1a
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
$ i2cdetect -y 11 | grep 1a
10: -- -- -- -- -- -- -- -- -- -- 1a -- -- -- -- -- 
$

@6by9:
Below fixes made camera_i2c and 1012x760 tool work again.
Is this the correct way to deal with kernel version differences?

Code: Select all

$ git diff
diff --git a/camera_i2c b/camera_i2c
...
 #raspi-gpio set 32 1
-i2cdetect -y 0
+r=`uname -r | head --bytes 1`
+if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi
+i2cdetect -y $i2c
 ;;
 
 *)
$

Code: Select all

$ diff 1012x760.old 1012x760
7a8,10
> r=`uname -r | head --bytes 1`
> if [ "$r" = "4" ]; then i2c=0; else i2c=10; fi
> 
10c13
< raspiraw -md 3 -vf -hf -t $1 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null
---
> raspiraw -md 3 -y $i2c -vf -hf -t $1 -ts tstamps.csv -hd0 hd0.32k -sr 1 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null
$

P.S:
raspiraw with HQ camera captured nicely on 4.x kernel.
Capturing on i2c-10 or i2c-11 results in completely white frames on 5.4 kernel.
5.4 has switched to using the i2c-mux-pinctrl kernel module to present pins 27&28 (GPIOS 0&1) on the 40pin header as i2c-0, and whatever pinmuxing is required to get to the camera or DSI display on i2c-10.
i2c-11 is an oddity from the mux and is the base controller without any muxing. It should be ignored as you can't guarantee where it will be routed to.
All pinmuxing options in camera_i2c are redundant (potentially counterproductive) if you use i2c-10, but you do need to enable the power by setting the relevant GPIO.
I was testing IMX477 with 5.4 on Friday, running only "./rpi3-gpiovirtbuf s 133 1" first, and -y 10 added to the raspiraw command line and all was fine. The kernel has nothing to do with controlling the camera as you're only using rawcam and directly sending I2C commands to the sensor.

PRs accepted.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RoboRob
Posts: 33
Joined: Tue Nov 05, 2013 8:43 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 6:43 pm

HermannSW wrote:
Sat May 23, 2020 2:31 pm
RoboRob wrote:
Sat May 23, 2020 11:38 am
UPDATE: well, removing dtoverlay=imx477 from /boot/config.txt and another power cycle seems to fix it... It works now...
Wow, so you have it work with 5.4 kernel?
As reported I only get white frames with 5.4 kernel and "-y 10".
Have you converted a raw frame captured and looked what you get?
I haven't figure out how to convert the video into images yet... so I can't tell you if the video was white or not...
HermannSW wrote:
Sat May 23, 2020 2:31 pm
RoboRob wrote:
Sat May 23, 2020 12:41 pm
Soo, how do I take just dump the unprocessed data of one picture with raspiraw and how can I control the parameters for shutter speed, analog gain, digital gain, etc.
Just add the settings you want to raspiraw line in 1012x760 script:

Code: Select all

raspiraw

raspiraw Camera App 0.0.3

-?, --help	: This help information
-md, --mode	: Set sensor mode <mode>
-hf, --hflip	: Set horizontal flip
-vf, --vflip	: Set vertical flip
-e, --ss	: Set the sensor exposure time (not calibrated units)
-g, --gain	: Set the sensor gain code (not calibrated units)
-o, --output	: Set the output filename
-hd, --header	: Write the BRCM header to the output file
-t, --timeout	: Time (in ms) before shutting down (if not specified, set to 5s)
-sr, --saverate	: Save every Nth frame
-b, --bitdepth	: Set output raw bit depth (8, 10, 12 or 16, if not specified, set to sensor native)
-c, --cameranum	: Set camera number to use (0=CAM0, 1=CAM1).
-eus, --expus	: Set the sensor exposure time in micro seconds.
-y, --i2c	: Set the I2C bus to use.
-awbg, --awbgains	: Set the AWB gains to use.
-r, --regs	: Change (current mode) regs
-hi, --hinc	: Set horizontal odd/even inc reg
-vi, --vinc	: Set vertical odd/even inc reg
-f, --fps	: Set framerate regs
-w, --width	: Set current mode width
-h, --height	: Set current mode height
-lt, --left	: Set current mode left
-tp, --top	: Set current mode top
-hd0, --header0	: Sets filename to write the BRCM header to
-hdg, --headerg	: Sets filename to write the .pgm header to
-ts, --tstamps	: Sets filename to write timestamps to
-emp, --empty	: Write empty output files
-m, --metadata	: Decode register metadata
-awb, --awb	: Use a simple grey-world AWB algorithm
-n, --nopreview	: Do not send the stream to the display
-P, --processing	: Pass images into an image processing function
-p, --preview	: Preview window settings <'x,y,w,h'>
-fs, --fullscreen	: Fullscreen preview mode
-op, --opacity	: Preview window opacity (0-255)
-PY, --processing_yuv	: Pass processed YUV images into an image processing function
-oY, --output_yuv	: Set the output filename for YUV data
Great, that was what I was looking for!!!

RoboRob
Posts: 33
Joined: Tue Nov 05, 2013 8:43 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 6:58 pm

I noticed in the return information of raspiraw it states that hte maximum exposure time is 65535 usec. Is it indeed not possible to expose up to the 200 sec. available to the HQ camera?

Code: Select all

raspiraw -y 10 --nopreview --output test.raw  --awbgains 3.3,1.53 --ss 5000000 --expus 5000000 --gain 1 --mode 3 --timeout 3000 --header

Using i2C device /dev/i2c-10
RaspiRaw: Probing sensor ov5647 on addr 36
RaspiRaw: Probing sensor imx219 on addr 10
RaspiRaw: Probing sensor adv7282 on addr 21
RaspiRaw: Probing sensor imx477 on addr 1A
RaspiRaw: Found sensor imx477 at address 1A
RaspiRaw: Setting exposure to 958588 from time 5000000us
RaspiRaw: Invalid exposure:958588, exposure range is 0 to 65535!

RaspiRaw: Invalid exposure:958588, vts range is 0 to 65535!

RaspiRaw: Set gain 0204 to 00
RaspiRaw: Set gain 0205 to 01
RaspiRaw: Encoding 41415270
No AWB
mmal: Set pack to 0, unpack to 0
mmal: Timing 6/2, 2/6/0, 0/0
mmal: Create pool of 6 buffers of size 983040
mmal: Create pool of 6 buffers of size 983040
mmal: Now streaming...

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 7:26 pm

6by9 wrote:
Sat May 23, 2020 4:23 pm
PRs accepted.
imx477.h mode numbering was not correct.
I fixed that, and added tool for each mode of HQ camera, and created pull request:
https://github.com/6by9/raspiraw/pull/35


Below are tool execution examples, avoiding output of frame delta distribution and frame skip listing sections via grep.
New is that framerate reported earlier is now named "majority framerate".
And that "average framerate" is reported as well.
And that dirty frames (with timestamps <=0) get excluded from reporting.

Demonstration of tools in order of descrending majority framerate:


195fps/155fps > 120fps:

Code: Select all

🍓 1012x760 4000 | grep -v "^[> ]"
removing /dev/shm/out.*.raw
capturing frames for 4000ms with 195fps requested
620 frames were captured
majority framerate 197fps
frame delta time[us] distribution
after skip frame indices (middle column)
21% frame skips
average framerate 155fps
🍓 

58fps/58fps > 50fps:

Code: Select all

🍓 2028x1080 4000 | grep -v "^[> ]"
removing /dev/shm/out.*.raw
capturing frames for 4000ms with 57fps requested
231 frames were captured
majority framerate 58fps
frame delta time[us] distribution
after skip frame indices (middle column)
0% frame skips
average framerate 58fps
🍓 

42fps/42fps < 50fps:

Code: Select all

🍓 2028x1520 4000 | grep -v "^[> ]"
removing /dev/shm/out.*.raw
capturing frames for 4000ms with 41fps requested
167 frames were captured
majority framerate 42fps
frame delta time[us] distribution
after skip frame indices (middle column)
0% frame skips
average framerate 42fps
🍓 

11fps/11fps > 10fps:

Code: Select all

🍓 4056x3040 4000 | grep -v "^[> ]"
removing /dev/shm/out.*.raw
capturing frames for 4000ms with 11fps requested
43 frames were captured
majority framerate 11fps
frame delta time[us] distribution
after skip frame indices (middle column)
49% frame skips
average framerate 11fps
🍓 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 7:59 pm

RoboRob wrote:
Sat May 23, 2020 6:43 pm
I haven't figure out how to convert the video into images yet... so I can't tell you if the video was white or not...
Capture with one of the 4 HQ mode tools I just commited, this captures 4 seconds 1012x760 frames:
https://github.com/Hermann-SW/raspiraw/ ... s/1012x760

Code: Select all

🍓 1012x760 4000 | grep -v "^[> ]"
removing /dev/shm/out.*.raw
capturing frames for 4000ms with 195fps requested
620 frames were captured
majority framerate 197fps
frame delta time[us] distribution
after skip frame indices (middle column)
21% frame skips
average framerate 155fps
🍓 

Convert eg. frame 20 captured in /dev/shm to .png:

Code: Select all

🍓 raw2ogg2anim name 20 20 1
For creating the .png "netpbm" packet needs to be installed.

For name.ogg video and name.anim.gif animated .gif creation, "ffmpeg" and gstreamer need to be installed.

Code: Select all

🍓 raw2ogg2anim 
format: raw2ogg2anim vname first last fps [d[d]]
🍓 
RoboRob wrote:
Sat May 23, 2020 6:58 pm
I noticed in the return information of raspiraw it states that hte maximum exposure time is 65535 usec. Is it indeed not possible to expose up to the 200 sec. available to the HQ camera?
The current implementation in omx477.h only allows for 16 bits:

Code: Select all

...
struct sensor_def imx477 = {
...
      .exposure_reg =         0x0202,
      .exposure_reg_num_bits = 16,

      .vts_reg =              0x0340,
      .vts_reg_num_bits =     16,
...
There is no publicly available imx477 datasheet yet.
ethanol100 provided pointer to imx477 register kernel sources for some other OS:
viewtopic.php?f=43&t=272926&p=1656904&h ... t#p1654528
And there only the lower 16bit are documented:

Code: Select all

	/* Frame Vertical Clock Count */
	{0x0340, CRL_REG_LEN_08BIT, 0x20}, /* Frame length [15:8] */
	{0x0341, CRL_REG_LEN_08BIT, 0x11}, /* Frame length [7:0]  */
Register 0x0342 is used for something else, and no match for "0x033".
You might try 0x033F for higher bits of "Frame length".

200s exposure is only available via rpi-update yet.
"RaspiCamControl.c" userland source shows that mml function gets called to set shutter speed:

Code: Select all

/**
 * Adjust the exposure time used for images
 * @param camera Pointer to camera component
 * @param shutter speed in microseconds
 * @return 0 if successful, non-zero if any parameters out of range
 */
int raspicamcontrol_set_shutter_speed(MMAL_COMPONENT_T *camera, int speed)
{
   if (!camera)
      return 1;

   return mmal_status_to_int(mmal_port_parameter_set_uint32(camera->control, MMAL_PARAMETER_SHUTTER_SPEED, speed));
}
I think mmal is closed source.


P.S:
Learned from 6by9 that Raspbian kernel source for imx477 is here:
https://github.com/raspberrypi/linux/bl ... c/imx477.c
Last edited by HermannSW on Sat May 23, 2020 10:02 pm, edited 1 time in total.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sat May 23, 2020 9:51 pm

6by9 wrote:
Sat May 23, 2020 4:23 pm
5.4 has switched to using the i2c-mux-pinctrl kernel module to present pins 27&28 (GPIOS 0&1) on the 40pin header as i2c-0, and whatever pinmuxing is required to get to the camera or DSI display on i2c-10.
i2c-11 is an oddity from the mux and is the base controller without any muxing. It should be ignored as you can't guarantee where it will be routed to.
Thanks for detailed explanation.
All pinmuxing options in camera_i2c are redundant (potentially counterproductive) if you use i2c-10, but you do need to enable the power by setting the relevant GPIO.
I was testing IMX477 with 5.4 on Friday, running only "./rpi3-gpiovirtbuf s 133 1" first,
I fixed camera_i2c along that explanation:
https://github.com/Hermann-SW/raspiraw/ ... d71bf1fd50
and -y 10 added to the raspiraw command line and all was fine.
In that commit the 4 tools were changed so that they select i2c device based on kernel version.
Now (my) raspiraw works for 4. as well as 5. kernels:

Code: Select all

[email protected]:~/raspiraw/hq1 $ uname -a
Linux raspberrypi4B2 5.4.42-v7l+ #1319 SMP Wed May 20 14:12:03 BST 2020 armv7l GNU/Linux
[email protected]:~/raspiraw/hq1 $ 1012x760 4000 | grep -v "^[> ]"
removing /dev/shm/out.*.raw
capturing frames for 4000ms with 195fps requested
577 frames were captured
majority framerate 197fps
frame delta time[us] distribution
after skip frame indices (middle column)
26% frame skips
average framerate 144fps
[email protected]:~/raspiraw/hq1 $ 

First frame captured with 5.4 kernel on /dev/i2c-10:
out.0500.ppm.d.png.jpg
out.0500.ppm.d.png.jpg
out.0500.ppm.d.png.jpg (66.85 KiB) Viewed 142 times
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sun May 24, 2020 8:28 am

The tools as checked in sofar do not pass $fps to raspiraw, so the (good) results are likely from free running mode.

I did a local copy of raspiraw/tools/1012x760 and added " --fps ${fps}" to raspiraw command there.
While requesting 120fps gives 120fps, there is a 5/6 scaling bug for framerates above 120fps, but for below as well:

Code: Select all

$ ./1012x760 4000 90 | grep -v "^[> ]" | grep fps
capturing frames for 4000ms with 90fps requested
majority framerate 75fps
average framerate 76fps
$

I did long runs requesting framerates from 120fps up to 300fps.
Highest request repeatedly achieving 5/6 of requested for majority framerate was 262fps with 220fps achieved majority framerate.
But with that average framerate is only 139fps -- maximal value for average framerate seen in 120..300 request range is 145fps.

Code: Select all

$ ./1012x760 4000 262 | grep -v "^[> ]" | grep fps
capturing frames for 4000ms with 262fps requested
majority framerate 220fps
average framerate 139fps
$ ./1012x760 4000 262 | grep -v "^[> ]" | grep fps
capturing frames for 4000ms with 262fps requested
majority framerate 220fps
average framerate 141fps
$ ./1012x760 4000 262 | grep -v "^[> ]" | grep fps
capturing frames for 4000ms with 262fps requested
majority framerate 220fps
average framerate 138fps
$ 

P.S:
I did use checked in tool and achieved 197fps majority framerate again, but with only 145fps average framerate:

Code: Select all

$ 1012x760 4000 | grep -v "^[> ]" | grep fps
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8)
capturing frames for 4000ms with 195fps requested
majority framerate 197fps
average framerate 145fps
$
Even after reboot.
But this was with 5.4 kernel after rpi-update.
On Pi4B with Buster 4.7 kernel average framerate seen is 155fps for majority framerate 197fps.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

User avatar
HermannSW
Posts: 2305
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sun May 24, 2020 10:15 am

The most important register for v1/v2 camera high framerates (up to 750fs/1007fps) was raspiraw "--height" command register.
On my fork-raspiraw I used yos_reg register of sensor for handling both, v1 and v2 camera:
https://github.com/Hermann-SW/fork-rasp ... aw.c#L1029

For imx477 sensor I created separate .reg files for easy diffing:

Code: Select all

🍓 sed -n "/imx477_4056x3040_regs/,/^}\;/p" imx477_modes.h > 3040.reg
🍓 sed -n "/imx477_2028x1520_regs/,/^}\;/p" imx477_modes.h > 1520.reg
🍓 sed -n "/imx477_2028x1080_regs/,/^}\;/p" imx477_modes.h > 1080.reg
🍓 sed -n "/imx477_1012x760_regs/,/^}\;/p" imx477_modes.h > 760.reg
🍓

"diff 1520.reg 1080.reg" only has few differences, and it turned out that 0x034[ef] registers are for height:

Code: Select all

🍓 grep 0x034[ef] *.reg
1080.reg:      {0x034e, 0x04},
1080.reg:      {0x034f, 0x38},
1520.reg:      {0x034e, 0x05},
1520.reg:      {0x034f, 0xf0},
3040.reg:      {0x034e, 0x0b},
3040.reg:      {0x034f, 0xe0},
760.reg:      {0x034e, 0x02},
760.reg:      {0x034f, 0xf8},
🍓 

These are the frame heights in hex:

Code: Select all

🍓 echo -e "obase=16\n1080\n1520\n3040\n760" | bc -q
438
5F0
BE0
2F8
🍓 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

Return to “Camera board”