andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jun 22, 2020 1:28 am

Hello :)

New here and hoping to try out raspiraw with the HQ camera. Tried following Hermann's instructions on P24 of this thread however I am getting an error?

pi@raspberrypi:~/raspiraw $ ./camera_i2c
setting GPIO for board revsion: c03112
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
Error: Could not open file `/dev/i2c-0' or `/dev/i2c/0': No such file or directory

Please to help, thank you


Mike_green
Posts: 10
Joined: Wed Oct 17, 2018 6:36 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jun 22, 2020 8:20 am

Hi there! Working with raspiraw on Pi4 i faced with missing Ethernet dropping down problem. I think camera_i2c file needs some modifications because, if i understand correctly, 28 and 29 gpio pins on Pi4 are responsible for gigabit Ethernet and this code dropping it down

Code: Select all

raspi-gpio set 28 ip 
raspi-gpio set 29 ip

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9927
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jun 22, 2020 9:55 am

andred14 wrote:
Mon Jun 22, 2020 1:28 am
Hello :)

New here and hoping to try out raspiraw with the HQ camera. Tried following Hermann's instructions on P24 of this thread however I am getting an error?

pi@raspberrypi:~/raspiraw $ ./camera_i2c
setting GPIO for board revsion: c03112
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
Error: Could not open file `/dev/i2c-0' or `/dev/i2c/0': No such file or directory
Add "dtparam=i2c_vc=on" to /boot/config.txt
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9927
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jun 22, 2020 9:55 am

Mike_green wrote:
Mon Jun 22, 2020 8:20 am
Hi there! Working with raspiraw on Pi4 i faced with missing Ethernet dropping down problem. I think camera_i2c file needs some modifications because, if i understand correctly, 28 and 29 gpio pins on Pi4 are responsible for gigabit Ethernet and this code dropping it down

Code: Select all

raspi-gpio set 28 ip 
raspi-gpio set 29 ip
You're quite right, it does. I just haven't gotten around to making the change as yet. PRs welcome.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

CanPi
Posts: 64
Joined: Wed May 20, 2020 12:21 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jun 22, 2020 10:25 am

This is a step by step of how I was able to make raspiraw work with a 3A+

viewtopic.php?f=43&t=276047#p1675364

Not exactly intuitive.

andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jun 29, 2020 6:39 am

6by9 wrote:
Mon Jun 22, 2020 9:55 am
andred14 wrote:
Mon Jun 22, 2020 1:28 am
Hello :)

New here and hoping to try out raspiraw with the HQ camera. Tried following Hermann's instructions on P24 of this thread however I am getting an error?

pi@raspberrypi:~/raspiraw $ ./camera_i2c
setting GPIO for board revsion: c03112
Raspberry Pi3B / Pi3B+ / 3A / 4B(1G/2G/4G)
Set state of 133 to 1
Error: Could not open file `/dev/i2c-0' or `/dev/i2c/0': No such file or directory
Add "dtparam=i2c_vc=on" to /boot/config.txt
thank you got it working now :)

andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Jul 05, 2020 2:46 pm

Hi Team,

So what I am trying to do with raspiraw is I am hoping to combine it with some motion sensing and then we can try to catch someone who is shooting bbs or pellets at my window during the night. Any suggestions for me? Do you think I am on the right track?

User avatar
HermannSW
Posts: 3198
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Jul 05, 2020 3:54 pm

andred14 wrote:
Sun Jul 05, 2020 2:46 pm
So what I am trying to do with raspiraw is I am hoping to combine it with some motion sensing and then we can try to catch someone who is shooting bbs or pellets at my window during the night.
Pellets or bbs are small and fast, so it is unlikely for you to detect them in real time, with raspiraw or raspivid[yuv].
In case you know where pellets fly you can capture them:
https://github.com/Hermann-SW/Raspberry ... gun_pellet
viewtopic.php?f=43&t=267563&p=1642029&h ... r#p1642029
Image

In your case (pellet hitting window) you could better use a sound trigger to trigger capturing of image or video, better with raspistill/raspivid than with raspiraw in that case:
https://github.com/Hermann-SW/Raspberry ... nd-trigger
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9927
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Jul 05, 2020 5:43 pm

andred14 wrote:
Sun Jul 05, 2020 2:46 pm
Hi Team,

So what I am trying to do with raspiraw is I am hoping to combine it with some motion sensing and then we can try to catch someone who is shooting bbs or pellets at my window during the night. Any suggestions for me? Do you think I am on the right track?
Why do you need raspiraw? Generally you're going to be better off using raspivid or similar. Use a NoIR camera and IR illumination, or PIR to improve nighttime performance.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Jul 05, 2020 10:20 pm

Thanks very much for your advice, will look into all you have said

So far i bought the HQ camera and got raspiraw working in hopes of catching this person, i suppose i will remove the IR filter? Not sure if I will be able to have IR illumination but there is a fair bit of street lighting outside.

ideally it would be great to get a thermal image so that I can see exactly where the projectile is coming from?

User avatar
jbeale
Posts: 3727
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jul 06, 2020 12:58 am

You can buy thermal imagers, but they tend to be expensive and there isn't one made for the Raspberry Pi specifically. Also, nothing in the remotely "affordable" category of thermal imagers has enough useful resolution to be able to identify facial features at any distance.

The raw mode specifically is harder to use than the regular raspistill or raspivid and doesn't really help you here, as far as I know. You are describing a typical security camera application. The main challenge in that case is getting useful detail at a distance in low light. The most effective thing you can do is add more light, if at all possible. Could be infrared if you have to, although visible light gives you better image quality- and might even deter the perpetrator in the first place!

andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Jul 06, 2020 4:05 am

jbeale wrote:
Mon Jul 06, 2020 12:58 am
You can buy thermal imagers, but they tend to be expensive and there isn't one made for the Raspberry Pi specifically. Also, nothing in the remotely "affordable" category of thermal imagers has enough useful resolution to be able to identify facial features at any distance.

The raw mode specifically is harder to use than the regular raspistill or raspivid and doesn't really help you here, as far as I know. You are describing a typical security camera application. The main challenge in that case is getting useful detail at a distance in low light. The most effective thing you can do is add more light, if at all possible. Could be infrared if you have to, although visible light gives you better image quality- and might even deter the perpetrator in the first place!
I agree with you about the price of thermal imagers certainly not within range atm :)

The main goal for now is to just see where the bb or pellet is coming from. it may actually be a sniper bb or pellet gun from blocks away (I am in a high rise apartment and there are other high rises around)

Once we can see what apartment the projectile comes from we can then use a spotting scope with smartphone attachment to get a closeup. This part is already setup and ready to go.

This is why I was thinking raspiraw so that we can try for some high fps video of the bb/pellet that is hitting my window?

Thanks for everyone's help with this

andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Jul 07, 2020 2:10 am

Thanks folks, any advice would be appreciated because this person literally harasses me every night and I am losing sleep.

To give an idea of my background, I am a veteran coder/linux geek but don't know much about video&photography :geek:

Just imagine that someone is keeping you up at night by shooting pellets or bbs at your window but it is not obvious where they are coming from, how can we catch them on film?

User avatar
jbeale
Posts: 3727
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Jul 07, 2020 4:52 am

It's an interesting technical problem, but I'm not sure the Pi camera (in raw or normal mode) is the best tool, or even a very feasible one. A tiny high-speed pellet or bb at night is a very difficult thing to photograph, without a huge amount of added light, and if you trigger after impact then you've already lost the incoming trajectory anyway. I'd be tempted to put up styrofoam sheets on the outside of the window at night (if you have access to the outside surface) as they might actually capture the pellet and/or show its path. Failing that, they might also damp out the sound so it's not so annoying. If you don't have access to the outside, even padding on the inside surface of the window might at least damp out some of the noise. I'm also thinking the moderators here may be tempted to move this thread as I'm not sure it's part of what "raw sensor access" was supposed to be about.

User avatar
HermannSW
Posts: 3198
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Jul 07, 2020 7:24 am

jbeale wrote:
Tue Jul 07, 2020 4:52 am
If you don't have access to the outside,
Interesting point, I have not thought on that.
Even in that case, you can put a bright IR light on inside of window direction outside.
You can capture say 1fps video with raspiraw.
Then exposure time will be roughly 1s, and you should "see" the trajectory on one of the frames.
See my animation I posted above, the slower moving (reflected) pellets show trajectory.
I did use 5000lm visible light led, not sure how much IR light is needed.
You can test that by capturing airsoft pistol pellet trajectories yourself (I used 12$ pistol, pellets are cheap).
I think that you would get problems when putting 5000lm led on window sill direction out.
I am not sure about whether same strength IR light is OK or not.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

andred14
Posts: 8
Joined: Mon Jun 22, 2020 1:17 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Jul 08, 2020 3:50 am

HermannSW wrote:
Tue Jul 07, 2020 7:24 am
jbeale wrote:
Tue Jul 07, 2020 4:52 am
If you don't have access to the outside,
Interesting point, I have not thought on that.
Even in that case, you can put a bright IR light on inside of window direction outside.
You can capture say 1fps video with raspiraw.
Then exposure time will be roughly 1s, and you should "see" the trajectory on one of the frames.
See my animation I posted above, the slower moving (reflected) pellets show trajectory.
I did use 5000lm visible light led, not sure how much IR light is needed.
You can test that by capturing airsoft pistol pellet trajectories yourself (I used 12$ pistol, pellets are cheap).
I think that you would get problems when putting 5000lm led on window sill direction out.
I am not sure about whether same strength IR light is OK or not.
Thank you so much Hermann and jbeale these are great ideas and I will try them. We do have limited access to the outside ie. we can open the window at one place and mount things around it but we cannot reach all areas of the ~ 6ft x 12ft window since I am on the 15th floor.

Don't worry about the motion detection part as that's where my skills are good whether it be motion, OpenCV, AI we can figure it out. Just need to know the best way to photograph or video record the problem preferably with high fps.

So currently we have the raspberry PI HQ camera working with raspiraw, should we remove the IR filter as well? Also do you think we should try to cover the sensor or lenses with an IR-pass filter?

Again thanks so much everyone for your help

User avatar
HermannSW
Posts: 3198
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Aug 02, 2020 8:50 pm

I was made aware that raspiraw for v1 camera on Pi4B is not able to capture 640x64 at 665fps framerate anymore.
Reporter did run newly installed Raspberry OS, so is on 5.4 kernel.

I did update and upgrade my 4.19 kernel Buster, and after reboot it runs 5.4 kernel as well.

Is it known that v1 camera can only capture up to 300fps anymore?
viewtopic.php?f=43&t=281728&p=1706641#p1706641

While capturing 640x144 at 282fps is possible, frame shows abnormal black horizontal lines:
out.0103.ppm.d.png
out.0103.ppm.d.png
out.0103.ppm.d.png (161.56 KiB) Viewed 1114 times

̶I̶ ̶h̶a̶v̶e̶ ̶n̶o̶t̶ ̶t̶e̶s̶t̶e̶d̶ ̶v̶2̶ ̶c̶a̶m̶e̶r̶a̶ ̶y̶e̶t̶,̶ ̶w̶i̶l̶l̶ ̶d̶o̶ ̶t̶h̶a̶t̶ ̶n̶o̶w̶.̶

P.S:
v2 camera was able to capture 640x320 at 295fps before.
I did need to request as low as 233fps to get succeed, above it fails.
So there seems to be a 233fps barrier for v2 camera with Pi4B and 5.4 kernel:

Code: Select all

🍓 ./640x320 2000 | head -9
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 233fps requested
300 frames were captured at 234fps
frame delta time[us] distribution
      1 -9223372036854775808
      1 
      1 0
     26 4264
    133 4265
🍓 vi 640x320 

Code: Select all

🍓 ./640x320 2000 | head -9
removing /dev/shm/out.*.raw
capturing frames for 2000ms with 234fps requested
283 frames were captured at 117fps
frame delta time[us] distribution
      1 -9223372036854775808
      1 
      1 0
      1 4242
      1 4244
🍓 

Where do the 300fps v1 camera and 233fps v2 camera limits come from?

P.P.S:
I attached 640x320 tool used for v2 camera.
This was done with this branch that knows how to deal with v1 and v2 camera (6by9's raspiraw only knows v1):
https://github.com/Hermann-SW/fork-raspiraw
Attachments
640x320.zip
640x320 tool used
(676 Bytes) Downloaded 16 times
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9927
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Aug 03, 2020 1:24 pm

Fails totally, or fails with black/corrupt lines?
The black lines are an internal FIFO overrunning and SDRAM. There are no conscious changes in the clocking, but there has been a fair amount of churn with regard general clock setup.

There is a firmware call that can be made to request a higher internal clock frequency. I'll check that that is being made by rawcam, but otherwise you should be able to mimic the same thing by setting core_freq_min in config.txt as per the overclocking docs. Pi4 should take 500. I don't recall the value for earlier Pis.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

2U3
Posts: 9
Joined: Sun Jun 10, 2018 1:13 am

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Aug 21, 2020 5:55 am

Hello people,

I think one can get images from any CSI camera of bare module using MML library, if one provide voltages to the module and set registers properly.

Is it right?

And, is there D-PHY on RPi?

please assume, the number of lanes are up to two.

Thank you.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9927
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Aug 21, 2020 6:53 am

2U3 wrote:
Fri Aug 21, 2020 5:55 am
I think one can get images from any CSI camera of bare module using MML library, if one provide voltages to the module and set registers properly.

Is it right?

And, is there D-PHY on RPi?

please assume, the number of lanes are up to two.
Documentation - https://www.raspberrypi.org/documentati ... 2-usage.md

V4L2 is recommended over MMAL's rawcam component.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

2U3
Posts: 9
Joined: Sun Jun 10, 2018 1:13 am

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Sep 04, 2020 4:06 am

Thank you, 6by9.

Using "Unicam", it seems that any CSI camera could be used on RPi. So, I'd like to connect another CSI camera to RPi as shown in a figure attatched.

Then, could you or anybody tell me if my plan would be correct or not?

All needed are:
1. connect balls on a camera chip to terminals of a flat cable,
2. set up camera chip properlly via I2C,
3. set up bridge chip properlly via I2C.

Is it enough?

And, are there more detailed documents, or tutorials?

Or, anybody who has connected another CSI camera to RPi?

Thank you.

P.S. the number of lanes of the camera chip is one or two.
Attachments
mipi_camera.gif
mipi_camera.gif (7.4 KiB) Viewed 728 times

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9927
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Sep 04, 2020 6:06 am

2U3 wrote:
Fri Sep 04, 2020 4:06 am
Using "Unicam", it seems that any CSI camera could be used on RPi. So, I'd like to connect another CSI camera to RPi as shown in a figure attatched.

Then, could you or anybody tell me if my plan would be correct or not?

All needed are:
1. connect balls on a camera chip to terminals of a flat cable,
2. set up camera chip properlly via I2C,
3. set up bridge chip properlly via I2C.

Is it enough?

And, are there more detailed documents, or tutorials?

Or, anybody who has connected another CSI camera to RPi?

Thank you.

P.S. the number of lanes of the camera chip is one or two.
There is no bridge chip on the Pi. The CSI2 lanes go direct to the Unicam peripheral on the SoC. The picture at https://picamera.readthedocs.io/en/late ... ecture.svg (part of the Picamera Python library docs) gives a more accurate depiction.

Bridge chip is the term generally used to describe format conversion chips, be it from HDMI or analogue video, or parallel cameras, to CSI2 or some other format. If you're using a native CSI2 camera then there is no need for a bridge chip.

Your camera board is likely to also need:
- voltage regulators to take 3.3V to whatever supply rails your camera module needs.
- potentially level shifters for the I2C to take 3.3V I2C and match your camera module.
- whatever frequency oscillator your camera module needs.

No documentation as they are commercial products, but Arducam make a number of compatible modules, as do Vision Components, and Innomaker
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
HermannSW
Posts: 3198
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Oct 02, 2020 5:56 pm

For high framerate work for v1 and v2 cameras it was no problem that the tools had integer framerate output only. For 4056x3040 tool of HQ camera I added floating number statistics output. When passing 2nd argument (after millisecond duration 1st argument), floating point framerates get set for HQ camera as well.

The modified tool can be found on my long_exposure branch:
https://github.com/Hermann-SW/raspiraw/ ... /4056x3040

I have verified that it works with 6by9's raspiraw as well:
https://github.com/6by9/raspiraw

Before 12MP raspiraw capturing happened at maximal framerate (free running?) of 11.38fps.
This is default behavior when running without 2nd argument.
No frameskips with that framerate, and 8 seconds of recording fill /dev/shm 100% leaving less than 16MB available on 4GB Pi4B:

Code: Select all

R㉝p🍓▐ ./4056x3040 8000 
removing /dev/shm/out.*.raw
capturing frames for 8000ms with 11.38fps requested
90 frames were captured
majority framerate 11.38fps
frame delta time[us] distribution
      1 
     11 87858
     66 87859
     12 87860
after skip frame indices (middle column)
0% frame skips
average framerate 11.38fps
R㉝p🍓▐ 

If passing 10.75fps as framerate, the tool produces that with 0 frameskips again. /dev/shm gets filled 95%:

Code: Select all

R㉝p🍓▐ ./4056x3040 8000 10.75
removing /dev/shm/out.*.raw
capturing frames for 8000ms with 10.75fps requested
85 frames were captured
majority framerate 10.75fps
frame delta time[us] distribution
      1 
      1 92999
      1 93000
      8 93001
     66 93002
      7 93003
      1 93007
after skip frame indices (middle column)
0% frame skips
average framerate 10.75fps
R㉝p🍓▐ 

For looking at 85th frame dcraw can be called directly.
I just call raw2ogg2anim tool and stop after dcraw .ppm conversion with CTRL-C:

Code: Select all

R㉝p🍓▐ raw2ogg2anim x 85 85 1
removing old auxiliary files
copying /dev/shm/out.????.raw files
85     
dcraw each .raw file (to .ppm)
out.0085.raw     
.ppm -> .ppm.d
out.0085.ppm     
.ppm.d -> .ppm.d.png
^C
R㉝p🍓▐ ls -l out.0085.ppm
-rw-r--r-- 1 pi pi 36990737 Oct  2 19:46 out.0085.ppm
R㉝p🍓▐ 
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

User avatar
HermannSW
Posts: 3198
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Oct 02, 2020 6:39 pm

This is just shot into the dark.
Nothing prevents to request a higher framerate than maximal possible 11.38fps.
This works until 20fps.
But only a little more than top half of lines gets updated, up to line 1704.
So extraction of 4056x1704@20fps 12bit/pixel video is possible.
Of course storing the lines below 1704 is a waste of memory.
Like with v1 or v2 camera "height" option and/or v1 camera "vertical increment" needs to be implemented for imx477 sensor with raspiraw.

No frameskips between frames 15 and 63:

Code: Select all

R㉝p🍓▐ ./4056x3040 4800 20
removing /dev/shm/out.*.raw
capturing frames for 4800ms with 20fps requested
87 frames were captured
majority framerate 19.99fps
frame delta time[us] distribution
      1 
      1 49989
      1 49996
      4 50000
     61 50001
     10 50002
      1 50006
      1 50013
      7 100002
after skip frame indices (middle column)
> 100002,2,38455570140
> 100002,3,38455670142
> 100002,4,38455770144
> 100002,5,38455870146
> 100002,6,38455970148
> 100002,8,38456120151
> 49989,14,38456420158
> 100002,64,38458970212
> 49996,68,38459170216
9% frame skips
average framerate 18.49fps
R㉝p🍓▐

This is recorded split frame scaled to 15% size as forum attachment.
I only have seen endless mirrors when camera was directed at display preview.
But from the side 9" HDMI display is a mirror for the recording:
out.0063.ppm.15%.jpg
out.0063.ppm.15%.jpg
out.0063.ppm.15%.jpg (14.95 KiB) Viewed 508 times
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/working_with_FPGAs

User avatar
RichShumaker
Posts: 323
Joined: Tue Jul 31, 2012 4:16 pm
Location: Sunny Southern CA near downtown LA
Contact: Website Facebook Twitter YouTube

Re: Raw sensor access / CSI-2 receiver peripheral

Fri Oct 02, 2020 8:07 pm

andred14 wrote:
Sun Jul 05, 2020 10:20 pm
Edit Added top quote
The main goal for now is to just see where the bb or pellet is coming from. it may actually be a sniper bb or pellet gun from blocks away (I am in a high rise apartment and there are other high rises around)

ideally it would be great to get a thermal image so that I can see exactly where the projectile is coming from?
SuperZoom is very easy and possible with the HQ camera at a DIY cost
Old 35mm manual lenses with 2x or 3x adapters allow me to easily take pictures of the moon.

Why am I bringing this up?
You want as clear an image as possible to "catch" this person.
So I would set up more than one camera in the directions I think it may come from. Also they probably do it from a few set locations and over a few days you could possibly move the camera and zoom in directly on the corner or window it is coming from. Lastly it is possible with multiple cameras to potentially triangulate where they are within about a meter(way way above my skill level but I know It's possible)

I am Not a Coder(I hobble/hack code badly) so I use RPi_Cam_Web_Interface as it has everything built in.
I bring this up as people have used this software and custom coded some of the scripts so 1 camera triggers a series of cameras and it can easily do loop recording to grab frames from each camera before you technically hit record(forget the real name of this feature.)

Also no one has discussed the Pi's interframe option. This is only GPU possible so limited to 120fps at 640x480 or up to 1080p 30fps. This is a super low bandwidth option that only shows the vector differences(for all I know it is how different motion triggers work as it is from years ago)

Final thoughts. I would take 3 or 4 M12 lens Noir Camera knock offs with wide angle lenses and IR light from 1 to 5 feet. I would create a triangulation methodology with the cameras. Put RPiCamWeb on each Pi and link them. One gets motion they capture. Since I only need the vectors to triangulate I would explore the Vector Motion. Then Once that triangulation is done if I could I would use a computer controlled Pan Tilt to move the Superzoom HQ recording 1080P video at 30fps. Raw gets you 4k 12bit 10fps(11.3 if you follow what HermannSW has done). Just understand that RAW data is super processor intensive and requires Post Work. I am trying to get the pic of the person ASAP as they will gun & run. Also the triangulation method would allow you to narrow your Suoerzoom HQ for the next day or even the same night without a fancy computerized pan tilt. The vector/triangulation should be near realtime so if they stay put for 15 seconds you could manually pan zoom and grab the persons image.

EDIT
I also forgot to mention StereoPi https://stereopi.com/ which can take stereo images from 2 cameras and could possible just give you triangulation inside a single Pi(2 Cameras is a Compute Module ONLY feature)
Way way above my level BUT research into triangulatuon using stereo Cameras on RasPi has been done here is an example Paper I found for Machine Vision https://www.google.com/url?sa=t&source ... nn1MtKnmc9

And another
https://www.google.com/url?sa=t&source ... EEETnFWEBg


Hope this helps.
Rich Shumaker
http://www.instructables.com/id/Pi-Zero-W-NoIR-8MP-Camera-Build-Overview-Introduct/

Return to “Camera board”