Arducam cameras fitting Raspberry CSI-2 interface

[HermannSW]

I do own 7 v1 cameras, as well as two v2.

I did buy some Arducam cameras as well, eg. ArduCam OV5647 Camera Board w/ M12 Mount, which I used for starting global external shutter captures before I learned to make Arduino v1 camera (clone) doing so as well:
https://www.raspberrypi.org/forums/view ... p?t=235523

In this posting I mentioned the sample cameras I received from Arducam, or were available for directly running from Raspberry CSI-2 interface:
https://www.raspberrypi.org/forums/view ... 3#p1485903

I just received a new package with two samples from Lee Jackson, the founder of Arducam. I tested the smaller sample first, 2MP monochrome global external shutter camera. Just followed instructions in Wiki for installing the drivers for now 6 Arducam Mipi cameras (0.3/1.0/2.0MP monochrome global shutter cameras, 13/16/18MP color rolling shutter cameras):
https://github.com/ArduCAM/MIPI_Camera/tree/master/RPI

This was first 1600x1300 image taken with "capture" sample. If the image looks dithered, then the image does not fit into your browser view. Just right click to open image in new tab, and use magnifying glass to see image 100% clearly:



This is the camera connected, the camera side has small connector as the Pi Zero has. The two connector cables are soldered to camera GND and FSIN pins, allowing to externally trigger capture. The other exposed pins are VCC and STROBE:



I have not tested sofar, but the camera video allows for [email protected] (the 0.3MP ov7251 allowed for [email protected]).

[HermannSW]

Now I tested the bigger sample I got, and it is quite impressive.
The rule until today was:

If you want to connect two cameras to a Pi, you need Compute module

This is not true anymore, Arducam Stereo camera hat allows to connect two cameras to any Raspberry Pi.
I got this sample so early that Lee did not even posted on his blog about it.
That posting will provide more details, I just describe what I have here.

The difference of the Arducam UC-625 stereo camera hat to the other Mipi cameras from Arducam is, that you can connect two Raspberry v1 or v2 cameras, and then use raspistill or raspivid -- Raspberry will not see that in fact two cameras are connected, it will see only one! In additon to that, you can connect two of the 6 Arducam Mipi cameras and use the Arducam SDK to get them record stereo video as well.

With Raspberry cameras and raspistill/raspivid left and right camera have only half horizontal resolution. With Arducam SDK you can get full resolution up to two times 16MP stereo.

I connected the two v1 (ov5647) cameras to the hat and the hat to Pi4B (it can be any other Pi) CSI-2 connector. After poewing the camera gets detected as if it was one:

Code: Select all

[email protected]:~ $ vcgencmd get_camera
supported=1 detected=1
[email protected]:~ $

The same i2c addresses are in use as with v1 camera (0x36), some crypto chip and the hat itself (0x0c, 0x24):

Code: Select all

[email protected]:~/raspiraw $ i2cdetect -y 0
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- 0c -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- 24 -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- 36 -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         
[email protected]:~/raspiraw $

So what does half width stereo look like?
I did record a 30fps mode1 1920x1080 video with raspivid:

Code: Select all

[email protected]:~ $ raspivid -md 1 -w 1920 -h 1080 -fps 30 -pts tst.pts -o tst.h264
[email protected]:~ $

I placed a mirror in the view so that you can see the hat and the two v1 cameras as well. I uploaded the video to youtube, what you see is a single frame from that video:
https://www.youtube.com/watch?v=S5dzfxs ... e=youtu.be



What you cannot see in that frame is that there are some cables soldered between the two cameras. They connect the oscillators of both cameras and provide synchronization of frame capturing at the nanosecond level:



What you can see as well is, that the hat connects to pins 1-6 of the Pi only, all other GPIO pins are freely accessible.

I used ptsanalyze
https://github.com/Hermann-SW/userland/ ... ptsanalyze

to see what framerate the recorded video really has. It is 28fps, a bit slower than the requested 30fps:

Code: Select all

[email protected]:~ $ ./ptsanalyze tst.pts 1
creating tstamps.csv
139 frames were captured at 28fps
frame delta time[us] distribution
      1 34690
      3 34693
      2 34694
      3 34695
      1 34696
      5 34697
      7 34698
      9 34699
     58 34700
     24 34701
      4 34702
      4 34703
      4 34704
      3 34705
      2 34706
      2 34707
      3 34708
      1 34711
after skip frame indices (middle column)
0 frame skips (0%)
[email protected]: $

Until now I only once played with stereo vision. Thread "Stereoscopic camera capture with single v2 camera" ended after I had to learn that a single camera alone without extra mechanisms can never provide any depth (split view in that thread did work):
https://www.raspberrypi.org/forums/view ... 3&t=230312

Now I can do real stereo camera stuff, others will be able to do when product is available.


P.S:
It is nice that I got the ov5647 version stereo cameras.
With my Rasberry v1 camera global external shutter project I will be able to capture 3000eps multiple exposure frames in stereo.
And I will be able to create 3000fps stereo video from that as described here:
https://github.com/Hermann-SW/Raspberry ... ure-frames


P.P.S:
I just executed mode7 capture with preview (screenshot taken via ssh with raspi2png):

Code: Select all

$ raspivid -md 7 -w 640 -h 480 -p 22,50,640,460 -t 0

[HermannSW]

P.S:
It is nice that I got the ov5647 version stereo cameras.
With my Rasberry v1 camera global external shutter project I will be able to capture 3000eps multiple exposure frames in stereo.

I tried that with the two ov5647 cameras connected to UC-625 stereo hat.
While the 1st stereo frame (1920x1080) captured below shows that initially the global shutter settings are effective (the mini drone propeller rotated with >20000rpm or >333 rotations per second), some frames later in the video cameras are back in rolling shutter mode, see 2nd stereo frame below (frames where created using pwm_ges tool and default settings, 3KHz PWM flashes of 8.33µs duration for 2s).

Not a big deal, since for taking stereo global shutter video two of the 0.3/1.0/2.0MP global shutter cameras can be taken.
What can be taken from my project is the 20$ diy highspeed flash:
https://github.com/Hermann-SW/Raspberry ... comparison

P.S:
SInce pins 1-6 were used by stereo camera hat, I had to connect the diy highspeed flash mosfets somewhere else.
I connected to pins 33, 25 and 17, easy to remember (starting high at odd pins, 3 free, SIG, 3 free, GND, 3 free, 3V3).

[Realizator]

What you cannot see in that frame is that there are some cables soldered between the two cameras. They connect the oscillators of both cameras and provide synchronization of frame capturing at the nanosecond level:

Hi Hermann,
We are looking for sync option too, and consider oscillators sync as the last approach (as it requires either soldering, or special PCB design). Just a curious, have you tried another approaches (like GPIO sync supported by IMX219) or, may be, another custom cameras with such a sync support?

[HermannSW]

We are looking for sync option too, and consider oscillators sync as the last approach (as it requires either soldering, or special PCB design).

What I know from Arducam is that you can either solder yourself, or buy camera bundle that is already soldered.

Just a curious, have you tried another approaches (like GPIO sync supported by IMX219)

I not, but some students from MIT used GPIO syncing to capture bullet after passing through apple from 16 cameras all at the same point in time (with v1 camera global reset and dark background, to get global external shutter captures). You can find the software they used for GPIO sync on their github repo:
https://www.raspberrypi.org/forums/view ... p?t=240442

or, may be, another custom cameras with such a sync support?

I started my work on v1 camera global external shutter with buying a special Arducam ov5647 camera. It is similar to v1 camera (clone), with some ov5647 pins exposed (most importantly for me was FREX, later it turned out that global reset can be triggered with I2C command as well):
https://www.raspberrypi.org/forums/view ... p?t=235523

[HermannSW]

The blog posting I mentioned has been done on Arducam site.
You can find much more details in it:
https://www.arducam.com/dual-camera-hat ... raspberry/

I was surprised by a new use case, makes sense:
"Panoramic (3D) photos and videos" (with fisheye lenses)



I got two ov9281 monochrome 1MP global shutter cameras from Lee some time ago. Since my experiment with global shutter mode for the two ov5647 cameras connected to the stereo hat did not work (see my 2nd last posting in this thread), I asked Lee for soldering instructions in order to make the two ov9281 modules work with stero hat. I just received the instructions (removing oscillator and crypto chip from one module, solder two jumping wires for GND (redundant) and oscillator clock between the two modules). When back at home on Friday I will try to follow those instructions and report back.

[HermannSW]

..., I asked Lee for soldering instructions in order to make the two ov9281 modules work with stero hat. I just received the instructions (removing oscillator and crypto chip from one module, solder two jumping wires for GND (redundant) and oscillator clock between the two modules). When back at home on Friday I will try to follow those instructions and report back.

OK, I tried today, but without success.

First I tried to desolder the crypto chip on module B. I did so with very big zoom with M12 lens showing the crypto chip big on HDMI monitor. After putting soldering iron into place and heating crypto chip pins solder, I tried to remove the fluid solder with solder sucker (right in photo):



After this did not really work, I used a small screw driver and removed layer after layer of the crypto chip with the screwdriver. Then I used pincer pliers and screw driver to get rid of module B ascillator as well. Finally I did solder a short cable to the pin of the removed oscillator I read about in the instruction:



On camera module A I did open one line per instruction and soldered short cable to it:



After I connected all together, I tried Arducam RPI/preview.c demo application, but no preview window did show up.
Then I realized that unlike the soldered pair of ov5647 had module B connected to MIPI CAM1 connector of camera module.
Switching the cameras (after shutdown, then boot up) preview demo still did not show a preview.

Finally I did remove the stereo hat, connected module A to Pi CSI-2 connector and preview demo did work.
Then, after powering down, I tested stereo hat with soldered pair of ov5647 cameras successfully.

Summary:
No damage besides unusable module B.
You should do this only with heatgun and good desoldering skill.
Ordering soldered camera bundle from Arducam will ensure working.

[HermannSW]

After my bad soldering experience I received an email from Lee:

BTW I told our engineer to design a stereo OV9281 camera adapter board, you don't worry about soldering any more . Do you have any requirements about the base line between these two cameras?

I did measure and distance between my pupil centers is 73mm.
After some problems with German TNT not knowing whether I am a private person (I am) or a company, a new packet from Arducam arrived.
I got two stereo camera modules, with both cameras being 80mm distant.
The left (connected) is new module with two ov5647 cameras (v1, UC-505), the disconnected has two ov9281 1MP monochrome global shutter cameras (UC-507):



Today I did test the ov9281 module with synchronized stereo hat (new one, UC-625, old works as well).
I did sync ArduCAM Mipi camera repo onto my Pi3B+ that has newly installed Buster:
https://github.com/ArduCAM/MIPI_Camera

Then I did follow the instructions in RPI/README.md to install library and build samples.

I looked for the available formats:

Code: Select all

[email protected]:~/MIPI_Camera/RPI $ ./list_format
mode: 0, width: 1280, height: 800, pixelformat: GREY, desc: (null)
mode: 1, width: 1280, height: 720, pixelformat: GREY, desc: (null)
mode: 2, width: 640, height: 400, pixelformat: GREY, desc: (null)
mode: 3, width: 320, height: 200, pixelformat: GREY, desc: (null)
mode: 4, width: 160, height: 100, pixelformat: GREY, desc: (null)
mode: 5, width: 1280, height: 800, pixelformat: GREY, desc: Used for ov9281 2lans raw8
mode: 6, width: 1280, height: 800, pixelformat: Y10P, desc: Used for ov9281 2lans raw10
mode: 7, width: 2560, height: 800, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 1280x800*2
mode: 8, width: 2560, height: 720, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 2560x720*2
mode: 9, width: 1280, height: 400, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 640x480*2
mode: 10, width: 640, height: 200, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 320x200*2
mode: 11, width: 320, height: 100, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 160x100*2
index: 0, CID: 0x00980914, desc: V4L2_CID_HFLIP, min: 0, max: 1, default: 0, current: 0
index: 1, CID: 0x00980915, desc: V4L2_CID_VFLIP, min: 0, max: 1, default: 0, current: 0
index: 2, CID: 0x00980911, desc: V4L2_CID_EXPOSURE, min: 0, max: 65535, default: 681, current: 3000
index: 3, CID: 0x00980913, desc: V4L2_CID_GAIN, min: 0, max: 15, default: 1, current: 1
close camera status = 0
[email protected]:~/MIPI_Camera/RPI $ 

Formats 7-11 are the interesting ones. As I said previously, for the v1 and v2 Raspberry cameras capturing with raspistill/raspivid is possible, but at half the resolution for left and right camera (because the stereo hat pretends to be a normal Raspberry camera). For the 6 different ArduCAM Mipi cameras that can be run directly on Pi CSI-2 interface, the stereo hat provides left and right image in full resolution. ov9281 is 1MP camera, with resolution 1280x800. Therefore mode7 has size 2560x800.

First I did preview, these commands do that:

Code: Select all

./preview_setMode 7
./preview_resolution 2560 800

Finally I captured a frame with this command which asks for mode (right click and zoom to see full 2560x800 image):

Code: Select all

[email protected]:~/MIPI_Camera/RPI $ ./capture
Open camera...
mode: 0, width: 1280, height: 800, pixelformat: GREY, desc: (null)
mode: 1, width: 1280, height: 720, pixelformat: GREY, desc: (null)
mode: 2, width: 640, height: 400, pixelformat: GREY, desc: (null)
mode: 3, width: 320, height: 200, pixelformat: GREY, desc: (null)
mode: 4, width: 160, height: 100, pixelformat: GREY, desc: (null)
mode: 5, width: 1280, height: 800, pixelformat: GREY, desc: Used for ov9281 2lans raw8
mode: 6, width: 1280, height: 800, pixelformat: Y10P, desc: Used for ov9281 2lans raw10
mode: 7, width: 2560, height: 800, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 1280x800*2
mode: 8, width: 2560, height: 720, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 2560x720*2
mode: 9, width: 1280, height: 400, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 640x480*2
mode: 10, width: 640, height: 200, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 320x200*2
mode: 11, width: 320, height: 100, pixelformat: Y10P, desc: Used for synchronized stereo camera HAT 160x100*2
index: 0, CID: 0x00980914, desc: V4L2_CID_HFLIP, min: 0, max: 1, default: 0, current: 0
index: 1, CID: 0x00980915, desc: V4L2_CID_VFLIP, min: 0, max: 1, default: 0, current: 0
index: 2, CID: 0x00980911, desc: V4L2_CID_EXPOSURE, min: 0, max: 65535, default: 681, current: 681
index: 3, CID: 0x00980913, desc: V4L2_CID_GAIN, min: 0, max: 15, default: 1, current: 1
Please choose sensor mode: 7
Setting the mode...
choose the mode 7
Choose resolution 2560x800
Current mode  is 7
Notice:You can use the list_format sample program to see the resolution and control supported by the camera.
Enable Software Auto Exposure...
Enable Software Auto White Balance...
Waiting for automatic adjustment to complete...
Capture image mode7.jpg...
Setting the focus...
Failed to set focus, the camera may not support this control.
mmal: Enable JPEG encoder.

Close camera...
[email protected]:~/MIPI_Camera/RPI $ 

Pretty cool and detailed image, house roof top height in this room is 4.30m.
Next I wanted to "see the image in 3D", but I have no viewer.
So I scaled down the image to 20%, see below.
Then I did place flat cardboard vertically to 22" monitor on the border between left and right image.
Next I placed my nose against cardboard and made both images overlay.
Wonderful 3D view, the lamp in front, my head behind, and roof top far away:



Besides the official posting, synchronized stereo hat is not available on uctronics.com right now.
Questions on pricing would have to go to Arducam support:
https://www.arducam.com/docs/cameras-fo ... amera-hat/