User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Mar 06, 2019 11:20 am

6by9 wrote:
Wed Mar 06, 2019 11:01 am
...
It's a fudge rather than a fix. It just removes all the signalling so video_render has no information to go on for stereoscopic, and hence interprets it as a single plane (quite probably squashed by decimation).
...
I'll see if there is an easy way to check whether the display is actually stereoscopic from within video_render. Switching to I420 has a fairly significant performance hit on the GPU.
That's why I decided to ask you. ;)

pculverhouse
Posts: 3
Joined: Thu Jun 27, 2019 9:45 am

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Jun 27, 2019 9:59 am

Hi,

I am trying to do something non-standard with stereo images from the dual camera computer board using a Pi3.

I want to capture a stereo pair (at reduced resolution - probably 640x480), and pass the memory buffer to my built-in IP socket, so that I can send stereo pairs on demand to a host connected to the other end of the IP socket.

So, I don't want to use GL rendering and I don't want to pass the file handle to the MMAL so that I can save the pair to file.

I am struggling with raspistill.c to hack out the bits I need to do the above. BUt I get completely un-stuck when it comes to MMAL interface.

Any pointers most gratefully received.

Regards Phil

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Jun 27, 2019 11:12 am

You'd have been better off starting a new thread, but never mind.
pculverhouse wrote:
Thu Jun 27, 2019 9:59 am
I am trying to do something non-standard with stereo images from the dual camera computer board using a Pi3.

I want to capture a stereo pair (at reduced resolution - probably 640x480), and pass the memory buffer to my built-in IP socket, so that I can send stereo pairs on demand to a host connected to the other end of the IP socket.
Built in to what?
pculverhouse wrote:So, I don't want to use GL rendering and I don't want to pass the file handle to the MMAL so that I can save the pair to file.

I am struggling with raspistill.c to hack out the bits I need to do the above. BUt I get completely un-stuck when it comes to MMAL interface.
raspistill takes still images and encodes them to JPEG (or similar), but it sounds like you want a stream.
raspivid would encode to H264 or MJPEG.
raspividyuv will deliver you the raw YUV or RGB frames.

Two options:
- Modify the source code of raspividyuv to write to your socket. camera_buffer_callback is where you get the buffer, with buffer->data being the pointer to buffer->length bytes of image data.
- Use "raspividyuv -o -" and it writes to stdout. Pipe stdout into whatever program you're using to control your socket.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

pculverhouse
Posts: 3
Joined: Thu Jun 27, 2019 9:45 am

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri Jun 28, 2019 7:42 am

Thanks.

The machine is the Plymouth robot OWL. (See https://web-dr.tis.plymouth.ac.uk/resea ... ymouth-owl). Currently we use Linuxprojects http IP server to stream video encoded as MJPEG to an OpenCV application sitting on the host. But MJPEG has no buffer flowcontrol. So when the host is taking its time to process a stereo pair (disparity calculation for example) the MJPEG buffer continues to fill. A solution is to empty the buffer in the processing loop. But that causes time delays between sampling the video stream and the OWL taking the original images - the OWL is an agile camera system so grabbing a stereo pair, processing it and moving the cameras is the loop.

So, a solution is to take images on demand, in our own IP socket connection to the host. So the host issues a grab command, and one stereo pair is sent to the host by the Pi3. I think I will leave the camera stream open but issue grabs when I need.

I will try the camera_buffer_callback method, but you suggest using raspistillyuv.c, I'll have to understand how to set that for stereo. I did a quick search of the source and could not find any options.

thanks!

Phil

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri Jun 28, 2019 9:22 am

pculverhouse wrote:
Fri Jun 28, 2019 7:42 am
The machine is the Plymouth robot OWL. (See https://web-dr.tis.plymouth.ac.uk/resea ... ymouth-owl). Currently we use Linuxprojects http IP server to stream video encoded as MJPEG to an OpenCV application sitting on the host. But MJPEG has no buffer flowcontrol. So when the host is taking its time to process a stereo pair (disparity calculation for example) the MJPEG buffer continues to fill. A solution is to empty the buffer in the processing loop. But that causes time delays between sampling the video stream and the OWL taking the original images - the OWL is an agile camera system so grabbing a stereo pair, processing it and moving the cameras is the loop.
Do you need the MJPEG encoding to reduce the network bandwidth?
MJPEG has no intra-frame dependencies, so grab the frames and drop them if you don't want them.
I will try the camera_buffer_callback method, but you suggest using raspistillyuv.c, I'll have to understand how to set that for stereo. I did a quick search of the source and could not find any options.
raspiVIDyuv. raspistill and raspistillyuv will take still images at the specified time, but that involves mode switches and other things that take time.

If you use raspivid/raspividyuv, the camera will be running continuously delivering frames. Make the choice in your application as to whether you wish to forward that frame on to your network socket or not.

Stereoscopic support for all 4 raspicam apps should be identical - it's in RaspiCamControl.c rather than the individual apps.
Then again I think there may be the setup lines missing in create_camera_component

Code: Select all

   status = raspicamcontrol_set_stereo_mode(camera->output[0], &state->camera_parameters.stereo_mode);
   status += raspicamcontrol_set_stereo_mode(camera->output[1], &state->camera_parameters.stereo_mode);
   status += raspicamcontrol_set_stereo_mode(camera->output[2], &state->camera_parameters.stereo_mode);

   if (status != MMAL_SUCCESS)
   {
      vcos_log_error("Could not set stereo mode : error %d", status);
      goto error;
   }
as are present in RaspiVid.c
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

pculverhouse
Posts: 3
Joined: Thu Jun 27, 2019 9:45 am

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri Jun 28, 2019 1:31 pm

Thanks for that introspection on the processes.

All I know is that we experience a lag, which we think is due to the MJPEG encoding and the streaming by the Apache web server. Our thinking is that we can remove some of those latencies by doing it ourselves. The advantage of synchronous capture and host use is that we know absolutely that the camera stereo pair were captured shortly before we start processing the images. The problem is that we are using OpenCV and the camera interface does not give us a MJPEG buffer flush, we have to loop to remove all frames in the buffer. I consider the reason is that MJPEG is designed to provide a resilient video interface despite variable network delays. This is not what we need.

Some thinking is required. I'll add those missing setup lines.

Thanks for your support.

Phil

User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Sep 10, 2019 6:27 am

@6by9, looks like stereoscopic support has been broken in the latest kernels update.
If it take Buster image (Raspbian Buster with desktop, 2019-07-10, kernel 4.19.57), both raspistill and raspivid works with -3d option. I mean that after running these commands I have one stereoscopic image and one stereoscopic video with appropriate names:

Code: Select all

raspistill -3d sbs -o 1.jpg
raspivid -3d sbs -w 1280 -h 720 -o 1.h264
Before all these tests I just put our dt-blob.bin to /BOOT and enable camera in raspi-config.

If I do system upgrade with piwiz (and kernel is updated to 4.19.66), both raspistill and raspivid stops working. Both raspistill and raspivid has the same behavior in this case:
- preview is started
- preview image freezes
- adding '-n' option (no preview) has no any effect
- no image or video saved
- system can not be shut down correctly (sudo poweroff), and frozen preview image is always on the screen

I've also tested stock Buster with rpi-update (and updated kernel is 4.19.71), but no luck too.
Could you please check this issue?
UPD> 4.19.73-v7+ has been tested. Problem still exist.

UPD2> New Buster (2019-09-26 release), 4.19.75-v7+ - problem still exist. Behavior has changed: no preview window now, and system can be shut down correctly by sudo poweroff.

UPD3> According to our GitHub discussion, current temporarily solution is to turn on old AWB algorithm once right after boot using this command:

Code: Select all

sudo vcdbg set awb_mode 0
I hope this info will be usefull for all who faced this problem.

wkeeling
Posts: 78
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas
Contact: Facebook

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri Mar 13, 2020 11:38 pm

pi@picam2:~ $ sudo vcdbg set awb_mode 0
vcdbg: symbol lookup error: vcdbg: undefined symbol: OpenVideoCoreMemoryFileWithOffsetAndSize
pi@picam2:~ $ uname -a
Linux picam2 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l GNU/Linux
pi@picam2:~ $


any ideas i have Compute Module with v2 cams -- can run 2 concurrent copies of raspivid to create 2 files but can't get -3d to work
Willie Keeling

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Mar 14, 2020 10:47 am

wkeeling wrote:
Fri Mar 13, 2020 11:38 pm
pi@picam2:~ $ sudo vcdbg set awb_mode 0
vcdbg: symbol lookup error: vcdbg: undefined symbol: OpenVideoCoreMemoryFileWithOffsetAndSize
pi@picam2:~ $ uname -a
Linux picam2 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l GNU/Linux
pi@picam2:~ $


any ideas i have Compute Module with v2 cams -- can run 2 concurrent copies of raspivid to create 2 files but can't get -3d to work
I'm guessing you've rebuilt userland, and I think there's a change to https://github.com/raspberrypi/userland ... /debug_sym that I've made in the firmware tree and not pulled across to userland - sorry.
If you reinstall /opt/vc/lib/libdebug_sym.so from apt (not possible for the single file), rpi-update, or similar, then I'd hope it'll get you working again. I'll also sort out pulling the change across to userland.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Mar 14, 2020 10:48 am

And I do have potential changes to the new AWB algorithm that may solve the issue, but I haven't had a stereoscopic rig setup to test.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Mar 14, 2020 12:59 pm

6by9 wrote:
Sat Mar 14, 2020 10:48 am
And I do have potential changes to the new AWB algorithm that may solve the issue, but I haven't had a stereoscopic rig setup to test.
I can send you a stereoscopic kit to test this. Please contact Gordon (I'm talking with him by Twitter DM) to discuss delivery details.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Mar 14, 2020 3:03 pm

Realizator wrote:
Sat Mar 14, 2020 12:59 pm
6by9 wrote:
Sat Mar 14, 2020 10:48 am
And I do have potential changes to the new AWB algorithm that may solve the issue, but I haven't had a stereoscopic rig setup to test.
I can send you a stereoscopic kit to test this. Please contact Gordon (I'm talking with him by Twitter DM) to discuss delivery details.
Thanks for the offer, but I have all the kit, just haven't found the time to set it up.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

wkeeling
Posts: 78
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas
Contact: Facebook

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Mon Mar 16, 2020 1:43 pm

I have rebuilt userland -- I will try the update you suggested -- thanks
Willie Keeling

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Mon Mar 16, 2020 4:27 pm

Userland has now been updated with the missing commit from firmware, therefore you should be able to rebuild a compatible libdebug_sym.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

wkeeling
Posts: 78
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas
Contact: Facebook

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Mar 17, 2020 12:46 am

[/quote]
I'm guessing you've rebuilt userland, and I think there's a change to https://github.com/raspberrypi/userland ... /debug_sym that I've made in the firmware tree and not pulled across to userland - sorry.
If you reinstall /opt/vc/lib/libdebug_sym.so from apt (not possible for the single file), rpi-update, or similar, then I'd hope it'll get you working again. I'll also sort out pulling the change across to userland.
[/quote]

I was not able to make install with my copy of userland work with either replacing debug_sym and rebuilding userland or by rpi-update.

So built another image at 4.19.50 and -3d works with now issues. After upgrading to 4.19.97 -3d hangs system if “sudo vcdbg set awb_mode 0” not run before. Also -awb off does not resolve the issue.

Thanks
Willie Keeling

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Apr 07, 2020 3:54 pm

I've finally found time to get a stereo rig set up again and tested. It seems to be all happy now with the new AWB algorithm, so you should be able to use the next rpi-update with it. I've tested on both a CMIO and StereoPi setup.

StereoPi seems to be slightly less than happy using the common firmware branch, so there is a small amount more digging to do. It's a touch trickier than normal as StereoPi doesn't have the firmware JTAG connector on. I'm going to try transplanting the SD card across to the CMIO (with CMIO dt-blob.bin) and see what is going on there.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Apr 11, 2020 3:49 pm

6by9 wrote:
Tue Apr 07, 2020 3:54 pm
I've finally found time to get a stereo rig set up again and tested. It seems to be all happy now with the new AWB algorithm, so you should be able to use the next rpi-update with it. I've tested on both a CMIO and StereoPi setup.
After rpi-update I got 4.19.114-v7+, but problem remains. That is I need to use 'awb_mode 0'. Is your changes available with rpi-update now, or I should wait for update?
6by9 wrote:
Tue Apr 07, 2020 3:54 pm
StereoPi seems to be slightly less than happy using the common firmware branch, so there is a small amount more digging to do. It's a touch trickier than normal as StereoPi doesn't have the firmware JTAG connector on. I'm going to try transplanting the SD card across to the CMIO (with CMIO dt-blob.bin) and see what is going on there.
I did rpi-update, and after that "sudo BRANCH=next rpi-update" on the StereoPi, and "sudo reboot".
System boot, and 'uname-r' got me "5.4.29-v7".
Wifi dongle requires remove-insert to work again.
System boots, raspistill -cs 0 works. Using -3d sbs hungs the system.
And... after that I did power-off power-on, and system not booting, just a black screen...

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Apr 11, 2020 4:22 pm

No, the changes haven't been merged as yet. I'll nudge people after the bank holidays.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Apr 15, 2020 12:59 pm

6by9 wrote:
Sat Apr 11, 2020 4:22 pm
No, the changes haven't been merged as yet. I'll nudge people after the bank holidays.
With the latest updates, stereoscopic capture now works out of the box (tested on both "old" and "common" firmware, and both 4.19.114-v7+ and 5.4.29-v7+ kernels). Thank you! By the way, new AWB algorithm dramatically increases image quality for indoor captures, no more "yellow" or "green" images on V1 sensor. It's great that we don't need to turn it off for the stereoscopic capture!

Also, with the common firmware system boots with both CM3 and CM3+ (I posted some details in a common firmware thread).

Just a curious, when it is planned to push these changes to the general Raspian update.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Apr 15, 2020 1:14 pm

Realizator wrote:
Wed Apr 15, 2020 12:59 pm
6by9 wrote:
Sat Apr 11, 2020 4:22 pm
No, the changes haven't been merged as yet. I'll nudge people after the bank holidays.
With the latest updates, stereoscopic capture now works out of the box (tested on both "old" and "common" firmware, and both 4.19.114-v7+ and 5.4.29-v7+ kernels). Thank you! By the way, new AWB algorithm dramatically increases image quality for indoor captures, no more "yellow" or "green" images on V1 sensor. It's great that we don't need to turn it off for the stereoscopic capture!

Also, with the common firmware system boots with both CM3 and CM3+ (I posted some details in a common firmware thread).
Thanks for the confirmation.
Realizator wrote:Just a curious, when it is planned to push these changes to the general Raspian update.
When things are considered stable.
We've got 2 things going on:
- switching to a common firmware for Pi0-3 and Pi4.
- switching to the 5.4 Linux kernel.
Both of which have a number of potential issues that need to be tested for. Once they are both considered stable, they'll get promoted to apt.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

wkeeling
Posts: 78
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas
Contact: Facebook

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Apr 29, 2020 3:23 pm

I have written a camera program that basically setup camera and encoder (h.264) components. These components are connected, and the output written by the encoder callback.

With stereo mode off it works at all resolutions. With side-by-side and works at all widths that are multiples of 32 (as noted in the rules at start of thread even if not 32 aligned generates frame with garbage) and any height. With Top-Bottom it only works and certain resolutions (480X864, 1040X1824, 1056X1856, 1080X1920, 1088X1920, 1112X1920, 1120X1920, 1144X1920 and 1152X1920).
The issue is I need TB for the application (race car stream front and rear cams) and need 720X1280 for throughput.

Any ideas on what I am doing wrong?

Program build from userland GitHub download. Source is at https://github.com/wkeeling63/userland/ ... cam/rcam.c. Camera mode is 5. Kernel is 4.19.108-v7+ #1298 SMP.

Thanks
William Keeling
Willie Keeling

User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Apr 30, 2020 1:31 pm

As for the brand new Raspberry Pi High Quality Camera - if it has some caveats for the stereoscopic mode? If RPi's GPU will allow us to capture 2x12 = 24 Mpix stereoscopic images? Do we need to fix dt-blob?
Also one more question - if "LED" line on CSI-2 still unused? (or on this module it is possible to use some lines for the cameras synchronization?..)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Apr 30, 2020 2:43 pm

wkeeling wrote:
Wed Apr 29, 2020 3:23 pm
I have written a camera program that basically setup camera and encoder (h.264) components. These components are connected, and the output written by the encoder callback.

With stereo mode off it works at all resolutions. With side-by-side and works at all widths that are multiples of 32 (as noted in the rules at start of thread even if not 32 aligned generates frame with garbage) and any height. With Top-Bottom it only works and certain resolutions (480X864, 1040X1824, 1056X1856, 1080X1920, 1088X1920, 1112X1920, 1120X1920, 1144X1920 and 1152X1920).
The issue is I need TB for the application (race car stream front and rear cams) and need 720X1280 for throughput.

Any ideas on what I am doing wrong?

Program build from userland GitHub download. Source is at https://github.com/wkeeling63/userland/ ... cam/rcam.c. Camera mode is 5. Kernel is 4.19.108-v7+ #1298 SMP.
Sorry, not really got time to test things out at present. Does the same thing work via raspivid?
I did notice something odd going on with the StereoPi rig which I do intend to get back to when I get a chance, and that was related to resolution not being a multiple of something. You may be seeing a similar thing.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11019
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Apr 30, 2020 2:58 pm

Realizator wrote:
Thu Apr 30, 2020 1:31 pm
As for the brand new Raspberry Pi High Quality Camera - if it has some caveats for the stereoscopic mode? If RPi's GPU will allow us to capture 2x12 = 24 Mpix stereoscopic images? Do we need to fix dt-blob?
Also one more question - if "LED" line on CSI-2 still unused? (or on this module it is possible to use some lines for the cameras synchronization?..)
I must confess to not having tested IMX477 in stereo mode. I only have one here (working from home) so can't test at present. But the theory is certainly that you should be able to combine the two sensors together into a stereo system delivering 24MPix images.
dt-blob is the same as before as you're only defining the control lines.

The LED line is indeed still unused. It's actually been removed from the Pi side on the Pi4 so you can't turn the LED of a v1 on even if you wanted to.

I've got an early production board here, and it has additional pads breaking out GPO, FSTROBE, XVS, and GPI, as well as the I2C. They aren't used by the firmware.
A quick check of the datasheet
GPO Digital output
FSTROBE Digital output. Flash strobe
XVS Digital I/O (Vertical sync signal) Used as a vertical synchronizing signal of Dual Camera
GPI no reference

Other references
6-9-5 Dual sensor synchronization operation
IMX477-AACK-C supports synchronized shooting operation of two image sensors by implementing both slave
and master mode for each sensor. To enable this feature, master/slave must be set for each sensor by software
control method. See “Dual Camera” of Software reference manual for details.

It looks like you link the XVS lines between the two sensors. Master sets it to output, slave sets it as an input.

6-9-6 Flash light control sequence
IMX477-AACK-C can internally generate the control pulse assuming to trigger the flash light emission and output
from the external pins (FSTROBE). See “Flash light control sequence” of Software reference manual for details.

It's not something that we've investigated, and not really a priority at present. And, no, I can't release the datasheet!
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Realizator
Posts: 69
Joined: Thu Jul 14, 2016 12:53 pm
Contact: Website Twitter

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Apr 30, 2020 3:02 pm

6by9 wrote:
Thu Apr 30, 2020 2:43 pm
wkeeling wrote:
Wed Apr 29, 2020 3:23 pm
... With stereo mode off it works at all resolutions. With side-by-side and works at all widths that are multiples of 32 (as noted in the rules at start of thread even if not 32 aligned generates frame with garbage) and any height.
....
Sorry, not really got time to test things out at present. Does the same thing work via raspivid?
I did notice something odd going on with the StereoPi rig which I do intend to get back to when I get a chance, and that was related to resolution not being a multiple of something. You may be seeing a similar thing.
@wkeeling and @6by9
It is mentioned in the PiCamera docs, that horizontal resolution should be multiple of 32, and vertical by 16. @wkeeling, in your post you did not mentioned about vertical resolution test (division by 16), but horizontal only.

Return to “Camera board”