dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 11:57 pm

Thanks a ton for your effort 6by9!
It would be perfect if stable 90fps would be achieved in this mode.

Best regards

dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Thu Feb 26, 2015 2:20 pm

6by9: I have checked 3d in raspivid from your userland repo, it works with up to 60fps in stereo mode.
Do you have an idea why the 90fps are not achieved?
From the calculation of pixels throughput, the MMAL should easily be able to handle it.

I also checked the V4L2 way of getting stereo camera but I was not able to get the desired mode there too.

Best Regards

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Thu Feb 26, 2015 3:07 pm

dweber wrote:6by9: I have checked 3d in raspivid from your userland repo, it works with up to 60fps in stereo mode.
Do you have an idea why the 90fps are not achieved?
From the calculation of pixels throughput, the MMAL should easily be able to handle it.
The ISP hardware should be able cope. It's likely to be some of the control loops that are taking too long when trying to run on 2 channels.
They normally take about 6-7ms per frame to work out appropriate new settings. The image then has to be processed by the hardware with those settings, and at 90fps you've got 11ms to get that all done. Add in a second channel and that's quite a lot of computation for not much time.

I haven't had a chance to investigate further as yet - I need to dig down into the processor occupancy on the GPU and various other bits. It's on my list.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Lstri
Posts: 3
Joined: Mon May 04, 2015 2:27 pm

Re: Stereoscopic camera capture - now implemented.

Fri May 08, 2015 7:46 am

6by9 wrote:
dweber wrote:6by9: I have checked 3d in raspivid from your userland repo, it works with up to 60fps in stereo mode.
Do you have an idea why the 90fps are not achieved?
From the calculation of pixels throughput, the MMAL should easily be able to handle it.
The ISP hardware should be able cope. It's likely to be some of the control loops that are taking too long when trying to run on 2 channels.
They normally take about 6-7ms per frame to work out appropriate new settings. The image then has to be processed by the hardware with those settings, and at 90fps you've got 11ms to get that all done. Add in a second channel and that's quite a lot of computation for not much time.

I haven't had a chance to investigate further as yet - I need to dig down into the processor occupancy on the GPU and various other bits. It's on my list.

Hi,6by9,is it possible to control two pi cameras with c++ or python?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Fri May 08, 2015 9:01 am

Lstri wrote:Hi,6by9,is it possible to control two pi cameras with c++ or python?
As stereoscopic, or as two independent cameras? In either case, yes.

For C (easy to convert to C++) https://github.com/raspberrypi/userland ... RaspiVid.c
Either -cs to select which camera, or as per posting.php?mode=quote&f=43&p=754479#pr700833 "-3d [none|sbs|tb]"

From Python http://picamera.readthedocs.org/en/rele ... ereoscopic, in the 1.8 release.
"Stereoscopic cameras are now tentatively supported on the Pi compute module. Please note I have no hardware for testing this, so the implementation is possibly (probably!) wrong; bug reports welcome! (#153)"
And http://picamera.readthedocs.org/en/rele ... amera.html defines that the constructor is
class picamera.camera.PiCamera(camera_num=0, stereo_mode='none', stereo_decimate=False, resolution=None, framerate=None, sensor_mode=0, led_pin=None)
That seems to give you both options.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

draco
Posts: 18
Joined: Fri Aug 15, 2014 6:42 am
Location: Melbourne, Australia

Re: Stereoscopic camera capture - now implemented.

Tue May 26, 2015 7:11 am

Great work on the stereo camera support. Thank you 6by9.

I've been testing the dual camera set up with manual camera settings, but it appears that the camera settings only apply to the primary camera. How do I apply my camera settings to both cameras?

The command that I'm running:

Code: Select all

raspistill -v -sh 100 -ISO 100 -awb horizon -ex off -ss 33333 -mm average -t 1 -q 100 -th none -n -w 5184 -h 1944 -3d sbs -3dswap -o /tmp/test.jpg
Thanks in advance.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Tue May 26, 2015 9:43 am

draco wrote:I've been testing the dual camera set up with manual camera settings, but it appears that the camera settings only apply to the primary camera. How do I apply my camera settings to both cameras?

The command that I'm running:

Code: Select all

raspistill -v -sh 100 -ISO 100 -awb horizon -ex off -ss 33333 -mm average -t 1 -q 100 -th none -n -w 5184 -h 1944 -3d sbs -3dswap -o /tmp/test.jpg
That sounds like a bug to me as you shouldn't need to change anything from the app side.
The tuners should operate in a master/slave mode. I'm guessing that the main one that is ignored is shutter speed as that goes direct to the sensor and it appears from the source that I'm only telling one sensor the new value. All the other parameters should be going to the tuner (or JPEG encoder) and should therefore be doing the right thing.

The stereoscopic support was never used in an actual product so there will be these minor issues lurking - just let me know and I'll try to fix them.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

draco
Posts: 18
Joined: Fri Aug 15, 2014 6:42 am
Location: Melbourne, Australia

Re: Stereoscopic camera capture - now implemented.

Wed May 27, 2015 1:49 am

6by9 wrote: That sounds like a bug to me as you shouldn't need to change anything from the app side.
The tuners should operate in a master/slave mode. I'm guessing that the main one that is ignored is shutter speed as that goes direct to the sensor and it appears from the source that I'm only telling one sensor the new value. All the other parameters should be going to the tuner (or JPEG encoder) and should therefore be doing the right thing.
I don't know which parameter applies to the sensor and which to the tuner, so I apologize if I'm asking stupid questions.

I've re-tested without the shutter speed parameter this time (to isolate problem to tuner only parameters), but I'm still getting a difference in image sharpness. The primary image appears sharper than the secondary at 100 sharpness.

ISO, AWB, and metering parameters appear to work as expected. The other parameters are harder to tell if they have been applied to both images.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20064
Joined: Sat Jul 30, 2011 7:41 pm

Re: Stereoscopic camera capture - now implemented.

Wed May 27, 2015 9:16 am

draco wrote:
6by9 wrote: That sounds like a bug to me as you shouldn't need to change anything from the app side.
The tuners should operate in a master/slave mode. I'm guessing that the main one that is ignored is shutter speed as that goes direct to the sensor and it appears from the source that I'm only telling one sensor the new value. All the other parameters should be going to the tuner (or JPEG encoder) and should therefore be doing the right thing.
I don't know which parameter applies to the sensor and which to the tuner, so I apologize if I'm asking stupid questions.

I've re-tested without the shutter speed parameter this time (to isolate problem to tuner only parameters), but I'm still getting a difference in image sharpness. The primary image appears sharper than the secondary at 100 sharpness.

ISO, AWB, and metering parameters appear to work as expected. The other parameters are harder to tell if they have been applied to both images.
Could it be differences in the camera module itself? ave you tried swapping them to see if the issues moves with the camera?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Wed May 27, 2015 9:18 am

draco wrote:I don't know which parameter applies to the sensor and which to the tuner, so I apologize if I'm asking stupid questions.
No problem. More a brain dump from me as a reminder when I come to fix it.
Only exposure time and analogue gain are done in the sensor, everything else is done in the GPU ISP hardware. So that could have meant that ISO wouldn't work, but that does still go via the tuner as it tweaks analogue and digital gain (digital gain being in the ISP). All internals that you shouldn't really need to care about!
draco wrote:I've re-tested without the shutter speed parameter this time (to isolate problem to tuner only parameters), but I'm still getting a difference in image sharpness. The primary image appears sharper than the secondary at 100 sharpness.

ISO, AWB, and metering parameters appear to work as expected. The other parameters are harder to tell if they have been applied to both images.
OK, I'll check sharpness as well if I get a chance. I can't think of a good reason why it wouldn't be reflected to the slave.

If you capture images from the second sensor with sharpness 100 and compare to the stereoscopic output do you get the same output? There is some variation between modules, and it could be that the second sensor isn't focused at quite the same distance as the first one.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

draco
Posts: 18
Joined: Fri Aug 15, 2014 6:42 am
Location: Melbourne, Australia

Re: Stereoscopic camera capture - now implemented.

Thu May 28, 2015 6:56 am

6by9 wrote:If you capture images from the second sensor with sharpness 100 and compare to the stereoscopic output do you get the same output? There is some variation between modules, and it could be that the second sensor isn't focused at quite the same distance as the first one.
I've configured both my cameras to the same focus distance (both are fitted with manual lens), and tested the different permutations of single/stereo camera with/without sharpness 100. I can confirm that sharpness 100 only applies to the primary image.

I've taken out all camera parameters except for sharpness:

Code: Select all

raspistill -sh 100 -q 100 -th none -n -w 5184 -h 1944 -3d sbs -o /tmp/image.jpg

draco
Posts: 18
Joined: Fri Aug 15, 2014 6:42 am
Location: Melbourne, Australia

Re: Stereoscopic camera capture - now implemented.

Wed Jun 03, 2015 2:26 am

Is it possible to have the 2 cameras on different ISO & shutter speed configuration? This is useful when both cameras are not exactly the same, and they need to be fine tuned individually (also helps with my post processing).

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Wed Jun 03, 2015 10:02 am

draco wrote:Is it possible to have the 2 cameras on different ISO & shutter speed configuration? This is useful when both cameras are not exactly the same, and they need to be fine tuned individually (also helps with my post processing).
No, they are deliberately slaved to the same values. The upper layers of the stack do almost nothing different for stereo vs single channel, only knowing how to request the formats, and how to process the output data.

For some projects the sensors were calibrated on the production line to compensate for the slight variation in lens shading and red/blue sensitivities between modules, but that isn't done on the Pi.
Overall sensitivity variation between sensors is normally fairly minimal, though I do remember those setting up the stereoscopic demo rig going through to pick out a couple of "golden" matched sensors.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

afanasjev
Posts: 2
Joined: Sat Jul 25, 2015 12:54 am

Re: Stereoscopic camera capture - now implemented.

Sat Jul 25, 2015 1:24 am

Is it possible to create 3d SBS or TB video with 1280x720 each frame?

raspivid -w 2560 -h 720 -3d sbs -o stereo.264
or
raspivid -w 1280 -h 1440 -3d tb -o stereo.264
always failes, i think due to 1920x1344 framebuffer limit

so there is no way to get 2560x720 sbs video on Pi compute module?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Sat Jul 25, 2015 6:45 am

afanasjev wrote:so there is no way to get 2560x720 sbs video on Pi compute module?
Not as H264. MJPEG should work.
1280x1440 TB may work - the 1344 height switches to a different (slightly more inefficient) image packing, but should work if everything uses the helper functions instead of making assumptions.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

natxopedreira
Posts: 9
Joined: Wed Mar 19, 2014 9:26 am

Re: Stereoscopic camera capture - now implemented.

Mon Aug 24, 2015 10:55 am

Hi

Im using raspivid to stream a stereoscopic SBS video.

Im using 1280x512 resolution at 30fps and works ok, but i can not managed to get more fps. I want to get at least 48fps but as soon as i increase the fps y get jagged edges on the image.

I tried other resolutions like 1280x480 but the same i only use 30fps, if i set more fps i get jagged edges.

Its possible to get a resolution of 1280x512 or similar at a higer framerate?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Mon Aug 24, 2015 7:40 pm

natxopedreira wrote:Hi

Im using raspivid to stream a stereoscopic SBS video.

Im using 1280x512 resolution at 30fps and works ok, but i can not managed to get more fps. I want to get at least 48fps but as soon as i increase the fps y get jagged edges on the image.

I tried other resolutions like 1280x480 but the same i only use 30fps, if i set more fps i get jagged edges.

Its possible to get a resolution of 1280x512 or similar at a higer framerate?
Are you decimating as (ie 640x512 for each image but squashed)?
It sounds like you're getting a sub optimal mode off the sensor and it is having to upscale the image. You can use the "-md X" option to force the sensor mode as the selection code may not deal with the sterescopic side ideally. You probably want md 4 or md 5 (4:3 or 16:9 versions of the same mode) - see elsewhere for the full list of modes.

NB The H264 block is only level 4.0, so 1280x720 @ 60fps max. Without the decimation option you'd be feeding in 2560x512, ah which actually won't work anyway as the width has a max of 2048. I guess that half answers my question above.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

natxopedreira
Posts: 9
Joined: Wed Mar 19, 2014 9:26 am

Re: Stereoscopic camera capture - now implemented.

Tue Aug 25, 2015 11:02 am

Thanks for reply.

I was not using decimation, i tried -dec in the raspivid command line.
raspivid -ih -3d sbs -dec -t 0 -w 1280 -h 480 -fps 48 ........

Now i don't get jagged edges, thanks !!!! but the image seems streched/distorted i don't understand the -dec flag.

Im just trying to stream the stereo video to view in a oculus, so i need to send 640x480 for each eye, so i assume that if i say
-w 1280 -h 480 -dec -3d sbs i will send two 640x480 images in sbs mode....... but seems like its not that size

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Tue Aug 25, 2015 4:18 pm

natxopedreira wrote:Thanks for reply.

I was not using decimation, i tried -dec in the raspivid command line.
raspivid -ih -3d sbs -dec -t 0 -w 1280 -h 480 -fps 48 ........

Now i don't get jagged edges, thanks !!!! but the image seems streched/distorted i don't understand the -dec flag.

Im just trying to stream the stereo video to view in a oculus, so i need to send 640x480 for each eye, so i assume that if i say
-w 1280 -h 480 -dec -3d sbs i will send two 640x480 images in sbs mode....... but seems like its not that size
The decimation option is to match the way many stereoscopic movie clips are rendered - squash the two full images by half either left/right or top/bottom, so the codec still sees the same number of pixels. In your case it sounds like that isn't applicable.

It sounds like you want
raspivid -ih -3d sbs -t 0 -w 1280 -h 480 -fps 48 -md 5
Stealing waveform80's write up of the modes from http://picamera.readthedocs.org/en/rele ... mera-modes, that should force the 1296x730 mode. I suspect that will give a reduced field of view, as the 4:3 version (mode 4, 1296x976) maxes out at 42fps.
I'm guessing that it was dropping into the VGA @30-60fps mode, so fewer pixels in the source image. <=30fps would have used the 1296x976 mode, hence the better quality.

The mode selection algorithm can make some odd mode choices sometimes, but it becomes nearly impossible to get an algorithm to choose the best mode under all circumstances. That's why we added manual selection for those who really want to get down to the nitty gritty.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

natxopedreira
Posts: 9
Joined: Wed Mar 19, 2014 9:26 am

Re: Stereoscopic camera capture - now implemented.

Wed Aug 26, 2015 8:50 am

Thanks for explaining, now i understand a little bit more how it works.
Using the -md flag did the trick, now im sending a stereo video stream and view that on rift..... its very coool

Thanks again!

Russeru
Posts: 1
Joined: Sat Oct 31, 2015 6:04 pm

Re: Stereoscopic camera capture - now implemented.

Sat Oct 31, 2015 6:26 pm

Hi,
I'm trying to do something similar (take two video streams and eventually send them over a network via gstreamer to display on an Oculus Rift), however the 3d option is only displaying one of the camera outputs, instead of both side-by-side.
I tried the command: raspivid -ih -3d sbs -t 0 -w 1280 -h 480 -fps 48 -md 5

Both cameras work individually; if I add the "-cs 1" option it displays video from the other camera.

Any ideas on what could be causing this? I noticed the original post says "You MUST enable stereoscopic mode before enabling the camera component or setting the camera number." Is this something I have to do myself or is it done automatically by the "-3d" option?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Sun Nov 01, 2015 2:11 pm

Russeru wrote:Hi,
I'm trying to do something similar (take two video streams and eventually send them over a network via gstreamer to display on an Oculus Rift), however the 3d option is only displaying one of the camera outputs, instead of both side-by-side.
I tried the command: raspivid -ih -3d sbs -t 0 -w 1280 -h 480 -fps 48 -md 5

Both cameras work individually; if I add the "-cs 1" option it displays video from the other camera.

Any ideas on what could be causing this? I noticed the original post says "You MUST enable stereoscopic mode before enabling the camera component or setting the camera number." Is this something I have to do myself or is it done automatically by the "-3d" option?
Have you switched into a 3D mode with tvservice?
What display modes does the Rift claim to support? (one of the tvservice commands, but can't remember the required options at the moment).
Or are you saying that the display is connected remotely and you're just streaming the paired stream?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

SebK
Posts: 7
Joined: Wed Jun 29, 2016 3:18 pm

Re: Stereoscopic camera capture - now implemented.

Wed Jun 29, 2016 5:07 pm

Hi everybody

I'm searching for help in this thread since I have a hard time getting some features of the stereoscopic mode to work properly.

I am testing the stereoscopic camera features on the Compute Module currently using raspivid. The problem arises if I try to record a Top-Bottom video in half-height mode. I took into account the maximum width and height sizes mentioned here and also the fact the picture size/height needs to be a multiple of 256.

Following code example from the first post in this thread does not work for me (I replaced the -hh option with -dec).

Code: Select all

raspivid -w 1792 -h 1080 -3d tb -dec -o tb_stereo.h264
The recording seems to be running properly (judging from the verbose raspivid output), but the output file is only 29 Bytes in total.
From what I was reading in this thread those width and height sizes should be applicable to a side-by-side recording. Indeed, the following call results in a proper 3d sbs recording.

Code: Select all

raspivid -w 1792 -h 1080 -3d sbs -dec -o sbs_stereo.h264.
I also got a working example for a full height top-bottom recording.

Code: Select all

raspivid -3d tb -w 640 -h 768 -dec -o tb_dec_stereo.h264
Here I took into account that 2*480=960 is not divisable by 256 and reduced it to 3*256=768.

Following this working example and the successful side-by-side decimate recording the following call should be working to.

Code: Select all

raspivid -3d tb -w 1920 -h 1024 -dec -o tb_dec_stereo.h264
But this only resulted in a faulty output file of 29 bytes. Could you give me a hint to where I am wrong about the recording settings? Thank you!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Wed Jun 29, 2016 10:35 pm

The restriction on a multiple of 256 is on the width when doing side by side. When doing top/bottom there shouldn't be an issue.
(The restriction comes from the internal image format being in columns of 128 pixels wide, and the hardware can't start the second image in the middle of a stripe. With top/bottom the second image just starts half way down, but is always at the start of the first column)

29 bytes of H264 is just the stream headers, no actual frames. Something is obviously stalling the pipe pretty comprehensively.

All of your commands look sensible - I'll have to get my Compute Module rig out to check it out.
If you wanted to give me a head start, then add "start_debug=1" to /boot/config.txt, reboot, try your recording and run "sudo vcdbg log assert" afterwards. It's likely to give some indication of what the GPU didn't like.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

SebK
Posts: 7
Joined: Wed Jun 29, 2016 3:18 pm

Re: Stereoscopic camera capture - now implemented.

Thu Jun 30, 2016 7:17 am

Thanks for the quick and detailed reply! I will follow your instructions regarding the debug output and post my results.

Also good to know that the size restriction should only apply to the image width. Strange thing though that I was unable to record a full height top-bottom video in 640x960. That only resulted in a 29 byte output file. I will try this again to confirm.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 3 guests