PiGraham
Posts: 3320
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Stereoscopic camera capture - now implemented.

Wed Nov 19, 2014 5:13 pm

edzieba wrote:
6by9 wrote:Please note that my diffs will NOT provide those command options. My diffs hard code the stereoscopic mode as I was just testing things. Adding them as options isn't difficult so had been left as an exercise for others whilst I had been looking at firmware issues - no one seems to have stepped up to the plate to do that work.
Ah, that's where I'd been going wrong, I though that diffs were to add that option to raspivid/raspistill.
If I'm understanding things correctly, as it stands the only way to produce stereo images or video with the Compute Module is to write a new interface that talks to the (MMAL?) library, pokes the GPU to encode things,, and whatever else is required?
I think you could quite easily adapt the patched raspivid to control the stereo parameters via command line options, such as that referred to in the OP.

edzieba
Posts: 25
Joined: Fri Jul 29, 2011 6:59 pm

Re: Stereoscopic camera capture - now implemented.

Sat Nov 22, 2014 12:04 am

Thanks, removing the errant -3d has got the modified raspistill working as intended. The modified raspivid only appears to be capturing from a single camera though, so I'm going over the code to check the diff applied correctly.
::EDIT:: Yup, diff misapplied.

dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Wed Dec 03, 2014 7:43 pm

Hello,
I have a question about stereo camera performance on the pi compute module.
We are currently using a pi to record videos at qvga (320x240) with 90fps. Is it possible to record stereo images at this framerate?
What are the supported / tested modes with stereo camera mode?
Best regards,
Damian

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20064
Joined: Sat Jul 30, 2011 7:41 pm

Re: Stereoscopic camera capture - now implemented.

Wed Dec 03, 2014 8:02 pm

I have a feeling that has never ever been tried. I suspect it would be OK though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Wed Dec 03, 2014 8:37 pm

jamesh: Correct (again!) - never tested but should be fine.

Please note all the rules in the first post about resolution, particularly
The hardware H264 block has an internal line buffer that will limit you at a maximum of 1920 wide. Please do not exceed that. There is another limitation on the frame height at 1344, so I'm afraid that also means that 720P top/bottom full frame is also out. The width of each image must also be a multiple of 128 (overall width a multiple of 256), so 1920x1080 doesn't work (1920/2 = 960. /128 = 7.5) . raspivid command will be something like raspivid -w 1792 -h 1080 -3d tb -hh -o stereo.264.
QVGA fails that criteria, so bad things will happen if you try side-by-side. Top/bottom should be fine, and you will probably get away without having to decimate either as the overall pixel rate should be low enough.
The main testing was at the higher resolutions as those stress the hardware more.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Mon Dec 08, 2014 9:24 pm

Thank you for your answers. Sounds very promising!
I just noticed, the Pi supports [email protected] Am I right assuming that dual camera with [email protected] is possible?
-> sidebyside: 2*640 width = 1280 (is a multiple of 256)
Pi is able to h264 encode [email protected]: 1920*1080*30 = 62208000 pixel/sec
[email protected]: 640*480*2*90 = 55296000 pixel/sec
-> h264 encoder should be able to handle data rate
This mode would enable many additional scenarios for realtime "high speed" image processing, eyetracking in our case.
I really would appreciate if someone out there with a appropriate hardware setup would test this configuration and report the result.
the command should be more or less the following one: raspivid -w 1280 -h 480 -3d sbs -hh -fps 90 -o stereo90fps.264

Best regards,
Damian

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Mon Dec 08, 2014 9:37 pm

dweber wrote:I just noticed, the Pi supports [email protected] Am I right assuming that dual camera with [email protected] is possible?
-> sidebyside: 2*640 width = 1280 (is a multiple of 256)
Pi is able to h264 encode [email protected]: 1920*1080*30 = 62208000 pixel/sec
[email protected]: 640*480*2*90 = 55296000 pixel/sec
-> h264 encoder should be able to handle data rate
This mode would enable many additional scenarios for realtime "high speed" image processing, eyetracking in our case.
I really would appreciate if someone out there with a appropriate hardware setup would test this configuration and report the result.
the command should be more or less the following one: raspivid -w 1280 -h 480 -3d sbs -hh -fps 90 -o stereo90fps.264
Sounds plausible, but I haven't tried it or got hardware to try it.
Please note that raspivid has NOT been updated for actually handling stereoscopic. There are diffs listed that enabled me to test the feature, but it has not been integrated into the app.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5171
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Stereoscopic camera capture - now implemented.

Fri Dec 12, 2014 6:31 pm

I had a go at adding the options to raspicam, but ran into some trouble. While I was at it, I moved the camera selection code outside the individual binaries into raspicam_control, then changed thing so that the camera component only gets enabled after the parameters have been applied. The code seems to have everything in place, but mmal complains about invalid arguments. Might need to dive into the firmware to see what happens there.

Bmohit
Posts: 6
Joined: Mon Dec 29, 2014 6:04 pm

Re: Stereoscopic camera capture - now implemented.

Mon Jan 05, 2015 5:45 pm

Hello everybody..
Trying to understand this thread from a couple of days.
Please clear my doubts.
1.I have doubt in first post itself. Is MMAL API a file or something which needs to be edited for stereo support?

2.which git repo should I clone exactly to enable stereo support?
Help is really appreciated.
Thank you.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5171
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Stereoscopic camera capture - now implemented.

Mon Jan 05, 2015 5:49 pm

Bmohit wrote:Hello everybody..
Trying to understand this thread from a couple of days.
Please clear my doubts.
1.I have doubt in first post itself. Is MMAL API a file or something which needs to be edited for stereo support?

2.which git repo should I clone exactly to enable stereo support?
Help is really appreciated.
Thank you.
The MMAL API is not a file, it's the API.
http://en.wikipedia.org/wiki/Applicatio ... _interface

The thread is about how to use the new functionality using the API.

The git repo to clone will depend on where you want to enable stereo support. Most people are interested in adding stereo mode to raspivid/raspistill, so the repo to clone for that is 'userland'.

Bmohit
Posts: 6
Joined: Mon Dec 29, 2014 6:04 pm

Re: Stereoscopic camera capture - now implemented.

Mon Jan 05, 2015 6:04 pm

Thanks alot..
I will clone

Code: Select all

'git://github.com/raspberrypi/userland.git'
by

Code: Select all

git clone --depth=1 https://github.com/raspberrypi/userland.git.
then 'buildme' procedure described on page one of this thread.
then pins connections as described in http://www.raspberrypi.org/documentatio ... -camera.md
and then raspistill command for capturing the stereo pair of images??m I thinking the correct way?

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5171
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Stereoscopic camera capture - now implemented.

Mon Jan 05, 2015 6:06 pm

I think you'll have to get frustrated and pull some hair out first while you figure it all out, but yes, that's the idea. I'll work on making all of this easier when the todo list clear up a little.

Bmohit
Posts: 6
Joined: Mon Dec 29, 2014 6:04 pm

Re: Stereoscopic camera capture - now implemented.

Mon Jan 05, 2015 6:15 pm

:? Frustration and hair pulling thing is happening right now.. particularly while searching for mmal-api. Presently reading the link you provided. Thanks alot for the support.
Desperately waiting for the simplified 'to-do' list. :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Mon Jan 05, 2015 6:23 pm

MMAL API headers are all in https://github.com/raspberrypi/userland ... rface/mmal
There is inline documentation within the headers which I recall someone has correctly extracted and stuck on a web page somewhere.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Thu Feb 19, 2015 3:45 pm

Are there any news on this topic?

Is somebody working on implementing the stereo mode in raspivid?

Best regards,
Damian

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5171
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 2:12 am

I got side-tracked with other things. It's not on top of the todo list for me right now, but of course anybody can send a pull request.

dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 2:38 pm

I work on a project where we need two cameras read out at the same time. We decided to implement it on a Pi Compute Module.
As written above in this thread, we will try to get stereo [email protected] working.

Is this thread the only information on how to get stereoscopic camera working?

Best regards,
Damian

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 3:12 pm

dweber wrote:Is this thread the only information on how to get stereoscopic camera working?
Pretty much.
I am still lurking and will provide limited support where needed - I did claim a Compute Module and pair of cameras off gsh a while ago with the intent of adding dual camera support to the V4L2 driver, so can test stuff out if things don't seem to be working (post test code to github or similar).
(I wasn't intending to add stereo support to V4L2 as the standard API didn't support it when I last looked)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informx
Posts: 5
Joined: Sat Dec 06, 2014 1:27 am

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 6:00 pm

dweber wrote: Is this thread the only information on how to get stereoscopic camera working?
No, there is one too:

http://www.raspberrypi.org/forums/viewt ... 98&t=90603

and

http://www.linux-projects.org/modules/s ... #example12

http://www.linux-projects.org/modules/s ... #example13

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 6:05 pm

Which tell you how to use UV4L (closed source project) to use the stereoscopic support. They all omit all the "rules" that I list at the top of this thread - be warned.

The only detail of the MMAL implementation is in this thread. It all depends on what level you want to program the system at.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

informx
Posts: 5
Joined: Sat Dec 06, 2014 1:27 am

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 6:23 pm

6by9 wrote: Which tell you how to use UV4L (closed source project) to use the stereoscopic support. They all omit all the "rules" that I list at the top of this thread - be warned.
The dual cameras can be programmed via the Video4Linux2 interface so the implementation rules you are talking about in this thread should be already present internally in their driver implementation and not are visible to the user - as it should be. This is enough to get the devices working as you prefer and answers what the op asked.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 9:18 pm

informx wrote:
6by9 wrote: Which tell you how to use UV4L (closed source project) to use the stereoscopic support. They all omit all the "rules" that I list at the top of this thread - be warned.
The dual cameras can be programmed via the Video4Linux2 interface so the implementation rules you are talking about in this thread should be already present internally in their driver implementation and not are visible to the user - as it should be. This is enough to get the devices working as you prefer and answers what the op asked.
<slightly grumpy response>
Hi RPiName or friend. I say that as you are obviously close to UV4L2. Also that was a very knowledgeable response as a 3rd post to the forum.

I see you have advertised VIDIOC_ENUM_FRAMESIZES that H264 does "Stepwise 256x64 - 3584x1088 with step 256/8" - nicely done.

Shame you don't enforce it via VIDIOC_S_FMT or VIDIOC_TRY_FMT

Code: Select all

[email protected] ~ $ v4l2-ctl -v width=352,height=288,pixelformat=H264
[email protected] ~ $ v4l2-ctl -V
Format Video Capture:
        Width/Height  : 352/288
        Pixel Format  : 'H264'
        Field         : None
        Bytes per Line: 352
        Size Image    : 262144
        Colorspace    : Unknown (00000000)
[email protected] ~ $ v4l2-ctl -v width=1920,height=1080,pixelformat=H264
[email protected] ~ $ v4l2-ctl -V
Format Video Capture:
        Width/Height  : 1920/1080
        Pixel Format  : 'H264'
        Field         : None
        Bytes per Line: 1920
        Size Image    : 262144
        Colorspace    : Unknown (00000000)
Oops!
My warning to obey the rules therefore is useful. It also gives an explanation for those who try something incompatible and wonder why it doesn't work or they get different settings to those they requested.

The post that also re-awoke this thread was http://www.raspberrypi.org/forums/viewt ... 25#p699408
Postby dweber » Thu Feb 19, 2015 3:45 pm
Are there any news on this topic?

Is somebody working on implementing the stereo mode in raspivid?

Best regards,
Damian
Your answer doesn't relate to raspivid. The followup asked for any extra information on implementing stereoscopic - I was taking that in context of the first question, which meant your response didn't help.
</slightly grumpy response>
Support still offered for working via MMAL (seeing as I've just setup my Compute Module, I'll have a look to see if i can get stereo support in raspivid done now)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dweber
Posts: 22
Joined: Wed Dec 03, 2014 7:35 pm

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 9:40 pm

Nice. Thank you both for the information provided

RpiName
Posts: 663
Joined: Sat Jul 06, 2013 3:14 am

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 10:11 pm

6by9 wrote:Hi RPiName or friend. I say that as you are obviously close to UV4L2. Also that was a very knowledgeable response as a 3rd post to the forum.
I think there are >50k users of UV4L now and I would not be surprised to find some expert developer among them posting in this forum.. I am one of them.
6by9 wrote: Shame you don't enforce it via VIDIOC_S_FMT or VIDIOC_TRY_FMT

Code: Select all

[email protected] ~ $ v4l2-ctl -v width=352,height=288,pixelformat=H264
[email protected] ~ $ v4l2-ctl -V
Indeed it looks like a bug. The first one I have heard of. Nothing is perfect. Thanks for reporting. I am sure the authors will promptly provide the fix.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5365
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture - now implemented.

Fri Feb 20, 2015 11:31 pm

dweber: I've just pushed an update to https://github.com/6by9/userland which should add the stereoscopic support to raspivid and raspistill (I haven't looked at the YUV apps).
New options are
-3d [none|sbs|tb]
-dec
-3dswap
For the stereoscopic mode, decimation, and swapping the sensors respectively.

I haven't tested it greatly, mainly as I don't have a 3D TV, but a quick test taking a JPEG and video recording both seem to imply the correct things are happening. Hopefully it'll help you out. I certainly get something looking vaguely sane from

Code: Select all

 raspivid -w 1280 -h 480 -3d sbs -o wibble.h264 -t 10000 -fps 90
I haven't investigated in detail, but I appear to have only got 608 video packets out of that 10seconds though, so there may be something odd happening shifting up from 60fps to 90. I'm jut picking up ethanol100's change which logs the timestamps to get some more accurate numbers.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 3 guests