husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 11:38 am

Hi there!

I want to make 2 video capture previews (live stream) side by side kind of like this:

http://postimg.org/image/kmn70tnyn/

Using the Raspberry Pi Camera through the CSi port.

I have tried literally just about everything. My first idea was to use Processing 2.1 and run a sketch on the raspberry pi, but it didn't really work out (used too much processing power, GSCapture and GStreamer library issues).

I then tried the Picamera 0.8 plugin for Python and tried to code a simple program for stereoscopic vision.

I tried grabbing 2 instances of the picamera:

camera = picamera.Picamera()
camera2 = picamera.Picamera()

then running 2 windows side by side, but I got an error saying it can't have 2 instances of Picamera running.

Then I tried running 2 python programs at once using subprocesses and my raspberry pi blew up (figuratively).

Is there anyway to copy the video preview (copy pixels) or something so that I can have 2 camera previews side by side?

The most important thing is high framerate/low latency. I can't have the camera running at 4fps or 10fps, it has to be minimum 30fps (or I would have used a webcam).

I am thinking about SimpleCV/OpenCV or OpenFrameworks and try from there, but I am literally about to pull out my hairs on this haha, this is SO frustrating!

If you have any advice or maybe a way to do it through python (best bet) that would be awesome!

Thank you :D !

I attached my sketches to this comment (processing, python) if you need to take a look.

Update:

I just need the camera output duplicated from the Raspicam. I don't really need stereoscopic 3d, just a side by side duplicated video from the CSi camera, thanks!
Attachments
Stereoscopic.zip
(1.15 KiB) Downloaded 258 times
Last edited by husamwadi on Tue Jan 07, 2014 6:10 pm, edited 1 time in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24162
Joined: Sat Jul 30, 2011 7:41 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 11:45 am

I'm confused. You cannot attach two cameras to a single Raspi. So stereo isn't possible. Or do you just want to duplicate the same image side by side in one JPG output file?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

mikerr
Posts: 2789
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 4:09 pm

(Stating the obvious)

For 3d stereoscopic image you need a slightly different viewing angle for each image half.
Otherwise it won't be 3d !

You could do that with a single pi camera if you moved the camera back and forth (e.g. on a servo) between each shot.
Speed of the servo and raspistill would mean that would only get you 1 fps,
using a rotating mirror might get you faster fps there

Other better options
2 Pi's with a raspicamera each and connected via network
or 1 pi with 1 usb webcam and 1 raspicam

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 6:05 pm

Sorry, what I meant is to duplicate just the camera output twice.

I am not concerned for it to be 3d, I just need a single output from the CSi camera preview doubled side by side.

But its been 2 weeks and I still haven't found a way to do it.

If someone knows a way or can point me in a direction that would be awesome.
Last edited by husamwadi on Tue Jan 07, 2014 6:12 pm, edited 1 time in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24162
Joined: Sat Jul 30, 2011 7:41 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 6:11 pm

I have no idea whether this would work, but you might be able to create a camera component, run the preview output through the splitter component and the output from those in to two preview components.

Someone on here had some success with a splitter, and I see no reason why you cannot create two previews.

Will require you to write some code though. Use raspivid and modify it appropriately.

EDIT: Or perhaps use the two ports on the camera component and direct to two previews....that might be easier.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

gordon77
Posts: 4303
Joined: Sun Aug 05, 2012 3:12 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 6:14 pm

One way to do it is to save the camera picture , open in python into a string, read every other element ( remember there will be rgb pixels for each element)save each line into a new string twice, and save as a new picture

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 6:15 pm

So I can have 2 devices attached to the port?

that would be useful, do you know where I can find the splitter?

But I was thinking it would be easier to copy the pixels to the other side of the screen somehow.

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 6:16 pm

I need it to be a live video stream, I looked up "python copy pixels" but they were all for pictures :?

gordon77
Posts: 4303
Joined: Sun Aug 05, 2012 3:12 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 6:27 pm

Oops missed the video requirement

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 8:18 pm

Any suggestions?

User avatar
DougieLawson
Posts: 36572
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 8:57 pm

husamwadi wrote:Any suggestions?
Two RPis, two raspicams and some software to timecode & synchronise the video streams from each camera.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

lagurus
Posts: 46
Joined: Wed Aug 07, 2013 8:02 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 9:12 pm

husamwadi wrote:Sorry, what I meant is to duplicate just the camera output twice.

I am not concerned for it to be 3d, I just need a single output from the CSi camera preview doubled side by side.

But its been 2 weeks and I still haven't found a way to do it.

If someone knows a way or can point me in a direction that would be awesome.
You could also modify raspistill which can use OpenGL for render preview. Then create new shader which split
video into two.

mhelin
Posts: 127
Joined: Wed Oct 17, 2012 7:18 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Tue Jan 07, 2014 9:20 pm

Best way to get stereoscopic 3D video or photo working on Pi is to make a new camera for Raspebby Pi. Omnivision OV5642 sensor for an example (also OV5645: http://www.ovt.com/products/sensor.php?id=126 ) has also a MIPI RX port, and it can also receive image data from another sensor via DVP. Ideally the two sensors would be mounted on adjustable mount to be able to set the stereo base according to distance to object (30:1 is the most commonly used rule). Also it would be better to be able to change the lenses and so use the CS mount for an example.

Another interesting sensor would be the OV5680 (http://www.ovt.com/products/sensor.php?id=109) which "can synchronize exposure and frame for stereo cameras", though this one needs two MIPI ports (might be possible to get working with two RPis).

MT0V024 sensor also has a slave video interface for stereo applications:
http://s1.archive.theimagingsource.com/ ... .en_US.pdf

It's used in this stereo camera reference design:
http://www.e-consystems.com/stereo-vision-camera.asp

However, if you want cheap stereo camera pick a car dash camera from ebay, they are less than $50 and some models have two sensors which can be adjusted 180 degrees to point parallel to the same direction, and the camera can make an SBS video and store it to SD card. Here's one example:

http://www.ebay.co.uk/itm/HD-X8000C-Dua ... 1e857b0800

There are also ones with USB output so you could use them with RPi:

http://www.ebay.co.uk/itm/New-2-0-TFT-H ... 3f1c014711

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 5:57 am

Wow thanks, didn't know about the dashcams

If I can bypass using a raspberry pi at all that would be great, just one less thing I have to worry about.

But I am not sure I can hack the dashcams enough to fit my purpose (live streaming, low latency, output to hdmi or av)

I will see though.

Also, where do I buy a OV5680? is that a commercial product or just a schematic?

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 6:00 am

Woah man, you're talking about creating my own camera? That's crazy but I like the idea.

So I would just get a camera, interface it to a sensor then output it to a microcontroller (arduino mega) and then output that to av or hdmi?

Seems really intense for the goggles I am making, but I did take 3 years electronics, maybe I can brush up and learn something.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24162
Joined: Sat Jul 30, 2011 7:41 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 9:59 am

Note that you cannot just plug any old camera in to the Raspi - it needs a specific driver in the GPU. In the Raspi case the driver is for the 5647 only.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
RaTTuS
Posts: 10501
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 10:10 am

why do you need 2 previews running - what is it that you are trying to achieve?
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

User avatar
recantha2
Posts: 287
Joined: Wed Nov 14, 2012 9:34 am
Location: Potton, Bedfordshire
Contact: Website Facebook Twitter

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 10:29 am

Thinking about your problem, I was just thinking how to do a stereoscopic camera... The only way, really, if you want decent FPS using the Pi camera module is to have two cameras, and therefore two Pis. You'd need to figure out some way to sync them so that the two video streams/photographs were displayed at the same time, together. That means you'd have a bit of a delay, but I'm sure that's not going to bother you if you're taking stereo stills. If you're interested in this concept, have a word with David Whale (@whaleygeek on Twitter) about his Pi-to-Pi networking workshop materials. He's got it set-up so that you program one Pi as a server, one Pi as a client and then establishes a chat session between the two. You could easily convert this into a take-a-still-now trigger.
--
Michael Horne - @recantha
Raspberry Pi blog - http://www.recantha.co.uk/blog

Cambridge Raspberry Jam
Website: http://camjam.me
Facebook: https://www.facebook.com/cambridgeraspberryjam
Follow the Cambridge Raspberry Jam on Twitter - @cambridgejam

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 10:30 am

I'm making it for the Oculus rift, It doesn't work with just 1 image. It's for a project involving different live perspectives, but since the Oculus is stereoscopic, the video output has to be stereoscopic.

Also, you lose the argumented reality feel if the quality is too low or the framerate too low, it has to be high framerate (60fps or 90fps optimal, 30fps minimum) and HD (720p works fine, and even 480p is doable).

I've butt heads for a long time with the raspberry pi, if ONLY it had another CSi connector lol.

Oh well, thanks to the previous post I will go with the dashcam, the raspberry pi was fun and interesting, but not what I need in my timeframe (too much troubleshooting and driver issues).

If someone does find a way to duplicate the preview please let me know, It would be nice for future projects (or even this one)!

Thanks!

mikerr
Posts: 2789
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 10:56 am

You could also do it at the lens level, there are a few 3d convertor/splitter lenses around:

e.g
http://www.amazon.co.uk/Panasonic-Inter ... B004445O3G

SB2020
Posts: 10
Joined: Thu Sep 06, 2012 1:34 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 4:07 pm

I was reading this thread thinking the same thing as Mikerr says above - use a mechanical method such as prisms and or mirrors to get stereo images from the sensor. Something I have been meaning to try out myself when time allows...
Good overview here: http://www.lhup.edu/~dsimanek/3d/stereo/3dgallery16.htm

toxibunny
Posts: 1382
Joined: Thu Aug 18, 2011 9:21 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 4:21 pm

Best solution would be for the raspi to have side-by-side compatible screen modes, as a setting in config.txt. I'm pretty sure only the foundation has the power to make that happen though.

Is it possible to output the video to an opengles texture? It would be then /relatively/ easy to display 2 copies on screen, and maybe even implement the oculus' lens distortion correction shaders...
note: I may or may not know what I'm talking about...

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24162
Joined: Sat Jul 30, 2011 7:41 pm

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 4:40 pm

toxibunny wrote:Best solution would be for the raspi to have side-by-side compatible screen modes, as a setting in config.txt. I'm pretty sure only the foundation has the power to make that happen though.

Is it possible to output the video to an opengles texture? It would be then /relatively/ easy to display 2 copies on screen, and maybe even implement the oculus' lens distortion correction shaders...
Yes, a recent change to raspivid has an option to do this (and some nice teapots to map it on to)

EDIT: It's actually raspitex you want look to for the code, which is included in raspistill

raspistill -g -gs teapot -t 0
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 6:10 pm

Actually the prisms was the first thing I tried 1 year ago,

They are nice, but very hard to setup and very touchy. I wanted a way to do it programmatically for more accuracy.

The lens are also an option, but the price tag on those things are ridiculous.

I will check up on raspitex, thanks!
Last edited by husamwadi on Wed Jan 08, 2014 6:22 pm, edited 1 time in total.

husamwadi
Posts: 18
Joined: Mon Jan 06, 2014 7:54 am

Re: Stereoscopic Raspberry Pi Camera ISSUE!

Wed Jan 08, 2014 6:13 pm

Here is the paper on prisms and mirrors that I based my first stereoscopic system on:

http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf

It is amazing how they set it up, the first system I've seen that doesn't take a large amount of space.

But perhaps I can talk with the Raspberry Pi people to give "Oculus Rift" support by allowing 2 previews or a duplicate preview of the camera.

Return to “Camera board”