Stereoscopic Raspberry Pi Camera ISSUE!

[husamwadi]

Hi there!

I want to make 2 video capture previews (live stream) side by side kind of like this:

http://postimg.org/image/kmn70tnyn/

Using the Raspberry Pi Camera through the CSi port.

I have tried literally just about everything. My first idea was to use Processing 2.1 and run a sketch on the raspberry pi, but it didn't really work out (used too much processing power, GSCapture and GStreamer library issues).

I then tried the Picamera 0.8 plugin for Python and tried to code a simple program for stereoscopic vision.

I tried grabbing 2 instances of the picamera:

camera = picamera.Picamera()
camera2 = picamera.Picamera()

then running 2 windows side by side, but I got an error saying it can't have 2 instances of Picamera running.

Then I tried running 2 python programs at once using subprocesses and my raspberry pi blew up (figuratively).

Is there anyway to copy the video preview (copy pixels) or something so that I can have 2 camera previews side by side?

The most important thing is high framerate/low latency. I can't have the camera running at 4fps or 10fps, it has to be minimum 30fps (or I would have used a webcam).

I am thinking about SimpleCV/OpenCV or OpenFrameworks and try from there, but I am literally about to pull out my hairs on this haha, this is SO frustrating!

If you have any advice or maybe a way to do it through python (best bet) that would be awesome!

Thank you !

I attached my sketches to this comment (processing, python) if you need to take a look.

Update:

I just need the camera output duplicated from the Raspicam. I don't really need stereoscopic 3d, just a side by side duplicated video from the CSi camera, thanks!

[jamesh]

I'm confused. You cannot attach two cameras to a single Raspi. So stereo isn't possible. Or do you just want to duplicate the same image side by side in one JPG output file?

[mikerr]

(Stating the obvious)

For 3d stereoscopic image you need a slightly different viewing angle for each image half.
Otherwise it won't be 3d !

You could do that with a single pi camera if you moved the camera back and forth (e.g. on a servo) between each shot.
Speed of the servo and raspistill would mean that would only get you 1 fps,
using a rotating mirror might get you faster fps there

Other better options
2 Pi's with a raspicamera each and connected via network
or 1 pi with 1 usb webcam and 1 raspicam

[husamwadi]

Sorry, what I meant is to duplicate just the camera output twice.

I am not concerned for it to be 3d, I just need a single output from the CSi camera preview doubled side by side.

But its been 2 weeks and I still haven't found a way to do it.

If someone knows a way or can point me in a direction that would be awesome.

[jamesh]

I have no idea whether this would work, but you might be able to create a camera component, run the preview output through the splitter component and the output from those in to two preview components.

Someone on here had some success with a splitter, and I see no reason why you cannot create two previews.

Will require you to write some code though. Use raspivid and modify it appropriately.

EDIT: Or perhaps use the two ports on the camera component and direct to two previews....that might be easier.

[gordon77]

One way to do it is to save the camera picture , open in python into a string, read every other element ( remember there will be rgb pixels for each element)save each line into a new string twice, and save as a new picture

[husamwadi]

So I can have 2 devices attached to the port?

that would be useful, do you know where I can find the splitter?

But I was thinking it would be easier to copy the pixels to the other side of the screen somehow.

[husamwadi]

I need it to be a live video stream, I looked up "python copy pixels" but they were all for pictures

[gordon77]

Oops missed the video requirement

[husamwadi]

Any suggestions?

[DougieLawson]

Any suggestions?

Two RPis, two raspicams and some software to timecode & synchronise the video streams from each camera.

[lagurus]

Sorry, what I meant is to duplicate just the camera output twice.

I am not concerned for it to be 3d, I just need a single output from the CSi camera preview doubled side by side.

But its been 2 weeks and I still haven't found a way to do it.

If someone knows a way or can point me in a direction that would be awesome.

You could also modify raspistill which can use OpenGL for render preview. Then create new shader which split
video into two.

[mhelin]

Best way to get stereoscopic 3D video or photo working on Pi is to make a new camera for Raspebby Pi. Omnivision OV5642 sensor for an example (also OV5645: http://www.ovt.com/products/sensor.php?id=126 ) has also a MIPI RX port, and it can also receive image data from another sensor via DVP. Ideally the two sensors would be mounted on adjustable mount to be able to set the stereo base according to distance to object (30:1 is the most commonly used rule). Also it would be better to be able to change the lenses and so use the CS mount for an example.

Another interesting sensor would be the OV5680 (http://www.ovt.com/products/sensor.php?id=109) which "can synchronize exposure and frame for stereo cameras", though this one needs two MIPI ports (might be possible to get working with two RPis).

MT0V024 sensor also has a slave video interface for stereo applications:
http://s1.archive.theimagingsource.com/ ... .en_US.pdf

It's used in this stereo camera reference design:
http://www.e-consystems.com/stereo-vision-camera.asp

However, if you want cheap stereo camera pick a car dash camera from ebay, they are less than $50 and some models have two sensors which can be adjusted 180 degrees to point parallel to the same direction, and the camera can make an SBS video and store it to SD card. Here's one example:

http://www.ebay.co.uk/itm/HD-X8000C-Dua ... 1e857b0800

There are also ones with USB output so you could use them with RPi:

http://www.ebay.co.uk/itm/New-2-0-TFT-H ... 3f1c014711

[husamwadi]

Wow thanks, didn't know about the dashcams

If I can bypass using a raspberry pi at all that would be great, just one less thing I have to worry about.

But I am not sure I can hack the dashcams enough to fit my purpose (live streaming, low latency, output to hdmi or av)

I will see though.

Also, where do I buy a OV5680? is that a commercial product or just a schematic?

[husamwadi]

Woah man, you're talking about creating my own camera? That's crazy but I like the idea.

So I would just get a camera, interface it to a sensor then output it to a microcontroller (arduino mega) and then output that to av or hdmi?

Seems really intense for the goggles I am making, but I did take 3 years electronics, maybe I can brush up and learn something.

[jamesh]

Note that you cannot just plug any old camera in to the Raspi - it needs a specific driver in the GPU. In the Raspi case the driver is for the 5647 only.

[RaTTuS]

why do you need 2 previews running - what is it that you are trying to achieve?

[recantha2]

Thinking about your problem, I was just thinking how to do a stereoscopic camera... The only way, really, if you want decent FPS using the Pi camera module is to have two cameras, and therefore two Pis. You'd need to figure out some way to sync them so that the two video streams/photographs were displayed at the same time, together. That means you'd have a bit of a delay, but I'm sure that's not going to bother you if you're taking stereo stills. If you're interested in this concept, have a word with David Whale (@whaleygeek on Twitter) about his Pi-to-Pi networking workshop materials. He's got it set-up so that you program one Pi as a server, one Pi as a client and then establishes a chat session between the two. You could easily convert this into a take-a-still-now trigger.

[husamwadi]

I'm making it for the Oculus rift, It doesn't work with just 1 image. It's for a project involving different live perspectives, but since the Oculus is stereoscopic, the video output has to be stereoscopic.

Also, you lose the argumented reality feel if the quality is too low or the framerate too low, it has to be high framerate (60fps or 90fps optimal, 30fps minimum) and HD (720p works fine, and even 480p is doable).

I've butt heads for a long time with the raspberry pi, if ONLY it had another CSi connector lol.

Oh well, thanks to the previous post I will go with the dashcam, the raspberry pi was fun and interesting, but not what I need in my timeframe (too much troubleshooting and driver issues).

If someone does find a way to duplicate the preview please let me know, It would be nice for future projects (or even this one)!

Thanks!

[mikerr]

You could also do it at the lens level, there are a few 3d convertor/splitter lenses around:

e.g
http://www.amazon.co.uk/Panasonic-Inter ... B004445O3G

[SB2020]

I was reading this thread thinking the same thing as Mikerr says above - use a mechanical method such as prisms and or mirrors to get stereo images from the sensor. Something I have been meaning to try out myself when time allows...
Good overview here: http://www.lhup.edu/~dsimanek/3d/stereo/3dgallery16.htm

[toxibunny]

Best solution would be for the raspi to have side-by-side compatible screen modes, as a setting in config.txt. I'm pretty sure only the foundation has the power to make that happen though.

Is it possible to output the video to an opengles texture? It would be then /relatively/ easy to display 2 copies on screen, and maybe even implement the oculus' lens distortion correction shaders...

[jamesh]

Best solution would be for the raspi to have side-by-side compatible screen modes, as a setting in config.txt. I'm pretty sure only the foundation has the power to make that happen though.

Is it possible to output the video to an opengles texture? It would be then /relatively/ easy to display 2 copies on screen, and maybe even implement the oculus' lens distortion correction shaders...

Yes, a recent change to raspivid has an option to do this (and some nice teapots to map it on to)

EDIT: It's actually raspitex you want look to for the code, which is included in raspistill

raspistill -g -gs teapot -t 0

[husamwadi]

Actually the prisms was the first thing I tried 1 year ago,

They are nice, but very hard to setup and very touchy. I wanted a way to do it programmatically for more accuracy.

The lens are also an option, but the price tag on those things are ridiculous.

I will check up on raspitex, thanks!

[husamwadi]

Here is the paper on prisms and mirrors that I based my first stereoscopic system on:

http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf

It is amazing how they set it up, the first system I've seen that doesn't take a large amount of space.

But perhaps I can talk with the Raspberry Pi people to give "Oculus Rift" support by allowing 2 previews or a duplicate preview of the camera.