worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Stereo camera PI processing

Thu Nov 21, 2019 4:40 pm

Hello. Please tell me how to do the following:
I have two cameras that are connected to the Raspberry PI.
I need to get an original size image from each camera. For example, 1024 * 1024.
Then cut the fragment size 640 * 480.
Mirror these two fragments and merge them into one image of 1280 * 480 size.
Then stream the stream through rtsp to the network.
Compression required h264. FPS 30.

I read that raspivid can set the region of interest and work with 2 cameras simultaneously in stereo.
But can he process the frames individually to solve my problem?
Please help with the solution. Thanks.

User avatar
mahjongg
Forum Moderator
Forum Moderator
Posts: 12400
Joined: Sun Mar 11, 2012 12:19 am
Location: South Holland, The Netherlands

Re: Stereo camera PI processing

Thu Nov 21, 2019 6:41 pm

Compute modules have two camera inputs, and software exists to combine the two camera's for stereo vision, and split screen display of both camera's

Using regular RPI's with two camera's is much harder, and involves a camera multiplexer, and the above software doesn't work.

User avatar
HermannSW
Posts: 1651
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Stereo camera PI processing

Thu Nov 21, 2019 7:26 pm

There is a way to connect two cameras to a single PI, can be any two Arducam cameras, or Raspberry v1/v2 cameras:
https://www.raspberrypi.org/forums/view ... 8#p1551503
Image
worldmaster wrote:
Thu Nov 21, 2019 4:40 pm
Then cut the fragment size 640 * 480.
Mirror these two fragments and merge them into one image of 1280 * 480 size.
The 2MP monochrome global shutter camera has a 640x400 mode.
That would give you 1280x400 size image without any work.
Above image is 2560x800 (two 1280x800 2MP images side-by-side).

P.S:
Scaled down image allows to see 3D image without any viewer (on 15" laptop display).
Just place hand vertically in the middle and make both images overlay:
Image
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 2:19 pm

I do not need the camera mode 640 * 480. I need to cut a fragment from each camera. A fragment can be anywhere in sight.

gordon77
Posts: 4293
Joined: Sun Aug 05, 2012 3:12 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 3:10 pm

"I have two cameras that are connected to the Raspberry PI."

What are these 2 cameras ?

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 4:03 pm

gordon77 wrote:
Sun Nov 24, 2019 3:10 pm
"I have two cameras that are connected to the Raspberry PI."

What are these 2 cameras ?
https://onpad.ru/catalog/cubie/raspberr ... /1864.html
Производитель: WaveShare
Поддерживает: Raspberry PI A+/B+/PI2
Сенсор: 5 мегапиксельный сенсор OV5647
Размер матрицы: 1/4
Диафрагма (F): 2,8
Фокусное расстояние: 3.37 mm
Диагональ: 72,4 градусов
Разрешение: 1080p
- 2592 × 1944 Разрешение изображения
Размеры: 25 х 24 мм х 9 мм
Поддерживает запись видео: 1080p30, 720p60 and 640x480p60/90

gordon77
Posts: 4293
Joined: Sun Aug 05, 2012 3:12 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 4:05 pm

How are they connected?

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 4:24 pm

gordon77 wrote:
Sun Nov 24, 2019 4:05 pm
How are they connected?
Stereopi
https://www.crowdsupply.com/virt2real/stereopi

User avatar
HermannSW
Posts: 1651
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Stereo camera PI processing

Sun Nov 24, 2019 5:54 pm

worldmaster wrote:
Sun Nov 24, 2019 2:19 pm
I do not need the camera mode 640 * 480. I need to cut a fragment from each camera. A fragment can be anywhere in sight.
Then using stereo hat with v1 camera is what can do that.
The stereo hat hides the fact that two cameras are connected.
You would use raspivid and it will think it takes a 5MP image.
In that image the left half will be from left camera, and right half from right.
Then just use raspivid's "--roi" option to specify region of interest.
Of course that does not make sense if you already use StereoPi.
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

gordon77
Posts: 4293
Joined: Sun Aug 05, 2012 3:12 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 6:00 pm

The stereopi site mentions using raspivid,

raspivid -3d sbs -w 1280 -h 480 -o 1.h264

can you use - roi with that?

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 7:24 pm

I repeat, I need fragments from each of the cameras. These fragments must be combined into a new frame. And then send the data stream to the network.
Image

gordon77
Posts: 4293
Joined: Sun Aug 05, 2012 3:12 pm

Re: Stereo camera PI processing

Sun Nov 24, 2019 7:32 pm

Roi will select a segment BUT I don't know if it will work for 3d, and select a different segment from each camera (which l don't think you mentioned before)

User avatar
HermannSW
Posts: 1651
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Stereo camera PI processing

Mon Nov 25, 2019 9:50 am

worldmaster wrote:
Sun Nov 24, 2019 7:24 pm
I repeat, I need fragments from each of the cameras. These fragments must be combined into a new frame. And then send the data stream to the network.
I have to repeat, the Stereo hat hides the fact that there are two cameras, Raspberry Pi thinks there is only one camera.
gordon77 wrote:
Sun Nov 24, 2019 7:32 pm
Roi will select a segment BUT I don't know if it will work for 3d, and select a different segment from each camera (which l don't think you mentioned before)
Again, Stereo hat provides "normal" frames, Pi does not know about two cameras connected, nor is any 3D option required for raspistill/raspivid.

But I was wrong with "-roi", since the Pi does not know about two cameras, it does not work the way needed.

This is simple stereo capture for this command:

Code: Select all

$ raspistill -md 4 -w 1296 -h 972 -p 22,50,648,486 -o p1.jpg
$
Image


Now with "-roi" I get different regions:

Code: Select all

$ raspistill -md 4 -w 1296 -h 972 -p 22,50,648,486 -o p2.jpg -roi 0.25,0.0,0.5,0.5
$
Image


Besides having received free samples of 0.3/1.0/2.0MP monochrome global shutter, 13MP/16MP color rolling shutter and 0.3MP color/2.0MP monochome stereo cameras from Lee at Arducam, I have no incentive in increasing Arducam sales.

Btw, Arducam stereo hat for Raspberry Pi with two 5MP camera bundle does cost 89.99€
https://www.uctronics.com/index.php/ard ... -2173.html

Comparable StereoPi starter kit does cost 125$ but includes a compute module 3+ lite:
https://www.crowdsupply.com/virt2real/stereopi

So if you already have a Pi, stereo hat is cheaper.

Stereo hat with two 8MP v2 cameras does cost 109.99$.
In addition to StereoPi it allows to get stereo hat with two Arducam cameras as well (all those I listed above).
I do have a sample here at home with two ov9281 monochrome global shutter cameras (my previous posting photo was taken with that, a "2 x 1280x800 = 2560x800" frame):
https://stamm-wilbrandt.de/en/forum/ArduCAM/mode7.jpg
The Arducam cameras do provide full and not half resolution as v1/v2 cameras.
But they only work with Arducam driver, not with raspivid/raspistill like the v1/v2 cameras:
https://github.com/ArduCAM/MIPI_Camera


So "-roi" does what it should above, capture 50% size in both dimensions , starting at coordinate (324,0).
What is needed for the OP request is to be able to specify two regions of interest.
I will come up with such a solution based on raspividyuv and i420toh264 tool (with GPU doing .h264 conversion) soon:
https://github.com/Hermann-SW2/userland ... i420toh264
What I can say for sure now is that that solution will work, and will allow to capture two arbitrarily located regions of interest.
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

User avatar
HermannSW
Posts: 1651
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Stereo camera PI processing

Wed Nov 27, 2019 10:22 am

HermannSW wrote:
Mon Nov 25, 2019 9:50 am
So "-roi" does what it should above, capture 50% size in both dimensions , starting at coordinate (324,0).
What is needed for the OP request is to be able to specify two regions of interest.
I will come up with such a solution based on raspividyuv and i420toh264 tool (with GPU doing .h264 conversion) soon:
https://github.com/Hermann-SW2/userland ... i420toh264
What I can say for sure now is that that solution will work, and will allow to capture two arbitrarily located regions of interest.
Done and commited, with new section added to README.md with details:
https://github.com/Hermann-SW2/userland ... g-pipeline

This command part of bash i420toh264 pipeline (x1,y1,x2,y1,w,d)

Code: Select all

...
./sample_yuv_dbl_roi 640 480 40,160,371,51,240,180 |
...
extracts regions with left top (40,160) and (371,51) from [email protected] raspividyuv video, both of dimension "w x d", and creates new frames of dimension "2w x d" from them for further processing. There are quite some assertions on coordinates, width, height, ... -- the 2nd frame has to be right of left frame (x1+w<=x2) for easy inplace copying to work, besides that their top left coordinates can be selected freely.

This is a frame from generated 480x180 video tst.h264:
Image


For comparison, this is full 640x480 view:
Image
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Wed Nov 27, 2019 2:41 pm

Thanks. And how now to broadcast the received stream to the network?

User avatar
HermannSW
Posts: 1651
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Stereo camera PI processing

Wed Nov 27, 2019 7:31 pm

That is a completely different question.
Depends how/where you want to receive it.
In browser, vlc, raw .h64 streaam, using rtsp:, ....

There are tons of solutions, many here in forum, or on the web.
You can send and receive with gstreamer pipeline via several protocols.
Or this:
http://frozen.ca/streaming-raw-h-264-fr ... pberry-pi/

In case you want to do it with i420toh264 bash pipeline, you just need to know that that tool accepts "-" as filename in order to send the GPU compressed .h264 stream to stdout like raspivid[yuv], and form there you can use any method for broadcast.
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Sun Dec 01, 2019 6:09 pm

Thanks for answers. Maybe you can tell a ready-made solution so that the stream received from i420toh264 is broadcast via rtsp with h264 compression?

User avatar
HermannSW
Posts: 1651
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Stereo camera PI processing

Sun Dec 01, 2019 8:53 pm

worldmaster wrote:
Sun Dec 01, 2019 6:09 pm
Thanks for answers. Maybe you can tell a ready-made solution so that the stream received from i420toh264 is broadcast via rtsp with h264 compression?
Replace "raspivid ... -o -" with "raspividyuv ... | ... | ./i420toh264 - width height" for options 1/2/4
https://raspberrypi.stackexchange.com/q ... 1429#31429


From
https://raspberrypi.stackexchange.com/q ... tcat#27083

you can change option 2 above with:

Code: Select all

... | cvlc -vvv stream:///dev/stdin --sout '#standard{access=http,mux=ts,dst=:8090}' :demux=h264
and access on client browser with:

Code: Select all

http://<IP-OF-THE-RPI>:8090
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

worldmaster
Posts: 8
Joined: Thu Nov 21, 2019 4:26 pm

Re: Stereo camera PI processing

Wed Dec 04, 2019 7:30 pm

Thank you so much. You have helped me a lot. I will try.

Return to “Camera board”