Pi to Pi Bi-Directional Camera Streaming and Preview

[fbutler]

Hi,
I've got a couple of Pi cameras so I've started experimenting with streaming video between 2 Pi's in both directions at once. This appears to be quite feasible and I'm not seeing any high CPU usage whilst doing this.

This screen capture best describes the setup:
http://cloud-wr-eu1.acronis.com/links/9 ... D085DFC979

The top windows show the commands being used to stream from Raspi1 to Raspi4 with preview
The middle windows show the commands being used to stream from Raspi4 to Raspi1 with preview
The bottom windows are showing CPU usage etc. on both Pi's.

Raspi4 is outputting to a TV. The issue I am seeing is that the TV is constantly doing auto-configuration for the HDMI input when displaying a video stream from Raspi1 to Raspi4 whilst at the same time displaying a preview from Raspi4 as it streams to Raspi1

The following short video clips showing the output from Raspi4 on the TV best explain what I am seeing. (WARNING: They may take a bit of time to download dependent on the speed of your Internet connection)
Apologies for the quality of the clips but this is due to low indoor lighting levels and is not a reflection on the quality of what the Raspberry camera can produce

Raspi1 streaming to Raspi4, with no issues:
http://cloud-wr-eu1.acronis.com/links/7 ... A52829E778

Raspi4 displaying a preview of what it is streaming to Raspi1 (The preview is in the bottom right corner of the screen):
http://cloud-wr-eu1.acronis.com/links/C ... 4DBE2534F2

Raspi1 streaming to Raspi4 whilst Raspi4 is displaying a preview (at 1320,650) of what it is streaming to Raspi1
http://cloud-wr-eu1.acronis.com/links/0 ... E50D275B64

The frequency of auto-reconfiguration differs according to where the preview from Raspi4 is placed on the screen (at 320, 700 this time) as shown by comparing this video with the previous one:
http://cloud-wr-eu1.acronis.com/links/7 ... 6116982EFE

The monitor attached to Raspi1 won't auto-adjust at all, (It gives an out of range error) when Raspi1 is displaying a preview and streaming to Raspi4

So my questions are:

• Is this being caused by the known preview issue?
  If not is there anything I can do to make the received video and preview play nice together. I've had a look at the preview code and couldn't see any adjustable parameters which might help.
  Is it just not possible to do what I am attempting to do?
  Is this a bug which can be fixed?

Thanks for taking the time to look at this post.

[jbeale]

I don't know anything about the monitor auto-reconfiguration, but thanks for posting the video examples and screen shots! Good to see streaming both ways can work.

Maybe not the place to discuss it here, but I think there should be some improvement possible with camera low-light performance. For example right now, at least by default the dark regions are apparently crushed to 0 (full black), this makes the video "clean" and good for compression. But for best low-light operation you want to see the sensor noise level above full black, the image will look more ugly, but at least some detail in the dark areas would be visible. Longer shutter speeds should also be possible, duplicating frames as needed.

[fbutler]

I don't know anything about the monitor auto-reconfiguration, but thanks for posting the video examples and screen shots! Good to see streaming both ways can work.

Maybe not the place to discuss it here, but I think there should be some improvement possible with camera low-light performance. For example right now, at least by default the dark regions are apparently crushed to 0 (full black), this makes the video "clean" and good for compression. But for best low-light operation you want to see the sensor noise level above full black, the image will look more ugly, but at least some detail in the dark areas would be visible. Longer shutter speeds should also be possible, duplicating frames as needed.

Yeah, I haven't started looking at exposure levels etc. The darkness in the videos is worse than it actually appears in reality. It's being exacerbated by the camera that is taking the videos.

I'm guessing that the auto-reconfiguration is being caused by some conflict between displaying the incoming stream and the preview, but I can't see how to address it myself. When it isn't auto-reconfiguring it seems to work quite well.

[PsycoMEn]

Hi fbutler,

Did you experienced any "jumpiness" with the setup after 1 min?
Framerate < 15fps

Sender: raspivid -t 99999999 -h 740 -w 1180 -fps 31 -b 1000000 -hf -n -o - | nc -v 192.168.1.84 5001
Reciever: nc -l -p 5001 > vid | hello_video.bin vid

BR,
PsycoMEn

[jbeale]

Sender: raspivid -t 99999999 -h 740 -w 1180 -fps 31 -b 1000000 -hf -n -o - | nc -v 192.168.1.84 5001
Receiver: nc -l -p 5001 > vid | hello_video.bin vid
n

You specified -fps 31 but does that work? I thought nothing over 30 fps was currently supported, even though the module is supposedly capable of 720p at 60 fps.

[fbutler]

Hi fbutler,
Did you experienced any "jumpiness" with the setup after 1 min?

No jumpiness

[PsycoMEn]

Sender: raspivid -t 99999999 -h 740 -w 1180 -fps 31 -b 1000000 -hf -n -o - | nc -v 192.168.1.84 5001
Receiver: nc -l -p 5001 > vid | hello_video.bin vid
n

You specified -fps 31 but does that work? I thought nothing over 30 fps was currently supported, even though the module is supposedly capable of 720p at 60 fps.

Hi,
As far as I can tell yes. It runs.

[jamesh]

Sender: raspivid -t 99999999 -h 740 -w 1180 -fps 31 -b 1000000 -hf -n -o - | nc -v 192.168.1.84 5001
Receiver: nc -l -p 5001 > vid | hello_video.bin vid
n

You specified -fps 31 but does that work? I thought nothing over 30 fps was currently supported, even though the module is supposedly capable of 720p at 60 fps.

Hi,
As far as I can tell yes. It runs.

It will run, but at the max frame frame supported, i.e. 30fps