Camera streaming with negligible delay

[MPikkemaat]

Hello all,

I've been searching for a way to stream video without delay, and although I found some examples, I had problems getting it right, until I found the following commands to make it happen with gstreamer. Thought I would share it here:

on the pi:

Code: Select all

/opt/vc/bin/raspivid -t 999999 -h 720 -w 1080 -fps 25 -b 2000000 -o - | gst-launch-0.10 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=192.168.1.44 port=5000

on a pc or laptop (this was using ubuntu 12.10)

Code: Select all

gst-launch-0.10 -v tcpclientsrc host=192.168.1.44 port=5000  ! gdpdepay !  rtph264depay ! ffdec_h264 ! autovideosink sync=false

Best of luck with it,
Marnix

[fbutler]

Hello all,

I've been searching for a way to stream video without delay, and although I found some examples, I had problems getting it right, until I found the following commands to make it happen with gstreamer. Thought I would share it here:

on the pi:

Code: Select all

/opt/vc/bin/raspivid -t 999999 -h 720 -w 1080 -fps 25 -b 2000000 -o - | gst-launch-0.10 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=192.168.1.44 port=5000

on a pc or laptop (this was using ubuntu 12.10)

Code: Select all

gst-launch-0.10 -v tcpclientsrc host=192.168.1.44 port=5000  ! gdpdepay !  rtph264depay ! ffdec_h264 ! autovideosink sync=false

I'm trying something based on this with Gstreamer 1.0 from a Pi to a Pi with a slightly modified command on the receiver side.
The Commands I am using are:

Sender:

Code: Select all

raspivid -t 999999 -h 720 -w 1080 -fps 25 -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=192.168.1.101 port=5000

Receiver:

Code: Select all

gst-launch-1.0 -v tcpclientsrc host=192.168.1.101 port=5000  ! gdpdepay !  rtph264depay ! h264parse ! omxh264dec ! videoconvert ! autovideosink sync=false

Although the video displays there is significant lag and the following types of warnings are continuously and repetitively being produced in great numbers on both the sender and the receiver:

Code: Select all

(gst-launch-1.0:10272): GStreamer-WARNING **: gstpad.c:3908:gst_pad_push_data:<gdppay0:src> Got data flow before stream-start event

(gst-launch-1.0:10272): GStreamer-WARNING **: gstpad.c:3677:gst_pad_chain_data_unchecked:<tcpserversink0:sink> Got data flow before stream-start event

(gst-launch-1.0:10272): GStreamer-WARNING **: gstpad.c:3908:gst_pad_push_data:<gdppay0:src> Got data flow before stream-start event

(gst-launch-1.0:10272): GStreamer-WARNING **: gstpad.c:3677:gst_pad_chain_data_unchecked:<tcpserversink0:sink> Got data flow before stream-start event

Has anyone got any thoughts on how to modify the pipelines to prevent/correct these warnings?

[HansSanitizer]

Here's the fastest pipelines I've been able to manage:

PI:
gst-launch-1.0 rpicamsrc ! 'video/x-h264, width=1280, height=720, framerate=30/1,profile=high' ! queue ! rtph264pay ! udpsink host=192.168.0.8 port=5000

Client:
gst-launch-1.0.exe udpsrc port=5000 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! decodebin ! autovideosink


Sorry I should probably show where to get the rpicamsrc element:
https://github.com/thaytan/gst-rpicamsrc

[codestar02]

Here's the fastest pipelines I've been able to manage:

PI:
gst-launch-1.0 rpicamsrc ! 'video/x-h264, width=1280, height=720, framerate=30/1,profile=high' ! queue ! rtph264pay ! udpsink host=192.168.0.8 port=5000

Client:
gst-launch-1.0.exe udpsrc port=5000 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H264" ! rtph264depay ! decodebin ! autovideosink


Sorry I should probably show where to get the rpicamsrc element:
https://github.com/thaytan/gst-rpicamsrc

This method works great! Best streaming I have tried so far.

One question:

One Pi shows a regular oriented preview, but the other pi shows it flipped. I am not sure how to flip the feed of the receiver pipeline. Any suggestions?

Thanks!

EDIT:

I tried using videoflip and it noticeably slows down the stream. The only way I can make the original two commands running lag-free is for one to be upside down....

[jamesh]

Try flipping the camera end instread...

[6by9]

Try flipping the camera end instread...

Sounds like a plan.
Hack at https://github.com/thaytan/gst-rpicamsr ... iew.c#L112 to set param.transform to an appropriate enum from MMAL_DISPLAYTRANSFORM_T (https://github.com/raspberrypi/userland ... deo.h#L109), and or MMAL_DISPLAY_SET_TRANSFORM into param.set.
As long as you are only flipping (not transposing), that'll enable a near zero-cost transform on displaying at the transmit end. It won't change the orientation of the data that is streamed. If you want to do that then it looks like there is a command-line option on gst-rpicamsrc to set up h & v flips (again zero cost as those are actually done on the sensor).

[dharasty]

Thank you so much for this thread, these examples, and the pointer to rpicamsrc.

Questions:

1) Can you explain (or point to docs) what the purpose of the second segment of that pipe does? (Beginning with "video/x-h264".)

2) Is that the right place to reduce the transmitted width, height? Or should I use an argument to "rpicamsrc"?

3) I find that the "client" has to be started first. Then I can start, stop, restart the camera side as often as I want. But if I start the client (display) after the camera (or stop-and-restart the display), it doesn't display. I tried fiddling with "keyframe-interval" in the source stream, but that didn't seem to help.)

4) Can you point to an example use of GST that allows me to show this stream on the RPi? (The "client" example works for me on Ubuntu, but crashes on the RPi. Do I need OpenGL on the RPi or something?)

Thanks!