Streaming lag free 1080p 25fps video. Gstreamer

[Gbaman]

Only for Mac OS and Linux

Over the past week after receiving my camera, I have been trying to set up a method to stream lag free video from my robot. It has to be lag free for the robot as it is going to be remote controlled from the live feed.
I have just posted a guide on my blog explaining how to do it for mac OS, should be very similar for linux based OSs.

The guide can be found at http://pi.gbaman.info/?p=150

Currently I don't see an easy way to do this on windows, I believe it requires gstreamer 1.0 and may not work with 0.10. If anyone finds a way, do let me know

[pmdelpino]

Works nice! much better than other options with VLC.

I have about 0,5s lag instead of 3-4 seconds.

Thank you!

[towolf]

You get gst-launch-0.10 when you download the GStreamer SDK for Windows.

As how you use a command line on Windows, I have no idea.

[Rushy]

For some reason I cannot get this to work it freezes on Pipeline is Prerolling or I get an error

ERROR: from element /GstPipeline:pipeline0/GstGDPDepay:gdpdepay0: Could not decode stream.

[fbutler]

The guide can be found at http://pi.gbaman.info/?p=150

Hi Gbaman,

This pipeline doesn't work when I try it. Here's what I am seeing when attempting to use the sender pipe with no modifications other than using the IP address of the sending Pi:

Code: Select all

pi@raspi1 ~ $ raspivid -t 999999 -h 720 -w 1080 -fps 25 -hf -b 2000000 -o - | gst-launch-1.0 -v fdsrc ! h264parse !  rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=192.168.1.101 port=5000
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0: current-port = 5000
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, width=(int)1080, height=(int)720, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0164001fffe1000f2764001fac2b402202df2f00f1226a01000528ee025cb0
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)"J2QAH6wrQCIC3y8A8SJq\,KO4CXLA\=", payload=(int)96, ssrc=(uint)924457899, timestamp-offset=(uint)3503283303, seqnum-offset=(uint)43364
/GstPipeline:pipeline0/GstGDPPay:gdppay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, sprop-parameter-sets=(string)"J2QAH6wrQCIC3y8A8SJq\,KO4CXLA\=", payload=(int)96, ssrc=(uint)924457899, timestamp-offset=(uint)3503283303, seqnum-offset=(uint)43364
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:sink: caps = video/x-h264, width=(int)1080, height=(int)720, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0164001fffe1000f2764001fac2b402202df2f00f1226a01000528ee025cb0
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: timestamp = 3503283303
/GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0: seqnum = 43364
/GstPipeline:pipeline0/GstGDPPay:gdppay0.GstPad:src: caps = application/x-gdp, streamheader=(buffer)< 01000100464e00000155ffffffffffffffff000000000000000000000000000000000000000000000000000000000000000000000000000000005d6700004773744576656e745365676d656e742c207365676d656e743d284773745365676d656e7429224773745365676d656e742c20666c6167733d284773745365676d656e74466c616773294753545f5345474d454e545f464c41475f4e4f4e452c20726174653d28646f75626c6529312c206170706c6965642d726174653d28646f75626c6529312c20666f726d61743d28477374466f726d6174294753545f464f524d41545f54494d452c20626173653d286775696e74363429302c206f66667365743d286775696e74363429302c2073746172743d286775696e74363429302c2073746f703d286775696e7436342931383434363734343037333730393535313631352c2074696d653d286775696e74363429302c20706f736974696f6e3d286775696e74363429302c206475726174696f6e3d286775696e7436342931383434363734343037333730393535313631353b223b00, 010001000002000000ff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000055d000006170706c69636174696f6e2f782d7274702c206d656469613d28737472696e6729766964656f2c20636c6f636b2d726174653d28696e742939303030302c20656e636f64696e672d6e616d653d28737472696e6729483236342c207370726f702d706172616d657465722d736574733d28737472696e6729224a32514148367772514349433379384138534a715c2c4b4f3443584c415c3d222c207061796c6f61643d28696e742939362c20737372633d2875696e74293932343435373839392c2074696d657374616d702d6f66667365743d2875696e7429333530333238333330332c207365716e756d2d6f66667365743d2875696e7429343333363400 >
/GstPipeline:pipeline0/GstTCPServerSink:tcpserversink0.GstPad:sink: caps = application/x-gdp, streamheader=(buffer)< 01000100464e00000155ffffffffffffffff000000000000000000000000000000000000000000000000000000000000000000000000000000005d6700004773744576656e745365676d656e742c207365676d656e743d284773745365676d656e7429224773745365676d656e742c20666c6167733d284773745365676d656e74466c616773294753545f5345474d454e545f464c41475f4e4f4e452c20726174653d28646f75626c6529312c206170706c6965642d726174653d28646f75626c6529312c20666f726d61743d28477374466f726d6174294753545f464f524d41545f54494d452c20626173653d286775696e74363429302c206f66667365743d286775696e74363429302c2073746172743d286775696e74363429302c2073746f703d286775696e7436342931383434363734343037333730393535313631352c2074696d653d286775696e74363429302c20706f736974696f6e3d286775696e74363429302c206475726174696f6e3d286775696e7436342931383434363734343037333730393535313631353b223b00, 010001000002000000ff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000055d000006170706c69636174696f6e2f782d7274702c206d656469613d28737472696e6729766964656f2c20636c6f636b2d726174653d28696e742939303030302c20656e636f64696e672d6e616d653d28737472696e6729483236342c207370726f702d706172616d657465722d736574733d28737472696e6729224a32514148367772514349433379384138534a715c2c4b4f3443584c415c3d222c207061796c6f61643d28696e742939362c20737372633d2875696e74293932343435373839392c2074696d657374616d702d6f66667365743d2875696e7429333530333238333330332c207365716e756d2d6f66667365743d2875696e7429343333363400 >

(gst-launch-1.0:4543): GStreamer-WARNING **: gstpad.c:3908:gst_pad_push_data:<gdppay0:src> Got data flow before stream-start event

(gst-launch-1.0:4543): GStreamer-WARNING **: gstpad.c:3677:gst_pad_chain_data_unchecked:<tcpserversink0:sink> Got data flow before stream-start event
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

(gst-launch-1.0:4543): GStreamer-WARNING **: gstpad.c:3908:gst_pad_push_data:<gdppay0:src> Got data flow before stream-start event

(gst-launch-1.0:4543): GStreamer-WARNING **: gstpad.c:3677:gst_pad_chain_data_unchecked:<tcpserversink0:sink> Got data flow before stream-start event

The last two warnings then repeat continuously until I interrupt it. These warnings occur before even starting the receiver.

Any thoughts on what could be the issue?

[jarrah31]

Has anyone else had problems installing gstreamer on rPi? I've tried on two different Pi's and Raspbian builds (after doing an update) but keep getting this error:

Code: Select all

pi@raspberrypi ~ $ sudo apt-get install gstreamer1.0
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package gstreamer1.0
E: Couldn't find any package by regex 'gstreamer1.0'

Thanks.

[Lauszus]

@jarrah31
You need to add:

Code: Select all

deb http://vontaene.de/raspbian-updates/ . main

To /etc/apt/sources.list

And then run:

Code: Select all

sudo apt-get update

[Gbaman]

Has anyone else had problems installing gstreamer on rPi? I've tried on two different Pi's and Raspbian builds (after doing an update) but keep getting this error:

Code: Select all

pi@raspberrypi ~ $ sudo apt-get install gstreamer1.0
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package gstreamer1.0
E: Couldn't find any package by regex 'gstreamer1.0'

Thanks.

Yes, my fault. I forgot I had another repo in my sources list which it was pulling it from, have updated the blog post

[fke]

Hi,
works fine in my setup (wlan - internet (vpn) - macbookpro ) -> so i can watch what's going on at home from the internet.
I' tried out some other solutions (vlc-streaming) but this is the best.
h 720
w 1000
fps 12
cpu < 10%

[Lauszus]

I just wanted to let people know that I got audio streaming working as well by using a USB soundcard for audio input:

To stream:

Code: Select all

gst-launch-1.0 alsasrc device=plughw:Set \
! mulawenc ! rtppcmupay ! udpsink host=clientIp port=5001

Replace the clientIp with your computers ip address. Normally I just set the end to 255, so it can be received on every computer on the network, for instance I set mine to: 10.0.0.255.

You will properly have to install the following packages: "alsa-utils", "alsa-firmware", and "alsa-plugins".

Also replace 'Set' with the name of your soundcard. You can get that by running:

Code: Select all

asoundconf list

You can also use:

Code: Select all

cat /proc/asound/cards
aplay -l

To get some more information about your soundcard.

You can set the default soundcard like so:

Code: Select all

asoundconf set-default-card cardname

To adjust your setting use:

Code: Select all

alsamixer

And then to store your settings:

Code: Select all

sudo alsactl store

To test the speakers run:

Code: Select all

speaker-test -c 2

To receive the audio on your computer:

Code: Select all

gst-launch-1.0 udpsrc port=5001 caps="application/x-rtp" \
! queue ! rtppcmudepay ! mulawdec ! audioconvert ! autoaudiosink sync=false

If you experience dropouts you might want to add "num-buffers=1000" at the server side, but in my experience it actually works better by not using it.

I will be working on muxing the signals toghether next

Btw you can test if your USB soundcard is working on the Raspberry Pi with the following command:

Code: Select all

gst-launch-1.0 audiotestsrc ! alsasink

And this will loop back the audio from your microphone:

Code: Select all

gst-launch-1.0 alsasrc device=plughw:Set ! alsasink

I also got streaming working via VLC:

Code: Select all

cvlc -vvv alsa://plughw:Set \
--sout '#transcode{acodec=mp2,ab=32}:rtp{dst=10.0.0.9,port=1234,sdp=rtsp://10.0.0.9:8085/stream.sdp}'

Be sure to replace 10.0.0.9 with the ip address of your Raspberry Pi.

And then just open:

Code: Select all

rtsp://10.0.0.9:8085/stream.sdp

In VLC on your computer, but the latency is much much larger.

Hope this will be useful to someone

[Lauszus]

I wrote a little followup to my previous comment at my blog with a little more information.
Check it out http://blog.tkjelectronics.dk/2013/06/h ... o-latency/

[oddballbutler]

For some reason I cannot get this to work it freezes on Pipeline is Prerolling or I get an error

ERROR: from element /GstPipeline:pipeline0/GstGDPDepay:gdpdepay0: Could not decode stream.

Sorry to revive a year old thread, but I am having the same issue and can't find much info with my googling skills. I can get it to work with UDP but it doesn't seem to like TCP for some reason. Anyone know why we are having this issue?