rndmplyr
Posts: 13
Joined: Fri May 31, 2013 4:09 pm

Re: Low latency wifi stream possible?

Tue Jun 10, 2014 4:42 pm

Thanks, sounds promising. I tried to use the commands you posted, noticed a small error: you need to cd to a subfolder mjpg-streamer-experimental, because the root folder is empty.
When I started the camera script, I get the error
DBG(/usr/src/mjpg-streamer/mjpg-streamer/mjpg-streamer-experimental/plugins/inp ut_raspicam/input_raspicam.c, worker_thread(), 553): Host init, starting mmal st uff
mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:EN OMEM)
mmal: mmal_component_create_core: could not create component 'vc.ril.camera' (1)
error create camera

Edit: the error is a general camera problem, as raspivid -d yields an error as well. Ill need to resolve this first :(

fimbox
Posts: 3
Joined: Sat Jun 21, 2014 2:15 pm

Re: Low latency wifi stream possible?

Sat Jun 21, 2014 2:47 pm

Hi
I tried a lot with mplayer to get reliable low latency and got down to around 250ms over wifi with 1280x720 by setting the framerate higher than the source video fps (>30fps)

The problem with the >30fps trick is that the picture gets choppy, thats because the player "thinks" the framerate is higher and always gets to the end of the cache and stalls there for a while, furthermore its not possible to get below 250ms-300ms.

So I ended up in writing my own h264 streamer that just collects enough data from the network stream to display one frame, its even possible to present the frame before it has all data for it to reduce the latency even more, but thats potentially generating artifacts. So its a quality vs. latency tradeoff.

Now I am down to 150-180ms with a reliable smooth hd picture, including my network ping to the pi with around 90ms, most of the time seems to be consumed by the network caching, I will try to use udb here to reduce that further

However with this pipeline it should be possible to fly fpv with hd resolution, finally :)

attached a picture where the camera captures a stopwatch on the screen
Attachments
pi_latency_watch.jpg
pi_latency_watch.jpg (43.19 KiB) Viewed 8019 times

gregd72002_
Posts: 49
Joined: Mon Apr 14, 2014 8:50 am

Re: Low latency wifi stream possible?

Sat Jul 19, 2014 7:58 pm

Hi fimbox,

Is there a chance you could share the code? Do you use raspivid to capture and then you stream it using your own solution? Or did you write the entire thing (capture & streaming) from scratch?

hamishSan
Posts: 1
Joined: Fri Aug 01, 2014 3:02 pm

Re: Low latency wifi stream possible?

Fri Aug 01, 2014 3:13 pm

I greatly reduced my streaming latency by
  1. lowering video stream bitrate, and
  2. running the receiving client the at a higher frame rate then the sender.
both of which have been mentioned earlier on this thread 8-)

Code: Select all

reciever$ nc -l -p 5001 | mplayer -fps 13 -demuxer h264es - 
raspberrypi$ raspivid -fps 12 -b 1000000 -t 0 -n -o - | nc 10.0.1.2 5001

peter64
Posts: 7
Joined: Thu Oct 23, 2014 7:32 am

Re: Low latency wifi stream possible?

Thu Oct 23, 2014 7:40 am

Hey fimbox,

Any possibility of you sharing your code. I'm pretty new to raspberrypi but I also wanted to use it for FPV as well some day.
It seems the OpenMax ffmpeg support isn't really complete yet as far as I can tell so I was hoping you might be willing to share your code or sell a copy.

Thanks,

Peter
fimbox wrote:Hi
I tried a lot with mplayer to get reliable low latency and got down to around 250ms over wifi with 1280x720 by setting the framerate higher than the source video fps (>30fps)

The problem with the >30fps trick is that the picture gets choppy, thats because the player "thinks" the framerate is higher and always gets to the end of the cache and stalls there for a while, furthermore its not possible to get below 250ms-300ms.

So I ended up in writing my own h264 streamer that just collects enough data from the network stream to display one frame, its even possible to present the frame before it has all data for it to reduce the latency even more, but thats potentially generating artifacts. So its a quality vs. latency tradeoff.

Now I am down to 150-180ms with a reliable smooth hd picture, including my network ping to the pi with around 90ms, most of the time seems to be consumed by the network caching, I will try to use udb here to reduce that further

However with this pipeline it should be possible to fly fpv with hd resolution, finally :)

attached a picture where the camera captures a stopwatch on the screen

jward
Posts: 8
Joined: Wed Mar 18, 2015 1:46 pm

Re: Low latency wifi stream possible?

Wed Mar 18, 2015 3:08 pm

MrBunsy wrote:
rndmplyr wrote:MrBunsy, did you use the commands from the blog instructions on the RPi side?
Not quite - I set the server (pi) side up so it was listening...and stuck that in a while loop with bash.
I prefer your idea of having the RPi listen so I used a similar command in a while loop but how do you keep it from getting out of hand and running excessively?
Where are the blog instructions that have been referenced?
MrBunsy wrote:
towolf wrote: So in other words stop netcatting video into mplayer. The lowest latency streaming I’ve seen to far was with Gstreamer’s rtpbin. I might be worth it to lower GOP with the new -g flag.
Ooo that looks useful. I shall investigate...
Did you find better performance?

jward
Posts: 8
Joined: Wed Mar 18, 2015 1:46 pm

Re: Low latency wifi stream possible?

Wed Mar 18, 2015 3:12 pm

rndmplyr wrote:...using the commands from the official tutorial...
Where is the tutorial? I've been looking for something official but have only found anecdotal tips.

User avatar
MrBunsy
Posts: 185
Joined: Mon Feb 20, 2012 1:48 pm
Location: Southampton, UK
Contact: Website

Re: Low latency wifi stream possible?

Wed Mar 18, 2015 9:21 pm

jward wrote:
MrBunsy wrote:
rndmplyr wrote:MrBunsy, did you use the commands from the blog instructions on the RPi side?
Not quite - I set the server (pi) side up so it was listening...and stuck that in a while loop with bash.
I prefer your idea of having the RPi listen so I used a similar command in a while loop but how do you keep it from getting out of hand and running excessively?
It waits for the next connection, so it just sits there not causing an issue.
jward wrote: Where are the blog instructions that have been referenced?
Honestly can't remember I'm afraid, and I don't have access to my usual PC at the mo, so I can't check unfortunately. Was it the official rp blog?
jward wrote:
MrBunsy wrote:
towolf wrote: So in other words stop netcatting video into mplayer. The lowest latency streaming I’ve seen to far was with Gstreamer’s rtpbin. I might be worth it to lower GOP with the new -g flag.
Ooo that looks useful. I shall investigate...
Did you find better performance?
Not found anything yet, but I've been working on other projects for the last few months. I'll come back to this in a few more hopefully!

rndmplyr
Posts: 13
Joined: Fri May 31, 2013 4:09 pm

Re: Low latency wifi stream possible?

Thu Mar 19, 2015 5:03 pm

jward wrote:
rndmplyr wrote:...using the commands from the official tutorial...
Where is the tutorial? I've been looking for something official but have only found anecdotal tips.
Hm. Apparently they have overhauled the blog. It was right in the article from when the camera was released, but I can't find it anymore.

fimbox
Posts: 3
Joined: Sat Jun 21, 2014 2:15 pm

Re: Low latency wifi stream possible?

Mon Aug 03, 2015 12:20 am

I created a package with a C++ program I made that streams an HD video from a rasperry pi to the pc with less than 200ms latency using netcat and FFmpeg. Depending on the network ping also 100ms should be possible.

There is also an oculus mode that duplicates and wraps the image to view on an oculus device (experimental)

You need a Windows PC ( tested on Windows 10 ) and a raspberry pi with a raspberry camera
see the README in the package for more info

basically it works like:
- change the IP in the script, execute it on the pi
- start the program on the windows pc, it should connect automatically
- enjoy 5mbit brilliant HD h264 video stream with low latency, fullscreen and oculus mode included

http://goo.gl/efkLNY

have fun & share the results!
Attachments
latency_test.jpg
latency_test.jpg (59.83 KiB) Viewed 6205 times

pescifranca
Posts: 2
Joined: Wed Oct 07, 2015 3:25 pm

Re: Low latency wifi stream possible?

Thu Oct 08, 2015 6:13 am

Hi Timbox, I've tried to use your software but, when i've launched the .bat file on a windows 7 PC (64bit), I've got an error message reporting:

kinect10.dll missed

Where Can I find this dll? I can't figure out...

Thanks
Bye

pescifranca
Posts: 2
Joined: Wed Oct 07, 2015 3:25 pm

Re: Low latency wifi stream possible?

Thu Oct 08, 2015 6:17 am

Hi fimbox, I've tried your software but I've got an error message when I've launched .bat file on windows 7 (64bit):
kinect10.dll missed?

Where can I find that .dll?

Thanks bye

rndmplyr
Posts: 13
Joined: Fri May 31, 2013 4:09 pm

Re: Low latency wifi stream possible?

Thu Oct 08, 2015 10:15 am

Hey folks, I found what seems to be the quite best method to stream video over wifi using the Pi: wifibroadcast by befinitv (https://befinitiv.wordpress.com/wifibro ... ideo-data/). It works with wifi in monitor mode, meaning no connection and authentication with a network is needed and connection reestablishment is very fast.

Vandan
Posts: 1
Joined: Sun Oct 11, 2015 8:33 am

Re: Low latency wifi stream possible?

Sun Oct 11, 2015 8:47 am

by towolf » Thu Jun 06, 2013 9:16 pm
I’ve filmed by screen with a clock on it and wit the Gstreamer RTP over UDP command it appears to have a delay of 200-300ms.

Perhaps this can be improved by tuning network device buffering or the Gstreamer properties of the various elements.
I'm using Mjpeg streamer to view the live video stream on a web browser using this tutorial :http://petrkout.com/electronics/low-lat ... er-opencv/

The tutorial quotes a latency of 0.4s. But I want to personally measure the latency. How did you implement the clocks to measure the latency ?

User avatar
jbeale
Posts: 3474
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Low latency wifi stream possible?

Mon Oct 12, 2015 7:39 pm

One easy way to measure latency is to have an onscreen clock on the same computer that is receiving and displaying the video. Point the camera so that both the clock and the camera's output window is visible on the screen , which will then after a few frames have several smaller versions of the clock visible because the camera records its own output so the process is recursive. The overall latency is then the difference between the time displayed between the clock image in successive frames. Several such screen shots have been posted on this thread. 0.4 seconds of latency is quite long, as you have seen here, people are managing 0.15 seconds or less.

fimbox
Posts: 3
Joined: Sat Jun 21, 2014 2:15 pm

Re: Low latency wifi stream possible?

Mon Oct 26, 2015 5:06 pm

for the kinect10.dll error, please install the kinect sdk from

http://www.microsoft.com/en-us/kinectfo ... Start.aspx

I will remove this dependency asap

RpiName
Posts: 712
Joined: Sat Jul 06, 2013 3:14 am

Re: Low latency wifi stream possible?

Fri Jan 15, 2016 8:53 am

720p hw-encoded, 30fps video to browsers (or to other WebRTC-compliant peers) with ~no latency and minimal CPU usage can now be done with UV4L at http://linux-projects.org :
it's now (also) possible to live stream hardware-encoded H264 HD video at full 30fps from the RaspberryPi 2 to any browser(*) over the Web, with imperceptible delay and minimal CPU usage.

This is possible thanks to the WebRTC extension for UV4L. No configuration is required! No magic gstreamer sauce here! You just need a browser to see the video ...well, and speakers if you also want to get an audio stream from the Rpi ;)

Since this is WebRTC technology, experts may already know that it's always possible to build more complex scenarios on the top of UV4L! And nothing forbids us to add Full HD in the future.

A new option for the server has been introduced allowing to specify the minimum bitrate for the adaptive streaming.
Original news:
http://www.linux-projects.org/modules/n ... toryid=190

subseatech
Posts: 1
Joined: Mon Dec 04, 2017 10:28 am

Re: Low latency wifi stream possible?

Mon Dec 04, 2017 10:31 am

Hi all,
Is it possible de have your "ToolBox.exe" code ?
Thank you

Return to “Camera board”