User avatar
HermannSW
Posts: 1239
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

@131's <0.1s latency h264 websocket browser player

Thu Feb 08, 2018 7:37 pm

I stumbled over this raspberrypi.stackexchange.com posting from @131
https://raspberrypi.stackexchange.com/q ... 8058#58058

referencing this github project:
https://github.com/131/h264-live-player:

I had no npm installed on Pi 2B and did "sudo apt-get install npm".
Then started the rpi server:

Code: Select all

[email protected]:~/h264-live-player $ nodejs server-rpi.js 
New guy
Incomming action 'REQUESTSTREAM'
raspivid -t 0 -o - -w 960 -h 540 -fps 12

Finally I browsed to http://pi2Bip:8080/ and it immediately worked after clicking on "Start Video".
I am impressed on this really low latency 950x540 live video in the browser!

Image


P.S:
Works fine under Linux Chrome browser, does not work on Android Chrome browser because of emscrypten/asm:
"Broadway provide the crazy emscripten/asm build of a h264 encoder accelerated by webGL canvas"
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/fork-raspiraw
https://github.com/Hermann-SW/userland
https://twitter.com/HermannSW

User avatar
HermannSW
Posts: 1239
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: @131's <0.1s latency h264 websocket browser player

Fri Feb 09, 2018 10:12 pm

Small update -- I still find this rpi camera websocket server really cool.
But I found two issues:
  1. After start you can click Stop Video, and video transmission to browser stops.
    But clicking Start Video again has no effect.
  2. Clicking Disconnect throws an unhandled js error and server stops.
For 1) ending server with CTRL-C and restarting gets it going again.
For 2) "ps -ef | grep raspivid" gives process ID of still running raspivid, "kill -9 ..." with that process ID and the restarting server gets it going again.

P.S:
This is not working for "server-rpi.js", for "nodejs server-static.js" all buttons work fine.

P.P.S:
I will use this rpi camera server for (mobile) raspivid development work without HDMI monitor:

Code: Select all

[email protected]:~/h264-live-player $ diff lib/raspivid.js.orig lib/raspivid.js
19c19
<     var msk = "raspivid -t 0 -o - -w %d -h %d -fps %d";
---
>     var msk = "../userland/build/bin/raspivid -t 0 -o - -stf 5 -w %d -h %d -fps %d";
22c22
<     var streamer = spawn('raspivid', ['-t', '0', '-o', '-', '-w', this.options.width, '-h', this.options.height, '-fps', this.options.fps, '-pf', 'baseline']);
---
>     var streamer = spawn('../userland/build/bin/raspivid', ['-t', '0', '-o', '-', '-stf', '5', '-w', this.options.width, '-h', this.options.height, '-fps', this.options.fps, '-pf', 'baseline']);
[email protected]:~/h264-live-player $ 

Since @131 explicitely asked for "sane" bitrates I will keep the 12fps, while working on vertical lines reductions and vertical increments for raspivid (that allowed raspiraw to do eg. 640x128_s at 665fps with v1 camera).

Development will be based on 640x480 mostly as with v1 camera work for raspiraw:

Code: Select all

[email protected]:~/h264-live-player $ diff lib/_server.js.orig lib/_server.js
16,17c16,17
<         width : 960,
<         height: 540,
---
>         width : 640,
>         height: 480,
[email protected]:~/h264-live-player $ 

If I send I2C commands (at 5th frame) after GPU is done with sending its I2C commands to camera, I will not only be able to change framerate (that worked successfully already, 640x480 at 180fps on v2 camera), but also do other commands commands. Working hypothesis is that if I tell camera to only capture 240 lines instead of 480, GPU will scale that 640x240 to the requested 640x480. Will see tomorrow ...
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/fork-raspiraw
https://github.com/Hermann-SW/userland
https://twitter.com/HermannSW

User avatar
HermannSW
Posts: 1239
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: @131's <0.1s latency h264 websocket browser player

Mon Feb 12, 2018 9:47 pm

WIll have to redo this experiment on LAN, but on train today over Android smartphone Wifi, end2end latency was 07:10.43-07:09.99=0.44s (websocket application plus Wifi):
Image


Yesterday I tried what happens when not using a "sane" bitrate as requested in Readme file.
This is "moving fingers" frame from browser displaying 640x480 at 75fps (over Android AP Wifi):
Image
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/fork-raspiraw
https://github.com/Hermann-SW/userland
https://twitter.com/HermannSW

User avatar
HermannSW
Posts: 1239
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: @131's <0.1s latency h264 websocket browser player

Wed Feb 14, 2018 5:54 pm

I tried "Streaming Video Using VLC Player":
http://www.raspberry-projects.com/pi/pi ... vlc-player

I used totem video play to look at the video streamed with this command for comparison to h264 player of this thread:

Code: Select all

$ raspivid -o - -t 0 -n -w 640 -h 480 -fps 12 | cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/}' :demux=h264
From looking at Android stopwatch on Android and on display I can say that delay with that method is >3s, so h264 websocket player is much better latency wise.


My son does not like emscripten compiled h264 player sent to browser and told me that HTML5 has video players built in.
Is there a streaming raspivid solution with h264 stream played in HTML5 browser?

I do like the empscripten based solution not only because of its cool low latency, but also because I used that same technique for allowing graph layouts in the browser by emscripten compiled Graphviiz layout library running in browser!
https://stamm-wilbrandt.de/GraphvizFiddle/


Image
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/fork-raspiraw
https://github.com/Hermann-SW/userland
https://twitter.com/HermannSW

User avatar
HermannSW
Posts: 1239
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: @131's <0.1s latency h264 websocket browser player

Thu Feb 15, 2018 8:23 pm

I met Jan Schmidt at GStreamer Conference 2017 in Prague last October, and he just explained the latency differences to me:
https://twitter.com/thaytan/status/964172047646474240
... why has rtsp solution 2.6s more latency than the websocket h264 player in browser solution?
because the RTSP client (Totem/Gstreamer/VLC) has a receiver jitterbuffer. GStreamer's is 2 seconds big by default.
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/fork-raspiraw
https://github.com/Hermann-SW/userland
https://twitter.com/HermannSW

Return to “Camera board”