alexanderbrevig
Posts: 3
Joined: Mon Dec 02, 2019 12:51 am

Best framework for maximum video performance?

Mon Dec 02, 2019 1:01 am

Hello everyone! My first post on here :D

I've been playing with a side project on and off and I've been struggeling to get my RPi to play 1080p videos, or rather - to minimize latency between swapping videos.

The end goal is to be able to play videos in sync with music through DMX (I will open source both the hat I made and the C++ program).


I have tried every combination I can think of between various codecs and H.264 settings - opencv - openmax - ffmpeg - QT - OpenFrameworks... The list is long.

I have ordered a couple of RPi 4 to see if they might help me, but my question still stands.

What do you recommend as the best framework for getting video loads fast and a stable 30fps 1080 video stream?

I would prefer to have a documented API at the lowest possible level (many of the frameworks I tried does not allow me to f.ex do double buffering (both in the next-frame-to-render sense, and in the load-next-movie-as-the-other-is-playing sense).

Thank you so much for any and all answers! Best regards,
Alexander

User avatar
dividuum
Posts: 228
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Best framework for maximum video performance?

Tue Dec 03, 2019 11:22 pm

I'm not entirely sure I understand exactly what you're looking for, but my software info-beamer pi could be a candidate. It's scriptable in Lua, supports playing videos and offers frame exact control over what's on the screen at any moment. Videos can be preloaded in the background and then started instantly. Documentation as well as documentation (older example code, various production packages) exist. The only downside might be that's it's commercial software. For purely private use it can be used for free.
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

alexanderbrevig
Posts: 3
Joined: Mon Dec 02, 2019 12:51 am

Re: Best framework for maximum video performance?

Mon Dec 09, 2019 8:40 pm

My question is which framework to use in order to have low level control over video playback from C or C++.


I need "next frame" accuracy, so when I get the signal to swap video - it should be ready next frame.

User avatar
HermannSW
Posts: 2957
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Best framework for maximum video performance?

Tue Dec 10, 2019 10:13 pm

I think MMAL is an efficient framework, but I cannot help with that.

There is a (C) framework I did build myself, and allows video processsing on they fly while capturing.
Typically bash pipeline starts with raspividyuv, then processing (modifying or not modifying) is done in C plugins, and finally i420toh264 tool uses GPU to compress the video as .h264 and store. I use non-modifying pipeline to control two steppers of stepper PT camera system in order to get an always centered video of approach for landing airplane:
https://github.com/Hermann-SW2/userland ... i420toh264

Since the YUV frame processing is done "by hand" in C, you have all the control and can do the buffering you want.
You just will have to decide what to do with frames you cannot process in time -- raspividyuv sends the frames when camera has them.

This is an example of modifying pipeline, animation is played at 3fps, 30 times slower than real.
The white square is displayed for graphical debugging of the pendulum follwing heuristic:
Image


P..S:
Ooops, you want playback, please look into /opt/vc/src/hello_pi/hello_video directory:
alexanderbrevig wrote:
Mon Dec 09, 2019 8:40 pm
My question is which framework to use in order to have low level control over video playback from C or C++.
https://stamm-wilbrandt.de/en/Raspberry_camera.html
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/raspiraw
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264

alexanderbrevig
Posts: 3
Joined: Mon Dec 02, 2019 12:51 am

Re: Best framework for maximum video performance?

Fri Dec 13, 2019 10:47 pm

HermannSW wrote:
Tue Dec 10, 2019 10:13 pm
P..S:
Ooops, you want playback, please look into /opt/vc/src/hello_pi/hello_video directory:
Thank you! Now this is what I've been looking for. I actually cannot believe I have not stumbled across this after all the paths I've been down.

Now, on to figure out multi-threading on the Pi (maybe pthreads works?) and a way to communicate with what seems to be a message queue of sorts with the

Code: Select all

ilclient
so I can interrupt play tunnel and the OMX_StateExecuting.

Thanks again HermannSW!

trejan
Posts: 2659
Joined: Tue Jul 02, 2019 2:28 pm

Re: Best framework for maximum video performance?

Fri Dec 13, 2019 11:39 pm

Be aware that OMX and MMAL aren't able to decode H265 even on a Pi 4. Read https://github.com/raspberrypi/firmware/issues/1168 If you're only going to be using H264 files then this isn't a problem.

Return to “C/C++”