Best framework for maximum video performance?

[alexanderbrevig]

Hello everyone! My first post on here

I've been playing with a side project on and off and I've been struggeling to get my RPi to play 1080p videos, or rather - to minimize latency between swapping videos.

The end goal is to be able to play videos in sync with music through DMX (I will open source both the hat I made and the C++ program).


I have tried every combination I can think of between various codecs and H.264 settings - opencv - openmax - ffmpeg - QT - OpenFrameworks... The list is long.

I have ordered a couple of RPi 4 to see if they might help me, but my question still stands.

What do you recommend as the best framework for getting video loads fast and a stable 30fps 1080 video stream?

I would prefer to have a documented API at the lowest possible level (many of the frameworks I tried does not allow me to f.ex do double buffering (both in the next-frame-to-render sense, and in the load-next-movie-as-the-other-is-playing sense).

Thank you so much for any and all answers! Best regards,
Alexander

[dividuum]

I'm not entirely sure I understand exactly what you're looking for, but my software info-beamer pi could be a candidate. It's scriptable in Lua, supports playing videos and offers frame exact control over what's on the screen at any moment. Videos can be preloaded in the background and then started instantly. Documentation as well as documentation (older example code, various production packages) exist. The only downside might be that's it's commercial software. For purely private use it can be used for free.

[alexanderbrevig]

My question is which framework to use in order to have low level control over video playback from C or C++.


I need "next frame" accuracy, so when I get the signal to swap video - it should be ready next frame.

[HermannSW]

I think MMAL is an efficient framework, but I cannot help with that.

There is a (C) framework I did build myself, and allows video processsing on they fly while capturing.
Typically bash pipeline starts with raspividyuv, then processing (modifying or not modifying) is done in C plugins, and finally i420toh264 tool uses GPU to compress the video as .h264 and store. I use non-modifying pipeline to control two steppers of stepper PT camera system in order to get an always centered video of approach for landing airplane:
https://github.com/Hermann-SW2/userland ... i420toh264

Since the YUV frame processing is done "by hand" in C, you have all the control and can do the buffering you want.
You just will have to decide what to do with frames you cannot process in time -- raspividyuv sends the frames when camera has them.

This is an example of modifying pipeline, animation is played at 3fps, 30 times slower than real.
The white square is displayed for graphical debugging of the pendulum follwing heuristic:



P..S:
Ooops, you want playback, please look into /opt/vc/src/hello_pi/hello_video directory:

My question is which framework to use in order to have low level control over video playback from C or C++.

[alexanderbrevig]

P..S:
Ooops, you want playback, please look into /opt/vc/src/hello_pi/hello_video directory:

Thank you! Now this is what I've been looking for. I actually cannot believe I have not stumbled across this after all the paths I've been down.

Now, on to figure out multi-threading on the Pi (maybe pthreads works?) and a way to communicate with what seems to be a message queue of sorts with the

Code: Select all

ilclient

so I can interrupt play tunnel and the OMX_StateExecuting.

Thanks again HermannSW!

[trejan]

Be aware that OMX and MMAL aren't able to decode H265 even on a Pi 4. Read https://github.com/raspberrypi/firmware/issues/1168 If you're only going to be using H264 files then this isn't a problem.