hjimbens
Posts: 50
Joined: Fri May 24, 2013 9:05 am

Playing video in an OpenGL(ES) application on Pi4

Thu Aug 01, 2019 10:21 am

I am trying to integrate hardware accelerated video playback in an OpenGL(ES) application with zero copy. /opt/vc/src/hello_pi/hello_videocube/video.c shows how to do that for a Pi1-3 with OpenMAX egl_render. For the Pi4 there are some challenges. First of all there are two different ways to do OpenGL(ES), via DRM/gbm and via X11. 6by9 kindly suggested the following:
6by9 wrote:
Wed Jun 26, 2019 4:19 pm
OpenMAX egl_render talks under the hood between decoder and 3D - that won't work.
Still bashing the last little bits into shape, but use MMAL or V4L2 to decode into a buffer, export that as a dmabuf, and then pass that into EGL or DRM for rendering. https://github.com/6by9/drm_mmal master branch should do that for DRM (ie do not start X), and the x11 or x11_export branches should do the right thing for EGL and X (x11_export will fail at present).
Unfortunately the master branch, doing DRM, does not use OpenGL and it is not immediately clear to me how to get the video frames into a texture. The x11 branch does not build out of the box. When I fix the Makefile to add some libraries that are missing, it builds, but it does not play the video correctly. Also, the x11 branch uses eglCreateImageKHR from /opt/vc/lib/libbrcmEGL.so, but I need to link with /usr/lib/arm-linux-gnueabihf/libEGL.so with fkms otherwise I will get runtime errors such as "* failed to add service - already in use?".

Is there any example code showing how to get hardware accelerated video playback into a texture that combines well with hardware accelerated OpenGL(ES)?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7149
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Playing video in an OpenGL(ES) application on Pi4

Thu Aug 01, 2019 1:15 pm

hjimbens wrote:
Thu Aug 01, 2019 10:21 am
I am trying to integrate hardware accelerated video playback in an OpenGL(ES) application with zero copy. /opt/vc/src/hello_pi/hello_videocube/video.c shows how to do that for a Pi1-3 with OpenMAX egl_render. For the Pi4 there are some challenges. First of all there are two different ways to do OpenGL(ES), via DRM/gbm and via X11. 6by9 kindly suggested the following:
6by9 wrote:
Wed Jun 26, 2019 4:19 pm
OpenMAX egl_render talks under the hood between decoder and 3D - that won't work.
Still bashing the last little bits into shape, but use MMAL or V4L2 to decode into a buffer, export that as a dmabuf, and then pass that into EGL or DRM for rendering. https://github.com/6by9/drm_mmal master branch should do that for DRM (ie do not start X), and the x11 or x11_export branches should do the right thing for EGL and X (x11_export will fail at present).
Unfortunately the master branch, doing DRM, does not use OpenGL and it is not immediately clear to me how to get the video frames into a texture. The x11 branch does not build out of the box. When I fix the Makefile to add some libraries that are missing, it builds, but it does not play the video correctly. Also, the x11 branch uses eglCreateImageKHR from /opt/vc/lib/libbrcmEGL.so, but I need to link with /usr/lib/arm-linux-gnueabihf/libEGL.so with fkms otherwise I will get runtime errors such as "* failed to add service - already in use?".
The Makefile is redundant on the x11 branch - see commit

Code: Select all

commit 1e7f30ec66d894c1bf918896654a7d202b342a22
Author: Eric Anholt <[email protected]>
Date:   Tue Feb 6 09:36:11 2018 +0000

    Add a meson build script.
    
    This lets me do cross-compiling easily.  Reminder for usage:
    
      meson build/
      ninja -C build/
      sudo ninja -C build/ install
I've removed Makefile.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Graphics programming”