kytart
Posts: 6
Joined: Fri Feb 26, 2021 3:06 pm

How to use dmabuf with OpenGL ES to achieve zero-copy behavior on RPi 4?

Thu Jun 24, 2021 4:54 pm

I'm writing my own video player for Raspberry Pi 4.

Right now I have a working version that is capable of very fast HW decoding of H.264 video (up to 50 fps) but the rendering part is not fast enough (around 24 fps).

I'm using OpenGL ES 3.0 and the program runs as a Wayland client on top of Weston, which is using DRM backend.

The way I render each frame is:
  1. I use ffmpeg for decoding. My program works with either MMAL decoder or video4linux decoder.
  2. I receive 3 buffers that contain YUV encoded image.
  3. I copy those buffers to a OpenGL texture using glTexSubImage2D.
  4. I convert YUV into RGB in the shader for better performance.
I've done some studying and the issue seems to be in the glTexSubImage2D, since this copies the entire frame from one place in memory to another place, which is slow.

If I'm correct, it should be possible to avoid the copying alltogether with use of DMA buffers. However I've never used DMA buffers and the information I found on how to do it, let alone in conjunction with OpenGL, is very incomplete and I'm still lost.

Could anyone point me in the right direction? Am I at least going in the right direction? Is there some literature or some other sources where I could find concrete examples of this?

randomher0
Posts: 25
Joined: Wed Apr 21, 2021 9:54 pm

Re: How to use dmabuf with OpenGL ES to achieve zero-copy behavior on RPi 4?

Thu Jun 24, 2021 8:29 pm

Hi, I've also been working with dma buffers recently - for a similar topic: I was trying to get zero copy screen capture working with mmal-h264.
You can see how to supply a dma-buf to the mmal encoder for zero-copy encoding see here), I suppose ther must be some way to get a dma-buf *from* mmal when doing decoding.

There are a few egl extensions that you can use to import/export dma-buf into a texture which can be processed/drawn in egl.
The link below should help you start out.

https://blaztinn.gitlab.io/post/dmabuf-texture-sharing/

RedMarsBlueMoon
Posts: 258
Joined: Mon Apr 06, 2020 3:49 am

Re: How to use dmabuf with OpenGL ES to achieve zero-copy behavior on RPi 4?

Thu Jun 24, 2021 10:08 pm

Is this what you want to do?
https://github.com/Fredrum/rpi_v4l2_tests

You said youre decoding files but part of the process might be the same.

Return to “Graphics programming”