MattSwarbrick
Posts: 27
Joined: Thu Oct 04, 2012 3:15 pm

gst-omx hardware rendering

Thu Oct 11, 2012 4:39 pm

So now most of us are decoding video files via the gpu and rendering the video via the CPU,

on a 128 split, this uses about 95% cpu usage. So I thought I'd start a thread about exploring the possibility of rendering the video back using the gpu.

I found the gst-plugins-gl gstreamer plugin and have built from source, but have encountered a few problems during the build which I'm currrently fixing.

Has anyone used this plugin before? It would be awesome to decode and render using the GPU via gstreamer.

Currently I'm playing big_buck_bunny_480p_h264.mov using the following gst-launch:

Code: Select all

gst-launch-0.10 --gst-debug=omx:5 filesrc location=big_buck_bunny_480p_h264.mov ! qtdemux name=demuxer \ demuxer. ! queue ! faad ! audioconvert ! alsasink device=hw:0,0 sync=false \ demuxer. ! h264parse ! omxh264dec ! ffmpegcolorspace ! fbdevsink

m][sko
Posts: 107
Joined: Fri Jul 20, 2012 6:37 am
Location: Slovakia

Re: gst-omx hardware rendering

Thu Oct 11, 2012 9:39 pm

If you render it with opengl
and you did YUV to RGB with shaders
you can get this
I get this with qt5 qml
20%CPU for 720x576 25fps
45%CPU for 1280x720
74% CPU for 1920x840

we can get even better experience if we share date inside broadcom media core
openmax -> opengl (zero copy)

we have now
openmax -> gstreamer(as YUV picture) -> opengl(texture)

MattSwarbrick
Posts: 27
Joined: Thu Oct 04, 2012 3:15 pm

Re: gst-omx hardware rendering

Fri Oct 12, 2012 11:21 am

Ah that sounds really interesting, I'll look into using that pipeline of:

openmax -> gstreamer -> opengl,

quite keen to get cpu usage right down, don't really want to use the cpu at all, since the gpu should handle everything

Not having massive luck with the gl plugin for gstreamer, but am making some progress, I'll post as sooon as I find something that might be useful for people. getting to the point where if I use the gst-launch:

Code: Select all

gst-launch-0.10 playbin2 uri=file:///home/pi/big_buck_bunny_480p_h264.mov video-sink=glimagesink
it says:

ERROR: from element /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstBin:bin0/GstGLImageSink:glimagesink0: Failed to create gl window

wickwire
Posts: 3
Joined: Thu Nov 22, 2012 1:52 pm

Re: gst-omx hardware rendering

Tue Nov 27, 2012 6:12 pm

m][sko wrote:If you render it with opengl
and you did YUV to RGB with shaders
you can get this
I get this with qt5 qml
20%CPU for 720x576 25fps
45%CPU for 1280x720
74% CPU for 1920x840

we can get even better experience if we share date inside broadcom media core
openmax -> opengl (zero copy)

we have now
openmax -> gstreamer(as YUV picture) -> opengl(texture)
Would it be possible for you to share a working example of this? In particular Qt5/QML also?

Thanks in advance

m][sko
Posts: 107
Joined: Fri Jul 20, 2012 6:37 am
Location: Slovakia

Re: gst-omx hardware rendering

Sat Dec 01, 2012 3:56 pm

wickwire wrote:
m][sko wrote:If you render it with opengl
and you did YUV to RGB with shaders
you can get this
I get this with qt5 qml
20%CPU for 720x576 25fps
45%CPU for 1280x720
74% CPU for 1920x840

we can get even better experience if we share date inside broadcom media core
openmax -> opengl (zero copy)

we have now
openmax -> gstreamer(as YUV picture) -> opengl(texture)
Would it be possible for you to share a working example of this? In particular Qt5/QML also?

Thanks in advance

http://qt-project.org/doc/qt-5.0/qml-qt ... layer.html


import QtQuick 2.0
import QtMultimedia 5.0

Item {
MediaPlayer {
id: mediaplayer
source: "groovy_video.mp4"
}

VideoOutput {
anchors: parent.fill
source: mediaplayer
}

MouseArea {
id: playArea
anchors.fill: parent
onPressed: mediaplayer.play();
}
}

Andrei
Posts: 18
Joined: Tue Nov 13, 2012 8:38 am

Re: gst-omx hardware rendering

Tue Dec 04, 2012 7:57 am

m][sko wrote:If you render it with opengl
and you did YUV to RGB with shaders
you can get this
I get this with qt5 qml
20%CPU for 720x576 25fps
45%CPU for 1280x720
74% CPU for 1920x840

we can get even better experience if we share date inside broadcom media core
openmax -> opengl (zero copy)

we have now
openmax -> gstreamer(as YUV picture) -> opengl(texture)
Could you please provide the instructions for how to install OpenGL plugins for GStreamer?

Timmmm
Posts: 14
Joined: Sun Jul 15, 2012 3:13 pm

Re: gst-omx hardware rendering

Thu Dec 27, 2012 1:25 pm

m][sko wrote:
wickwire wrote: Would it be possible for you to share a working example of this? In particular Qt5/QML also?

Thanks in advance
http://qt-project.org/doc/qt-5.0/qml-qt ... layer.html
Heh that's the easy part! I think he meant how do you get QtMultimedia to work with OpenMAX (or gst-omx)? You're the first person I've found who has claimed to get it to work.

m][sko
Posts: 107
Joined: Fri Jul 20, 2012 6:37 am
Location: Slovakia

Re: gst-omx hardware rendering

Fri Dec 28, 2012 3:00 pm

Just try to put thing together


Qt use gstreamer as media framework
and there is gstreamer openmax plugin :)
http://cgit.freedesktop.org/gstreamer/gst-omx
I would like to try gstreamer 1.0 and omx
it looks like gst-omx move to 1.0 API
btw only different in rasppery branch is call bmc_init();

plus some info here:
http://www.raspberrypi.org/phpBB3/viewt ... 38&t=19606

Timmmm
Posts: 14
Joined: Sun Jul 15, 2012 3:13 pm

Re: gst-omx hardware rendering

Fri Dec 28, 2012 11:26 pm

m][sko wrote:I would like to try gstreamer 1.0 and omx
it looks like gst-omx move to 1.0 API
btw only different in rasppery branch is call bmc_init();
I may be wrong but I think when I checked there were more differences. Also you'll have to cross-compile all of gstreamer-1.0 for the Pi, since it isn't in the repositories.

Also, I'm going to link to this blog post again because it has exactly what I want (code notwithstanding), and I wish I had found Luca's blog earlier!

http://thebugfreeblog.blogspot.it/

m][sko
Posts: 107
Joined: Fri Jul 20, 2012 6:37 am
Location: Slovakia

Re: gst-omx hardware rendering

Sat Dec 29, 2012 9:19 pm

Timmmm wrote:
m][sko wrote:I would like to try gstreamer 1.0 and omx
it looks like gst-omx move to 1.0 API
btw only different in rasppery branch is call bmc_init();
I may be wrong but I think when I checked there were more differences. Also you'll have to cross-compile all of gstreamer-1.0 for the Pi, since it isn't in the repositories.

Also, I'm going to link to this blog post again because it has exactly what I want (code notwithstanding), and I wish I had found Luca's blog earlier!

http://thebugfreeblog.blogspot.it/
I read that blog post yesterday pretty interesting. I would like to see source codes.

if you are talking about compilation
look for compilation on raspberry pi
but you can always use distcc distributed complimation and use qemu machines as clients
it was standard couple years ago as on gentoo distribution is all about package compilation :)

related to gstreamer 1.0
there is unstable repository in debian
look for it.
it isn't raspbian :(

10robinho
Posts: 44
Joined: Thu Feb 28, 2013 7:42 pm

Re: gst-omx hardware rendering

Thu Feb 28, 2013 7:49 pm

m][sko wrote: we can get even better experience if we share date inside broadcom media core
openmax -> opengl (zero copy)

we have now
openmax -> gstreamer(as YUV picture) -> opengl(texture)
Do you have any idea how low CPU usage one could have with modifying omxh264dec to output on opengl texture (would that even be possible??)? Maybe you can try gst-launch without gl part to see?

Could you test CPU usage with 1920x1080 60fps video? If video is watchable, would it be OK to build apps onto omx -> yuv -> opengl principle?

luc4
Posts: 49
Joined: Mon Nov 12, 2012 12:28 am

Re: gst-omx hardware rendering

Fri Mar 01, 2013 7:19 am

m][sko wrote:
Timmmm wrote:
m][sko wrote:I would like to try gstreamer 1.0 and omx
it looks like gst-omx move to 1.0 API
btw only different in rasppery branch is call bmc_init();
I may be wrong but I think when I checked there were more differences. Also you'll have to cross-compile all of gstreamer-1.0 for the Pi, since it isn't in the repositories.

Also, I'm going to link to this blog post again because it has exactly what I want (code notwithstanding), and I wish I had found Luca's blog earlier!

http://thebugfreeblog.blogspot.it/
I read that blog post yesterday pretty interesting. I would like to see source codes.

if you are talking about compilation
look for compilation on raspberry pi
but you can always use distcc distributed complimation and use qemu machines as clients
it was standard couple years ago as on gentoo distribution is all about package compilation :)

related to gstreamer 1.0
there is unstable repository in debian
look for it.
it isn't raspbian :(
I shared all the source code.

nemilos
Posts: 27
Joined: Sat Jan 05, 2013 4:16 am

Re: gst-omx hardware rendering

Sat Jun 01, 2013 10:40 pm

Hello,

I'am a little bit confused !

It's now possible to play video ( hw accelerated ) with :

Qt5 + gstreamer 0.10 + gst-omx ?

thx

albertd
Posts: 51
Joined: Thu Aug 30, 2012 8:10 pm

Re: gst-omx hardware rendering

Mon Jun 03, 2013 9:56 pm

luc4 wrote:I shared all the source code.
Hi luc4, would you be interested in putting the code into our buildroot environment?

https://github.com/albertd/buildroot-rpi

We have a full Qt 5.0.2 environment.

Cheers,
Albert

luc4
Posts: 49
Joined: Mon Nov 12, 2012 12:28 am

Re: gst-omx hardware rendering

Tue Jun 04, 2013 6:07 am

Hi albertd,

I don't think the code is ready to be included in a filesystem. It is more like a proof of concept at the moment and needs fixes and improvements.

Return to “Graphics, sound and multimedia”