QtMultimedia, OpenMAX and GStreamer - will it ever work?

Ok, so I've been on a mission to get Qt compiled for the pi, with QtMultimedia support so I can play hardware accelerated videos from a QtQuick app. I'm starting to think that is impossible.

I've read the mega-thread here: http://www.raspberrypi.org/phpBB3/viewtopic.php?t=6852 and I don't want to lengthen it, cause who likes wading through threads? Instead I've written instructions on how to get to the stage that I am at, which includes *all* the steps, including the ones that people seem to magically omit.

Here's where I've got so far: http://creeder.com/?page=RaspberryPi I tried to make it beginner-friendly because the amount of undocumented options and unnecessary complication is insane! Building software shouldn't be this hard.

Anyway, if you follow all the instructions, it basically won't work. You'll end up with successfully installed gst-omx plugins, but if you try to use them with gst-launch it doesn't work. Additionally, I don't even know if Qt will take advantage of OpenMAX, but if you try to use QtMultimedia it says:

Code: Select all

Module 'QtMultimedia' does not contain a module identifier directive - it cannot be protected from external registrations.
defaultServiceProvider::requestService(): no service found for - "org.qt-project.qt.mediaplayer"

Really annoying since I've written a simple media centre in Qt Quick, and the UI is nice and speedy (very unlike XBMC!). It just can't play video on the Pi. I'm basically giving up on gstreamer/QtMultimedia but I figured this might be useful to other people. Instead I'm going to try to integrate omxplayer into my app, even though I know that is going to be a massive pain. But hey at least it definitely works, judging by XBMC.

So I guess my questions are:

1. Has anyone actually got gst-omx to work properly? Can you post *detailed* instructions?
2. Even if gst-omx works, would Qt use it?
3. What's the best way to integrate omxplayer into my app? Have the XBMC guys turned it into a library or anything?

1. Has anyone actually got gst-omx to work properly? Can you post *detailed* instructions?
2. Even if gst-omx works, would Qt use it?
3. What's the best way to integrate omxplayer into my app? Have the XBMC guys turned it into a library or anything?

Omxplayer library under way.

Omxplayer library under way.

Ah cool, how far through are you? Github? Just for reference, here's what my ideal library would have at least the following:

1. The ability to set the layer of the output so it can appear "above" my Qt program.
2. A way to query the playback position.
3. Play/pause/seek commands.
4. The ability to play back from an arbitrary stream; not just files. I.e. I provide read and seek callbacks.

Obviously there's way more you could do but that would be a nice start.

Upon further searching, it seems Luca Carlon (luca4) has done almost exactly what I want:

http://thebugfreeblog.blogspot.co.uk/20 ... ering.html

Upon further searching, it seems Luca Carlon (luca4) has done almost exactly what I want:

http://thebugfreeblog.blogspot.co.uk/20 ... ering.html

So have you managed to get qt.mediaplayer to work? I have exactly the same problem:

Code: Select all

defaultServiceProvider::requestService(): no service found for - "org.qt-project.qt.mediaplayer"


When I configured Qt I had gstreamer support enabled but in /usr/local/qt5pi/plugins/mediaservice there’s only one file: libqtmedia_audioengine.so. According to this: https://bugreports.qt-project.org/browse/QTBUG-26875
there should also be libgstmediaplayer.so

Yes, it should be in qt_install_dir/plugins/mediaservice. Did you compile and install the qtmultimedia module correctly? Did you try the qmlvideo sample? It seemed to work for me.

Did you compile and install the qtmultimedia module correctly?

I think so, I used this script: https://gist.github.com/raw/3488286/d1d ... eqtpi.bash
so the module was installed simply like that:

Code: Select all

cd your_qt_module
your_qt_prefix/bin/qmake
make -j(number_of_cpus + 1)
sudo make install

Could you please paste here the qml code you used for testing?

Yes, qtmultimedia seems to be compiled. Check the config.log in qtmultimedia to see if there's anything missing. I used the qmlvideo sample in Qt sources and seemed to work.

Check the config.log in qtmultimedia to see if there's anything missing.

Thanks for the hint. I was able to solve the previous problem but now I have another one:

Code: Select all

Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started

This appears to be related to audio but my video file doesn’t even have audio stream. I used following qml code in my main.qml

Code: Select all

    Video {
        id: video
        width : 800
        height : 600
        source: "test.avi"
        autoPlay: true
    }

Yes, jack is required. Did you try the qmlvideo demo?

Yes, jack is required. Did you try the qmlvideo demo?

Getting jack to work seems to be another long story: viewtopic.php?f=41&t=5787
I don’t really need a sound at this moment. Is there any way to get only the video working without the sound?

Again: did you try the qmlvideo sample application? It worked for me some time in the past. Of course, only small resolutions, like < 480p, but it was "working" (working is a complex word: it rendered some videos I tested). I also saw an animation of the video resizing but I remember that for some strange reason after a couple of restart of the application it required a system reboot.
I didn't investigate any of the issues I encountered and I completely abandoned this solution after I saw how gst-omx was working, so I have not much more info to provide, sorry.

Again: did you try the qmlvideo sample application?

Do you mean this example? http://doc.qt.digia.com/qtmobility/video-qmlvideo.html
If so then it seems like pretty complex example and I am not sure if I am able to reconfigure and compile it for RPi. Actually I don’t even need all the functionality provided by this example.

Now I managed to get the video working somehow. Last time I forgot that it is necessary to roll back the firmware in order to get the gstreamer to work. So now I have following setup:

• 2012-12-16-wheezy-raspbian

• GStreamer and OpenMAX installed by the instructions posted in this thread: viewtopic.php?p=191153#p191153

• Firmware rolled back to 17ff0bb5bb46210bad235d32589d35f951f269cf

• Qt5 installed by these instructions: http://qt-project.org/wiki/RaspberryPi_Beginners_guide
I used bakeqtpi script with small modification: I configured qtbase without -reduce-relocations and -reduce-exports flags in order to get gstreamer support.

• My application is similar to this example: https://gitorious.org/qt5-raspberrypi-example
In main.qml file I have Video qml element which I have posted previously.

• I used following videos from http://www.bigbuckbunny.org/index.php/download/ for testing:
big_buck_bunny_480p_h264.mov
BigBuckBunny_320x180.mp4

The problem is that while playing a video the CPU usage is always almost 100%. Also I was not able to play these videos to the end because at some point the video just halts. The application continues to function so I assume that it is the gstreamer that crashes. I think cause of the crash may be related to the high CPU usage.

In this tread viewtopic.php?p=192425#p192425
m][sko has posted following CPU usages when playing videos in the similar way as I am trying now:

Code: Select all

20%CPU for 720x576 25fps
45%CPU for 1280x720
74% CPU for 1920x840

So there’s clearly something wrong with my setup.

I have had the same results, I believe the cpu precentages m[]sko refers to is without rendering... The only place where I've seen video running from Qt is from luc4 on his blog:

http://thebugfreeblog.blogspot.co.uk/20 ... ering.html

I've had video playing from Qt,using the Qml video element which in turn pumps it into Gstreamer and it's especially laggy, some videos fail to even play (play fine with omxplayer) so I think gst-omx needs a bit more work... I looked into writing a new gstreamer element, but was looking trickier than first impressions.

Does anyone know if Gstreamer installed from raspbian repos, i.e gstreamer-0.10 is hard float?? And would ffmpeg need to be built with hard float in mind?

I have had the same results, I believe the cpu precentages m[]sko refers to is without rendering... The only place where I've seen video running from Qt is from luc4 on his blog:

http://thebugfreeblog.blogspot.co.uk/20 ... ering.html

I've had video playing from Qt,using the Qml video element which in turn pumps it into Gstreamer and it's especially laggy, some videos fail to even play (play fine with omxplayer) so I think gst-omx needs a bit more work... I looked into writing a new gstreamer element, but was looking trickier than first impressions.

Does anyone know if Gstreamer installed from raspbian repos, i.e gstreamer-0.10 is hard float?? And would ffmpeg need to be built with hard float in mind?

look for how to enable debug info in gstreamer
because it looks like you get software decoding with gstreamer ffmpeg

You sould start with simple pure gstreamer pipeline like
viewtopic.php?t=6852

GST_DEBUG="*:3" gst-launch-0.10 uridecodebin uri="file:///home/misko/test/1778150-MZNqiO.mp4" ! fakesink sync=true

If you want only omx plugin debug
GST_DEBUG="omx:5"

look for how to enable debug info in gstreamer
because it looks like you get software decoding with gstreamer ffmpeg

Well, the first lines of omx debug are:

Code: Select all

omx gstomx.c:2086:plugin_init: Registering element 'omxmpeg4videodec'
omx gstomx.c:2086:plugin_init: Registering element 'omxh264dec'
omx gstomx.c:107:gst_omx_core_acquire: Successfully loaded core '/opt/vc/lib/libopenmaxil.so'
omx gstomx.c:122:gst_omx_core_acquire: Successfully initialized core '/opt/vc/lib/libopenmaxil.so'

I get the same debug output at all times, regardless of the way I use to start the video. E.g. this way:

Code: Select all

gst-launch-0.10 uridecodebin uri="file:///home/pi/test.mov" ! fakesink sync=true

or this way:

Code: Select all

gst-launch-0.10 playbin2 uri="file:///home/pi/test.mov" video-sink=fbdevsink

or from my Qt app.

So I think it is using hardware decoding but for some reason the CPU usage is still very high.

Well, the first lines of omx debug are:

Code: Select all

omx gstomx.c:2086:plugin_init: Registering element 'omxmpeg4videodec'
omx gstomx.c:2086:plugin_init: Registering element 'omxh264dec'
omx gstomx.c:107:gst_omx_core_acquire: Successfully loaded core '/opt/vc/lib/libopenmaxil.so'
omx gstomx.c:122:gst_omx_core_acquire: Successfully initialized core '/opt/vc/lib/libopenmaxil.so'

gstreamer always init all plugins

just simple enable debug level 5 for omx
and look for openmax real decoding

you should see something like openmax channel comunication

just simple enable debug level 5 for omx
and look for openmax real decoding

Here’s a log of omx debug output (level 5) when playing video from Qt for a few seconds: http://pastebin.com/raw.php?i=t6pn0baR

The December raspian, gstreamer and OpenMax dont work.. Since QT uses gstreamer, rather look there.

GSTOMX stops processing at a point. I am still trying to find out what changed and why.

just simple enable debug level 5 for omx
and look for openmax real decoding

Here’s a log of omx debug output (level 5) when playing video from Qt for a few seconds: http://pastebin.com/raw.php?i=t6pn0baR

It looks fine

can you share you video somewhere or can you write detail about audio video codec resolution,..

As I wrote somewhere else
qt5 + qml + gstreamer that is my experience
20%CPU for 720x576 25fps
45%CPU for 1280x720
74% CPU for 1920x840

do you have at least same results without qt with fakesink sync=true ?