gkreidl
Posts: 5207
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

gstreamer-omx not working on Stretch

Wed Sep 13, 2017 10:44 am

On Wheezy (gstreamer1.0 version 1.2) and on Jessie (gstreamer1.0 version 1.4) we had a gstreamer-omx module (version 1.0.0.1) which could be used for decoding (different codecs) and encoding (H264 only) using the GPU. I'm using it in my TV application to transcode DVB streams to lower bit rate (resolution) streams (accessible from the internet, for example) and also in the "rtranscode" package which I have published.

On Stretch we now have gstreamer1.0 version 1.10.4, but the gstreamer-omx module is still version 1.0.0.1. After adpating my tool chains to gstreamer 1.10.4 (there have been a number of changes), I first tried to just repackage audio and video to make sure everything besides transcoding does work. But when I add decoding / encoding via gstreamer-omx it just breaks (I'm testing my tool chains with gst-launch-1.0). It definitely looks like gstreamer-omx 1.0.0.1 cannot be used on Stretch at all. A newer version is currently not available.

As a next step I tried to compile the gstreamer-omx version 1.10.5 from source using the specific RPi options. Compilation broke because of missing GL stuff. gstreamer-omx depends on the plugins-bad module. The version from the Raspbian repository has been compiled for OpenGL and doesn't use the RPi specific libraries (OpenGL ES and EGL).

So the next logical step was to compile the plugins-bad module from source (1.10 branch, current version = 1.10.5) with RPi specific settings. After adding lots of dependencies it compiled successfully and the things I'm using in my tool chains from this module proved to be working.

Now I could also compile gstreamer-omx 1.10.5 sucessfully.But when I try to use the omx module I run into errors. At least it's loading now (other than the 1.0.0.1 version), but it stops with an error message (I've only included the last part here):

Code: Select all

/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
Additional debug info:
gstomxvideodec.c(1599): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason not-negotiated
The tool chain I'm using looks like this (it does work if I remove the transcoding section):

Code: Select all

gst-launch-1.0 -vvv souphttpsrc location="http://localhost:9082/bysid/10301" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0_13ee ! queue ! mpegaudioparse ! matroskamux name=stream streamable=true demux.video_0_13ed ! queue ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! videoconvert ! omxh264enc target-bitrate=2621440 control-rate=variable ! video/x-h264,stream-format=byte-stream,profile=high,width=1024,height=576,framerate=50/1 ! h264parse ! queue ! stream.
At this point I am stuck. Has anybody else successfully compiled gstreamer-omx version 1.10(.x) and got it to work?

--- Additional information ---

For compilation I have used lots of information from here: https://gist.github.com/sphaero/02717b0b35501ad94863

To complile the bad plugins, I have used the following:

Code: Select all

export LDFLAGS='-L/opt/vc/lib' CFLAGS='-I/opt/vc/include -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2' CPPFLAGS='-I/opt/vc/include -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2'

./autogen.sh --disable-gtk-doc --disable-examples --disable-x11 --disable-glx --disable-opengl --enable-dispmanx --enable-introspection=yes --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib/arm-linux-gnueabihf

make CFLAGS+="-Wno-error -Wno-redundant-decls" CXXFLAGS+="-Wno-redundant-decls" LDFLAGS+="-L/opt/vc/lib -lbrcmGLESv2 -lbrcmEGL" 
For gstreamer-omx I have used:

Code: Select all

export LDFLAGS='-L/opt/vc/lib' CFLAGS='-I/opt/vc/include -I/opt/vc/include/IL -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/IL -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2' CPPFLAGS='-I/opt/vc/include -I/opt/vc/include/IL -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/IL -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2'

./autogen.sh --disable-gtk-doc --with-omx-target=rpi --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib/arm-linux-gnueabihf --disable-examples --with-omx-header-path=/opt/vc/include/IL --with-omx-struct-packing=4

make CFLAGS+="-Wno-error -Wno-redundant-decls" LDFLAGS+="-L/opt/vc/lib -lbrcmGLESv2 -lbrcmEGL -lbcm_host"
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

gkreidl
Posts: 5207
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Sat Sep 23, 2017 9:54 am

After trying lots of different options I have meanwhile been able to build a bad-plugin with full OpenGL ES suppport and also a working gstreamer-omx 1.10.5 module (including one upstream patch).

A few days ago the Foundation published a RPi specific gstreamer-omx v. 1.10.4 module, which I have also tested together with the bad-plugins from the Raspbian archive (compiled for use with OpenGL).

The easiest way to test if encoding and decoding work is a simple tool chain like the following:

Code: Select all

gst-launch-1.0 -v videotestsrc ! omxh264enc ! h264parse ! omxh264dec ! glimagesink
With my self-compiled version this produces a small video overlay area showing the typical test image (video). This proves that both hardware encoding and decoding are working.

If I use the Foundation version, it doesn't display anything, because the software OpenGL (mesa) driver is too slow. When I enable the alpha OpenGL driver (not the full video driver, but just OpenGL), it does work. But unfortunately things like omxplayer(GUI) don't work any more, so this solution is completely useless for my TV application.

As a next step I created a tool chain to display a real video, in this case a 720p50 TV stream from my video server (based on mumdvb).

Code: Select all

gst-launch-1.0 -v souphttpsrc location="http://localhost:9082/bysid/10301" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0_13ee ! mpegaudioparse ! mpg123audiodec ! audioconvert dithering=0 ! audio/x-raw,channels=2 ! queue ! omxhdmiaudiosink demux.video_0_13ed ! queue ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! queue ! glimagesink
This displays the video in a 720p overlay in the center of the screen and also plays the selected audio channel. Sometimes the audio sounded a bit shaky, but if I don't do anything else it works quite well.

Using the Foundation version with the alpha OpenGL driver enabled, I also get video and sound, but the video runs for about half a second, stops for about 2 seconds, runs again for half a second, stops again and so on. Obiously the OpenGL driver is too slow to display the video in real time. So it looks like the Foundation version is quite useless at the moment.

But I don't want to use gstreamer-omx for video display; I need it for transcoding. This means that I have to feed the output of the decoder into an encoder again. And this is where both versions break: If I try to connect the output of the decoder to anything else besides glimagesink, I get stream errors (usually a "not-negotiated" message).

Result: Both versions don't solve my problem of using gstreamer-omx for transcoding as I have done in the past. I've got an idea what might be going wrong: The decoder output resides in the GPU memory, which cannot be accessed by normal tool chain elements.

As a next step I'll try to contact the gstreamer developers and ask them for a solution.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

Return to “Raspbian”

Who is online

Users browsing this forum: No registered users and 18 guests