gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

gstreamer-omx not working on Stretch

Wed Sep 13, 2017 10:44 am

On Wheezy (gstreamer1.0 version 1.2) and on Jessie (gstreamer1.0 version 1.4) we had a gstreamer-omx module (version 1.0.0.1) which could be used for decoding (different codecs) and encoding (H264 only) using the GPU. I'm using it in my TV application to transcode DVB streams to lower bit rate (resolution) streams (accessible from the internet, for example) and also in the "rtranscode" package which I have published.

On Stretch we now have gstreamer1.0 version 1.10.4, but the gstreamer-omx module is still version 1.0.0.1. After adpating my tool chains to gstreamer 1.10.4 (there have been a number of changes), I first tried to just repackage audio and video to make sure everything besides transcoding does work. But when I add decoding / encoding via gstreamer-omx it just breaks (I'm testing my tool chains with gst-launch-1.0). It definitely looks like gstreamer-omx 1.0.0.1 cannot be used on Stretch at all. A newer version is currently not available.

As a next step I tried to compile the gstreamer-omx version 1.10.5 from source using the specific RPi options. Compilation broke because of missing GL stuff. gstreamer-omx depends on the plugins-bad module. The version from the Raspbian repository has been compiled for OpenGL and doesn't use the RPi specific libraries (OpenGL ES and EGL).

So the next logical step was to compile the plugins-bad module from source (1.10 branch, current version = 1.10.5) with RPi specific settings. After adding lots of dependencies it compiled successfully and the things I'm using in my tool chains from this module proved to be working.

Now I could also compile gstreamer-omx 1.10.5 sucessfully.But when I try to use the omx module I run into errors. At least it's loading now (other than the 1.0.0.1 version), but it stops with an error message (I've only included the last part here):

Code: Select all

/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, width=(int)1280, height=(int)720, framerate=(fraction)50/1, parsed=(boolean)true, profile=(string)high-10, level=(string)4
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)50/1
ERROR: from element /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0: Internal data stream error.
Additional debug info:
gstomxvideodec.c(1599): gst_omx_video_dec_loop (): /GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0:
stream stopped, reason not-negotiated
The tool chain I'm using looks like this (it does work if I remove the transcoding section):

Code: Select all

gst-launch-1.0 -vvv souphttpsrc location="http://localhost:9082/bysid/10301" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0_13ee ! queue ! mpegaudioparse ! matroskamux name=stream streamable=true demux.video_0_13ed ! queue ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! videoconvert ! omxh264enc target-bitrate=2621440 control-rate=variable ! video/x-h264,stream-format=byte-stream,profile=high,width=1024,height=576,framerate=50/1 ! h264parse ! queue ! stream.
At this point I am stuck. Has anybody else successfully compiled gstreamer-omx version 1.10(.x) and got it to work?

--- Additional information ---

For compilation I have used lots of information from here: https://gist.github.com/sphaero/02717b0b35501ad94863

To complile the bad plugins, I have used the following:

Code: Select all

export LDFLAGS='-L/opt/vc/lib' CFLAGS='-I/opt/vc/include -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2' CPPFLAGS='-I/opt/vc/include -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2'

./autogen.sh --disable-gtk-doc --disable-examples --disable-x11 --disable-glx --disable-opengl --enable-dispmanx --enable-introspection=yes --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib/arm-linux-gnueabihf

make CFLAGS+="-Wno-error -Wno-redundant-decls" CXXFLAGS+="-Wno-redundant-decls" LDFLAGS+="-L/opt/vc/lib -lbrcmGLESv2 -lbrcmEGL" 
For gstreamer-omx I have used:

Code: Select all

export LDFLAGS='-L/opt/vc/lib' CFLAGS='-I/opt/vc/include -I/opt/vc/include/IL -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/IL -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2' CPPFLAGS='-I/opt/vc/include -I/opt/vc/include/IL -I/opt/vc/include/interface/vcos/pthreads -I/opt/vc/include/interface/vmcs_host/linux -I/opt/vc/include/IL -I/opt/vc/include/EGL -I/opt/vc/include/GLES -I/opt/vc/include/GLES2'

./autogen.sh --disable-gtk-doc --with-omx-target=rpi --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib/arm-linux-gnueabihf --disable-examples --with-omx-header-path=/opt/vc/include/IL --with-omx-struct-packing=4

make CFLAGS+="-Wno-error -Wno-redundant-decls" LDFLAGS+="-L/opt/vc/lib -lbrcmGLESv2 -lbrcmEGL -lbcm_host"
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Sat Sep 23, 2017 9:54 am

After trying lots of different options I have meanwhile been able to build a bad-plugin with full OpenGL ES suppport and also a working gstreamer-omx 1.10.5 module (including one upstream patch).

A few days ago the Foundation published a RPi specific gstreamer-omx v. 1.10.4 module, which I have also tested together with the bad-plugins from the Raspbian archive (compiled for use with OpenGL).

The easiest way to test if encoding and decoding work is a simple tool chain like the following:

Code: Select all

gst-launch-1.0 -v videotestsrc ! omxh264enc ! h264parse ! omxh264dec ! glimagesink
With my self-compiled version this produces a small video overlay area showing the typical test image (video). This proves that both hardware encoding and decoding are working.

If I use the Foundation version, it doesn't display anything, because the software OpenGL (mesa) driver is too slow. When I enable the alpha OpenGL driver (not the full video driver, but just OpenGL), it does work. But unfortunately things like omxplayer(GUI) don't work any more, so this solution is completely useless for my TV application.

As a next step I created a tool chain to display a real video, in this case a 720p50 TV stream from my video server (based on mumdvb).

Code: Select all

gst-launch-1.0 -v souphttpsrc location="http://localhost:9082/bysid/10301" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0_13ee ! mpegaudioparse ! mpg123audiodec ! audioconvert dithering=0 ! audio/x-raw,channels=2 ! queue ! omxhdmiaudiosink demux.video_0_13ed ! queue ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! queue ! glimagesink
This displays the video in a 720p overlay in the center of the screen and also plays the selected audio channel. Sometimes the audio sounded a bit shaky, but if I don't do anything else it works quite well.

Using the Foundation version with the alpha OpenGL driver enabled, I also get video and sound, but the video runs for about half a second, stops for about 2 seconds, runs again for half a second, stops again and so on. Obiously the OpenGL driver is too slow to display the video in real time. So it looks like the Foundation version is quite useless at the moment.

But I don't want to use gstreamer-omx for video display; I need it for transcoding. This means that I have to feed the output of the decoder into an encoder again. And this is where both versions break: If I try to connect the output of the decoder to anything else besides glimagesink, I get stream errors (usually a "not-negotiated" message).

Result: Both versions don't solve my problem of using gstreamer-omx for transcoding as I have done in the past. I've got an idea what might be going wrong: The decoder output resides in the GPU memory, which cannot be accessed by normal tool chain elements.

As a next step I'll try to contact the gstreamer developers and ask them for a solution.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

mahrk
Posts: 7
Joined: Wed Apr 23, 2014 3:14 pm

Re: gstreamer-omx not working on Stretch

Fri Oct 13, 2017 7:45 pm

Hey there, I have run into the same issue and was wondering if you got anywhere. Thanks.

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Sat Oct 14, 2017 3:20 am

mahrk wrote:
Fri Oct 13, 2017 7:45 pm
Hey there, I have run into the same issue and was wondering if you got anywhere. Thanks.
No, not really. I have just finished my bug report and will publish it within the next days on bugzilla.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

UpNorth
Posts: 4
Joined: Thu Dec 22, 2016 7:00 pm

Re: gstreamer-omx not working on Stretch

Mon Nov 20, 2017 3:08 pm

I have a similar problem. Using the same starting point https://gist.github.com/sphaero/02717b0b35501ad94863 I was able to build GStreamer v1.10 on Jessie with GPU support for transcoding. I changed the branch to 1.12 and tried building it on on Stretch. While it appears to work, there is no GPU acceleration. My test case is simply: gst-launch-1.0 playbin uri=file://someh264videofile.mp4

If anyone comes across a solution, please post.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4829
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: gstreamer-omx not working on Stretch

Thu Dec 21, 2017 1:52 pm

It looks like GStreamer has changed to passing GL buffers around wherever it can. omxvideodec supports it by stuffing an egl_render into the pipe. omxvideoenc can't cope with that.

Code: Select all

$ gst-launch-1.0 -vvv \
> filesrc location=bbb.264 ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! omxh264enc ! fakesink
...
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
vs

Code: Select all

$ gst-launch-1.0 -vvv filesrc location=bbb.264 ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! videoconvert ! omxh264enc ! fakesink
...
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)sRGB, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, format=(string)BGR, interlace-mode=(string)progressive
**
ERROR:gstomxvideoenc.c:1075:gst_omx_video_enc_set_format: code should not be reached
videoconvert appears to be wanting to output RGB which the encoder won't take, but also I believe because videoconvert may support GL buffers on the input, but then can't negotiate correctly on the output.

TBH I'm not totally clear what videoconvert is doing in the pipeline anyway as you want I420 out of omxvideodec, and I420 into omxvideoenc. AIUI you want videoscale to resize the video (I'm guessing that's what you actually want with the new size specified on the OUTPUT of the encoder)
(Annoyingly I've just messed up GST on my Pi, so can't test some stuff at the moment).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Fri Dec 22, 2017 8:14 am

I started with working tool chains from my Jessie applications and just modified the syntax of the tsdemuxer (I had to check the source code to understand the meaning, as there is no documentation).

'videoconvert' is not really needed at all. It's a remnant from my older TV server tool chains and has been removed in my newer rtranscode tool chain. I have also removed it from my test pipelines on Stretch.

"videoscale" was never required. Just defining the video resolution for the output of the encoder always worked for me.

My current problem is, that a soon as I include both omxdecoder and omxencoder into the pipeline, connecting to the video output of the tsdemuxer doesn't work any more ("delayed linking error message"), while the same kind of linking works if I use only the decoder and produce an output with glimagesink.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4829
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: gstreamer-omx not working on Stretch

Fri Dec 22, 2017 11:55 am

gkreidl wrote:
Fri Dec 22, 2017 8:14 am
I started with working tool chains from my Jessie applications and just modified the syntax of the tsdemuxer (I had to check the source code to understand the meaning, as there is no documentation).
Generally gst-inspect-1.0 gives sufficient info, but I'll agree that GStreamer is a beast with little documentation.
gkreidl wrote:'videoconvert' is not really needed at all. It's a remnant from my older TV server tool chains and has been removed in my newer rtranscode tool chain. I have also removed it from my test pipelines on Stretch.

"videoscale" was never required. Just defining the video resolution for the output of the encoder always worked for me.
Relying on magic working within the pipe - eek.
The only way an encoder (or decoder) can resize is by cropping, and that is rarely the desired approach. I don't know how else GStreamer is actioning the resize.
gkreidl wrote:My current problem is, that a soon as I include both omxdecoder and omxencoder into the pipeline, connecting to the video output of the tsdemuxer doesn't work any more ("delayed linking error message"), while the same kind of linking works if I use only the decoder and produce an output with glimagesink.
Simple test cases requried if you really want help. At a minimum:
- a sample TS stream that can be loaded with filesrc (or similar)
- the pipeline you're trying to run
- and if necessary an explanation of what you're trying to achieve in terms of resizing/cropping.
Last edited by 6by9 on Fri Dec 22, 2017 2:24 pm, edited 1 time in total.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Fri Dec 22, 2017 2:03 pm

I'm successfully transcoding real time TV streams (SD and different modes of HD TV) for more than 3 years now, using gstreamer (1.2 on Wheezy, 1.4 on Jessie) and gstreamer-omx 1,0 (after I patched a memory leak and the patch was included into the Foundation repository version). And I'm mostly using lower resolutions for output (and have to for 1080p) and it has always been done by setting the output resolution. And it does not crop but shows the full image. I have no idea how the gstreamer developers implemented this. Here's and example, using omxplayer -i to get the stream details.

Original Stream:

Code: Select all

Stream #0:0[0x3ff]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0x403](deu): Audio: ac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 192 kb/s (clean effects)
Transcoded stream:

Code: Select all

Stream #0:0(ger): Audio: ac3, 48000 Hz, stereo, fltp, 64 kb/s (default)
Stream #0:1(eng): Video: h264 (High), yuv420p, 910x512, SAR 1:1 DAR 455:256, 25 fps, 25 tbr, 1k tbn, 2k tbc (default)
using the following pipeline (which is sent to http-launch):

Code: Select all

souphttpsrc location="http://192.168.0.34:9082/bysid/108" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0403 ! queue ! ac3parse ! a52dec ! audioconvert dithering=0 ! audio/x-raw,channels=2 ! avenc_ac3 compliance=-2 bitrate=65536 ! matroskamux name=stream streamable=true demux. ! queue ! h264parse ! omxh264dec ! omxh264enc target-bitrate=1638400 control-rate=variable ! video/x-h264,stream-format=byte-stream,profile=high,width=1024,height=576,framerate=25/1 ! h264parse ! queue ! stream.
I'll prepare a testing environment after Christmas.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

horai
Posts: 12
Joined: Fri Apr 21, 2017 2:45 pm

Re: gstreamer-omx not working on Stretch

Sat Dec 23, 2017 6:30 pm

Dear all,

I am trying to contribute to this problem, I guess I encountered similar problem, I described it here:
http://gstreamer-devel.966125.n4.nabble ... 85740.html
Did you find out any solution for this matter?

horai
Posts: 12
Joined: Fri Apr 21, 2017 2:45 pm

Re: gstreamer-omx not working on Stretch

Mon Dec 25, 2017 7:20 am

gkreidl wrote:
Sat Sep 23, 2017 9:54 am
After trying lots of different options I have meanwhile been able to build a bad-plugin with full OpenGL ES suppport and also a working gstreamer-omx 1.10.5 module (including one upstream patch).

A few days ago the Foundation published a RPi specific gstreamer-omx v. 1.10.4 module, which I have also tested together with the bad-plugins from the Raspbian archive (compiled for use with OpenGL).

The easiest way to test if encoding and decoding work is a simple tool chain like the following:

Code: Select all

gst-launch-1.0 -v videotestsrc ! omxh264enc ! h264parse ! omxh264dec ! glimagesink
With my self-compiled version this produces a small video overlay area showing the typical test image (video). This proves that both hardware encoding and decoding are working.

If I use the Foundation version, it doesn't display anything, because the software OpenGL (mesa) driver is too slow. When I enable the alpha OpenGL driver (not the full video driver, but just OpenGL), it does work. But unfortunately things like omxplayer(GUI) don't work any more, so this solution is completely useless for my TV application.

As a next step I created a tool chain to display a real video, in this case a 720p50 TV stream from my video server (based on mumdvb).

Code: Select all

gst-launch-1.0 -v souphttpsrc location="http://localhost:9082/bysid/10301" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0_13ee ! mpegaudioparse ! mpg123audiodec ! audioconvert dithering=0 ! audio/x-raw,channels=2 ! queue ! omxhdmiaudiosink demux.video_0_13ed ! queue ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! queue ! glimagesink
This displays the video in a 720p overlay in the center of the screen and also plays the selected audio channel. Sometimes the audio sounded a bit shaky, but if I don't do anything else it works quite well.

Using the Foundation version with the alpha OpenGL driver enabled, I also get video and sound, but the video runs for about half a second, stops for about 2 seconds, runs again for half a second, stops again and so on. Obiously the OpenGL driver is too slow to display the video in real time. So it looks like the Foundation version is quite useless at the moment.

But I don't want to use gstreamer-omx for video display; I need it for transcoding. This means that I have to feed the output of the decoder into an encoder again. And this is where both versions break: If I try to connect the output of the decoder to anything else besides glimagesink, I get stream errors (usually a "not-negotiated" message).

Result: Both versions don't solve my problem of using gstreamer-omx for transcoding as I have done in the past. I've got an idea what might be going wrong: The decoder output resides in the GPU memory, which cannot be accessed by normal tool chain elements.

As a next step I'll try to contact the gstreamer developers and ask them for a solution.
How do you know that your hardware encoder is running? Did you export GST_DEBUG=3 in order to see error? If not, could you please share the way you realized that hardware GPU decoder is running? I had problems running GPU acceleration with repository packages yielding this error "Failed to negotiate RGBA for EGLImage" which I considered as a proof that GPU acceleration is not running.
I only recompiled 1.10.4 OMX package,set it up and the error message "Failed
to negotiate RGBA for EGLImage" is gone, the CPU consumption is still rather
high, but i hope I now have hardware accelerated video decoding, if you know
some other way how to verify the hardware acceleration is running, please,
if you could be so nice and let me know the now. I really need to verify whether GPU acceleration is running.
No I remained only with this FIXMEs.
FIXME default
gstutils.c:3826:gst_pad_create_stream_id_internal:<fakesrc0:src> Creating
random stream-id, consider implementing a deterministic way of creating a
stream-id
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:01.291350354 1037 0xb1e0e660 FIXME rtpjitterbuffer
gstrtpjitterbuffer.c:1488:gst_jitter_buffer_sink_parse_caps:<rtpjitterbuffer0>
Unsupported timestamp reference clock
0:00:01.295135325 1037 0xb1e0e660 FIXME rtpjitterbuffer
gstrtpjitterbuffer.c:1496:gst_jitter_buffer_sink_parse_caps:<rtpjitterbuffer0>
Unsupported media clock


Actually, your test:
gst-launch-1.0 -v videotestsrc ! omxh264enc ! h264parse ! omxh264dec ! glimagesink
yields no error, but consumes up to 80% of CPU via top command.
Isn't this strange?

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Mon Dec 25, 2017 10:33 am

horai wrote:
Mon Dec 25, 2017 7:20 am

Actually, your test:
gst-launch-1.0 -v videotestsrc ! omxh264enc ! h264parse ! omxh264dec ! glimagesink
yields no error, but consumes up to 80% of CPU via top command.
Isn't this strange?
I assume you are using the repository version of both the bad plugins and gstreamer-omx. Both are based on OpenGL.
If you do not use the experimental OpenGL driver, a software replacement is used. That explains the high CPU. It won't work for larger image size at all.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Mon Dec 25, 2017 3:39 pm

Meanwhile I've got something to work. Here's an example pipeline (fed into http-launch) which converts an MPEG2 SD channel to a lower resolution MKV/H264 stream:

Code: Select all

souphttpsrc location="http://localhost:9082/bysid/28106" is-live=true keep-alive=true do-timestamp=true retries=10 typefind=true blocksize=16384 ! tsdemux parse-private-sections=false name=demux demux.audio_0_0066 ! queue ! mpegaudioparse ! matroskamux name=stream streamable=true demux. ! queue ! mpegvideoparse ! omxmpeg2videodec ! videoscale method=0 qos=false add-borders=false sharpen=0 envelope=1 ! video/x-raw,width=480,height=384,framerate=25/1 ! omxh264enc ! h264parse ! queue ! stream.
I've also tested it with 720p50 and 1080i50 HD TV streams. The video scaler is the critical part and may easily use too much CPU (especially with p50 streams).

This uses the default target-bitrate (which isn't good, especially with higher resolutions). As soon as I set omxh264enc properties control-rate and/or target-bitrate, the stream crashes. This is the same bug as discussed in this thread: viewtopic.php?f=29&t=200306&p=1248584.

As soon as this bug is fixed, I could create a Stretch version of my transcoder package. It might not be as efficient as the Jessie release, but it should at least work.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

horai
Posts: 12
Joined: Fri Apr 21, 2017 2:45 pm

Re: gstreamer-omx not working on Stretch

Tue Dec 26, 2017 6:32 am

Hello,

I am using gstreamer-bad plugin from repository, but the omx plugin I built from sources because according to the problem I described thoroughly in previous posts as I had the impression that GPU acceleration is not being used.
Would you be so kind to tell me what is your CPU consumption when running your encoder decoder test and could you also tell me which Raspberry hardware do you posses?
I thought that by recompiling the omx plugin and consequent removal of the error message I fired up the hardware acceleration, but the CPU load is really high. Do you know any other way how to verify that hardware decoding is being used?
Do you suggest also recompiling gstreamer-bad plugin? What linkage is there related in between gstreamer-bad and omx-plugin?

horai
Posts: 12
Joined: Fri Apr 21, 2017 2:45 pm

Re: gstreamer-omx not working on Stretch

Tue Dec 26, 2017 6:34 am

Actually, I also experienced crashes of omxh264enc and fixed it by adding caps to encoder:
omxh264enc ! video/x-h264, profile=(string)high ! h264parse

Hope it helps you as well

gkreidl
Posts: 5457
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: gstreamer-omx not working on Stretch

Tue Dec 26, 2017 12:19 pm

horai wrote:
Tue Dec 26, 2017 6:32 am
Hello,

I am using gstreamer-bad plugin from repository, but the omx plugin I built from sources because according to the problem I described thoroughly in previous posts as I had the impression that GPU acceleration is not being used.
Would you be so kind to tell me what is your CPU consumption when running your encoder decoder test and could you also tell me which Raspberry hardware do you posses?
I thought that by recompiling the omx plugin and consequent removal of the error message I fired up the hardware acceleration, but the CPU load is really high. Do you know any other way how to verify that hardware decoding is being used?
Do you suggest also recompiling gstreamer-bad plugin? What linkage is there related in between gstreamer-bad and omx-plugin?
The bad plugins consist of multiple packages: gstreamer1.0-plugins-bad, gstreamer1.0-plugins-bad-doc, libgstreamer-plugins-bad1.0 and libgstreamer-plugins-bad1.0-dev.
gstreamer-0mx depends on libgstreamer-plugins-bad1.0 only.

I started my transcoding experiments on Stretch before the Foundation published the new gstreamer-omx 1.10.4 version. I also compiled the bad plugins from source (which contains all packages) to use OpenGL ES instead of OpenGL to get accelerated video output (glimagesink) without having to use the experimental OpenGL driver which conflicts with too many things I'm using.

My 3 work systems are all running on Jessie still and I won't upgrade any of them until transcoding works. They are all running from HDDs (I only use the boot sector from the SD cards) and plan a live upgrade to Stretch as soon as I get everything to work.

My mein Pi3 system has two HDDs connected to it, one of them with a Jessie OS, the other one with a Stretch OS, which is currently my experimental system. To switch between them, I have to shut down, replace the SD card and restart. I only do that from time to time, when the services which are running on it are not needed for a while.

Both the omx encoder and the decoder(s) do not fall back to software encoding/decoding. That means that HW acceleration is used if they work at all. You could replace glimagesink by fakesink or activate the experimental OpenGL driver to reduce the CPU performance in the test pipeline. At the moment I cannot test the CPU usage, because I'm working on Jessie.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

horai
Posts: 12
Joined: Fri Apr 21, 2017 2:45 pm

Re: gstreamer-omx not working on Stretch

Tue Dec 26, 2017 6:10 pm

Thank you. I just wanted to know how do you recognize that encoding is not falling to software decoding. I can only see that according to gstreamer debug mode or CPU load, I would like to kindly ask you if there is any file or any other way how to see that gstreamer is definitely using HW GPU decoding (encoding in case of camera), it would be perfect.

Return to “Raspbian”

Who is online

Users browsing this forum: HiassofT and 17 guests