gstreamer openmax


264 posts   Page 4 of 11   1, 2, 3, 4, 5, 6, 7 ... 11
by naplam » Tue Sep 04, 2012 4:24 pm
I have some new results. After some tests with the latest 0.10 I got an mp4 video to play, but very slowly at 1fps or less (no, it's not using the cpu to decode I can only decode mp4 or h264 with the openmax plugins and cpu usage is like 1%).
Code: Select all
 gst-launch-0.10 filesrc location="test.mp4" ! decodebin2 ! ffmpegcolorspace ! fbdevsink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Missing element: MPEG-4 AAC decoder
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 56707649902 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...


I also tried the test.h264 file that comes with the pi again, and after a very long time it renders one frame. I stopped it after that. (again, almost no cpu use, as if there was a deadlock somewhere).

I tried a movie trailer as well (.mp4 file with h264 and aac inside): I get the first black frame and either it's taking forever to fade in or it's not decoding anything else.

At that point i tried the first mp4 file again and got this messge repeateadly (I was using GST_DEBUG=2 this time): "0:00:36.603317141 7898 0x1bcdd50 WARN basevideodecoder gstbasevideodecoder.c:2107:gst_base_video_decoder_alloc_src_frame:<omxmpeg4videodec-omxmpeg4videodec0> failed to get buffer not-linked"

Maybe something was screwed up internally in the GPU at this point because the hello_video sample doesn't work anymore either. -- It can't mount root anymore now after rebooting.. great! the pi living up to expectations, as usual. I'll now have to read it in a computer and try to fix it...
Posts: 43
Joined: Tue Aug 14, 2012 11:04 pm
by naplam » Tue Sep 04, 2012 4:55 pm
Ok.. I've fixed the SD card. Now, after rebooting hello_video works again and I can decode h264 and mp4 with gstreamer and the omx plugins, but only when using playbin2 AND it uses a lot of cpu (it can't decode 1080p in real time like the hello_video sample can with low cpu use- and it uses like 50% cpu for 30fps 300x400 or something like that).

For example in one of the videos (the low-res one) it starts playing ok with decodebin2 (GST_DEBUG=2 gst-launch-0.10 -v filesrc location="test.mp4" ! decodebin2 ! ffmpegcolorspace ! fbdevsink), but halfway through, this happens, always at the same point:
Code: Select all
0:00:02.667443224  1961  0x11c31b0 WARN             omxvideodec gstomxvideodec.c:1477:gst_omx_video_dec_finish:<omxmpeg4videodec-omxmpeg4videodec0> Component does not support empty EOS buffers
Got EOS from element "pipeline0".
Execution ended after 1183670758 ns.
Posts: 43
Joined: Tue Aug 14, 2012 11:04 pm
by dom » Tue Sep 04, 2012 5:13 pm
naplam wrote:Ok.. I've fixed the SD card. Now, after rebooting hello_video works again and I can decode h264 and mp4 with gstreamer and the omx plugins, but only when using playbin2 AND it uses a lot of cpu (it can't decode 1080p in real time like the hello_video sample can with low cpu use- and it uses like 50% cpu for 30fps 300x400 or something like that).


I guess oprofile can tell where the time is spent. Probably ffmpegcolorspace.
Note that hello_video creates a sequence of OpenMAX componenents with tunneling between them, so the ARM doesn't have to handle the decoded video data. I don't know if gstreamer allows that.

It should certainly be possible for gstreamer to create a video_render component which can be connected to video_decode which means the ffmpegcolorspace shouldn't be necessary. (Although I know nothing about gstreamer).
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5083
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by naplam » Tue Sep 04, 2012 6:36 pm
dom wrote:
naplam wrote:Ok.. I've fixed the SD card. Now, after rebooting hello_video works again and I can decode h264 and mp4 with gstreamer and the omx plugins, but only when using playbin2 AND it uses a lot of cpu (it can't decode 1080p in real time like the hello_video sample can with low cpu use- and it uses like 50% cpu for 30fps 300x400 or something like that).


I guess oprofile can tell where the time is spent. Probably ffmpegcolorspace.
Note that hello_video creates a sequence of OpenMAX componenents with tunneling between them, so the ARM doesn't have to handle the decoded video data. I don't know if gstreamer allows that.

It should certainly be possible for gstreamer to create a video_render component which can be connected to video_decode which means the ffmpegcolorspace shouldn't be necessary. (Although I know nothing about gstreamer).

Yeah, I've heard ffmpegcolorspace is particularly slow and lacking optimization. Anyway, I'm more interested in encoding so decoding troubles are not a big issue. I just wanted to get it working to see if openmax encoding could be done easily with gstreamer, because gst-omx is supposed to be more or less adaptable to different vendors using openmax. But that seems not to be the case -if I have to modify the source code, gstreamer's supposed advantage is no more, so I think I'll try to find openmax encoding code somewhere. I have code from several vendors, mostly for decoding, but not from Broadcom for encoding. Do you know where I could find some code for openmax encoding on Broadcom videocore?
Posts: 43
Joined: Tue Aug 14, 2012 11:04 pm
by gkiagia » Wed Sep 05, 2012 8:20 am
naplam wrote:legacyh264parse is what I get when I clone the 0.10 branch for gst-plugins-bad:
Code: Select all
grep legacy gst/h264*/*
gst/h264parse/gsth264parse.c:  GST_DEBUG_CATEGORY_INIT (h264_parse_debug, "legacy h264parse", 0,
gst/h264parse/gsth264parse.c:      "legacy h264 parser");
gst/h264parse/gsth264parse.c:  return gst_element_register (plugin, "legacyh264parse",

Isn't everybody here using the 0.10 package that comes with raspbian? I have compiled the 0.10 current version now (not the latest tagged release) and it still doesn't work. What am I supposed to do to make this to work? I tried 0.11 at first and it didn't work either.


h264parse is in videoparsersbad from gst-plugins-bad:

Code: Select all
% gst-inspect-0.10 | grep h264parse                                             
videoparsersbad:  h264parse: H.264 parser
h264parse:  legacyh264parse: H264Parse
Posts: 6
Joined: Mon Sep 03, 2012 9:24 am
by gkiagia » Wed Sep 05, 2012 8:30 am
naplam wrote:For example in one of the videos (the low-res one) it starts playing ok with decodebin2 (GST_DEBUG=2 gst-launch-0.10 -v filesrc location="test.mp4" ! decodebin2 ! ffmpegcolorspace ! fbdevsink), but halfway through, this happens, always at the same point:
Code: Select all
0:00:02.667443224  1961  0x11c31b0 WARN             omxvideodec gstomxvideodec.c:1477:gst_omx_video_dec_finish:<omxmpeg4videodec-omxmpeg4videodec0> Component does not support empty EOS buffers
Got EOS from element "pipeline0".
Execution ended after 1183670758 ns.

This is a normal condition that happens when the input ends (EOS = end of stream). The warning there shouldn't really be a warning imho, it's because the element is configured with the "no-empty-eos-buffer" hack.
Posts: 6
Joined: Mon Sep 03, 2012 9:24 am
by gkiagia » Wed Sep 05, 2012 8:43 am
dom wrote:I guess oprofile can tell where the time is spent. Probably ffmpegcolorspace.
Note that hello_video creates a sequence of OpenMAX componenents with tunneling between them, so the ARM doesn't have to handle the decoded video data. I don't know if gstreamer allows that.

GStreamer cannot tunnel OpenMAX components, as it needs to be able to control the flow itself. It could, however, pass around pointers to GPU buffers from one component to another. This is not something that is doable with the OpenMAX standard, though, so it may require modifications to the OpenMAX implementation and of course input from its developers.

ffmpegcolorspace, fbdevsink and the ~3 copies of each buffer that happen in the pipeline flow are for sure enough cause for low performance.

dom wrote:It should certainly be possible for gstreamer to create a video_render component which can be connected to video_decode which means the ffmpegcolorspace shouldn't be necessary. (Although I know nothing about gstreamer).

That should be possible, but the components cannot be tunnelled (which means we won't avoid the 3 buffer copies). It needs a bit of work, though, and it is a bit crappy imho, as it doesn't integrate with windowing systems.
Posts: 6
Joined: Mon Sep 03, 2012 9:24 am
by fbutler » Wed Sep 05, 2012 6:36 pm
Hi,

I'm trying to stream the hello-video test.h264 file to the display using the following pipeline:

gst-launch-0.10 -v --gst-debug-level=2 filesrc location="./test.h264" ! h264parse ! omxh264dec ! ffmpegcolorspace ! fbdevsink sync=false

It displays the first frame and then stops and starts showing "sync point doesn't have timestamp" warnings.

The video plays fine using both hello_video and omxplayer.

Am I missing some required element/s from the pipeline, and if so does anyone know what it/they are?

Here are the warnings that I am seeing:
Code: Select all
 0:00:00.586440209  6279  0x115e6c0 WARN                     omx gstomx.c:2018:gst_omx_parse_hacks: Unknown hack: bcm-host
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, width=(int)1920, height=(int)1080, parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, width=(int)1920, height=(int)1080, parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false
/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:src: caps = video/x-raw-rgb, bpp=(int)16, depth=(int)16, endianness=(int)1234, red_mask=(int)63488, green_mask=(int)2016, blue_mask=(int)31, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false
/GstPipeline:pipeline0/GstFFMpegCsp:ffmpegcsp0.GstPad:sink: caps = video/x-raw-yuv, format=(fourcc)I420, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false
/GstPipeline:pipeline0/GstFBDEVSink:fbdevsink0.GstPad:sink: caps = video/x-raw-rgb, bpp=(int)16, depth=(int)16, endianness=(int)1234, red_mask=(int)63488, green_mask=(int)2016, blue_mask=(int)31, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, pixel-aspect-ratio=(fraction)1/1, interlaced=(boolean)false
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:04.223618658  6279  0x115e6c0 WARN                     bin gstbin.c:2395:gst_bin_do_latency_func:<pipeline0> did not really configure latency of 0:00:00.000000000
New clock: GstSystemClock
0:00:04.318143629  6279  0x1178bb0 WARN        basevideodecoder gstbasevideodecoder.c:1433:gst_base_video_decoder_prepare_finish_frame:<omxh264dec-omxh264dec0> sync point doesn't have timestamp
0:00:05.803172042  6279  0x1178bb0 WARN        basevideodecoder gstbasevideodecoder.c:1433:gst_base_video_decoder_prepare_finish_frame:<omxh264dec-omxh264dec0> sync point doesn't have timestamp
0:00:05.870051899  6279  0x1178bb0 WARN        basevideodecoder gstbasevideodecoder.c:1433:gst_base_video_decoder_prepare_finish_frame:<omxh264dec-omxh264dec0> sync point doesn't have timestamp
I'm new to gstreamer so any guidance would be greatly appreciated.
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by dom » Wed Sep 05, 2012 6:45 pm
fbutler wrote:I'm trying to stream the hello-video test.h264 file to the display using the following pipeline:

gst-launch-0.10 -v --gst-debug-level=2 filesrc location="./test.h264" ! h264parse ! omxh264dec ! ffmpegcolorspace ! fbdevsink sync=false

It displays the first frame and then stops and starts showing "sync point doesn't have timestamp" warnings.

Well, test.h264 is a raw h264 bitstream, and as such doesn't have timestamps. You need to put it in a container file to get timestamps. Perhaps your pipeline expects a container file?
Can you try with an mp4 file?
http://www.bigbuckbunny.org/index.php/download/
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5083
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by fbutler » Wed Sep 05, 2012 8:37 pm
dom wrote:Well, test.h264 is a raw h264 bitstream, and as such doesn't have timestamps. You need to put it in a container file to get timestamps. Perhaps your pipeline expects a container file?
Can you try with an mp4 file?
http://www.bigbuckbunny.org/index.php/download/


I've tried it with big_buck_bunny_480p_h264.mov and the following pipe and nothing is displayed:

gst-launch-0.10 -v --gst-debug-level=2 filesrc location="./big_buck_bunny_480p_h264.mov" ! h264parse ! omxh264dec ! ffmpegcolorspace ! fbdevsink sync=false

From the warnings I'm guessing that I'm still using the correct file format.

I've sucessfully used the h264parse and omxh264dec elements by feeding the pipe from a program capturing live H264 video from a Logitech c920 webcam, although that has some artifacts during play. However the same artifacts are present when the program captures direct to disk. In that case I wonder if the transfer rate from the webcam in H264 mode is too much for the USB? The artifacts are present even when capturing at a low resolution with a low framerate

The warnings playing the big_buck_bunny_480p_h264.mov file are:
Code: Select all
0:00:00.436301132 ^[[336m 2093^[[00m  0x18728c0 ^[[33;01mWARN   ^[[00m ^[[00m                 omx gstomx.c:2018:gst_omx_parse_hacks:^[[00m Unknown hack: bcm-host
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
0:00:00.824272799 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m   codecparsers_h264 gsth264parser.c:1433:gst_h264_parse_sps:^[[00m failed to read UE
0:00:00.824678786 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m   codecparsers_h264 gsth264parser.c:1556:gst_h264_parse_sps:^[[00m error parsing "Sequence parameter set"

** (gst-launch-0.10:2093): CRITICAL **: nal_reader_get_ue: assertion `i <= 32' failed
0:00:00.825421762 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m   codecparsers_h264 gsth264parser.c:1433:gst_h264_parse_sps:^[[00m failed to read UE
0:00:00.825640755 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m   codecparsers_h264 gsth264parser.c:1556:gst_h264_parse_sps:^[[00m error parsing "Sequence parameter set"
0:00:00.825997744 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m   codecparsers_h264 gsth264parser.c:1756:gst_h264_parser_parse_slice_hdr:^[[00m couldn't find associated picture parameter set with id: 0
0:00:00.826274735 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m   codecparsers_h264 gsth264parser.c:1756:gst_h264_parser_parse_slice_hdr:^[[00m couldn't find associated picture parameter set with id: 22
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au
0:01:32.060339677 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m           h264parse gsth264parse.c:727:gst_h264_parse_check_valid_frame:<h264parse0>^[[00m input stream is corrupt; it contains a NAL unit of length 1
0:01:32.067109463 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m           h264parse gsth264parse.c:727:gst_h264_parse_check_valid_frame:<h264parse0>^[[00m input stream is corrupt; it contains a NAL unit of length 1
0:01:32.072051306 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m         omxvideodec gstomxvideodec.c:1539:gst_omx_video_dec_drain:<omxh264dec-omxh264dec0>^[[00m Component does not support empty EOS buffers
0:01:32.191894496 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m           h264parse gsth264parse.c:727:gst_h264_parse_check_valid_frame:<h264parse0>^[[00m input stream is corrupt; it contains a NAL unit of length 1
0:01:32.194267421 ^[[336m 2093^[[00m  0x189ae60 ^[[33;01mWARN   ^[[00m ^[[00m           h264parse gsth264parse.c:727:gst_h264_parse_check_valid_frame:<h264parse0>^[[00m input stream is corrupt; it contains a NAL unit of length 1
I'll try and see if I can work it out tomorrow night.
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by naplam » Wed Sep 05, 2012 9:29 pm
fbutler wrote: In that case I wonder if the transfer rate from the webcam in H264 mode is too much for the USB? The artifacts are present even when capturing at a low resolution with a low framerate

On the contrary, h264 data is compressed and the amount of data will be much less than using yuv directly (or even mjpeg). Maybe some of the data is being corrupted (presumably because of the crappy SD driver, USB driver and/or USB IP in the Broadcom chip) - I haven't tested much with the latest kernel but previously my webcam stopped streaming after a minute or so, and I'm still having 50%+ frames dropped on a USB wireless NIC.

About gstreamer itself, I've had mixed results (mostly bad) when playing videos using h264parse ! omxh264dec, but it works with playbin2 or decodebin2. Try playbin2 or decodebin2, even the test.h264 file should work with that.
Posts: 43
Joined: Tue Aug 14, 2012 11:04 pm
by fbutler » Thu Sep 06, 2012 6:08 am
naplam wrote:On the contrary, h264 data is compressed and the amount of data will be much less than using yuv directly (or even mjpeg). Maybe some of the data is being corrupted (presumably because of the crappy SD driver, USB driver and/or USB IP in the Broadcom chip) - I haven't tested much with the latest kernel but previously my webcam stopped streaming after a minute or so, and I'm still having 50%+ frames dropped on a USB wireless NIC.


Interesting, I think you may be correct there. Using yuv from the C920 cam produces a clear but jerky video whatever the framerate is set to , whereas H264 seems to run at full framerate, even 30fps, and is smooth, but with the artifacts. I've noticed that H264 support appears to be available for v4l2src in release 11.91 of the plugins-good package , however debian wheezy is currently only up to release 10.31-3. It would be good to be able to use v4l2src with H264 to eliminate the capture program from being the cause of the artifacts, however I do suspect that the artifacts are another symptom of an underlying USB driver issue with high speed transfers, even with all of the current fixes applied.

naplam wrote:About gstreamer itself, I've had mixed results (mostly bad) when playing videos using h264parse ! omxh264dec, but it works with playbin2 or decodebin2. Try playbin2 or decodebin2, even the test.h264 file should work with that.

I've just tried test.h264 with decodebin2, and got the same timestamp issues. The good news is that both big_buck_bunny_480p_h264.mov and big_buck_bunny_480p_surround-fix.avi played using decodebin2, so some more progress there :D . The bad news is that playing them consumed over 90% CPU. :o

I'll try out some more tests tonight.
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by fbutler » Fri Sep 07, 2012 6:49 am
Some progress.

I've managed to get the big_buck_bunny_480p_h264.mov file to play without using decodebin2 with both video, using omxh264dec, and audio, using alsa :) with the following pipeline:

gst-launch-0.10 filesrc location=big_buck_bunny_480p_h264.mov ! qtdemux name=demuxer \ demuxer. ! queue ! faad ! audioconvert ! alsasink device=hw:0,0 sync=false \ demuxer. ! h264parse ! omxh264dec ! ffmpegcolorspace ! fbdevsink

The video plays mostly smoothly, but the audio is very choppy and CPU usage is ~94%. Sometimes the audio drops out during play, or both the audio and video stop, however it is a good step forward into looking at this.

If you try this and don't hear any audio try forcing the audio output channel using:

sudo amixer cset numid=3 <n> where n is 0=auto, 1=headphones, 2=hdmi.

Now to work out what's devouring the CPU resource.
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by Richard_P » Sun Sep 09, 2012 12:34 pm
From what I have seen, the fbdevsink is one the main culprits for hogging the CPU. The decoded video *SHOULD NOT* be coming back in to userspace, but decoded and displayed directly on to video surface at the OMX layer.

I don't know how omxplayer does it, perhaps someone could enhance the Gstreamer plugin to include a 'LIVE' option that just sends data to the Video and in that case a fakesink can be used to terminate the pipeline.

Richard
Posts: 39
Joined: Mon Jun 11, 2012 10:43 am
by fbutler » Sun Sep 09, 2012 1:21 pm
Richard_P wrote:From what I have seen, the fbdevsink is one the main culprits for hogging the CPU. The decoded video *SHOULD NOT* be coming back in to userspace, but decoded and displayed directly on to video surface at the OMX layer.

I don't know how omxplayer does it, perhaps someone could enhance the Gstreamer plugin to include a 'LIVE' option that just sends data to the Video and in that case a fakesink can be used to terminate the pipeline.

Richard


It's mainly ffmpegcolorspace that's consuming it.
I installed oprofile and here's what it shows:

Code: Select all
Every 1.0s: opcontrol --dump && opreport -l 2>/dev/null | head -30 ; opcontrol --reset                           Sun Sep  9 14:15:33 2012

CPU: CPU with timer interrupt, speed 1347 MHz (estimated)
Profiling through timer interrupt
samples  %        image name               app name                 symbol name
145      33.4873  libgstffmpegcolorspace.so libgstffmpegcolorspace.so /usr/lib/arm-linux-gnueabihf/gstreamer-0.10/libgstffmpegcolorspace.
so
74       17.0901  no-vmlinux               no-vmlinux               /no-vmlinux
44       10.1617  libcofi_rpi.so           libcofi_rpi.so           mis_2_loop
40        9.2379  libfaad.so.2.0.0         libfaad.so.2.0.0         /usr/lib/arm-linux-gnueabihf/libfaad.so.2.0.0
37        8.5450  libgstaudioconvert.so    libgstaudioconvert.so    /usr/lib/arm-linux-gnueabihf/gstreamer-0.10/libgstaudioconvert.so
24        5.5427  libcofi_rpi.so           libcofi_rpi.so           fast_loop
12        2.7714  libc-2.13.so             libc-2.13.so             /lib/arm-linux-gnueabihf/libc-2.13.so
8         1.8476  libm-2.13.so             libm-2.13.so             /lib/arm-linux-gnueabihf/libm-2.13.so
5         1.1547  ld-2.13.so               ld-2.13.so               /lib/arm-linux-gnueabihf/ld-2.13.so
5         1.1547  libglib-2.0.so.0.3200.3  libglib-2.0.so.0.3200.3  /lib/arm-linux-gnueabihf/libglib-2.0.so.0.3200.3
4         0.9238  libcofi_rpi.so           libcofi_rpi.so           memcpy
4         0.9238  libcofi_rpi.so           libcofi_rpi.so           post_misalignment_2_loop
3         0.6928  libgstbase-0.10.so.0.30.0 libgstbase-0.10.so.0.30.0 /usr/lib/arm-linux-gnueabihf/libgstbase-0.10.so.0.30.0
3         0.6928  libgstreamer-0.10.so.0.30.0 libgstreamer-0.10.so.0.30.0 /usr/lib/arm-linux-gnueabihf/libgstreamer-0.10.so.0.30.0
3         0.6928  libpthread-2.13.so       libpthread-2.13.so       __pthread_mutex_unlock_usercnt
3         0.6928  libpthread-2.13.so       libpthread-2.13.so       pthread_mutex_lock
2         0.4619  libcofi_rpi.so           libcofi_rpi.so           tail_fast_loop
2         0.4619  libgstfbdevsink.so       libgstfbdevsink.so       /usr/lib/arm-linux-gnueabihf/gstreamer-0.10/libgstfbdevsink.so
2         0.4619  libgstopenmax.so         libgstopenmax.so         gst_omx_video_dec_fill_buffer
2         0.4619  libpthread-2.13.so       libpthread-2.13.so       pthread_getspecific
1         0.2309  [vectors] (tgid:5901 range:0xffff0000-0xffff1000) dash                     [vectors] (tgid:5901 range:0xffff0000-0xffff
1000)
1         0.2309  init                     init                     /sbin/init
1         0.2309  libcofi_rpi.so           libcofi_rpi.so           full_out
1         0.2309  libcofi_rpi.so           libcofi_rpi.so           main_loop
1         0.2309  libcofi_rpi.so           libcofi_rpi.so           post_misalignment_2
1         0.2309  libgstopenmax.so         libgstopenmax.so         FillBufferDone
1         0.2309  libpthread-2.13.so       libpthread-2.13.so       sem_post@@GLIBC_2.4
Signalling daemon... done
As you say, hopefully someone with the required knowledge will be able to improve this in the Gstreamer OMX plug-in.
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by wlk » Fri Sep 21, 2012 5:58 am
Hi,
I am also fighting with gst-omx. Right now, I am able to decode mp4 files. big_buck_bunny_480p_h264.mov is played on fbdevsink without problems, but encoding is a different story. After some tweaking with command line and gstomx.conf I am able to run gstreamer without runtime errors (v4l2src -> ..... omxh264enc ... udpsink), but no h264 stream is sent to the network. I will post command line, logs and gstomx.conf when I will be back home.
And the last issue: I am using USB webcam configured to 320x200@30fps. It plays smoothly on fbdevsink (raw stream) only when I overclock CPU to >=800MHz.
Any ideas about encoding? Was anyone successful using OpenMAX to encode?
Posts: 1
Joined: Fri Sep 21, 2012 5:27 am
by nx5 » Fri Sep 21, 2012 9:39 am
wlk wrote:Any ideas about encoding? Was anyone successful using OpenMAX to encode?

I tried but I had the same problem as you, it won't encode anything. We probably need some modifications to the gst-omx code. About the usb issues I recommend that you follow this topic:
viewtopic.php?f=28&t=5249&p=178366#p178366

However, recording from a webcam and stuff like that, anything that depends on reliable usb traffic or reliable sd card driver behaviour, won't work properly on the pi. I've sold mine on ebay for more than it cost me. Getting trolled by the fanboys for stating the obvious and eventually banned for a week for stating the truth on this forum (just a couple of posts, no propaganda effort on my part) was the tipping point. Goodbye Raspberry Pi.
Posts: 6
Joined: Wed Sep 12, 2012 9:16 pm
by MattSwarbrick » Fri Oct 05, 2012 4:40 pm
Hey,

I'm completely new to gstreamer, but very eager to learn.
So I'm trying to get gstreamer to use the gst-omx plugin to play video via hardware acceleration...
Currently I'm experiencing the same problems as previous posters, but have got quite far, when I try to use to gst-launch so that:

Code: Select all
gst-launch-0.10 playbin2 uri=file:///root/test.h264


I've set GST_DEBUG to "*omx*:5" so that I see what is going on with everything omx and it says that it fails to initialise the core as what other people have experienced:

Code: Select all
omxvideodec gstomxvideodec.c:258:gst_omx_video_dec_open:<omxh264dec-omxh264dec0> Opening decoder
0:00:01.412601175  2567   0x9d5860 DEBUG                    omx gstomx.c:77:gst_omx_core_acquire: Successfully loaded core '/opt/vc/lib/libopenmaxil.so'
0:00:01.415832127  2567   0x9d5860 ERROR                    omx gstomx.c:87:gst_omx_core_acquire: Failed to initialize core '/opt/vc/lib/libopenmaxil.so': 0x80001009
Missing element: H.264 decoder


and thus fails, I'm using the test.h264 full HD so that if it works it works and i know it works using hardware acceleration, any help would be great as I'm beginning to understand how it works,

Matt
Posts: 27
Joined: Thu Oct 04, 2012 3:15 pm
by fbutler » Sat Oct 06, 2012 6:51 am
MattSwarbrick wrote:
Code: Select all
Missing element: H.264 decoder


What does the command: gst-inspect-0.10 | grep omx show?

Here's the complete list of commands I used to successfully install gstreamer with gst-omx on a fresh Raspian Wheezy install in case this helps you:

Code: Select all
# Get and install  rpi-update if necessary
sudo wget http://goo.gl/1BOfJ -O /usr/bin/rpi-update && sudo chmod +x /usr/bin/rpi-update
sudo apt-get -y install git-core

# Upgrade to the latest packages and firmware
sudo apt-get update
sudo apt-get upgrade -y
sudo rpi-update
sudo reboot

# Add the qt50-snapshot to the sources list and install it
sudo sh -c 'echo deb http://archive.qmh-project.org/rpi-wheezy/debian unstable main >> /etc/apt/sources.list'
sudo apt-get update
sudo apt-get -y --force-yes install qt50-snapshot

# Get gst-omx source
cd $HOME
git clone -b raspberry git://anongit.freedesktop.org/gstreamer/gst-omx

# Install the Gstreamer packages, and the packages required to build omx
sudo apt-get install -y autoconf gtk-doc-tools libtool

# Autogenerate the configure script, configure, make and install gst-omx
cd gst-omx
./autogen.sh --noconfigure
./configure --prefix=/home/pi/omx
make
make install

# Set up the gst-omx environment for the pi user
cp  omx/gstomx-raspberry.conf $HOME/omx/lib/gstreamer-0.10/gstomx.conf
cd $HOME
echo -e \\n# Gstreamer environment >> .profile
echo export GST_PLUGIN_PATH=$HOME/omx/lib/gstreamer-0.10/ >> .profile
echo export GST_OMX_CONFIG_DIR=$HOME/omx/lib/gstreamer-0.10/ >> .profile
echo export LD_LIBRARY_PATH=$HOME/omx/lib/gstreamer-0.10/ >> .profile
. ./.profile

# Install the GStreamer Tools
sudo apt-get install gstreamer0.10-tools

# Verify that gst-omx has been installed correctly. If it has the following command should show these plug-ins:
# openmax:  omxmpeg4videodec: OpenMAX MPEG4 Video Decoder
# openmax:  omxh264dec: OpenMAX H.264 Video Decoder
gst-inspect-0.10 | grep omx

# List all of the currently installed GStreamer plug-ins and features
gst-inspect-0.10

# A list detailing other GGtreamer plug-ins and what packages they are in is available at: http://gstreamer.freedesktop.org/documentation/plugins.html
# Install  a basic core of Gstreamer plug-ins
sudo apt-get install -y gstreamer0.10-plugins-base  gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-ugly gstreamer0.10-alsa

# List all of the currently installed GStreamer plug-ins and features
gst-inspect-0.10 | more

# Install the v4l-utils to be able to use the v4l2-ctl command to configure v4l2 input devices, such as webcams, if required
sudo apt-get install v4l-utils
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by m][sko » Mon Oct 08, 2012 7:26 am
Hi
it looks like with latest update (firmware and rapsbian)
gst-omx don't work

gst-omx lock in preloading

any idea how to fix it?
Posts: 97
Joined: Fri Jul 20, 2012 6:37 am
Location: Slovakia
by MattSwarbrick » Mon Oct 08, 2012 10:18 am
fbutler wrote:
MattSwarbrick wrote:
Code: Select all
Missing element: H.264 decoder

What does the command: gst-inspect-0.10 | grep omx show?


Thanks for your reply, the gst-inspect-0.10 | grep omx shows:

Code: Select all
root@raspberrypi:~# gst-inspect-0.10 |grep omx
openmax:  omxmpeg4videodec: OpenMAX MPEG4 Video Decoder
openmax:  omxh264dec: OpenMAX H.264 Video Decoder


Did you manage to play the test.h264 video?

I'll have a look at your instructions, they seem similar but slightly different to what I've done. Which may provide the difference needed!
Posts: 27
Joined: Thu Oct 04, 2012 3:15 pm
by fbutler » Mon Oct 08, 2012 1:26 pm
MattSwarbrick wrote:Did you manage to play the test.h264 video?

No, it had errors as per my posting here:
viewtopic.php?p=166842#p166842.

The errors were because it was a raw file. Dom suggested using the big_buck_bunny movies and I got it working with them.

The command that I used is shown in one of the threads above, and that gave me both video and choppy audio
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by MattSwarbrick » Mon Oct 08, 2012 2:30 pm
I followed your instructions step by step and am using the "big_buck_bunny_480p_surround-fix.avi" file but seems to hang on prerolling, did you find that before? How did you get around that to be able to play some video output? Thanks for your help btw, seem to be a bit closer, but still feel far away!
Posts: 27
Joined: Thu Oct 04, 2012 3:15 pm
by fbutler » Mon Oct 08, 2012 2:56 pm
MattSwarbrick wrote:I followed your instructions step by step and am using the "big_buck_bunny_480p_surround-fix.avi" file but seems to hang on prerolling, did you find that before? How did you get around that to be able to play some video output? Thanks for your help btw, seem to be a bit closer, but still feel far away!

I had lots of various issues before getting it to finally work, most of which were down to learning. I found setting the debugging level to 2 to be useful in working out what was wrong:

If you run it with:

gst-launch-0.10 -v --gst-debug-level=2

are there any messages shown?

I've just tried running one of my previously working pipelines and I'm getting the following messages:

Code: Select all
gst-launch-0.10  --gst-debug-level=2 filesrc location=big_buck_bunny_480p_h264.mov ! qtdemux name=demuxer \ demuxer. ! queue ! faad ! audioconvert ! alsasink device=hw:0,0 sync=false \ demuxer. ! h264parse ! omxh264dec ! ffmpegcolorspace ! fbdevsink
0:00:00.384496280  2613  0x1d4e240 WARN                 default ./grammar.y:889:priv_gst_parse_yyerror: Error during parsing: syntax error, unexpected $undefined
0:00:00.479506821  2613  0x1d4e240 WARN                 default ./grammar.y:889:priv_gst_parse_yyerror: Error during parsing: syntax error, unexpected $undefined
Setting pipeline to PAUSED ...
0:00:00.777840800  2613  0x1d4e240 ERROR                    omx gstomx.c:446:gst_omx_component_new:<omxh264dec-omxh264dec0> Failed to get component handle 'OMX.broadcom.video_decode' from core '/opt/vc/lib/libopenmaxil.so': 0x80001000
ERROR: Pipeline doesn't want to pause.
Setting pipeline to NULL ...
Freeing pipeline ...

I'm not sure what is causing this yet. It could be that something in my environment has changed since I last tried it a few weeks ago, or it may be something else that has changed with the various firmware updates etc in the meantime. I'll investigate this further when I have some free time tonight.
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England
by fbutler » Mon Oct 08, 2012 4:28 pm
m][sko wrote:Hi
it looks like with latest update (firmware and rapsbian)
gst-omx don't work

gst-omx lock in preloading

any idea how to fix it?


What's the full error message you get with --gst-debug-level=2
User avatar
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England