RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 9:49 am

Hi Folks,
we are trying to convert live stream (for example, live RTSP source) to H264 with gstreamer omxh264enc encoder.

With GST_DEBUG we receive a lot of:
omxvideoenc gstomxvideoenc.c:626:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> No corresponding frame found
And results on the media player receiver is a sort of broken video in the bottom.

After hundreds of tests, we discover that this problem is present ONLY when forcing parameter "target-bitrate".
For example:
GST_DEBUG=3 gst-launch-1.0 rtsp://192.168.1.1 ! rtph264depay ! h264parse ! omxh264dec ! omxh264enc target-bitrate=5000000 control-rate=variable ! rtph264pay pt=96 ! udpsink host=192.168.1.100 port=15000 sync=false async=false
If you use a lower value for target-bitrate, phenomenon is less frequent (for example with 500000 - 500kbit). Otherwise, if you increase this value phenomenon is very present (like previous example, with 5000000 - 5Mbit).

With this pipeline we don't have this problem:
GST_DEBUG=3 gst-launch-1.0 rtsp://192.168.1.1 ! rtph264depay ! h264parse ! omxh264dec ! omxh264enc ! rtph264pay pt=96 ! udpsink host=192.168.1.100 port=15000 sync=false async=false
But in this second case we don't have VBR under our control: bitrate is very variable, starting from 1 kb/s, with poor final results.

This is our configuration:
  • Raspbian Stretch totally upgraded
  • rpi-udate at last version
  • Gstreamer 1.0 with all plugins and gstreamer1.0-omx-rpi (stock packages)
  • /boot/config.txt with gpumem=128

Final effect is similar to this:
Image

Described in this question:
https://stackoverflow.com/questions/295 ... spberry-pi

Any help is appreciated.

Bye

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 10:49 am

UPDATE:

The same condition is verified with FFMPEG recompiled with omx support.
ffmpeg version N-89088-gce001bb8fc Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1) 20170516
configuration: --arch=armel --target-os=linux --enable-gpl --enable-omx --enable-omx-rpi --enable-nonfree --enable-libmp3lame --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-libxcb-shape --enable-mmal

Forcing bitrate target, the results are similars to previous message. For example with this command line:

Code: Select all

./ffmpeg -re -c:v h264_mmal -i rtsp://192.168.1.1/  -c:v h264_omx -vprofile baseline -an -b:v 1500k -f rtp rtp://192.168.1.100:15000

Very frustratring.

Bye,
Raoul

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 11:22 am

So what bitrate is GST defaulting to?

My immediate guess is that it's linked to the Pi video encoder being happy to return a frame split over multiple buffers, with only the last flagged with OMX_BUFFERFLAG_ENDOFFRAME (as is permitted by the IL spec). That is why the default output buffer size is only 64kB.

A quick check in the gst-omx source at https://github.com/GStreamer/gst-omx/tree/master/omx would imply that it sets OMX_BUFFERFLAG_ENDOFFRAME on the decoder input buffers, but can't be bothered looking at them on the encoded output.
Slapping a

Code: Select all

port_def.nBufferSize= 512<<10;
to https://github.com/GStreamer/gst-omx/bl ... enc.c#L539 may help you out at the expense of a bundle of memory. (Add a log message of nBufferCountActual at that point to determine how much you're actually looking at - default is 1, so not too bad).

ffmpeg appears to be at least looking for the ENDOFFRAME flags, so it's more difficult to say what you're seeing as you haven't given any logs.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 5552
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 11:55 am

On Raspbian Jessie it should work. I do a lot of transcoding, but didn't get it to work on Stretch yet.
viewtopic.php?f=66&t=193152
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 1:41 pm

6by9, thanks for the reply.

It's difficult for us to retrieve default bitrate of gstreamer because we are using default stretch package. We did'nt found an effective debug configuration to retrieve this information.
Empirically, (through VLC client statistics) seems to be a minimum of 64kbit (with static images) and heavy VBR activity (up to 2Mbit) with a lot of movements in the scene. Unfortunately, in this default configuration and scenes with "normal" movements, video quality in output is very poor.

We also fails to recompile gstreamer on stretch with omx support (we have a problem with h264parse plugin that is not found after recompilation). If you suggest to modify gstomxh264enc.c, we need first of all to achieve this goal.

In this moment, (with --gst-debug=omxvideoenc:5) the only useful logs are:
0:21:22.613191072 1699 0x6f0ea4f0 ERROR omxvideoenc gstomxvideoenc.c:626:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> No corresponding frame found
0:21:24.652042381 1699 0x6f0ea4f0 ERROR omxvideoenc gstomxvideoenc.c:626:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> No corresponding frame found

We have done a lot of research on the web and we can not understand why nobody reports this kind of problem which practically nullifies the possibility of transcoding with raspberry.
Can it be caused by our stupid mistake?

gkreidl, we think that our problem is different. We are successfully using gstreamer and omx support, installed from standard stretch repository.

Do you think that downgrading to jessie could be resolve our problems? Do you tried to use bitrate target in live transcoding?

For your convenience, this is the list of our installed packages (all in 1.10.4 versions including gstreamer1.0-omx-rpi):
i gstreamer1.0-alsa GStreamer plugin for ALSA 1.10.4-1
i gstreamer1.0-doc GStreamer core documentation and manuals 1.10.4-1
i gstreamer1.0-libav libav plugin for GStreamer 1.10.4-1
i gstreamer1.0-libav-dbg libav plugin for GStreamer (debug symbols) 1.10.4-1
i gstreamer1.0-omx GStreamer OpenMAX plugins 1.10.4-1+rpt2
i gstreamer1.0-omx-rpi OpenMax plugins for GStreamer 1.10.4-1+rpt2
i gstreamer1.0-omx-rpi-config OpenMax plugins for GStreamer 1.10.4-1+rpt2
i gstreamer1.0-omx-rpi-dbgsym Debug symbols for gstreamer1.0-omx-rpi 1.10.4-1+rpt2
i gstreamer1.0-plugins-bad GStreamer plugins from the "bad" set 1.10.4-1
i gstreamer1.0-plugins-bad-dbg GStreamer plugins from the "bad" set (debug symbols) 1.10.4-1
i gstreamer1.0-plugins-bad-doc GStreamer documentation for plugins from the "bad" set 1.10.4-1
i gstreamer1.0-plugins-base GStreamer plugins from the "base" set 1.10.4-1
i gstreamer1.0-plugins-base-apps GStreamer helper programs from the "base" set 1.10.4-1
i gstreamer1.0-plugins-base-dbg GStreamer plugins from the "base" set 1.10.4-1
i gstreamer1.0-plugins-base-doc GStreamer documentation for plugins from the "base" set 1.10.4-1
i gstreamer1.0-plugins-good GStreamer plugins from the "good" set 1.10.4-1
i gstreamer1.0-plugins-good-dbg GStreamer plugins from the "good" set 1.10.4-1
i gstreamer1.0-plugins-good-doc GStreamer documentation for plugins from the "good" set 1.10.4-1
i gstreamer1.0-plugins-ugly GStreamer plugins from the "ugly" set 1.10.4-1
i gstreamer1.0-plugins-ugly-dbg GStreamer plugins from the "ugly" set (debug symbols) 1.10.4-1
i gstreamer1.0-plugins-ugly-doc GStreamer documentation for plugins from the "ugly" set 1.10.4-1
i gstreamer1.0-pulseaudio GStreamer plugin for PulseAudio 1.10.4-1
i gstreamer1.0-tools Tools for use with GStreamer 1.10.4-1
i gstreamer1.0-x GStreamer plugins for X11 and Pango 1.10.4-1
i libgstreamer-plugins-bad1.0-0 GStreamer development files for libraries from the "bad" set 1.10.4-1
i libgstreamer-plugins-base1.0-0 GStreamer libraries from the "base" set 1.10.4-1
i libgstreamer1.0-0 Core GStreamer libraries and elements 1.10.4-1
i libgstreamer1.0-dev GStreamer core development files 1.10.4-1
Attachments
omxh264enc_bitrate_problem_1.jpg
omxh264enc_bitrate_problem_1.jpg (112.42 KiB) Viewed 2028 times

gkreidl
Posts: 5552
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 2:05 pm

RaoulSerio wrote:
Tue Dec 19, 2017 1:41 pm
...
gkreidl, we think that our problem is different. We are successfully using gstreamer and omx support, installed from standard stretch repository.

Do you think that downgrading to jessie could be resolve our problems? Do you tried to use bitrate target in live transcoding?
Yes, and I can set the target bitrate in my rtranscode application: viewtopic.php?f=38&t=123876
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 3:34 pm

RaoulSerio wrote:
Tue Dec 19, 2017 1:41 pm
It's difficult for us to retrieve default bitrate of gstreamer because we are using default stretch package. We did'nt found an effective debug configuration to retrieve this information.
Checking the gst-omx source, it appears to be using whatever the defaults are on the component, which is eControlRate = OMX_Video_ControlRateDisable, nTargetBitrate = 0;
RaoulSerio wrote:Empirically, (through VLC client statistics) seems to be a minimum of 64kbit (with static images) and heavy VBR activity (up to 2Mbit) with a lot of movements in the scene. Unfortunately, in this default configuration and scenes with "normal" movements, video quality in output is very poor.

We also fails to recompile gstreamer on stretch with omx support (we have a problem with h264parse plugin that is not found after recompilation). If you suggest to modify gstomxh264enc.c, we need first of all to achieve this goal.

In this moment, (with --gst-debug=omxvideoenc:5) the only useful logs are:
0:21:22.613191072 1699 0x6f0ea4f0 ERROR omxvideoenc gstomxvideoenc.c:626:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> No corresponding frame found
0:21:24.652042381 1699 0x6f0ea4f0 ERROR omxvideoenc gstomxvideoenc.c:626:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> No corresponding frame found
It'd be worth increasing the log level to see what else is going on and putting them on pastebin or similar. I can't see your system, so otherwise I'm guessing.
The error message is coming from https://github.com/GStreamer/gst-omx/bl ... enc.c#L630, which will have been called from https://github.com/GStreamer/gst-omx/bl ... enc.c#L765. I still see no handling for fragmented OMX buffers. One thing to check would be the size of the encoded frames using "ffprobe -show_packets <file>" or similar. If there are video packets coming out at >64kB then it would confirm my theory.
RaoulSerio wrote:We have done a lot of research on the web and we can not understand why nobody reports this kind of problem which practically nullifies the possibility of transcoding with raspberry.
Can it be caused by our stupid mistake?
The number of people transcoding anything on any platform is likely to be low. Neither Gstreamer nor FFmpeg are maintained by Pi Towers, so the first we know is when someone shouts, and then we don't necessarily have the resource to investigate (everything has to be prioritised).

As to gkreidl's issue, I hadn't picked up on it before, and again I haven't seen full logs. There's nothing really changed on the firmware side, but I do vaguely recall there being a significant change in that h264parse wasn't required in the more recent versions of GStreamer due to an architecture change. That should be independent of format negotiation between decode and encode though.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Dec 19, 2017 5:16 pm

Had a quick play with GST installed from the Raspbian repos for Stretch.

Code: Select all

GST_DEBUG=*:3,omxvideodec:6 gst-launch-1.0 filesrc location=bbb.264 ! h264parse ! omxh264dec ! omxh264enc ! filesink location=gst.h264
is happy and produces a sensible output.

Code: Select all

GST_DEBUG=*:3,omxvideodec:6 gst-launch-1.0 filesrc location=bbb.264 ! h264parse ! omxh264dec ! omxh264enc control-rate=variable target-bitrate=500000 ! filesink location=gst.h264
does throw the error messages you report.

Analysing the logs more closely with log level 5 for omxvideoenc:

Code: Select all

0:00:27.243348648  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.243335054  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000410 17933333
0:00:27.243422502  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.263523913  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.263824225  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:27.274406857  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:27.285606414  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:770:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:27.285751726  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.285942611  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.287742555  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000410 17966666
0:00:27.306605897  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.306815219  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:27.325221114  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:27.325687623  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:770:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:27.325811790  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.325795019  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000490 0
0:00:27.325902519  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.347310386  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.347550177  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:27.350822461  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:770:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:27.350976940  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.350965273  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000420 17999999
0:00:27.351055012  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.376394172  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.376613130  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:27.397794956  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:27.399687035  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:770:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:27.399823753  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.399809951  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000520 17999999
0:00:27.399909951  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.419698082  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.419912196  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:27.440524752  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:27.442490008  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:770:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:27.442621154  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.442607404  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000520 17999999
0:00:27.442711466  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.462377358  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.462594493  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:27.482638248  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:27.484521890  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:770:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:27.484658296  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1460:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.484642879  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000530 17999999
0:00:27.484744754  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1553:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:27.505045957  6869 0x723024c0 DEBUG            omxvideoenc gstomxvideoenc.c:1612:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:27.505262727  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:27.515741244  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:762:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
In those "Handling buffer: " messages, the first hex value is the buffer flags.
0x400 is OMX_BUFFERFLAG_ENDOFNAL (ignore this)
0x100 is OMX_BUFFERFLAG_TIME_UNKNOWN
0x80 is OMX_BUFFERFLAG_CODECCONFIG (header bytes)
0x20 is OMX_BUFFERFLAG_SYNCFRAME
0x10 is OMX_BUFFERFLAG_ENDOFFRAME
I see:

Code: Select all

0:00:27.243335054  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000410 17933333
P-frame at 17933333
0:00:27.287742555  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000410 17966666
P-frame at 17966666
0:00:27.325795019  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000490 0
Header bytes (someone's enabled inline headers)
0:00:27.350965273  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000420 17999999
I-frame pt 1 at 17999999
0:00:27.399809951  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000520 17999999
I-frame pt 2 (time unknown)
0:00:27.442607404  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000520 17999999
I-frame pt 3 (time unknown)
0:00:27.484642879  6869 0x71803b20 DEBUG            omxvideoenc gstomxvideoenc.c:753:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000530 17999999
I-frame pt 4 (time unknown), completes the frame
So I think that confirms my suspicion of the cause.

I've not got time to look at FFmpeg at the moment, but as I'd said above, that does appear to do something with flags without OMX_BUFFERFLAG_ENDOFFRAME set.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 5552
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: GStreamer and omx encoding from live source.. a naive problem

Wed Dec 20, 2017 12:40 am

I'll check tomorrow what happens to my transcoding pipeline, if I remove the "control-rate=variable target-bitrate=..." settings.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Jan 02, 2018 1:46 pm

I've rebuilt 1.10 based on gkreidl's instructions at https://gist.githubusercontent.com/spha ... r-build.sh (substitute "1.10" instead of master asthe branch). I'm looking at 1.10 as 1.10.4 is what is in the Raspbian repos.

With the vanilla code and the launch line of

Code: Select all

GST_DEBUG=*:3 gst-launch-1.0 -vvv filesrc location=bbb.h264 ! h264parse ! video/x-h264,alignment=au ! omxh264dec ! videoscale ! video/x-raw,pixelformat=I420,width=1280,height=720 ! omxh264enc control-rate=variable target-bitrate=5000000 ! filesink location=output.h264
I get the " No corresponding frame found" error message logged every so often.

With the one suggested diff

Code: Select all

diff --git a/omx/gstomxh264enc.c b/omx/gstomxh264enc.c
index aa33ae5..a091fa1 100644
--- a/omx/gstomxh264enc.c
+++ b/omx/gstomxh264enc.c
@@ -310,6 +310,7 @@ gst_omx_h264_enc_set_format (GstOMXVideoEnc * enc, GstOMXPort * port,
   gst_omx_port_get_port_definition (GST_OMX_VIDEO_ENC (self)->enc_out_port,
       &port_def);
   port_def.format.video.eCompressionFormat = OMX_VIDEO_CodingAVC;
+  port_def.nBufferSize = 512<<10;
   err =
       gst_omx_port_update_port_definition (GST_OMX_VIDEO_ENC
       (self)->enc_out_port, &port_def);
I don't get the error logged, and the encoded file looks fine from a cursory viewing.

TBH It isn't really the correct fix as it would be possible for a frame to encode up to more 512kB, but the chances are pretty low. The correct fix is for gst-omx to handle fragments out of the encoder. I'll have a quick play to see if I can achieve that, but no guarantees.
I'll also talk to those who manage the Raspbian repo to see whether they'd accept that one line change into Raspbian.

Another minor observation is that trying to resize the video to 640x360 results in colour issues at the bottom of the frame. 360 is not a multiple of 16, therefore I suspect something isn't setting up pointers correct, even though omxh264enc appears to have code which is doing a sensible thing. Yet another thing to look at.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Jan 02, 2018 2:16 pm

Hmm, that seemed to fall out more easily than expected. Try this patch for combining any fragmented buffers into one output GST buffer

Code: Select all

diff --git a/omx/gstomxvideoenc.c b/omx/gstomxvideoenc.c
index bfad2f3..ad02461 100644
--- a/omx/gstomxvideoenc.c
+++ b/omx/gstomxvideoenc.c
@@ -585,16 +585,42 @@ gst_omx_video_enc_handle_output_frame (GstOMXVideoEnc * self, GstOMXPort * port,
 
     GST_DEBUG_OBJECT (self, "Handling output data");
 
-    if (buf->omx_buf->nFilledLen > 0) {
-      outbuf = gst_buffer_new_and_alloc (buf->omx_buf->nFilledLen);
-
-      gst_buffer_map (outbuf, &map, GST_MAP_WRITE);
-      memcpy (map.data,
-          buf->omx_buf->pBuffer + buf->omx_buf->nOffset,
-          buf->omx_buf->nFilledLen);
-      gst_buffer_unmap (outbuf, &map);
+    if (!frame->output_buffer) {
+      if (buf->omx_buf->nFilledLen > 0) {
+        outbuf = gst_buffer_new_and_alloc (buf->omx_buf->nFilledLen);
+
+        gst_buffer_map (outbuf, &map, GST_MAP_WRITE);
+        memcpy (map.data,
+            buf->omx_buf->pBuffer + buf->omx_buf->nOffset,
+            buf->omx_buf->nFilledLen);
+        gst_buffer_unmap (outbuf, &map);
+      } else {
+        outbuf = gst_buffer_new ();
+      }
     } else {
-      outbuf = gst_buffer_new ();
+      if (buf->omx_buf->nFilledLen > 0) {
+        GstMapInfo map_fragment = GST_MAP_INFO_INIT;
+
+        outbuf = gst_buffer_new_and_alloc (gst_buffer_get_size(frame->output_buffer) + buf->omx_buf->nFilledLen);
+
+        gst_buffer_map (outbuf, &map, GST_MAP_WRITE);
+
+        gst_buffer_map (frame->output_buffer, &map_fragment, GST_MAP_READ);
+        memcpy(map.data,
+            map_fragment.data,
+            gst_buffer_get_size(frame->output_buffer));
+        gst_buffer_unmap (frame->output_buffer, &map_fragment);
+
+        memcpy (map.data + gst_buffer_get_size(frame->output_buffer),
+            buf->omx_buf->pBuffer + buf->omx_buf->nOffset,
+            buf->omx_buf->nFilledLen);
+        gst_buffer_unmap (outbuf, &map);
+
+        gst_buffer_unref(frame->output_buffer);
+        frame->output_buffer = NULL;
+      } else {
+        outbuf = frame->output_buffer;
+      }
     }
 
     GST_BUFFER_TIMESTAMP (outbuf) =
@@ -620,8 +646,13 @@ gst_omx_video_enc_handle_output_frame (GstOMXVideoEnc * self, GstOMXPort * port,
 
     if (frame) {
       frame->output_buffer = outbuf;
-      flow_ret =
-          gst_video_encoder_finish_frame (GST_VIDEO_ENCODER (self), frame);
+      if (buf->omx_buf->nFlags & OMX_BUFFERFLAG_ENDOFFRAME) {
+        GST_INFO_OBJECT(self, "Passing buffer as frame complete - nFlags %04X, size %d",
+            buf->omx_buf->nFlags,
+            gst_buffer_get_size(frame->output_buffer));
+        flow_ret =
+            gst_video_encoder_finish_frame (GST_VIDEO_ENCODER (self), frame);
+      } else {
+        GST_ERROR_OBJECT(self, "NOT passing buffer as it is a fragment - nFlags %04X, now size %d",
+            buf->omx_buf->nFlags,
+            gst_buffer_get_size(frame->output_buffer));
+      }
     } else {
       GST_ERROR_OBJECT (self, "No corresponding frame found");
       flow_ret = gst_pad_push (GST_VIDEO_ENCODER_SRC_PAD (self), outbuf);
Yes there's extra logging in there which isn't necessary, but it does mean it screams as and when this happens.
I've just seen in the logs

Code: Select all

0:00:13.626942325  2168 0x71d02c30 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:00:13.627881326  2168 0x71d02c30 INFO             omxvideoenc gstomxvideoenc.c:652:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Passing buffer as frame complete - nFlags 0530, size 78358
...
0:00:16.890567810  2168 0x71d02c30 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:00:16.891761862  2168 0x71d02c30 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0520, now size 131072
0:00:16.892754509  2168 0x71d02c30 INFO             omxvideoenc gstomxvideoenc.c:652:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Passing buffer as frame complete - nFlags 0530, size 142618
and using "avprobe -show_packets output.h264 | grep size"

Code: Select all

size=78386
...
size=142646
which fairly nicely match up. The 28 byte difference sounds like the header bytes to me.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 5552
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Jan 02, 2018 6:32 pm

6by9 wrote:
Tue Jan 02, 2018 2:16 pm
Hmm, that seemed to fall out more easily than expected. Try this patch for combining any fragmented buffers into one output GST buffer

Code: Select all

diff --git a/omx/gstomxvideoenc.c b/omx/gstomxvideoenc.c
index bfad2f3..ad02461 100644
--- a/omx/gstomxvideoenc.c
+++ b/omx/gstomxvideoenc.c
@@ -585,16 +585,42 @@ gst_omx_video_enc_handle_output_frame (GstOMXVideoEnc * self, GstOMXPort * port,
 
     GST_DEBUG_OBJECT (self, "Handling output data");
 
-    if (buf->omx_buf->nFilledLen > 0) {
-      outbuf = gst_buffer_new_and_alloc (buf->omx_buf->nFilledLen);
-
-      gst_buffer_map (outbuf, &map, GST_MAP_WRITE);
-      memcpy (map.data,
-          buf->omx_buf->pBuffer + buf->omx_buf->nOffset,
-          buf->omx_buf->nFilledLen);
-      gst_buffer_unmap (outbuf, &map);
+    if (!frame->output_buffer) {
+      if (buf->omx_buf->nFilledLen > 0) {
+        outbuf = gst_buffer_new_and_alloc (buf->omx_buf->nFilledLen);
+
+        gst_buffer_map (outbuf, &map, GST_MAP_WRITE);
+        memcpy (map.data,
+            buf->omx_buf->pBuffer + buf->omx_buf->nOffset,
+            buf->omx_buf->nFilledLen);
+        gst_buffer_unmap (outbuf, &map);
+      } else {
+        outbuf = gst_buffer_new ();
+      }
     } else {
-      outbuf = gst_buffer_new ();
+      if (buf->omx_buf->nFilledLen > 0) {
+        GstMapInfo map_fragment = GST_MAP_INFO_INIT;
+
+        outbuf = gst_buffer_new_and_alloc (gst_buffer_get_size(frame->output_buffer) + buf->omx_buf->nFilledLen);
+
+        gst_buffer_map (outbuf, &map, GST_MAP_WRITE);
+
+        gst_buffer_map (frame->output_buffer, &map_fragment, GST_MAP_READ);
+        memcpy(map.data,
+            map_fragment.data,
+            gst_buffer_get_size(frame->output_buffer));
+        gst_buffer_unmap (frame->output_buffer, &map_fragment);
+
+        memcpy (map.data + gst_buffer_get_size(frame->output_buffer),
+            buf->omx_buf->pBuffer + buf->omx_buf->nOffset,
+            buf->omx_buf->nFilledLen);
+        gst_buffer_unmap (outbuf, &map);
+
+        gst_buffer_unref(frame->output_buffer);
+        frame->output_buffer = NULL;
+      } else {
+        outbuf = frame->output_buffer;
+      }
     }
 
     GST_BUFFER_TIMESTAMP (outbuf) =
@@ -620,8 +646,13 @@ gst_omx_video_enc_handle_output_frame (GstOMXVideoEnc * self, GstOMXPort * port,
 
     if (frame) {
       frame->output_buffer = outbuf;
-      flow_ret =
-          gst_video_encoder_finish_frame (GST_VIDEO_ENCODER (self), frame);
+      if (buf->omx_buf->nFlags & OMX_BUFFERFLAG_ENDOFFRAME) {
+        GST_INFO_OBJECT(self, "Passing buffer as frame complete - nFlags %04X, size %d",
+            buf->omx_buf->nFlags,
+            gst_buffer_get_size(frame->output_buffer));
+        flow_ret =
+            gst_video_encoder_finish_frame (GST_VIDEO_ENCODER (self), frame);
+      } else {
+        GST_ERROR_OBJECT(self, "NOT passing buffer as it is a fragment - nFlags %04X, now size %d",
+            buf->omx_buf->nFlags,
+            gst_buffer_get_size(frame->output_buffer));
+      }
     } else {
       GST_ERROR_OBJECT (self, "No corresponding frame found");
       flow_ret = gst_pad_push (GST_VIDEO_ENCODER_SRC_PAD (self), outbuf);
Yes there's extra logging in there which isn't necessary, but it does mean it screams as and when this happens.
I've just seen in the logs

Code: Select all

0:00:13.626942325  2168 0x71d02c30 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:00:13.627881326  2168 0x71d02c30 INFO             omxvideoenc gstomxvideoenc.c:652:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Passing buffer as frame complete - nFlags 0530, size 78358
...
0:00:16.890567810  2168 0x71d02c30 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:00:16.891761862  2168 0x71d02c30 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0520, now size 131072
0:00:16.892754509  2168 0x71d02c30 INFO             omxvideoenc gstomxvideoenc.c:652:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Passing buffer as frame complete - nFlags 0530, size 142618
and using "avprobe -show_packets output.h264 | grep size"

Code: Select all

size=78386
...
size=142646
which fairly nicely match up. The 28 byte difference sounds like the header bytes to me.
You're my hero! After adding your patches my transcoder software is running like charm. No errors any more, except upstream errors (from the original stream).
I've also added an upstream patch from github: (3f5d98a360d657e904fadedb4804b03952204f63) to gst_omx_video_enc_propose_allocation, but it also works without this patch.

There's one small problem left: every 1.2 to 2.5 seconds (depending on the format) I'll get a warning like this:
0:00:44.143892732 5261 0xf3bf80 WARN matroskamux matroska-mux.c:3558:gst_matroska_mux_write_data:<stream:video_0> Invalid buffer timestamp; dropping buffer
This did not happen on Jessie with gst-omx 1.0, but it doesn't make a difference visible in the video.

Thanks a lot. I hope your fixes will soon find their way into the (Foundation) repository version.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Tue Jan 02, 2018 6:39 pm

Please note that the second change should invalidate the first - you don't need big buffers if the code is handling fragmentation.

At a guess, based on the duration between errors, your mkv error is it seeing header bytes in-band which probably come through with a 0 or unknown timestamp, but it'll need a little work to determine if that guess is correct or not.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Wed Jan 03, 2018 12:33 pm

OK, minimal requirements for getting Raspbian repos updated (at least for me as I can talk directly to our maintainer).
I've got one colleague here who is vaguely familiar with GStreamer, so I will get him to review it before pushing, but otherwise we should be able to get our local version patched in the next day or so.

For some reason I can't make it mux back into an mkv file (errors on not knowing the total length, and complaints on sticky events), so can't check out why you are having errors thrown from it. I'm building 1.12 to see if that helps out.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 5552
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: GStreamer and omx encoding from live source.. a naive problem

Wed Jan 03, 2018 1:49 pm

6by9 wrote:
Wed Jan 03, 2018 12:33 pm
OK, minimal requirements for getting Raspbian repos updated (at least for me as I can talk directly to our maintainer).
I've got one colleague here who is vaguely familiar with GStreamer, so I will get him to review it before pushing, but otherwise we should be able to get our local version patched in the next day or so.

For some reason I can't make it mux back into an mkv file (errors on not knowing the total length, and complaints on sticky events), so can't check out why you are having errors thrown from it. I'm building 1.12 to see if that helps out.
Today I tried to add the same patch to gstreamer-omx-1.0 for Jessie, which is easy, as the code off this function hasn't changed except for a small detail.

I used the standard Debian tools to get the sources. First I rebuilt the package from the unmodified (...rpi18) sources, then I added the patch (number 0031) using quilt and built the package ...rpi19. Both packages have been built with "debuild -b -uc -us"

But both packages I built don't work and throw the same error:
WARN GST_PLUGIN_LOADING gstplugin.c:739:gst_plugin_load_file: module_open failed: /opt/vc/lib/libEGL.so: undefined symbol: glDiscardFramebufferEXT
(gst-plugin-scanner:12986): GStreamer-WARNING **: Failed to load plugin '/usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstomx.so': /opt/vc/lib/libEGL.so: undefined symbol: glDiscardFramebufferEXT

Is there some special trick I don't know about when building this package? Perhaps you could ask the maintainer who has created the Foundation version (currently ...rpi18pi1g). The omx module 1.0 throws the same errors but less frequently and they are not as obtrusive, but it would be nice to have the fix in Jessie, too.

I have attached the patch for reference.
Attachments
0031-omxvideoenc-combine-fragmented-buffers.patch.bz2
(849 Bytes) Downloaded 19 times
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5022
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: GStreamer and omx encoding from live source.. a naive problem

Wed Jan 03, 2018 1:54 pm

This is probably an incomplete answer, since I don't know what it's trying to do with EGL, but there shouldn't be a '/opt/vc/lib/libEGL.so' on Stretch.

gkreidl
Posts: 5552
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: GStreamer and omx encoding from live source.. a naive problem

Wed Jan 03, 2018 2:07 pm

ShiftPlusOne wrote:
Wed Jan 03, 2018 1:54 pm
This is probably an incomplete answer, since I don't know what it's trying to do with EGL, but there shouldn't be a '/opt/vc/lib/libEGL.so' on Stretch.
I'm talking about Jessie and gst-omx-1.0 for Jessie. I have both libbrcmEGL.so and libEGL.so in '/opt/vc/lib'.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5022
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: GStreamer and omx encoding from live source.. a naive problem

Wed Jan 03, 2018 2:21 pm

Ah, my mistake. Should've read your post more thoroughly.

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 8:30 am

Good year guys
in stretch, with branch 1.10, I tried to apply gkreidl patch attached in previous post.
By copying & paste 6by9 patch directly from the web, patch command give me an error on line 73.

After compilation, I tried to get an h264 stream for example from an IP camera to decode, re-encode and stream in udp:

Code: Select all

GST_DEBUG=3 gst-launch-1.0 rtsp://192.168.1.1 ! rtph264depay ! h264parse ! omxh264dec ! omxh264enc target-bitrate=5000000 control-rate=variable ! rtph264pay ! udpsink host=192.168.1.100 port=15000 sync=false async=false
It's working at the beginning, but after some seconds of activity and these logs:

Code: Select all

0:34:58.153923436  9045 0x72e01f20 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<enc> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:35:01.340066611  9045 0x72e01f20 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<enc> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:35:04.527713325  9045 0x72e01f20 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<enc> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:35:07.705775712  9045 0x72e01f20 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<enc> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
gstreamer does not stream anymore: i don't have any further errors, segmentation faults or other logs. It seems in "pause" or something like that. The receiver does not receive packets anymore.

I tried to simplify pipeline in this way:

Code: Select all

GST_DEBUG=3 gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720 ! videoconvert ! videoscale ! omxh264enc target-bitrate=5000000 control-rate=variable ! rtph264pay ! udpsink host=192.168.150.100 port=15000 sync=false async=false
In this case, I don't have logs, but final result is identical.

SDP file to receive stream with VLC attached.
Attachments
receive_stream_VIDEO_H264_15000.zip
(353 Bytes) Downloaded 16 times

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 8:36 am

This is the last log lines before "hang":

Code: Select all

0:00:16.793390733   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:798:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:16.793625525   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:806:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:16.832603509   762   0x7fba90 DEBUG            omxvideoenc gstomxvideoenc.c:1496:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:16.832696946   762   0x7fba90 DEBUG            omxvideoenc gstomxvideoenc.c:1589:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Handling frame
0:00:16.840923772   762   0x7fba90 DEBUG            omxvideoenc gstomxvideoenc.c:1648:gst_omx_video_enc_handle_frame:<omxh264enc-omxh264enc0> Passed frame to component
0:00:16.850603984   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:789:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000490 0
0:00:16.850736067   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:798:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:16.850907943   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:806:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:16.863134405   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:789:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000420 16169186
0:00:16.863263103   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:16.863411281   762 0x6f48b120 ERROR            omxvideoenc gstomxvideoenc.c:658:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> NOT passing buffer as it is a fragment - nFlags 0420, now size 65536
0:00:16.863452479   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:798:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:16.863821906   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:806:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
0:00:16.863889614   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:789:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Handling buffer: 0x00000530 16169186
0:00:16.863961906   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:586:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Handling output data
0:00:16.864360916   762 0x6f48b120 INFO             omxvideoenc gstomxvideoenc.c:652:gst_omx_video_enc_handle_output_frame:<omxh264enc-omxh264enc0> Passing buffer as frame complete - nFlags 0530, size 93618
0:00:16.867237584   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:798:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Finished frame: ok
0:00:16.873008888   762 0x6f48b120 DEBUG            omxvideoenc gstomxvideoenc.c:806:gst_omx_video_enc_loop:<omxh264enc-omxh264enc0> Read frame from component
after this, gstreamer seems dead.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 9:43 am

Normal mantra for GStreamer - if in doubt, insert a queue component. At a guess, something upstream of the codecs is waiting for things to be processed before returning the buffer, and that isn't happening. Many of the intra-component calls get done in the same thread context, inserting a queue breaks it into a new context.
I wonder if rtph264pay has a maximum buffer size, or udpsink is doing something odd when it has to fragment the UDP packet (also happens at around the 64kB mark).

NB "NOT passing buffer as it is a fragment - nFlags 0420, now size 65536" is NOT actually an error - it was set to a high log level to show up easily in the logs whilst developing.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 10:39 am

Thank you 6by9, but I have same results with filesink:

Code: Select all

GST_DEBUG=3 gst-launch-1.0 rtsp://192.168.1.1 ! rtph264depay ! h264parse ! omxh264dec ! queue ! omxh264enc target-bitrate=5000000 control-rate=variable ! filesink location='test.h264'
Only 3-6 seconds of writing

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 10:52 am

Adding videoconvert and videorate, the pipeline seems to work better.... For example in this way:

Code: Select all

GST_DEBUG=3 gst-launch-1.0 rtsp://192.168.1.1 ! rtph264depay ! h264parse ! omxh264dec ! queue ! videoconvert ! videorate ! omxh264enc target-bitrate=5000000 control-rate=variable ! h264parse ! queue ! rtph264pay ! queue ! udpsink host=192.168.1.100 port=15000 sync=false async=false

I'm doing more tests

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4953
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 11:11 am

Code: Select all

GST_DEBUG=*:3,omxvideoenc:5 gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720 ! videoconvert ! videoscale ! omxh264enc target-bitrate=5000000 control-rate=variable ! filesink location=testsrc.h264
is working fine for me and produced >30 seconds.
Likewise

Code: Select all

GST_DEBUG=*:3,omxvideoenc:5 gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720,pixelformat=I420 ! omxh264enc target-bitrate=5000000 control-rate=variable ! filesink location=testsrc.h264
There may be something funny in rebuilding just the OMX plugin - just due to the way I've been doing development I've got everything else built from 1.10.5 and gst-omx from the Raspbian source + the one patch.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RaoulSerio
Posts: 10
Joined: Tue Dec 19, 2017 9:12 am

Re: GStreamer and omx encoding from live source.. a naive problem

Fri Jan 05, 2018 11:24 am

I was thinking the same.. it's probably a recompilation problem.
I followed the script (entirely) for recompilation and I used the gst-omx branch 1.10 sources from the main source, modifying the script in the gst-omx section adding "git checkout -t origin/$BRANCH || true" that was missing.

Now I will try to use rpi source of gst-omx
Last edited by RaoulSerio on Fri Jan 05, 2018 3:47 pm, edited 1 time in total.

Return to “Advanced users”

Who is online

Users browsing this forum: pws and 16 guests