david_Est
Posts: 12
Joined: Sun Jun 17, 2012 4:58 pm

Re: Real Time Transcoding of DVB TV Streams

Mon Jul 29, 2019 7:43 am

6by9 wrote:
Wed Jul 24, 2019 9:23 am
6by9 wrote:
Mon Jun 24, 2019 7:38 am
david_Est wrote:
Mon Jun 24, 2019 7:24 am
Would be great if new rpi4 has the capabilities to support this. Even greater if HEVC codec is selectable for encoding
Hevc is decode only on the pi4.

We are chasing down an issue that only showed up in the last few days where simultaneous encode and decode with the h264 codec block stalls. We're actively digging into it at the moment. It seems to be some hardware quirk as the same code on Pi3 works fine.
Last night's rpi-update release has a fix for the simultaneous encode/decode issue on Pi4, so hopefully all is good to go now.
this sounds great!!!

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Transcoding on Buster and the RPi 4

Mon Sep 16, 2019 3:17 pm

A few weeks ago I updated my first system to Buster (a RPi 2B running from HDD with Boot partition on SD card). The first step after fine tuning the system was to update my published software to Buster (kweb suite, rpiotrtool).

Then I started to work on rtranscode V 4.0. I had only to modify a few lines of code and to recompile the Python applications / modules with nuitka and it worked with the same restrictions as on Stretch (no HW video scaling, software deinterlacing required). Full HD resolutions (1920x1080 interlaced) did not really work but that was to be expected as the RPi 2B simply doesn't have the power to do all the software deinterlacing and scaling in real time. Decoding (MPEG2 and H264) and encoding were done by the gstreamer-omx module which seemed to work out of the box.

Then I replaced the RPi 2B with my new shiny RPi 4B/4G, hoping that the much more powerful processor and the faster memory interface would give enough power for a decent real time transcoding solution. But I ran into a lot of problems.

I knew, that the HW MPEG2 decoders are not available on the RPi 4 any more. I replaced them with a software decoder and this worked well on SD channels (I do not have access to HD MPEG2 channels).

But all H264 encoded channels (SD and HD) didn't work. gst-omx decoding simply doesn't work on the RPi 4 any more.

So I tried to replace the gst-omx H264 decoder by the V4L2 H264 decoder. But this resulted in a very restricted success: v4l2h264dec doesn't work for interlaced video and most video channels (SD as well as full HD) are interlaced. It only worked for 720p50 streams, which are used by some German public TV stations on Astra.

As a last resort I found avdec_h264_mmal. This does work with all kinds of streams but creates another problem: video and audio are out of sync, especially on HD channels (up to a few seconds). And also it is very critical with clock jittering and simply crashes on some channels.

At the moment I am stuck. Possible solutions might be:

Someone fixes the gst-omx decoder so that it works on the RPi 4. But this is not very realistic, as it seems to be incompatible with the new video drivers.

Someone fixes the timing (clock) problems of avdec_h264_mmal. Not a very realistic perspective either, IMHO.

This leaves v4l2h264dec as a last resort. It has been stated here that the RPi developers believe that the V4L2 components will be the future. Why can't the decoder simple interweave to 2 half images into one, as OpenMax and MMAL could do?

And there are more problems with V4L2: all other tools I've tried to use (v4l2convert which I have used for scaling and also v4l2h264enc) break, if the original stream is flagged as interlaced. If I deinterlace the stream, v4l2convert does work. But deinterlacing is not always required, even if the original stream is interlaced. A typical movie channel needs no deinterlacing, because the original material is recorded with 24 images/sec. There is no time difference between both half images. It is even better, not to deinterlace such channels, because you loose resolution by deinterlacing. A typical sports channel needs deinterlacing, because otherwise you will see ragged edges between video lines. It should be decided by the application developer (or the user) to decide, if deinterlacing is required or not. That means, that both v4l2convert and v4l2h264enc should simply ignore, if the stream is flagged as interlaced or not and just treat it as a progressive stream. This is the way gst-omx worked.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9574
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Real Time Transcoding of DVB TV Streams

Mon Sep 16, 2019 4:53 pm

gkreidl wrote:
Mon Sep 16, 2019 3:17 pm
But all H264 encoded channels (SD and HD) didn't work. gst-omx decoding simply doesn't work on the RPi 4 any more.
Groan, they merged in the use of egl_render to convert video frames into EGL frames. That's going to fail in a big way as we have no EGL backend to register with.
That appears to be confirmed by the debug output from a Pi3 running "gst-launch-1.0 -e -vvvv filesrc location=bbb_sunflower_1080p_30fps_normal.mp4 ! qtdemux ! h264parse ! omxh264dec ! kmssink"

Code: Select all

GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), 
It's that memory:GLMemory that is the issue.

Rebuilding having culled all the bits in gstomxvideodec.c behind "#if defined (USE_OMX_TARGET_RPI) && defined (HAVE_GST_GL)" should work. There are a couple of other options behind USE_OMX_TARGET_RPI that are still wanted, so it may be safe to disable HAVE_GST_GL within that one file.
gkreidl wrote:This leaves v4l2h264dec as a last resort. It has been stated here that the RPi developers believe that the V4L2 components will be the future. Why can't the decoder simple interweave to 2 half images into one, as OpenMax and MMAL could do?

And there are more problems with V4L2: all other tools I've tried to use (v4l2convert which I have used for scaling and also v4l2h264enc) break, if the original stream is flagged as interlaced. If I deinterlace the stream, v4l2convert does work. But deinterlacing is not always required, even if the original stream is interlaced. A typical movie channel needs no deinterlacing, because the original material is recorded with 24 images/sec. There is no time difference between both half images. It is even better, not to deinterlace such channels, because you loose resolution by deinterlacing. A typical sports channel needs deinterlacing, because otherwise you will see ragged edges between video lines. It should be decided by the application developer (or the user) to decide, if deinterlacing is required or not. That means, that both v4l2convert and v4l2h264enc should simply ignore, if the stream is flagged as interlaced or not and just treat it as a progressive stream. This is the way gst-omx worked.
You've been processing an interlaced frame (2 fields) as if it were progressive? This can work if you consider it to be double field width x field height, but if you consider it as field width x frame height then the chroma subsampling is offset - chroma line 2 will be considered to be for luma lines 3&4, when in reality it is 2&4.
The H264 encoder has never supported interlaced video, so I'm not sure how you've told omxh264enc that the material is interlaced.

I suspect in both cases no one has been bothered about the signalling, and silently accepted it.

v4l2h264dec could support interlacing - I hadn't added it as it is tricky plumbing to get the interlacing information and order out from the codec at an appropriate point to update the V4L2 format. Each buffer also then has to be signalled correctly.
It's just a faff of plumbing, most of which is hidden from you in OMX. I think I see why gst-omx and ffmpeg mmaldec have swept it under the carpet. I'm not sure either IL or MMAL actually export the information externally, it's passed down IL tunnels in-band should you care to connect suitable components.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Tue Sep 17, 2019 10:56 am

@6by9
Some of the things you describe are just "black boxes" for me. But I'm beginning to understand how it worked in the past.

gst-omx renders into a EGL buffer in RGBA format, which means it includes a conversion into RGBA. Both half-images are rendered into one images, which contains the first image in lines 1,3,5 ... and the second image in lines 2,4,5 ...

This single image is fed into the omxencoder, which renders it as a progressive stream.

I've tested it on Stretch by feeding my toolchain into gst-launch in verbose mode (usimg a H264 encoded SD stream):

Code: Select all

New clock: GstSystemClock
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstMpegAudioParse:mpegaudioparse0.GstPad:src: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)2, rate=(int)48000, channels=(int)2, parsed=(boolean)true
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstMatroskamuxPad:audio_0: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)2, rate=(int)48000, channels=(int)2, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstH264Parse:h264parse1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstH264Parse:h264parse1.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)25/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)25/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)25/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstMatroskamuxPad:video_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)25/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstPad:src: caps = video/x-matroska
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstPad:src: caps = video/x-matroska, streamheader=(buffer)< 1a45dfa301000000000000144282896d6174726f736b610042878102428581021853806701ffffffffffffff1549a966010000000000006973a4904e415be0c8fcda01522543d705f4010a2ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31302e34005741994753747265616d6572204d6174726f736b61206d757865720044618808318cbe09e62c001654ae6b01000000000000cfae010000000000004cd7810183810273c588904ad62239a56de323e38384016e360022b59c8467657200536e86417564696f00e1010000000000000db58840e77000000000009f8102868a415f4d5045472f4c3200ae0100000000000071d7810283810173c5886be661e65daca4ab23e3838402625a00536e86566964656f00e00100000000000012b08202d0ba82024054b082041754ba8202408690565f4d504547342f49534f2f4156430063a29f01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0 >
If I add software deinterlacing (using scalerbob and both fields) I get a doubled 50Hz progressive stream:

Code: Select all

New clock: GstSystemClock
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = audio/mpeg, mpegversion=(int)1
/GstPipeline:pipeline0/GstMpegAudioParse:mpegaudioparse0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstMpegAudioParse:mpegaudioparse0.GstPad:src: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)2, rate=(int)48000, channels=(int)2, parsed=(boolean)true
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstMatroskamuxPad:audio_0: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)2, rate=(int)48000, channels=(int)2, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)16/11, width=(int)720, height=(int)576, framerate=(fraction)25/1, parsed=(boolean)true, profile=(string)high, level=(string)3
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDeinterlace:deinterlace0.GstPad:src: caps = video/x-raw, format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstDeinterlace:deinterlace0.GstPad:sink: caps = video/x-raw, format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)RGBA, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, colorimetry=(string)sRGB, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstDeinterlace:deinterlace0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstDeinterlace:deinterlace0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)720, height=(int)576, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)16/11, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstOMXH264Enc-omxh264enc:omxh264enc-omxh264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstH264Parse:h264parse1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)50/1
/GstPipeline:pipeline0/GstH264Parse:h264parse1.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)50/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)50/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)50/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstMatroskamuxPad:video_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)high, level=(string)4, width=(int)720, height=(int)576, pixel-aspect-ratio=(fraction)16/11, framerate=(fraction)50/1, parsed=(boolean)true, codec_data=(buffer)01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstPad:src: caps = video/x-matroska
/GstPipeline:pipeline0/GstMatroskaMux:stream.GstPad:src: caps = video/x-matroska, streamheader=(buffer)< 1a45dfa301000000000000144282896d6174726f736b610042878102428581021853806701ffffffffffffff1549a966010000000000006973a4909d630ae9629d26c0256489e663050f492ad7b1830f42404d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31302e34005741994753747265616d6572204d6174726f736b61206d757865720044618808318cf40e2d3c001654ae6b01000000000000cfae010000000000004cd7810183810273c5880f159e49d31f5ad323e38384016e360022b59c8467657200536e86417564696f00e1010000000000000db58840e77000000000009f8102868a415f4d5045472f4c3200ae0100000000000071d7810283810173c5881a1f22456373533623e3838401312d00536e86566964656f00e00100000000000012b08202d0ba82024054b082041754ba8202408690565f4d504547342f49534f2f4156430063a29f01640028ffe1000f27640028ac2b405a09360803c489a801000528ee025cb0 >
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9574
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Real Time Transcoding of DVB TV Streams

Tue Sep 17, 2019 11:15 am

gkreidl wrote:
Tue Sep 17, 2019 10:56 am
gst-omx renders into a EGL buffer in RGBA format, which means it includes a conversion into RGBA. Both half-images are rendered into one images, which contains the first image in lines 1,3,5 ... and the second image in lines 2,4,5 ...
The image data will be incorrectly interpreted during that conversion to RGBA.

YUV4:2:0 has the chroma planes subsampled by a factor of 2 both horizontally and vertically https://en.wikipedia.org/wiki/Chroma_subsampling#4:2:0.
As stated there
There are three variants of 4:2:0 schemes, having different horizontal and vertical siting.[7]
- In MPEG-2, Cb and Cr are cosited horizontally. Cb and Cr are sited between pixels in the vertical direction (sited interstitially).
- In JPEG/JFIF, H.261, and MPEG-1, Cb and Cr are sited interstitially, halfway between alternate luma samples.
- In 4:2:0 DV, Cb and Cr are co-sited in the horizontal direction. In the vertical direction, they are co-sited on alternating lines.
Assuming MPEG2 with an interlaced frame, the first line of chroma should be treated as applying between the first two lines of the first field, which are lines 1 & 3 in the frame. The second line of chroma should apply to the first two lines of the second field, which are lines 2 & 4 in the frame.
If you're ignoring this and treating the frame as progressive, then the first line of chroma will be applied to lines 1&2, and the second line of chroma will be applied to lines 3&4, therefore your format conversion is incorrect and will suffer from colour bleed.

You may say you don't care, but I'm afraid I do, and therefore I won't be hacking things about to allow the interlacing flags to be just dropped. We can probably get omxh264dec back up and running for Pi4 though (making it a common binary may be harder).

The image conversion to RGBA for omxh264dec is totally redundant anyway for transcoding, as the encoder will have to convert it back to YUV4:2:0 before it can touch it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Tue Sep 17, 2019 11:42 am

I did not invent that conversion to RGBA. It simply is the way it has been done in the past. And I cannot see any kind of problem in the converted images and so I have to assume it is done in the right way.

You have said yourself, that the RPi encoder does not support interlaced streams. So there has to be a way of creating a progressive stream, even if the original material is interlaced.

I'll be happy with any solution that will allow real time transcoding of TV-streams again in the future.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Sat Oct 05, 2019 11:37 am

Hello, I have Rpi3B + Stretch.
Can I run transcoding on two channels at the same time?

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Sat Oct 05, 2019 12:00 pm

typer08 wrote:
Sat Oct 05, 2019 11:37 am
Hello, I have Rpi3B + Stretch.
Can I run transcoding on two channels at the same time?
For SD streams this might work. You need to run two instances of rtranscode using different settings (especially the streaming port must be different).
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Sat Oct 05, 2019 12:57 pm

And how run two instances of rtranscode using different settings?

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Sat Oct 05, 2019 1:03 pm

typer08 wrote:
Sat Oct 05, 2019 12:57 pm
And how run two instances of rtranscode using different settings?
rtranscode -c=/path/to/settingsfile ....

If you only want to run the second istance on a different port, you can also use
rtranscode -p=9081 ....
(defaut streaming port is 9080).
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Sat Oct 05, 2019 3:22 pm

I managed to run 2 HD streams, but the video and audio are cut every 5-10 seconds. CPU load I have about 35%. Is this too much load on the processor, or something else can be improved? With 3 SD streams, everything works fine.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Sun Oct 06, 2019 9:07 am

typer08 wrote:
Sat Oct 05, 2019 3:22 pm
I managed to run 2 HD streams, but the video and audio are cut every 5-10 seconds. CPU load I have about 35%. Is this too much load on the processor, or something else can be improved? With 3 SD streams, everything works fine.
It's simply the limit of the H264 decoder/encoder in the GPU.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

antoniosatta
Posts: 1
Joined: Tue Oct 04, 2016 4:54 pm

Re: Real Time Transcoding of DVB TV Streams

Tue Nov 12, 2019 4:54 pm

Wow I liked a lot! I would like to contact you for a project

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Tue Nov 12, 2019 9:07 pm

antoniosatta wrote:
Tue Nov 12, 2019 4:54 pm
Wow I liked a lot! I would like to contact you for a project
You'll my email address in all my debian packages.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Mon Nov 25, 2019 8:39 am

Hi, I can't enable audio encoding. If I choose an option other than the original, the stream will not be encoded.
What could be the reason? RPi Stretch.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Mon Nov 25, 2019 10:35 am

typer08 wrote:
Mon Nov 25, 2019 8:39 am
Hi, I can't enable audio encoding. If I choose an option other than the original, the stream will not be encoded.
What could be the reason? RPi Stretch.
Check if the gsteamer bad plugins and the gstreamer libav plugins are installed.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Mon Nov 25, 2019 12:19 pm

Thanks, the sound works after installing gstreamer.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9574
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Real Time Transcoding of DVB TV Streams

Mon Jan 27, 2020 4:41 pm

Possibly against my better judgement, I've had a quick look at gst-omx.

There is a minor firmware bug in that the video_decode input port defaulted to MPEG4, however MPEG4 isn't supported on the Pi4. gst-omx does GetParameter(OMX_IndexParamPortDefinition) and then SetParameter(OMX_IndexParamPortDefinition) without apparently having changed anything. That failed, and upset the rest of the pipeline.

Most of the code appears to be correctly done under a conditional (self->egl_image) at runtime, so it's probably only one or two places that are doing the wrong thing.
I did a quick hack to compile with HAVE_GST_GL being undefined, but USE_OMX_TARGET_RPI defined and that broadly works. Trying to get the logic right to only advertise GL_MEMORY under some circumstances could be harder.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Mon Jan 27, 2020 5:17 pm

6by9 wrote:
Mon Jan 27, 2020 4:41 pm
Possibly against my better judgement, I've had a quick look at gst-omx.

There is a minor firmware bug in that the video_decode input port defaulted to MPEG4, however MPEG4 isn't supported on the Pi4. gst-omx does GetParameter(OMX_IndexParamPortDefinition) and then SetParameter(OMX_IndexParamPortDefinition) without apparently having changed anything. That failed, and upset the rest of the pipeline.

Most of the code appears to be correctly done under a conditional (self->egl_image) at runtime, so it's probably only one or two places that are doing the wrong thing.
I did a quick hack to compile with HAVE_GST_GL being undefined, but USE_OMX_TARGET_RPI defined and that broadly works. Trying to get the logic right to only advertise GL_MEMORY under some circumstances could be harder.
Does that mean, that we may get a working gst-omx decoder in the near future?
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9574
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Real Time Transcoding of DVB TV Streams

Mon Jan 27, 2020 7:38 pm

gkreidl wrote:
Mon Jan 27, 2020 5:17 pm
Does that mean, that we may get a working gst-omx decoder in the near future?
I have had it running, but seeing as the two options of HAVE_GST_GL and not having it are behind compile time switches, creating a generic set of components is a different thing.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

lcmg
Posts: 1
Joined: Sun Feb 09, 2020 10:21 pm

Re: Real Time Transcoding of DVB TV Streams

Sun Feb 09, 2020 10:58 pm

Hi,
First of all many thanks to the developer for his work and sharing!!!
I've found this topic when I was checking the possibility of transcoding with pi and finally this weekend I had some time to give it a try.

I'm currently on buster and didn't wanted to downgrade to stretch or jessie. I saw that rtranscode doesn't work on buster yet but I gave it a try anyway and didn't work (as expected). I didn't want to give up, and since I'm a huge fan of docker on raspberry, I've looked for an image of raspbian/stretch and found it so I've decided to give a try on this possible method to run rtranscode on docker container with raspbian/stretch.
My setup is: rpi3b+, raspbian buster lite, SSD, running as headless server with several docker services.
So I've setup the the raspbian/stretch container, updated the packages, and followed the guide to install rtranscode. This was a tentative/error process so during the process, I've had to install other needed dependencies for rtranscode (google was my friend) and I had to add to the container the port for the transcoded stream (I use the standard) and also the "/dev/vchiq" that as far as I understood is the GPU. For the source TS stream I used tvheadend. tvheadend is running on same rpi but on a different docker container.

The good news is that at the end I managed to get the rtranscode to transcode, the bad news is that after more or less 1 min it starts to get some freezes, I need further testing to try to optimize the solution and see if this can be a workaround while rtranscode is not on buster, but run out of time for the weekend :D. Any sugestion are also very welcome.

If someone else is interested in trying I can try to detail the steps I describe above.

note - I'm just a user of docker containers, I don't have the knowledge to build them, but would be great to have this great piece of sw running on a docker container. Probably I will try to learn something on this topic on near future :).

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Sun Mar 08, 2020 8:48 pm

Hello, can you create a link to every channel that would launch the video player when clicked?
Something like OpenWebIf with Enigma2.

Image

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Mon Mar 09, 2020 7:09 am

typer08 wrote:
Sun Mar 08, 2020 8:48 pm
Hello, can you create a link to every channel that would launch the video player when clicked?
Something like OpenWebIf with Enigma2.

Image
I'm not sure how this is related to transcoding.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

typer08
Posts: 12
Joined: Sun Mar 10, 2019 1:27 am

Re: Real Time Transcoding of DVB TV Streams

Mon Mar 09, 2020 9:18 am

Maybe I will say differently.
Now in the rtranscode 4.0 web interface I have to double click to start the channel and run VLC with the address http: // ***: 9080 / xyz.mkv.
I would like to click the channel in the rtranscode 4.0 Web interface once.
In the background, the script sends the command just like after clicking Transcode, and I get the m3u link to run in VLC.

Or maybe you can integrate an external video player with the rtranscode 4.0 network interface and play the streams in the browser.

gkreidl
Posts: 6355
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Real Time Transcoding of DVB TV Streams

Mon Mar 09, 2020 9:59 am

typer08 wrote:
Mon Mar 09, 2020 9:18 am
Maybe I will say differently.
Now in the rtranscode 4.0 web interface I have to double click to start the channel and run VLC with the address http: // ***: 9080 / xyz.mkv.
I would like to click the channel in the rtranscode 4.0 Web interface once.
In the background, the script sends the command just like after clicking Transcode, and I get the m3u link to run in VLC.

Or maybe you can integrate an external video player with the rtranscode 4.0 network interface and play the streams in the browser.
OK, this could be done. I use kweb and can just click the "stream" button and omxplayerGUI will open and run the stream.

But adding a local player could be done. At the moment I have stopped development because of the problems with RPi 4.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

Return to “Graphics, sound and multimedia”