Incompatibility between h264_omx and mpegts in ffmpeg?

[g8gkq]

Hi

The Short Version

I'm using ffmpeg on a RPi3 to generate an H264 transport stream using either libx264 or h264_omx and mpegts. I can get it to work properly with libx264, but the CPU is maxed out. With h264_omx it appears to work, but the resulting TS has an undefined frame rate. VLC will play the TS, but it is unsuitable for transmission using the DVB-S standard. I know that h264_omx is at least functioning, because it works with flv output.

I realise that this a very specialist question and I am not expecting answers - just pointers to where I might find any more information on how to force mpegts to define the frame rate. Google has not been helpful!

Dave

The Long Version

This code to produce a web stream using ffmpeg, libx264 and flv works well but maxes out the CPU:

Code: Select all

/home/pi/rpidatv/bin/ffmpeg \
  -f v4l2 -input_format h264 \
  -i /dev/video0 -thread_queue_size 2048 \
  -f alsa -ac 1 -ar 44100 \
  -i hw:1,0 \
  -framerate 15 -video_size 720x576 -c:v libx264 -b:v 512k -minrate:v 512k -maxrate:v 512k \
  -b:a 22050 -ac 1 \
  -g 25 \
  -f flv rtmp://sitename.org.uk/live/keykey &

Using omx_h264 in this application also works well with much lower CPU usage:

Code: Select all

/home/pi/rpidatv/bin/ffmpeg \
  -f v4l2 -input_format h264 \
  -i /dev/video0 -thread_queue_size 2048 \
  -f alsa -ac 1 -ar 44100 \
  -i hw:1,0 \
  -framerate 15 -video_size 720x576 -c:v h264_omx -b:v 512k -minrate:v 512k -maxrate:v 512k \
  -b:a 22050 -ac 1 \
  -g 25 \
  -f flv rtmp://sitename.org.uk/live/keykey &

So moving on to using mpegts output format. This code works, but maxes out the CPU cores:

Code: Select all

/home/pi/rpidatv/bin/ffmpeg -loglevel debug \
  -f image2 -loop 1 -pix_fmt yuv420p\
  -framerate 25 -pix_fmt yuv420p \
  -i /home/pi/rpidatv/video/tcf.jpg \
  -c:v libx264 -pix_fmt yuv420p \
  -b:v 320600 -minrate:v 320600 -maxrate:v 320600 \
  -f mpegts -blocksize 1880 \
  -mpegts_original_network_id 1 -mpegts_transport_stream_id 1 \
  -mpegts_pmt_start_pid 4095 -streamid 0:256 -streamid 1:257 \
  -metadata service_provider=G8GKQ -metadata service_name=G8GKQ \
  -muxrate 1382352 -y videots &

videots is a fifo that is read by another application that converts the transport stream into I and Q signals for a DVB-S TV transmission.

If I substitute h264_omx for libx264, it runs with lower CPU usage, but produces a non-compliant transport stream with a (video) frame rate undefined:

Code: Select all

/home/pi/rpidatv/bin/ffmpeg -loglevel debug \
  -f image2 -loop 1 -pix_fmt yuv420p\
  -framerate 25 -pix_fmt yuv420p \
  -i /home/pi/rpidatv/video/tcf.jpg \
  -c:v h264_omx -pix_fmt yuv420p \
  -b:v 320600 -minrate:v 320600 -maxrate:v 320600 \
  -f mpegts -blocksize 1880 \
  -mpegts_original_network_id 1 -mpegts_transport_stream_id 1 \
  -mpegts_pmt_start_pid 4095 -streamid 0:256 -streamid 1:257 \
  -metadata service_provider=G8GKQ -metadata service_name=G8GKQ \
  -muxrate 1382352 -y videots &

I have taken the audio codec out of the examples above for clarity. When the audio codec was present, I could decode the audio from the stream.

I also tested with writing the output to a .ts file and using Promax TS Analyser software to look at the stream properties. It said that the (video) frame rate in the h264_omx stream was 0.0 fps. VLC played both files OK, but when checking codec properties, there was no entry for the frame rate in the h264_omx-produced file. In the libx264 file the frame rate was correctly described as 25.0 fps. I think that this is the reason it does not work in my application - so I am looking for a way to make mpegts define the (video) frame rate.

Here is my ffmpeg configuration:

Code: Select all

ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/home/pi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/pi/ffmpeg_build/include --extra-ldflags=-L/home/pi/ffmpeg_build/lib --enable-gpl --enable-libfreetype --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-nonfree --enable-mmal --enable-omx --enable-omx-rpi
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100

I have played with trying to define a frame rate in just about everywhere in the ffmpeg statement. I'm about to give up on this, but just wondered if anyone had seen anything like it before.

The software is intended to be an enhancement to this RPi-based Digital Amateur TV Transmitter: https://wiki.batc.tv/The_Portsdown_Transmitter The existing code is on GitHub here: https://github.com/BritishAmateurTelevisionClub/rpidatv

Thanks in advance

Dave

[lawin1]

I can confirm also that ffmpeg with h264_omx encoder produces incorrect/faulty ts streams. There is a simple workaround. You can pipe ffmpeg flv output to vlc which will produce correct ts stream without video/audio re-encoding.

Code: Select all

ffmpeg -c:v:0 h264_mmal -i "http://input" -c:v:0 h264_omx -acodec aac -ac 2 -ab 128k -f flv - | cvlc -vvv - --http-reconnect --loop --repeat --sout="#gather:std{access=http,mux=ts,dst=:7095/cam1}" --sout-keep --network-caching=1000 --sout-mux-caching=1000

[6by9]

h264_omx doesn't produce a TS (Transport Stream), it produces an Elementary Stream (ES).
Within FFmpeg you use mpegts to convert from one or more ES to a TS.

There is an optional section in the H264 where the VUI (Video usability information) section of the SPS (Sequence Parameter Set) can provide a timing value. The Pi encoder omits this as most use cases use timestamps in the container (eg MKV, MP4, TS) rather than the SPS header because they are more accurate.
You can enable it on the encoder via the parameter OMX_IndexParamBrcmVideoAVCSPSTimingEnable, but h264_omx won't be doing that as standard. I have investigated enabling it on a couple of occasions, but the full implications are unclear, and at the moment it can only handle integer values anyway, so no 29.97fps streams so beloved of NTSC. https://github.com/raspberrypi/firmware/issues/1007

[g8gkq]

Thanks 6by9

Most of my users are in PAL-land so 25 fps is great. Those that aren't will just have to use 30.

I'll be calling the h264_omx encoder from ffmpeg in bash, so where/how do I set the OMX_IndexParamBrcmVideoAVCSPSTimingEnable parameter?

Dave

[6by9]

Most of my users are in PAL-land so 25 fps is great. Those that aren't will just have to use 30.

I'll be calling the h264_omx encoder from ffmpeg in bash, so where/how do I set the OMX_IndexParamBrcmVideoAVCSPSTimingEnable parameter?

It would need to be a patch to https://github.com/FFmpeg/FFmpeg/blob/m ... odec/omx.c, and then rebuild FFmpeg.

I'm reviewing this area of the code anyway for other purposes, so it may be possible to change the default without breaking stuff. It needs a bundle of testing first though.

[g8gkq]

Thanks

I'm overdue a recompile of the ffmpeg binary that I use, so I'll take a look at it when I do that.

Dave