Hardware encoding a webcam stream to Youtube

[MichaelF91]

Hi all,

I've been going round in circles for a couple of weeks on this now.

I'm running 64-bit Ubuntu 19.10 on a Raspberry Pi 4B 4GB with Gnome desktop as a server. I have a couple of Rapsberry Pi Zero-based cameras running RPi Cam Web Interface which streams Motion JPEG on my local network. The goal is for the the RPi 4B server to take in these streams and beam them up to Youtube, which I can do without much problem using software H264 encoding with ffmpeg.

The issue is that using software encoding is obviously very CPU intensive - I would like to do the heavy lifting in hardware.

With the differences between older RPis and the RPi 4B, as well as using Ubuntu vs Raspbian and 64-bit vs 32-bit, I just haven;t been able to get my head around how to make this all work with hardware encoding.

The closest I can get is by running this command (on a random .gif from the internet):

Code: Select all

ffmpeg -i test.gif -c:v h264_omx test.mp4 -y

Which produces the following output:

Code: Select all

pi@pi4:~$ ffmpeg -i test.gif -c:v h264_omx test.mp4 -y
ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)
  configuration: --prefix=/usr --extra-version=1build2 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, gif, from 'test.gif':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: gif, bgra, 500x375, 14.25 fps, 28.58 tbr, 100 tbn, 100 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
[h264_omx @ 0xaaaacb5fe4a0] libOMX_Core.so not found
[h264_omx @ 0xaaaacb5fe4a0] libOmxCore.so not found
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

Or, if I use the encoder "h264_v4l2m2m", I get:

Code: Select all

pi@pi4:~$ ffmpeg -i test.gif -c:v h264_v4l2m2m test.mp4 -y
ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)
  configuration: --prefix=/usr --extra-version=1build2 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Input #0, gif, from 'test.gif':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: gif, bgra, 500x375, 14.25 fps, 28.58 tbr, 100 tbn, 100 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (gif (native) -> h264 (h264_v4l2m2m))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0xaaaafe6264a0] Could not find a valid device
[h264_v4l2m2m @ 0xaaaafe6264a0] can't configure encoder
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

Anyone got any pointers?

[dan s]

I'm not sure what you are asking.

• Are you asking how to grab the feed, so you can feed it into ffmeg?
• Are you asking how to ensure hardware/gpu acceleration?
• Are you wanting to write videos to the harddrive to be uploaded later, stream live to youtube?
• Or are you just needing some general help with ffmpeg?

[MichaelF91]

I'm not sure what you are asking.

• Are you asking how to grab the feed, so you can feed it into ffmeg?
• Are you asking how to ensure hardware/gpu acceleration?
• Are you wanting to write videos to the harddrive to be uploaded later, stream live to youtube?
• Or are you just needing some general help with ffmpeg?

Thanks for your reply. To clarify, I'm stuck on point #2. The only problem I have at the moment is using hardware acceleration for the encoding.

Specifying the video codec as h264_omx in ffmpeg on Raspbian works out of the box - the process is hardware accelerated. I am having trouble replicating that when using Ubuntu. (Of course, the easy answer would be to say "Well just use Raspbian, then", but I feel that this should be technically possible and I would really like to run Ubuntu on my Raspberry Pi.)

[dan s]

Thanks for your reply. To clarify, I'm stuck on point #2. The only problem I have at the moment is using hardware acceleration for the encoding.

Specifying the video codec as h264_omx in ffmpeg on Raspbian works out of the box - the process is hardware accelerated. I am having trouble replicating that when using Ubuntu. (Of course, the easy answer would be to say "Well just use Raspbian, then", but I feel that this should be technically possible and I would really like to run Ubuntu on my Raspberry Pi.)

Run this command and see what's listed.

Code: Select all

ffmpeg -encoders

It might be that the proper drivers aren't available in Ubuntu.

[MichaelF91]

Thanks for your reply. To clarify, I'm stuck on point #2. The only problem I have at the moment is using hardware acceleration for the encoding.

Specifying the video codec as h264_omx in ffmpeg on Raspbian works out of the box - the process is hardware accelerated. I am having trouble replicating that when using Ubuntu. (Of course, the easy answer would be to say "Well just use Raspbian, then", but I feel that this should be technically possible and I would really like to run Ubuntu on my Raspberry Pi.)

Run this command and see what's listed.

Code: Select all

ffmpeg -encoders

It might be that the proper drivers aren't available in Ubuntu.

Code: Select all

pi@pi4:~$ ffmpeg -encoders | grep h264
ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)
  configuration: --prefix=/usr --extra-version=1build2 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
 V..... libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V..... libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V..... h264_vaapi           H.264/AVC (VAAPI) (codec h264)

[dan s]

So it sees the encoders, have you tested with an actual mjpeg video file?

[MichaelF91]

So it sees the encoders, have you tested with an actual mjpeg video file?

Yup, exactly the same output as the original post.

[dan s]

add one of these commands before the input and see if it spits out anything useful.

Code: Select all

-loglevel repeat+level+verbose

Code: Select all

-loglevel repeat+level+trace