senten
Posts: 6
Joined: Thu Oct 13, 2016 8:40 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Sun May 13, 2018 2:45 pm

Thanks a lot for this excellent tutorial.

I was curious if this would work also using Ubuntu Mate on the Raspberry Pi 3 B. If anyone else is searching for the answer, I tried it today, it appears it does, at least for HW decoding :)

I followed the steps using Ubuntu Mate 16.04.4. I only checked the HW decoding, I did not follow the steps for HW encoding.

In steps 2 and 12, you can set the GPU memory by opening the /boot/config.txt and searching for "gpu_mem=", setting it there and uncommenting the line, then rebooting.

In step 3, linux-headers-rpi2 was not available, I installed raspberrypi-kernel-headers instead, I'm not sure if this is necessary or not.

Also, for step 14, in associating mpv with various video file types, I did not see an option for "Execute in terminal emulator", and because of this, I didn't get the keyboard control of mpv working yet.

Thanks again. (I'm not asking for any support here, just posting it for anyone looking for information on whether it's worth their while doing it on Ubuntu Mate on the Rpi3)

Using this in combination with acestream-engine and acestream-launcher, I can play 1080p fullscreen video from acestream using mpv. Works perfectly. :)

bloxwichkiwi
Posts: 1
Joined: Fri May 18, 2018 10:50 am

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Fri May 18, 2018 11:04 am

Hi Mike,

Brilliant tutorial. Thanks very much. Must have taken a lot of effort.
I did have a problem though with step 12;

I get this error with wscript modified as suggested;

...
Checking for Android support : no
Checking for Raspberry Pi support : no
You manually enabled the feature 'rpi', but the autodetection check failed.
[email protected]:/FFmpeg_Build/mpv-0.27.0 $

Modifying the last few lines of the section of script to the foillowing.......

lib=['mmal_core', 'mmal_util', 'mmal_vc_client', 'bcm_host']),
# We still need all OpenGL symbols, because the vo_opengl code is
# generic and supports anything from GLES2/OpenGL 2.1 to OpenGL 4 core.
check_cc(lib=["EGL","GLESv2"]),
),

gives...

...
Checking for Android support : no
Checking for Raspberry Pi support : yes
Checking for iOS OpenGL ES hardware decoding interop support : no
Checking for OpenGL without platform-specific code (e.g. for libmpv) : not found any of libmpv-shared, libmpv-static
Checking for MALI via Linux fbdev : no
Checking for OpenGL video outputs : yes
Checking for EGL helper functions : yes
Checking for FFmpeg/Libav present : yes
Checking for libav* is FFmpeg : no
Checking for libav* is Libav : no
Checking for Libav/FFmpeg library versions : not found any of is_ffmpeg, is_libav
Unable to find development files for some of the required FFmpeg/Libav libraries. You need at least FFmpeg 3.2.2 or Libav 12. Aborting.
[email protected]:/FFmpeg_Build/mpv-0.27.0 $

Any idea what I am doing wrong?

Thanks,
Dave.

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Sat May 19, 2018 10:25 pm

bloxwichkiwi wrote:
Fri May 18, 2018 11:04 am
[email protected]:/FFmpeg_Build/mpv-0.27.0 $

Checking for Libav/FFmpeg library versions : not found any of is_ffmpeg, is_libav
Unable to find development files for some of the required FFmpeg/Libav libraries. You need at least FFmpeg 3.2.2 or Libav 12. Aborting.

From just one tiny clue, I can tell your system is UNORTHODOX – or totally messed up!

That tiny clue is your command prompt.

On a completely "normal" Raspbian system, your command prompt while inside the mpv build directory should look EXACTLY like this:

[email protected]:~/FFmpeg_Build/mpv-0.27.0 $

If I enter the pwd command in this location on my perfectly normal system, I get this:

/home/pi/FFmpeg_Build/mpv-0.27.0

The pwd command stands for "print working directory" – although conceptually, I prefer to think of it as the "present working directory". It shows the FULL, EXPLICIT PATH that Terminal is currently located in.

But in your case, your command prompt says this:

[email protected]:/FFmpeg_Build/mpv-0.27.0 $

If you compare your command prompt to mine and look very carefully, you'll notice the little squiggly tilde symbol (~) is missing from your path. The tilde symbol acts as an abbreviation for your home directory – which on any "normal" Raspberry system is /home/pi. To the untrained eye, it may be "just a little symbol". But it actually has HUGE implications.

On your system, if you run the pwd command while Terminal is inside the mpv build folder, it will say this:

/FFmpeg_Build/mpv-0.27.0

That means something astounding – you placed the FFmpeg_Build folder at the ROOT level of your system! That means it exists at "/" instead of "/home/pi". This is definitely NOT an acceptable practice. The root has special properties and is intended "for system use only" – it's definitely not a location for you to be placing source code build folders. Among other things, it's not even owned by the user named "pi" – instead, it's owned by the user named "root". As a result, when logged in normally as user "pi", it has completely alien permissions. That means in Step 4 of my tutorial, you would have needed to deviate from my instructions by using "sudo mkdir FFmpeg_Build" instead of "mkdir FFmpeg_Build" to create the folder in the first place! Otherwise, you would have encountered a "permission denied" error message. By using "sudo", that also means the FFmpeg_Build folder would automatically be owned by root, not pi. Given my detailed instructions, my tutorial has every right to assume that folder is owned by pi – yet in your case, it would be owned by root.

Not to sound like a scolding professor, but this alone tells me that at least to some degree, you decided to do things "your own way". Not a good idea – especially when you consider how complicated and utterly unforgiving it is to build MULTIPLE programs from raw source code. And just to cover myself: Maybe you didn't encounter a permissions issue and therefore didn't have to use "sudo". Well, that would be equally messed up – because that would mean you previously made a completely non-standard, fundamental alteration to your root's permissions.

The missing tilde symbol also means you either don't have a normal home directory (which means you "tampered" with your system) – or for some reason, you do have a home directory (perhaps even /home/pi) – but, as already mentioned, you chose to do things your own way and created the build folder right off your root. My instructions clearly say to open Terminal – which, by default, will automatically open inside your home directory (/home/pi) on any standard, unaltered Raspbian system.

As you may have noticed, I repeatedly emphasized throughout my tutorial that your system must be completely "standard" and "normal" – I even defined those terms in great detail.

As if that weren't enough, I can tell something else from your command prompt – your host name is also unorthodox and non-standard.

Your host name is "hollandpi". On a normal system, it should simply be "raspberrypi". I'm not saying there's anything technically "wrong" about that in and of itself. After all, people are free to alter their host names – or do anything else they wish with their Raspberries!

But my point is that these unusual arrangements don't "just happen". They tell me that you're either an advanced user doing advanced things and thereby creating an unorthodox system in the process – or you're mostly just a regular user trying to do "advanced" things without really understanding their implications!

Either way, it spells trouble. It's a clear "violation" of my tutorial's well-defined rules that emphasize normality and standardization – so I'm not at all surprised that you're having problems. Since I'm able to extract all this concerning information from just a single command prompt, it makes me justifiably suspicious about other unknown "tweaks" you've probably made as well.

You also took it upon yourself to experiment with the wscript file. I realize you probably did that only after you encountered problems. But trust me – my version is the only correct one. By altering the script and leaving out SEVERAL critical elements, you were only digging an even deeper hole for yourself. Remember – everything in my tutorial has been thoroughly tested and proven to work. There's no legitimate reason to change ANYTHING!

Finally, be aware that just because your problems MANIFESTED in Step 12 does not necessarily mean that the SOURCE of your problems have anything to do with Step 12! For example, since mpv is totally dependent on my customized build of the FFmpeg engine in Step 11, even the slightest mistake in building FFmpeg can easily result in errors that only manifest later on while building mpv in Step 12. The untrained observer might therefore think "oh gee – what's wrong with mpv?" When in fact, the question itself is not even right!

Then again, FFmpeg might be perfectly fine. In many cases, the primary error you report – "Checking for Libav/FFmpeg library versions : not found any of is_ffmpeg" – can easily be caused by accidentally skipping the "export LIBRARY_PATH=/opt/vc/lib" line in my mpv build instructions. That essential line comes immediately after the wscript editing phase in my procedure – so it's entirely possible that you got distracted by your edits of the wscript file and accidentally skipped over that critical export command. I'm NOT saying that's what happened – your error messages are generic enough that several other possibilities exist.

No matter what your particular circumstances, however, there's really only one big thing to remember – when followed to the letter, my tutorial is proven to work.

BOTTOM LINE: Try a "tactical nuke" on the mpv build folder and see what happens. If that doesn't work, you'll need to do a "strategic nuke" on your entire system! I'm strongly guessing you'll need the latter.

What do I mean by tactical and strategic nukes? See my detailed troubleshooting guide that I wrote for someone else – it explains almost EVERYTHING you need to know. It also covers almost every conceivable mistake and oversight someone could possibly make on my tutorial:

viewtopic.php?f=38&t=199775&start=50#p1281045

johndavies
Posts: 155
Joined: Fri Dec 20, 2013 1:00 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Sun May 20, 2018 6:30 am

I added the following post to the Graphics, Sound and Media Section some time ago. Afterwards begin to think that I should have included it in the excellent TUTORIAL. Accordingly here it is.


Those who have built RPi_Mike's excellent enhanced mpv can use it to play and save some streamed material.To play, type mpv and the URL of the compatible streamed material into a Terminal and press Enter e.g.

Code: Select all

mpv https://abcnews24mha-lh.akamaihd.net/i/[email protected]/master.m3u8

To save material which can be played later you you add an "EXPERIMENTAL" mpv command --record-file="path to your chosen name for the recorded file.ts". An example is

Code: Select all

mpv https://abcnews24mha-lh.akamaihd.net/i/[email protected]/master.m3u8 --record-file="/home/pi/Aussinews.ts"
According to a post on the Reddit site, only .mkv and .ts files can be saved using this "EXPERIMENTAL" command. m3u8 files need to be saved as .ts files while all the others need to be saved as .mkv files The file is saved when the stream comes to an end or when the playback is stopped. I cannot give any more guidance as I know very little about streaming and the iptv world. A site where you can find streamed material to experiment with is https://github.com/notanewbie/LegalStream. You will need to click on most grouped URL file names to open them up to find out single URLs.

geevee
Posts: 1
Joined: Sat Jun 02, 2018 10:56 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Tue Jun 05, 2018 11:52 am

RPi_Mike: You are a saint and a beautiful man, you solved the problem I was experiencing PERFECTLY!

I followed your instructions once and got a perfect result. THANK YOU!

...and for the rest of you ungrateful human waste, you should all be thanking this guy.

-GEVEE

G160
Posts: 11
Joined: Sun Dec 23, 2012 1:32 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Tue Jul 10, 2018 4:41 pm

Thanks for this guide, its really well laid out and easy to follow.

I have one problem though, when using mpv for viewing hls streams, I'm getting quite a lot of buffering, after a bit of googling i discovered that quite a few changes were made to mpv from version 0.28 onwards which address this issue.

Is it possible to use version 0.28 with your method and if so how hard would it be?
I tried following your guide and installing 0.28 instead of 0.27 but i get errors at the step:

Code: Select all

./waf configure --prefix=/usr --enable-rpi
with that version.

Any help is appreciated.
Failing that if anyone knows of any other method/guide to get ffmpeg/mpv(v0.28) working with mmal acceleration, it would also be appreciated.

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Jul 11, 2018 6:02 am

G160 wrote:
Tue Jul 10, 2018 4:41 pm
Is it possible to use version 0.28 with your method and if so how hard would it be?
I tried following your guide and installing 0.28 instead of 0.27 but i get errors at the step:

NOTE TO READERS – JUST SO NO ONE GETS SCARED OFF BY MY TUTORIAL: This person chose to step outside my instructions and experiment on his own. There's nothing inherently wrong with that, but it obviously didn't work out too well in this case! My tutorial still works perfectly.

NOTE TO POSTER (G160): It is true that mpv's source code developers have made some improvements in the latest version (0.28). For most people, however, that slightly newer version offers no big advantage. My build (0.27) still works great and meets almost everyone's needs. However, just by coincidence, the one area you mention – HLS-based streaming – did receive a significant improvement. They enhanced the caching, so HLS-related buffering should work better in the newer version.

I therefore understand why you were tempted to build 0.28 – but there's one big catch you overlooked: As the British might say, your decision had unintended "knock-on" effects.

You see, the latest version of mpv also requires the latest version of FFmpeg!

As just one of many examples, mpv 0.28.x requires libavcodec version >= 58.7.100 as indicated here.

But the version of FFmpeg that my tutorial builds (3.4.x) contains libavcodec version 57.107.100 as indicated here.

FFmpeg is the "engine" that mpv uses – so these are absolutely critical dependencies that must be met. If mpv doesn't have EXACTLY what it needs, it simply won't work.

As of this writing – July 11, 2018 – the developers of mpv have not issued a fundamentally new "whole number" version of mpv in 7 months (since mpv 0.28.0 came out on December 24, 2017). Since that's only a few months newer than my build's version, it simply didn't make sense for me to spend hours and hours – or even days – completely re-doing the development, testing and re-writing of the entire tutorial just to update mpv's version by a matter of weeks. This is especially true when you consider that I'm doing all this for free!

However, I do keep my eye on mpv's "latest release" GitHub page. As soon as I see they've issued a fundamentally new version, I will probably do a complete update of my tutorial – assuming (1) I have the time to do all that and (2) I don't encounter an unexpected showstopper. New versions have new code, after all – so who knows what might happen! For all I know, it might not even work on the Raspberry.

Until then, you're free to experiment on your own. First and foremost, of course, you would need to build the latest version of FFmpeg (4.0.x) from raw source code. Also, be aware that if you do wish to experiment, you'll probably have to build the latest version of libass as well, since that's also a critical dependency. And to avoid any weird conflicts before you proceed, you'll also need to do a complete removal of all the packages you previously built. If you're not exactly sure how to do that, your best bet is to "nuke" your entire system with a brand-new install of Raspbian and try all over again from scratch.

Or better yet... just wait for RPi_Mike to set ya straight! :-)

G160
Posts: 11
Joined: Sun Dec 23, 2012 1:32 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Jul 11, 2018 6:30 am

@RPI_Mike
Yes i should clarify, i followed your tutorial to the letter and it worked perfectly, not a single error.
Playing backnormal MKVs,MP4s,etc worked excellently, so do normal internet streams, and weirdly even some HLS streams, just certain ones were buffering so i decided to do a little investigation.

I have mpv installed on my mac, which plays literally everything perfectly, that version installed from brew showed v0.28 so then it all started to become a little clearer.

Thanks for your explanation of the issue, i have a little time today and my raspian was a fresh install, so I'll keep a backup of the .debs created so if my attempts fail then i can just revert to your tried and tested method.

Many thanks again Mike, your work on this IS appreciated by some.

G160
Posts: 11
Joined: Sun Dec 23, 2012 1:32 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Jul 11, 2018 10:21 am

@RPI_Mike
Well that was an experience!
After building the latest libass, ffmpeg and mpv, it all went really smoothly until i attempted to play something when i was greeted with the dreaded 'Segfault' error.

God knows what the problem is, so ive reinstalled the .debs from your guide and im back up and runnning.
Shame i cant really use it for HLS streams as it is, but i'll have to live with it until someone can figure out what the hell is going on.
Cheers anyway.

senten
Posts: 6
Joined: Thu Oct 13, 2016 8:40 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Mon Jul 16, 2018 3:03 pm

@G160, as a possible workaround for playing back HLS streams, I wonder if streamlink would do the job? https://github.com/streamlink/streamlink

After installing it, you could try running

Code: Select all

streamlink <your HLS stream URL> best --player=mpv
Streamlink should take care of the HLS stuff, and pass it to mpv for decoding. (there's also a streamlink setting for using different cache sizes)

Of course this is going way off from RPi_Mike's intentions, and streamlink running in the background doing the HLS streaming business may mean the raspi cannot keep up, but might be worth trying out.

G160
Posts: 11
Joined: Sun Dec 23, 2012 1:32 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Mon Jul 16, 2018 5:18 pm

@senten
Thanks for the idea, i'll give it a go and see how i get on.
Nothing to lose really.

getty23
Posts: 1
Joined: Mon Aug 06, 2018 5:08 am

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Mon Aug 06, 2018 6:09 am

Hello,

I was wondering if anyone has already tried it on Ubuntu Core with a snap?
I try to get MPV running on Ubuntu Core but I am having some trouble (compare: https://forum.snapcraft.io/t/hdmi-outpu ... ay/5858/23).

If you have advices or tricks, just let me know.

Thanks and best regards

sramctc
Posts: 2
Joined: Thu Jun 14, 2018 8:26 am

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Tue Aug 07, 2018 1:21 am

Dear Mike,

I got this error message while building and installing MPV [mpv will not work without libass and FFmpeg] (STEP 12)

vutil >= 55.34.100 libavcodec >= 57.64.100 libavformat >= 57.56.100 libswscale >= 4.2.100 libavfilter >= 6.65.100 libswresample >= 2.3.100' not found)
Unable to find development files for some of the required FFmpeg/Libav libraries. You need at least FFmpeg 3.2.2 or Libav 12. Aborting.

is it normal or not?

Thanks you very much

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Aug 08, 2018 9:52 pm

getty23 wrote:
Mon Aug 06, 2018 6:09 am
I was wondering if anyone has already tried it on Ubuntu Core with a snap? I try to get MPV running on Ubuntu Core but I am having some trouble.

SO LET ME GET THIS STRAIGHT: You're trying to build a very intense video rendering program – mpv — and get it to work on an obscure, stripped-down, bare-bones operating system that's primarily built for IoT applications? And on top of that, you want it to work inside a sandboxed snap? And you want all this to happen on a little $35 Raspberry?

WOW! Unless you're a total genius with access to top-secret equipment that can "see" the tiny electrical impulses deep inside the Broadcom chip – and can use those insights to reverse engineer exactly how it handles video – your chance of pulling all that off is about... ZERO!

Even if you could somehow get mpv "working", it's extremely unlikely that it would ever support GPU-based hardware acceleration like my tutorial does. That means your frame rate for 1080p video would be about 2 SPF. And no – that's not a typo or a measure of sunscreen protection! That's 2 seconds per frame – not 2 frames per second! It reminds me of the 62-metric-ton M1A2 main battle tank. If you asked the guys who drive those things how many miles per gallon it gets, they would just laugh – because the only meaningful question is "how many gallons per mile does it get!" [The answer is almost 2, by the way – 2 gallons per mile!]

As I clearly indicated, my tutorial is designed to work with one specific operating system – the Full Desktop version of Raspbian Stretch, the Raspberry's official OS. The other official Raspberry operating system – the stripped-down "lite" version of Raspbian Stretch – will definitely NOT work.

One individual, however, did report some measure of success with Ubuntu Mate – because its underlying architecture is very similar to the full version of Raspbian. He did it by slightly modifying my tutorial (see post at the top of this page). But Ubuntu Core inside a snap? I would abandon all hope on that one!



sramctc wrote:
Tue Aug 07, 2018 1:21 am
I got this error message while building and installing MPV....... Unable to find development files....... You need at least FFmpeg 3.2.2....... Aborting.

is it normal or not?

SO LET ME GET THIS STRAIGHT: You're asking me if it's "normal" to get a catastrophic error message where the entire build comes to a crashing halt? No, that's definitely not normal! For those who have read my other posts over many months, I'm sure this will sound like a broken record: My tutorial, if followed to the letter, is documented to work perfectly. Tons of posts on here, from people all around the world, have firmly established that basic fact. So that's the giant elephant in the room. In other words, it means it's almost certain that you made a mistake. [Just to cover myself, there's always the very remote possibility that the Raspberry Pi Foundation just issued a Raspbian update a few days ago that somehow broke my tutorial. I would have to re-compile everything from scratch, every single day, to exclude that possibility with mathematical certainty – and I'm obviously not going to do that!]

But just to quickly address the error messages you mention: "Unable to find development files" would suggest that you forgot to install (or improperly installed) some or all of the critical dependencies in Step 3. One person who posted on here months ago, who complained about things not working, later admitted that he was "in a rush" and "probably skipped" that step. I honestly don't know why people even bother with my tutorial if they're not going to take it seriously. I think it's pretty obvious that it's not some casual thing you can do while watching your favorite TV show. It's literally 100% or nothing. 99% won't cut it.

And that message from mpv about you needing "at least FFmpeg 3.2.2"? It's hard not to chuckle on that one – because the whole point of the giant step just before it was to build FFmpeg 3.4.1. And while it's true I'm not a Fields Medalist like Ed Witten, I'm pretty sure that 3.4.1 is greater than 3.2.2. In other words, it means you didn't successfully build FFmpeg in the first place. Without a proper build of FFmpeg, mpv is pure fantasy – because FFmpeg is the "engine" that mpv uses.

Fortunately for you, I previously wrote an extremely detailed troubleshooting guide that covers virtually every single mistake a person could possibly make on my tutorial. As long as you follow my advice, you'll be in good shape. If you're not 100% certain of the current status of your Raspberry in every regard – and I suspect you're not – your best bet is to use my "strategic nuke" method. But check out all my thoughts before you make your final decision:

RPi_Mike's FFmpeg / mpv Troubleshooting Guide

wnorfl
Posts: 2
Joined: Tue Aug 21, 2018 2:22 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Tue Aug 21, 2018 3:01 pm

Can the capability to play audio CDs be added? Currently I get:

Code: Select all

$ mpv cdda://
Playing: cdda://
No protocol handler found to open URL cdda://
The protocol is either unsupported, or was disabled at compile-time.


Exiting... (Errors when loading file)
$
Of course, I've tried adding various mpv options such as "--cdrom-device=/dev/cdrom" without luck. "mplayer cdda:// -cache 1024" plays fine.

Superb HOWTO, the best I've ever seen. Thanks!

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Aug 22, 2018 12:42 am

wnorfl wrote:
Tue Aug 21, 2018 3:01 pm
Can the capability to play audio CDs be added?

Superb HOWTO, the best I've ever seen. Thanks!

Wow, music CDs? I honestly haven't touched that dinosaur technology in at least one-tenth of a CENTURY.

As you can tell from my tutorial, I carefully test my methods before I publish anything. Although I might have an old CD player collecting dust in a closet somewhere, I certainly don't have an even more obscure device just laying around – an "external USB CD/DVD player". That highly specific device is obviously what I would need to play audio CDs on my Raspberry.

So there's no way I would add "support" for an antiquated technology I can't even test – especially since it might break my entire tutorial! Based on my considerable experience in porting FFmpeg and mpv to the Raspberry platform, I've been shocked and surprised several times at how the slightest little change in procedure can produce all kinds of unanticipated "knock on" effects.

Nonetheless, if you feel daring and would like to experiment, I took the time to write up a THEORETICAL method to add CD support to mpv. I say "theoretical" because, as I just explained, I don't even have an "external USB CD player" to test my theory – and I certainly won't be going on Amazon to buy one!

Before I share my untested theoretical idea, let me say one more thing: I don't want to clutter this thread with a bunch of off-topic back-and-forth about a technology I don't "support". So if my theoretical approach works for you, that's great and I'd be happy to hear a simple "thank you". But if it doesn't work, I don't want to get sucked into troubleshooting something I don't even "support" and don't have the equipment to test.


So here's my UNTESTED and UNVERIFIED procedure to add audio CD support to mpv (on your existing system that you already did my tutorial on):


STEP 1: Completely remove both FFmpeg and mpv:

sudo dpkg -r ffmpeg mpv



STEP 2: Completely delete the build folders for FFmpeg and mpv:

sudo rm -r /home/pi/FFmpeg_Build/ffmpeg-3.4.1

sudo rm -r /home/pi/FFmpeg_Build/mpv-0.27.0



STEP 3: Completely delete the mpv configuration folder:

sudo rm -r /home/pi/.config/mpv



STEP 4: Install these CD-related dependencies from the official Raspbian Stretch repository:

sudo apt-get install libcdio-cdda-dev libcdio-paranoia-dev



STEP 5: Reboot – just to make absolutely sure everything is totally "fresh" on your system!



STEP 6: Completely re-do the entire build and installation of FFmpeg (re-do Step 11 in my tutorial from scratch) – *EXCEPT* carefully add the following item to the end of the "./configure" stanza. And when I say carefully, I mean carefully – make sure there's exactly ONE space between the "--enable" and the previous item, etc:

--enable-libcdio



STEP 7: Completely re-do the entire build and installation of mpv (re-do Step 12 in my tutorial from scratch) – *EXCEPT* carefully add the following item to the end of the "./waf configure" line:

--enable-cdda



STEP 8: Completely shut down your Raspberry and cut the power. Then attach your CD player and turn it on. Then wait about 10 seconds and turn your Raspberry back on. If you're super lucky, everything will work!



FINAL NOTE: Even if my procedure is perfect, you still have to use the proper syntax to get mpv to play a CD. It would be a real shame if my procedure worked perfectly, but because you were typing in the wrong mpv command to play a CD, it created the false impression that my procedure didn't work! So if it still doesn't work, make sure you do tons of Googling to see how others play a CD with mpv. The correct command appears to be the one you tried – mpv cdda:// – but again, I have no way of verifying anything without buying an "external USB CD player"!

And forget about mpv for a moment – you also need to make sure your Raspberry itself is recognizing your external USB CD/DVD drive in the first place! So before you do anything else, you need to verify that it's active and present at /dev/cdrom or /dev/sr0, for example. If you have an old data CD laying around, I would check that as well to see if you can open files from it on your Raspberry. Even the contents of a standard audio CD should at least be passively viewable. If you can't even do that, nothing you do with mpv will matter anyway. For more on this fundamental issue that has nothing to do with mpv, do a Google search ending with site:raspberrypi.org to constrain your results to this website. Good luck!

wnorfl
Posts: 2
Joined: Tue Aug 21, 2018 2:22 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Aug 22, 2018 6:04 pm

Code: Select all

Wow, music CDs? I honestly haven't touched that dinosaur technology in at least one-tenth of a CENTURY.
Audio CDs play great after the modifications detailed immediately above. While it's easy to cycle through tracks with "Page Up" and "Page Down", I've not hit upon a means to jump to a specified track. Otherwise, flawless. Many thanks!

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Fri Aug 24, 2018 1:02 am

wnorfl wrote:
Wed Aug 22, 2018 6:04 pm
Audio CDs play great after the modifications detailed immediately above. While it's easy to cycle through tracks with "Page Up" and "Page Down", I've not hit upon a means to jump to a specified track. Otherwise, flawless. Many thanks!

I'm glad my theoretical procedure worked perfectly and that you can now play audio CDs with mpv!

Now this may sound like nit-picking – although it's really not when you consider that mpv developers have literally spent tens of thousands of hours of "blood, sweat and tears" designing their completely FREE software: The fact you can't directly "dial in" specific track numbers on a CD is no more a "flaw" than the fact that the CD player inside a Rolls Royce can't do that either. In fact, I've never seen a physical CD player that comes attached with a numeric keypad like a telephone. Instead, you simply tap the "track forward" button until you get to the song you want. A Rolls Royce also can't fly. Although some might consider that a "flaw" as well, the correct perspective is that all of these things are simply "non-features" – not flaws.

Mpv has never claimed to be a specialized "CD app" or "CD navigation software". There are other programs out there that cater to that old technology. Mpv was built from the ground up to be one thing only – an outstanding file-based media player. In fact, no media player in the world – at least not on the Raspberry – does a better job at playing all these files: MP4, WebM, MKV, MOV, MPG, TS, AVI, WMV, MP3, M4A, WAV, FLAC... and many others too numerous to mention!

User avatar
rpdom
Posts: 12804
Joined: Sun May 06, 2012 5:17 am
Location: Ankh-Morpork

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Fri Aug 24, 2018 4:46 am

RPi_Mike wrote:
Fri Aug 24, 2018 1:02 am
In fact, I've never seen a physical CD player that comes attached with a numeric keypad like a telephone.
Seriously? Admittedly I've only seen a few. Otherwise the direct track select option is on the remote control keypad for all players that have one (which is most of larger ones).

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Fri Aug 24, 2018 5:43 am

rpdom wrote:
Fri Aug 24, 2018 4:46 am
RPi_Mike wrote:
Fri Aug 24, 2018 1:02 am
In fact, I've never seen a physical CD player that comes attached with a numeric keypad like a telephone.
Seriously? Admittedly I've only seen a few. Otherwise the direct track select option is on the remote control keypad for all players that have one (which is most of larger ones).

Yup, seriously! The key word in my statement was "attached" — as in an integrated part of the CD player itself. I admit, though, that I'm certainly no audiophile.

I've had cars with CD players — a simple slot to place the CD and a few control buttons: Up and down for volume, up and down for tracks.

And then I had a CD player that was part of a stereo component system. It had a remote control, but only basic buttons — no fancy "calculator pad" to dial in tracks. With a typical CD having only 10 or 12 songs, they probably figured there was no point.

rgriggs
Posts: 5
Joined: Mon Aug 27, 2018 10:15 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Tue Aug 28, 2018 3:54 pm

First, THANK YOU RPI_Mike for the great tutorial! I am able to play video from my webcam fullscreen with no problems at very low CPU utilization!

I'm wondering if you can advise on the following:

I would like to grab JPG screenshots from a h.264 video every 1 second. When I try to do this, the CPU utilization goes through the roof.
(My camera's stream is 1080P 1980x1080 resolution at 15fps)

Here is my command:

ffmpeg -vcodec h264_mmal -i rtsp://192.168.1.88:554/11 -r 1 test%03d.jpg

(To summarize, it uses the h264_mmal accelerated codec, grabs the input from the IP address specified, sets frame rate to 1 second, and saves to test___.jpg where ___ is an incremented number).

When I run this, it uses the mmal codec according to the log, but my CPU utilization goes to 100% for a single core.

I really need to be able to grab frames from up to 8 camera streams simultaneously, preferably at 5 fps, so this type of utilization is not acceptable.

I was hoping you could suggest some way to improve the performance? Again thank you very much for the tutorial and thank you for your time to read this question!

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Wed Aug 29, 2018 9:10 am

rgriggs wrote:
Tue Aug 28, 2018 3:54 pm
First, THANK YOU RPI_Mike for the great tutorial! I am able to play video from my webcam fullscreen with no problems at very low CPU utilization!

I'm glad my tutorial worked perfectly for you!



rgriggs wrote:
Tue Aug 28, 2018 3:54 pm
I'm wondering if you can advise on the following:

I would like to grab JPG screenshots from a h.264 video every 1 second. When I try to do this, the CPU utilization goes through the roof.
(My camera's stream is 1080P 1980x1080 resolution at 15fps)

Here is my command:

ffmpeg -vcodec h264_mmal -i rtsp://192.168.1.88:554/11 -r 1 test%03d.jpg

I really need to be able to grab frames from up to 8 camera streams simultaneously, preferably at 5 fps, so this type of utilization is not acceptable.

This doesn't have much of anything to do with my tutorial, but I'll respond anyway.

There are several false assumptions being made, but here are just a few:

First of all, invoking h264_mmal in your FFmpeg command line uses the GPU to provide hardware-accelerated DECODING of the H.264 video stream INPUT. That's perfectly fine and you seem to understand that part. But the "test%03d.jpg" component of your command line – which performs JPEG ENCODING and OUTPUT – has absolutely nothing to do with the totally separate H.264 input and decoding.

Although they are part of the same command line, the JPEG encoding and output is an unrelated function that FFmpeg is performing separately without any assistance from the GPU. Instead, that part of your command line is asking FFmpeg to perform software-based JPEG encoding on the CPU, not the GPU. It also has to extract the frame before it can do anything with it in the first place. There's nothing wrong with all that, but it does explain the CPU activity.

I have not personally tried it, but apparently some people have repurposed the GPU's built-in support for MJPEG encoding (an old video standard based on individual JPEGs) to do hardware-based JPEG encoding of individual high-resolution pictures at somewhere between 15 and maybe 30 pictures per second. But that's using a raw feed from the official Raspberry camera (not a third-party webcam, as you apparently are using). They're also not trying to stream and process 2 completely different formats simultaneously (H.264 and JPEG). When it comes to the Raspberry, nothing is faster than the official Raspberry camera – simply because it has been specifically designed to work with the Raspberry's hardware through a dedicated interface. Any third-party webcam would have lots of "connectivity overhead" that would slow things down even more – whether it's over a USB connection or WiFi.

I'm not sure what you're really trying to do, but your description sounds very similar to a multi-camera surveillance system. I have personally used MotionEyeOS at 1080p with ONE official Raspberry camera attached to ONE Raspberry and it worked great. But the best way to implement a high-definition "simultaneous multi-camera system" on the cheap – and whether multiple Raspberries are even appropriate for such a role – is not something I've ever looked into. Whatever the case, it's certainly not a topic for this thread.

But let's step back for a moment and contemplate what you want a single Raspberry to do: You want it to decode 8 separate 1080p camera feeds simultaneously while also capturing and encoding 40 1920x1080 JPEGs per second at the same time – 5 per second, simultaneously, from 8 independent videos.

Let's do some quick math:

(1920 x 1080 x 3 bytes of color data per pixel x 15 FPS x 8 simultaneous camera feeds) + (1920 x 1080 x 3 bytes of color data per pixel x 40 JPEGs per second) = PROCESSING ONE BILLION BYTES OF RAW PIXEL DATA PER SECOND!

Now this is certainly apples and oranges on several levels, but it should still give you at least some sense of the sheer magnitude of "data per second" you're talking about: When transferring a single contiguous file from my high-end Samsung internal SD card to an equally high-end Samsung USB thumb drive, I typically get a data transfer rate of only 15 MB per second on my Raspberry 3B+ (due to the slow internal bus speed, not my storage devices). That's a data rate 67 times slower than the kind of massive pixel data you're talking about. Realize also that when I transfer a file, that's just a simple movement of unprocessed data. There's no encoding or decoding taking place on top of it.

To be honest, this reminds me of having a scooter that only goes 10 miles an hour and wondering if some wizard out there can figure out how to make it go 670 miles an hour – while also popping wheelies at the same time.

Even that doesn't fully capture the magnitude of what you're talking about, however – because 8 separate 1080p camera feeds also means 8 separate input channels functioning simultaneously.

Since the total retail price of a Raspberry Pi 3 is only $35, I'm guessing that the "effective retail price" of the VideoCore GPU – which is not even a separate part but instead comes integrated with the overall Broadcom "system on a chip" – is no more than about $5 (and probably quite a bit less).

That means you're dreaming about the kind of extreme capability you would find on a high-end GPU card that consumes 250 watts (one-third horsepower) and costs 100 to 200 times as much! And even that wouldn't end the damage to your bank account – because a monster like that would also need a good home: A desktop computer that costs nearly 100 times as much as a Raspberry!

And what's the point of having 8 separate 1080p camera feeds if you can't watch all of them at the same time? A 77" 4K OLED TV would certainly do the trick. Although if you want to watch all the camera feeds at full resolution, you'd need to get two 4K TVs – and probably a second GPU to make sure it all runs smoothly!

rgriggs
Posts: 5
Joined: Mon Aug 27, 2018 10:15 pm

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Thu Aug 30, 2018 1:47 am

@RPi_Mike
First, thank you for taking the time to reply, even though my question was off-topic. The reason I asked in this topic was that it was related to the need for FAST hardware decoding as well as encoding (granted, not encoding to h.264, but encoding nonetheless...). I'm happy to start a new topic if you feel that would be better-suited to this discussion.

To put you in the picture as to WHY I thought this might be possible, please see this thread: https://www.raspberrypi.org/forums/view ... p?t=133875

TL;DR: This user reports he is able to decode, resize, and display NINE 1080p streams simultaneously by the Pi at around 6.5 fps!

After reading this post, I thought that since the pixel data was already IN MEMORY from being decoded and displayed, maybe it would not be much more CPU-intensive to grab and encode the image data to jpeg format. (Or I could simply forget about *displaying* the videos and do direct-to-jpeg encoding with similar performance as reported by this user.) I didn't think native jpeg encoding was that much of a CPU hog. My thought was that if the GPU could simultaneously decode all those streams (up to about 30mBit max), it should be easy for the CPU to grab a frame occasionally and write it to disk or ramdisk.

Hopefully this shows you where I was coming from - why I thought this might be possible.

Anyway, thank you for your time in your previous response. I appreciate your input.

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Fri Aug 31, 2018 12:19 am

rgriggs wrote:
Thu Aug 30, 2018 1:47 am
My thought was that if the GPU could simultaneously decode all those streams (up to about 30mBit max), it should be easy for the CPU to grab a frame

FRAME-GRAB TEST: You got me curious so I did a frame-grab test. I used the latest version of FFmpeg to extract JPEG images from a completely standard H.264 1080p video. The video I used was a "pre-made" MP4 that I previously downloaded from YouTube (it was the brief neutron star video from NASA that I reference in my tutorial). The point is, there was no streaming or "live input" going on, nor was there any video display or any other software running on the system – so my Raspberry 3B+ was free to devote almost all of its resources to this one task.

After extracting exactly 1,269 frames, the average frame-grabbing rate came out to only 5.2 FPS. Keep in mind this was for only ONE MP4 video. Speed, of course, is in the eyes of the beholder – but that kind of software-based frame rate is very typical for FFmpeg on the Raspberry. In other words, it's perfectly normal – yet utterly pathetic at the same time if your needs are 10 or 100 times greater.

Here's a brief summary of my Terminal session:

ffmpeg -i video.mp4 -vf fps=30 %04d.jpg

ffmpeg version 4.0.2 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1)

encoder: Lavc58.18.100 mjpeg

frame= 1269 fps=5.2 q=24.8 Lsize=N/A time=00:00:42.30 bitrate=N/A speed=0.173x


THE 25% REALITY: As you can see, FFmpeg is using the standard "Lavc" library of encoders to convert to JPEG. Lavc is short for libavcodec. My build of FFmpeg is also using the latest version of libavcodec – 58.18.100.

Like so many things in life, there's theory and then there's reality.

For example, in theory the Raspberry's CPU has 4 cores that can speed everything up!

In reality, the CPU has only ONE core, or at most "1.4 cores", for almost everything it does. I say 1.4 cores because I rarely ever see it go above 35%. I was actually impressed that when FFmpeg exported the JPGs, it hovered between 33 and 35%.

There are a few exceptions to what I'm saying – such as the amazing GCC compiler or the x264 encoder with NEON support. They make full 25% x 4 = 100% use of the CPU.

But I look at my CPU monitor all the time, and by far the most common reading I see when it's tasked with a big job is 25%. This is even true for "native" applications like LibreOffice Writer that come standard with Raspbian. Just do a simple search and replace on a giant text document and you'll see the CPU maxes out at only 25%. That's because it's very difficult to write genuine parallel code that's also customized for a specific processor architecture. ARM is also a bit of a stepchild in the computer world. A lot of developers that take the extra time to create custom parallel code for Intel/AMD-based systems simply aren't going to bother doing the same thing for ARM-based systems. Let's face it – ARM is dominant in the "tiny pocket computer" world. Intel/AMD are the dominant systems in the "big boy" world. Things are slowly improving in the ARM world, but it makes sense that for heavy-duty nerd applications like video encoding, only so many developers are going to do the extremely difficult extra work to customize their code for the ARM.

In fact, here's a shocking but very common example of what I'm talking about:

When I use the highly-regarded GIMP imaging program on my Raspberry to export a single PNG from a 1920 x 1080 image, it takes SEVERAL SECONDS to process! Now it's true that PNG exporting is much more computationally intensive than JPG, so this is certainly a bit of apples and oranges. It's also true that the "computational complexity" of images can vary dramatically, which has a big impact on the exporting time. Nonetheless, it still gives a good idea of how a seemingly simple task can eat up a tremendous amount of CPU time.

Here's the relevant timestamp data – the inode creation time (when the file was born) vs. the file modification time (when the last bit was written to it):

File: test.png
crtime: 2018-08-30 18:58:29.747153460
mtime: 2018-08-30 18:58:41.245832484

As you can see, this particular PNG – a single image – took almost 12 seconds to export. That's a frame rate of only 0.08 FPS. That's 375 times slower than real time (30 FPS)!

And guess what? The entire time the CPU was stuck at a very typical 25%.


THE "REAL" CPU: In theory, the Raspberry 3 has an impressive 64-bit ARMv8. In practice, it really has a 32-bit ARMv6 / ARMv7. Why do I say this? For understandable reasons, the Raspberry Pi Foundation wants to maintain backward compatibility with older models. They also don't want the nightmare of having to maintain two completely separate operating systems (32-bit and 64-bit) – one for older systems, and one for newer systems. Remember – it's a non-profit, not a multi-billion dollar corporation. Unfortunately, since the operating system is the "middle man" between you and the CPU, that has the effect of "watering down" the true capabilities of the CPU. When I compile FFmpeg and x264, for example, the operating system prevents me from making anything more than 32-bit ARMv6 optimizations. On top of that, those optimizations only apply to a limited subset of FFmpeg's total features. So for many things, it's not getting any specialized CPU acceleration.


SIMPLE ISN'T SIMPLE: In theory, all that FFmpeg has to do is simply "export the JPG". Now I'm certainly not an expert on the inner workings of FFmpeg's source code or the Libavcodec suite, but my guess is that there's a whole bunch of "pre-processing" that has to first take place before it can even think about the actual JPG exporting process. A classic example of this would be the fact that unlike MJPEG video streams – where each frame is a completely normal "standalone" picture – H.264 video is vastly more complex. Instead of being a stitched-together series of simple pictures, it actually consists of (I)ntra-coded frames, (P)redicted frames and (B)i-directional frames. The I frames are fairly simple – because they're the rough equivalent of a standalone JPEG image. But P and B frames are not complete pictures. Instead, they contain some combination of motion vector displacement and image data. So FFmpeg almost certainly has to perform a considerable analysis of the "incomplete" P and B frames in the larger context of the video before it can process them into legible images that are suitable for export. Some might say, "oh, but the fancy hardware-based H.264 decoder in the GPU has already done that part." But keep in mind that's only relevant to FFmpeg if the developers have taken the time to write custom code so that FFmpeg can directly communicate with the internal workings of the Raspberry's proprietary VideoCore GPU. I seriously doubt that is the case. So FFmpeg is probably forced to look at the stream completely "fresh" through its own eyes and do everything in software on the CPU.


VIDEO WALL: What that person did in displaying 9 simultaneous videos (3 x 3) with a single Raspberry was truly impressive and I would have no clue how to do that myself. But when you dig into the details, it's not quite what it seems. Here's what I mean. In his final update on the link you shared, he reported these stats on a non-overclocked Raspberry:

Non-overclocked
1x1 1080p 50 fps
2x2 1080p 12 fps but screen is blank
3x3 640x480 32 fps


Now let's consider for a moment how the "video wall" compares to my tutorial. In other words, how many pixels can each one pump out each second? As you know, my tutorial cranks out 60 frames per second at 1080p. So here's the math:

MY TUTORIAL BUILD – 1920 x 1080 x 1 @ 60 FPS:
1920 x 1080 = 2,073,600 pixels per frame. 2,073,600 x 60 FPS = 124,416,000 pixels per second

3 X 3 VIDEO WALL – 640 x 480 x 9 @ 32 FPS:
640 x 480 = 307,200. 307,200 x 9 = 2,764,800 pixels per frame. 2,764,800 x 32 FPS = 88,473,600 pixels per second

That means my customized build of FFmpeg / mpv is actually 40% faster. It's pumping out 36 MILLION more pixels per second than the video wall!

That's why I maintain that if anyone wants to decode and display 8 simultaneous 1080p streams while simultaneously exporting JPEG frame grabs from each independent video simultaneously, they definitely should be thinking of a system a lot closer to $3,500 than $35. Maybe a brand-new $2,000 system – but I can't imagine going much lower than that!

User avatar
RPi_Mike
Posts: 91
Joined: Sat Dec 09, 2017 12:57 am
Location: United States

Re: TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv

Sat Sep 01, 2018 6:46 pm

TO ALL THE READERS OUT THERE: Within the hour, I'll be posting a massive new update of my entire tutorial. I would describe it as "version 3.0". I'm very pleased with the results and it will probably be my final word on this subject – at least until Raspbian Buster, the next whole-number release of the Raspberry's operating system, comes out in the middle or end of 2019.

After everything's published and you've had a chance to review it, please feel free if you have any questions!

Return to “Graphics, sound and multimedia”

Who is online

Users browsing this forum: No registered users and 7 guests