MAJOR UPDATE – JANUARY 31, 2018 – ALL 8 TUTORIAL APPENDICES:
I have made numerous additions and enhancements to my FFmpeg and mpv tutorial – including the elimination of rpi-update. So much has been added, in fact, that I have now exceeded the 60,000 character limit!
As a result, I had to free up space by removing several appendices from my tutorial's main page:
TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv
I am therefore including all 8 appendices in this (separate) post:
APPENDIX 1: MOST ESSENTIAL MPV KEYBOARD CONTROLS AND OPTIONS:
Toggle the video between full screen and actual size
Toggle the video (or audio) between pause and play
left and right arrow keys:
Rewind and fast-forward by 6 seconds
up and down arrow keys:
Rewind and fast-forward by 1 minute
9 and 0 keys:
Lower and raise the volume
Make the on-screen display briefly appear to show time remaining, etc.
[ and ] keys:
Speed up or slow down the video
Quit out of the video (or audio)
REMINDER: Be sure to close any open Terminal windows before you use mpv, or you will lose keyboard control.
CUSTOMIZE MPV's BEHAVIOR:
If you wish to customize things further, you can always modify mpv's configuration file (mpv.conf). The mpv.conf file is a standard text file (but with a ".conf" extension) that can be quickly modified with the standard Text Editor. It's located at /home/pi/.config/mpv (you may have to temporarily click View | Show Hidden in order to see these hidden files and folders in File Manager). In my instructions, I already included 2 of the most useful customizations inside the mpv.conf file:
<---This forces all videos, both large and small, to automatically expand to fill your screen. To prevent any distortion, this option always maintains the native aspect ratio of the video. By design, it will never "stretch" or "squeeze" the video to fit the shape of your monitor or TV. But as long as the video's aspect ratio matches the aspect ratio of your screen, the video will completely and perfectly FILL the screen. For example, if you have a 480p, 720p or 1080p video (all of which would normally have a 16:9 aspect ratio) – and you also have a monitor or TV with a 16:9 aspect ratio – then your entire screen will be filled with active pixels.
<---Sometimes, you may watch an old video – or one that's simply not as "wide" as a 16:9 aspect ratio. For example, you might have an old video from a VHS tape with a low resolution of 320 x 240 (a 4:3 aspect ratio). Or you might have a DVD video with a resolution of 720 x 480 (a 3:2 aspect ratio). Either way, these videos are not "wide" enough to perfectly fit the shape of a modern 16:9 display without being stretched or distorted. In those cases, the "rpi-background=yes" option will automatically add black bars to the left and right sides of your screen so that there are no visual distractions in the background!
<---I did NOT add this option in my instructions because many people would not want this particular behavior. But if you're using your Raspberry to power a display at a trade show or other event, for example, this is a great way to make the video automatically play over and over again.
<---This mpv.conf line will force a 1080p video to play at only 4% of the total surface area of a 1080p monitor or TV – and it will place it in the upper-left corner of your screen. NOTE: In case you're wondering how 20% becomes 4%, it all comes down to understanding the difference between linear dimensions and surface dimensions. Assuming the native resolution of the video is the same as your screen's resolution, 20% smaller across the X axis AND 20% smaller across the Y axis means the video would only cover 4% of your screen's total surface area (20% x 20% = 4%).
<---And what if I want a tiny video to appear in the upper-right corner of my screen? For this mpv.conf example, simple math is in order: My 1080p monitor is 1920 pixels across. 20% of 1920 is 384. And 1920 - 384 = 1536. That means the upper-left corner of the video should start at 1536 on the X axis and 0 on the Y axis. In both of these examples, tapping the "f" key will instantly toggle the video back and forth between FULL SCREEN and TINY SCREEN. Mpv handles it beautifully in the blink of an eye.
TAKE MANUAL CONTROL OF MPV:
The contents of the mpv.conf file affects mpv's "fundamental" behavior. In other words, it will always affect the way mpv behaves when it plays video and audio. But for some options, you can temporarily invoke a behavior on a one-time basis without modifying your mpv.conf file. Generally, this only works for options that begin with 2 dashes (--). For example, let's say you do NOT have the "--loop" option in your mpv.conf file, but you still want to launch a video with that behavior on a one-time basis. To do that, you can manually launch the video inside Terminal (instead of double-clicking it in File Manager as you would normally do). Here are some common syntax examples for manual launching of multimedia files:
This is the MANUAL way to play a video with mpv. Normally, as I just mentioned, you would simply double-click the file and it would automatically start playing – but I'm including these manual examples for the benefit of interested users. In this example, we are NOT invoking any special options (other than what's already in your mpv.conf file). It also assumes that Terminal is currently open inside the same folder as your video:
This manually instructs the video to loop continuously (in addition to whatever else your mpv.conf file may have it do). It also assumes that Terminal is currently open inside the same folder as your video:
mpv --loop video.mp4
This does the same thing as Example B, but it does so for a video that's in a completely different folder than the one Terminal is currently located in (which is normally your "home" folder by default – /home/pi). In other words, in this example, you're using the FULL DIRECT PATH to the video. The full path to a file can be easily obtained by right-clicking any file in File Manager and selecting "Copy Path(s)". This syntax is necessary whenever Terminal and the video are in different locations:
mpv --loop /home/pi/vacations/hawaii/video.mp4
This also does the same thing as Example B, but it places the full path inside standard quote marks. Why is this important? You must do this any time there are SPACES in the file path. That's a Linux thing, not a mpv thing. For example:
mpv --loop "/home/pi/truly awesome vacations/hawaii/video.mp4"
MPV HAS HUNDREDS OF OTHER OPTIONS AND FEATURES:
For complete details, please review mpv's extremely thorough manual at their official website:
Official Manual: mpv
APPENDIX 2: DOWNLOAD AND RETAIN HIGH-QUALITY VIDEOS FROM HUNDREDS OF WEBSITES – WITH YOUTUBE-DL:
Although youtube-dl is available in the Raspbian software repository, it’s nearly a year old. To acquire the very latest version from youtube-dl's official website, enter the following two command lines, one at a time, in the Terminal window:
sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl
DOWNLOADING MP4 VIDEOS (H.264 / AAC):
Let’s say you have your Raspberry hooked up to a high-definition 1920 x 1080 monitor or TV and you wish to download and enjoy a NASA video, at maximum resolution and quality, of two neutron stars colliding – with the screen completely filled-up by the video. Here’s how you do that with youtube-dl:
Identify the URL of the page that contains the video you want. In this example, it’s a YouTube video at the following URL:
Open the Terminal window and enter the following code (make sure you use a capital “-F”):
youtube-dl -F https://www.youtube.com/watch?v=x_Akn8fUBeQ
You will then see an impressive list of all the different audio and video formats that the YouTube server has available for the NASA video. If you look carefully, you will see there’s a video track with a resolution of 1920 x 1080 that uses the “avc1” codec at 30 FPS (Frames Per Second). “AVC1” is simply another name for the H.264 codec. The Raspberry’s GPU has been specifically optimized to play H.264 video with full-blown hardware acceleration. On the far left column, you will see a “format code” for that particular video format. In this case, it’s code “137”. You will also see a high-quality audio track that’s described as an “m4a” with a sampling rate of 44100Hz. That’s YouTube's terminology for an AAC audio track. It has format code “140”. It's the best choice for this example, since an AAC audio track is specifically designed to work with an H.264 video track when creating an MP4 video. To download the high-resolution video (with sound), you will therefore issue the following command (make sure you use a lower-case “-f”):
1920 x 1080 MP4 – 30 FPS:
youtube-dl -f 137+140 https://www.youtube.com/watch?v=x_Akn8fUBeQ
Once the download is complete, be sure to close the Terminal window or you will lose keyboard control of mpv.
Then, simply double-click the MP4 video you just downloaded and mpv will automatically start playing it full screen! If you downloaded a resolution smaller than your screen, you can tap the "f" key to toggle back and forth between actual size and full screen mode.
DOWNLOADING WEBM VIDEOS (VP9 / Opus):
If you want a much smaller file size at a surprisingly high quality – albeit with lower resolutions of 720p or 480p – the “VP9” format for video and the “Opus” format for audio are the ones you want. They are the most advanced compression technologies currently available. The two tracks will be automatically combined by youtube-dl (using the FFmpeg engine) and merged into a standard WEBM video. Be aware that the Raspberry’s GPU is only designed to provide full hardware acceleration for MP4 videos using H.264 – so a 1080p VP9-based WEBM video will push your Raspberry to the absolute limit and will likely result in at least some stutter. That's why I recommend 720p and 480p resolutions for VP9-based videos. They also, of course, should not exceed 30 FPS. Finally, if you wish to share the video with someone else who doesn’t have an advanced player like mpv, it’s best to stick with a standard H.264-based MP4 video for compatibility reasons. For the NASA video, here are the two best command lines if you want smaller file sizes (for example, if you're using a cellular connection and need to conserve your data usage). When it's competently encoded by the uploader, I've been consistently amazed by how good a humble 480p video can look when viewed full screen on my large 24 inch 1080p monitor.
1280 x 720 WEBM – 30 FPS:
youtube-dl -f 247+249 https://www.youtube.com/watch?v=x_Akn8fUBeQ
854 x 480 WEBM – 30 FPS:
youtube-dl -f 244+249 https://www.youtube.com/watch?v=x_Akn8fUBeQ
HIGH FRAME-RATE CAPABILITY:
In my testing, I was surprised to discover that the Raspberry’s GPU can handle 1920 x 1080 H.264 video at DOUBLE the normal frame rate – 60 FPS without dropping even one frame! Some newer videos on YouTube are now available in 60 FPS format, and the NASA video is one of them. If you wish to download the 60 FPS version of the above 1080p video, you would simply change the video format code from 137 to 299. In other words, you would use this command line instead:
1920 x 1080 MP4 – 60 FPS:
youtube-dl -f 299+140 https://www.youtube.com/watch?v=x_Akn8fUBeQ
Don't assume the format codes will always be the same. YouTube is quite consistent – but even with them, the available formats can vary from one video to another. For example, I've noticed that YouTube's servers will often place different formats in a queue, prioritizing H.264 over VP9. Even for the mighty Google, encoding video is very computationally intensive and time consuming – so it makes sense for them to quickly get the H.264 version up on their server, and then process the VP9 version an hour or half hour later. Other video sites, however, use completely different format codes and are typically limited to H.264 videos – so for them, you'll want to first run youtube-dl's "-F" option (capital F), as explained above, to see exactly what formats are available. Finally, be aware that most other sites tend to integrate the video and audio tracks into a single pre-packaged MP4 video – so there won't be separate video and audio formats to choose from. YouTube does this as well, by the way. It's usually listed in the last couple lines of their available formats. In YouTube's case, you'll know if everything's already been combined if it says both "avc1" and "mp4a" in the description. In those cases, you would not use the "+" symbol. Instead, you would simply use a single format code. For more information on the many other features and capabilities of youtube-dl, visit their official website at yt-dl.org
APPENDIX 3: PLAY ONLINE VIDEOS AUTOMATICALLY WITH MPV (WITHOUT ANY MANUAL DOWNLOADING):
Personally, I like to review all the available video and audio formats before I download a video. If it's a concert or music video, I may want to enjoy it multiple times. By downloading it, as described in Appendix 2, I have immediate access to it at the highest quality – even if I'm offline. Downloading is also crucial if you're using a cellular data connection and need to conserve your data. Even with an "unlimited" plan, your carrier may still throttle back your speed once you reach a monthly limit. For those instances, I've found that 480p videos that use the advanced VP9 video codec and Opus audio codec at 50 or 70 kbps are the perfect balance between file size and quality. In an HD aspect ratio, they typically have resolutions of 854 x 480. For a relatively small video, they look surprisingly good on my large 1080p monitor in full screen mode. Of course, if you want the absolute best in quality on a 1080p screen and data use is not a concern, there's nothing like a 1080p video.
In many cases, however, data use may not be a concern. You also may not have any desire to manually download (and retain) the video. And finally, you may not really care if you're viewing the most optimal format and quality – because you just want to watch a video without any thought. In those cases, I have configured my procedure to support tight integration between mpv and youtube-dl. As long as you have properly followed this tutorial and have installed a recent copy of youtube-dl (see Appendix 2), all you have to do is enter a URL after the mpv command! That will automatically invoke youtube-dl, which will start downloading the first several seconds of the video (invisibly as a background process). It does this to create a sufficient cache to help prevent buffering interruptions while playing. Once it's done that, mpv will automatically start playing the video! Generally, when operating in this mode, youtube-dl's algorithm will attempt to choose the "best quality" version of the video. It works well on leading video sites like YouTube, but there are no guarantees with less sophisticated websites. If you're unsatisfied with the results on other sites, nothing of course beats a manual review of the available formats (as described in Appendix 2).
As an example, if you want to automatically play the NASA video mentioned in Appendix 2, this simple command line is all it takes:
APPENDIX 4: SAMPLE FFMPEG COMMAND LINES – TO ENCODE OR PROCESS VIDEO AND AUDIO:
NOTE: In the following examples, simply right-click any folder that contains the media files you wish to encode or process and click “Open in Terminal”. Then, run the following commands (changing the file names and settings where appropriate):
EXAMPLE 1: Convert an old AVI video (with sound) – or any other input video format you choose – into a modern MP4 (with H.264 video and AAC audio):
ffmpeg -i input.AVI -vf format=yuv420p -c:v libx264 -crf 23 -c:a libfdk_aac -b:a 128k output.MP4
NOTE: This example uses the higher-quality (but slower) x264 software-based encoder. That means there is no GPU hardware acceleration – however it does use a CPU optimization known as “NEON” that helps to speed up the encoding. The “-crf 23” setting is the default video quality and generally produces excellent results. The lower the number, the higher the encoding quality (at the expense of a larger file size). A setting of 18 is said to be “visually lossless” – but the file size will be almost double that of a 23 setting. Be aware that the number scale is non-linear – but as a rough rule of thumb, a change of 5 to 6 will double (or cut in half) the file size. In many cases, when a smaller file size is important, a setting of 26 still maintains a high-quality appearance. Finally, the “-c:a libfdk_aac -b:a 128k” part indicates that the audio will be converted with the Fraunhofer AAC codec, at a bit rate of 128 kbps.
EXAMPLE 2: Resize a large “1080p” MP4 video (1920 x 1080) with sound into a smaller “480p” MP4 video (854 x 480) with sound – without (unnecessarily) re-encoding the original audio:
ffmpeg -i input.MP4 -vf "format=yuv420p, scale=854:480:flags=lanczos" -c:v libx264 -crf 23 -c:a copy output.MP4
EXAMPLE 3: Extract every frame from a 30 FPS MP4 video and output it as a padded, 7-digit numerical JPEG image sequence:
ffmpeg -i video.mp4 -vf fps=30 -q:v 2 %07d.jpg
EXAMPLE 4: Convert an alphanumeric image sequence of JPEG pictures into a 30 FPS MP4 video:
ffmpeg -framerate 30 -pattern_type glob -i "*.jpg" -vf format=yuv420p -c:v libx264 -crf 23 output.MP4
EXAMPLE 5: Add a transparent overlay – such as text or a logo – on top of a video:
ffmpeg -i input.MP4 -i overlay.png -filter_complex "[0:v][1:v] overlay=0:0" -c:v libx264 -crf 23 -c:a copy output.MP4
NOTE: The “overlay.png” file must be a transparent PNG file (with alpha channel) that includes your text or logo. This can be created with any standard image-manipulation program like GIMP or Photoshop. Also, in this particular example, the inputted MP4 video is assumed to have a proper sound track that can simply be copied over without re-encoding. If the sound does require encoding, simply change “-c:a copy” to “-c:a libfdk_aac -b:a 128k”.
EXAMPLE 6: Use the hardware-accelerated H.264 encoder to convert an AVI into an MP4:
ffmpeg -i input.AVI -c:v h264_omx -b:v 4500k -c:a copy output.MP4
NOTE: The hardware encoder, though about 3 times faster, has fewer options and generally, bit for bit, produces a decent but lower-quality result. The software-based encoder, which relies on the versatile and general-purpose CPU, provides the programmers maximum flexibility to use whatever coding techniques they like – but a hardware encoder, by its very nature, is constrained by the features the GPU supports. Like most things in life, there are pros and cons to each approach. In this example, the “-b:v 4500k” part indicates a bit rate of 4,500 kbps – which is generally good for a large 1080p video. Unlike the software-based encoder, there is no “crf” quality setting, since the hardware-based encoder uses bit rate as a proxy for quality. If your video is smaller than 1080p, you will obviously want to lower the bit rate accordingly – to 2,000, 1,500, etc. Finally, this example assumes the existing audio is already in an MP4-compatible format, such as an AAC or MP3, and can thus be copied over. If you do need to transcode the audio, simply change “-c:a copy” to “-c:a libfdk_aac -b:a 128k”.
FINAL NOTE: The 6 examples I just provided are only a small sample of the astonishing capabilities of FFmpeg. For more information, visit their official website at FFmpeg.org.
APPENDIX 5: IF FOR SOME REASON YOU WISH TO COMPLETELY UNINSTALL ALL THE PROGRAMS YOU JUST INSTALLED IN THIS TUTORIAL, JUST RUN THIS COMMAND LINE IN TERMINAL:
sudo dpkg -r mpv ffmpeg libass lame opus libvpx fdk-aac x264-snapshot-20180125-2245
THEN, DELETE THE "preferences" FILE YOU USED TO "PIN" THE PROGRAMS (IF YOU HAVE MANUALLY PINNED OTHER PROGRAMS UNRELATED TO THIS TUTORIAL, YOU'LL OBVIOUSLY WANT TO EDIT THIS FILE, NOT DELETE IT):
sudo rm /etc/apt/preferences
APPENDIX 6: IF FOR SOME REASON YOU NEED TO RE-INSTALL ALL THE PROGRAMS IN THIS TUTORIAL ON YOUR *EXISTING* SYSTEM, YOU CAN DO IT IN LESS THAN ONE MINUTE! BY "EXISTING" SYSTEM, I MEAN THE SAME RASPBERRY WITH THE SAME OPERATING SYSTEM THAT YOU WERE USING WHEN YOU DID MY TUTORIAL. BY "EXISTING" SYSTEM, I ALSO MEAN THAT YOUR CORE SYSTEM AND "FIRMWARE" FILES HAVE NOT BEEN ALTERED. IF YOU HAVE SINCE RE-INSTALLED THE OPERATING SYSTEM OR IT'S A NEW SYSTEM – OR IT JUST ISN'T YOUR "EXISTING" SYSTEM FOR WHATEVER REASON – YOU NEED TO USE APPENDIX 7 INSTEAD. BUT IF IT TRULY IS YOUR EXISTING SYSTEM, SIMPLY OPEN TERMINAL IN WHATEVER FOLDER YOU PLACED ALL 8 .DEB FILES (SEE STEP 13 OF MY TUTORIAL FOR MORE ON THIS TOPIC) AND RUN THIS COMMAND:
sudo dpkg -i *.deb
APPENDIX 7: IF YOU WISH TO INSTALL ALL THE PROGRAMS IN THIS TUTORIAL ON ANOTHER RASPBERRY PI 3 – OR ON THE SAME RASPBERRY PI 3 AFTER A FRESH INSTALL OF THE OPERATING SYSTEM – IT ONLY TAKES A FEW MINUTES! JUST FOLLOW THESE 7 SIMPLE STEPS:
Make sure the other Raspberry has a completely normal, standard copy of the Raspbian Stretch operating system. Please see my tutorial to find out exactly what I mean by "normal" – both the requirements section and Step 2 have critical information on this topic. If you're certain that everything is perfectly "normal" with the other Raspberry, simply open Terminal and do a standard update to make sure everything is current:
sudo apt-get update
sudo apt-get dist-upgrade
Set your GPU memory allocation to 128 MB (mpv requires 128 MB to work). Proceed as follows:
Click the Raspberry Menu | Preferences | Raspberry Pi Configuration | Performance Tab
Change "GPU Memory" to 128
Click OK | Click No to reboot (You’ll be rebooting soon anyway – so rebooting now is not necessary.)
Install all the basic “dependencies” that FFmpeg and mpv require by running this entire stanza in Terminal:
sudo apt-get install autoconf automake checkinstall libsdl2-dev libtool libva-dev libvdpau-dev libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev texinfo libfribidi-dev libfontconfig1-dev libjpeg-dev libgnutls28-dev libluajit-5.1-dev python-docutils libbluray-dev libtheora-dev libvorbis-dev linux-headers-rpi2 libomxil-bellagio-dev
Install the OpenGL ES (GLES) and EGL files from the official Raspberry GitHub site. These GPU-related files are essential for mpv to function. Since we're installing pre-compiled ".deb" files in this procedure, the compiling has already "baked in" 2 of the 4 GPU-related files – which explains why we only need to install 2 files in this step:
sudo wget -P /opt/vc/lib https://github.com/raspberrypi/firmware/raw/master/hardfp/opt/vc/lib/libEGL.so
sudo wget -P /opt/vc/lib https://github.com/raspberrypi/firmware/raw/master/hardfp/opt/vc/lib/libGLESv2.so
Place the 8 “.deb” installer files in an empty folder on the Raspberry Pi 3 (see Step 13 of my tutorial for more on this topic). Then, open Terminal inside that folder (right-click the folder and select "Open in Terminal"). Finally, run the following command to install all the programs:
sudo dpkg -i *.deb
Run this command in Terminal to make sure all the “shared libraries” are properly recognized by FFmpeg and mpv. This step is critical, so don't overlook it:
Pin the advanced versions of the programs you just installed so that your system does not inappropriately "upgrade" them with older generic versions when you run "sudo apt-get upgrade" or "sudo apt-get dist-upgrade". Hat tip to "rpdom" for his pinning suggestion!
sudo nano /etc/apt/preferences
WHEN THE NANO EDITOR APPEARS INSIDE TERMINAL, COPY AND PASTE THE FOLLOWING 8 STANZAS INTO THE EMPTY DOCUMENT:
Pin: version 0.27.0-1
Pin: version 3.4.1-1
Pin: version 0.13.7-1
Pin: version 3.100-1
Pin: version 1.2.1-1
Pin: version 1.6.1-1
Pin: version 0.1.5-1
Pin: version 20180125-1
TO SAVE THE FILE, PRESS "CTRL + X" TO EXIT OUT OF NANO – THEN TAP "Y" FOR YES – THEN TAP THE ENTER KEY TO ACCEPT THE CHANGE.
APPENDIX 8: TROUBLESHOOTING:
If you encounter any issues in the future, a good thing to try is to simply re-install all the ".deb" packages you backed up in Step 13. The entire process takes less than one minute! See Appendix 6 for details.
If mpv generates any "port" or "MMAL" errors, make sure your GPU memory allocation is 128 MB (as indicated in my mpv instructions). And if mpv generates a "failed to add service" error, make sure you're not using the experimental desktop GL drivers known as Full KMS and Fake KMS (as indicated in my requirements section).
Finally, as mentioned previously, be sure that all Terminal windows are closed before you use mpv (in order to maintain keyboard control).
THIS PAGE ONLY CONTAINS THE APPENDICES – PLEASE VISIT MY MAIN TUTORIAL PAGE:
TUTORIAL: Play or Encode High-Quality Video and Audio – with FFmpeg and mpv