dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5502
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Moving Linux Kernel to 5.4

Thu Apr 02, 2020 7:30 pm

We are planning to move to the 5.4 kernel in the near future (perhaps a few months).
One advantage is there is more raspberry pi specific support in the newer upstream kernels which reduces the number of downstream raspberry pi patches and so reduces maintenance effort.

5.4 also includes the kms driver for Pi4 (this means driver code for display, like the hardware video scaler and HDMI runs on the arm rather than closed source firmware), and support for HEVC video decode using V4L2 on Pi4.

The kms driver also includes a new arm side hdmi alsa driver. See this post to enable.

Obviously a newer kernel will bring additional kernel features and support for more devices (e.g. USB devices like wifi and dvb).
You can read up on what appeared in 4.20, 5.0 ,5.1, 5.2, 5.3 and 5.4.

We will do this in a few stages. Starting now, we are updating the "next" firmware branch to the 5.4 kernel.
Assuming testing is positive, we'll update the master rpi-update firmware branch.
Eventually the 5.4 kernel will appear with "apt upgrade" and on new raspbian images in the future.

The 5.4 kernel is an LTS kernel (long term support) and has been used in LibreELEC nightly builds for a while.

Update with:

Code: Select all

sudo rpi-update
and reboot. Ideally nothing much will change.
Please report if there are any regressions compared to the 4.19 kernel.

If you are feeling especially bold, you can try the kms driver. Rather than dtoverlay=vc4-fkms-v3d, switch to dtoverlay=vc4-kms-v3d-pi4 on a pi4, or dtoverlay=vc4-kms-v3d on Pi2/3 (not recommended on Pi0/1).

Note: kms won’t ever support applications that use firmware apis for display like dispmanx, or mmal/openmax video_render, including omxplayer, raspivid and non-gbm kodi. Similarly tvservice is not useful in this environment.

Obviously this release is for testers who like to live on the bleeding edge. It's not intended for beginners, or users who need stability.
Backing up is always advisable. There may be regressions. There may be improvements. But we won't know without testers trying it and reporting back.

Note: the next firmware branch also includes common firmware which is also being tested.

EDIT: 5.4 kernel testing is now on master branch, as is common firmware.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8717
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 7:37 pm

Please note that the bcm2835-unicam driver isn't present on this branch as yet. It should hopefully be there in the next few days.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
DougieLawson
Posts: 38851
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 7:51 pm

Ooh shiny new toy to stave off the interminable boredom. Bring it on.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 8:18 pm

Initial observations, with the -v8+ kernel:

dtoverlay=vc4-kms-v3d-pi4 causes it to give up on console output after a message saying it's switched from 'simple' to $something (I missed what, sorry).

omxplayer spits out a lot of normal OMX-I'm-screwed error messages of the sort 6by9 so loves (eg: 'COMXVideo::Decode - OMX_EmptyThisBuffer() failed with result(0x80001009)'), but doesn't produce any video.

vlc will do HEVC, but with an endless stream of

Code: Select all

vcsm_clean_invalid2 failed: errno=22
to stderr.

1-wire still works, which is nice.

Code: Select all

root@pi4:~# tvservice -m CEA
tvservice-client: Failed to download CEA cache in [vc_tv_hdmi_get_supported_modes_new_id]
Group CEA has 0 modes:
root@pi4:~# tvservice -m DMT
tvservice-client: Failed to download DMT cache in [vc_tv_hdmi_get_supported_modes_new_id]
Group DMT has 0 modes:
root@pi4:~# 
So not quite there yet. Anything else you'd like tested?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26432
Joined: Sat Jul 30, 2011 7:41 pm

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 8:21 pm

dickon wrote:
Thu Apr 02, 2020 8:18 pm
omxplayer spits out a lot of normal OMX-I'm-screwed error messages of the sort 6by9 so loves (eg: 'COMXVideo::Decode - OMX_EmptyThisBuffer() failed with result(0x80001009)'), but doesn't produce any video.
I don't think OMXplayer will work with the KMS driver. Does it work OK with FKMS?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

trejan
Posts: 1855
Joined: Tue Jul 02, 2019 2:28 pm

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 8:26 pm

Just FYI. github is having issues right now so rpi-update act strange.

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 8:42 pm

jamesh wrote:
Thu Apr 02, 2020 8:21 pm
dickon wrote:
Thu Apr 02, 2020 8:18 pm
omxplayer spits out a lot of normal OMX-I'm-screwed error messages of the sort 6by9 so loves (eg: 'COMXVideo::Decode - OMX_EmptyThisBuffer() failed with result(0x80001009)'), but doesn't produce any video.
I don't think OMXplayer will work with the KMS driver. Does it work OK with FKMS?
No:

Code: Select all

root@pi4:~# omxplayer -d -r -g /mnt/newwatcher/video/[...]
Chapter : 	0 	00:00:00.000 	    0.00
Chapter : 	1 	00:00:07.007 	    7.00
Chapter : 	2 	00:01:50.485 	  110.00
Chapter : 	3 	00:03:38.760 	  218.00
Chapter : 	4 	00:09:27.776 	  567.00
Chapter : 	5 	00:17:01.979 	 1021.00
Chapter : 	6 	00:25:16.265 	 1516.00
Chapter : 	7 	00:36:17.842 	 2177.00
Chapter : 	8 	00:46:36.252 	 2796.00
Chapter : 	9 	00:56:51.325 	 3411.00
tvservice-client: Failed to download CEA cache in [vc_tv_hdmi_get_supported_modes_new_id]
Video codec omx-h264 width 1920 height 1080 profile 100 fps 23.976025
Audio codec dca channels 6 samplerate 48000 bitspersample 16
Subtitle count: 1, state: off, index: 1, delay: 0
COMXVideo::Decode - OMX_EmptyThisBuffer() failed with result(0x80001009)
COMXVideo::Decode - OMX_EmptyThisBuffer() failed with result(0x80001009)
COMXVideo::Decode - OMX_EmptyThisBuffer() failed with result(0x80001009)
/boot/config.txt:

Code: Select all

root@rtr-dickonh:~# cat /var/local/nfsroot/telly-buster/boot/config.txt 
# For more options and information see
# http://rpf.io/configtxt
# Some settings may impact device functionality. See link above for details

arm_64bit=1
gpu_mem=128

initramfs initrd.gz
hdmi_enable_4k=1

dtoverlay=w1-gpio
hdmi_ignore_cec_init=1

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

[pi4]
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
dtoverlay=vc4-fkms-v3d
# dtoverlay=vc4-kms-v3d-pi4
max_framebuffers=2
enable_uart=1
dtoverlay=uart5
start_x=1

[all]
#dtoverlay=vc4-fkms-v3d
hdmi_enable_4kp60=1
root@rtr-dickonh:~# 

Rascas
Posts: 668
Joined: Tue Mar 11, 2014 6:18 pm
Location: Porto, Portugal
Contact: Website

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 9:00 pm

Code: Select all

pi@raspberrypi:~ $ sudo BRANCH=next rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 !!! Failed to download update for rpi-update!
 !!! Make sure you have ca-certificates installed and that the time is set correctly
ca-certificates package is installed and date/time is correct. Any thoughts ?

EDIT: Forget it, it's updating now without changes, maybe a github issue.
Last edited by Rascas on Thu Apr 02, 2020 9:05 pm, edited 1 time in total.

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 9:05 pm

As trejan said, Github are having issues. Try it again.

I assumed it was eatmydata that was causing the problem, but no...

User avatar
DougieLawson
Posts: 38851
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 9:06 pm

Rascas wrote:
Thu Apr 02, 2020 9:00 pm

Code: Select all

pi@raspberrypi:~ $ sudo BRANCH=next rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 !!! Failed to download update for rpi-update!
 !!! Make sure you have ca-certificates installed and that the time is set correctly
ca-certificates package is installed and date/time is correct. Any thoughts ?
No weirdness here.
Run it with
sudo -i
UPDATE_SELF=0 BRANCH=next rpi-update
exit
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8717
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 9:15 pm

omxplayer and 64bit kernel is not going to work, and we're not fixing OMX for 64bit. It should work fine on a 32bit kernel.

tvservice has near zero use with vc4-kms-v3d(-pi4) as it will only report the last mode that the firmware requested. Use modetest, or look at the /sys/class/drm/card1/(something)/state status dump.

I must confess to not having tried the 64bit kernel myself. I guess I'll be downloading it tomorrow.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 9:29 pm

Really? It does happily on the 4.19 series kernels. I'm not expecting a 64b userland omx stack, but omxplayer has been fine on the -v8+ kernels on the 3+ and 4 since you started releasing them.

tvservice -m also works on 4.19 aarch64. -s is also reporting the current mode correctly.

Rascas
Posts: 668
Joined: Tue Mar 11, 2014 6:18 pm
Location: Porto, Portugal
Contact: Website

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 10:06 pm

From a very quick test here is my results on a Pi4 with 32bit kernel (default, pretty much everything stock):

With dtoverlay=vc4-fkms-v3d:

Kodi runs fine and HDR/10bit videos looks like the colors is now correct. Performance looks the same though, some high bitrate 4K HEVC HDR still skips frames. But I understand that the focus for that is now on v4l2 rather than MMAL.
Desktop Environment and other graphical applications looks like the same as the "stable" kernel.

With dtoverlay=vc4-kms-v3d-vc4:

Kodi runs now and without tearing (Vsync On on the interface). But most of the video test samples that I have don't play, the screen looses signal through HDMI, probably because of Kodi's MMAL. But again, I understand that the focus for that is now on v4l2 rather than MMAL.
Desktop Environment and other graphical applications works, VLC looks like it has the same problem as Kodi, most videos don't play.
Strangely, on Chromium, videos looks like working fine, but needs more testing.

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 11:23 pm

Rascas wrote:
Thu Apr 02, 2020 10:06 pm
From a very quick test here is my results on a Pi4 with 32bit kernel (default, pretty much everything stock):

With dtoverlay=vc4-fkms-v3d:

Kodi runs fine and HDR/10bit videos looks like the colors is now correct. Performance looks the same though, some high bitrate 4K HEVC HDR still skips frames.
Really? My TV isn't showing the HDR signalling, so the colours are sitll all wrong. This is on a 32b kernel.
With dtoverlay=vc4-kms-v3d-pi4:
I get a static screen with the end of the boot process on it -- exactly at which point seems to vary -- which I assume is at the point it switched from the 'simple' to 'kms' console, and after that, nothing changes; not even the cursor blinks. omxplayer (run from an ssh session) seems happy, but there's no actual video output; audio works, though.
Last edited by dickon on Fri Apr 03, 2020 12:18 am, edited 1 time in total.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5502
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 11:55 pm

Added:
Note: kms won’t ever support applications that use firmware apis for display like dispmanx, or mmal/openmax video_render, including omxplayer, raspivid and non-gbm kodi.

If you don’t enable kms then 5.4 kernel will continue to support the firmware display APIs, although we are moving away from the proprietary firmware APIs and towards standard Linux APIs.

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Thu Apr 02, 2020 11:59 pm

OK, noted, but what's the alternative console video player that can be run without X from an ssh session? vlc?

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5502
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 12:05 am

Some kms notes:
HDR isn’t supported yet but will come.
Omxplayer is deprecated in favour of vlc which should work windowed, but I don’t think full screen yet (it will come).
Chromium I believe should work with video playback (mmal for decode works fine with kms but not for display).
Kodi works when built in gbm rather than mmal mode and supports the v4l2 hevc driver.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5502
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 12:09 am

Rascas wrote:
Thu Apr 02, 2020 10:06 pm
With dtoverlay=vc4-kms-v3d-vc4:
vc4-kms-v3d-pi4

(support is being added for aliasing dtoverlay names so we will be able to use vc4-kms-v3d on Pi3 or Pi4 in the near future).

User avatar
dickon
Posts: 1435
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 12:19 am

dom wrote:
Fri Apr 03, 2020 12:09 am
vc4-kms-v3d-pi4
<fx: hastily edits post before last>

What you said.

mby
Posts: 36
Joined: Sat Dec 15, 2018 3:05 pm

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 7:30 am

Runs smoothly @ 64 bit, needed to reinstall iptables, now everything's fine again.

User avatar
DougieLawson
Posts: 38851
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 8:24 am

Got it running on a four Buster systems B+, A+, Zero and 3A+. No problems so far.

Will try a Buster 3B next. Then a 3B+ (running Stretch because I've not worked out where I'm going with the Google AIY stuff).
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

jahboater
Posts: 5668
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 8:34 am

Runs fine on my Pi4 4GB, Raspbian Lite, 64-bit kernel.
Pi4 8GB running PIOS64

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8717
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 9:50 am

dickon wrote:
Thu Apr 02, 2020 9:29 pm
Really? It does happily on the 4.19 series kernels. I'm not expecting a 64b userland omx stack, but omxplayer has been fine on the -v8+ kernels on the 3+ and 4 since you started releasing them.
You're right. It was late yesterday and my brain was getting confused.
64b kernel with 32b userland should work. 64b userland won't.
The exception being video_render when vc4-kms-v3d-pi4 is loaded, at which point the VPU can't add any overlays to the screen.
dickon wrote:tvservice -m also works on 4.19 aarch64. -s is also reporting the current mode correctly.
tvservice -m will as it will have cached the modes available from the early boot process.
tvservice -s won't reflect the mode as soon as it has been changed when using vc4-kms-v3d-pi4. (It does on vc4-fkms-v3d as that is still using the firmware composition path).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5502
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 10:12 am

KMS also comes with its own arm-side HDMI audio driver. If you want to test this, then remove dtparam=audio=on from config.txt
Replace /usr/share/alsa/cards/vc4-hdmi.conf with the contents here:

Code: Select all

# Configuration for the VC4-HDMI sound card using software IEC958
# subframe conversion

<confdir:pcm/hdmi.conf>
vc4-hdmi.pcm.hdmi.0 {
	@args [ CARD AES0 AES1 AES2 AES3 ]
	@args.CARD {
		type string
	}
	@args.AES0 {
		type integer
	}
	@args.AES1 {
		type integer
	}
	@args.AES2 {
		type integer
	}
	@args.AES3 {
		type integer
	}
	type iec958
	slave {
		format IEC958_SUBFRAME_LE
		pcm {
			type hooks
			slave.pcm {
				type hw
				card $CARD
				device 0
			}
			hooks.0 {
				type ctl_elems
				hook_args [
				{
					name "IEC958 Playback Default"
					optional true
					lock true
					preserve true
					value [ $AES0 $AES1 $AES2 $AES3 ]
				}
				]
			}
		}
	}
	status [ $AES0 $AES1 $AES2 $AES3 ]
}

# default with plug
vc4-hdmi.pcm.default {
	@args [ CARD ]
	@args.CARD {
		type string
	}
	type plug
	slave.pcm {
		type softvol
		slave.pcm {
			@func concat
			strings [ "hdmi:" $CARD ]
		}
		control {
			name "PCM Playback Volume"
			card $CARD
		}
	}
}
and you should havea new alsa device. (Only available when running with dtoverlay=vc4-kms-v3d-pi4).

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8717
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Moving Linux Kernel to 5.4

Fri Apr 03, 2020 11:19 am

dom wrote:
Fri Apr 03, 2020 10:12 am
and you should havea new alsa device. (Only available when running with dtoverlay=vc4-kms-v3d-pi4).
Dual HDMI audio should work from the driver level. Is it a case of replicating the vc4-hdmi.pcm.hdmi.0 section as vc4-hdmi.pcm.hdmi.1?
(I was just going to try it, but the TV I have as a second monitor appears to have decided against displaying any input given to it)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Advanced users”