jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

RPI4 Direct USB Boot Ubuntu 20.04

Wed Jul 01, 2020 2:29 am

=========== UPDATE 15Sep2020==============
Here are downloads for confirmed working USB booting for the pi4. If you guys will post more images for other linux flavors, I will add them to this list.

Ubuntu Server 20.04
https://mega.nz/file/DB1g2SyY#wVn8avwND ... ZmNDVOmoF0

Ubuntu Mate 20.04
https://mega.nz/file/Xc8ClACQ#6knpKl2n8 ... XTNyIxSkCs

openSUSE
https://mega.nz/file/zNNklQSC#acFBMaYmz ... lTmR6pJeWs

Instructions on this post: viewtopic.php?f=131&t=278791&start=75#p1727980

Manjaro (Works right out of the box)
https://osdn.net/frs/redir.php?m=xtom_u ... .08.img.xz

=========== UPDATE 04Sep2020==============

Here is a working image for USB booting Ubuntu Mate on the RPI4 for anyone who wants it.
https://mega.nz/file/nFlhxJiT#ejmbPr7mw ... NjNhAyY1EU

=========== UPDATE 30Aug2020==============

It seems that some people are having issues creating a bootable USB drive. I have created a confirmed working image. If you find that you are having ah hard time getting this to work, please download my provided image. This was created using the most recent Ubuntu server image for the RaspberryPi 4 64-bit image as of 30Aug2020.

https://archive.org/details/rpi-usb-boot
https://mega.nz/file/yQVGxYKQ#Sv9ZutsMb ... j59tzbQDmE
MD5 = 0e90024774ef0d098dc25218839fec03

**Please note that you must have updated the RaspberryPi 4 EEPROM image, containing the bootloader, to support USB booting. See the link in step 3 below, and read ALL sections regarding updating EEPROM and USB booting.

For even more information about updating to a compatible bootloader see https://www.raspberrypi.org/documentati ... teeprom.md.
If this is not done, USB booting will not work.
Please update the bootloader.

If things don't work, please let us know.

=========== UPDATE 11Jul2020==============

Setp 3 was updated to make sure that you are using the most current "Stable" EEPROM for the RPI4. If you do not use the updated EEPROM, them you cannot boot from USB. There is no "Critical" release with the USB booting option enabled, and the RPI4 is certainly not shipping with it enabled yet.

=========== UPDATE 05Jul2020==============

Just so everyone can get this working without an issue, once you do the initial setup on the boot partition of the USB drive, there is another step so that you do not have to manually decompress the image after each updated. This was particularly annoying with background updates, and your system would not boot after restart for no apparent reason.

So Here are the steps:

1) Download the Ubuntu image for raspberry pi 4 form the Ubuntu official website.

2) Flash the image to a USB drive (USB 3.0 SSD or UBS flash drive).

3) Download the updated firmware files from the raspberry pi github site (https://github.com/raspberrypi/firmware ... aster/boot). Copy all *.dat and *.elf files to the Ubuntu boot partition. (Overwrite the files that were previously there).
NOTE: You must have MSD Booting EEPROM flashed onto your RPI4, or else this will not work!!! See: https://www.raspberrypi.org/documentati ... torageboot

Note: As of August 2020, you may not need to perform step 3. It has been reported that the current Ubuntu image contains the correct firmware files. I will leave this step here in case it does help someone. Be aware that it may not be necessary, but does not hurt to do anyway.

4) Decompress vmlinuz on the boot partition

Code: Select all

zcat vmlinuz > vmlinux
5) Update the config.txt as follows for the [pi4] section:

Code: Select all

[pi4]
max_framebuffers=2
dtoverlay=vc4-fkms-v3d
boot_delay
kernel=vmlinux
initramfs initrd.img followkernel
6) Add a new script to the boot partition called auto_decompress_kernel with the following:

Code: Select all

#!/bin/bash -e

#Set Variables
BTPATH=/boot/firmware
CKPATH=$BTPATH/vmlinuz
DKPATH=$BTPATH/vmlinux

#Check if compression needs to be done.
if [ -e $BTPATH/check.md5 ]; then
	if md5sum --status --ignore-missing -c $BTPATH/check.md5; then
	echo -e "\e[32mFiles have not changed, Decompression not needed\e[0m"
	exit 0
	else echo -e "\e[31mHash failed, kernel will be compressed\e[0m"
	fi
fi

#Backup the old decompressed kernel
mv $DKPATH $DKPATH.bak

if [ ! $? == 0 ]; then
	echo -e "\e[31mDECOMPRESSED KERNEL BACKUP FAILED!\e[0m"
	exit 1
else 	echo -e "\e[32mDecompressed kernel backup was successful\e[0m"
fi

#Decompress the new kernel
echo "Decompressing kernel: "$CKPATH".............."

zcat $CKPATH > $DKPATH

if [ ! $? == 0 ]; then
	echo -e "\e[31mKERNEL FAILED TO DECOMPRESS!\e[0m"
	exit 1
else
	echo -e "\e[32mKernel Decompressed Succesfully\e[0m"
fi

#Hash the new kernel for checking
md5sum $CKPATH $DKPATH > $BTPATH/check.md5

if [ ! $? == 0 ]; then
	echo -e "\e[31mMD5 GENERATION FAILED!\e[0m"
	else echo -e "\e[32mMD5 generated Succesfully\e[0m"
fi

#Exit
exit 0
7) Make the script executable -- This is not actually needed as the boot partition is a fat32 partition. There is no executable bit to set.

Code: Select all

sudo chmod +x auto_decompress_kernel
8) Create a script in the /ect/apt/apt.conf.d/ directory and call it 999_decompress_rpi_kernel. The script should contain the following:

Code: Select all

DPkg::Post-Invoke {"/bin/bash /boot/firmware/auto_decompress_kernel"; };
9) Make the script executable

Code: Select all

sudo chmod +x 999_decompress_rpi_kernel
Steps 8 and 9 can be done before first boot if you can mount the root file system on your computer, if you cannot, you can do them after you boot up the pi and log on to ubuntu for the first time. I recommend that you do it before first boot, but if you cannot, once you have the files in the correct place, please run the script auto_decompress_kernel from the /boot/firmware directory. If you do not, and you cannot boot after a restart, you will need to manually decompress your kernel again before this script will actually work (step 3).

10) Enjoy Ubuntu on the RPI4 without hassle :D

-- Link to the post that has this information viewtopic.php?f=131&t=278791&p=1691270#p1691270

=========================================

So, this is my first post, but this is something that was driving me crazy, so now that I have it working, I thought I would let others also know.

In order to get Ubuntu Server 20.04 to boot on the RPI4 using USB only, follow these steps:

1) Download the Ubuntu image for raspberry pi 4 form the Ubuntu official website.

2) Flash the image to a USB drive (USB 3.0 SSD or UBS flash drive).

3) Until they update their image, or you want to compile u-boot, you will need to decompress the vmlinuz kernel
- Easiest way I have found is to execute the following command "dd if=vmlinuz bs=1 | zcat > vmlinux" in the "system-boot" partition of the USB

4) Update the config.txt as follows for the [pi4] section:

[pi4]
max_framebuffers=2
#dtoverlay=vc4-fkms-v3d
boot_delay
kernel=vmlinux
initramfs initrd.img followkernel

That's it. You can now have Ubuntu running on the RPI4 booting directly from USB. You can probably remove the boot_delay. I was testing off a very slow USB flash drive, and that helped get it working, so I just never removed it. Pretty quick once it is running of an SSD.
Last edited by jsciamms on Thu Sep 17, 2020 5:42 am, edited 21 times in total.

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Jul 01, 2020 1:00 pm

I forgot to add that you need to replace all the *.elf and *.dat files in the boot folder with the ones from the most recent raspberry pi firmware.

Download them from https://github.com/raspberrypi/firmware

kazin08
Posts: 19
Joined: Thu Feb 14, 2019 10:08 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 12:39 am

jsciamms wrote: So, this is my first post, but this is something that was driving me crazy, so now that I have it working, I thought I would let others also know.

In order to get Ubuntu Server 20.04 to boot on the RPI4 using USB only, follow these steps:

1) Download the Ubuntu image for raspberry pi 4 form the Ubuntu official website.

2) Flash the image to a USB drive (USB 3.0 SSD or UBS flash drive).

3) Until they update their image, or you want to compile u-boot, you will need to decompress the vmlinuz kernel
- Easiest way I have found is to execute the following command "dd if=vmlinuz bs=1 | zcat > vmlinux" in the "system-boot" partition of the USB

4) Update the config.txt as follows for the [pi4] section:

[pi4]
max_framebuffers=2
#dtoverlay=vc4-fkms-v3d
boot_delay
kernel=vmlinux
initramfs initrd.img followkernel

That's it. You can now have Ubuntu running on the RPI4 booting directly from USB. You can probably remove the boot_delay. I was testing off a very slow USB flash drive, and that helped get it working, so I just never removed it. Pretty quick once it is running of an SSD.
I can confirm this! it's working! i can boot directly from my USB!

To execute the command, just copy the file vmlinuz in another directory like Documents, then in command promp:

Code: Select all

sudo dd if=vmlinuz bs=1 | zcat > vmlinux
then wait. When finish just copy with the command:

Code: Select all

sudo cp ~/Documents/vmlinux /boot/firmware/
Then download the latest *.elf and *.dat from the link above and replace it in /boot/firmware, and it's done.

Dont forget to modify the config.txt. You can enable "dtoverlay=vc4-fkms-v3d"

I'm using my old HDD WD blue with USB adapter with ubuntu 20.04 and works very well!

Regards

Amen8
Posts: 5
Joined: Mon Jun 22, 2020 6:37 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 9:54 am

i also confirm this is working ! thank you sooooooooooooooooooo so much
i almost cry of happiness

thatchunkylad198966
Posts: 253
Joined: Thu Jul 04, 2019 10:21 am
Location: UK, Birmingham

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 12:46 pm

This works for the first bootup, but after you reboot (updated everything or not) you get a failure of boot because of local-premount and writeable not existing. :(
One man's trash is another man's treasure! :) Pi's I have; Pi Zero, Pi Zero W, Pi 2 x2, Pi 3 x2, Pi 4 4GB x2.

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 1:22 pm

If you update Ubuntu, a new vmlinuz is added to the boot directory, and the old one is turned to vmlinuz.bak.

What you need to do is rename your vmlinux decompressed image to vmlinux.bak (or delete it), take the new vmlinuz image and decompress it as stated above.

Once you are using the updated and decompressed image, it will boot without issue.

Ubuntu will always update on the first boot. If you leave it running long enough, a silent update process runs in the background. I had the same issue, and it took some time to figure it out.

It boils down to the fact that it is expecting you to boot using u-boot which will decompress a compressed kernel. The normal raspberry pi bootloader will not accept a compressed image, so it can’t boot. Any time you do an update, you just have to decompress the new kernel image and everything will work as expected.

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 2:33 pm

It may also be worth noting that some packages, when installed, will also cause the kernel to update, and break the boot process until your kernel image is again manually decompressed.

thatchunkylad198966
Posts: 253
Joined: Thu Jul 04, 2019 10:21 am
Location: UK, Birmingham

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 2:56 pm

jsciamms wrote:
Thu Jul 02, 2020 1:22 pm
If you update Ubuntu, a new vmlinuz is added to the boot directory, and the old one is turned to vmlinuz.bak.

What you need to do is rename your vmlinux decompressed image to vmlinux.bak (or delete it), take the new vmlinuz image and decompress it as stated above.

Once you are using the updated and decompressed image, it will boot without issue.

Ubuntu will always update on the first boot. If you leave it running long enough, a silent update process runs in the background. I had the same issue, and it took some time to figure it out.

It boils down to the fact that it is expecting you to boot using u-boot which will decompress a compressed kernel. The normal raspberry pi bootloader will not accept a compressed image, so it can’t boot. Any time you do an update, you just have to decompress the new kernel image and everything will work as expected.
I didn't do an update though, a simple reboot and the system wouldn't boot.
One man's trash is another man's treasure! :) Pi's I have; Pi Zero, Pi Zero W, Pi 2 x2, Pi 3 x2, Pi 4 4GB x2.

GlowInTheDark
Posts: 846
Joined: Sat Nov 09, 2019 12:14 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 3:03 pm

There is, of course, no need for using 'dd' or a pipeline.

All you need is:

$ gzip -dc < vmlinuz > vmlinux
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

User avatar
rpdom
Posts: 17418
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 4:40 pm

GlowInTheDark wrote:
Thu Jul 02, 2020 3:03 pm
There is, of course, no need for using 'dd' or a pipeline.

All you need is:

$ gzip -dc < vmlinuz > vmlinux
Or even simpler:

$ zcat vmlinuz >vmlinux
Unreadable squiggle

kazin08
Posts: 19
Joined: Thu Feb 14, 2019 10:08 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 6:37 pm

thatchunkylad198966 wrote:
Thu Jul 02, 2020 2:56 pm
jsciamms wrote:
Thu Jul 02, 2020 1:22 pm
If you update Ubuntu, a new vmlinuz is added to the boot directory, and the old one is turned to vmlinuz.bak.

What you need to do is rename your vmlinux decompressed image to vmlinux.bak (or delete it), take the new vmlinuz image and decompress it as stated above.

Once you are using the updated and decompressed image, it will boot without issue.

Ubuntu will always update on the first boot. If you leave it running long enough, a silent update process runs in the background. I had the same issue, and it took some time to figure it out.

It boils down to the fact that it is expecting you to boot using u-boot which will decompress a compressed kernel. The normal raspberry pi bootloader will not accept a compressed image, so it can’t boot. Any time you do an update, you just have to decompress the new kernel image and everything will work as expected.
I didn't do an update though, a simple reboot and the system wouldn't boot.
It happened to me even when i boot from SD card (at least one reboot work, the next reboot wont) and this happen here too. Because i got an 3.5" HDD i can hear when it turn on or off. When i connect my pi and connect the HDD, when the want to boot it turn off the HDD and turn on again (this is like one reboot) then when i reboot the pi it wont boot. so maybe the problem it's my cheap usb adapter or the firmware of the pi.

Regards

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 6:40 pm

thatchunkylad198966 wrote:
Thu Jul 02, 2020 2:56 pm
jsciamms wrote:
Thu Jul 02, 2020 1:22 pm
If you update Ubuntu, a new vmlinuz is added to the boot directory, and the old one is turned to vmlinuz.bak.

What you need to do is rename your vmlinux decompressed image to vmlinux.bak (or delete it), take the new vmlinuz image and decompress it as stated above.

Once you are using the updated and decompressed image, it will boot without issue.

Ubuntu will always update on the first boot. If you leave it running long enough, a silent update process runs in the background. I had the same issue, and it took some time to figure it out.

It boils down to the fact that it is expecting you to boot using u-boot which will decompress a compressed kernel. The normal raspberry pi bootloader will not accept a compressed image, so it can’t boot. Any time you do an update, you just have to decompress the new kernel image and everything will work as expected.
I didn't do an update though, a simple reboot and the system wouldn't boot.
Did you check the boot volume of the USB? I’m betting there is a vmlinuz.bak. If that file exists, something was installed, or a process ran, that regenerated the kernel image. All you should need to do is decompress the kernel again.

At this point, even if that file isn’t there, try decompressing the kernel and booting. More efficient ways of doing that have been posted by 2 other people.

GlowInTheDark
Posts: 846
Joined: Sat Nov 09, 2019 12:14 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 02, 2020 9:45 pm

Or even simpler:

$ zcat vmlinuz >vmlinux
Yeah, that'll work, too. All I can say is that when gzip first came out and I first started learning how to use it, I found all the aliases and stuff confusing and wanted to learn as few commands as possible - i.e., just the minimum amount of memorization.

I found that knowing just "gzip -c" and "gzip -dc" covers 99% of the cases.
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

RulerOfDevNull
Posts: 1
Joined: Sat Jul 04, 2020 12:11 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Jul 04, 2020 12:22 pm

First of all thanks for the writeup!

I had a similar working setup procedure, but the installations kept breaking after one or two reboots. The pointer with the kernel updates seems to have resolved that issue for now... hopefully. But I guess manually updating the kernel isn't a satisfying solution. I mean how is this "machines take over the world" scenario supposed to happen, if they can't even update themselves properly? :lol:

I've limited knowledge of linux and would be happy if anyone could check if the following script is even a proper solution (that is a WARNING - you should not just copy&paste the following). I've created the following script in /etc/kernel/postinst.d/kerneldecompress:

Code: Select all

#!/bin/sh -e

zcat /boot/firmware/linuz >/boot/firmware/vmlinux

perlsite
Posts: 3
Joined: Mon Jun 15, 2020 8:43 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Jul 04, 2020 1:38 pm

Thank you folks for all the information!
@jsciamms I'm going soon to try this one on my Pi. I'm wondering one thing - do you know what is the exact command that is used when the OS updates the kernel? What I'm thinking is wrapping that command to do the uncompressing the image every time! You can do that at any other level, including rc.0/rc.6, where you could check whether vmlinux is up to date. What do you think?

perlsite
Posts: 3
Joined: Mon Jun 15, 2020 8:43 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Jul 04, 2020 1:39 pm

Thank you folks for all the information!
jsciamms wrote:
Thu Jul 02, 2020 2:33 pm
It may also be worth noting that some packages, when installed, will also cause the kernel to update, and break the boot process until your kernel image is again manually decompressed.
I'm going soon to try this one on my Pi. I'm wondering one thing - do you know what is the exact command that is used when the OS updates the kernel? What I'm thinking is wrapping that command to do the uncompressing the image every time! You can do that at any other level, including rc.0/rc.6, where you could check whether vmlinux is up to date. What do you think?

geev03
Posts: 188
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Jul 04, 2020 3:29 pm

GlowInTheDark wrote:
Thu Jul 02, 2020 3:03 pm
There is, of course, no need for using 'dd' or a pipeline.

All you need is:

$ gzip -dc < vmlinuz > vmlinux
Thank you.
Ubuntu Mate is now even more desirable for use cases as a 'desktop computer'

Code: Select all

ubuntu@ubuntu:~$ neofetch
            .-/+oossssoo+/-.               ubuntu@ubuntu 
        `:+ssssssssssssssssss+:`           ------------- 
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 20.04 LTS aarch64 
    .ossssssssssssssssssdMMMNysssso.       Host: Raspberry Pi 4 Model B Rev 1.4 
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 5.4.0-1012-raspi 
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 2 mins 
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 2356 (dpkg), 6 (snap) 
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: bash 5.0.16 
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 1920x1080 
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   DE: MATE 
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   WM: Metacity (Marco) 
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   WM Theme: Ambiant-MATE-Dark 
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Theme: Ambiant-MATE-Dark [GTK2/3] 
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/    Icons: Ambiant-MATE [GTK2/3] 
  +sssssssssdmydMMMMMMMMddddyssssssss+     Terminal: mate-terminal 
   /ssssssssssshdmNNNNmyNMMMMhssssss/      Terminal Font: Ubuntu Mono 13 
    .ossssssssssssssssssdMMMNysssso.       CPU: BCM2835 (4) @ 1.500GHz 
      -+sssssssssssssssssyyyssss+-         Memory: 721MiB / 7810MiB 
        `:+ssssssssssssssssss+:`
            .-/+oossssoo+/-.                                       
                                                                   


ubuntu@ubuntu:~$ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0    7:0    0 85.4M  1 loop /snap/core/9441
loop1    7:1    0 48.4M  1 loop /snap/core18/1708
loop2    7:2    0 61.3M  1 loop /snap/lxd/14808
loop3    7:3    0 48.4M  1 loop /snap/core18/1753
loop4    7:4    0  3.6M  1 loop /snap/hw-probe/831
loop5    7:5    0 63.6M  1 loop /snap/lxd/15938
loop6    7:6    0 23.5M  1 loop /snap/snapd/7267
loop7    7:7    0 25.9M  1 loop /snap/snapd/8147
sda      8:0    1 29.2G  0 disk 
├─sda1   8:1    1  256M  0 part /boot/firmware
└─sda2   8:2    1 28.9G  0 part /
ubuntu@ubuntu:~$ 

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Jul 04, 2020 5:39 pm

I have thought about doing something similar. Usually installs and updates cause the kernel repackage and compress. I don’t know if you can wrap those apt calls like that. It might be easier to run a script on shutdown.

Maybe something that would mount the boot partition, and check the creation dates of the vmlinuz vs vmlinux files, and if they are not within the same, day or whatever time period, decompress the kernel and replace the vmlinux file. It may be even easier if we can find out why Ubuntu is compressing the kernel in the first place. I have no idea why that is the case. Maybe someone here knows the answer. Then it wouldn’t matter if something is updated.

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Mon Jul 06, 2020 1:15 am

RulerOfDevNull wrote:
Sat Jul 04, 2020 12:22 pm
First of all thanks for the writeup!

I had a similar working setup procedure, but the installations kept breaking after one or two reboots. The pointer with the kernel updates seems to have resolved that issue for now... hopefully. But I guess manually updating the kernel isn't a satisfying solution. I mean how is this "machines take over the world" scenario supposed to happen, if they can't even update themselves properly? :lol:

I've limited knowledge of linux and would be happy if anyone could check if the following script is even a proper solution (that is a WARNING - you should not just copy&paste the following). I've created the following script in and placed it in: /etc/kernel/postinst.d/zzz_decompress_rpi_kernel

Code: Select all

#!/bin/sh -e

zcat /boot/firmware/linuz >/boot/firmware/vmlinux

You are absolutely right, where is the fun in decompressing your own kernel:): This is the exact script that I was using:

Code: Select all

 
 #!/bin/bash -e

#Set Path Variables
BTPATH=/boot/firmware
CKPATH=$BTPATH/vmlinuz
DKPATH=$BTPATH/vmlinux

#Backup the old decompressed kernel
mv $DKPATH $DKPATH.bak

#Decompress the new kernel
zcat $CKPATH > $DKPATH

exit 0
Issue is that it decompresses the old file. The new kernel image is not added until about 1 second after the decompression is complete. Not sure how to resolve this. I have even added the script as "ZZZ_decompress" since everything in that folder is executed in alphabetical order, but I am still decompressing the old kernel. I have verified by performing hashes on both the original decompressed image and the image decompressed by the script. They are exactly the same. Any thoughts on where to go from here?

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Mon Jul 06, 2020 2:24 am

Ok, made some progress.

I have updated the script a bit to make sure that the the kernel needs to be decompressed, and placed it in the boot partition of the raspberry pi and called it auto_decompress_kernel. The script is as follows:

Code: Select all

#!/bin/bash -e

#Set Variables
BTPATH=/boot/firmware
CKPATH=$BTPATH/vmlinuz
DKPATH=$BTPATH/vmlinux

#Check if compression needs to be done.
if [ -e $BTPATH/check.md5 ]; then
	if md5sum --status --ignore-missing -c $BTPATH/check.md5; then
	echo -e "\e[32mFiles have not changed, Decompression not needed\e[0m"
	exit 0
	else echo -e "\e[31mHash failed, kernel will be compressed\e[0m"
	fi
fi

#Backup the old decompressed kernel
mv $DKPATH $DKPATH.bak

if [ ! $? == 0 ]; then
	echo -e "\e[31mDECOMPRESSED KERNEL BACKUP FAILED!\e[0m"
	exit 1
else 	echo -e "\e[32mDecompressed kernel backup was successful\e[0m"
fi

#Decompress the new kernel
echo "Decompressing kernel: "$CKPATH".............."

zcat $CKPATH > $DKPATH

if [ ! $? == 0 ]; then
	echo -e "\e[31mKERNEL FAILED TO DECOMPRESS!\e[0m"
	exit 1
else
	echo -e "\e[32mKernel Decompressed Succesfully\e[0m"
fi

#Hash the new kernel for checking
md5sum $CKPATH $DKPATH > $BTPATH/check.md5

if [ ! $? == 0 ]; then
	echo -e "\e[31mMD5 GENERATION FAILED!\e[0m"
	else echo -e "\e[32mMD5 generated Succesfully\e[0m"
fi

#Exit
exit 0
This is then called by apt any time that apt is used to install or update software. I think this should catch most use cases. In order to get this working, make a file in the /ect/apt/apt.conf.d/ directory and call it 999_decompress_rpi_kernel

That file needs to contain the following: DPkg::Post-Invoke {"/bin/bash /boot/firmware/auto_decompress_kernel"; };

There are two ways to make this work. You can set it all up before your first boot (follow directions above), or after first boot with these commands:

Code: Select all

sudo echo "DPkg::Post-Invoke {"/bin/bash /boot/firmware/auto_decompress_kernel"; };" > /etc/apt/apt.conf.d/999_decompress_rpi_kernel
sudo chmod +x /etc/apt/apt.conf.d/999_decompress_rpi_kernel
sudo nano /boot/firmware/auto_decompress_kernel
Copy the script above into that file, then save the file. Then make it executable and run it, just in case there have been background updates since boot.

Code: Select all

sudo chmod +x /boot/firmware/auto_decompress_kernel
sudo /boot/firmware/auto_decompress_kernel
That should do it. Goodbye manual decompression! 8-)

The only potential issue here is if you stop the unattended upgrades by killing the process (ie: pulling power cable). I believe that a proper restart will finish up the upgrades and keep everything running smooth. I have tested on forced upgrades and by letting the pi sit for a while on first boot and rebooting. Worked like a charm in both situations.

Decostre
Posts: 2
Joined: Wed Jul 08, 2020 6:02 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Jul 08, 2020 6:05 am

I had a problem! My RPi worked for 4 days and then I had to turn it off and I could not turn it on. Then I did it all over again "dd if=vmlinuz bs=1 | zcat > vmlinux" in the "system-boot", replaced the .elf and .dat files, but USB boot failed. Can't you tell me where I was wrong?

jsciamms
Posts: 38
Joined: Wed Jul 01, 2020 2:11 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Jul 08, 2020 1:36 pm

Where did it fail? Did it get past the initial firmware and see you USB drive? Did it fail at the color screen? Did it stop after giving you an initramfs prompt?

We will need a bit more information to be able to help you out.

User avatar
ranrinc
Posts: 15
Joined: Tue Feb 25, 2020 7:23 am
Location: Jakarta Indonesia
Contact: Website

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Jul 08, 2020 3:36 pm

Is this mean I can boot ubuntu without microSD? If yes.. need to try it soon

Decostre
Posts: 2
Joined: Wed Jul 08, 2020 6:02 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Jul 08, 2020 3:59 pm

jsciamms wrote:
Wed Jul 08, 2020 1:36 pm
Where did it fail? Did it get past the initial firmware and see you USB drive? Did it fail at the color screen? Did it stop after giving you an initramfs prompt?

We will need a bit more information to be able to help you out.
Hey! Thanks for all info! The first time your guide worked for me. My Rpi worked for 4 days via SSD ubuntu. I was manually copying the files .elf and .dat after the updates and also performed this command

Code: Select all

dd if=vmlinuz bs=1 | zcat > vmlinux
before rebooting. now I do the same process again but the system doesn't start. it takes a long time to load, and I get messages like:

Code: Select all

 [FAILED] Failed to start Apply Kernel Variables.
"

Code: Select all

A start job is running for load apparmor profiles
"
and etc.

Why can this happen and how can I fix it?

This shouldn't work via USB 3.0?

NicknDi
Posts: 12
Joined: Tue Jul 07, 2020 9:47 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Jul 09, 2020 5:35 pm

First, a huge "thank you" to jsciamms for this post.

Today I followed the July 5th update exactly, but sadly haven't succeeded in booting from my USB3 SSD drive. I get to the rainbow rectangle splash, then just a blank screen.

At first I didn't even get that far, but then saw the July 1st post about forgetting to replace the .dat and .elf files, so did that, and then got to the rainbow rectangle, so progress of sorts. The other thing to note is that at Step 3 zcat returned a message saying vmlinuz was not a zip (or similar). Could that be the problem? I've been using linux for many years now, but I'm still at the edge of my comfort zone here. Would welcome any suggestions how to proceed.

PS I did all the editing on my linux (Manjaro) pc, so was attempting to run from the usb drive at first boot.

Return to “Ubuntu”