hedefalk
Posts: 3
Joined: Tue Oct 20, 2020 8:56 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Tue Oct 20, 2020 9:04 am

I'm following this guide but I'm having issues bootstrapping with cloud-init. I previously installed Ubuntu 20.04.1 64-bit on SD card and just added a user-config with my public ssh key to be able to login via ssh.

The same route doesn't seem to work here. I add a user-config:

Code: Select all

ssh_pwauth: false

hostname: _host

users:
  - name: viktor
    ssh_authorized_keys:
      - ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXX
    sudo: ALL=(ALL) NOPASSWD:ALL
    lock_passwd: true
    shell: bash

This worked fine when using an SD, but doesn't seem to do the trick here. I've tried all things I can figure out about allowing root-login, pwd as well as keys and manually configuring ssh server:

Code: Select all

## https://cloudinit.readthedocs.io/

ssh_pwauth: yes

users:
  - default
    ssh_authorized_keys:
      - ssh-rsa XXXXXX
  - name: viktor
    ssh_authorized_keys:
      - ssh-rsa XXXXX
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: users, admin
    shell: bash

disable_root: false
runcmd:
  - sed -i -e '$aAllowUsers viktor' /etc/ssh/sshd_config
  - restart ssh

But it seems to me the cloud-init simply isn't applied.

My problem is that I'm headless. I don't even have an hdmi cable or keyboard so really need to be able to login via ssh after boot. Anyone had success with this on USB-boot? Just weird that the same thing worked fine with SD…

dreamlax007
Posts: 11
Joined: Tue Oct 13, 2020 1:36 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Tue Oct 20, 2020 10:43 am

I had success with USB-boot without sdcard. I used balenaetcher to write the MATE-USB-BOOT.img. I had to manually resize the writable partition to get this working.

After a couple of days of usage, I got a green screen. So I rebooted the pi and the whole usb boot failed again. The mate splash screen comes up but no progress. Does anyone know how to look at what the kernel is doing during boot up?

dreamlax007
Posts: 11
Joined: Tue Oct 13, 2020 1:36 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Tue Oct 20, 2020 4:05 pm

It seems I unable to boot from SSD because it has gotten some badblocks. This was a new SSD so the badblocks must have come because of the booting Mate on pi4. Most likely its the external ssd reader thats not getting the power from the pi board.

Retreating back to sd card boot :(

shurx
Posts: 3
Joined: Mon Oct 19, 2020 3:15 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Oct 21, 2020 2:48 am

I have ubuntu 18.04 64bit and it won't boot from usb. I tried different ways. Bootloader is the latest. Only RPi OS was loaded from usb, which technically does not suit me due to the mariadb database, since i am using mysql 5.7

hedefalk
Posts: 3
Joined: Tue Oct 20, 2020 8:56 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Oct 21, 2020 8:03 am

hedefalk wrote:
Tue Oct 20, 2020 9:04 am

But it seems to me the cloud-init simply isn't applied.
My problem was definitely not that cloud-init wasn't applied. I finally gave up on it and just did without and then everything was fine. So definitely applied, but something with how my cloud init settings was applied to sshd_config messed stuff up. But I'm not sure how since I can't reach the system and not even find any logs on the usb disk without going through the hassle of fuse on my mac or something. My problem was that I got "Unahtorixed (public key)" from ssh server even though I put my pub key there with cloudinit

For now I just did everything manually, letting the ubuntu/ubuntu be there and logging in with pwd. Then::

* add my user account with pwd
* push my pub key there:
cat ~/.ssh/id_rsa.pub | ssh viktor@x.x.x.x "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
* Disable pwd ssh:
sed -i 's/PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo service ssh restart

Classic manual labour I really could do without, but yeah. I guess I could just do this with cloud-init manually by simply replacing /etc/ssh/sshd_config but I don't know what was wrong with my cloudinit to begin with and just want to progress now.

dreamlax007
Posts: 11
Joined: Tue Oct 13, 2020 1:36 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Oct 21, 2020 9:48 am

shurx wrote:
Wed Oct 21, 2020 2:48 am
I have ubuntu 18.04 64bit and it won't boot from usb. I tried different ways. Bootloader is the latest. Only RPi OS was loaded from usb, which technically does not suit me due to the mariadb database, since i am using mysql 5.7
How are you flashing the image? Does the boot start from usb and hang midway?

shurx
Posts: 3
Joined: Mon Oct 19, 2020 3:15 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Oct 21, 2020 1:55 pm

dreamlax007 wrote:
Wed Oct 21, 2020 9:48 am
How are you flashing the image? Does the boot start from usb and hang midway?
The image was copied and recorded using Win32DiskImager. It clones memory entirely. I transferred the system from one sd card to another. Here the instructions indicate that you need an uncompressed kernel, although I already have it uncompressed. Even if you specify this kernel, the boot gets stuck half way. The fact that the developers did boot from usb is good, but useless for ubuntu, and I don’t want to use the official rpi os in principle, because 32 bits.

dreamlax007
Posts: 11
Joined: Tue Oct 13, 2020 1:36 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Fri Oct 23, 2020 3:07 pm

Try resizing the writeable partition. My boot was getting hanged as well. The resizing worked.

dreamlax007
Posts: 11
Joined: Tue Oct 13, 2020 1:36 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Fri Oct 23, 2020 4:05 pm

Just tried the latest ubuntu 20.10 for raspi 4. It booted from usb out the box without any hiccups :). Would highly recommed that.

shurx
Posts: 3
Joined: Mon Oct 19, 2020 3:15 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Fri Oct 23, 2020 5:13 pm

Guys, I was able to run ubuntu 18.04 with usb. Made changes to config.txt. Another problem was in the usb 3.0 - sata adapter. After replacing it, it worked.

Code: Select all

[pi4]
#kernel=uboot_rpi_4.bin
max_framebuffers=2

[all]
arm_64bit=1
#device_tree_address=0x03000000
kernel=vmlinuz
cmdline=btcmd.txt
initramfs initrd.img followkernel

xyzulu
Posts: 8
Joined: Wed Aug 05, 2020 12:59 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Oct 24, 2020 8:49 am

So now that 20.10 makes the brilliant work of the original poster redundant .. what changes need to be made to revert to the stock config.txt?

stef70
Posts: 3
Joined: Sun Jul 19, 2020 2:07 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Wed Nov 04, 2020 4:47 pm

@xyzulu I do not know if and how 20.10 solved the USB boot problem but I noticed that the linux-image-5.8.0-1006-raspi package now contains an uncompressed kernel that is still installed as /boot/firmware/vmlinuz.

Consequently, the zcat command is failing in auto_decompress_kernel thus making the system unbootable after the upgrade.

An easy fix is to modify /boot/firmware/config.txt with kernel=vmlinuz and to remove auto_decompress_kernel and its DPkg::Post-Invoke command in /etc/apt/apt.conf.d

Another possibility is to modify auto_decompress_kernel to detect the file type and then act accordingly. This is probably overdone but I do not want to risk breaking my system (fully encrypted so a pain to repair) if Ubuntu suddenly decides to compress their kernel again. Also I want to be able to boot the old 5.4 kernel is something is not working well with 5.8. Consequently, I changed the zcat command as follow in auto_decompress_kernel:

Code: Select all

#
# Decompress the new kernel if needed
#
TYPE=$(file --mime-type --brief "$CKPATH")
case "$TYPE" in
    application/x-dosexec)
      echo -e "$CKPATH is not compressed! Copying to $DKPATH";
      cp $CKPATH $DKPATH
      if [ ! $? == 0 ]; then
          exit 1
      fi
      ;;
    application/gzip)    
      # Decompress the new kernel 
      echo "Decompressing kernel: "$CKPATH".............."
      zcat $CKPATH > $DKPATH
      if [ ! $? == 0 ]; then
          echo -e "\e[31mKERNEL FAILED TO DECOMPRESS!\e[0m"
          exit 1
      else
          echo -e "\e[32mKernel Decompressed Succesfully\e[0m"
      fi
      ;;
    *)
      echo -e "\e[32mUNKNOWN KERNEL FILE TYPE: $TYPE\e[0m"
      exit 1
      ;;      
esac


vladoportos
Posts: 16
Joined: Thu Dec 08, 2016 10:31 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Nov 05, 2020 6:45 am

Hi @jsciamms thanks fro the Ubuntu 20.04 usb boot image it works great, but it contains bugged netplan and wifi will not start ( apply ). It require update first to work ( see the bug report here: https://bugs.launchpad.net/ubuntu/+sour ... ug/1874377 )
I'm also curious how did you achieve that after kernel update it still boots up ok, without unpacking kernel as mentioned that we have to do it every time new kernel comes up ( I know it works, because I just updated and it booted the new kernel without issue )

Chris_K
Posts: 1
Joined: Wed Nov 04, 2020 3:17 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Nov 07, 2020 6:58 pm

Hi, I hope someone can help with my problem. I have used an 128GB sdcard with Ubuntu Server so far but due to reliability concerns I tried to switch to SSD on my raspberry pi 4.

I have removed the 128GB sdcard and inserted an SD card with Raspberry Pi OS and ran the commands

Code: Select all

sudo apt update
sudo apt full-upgrade
sudo reboot now
I also checked for the bootloader version and it said Sept 2020.

While I updated the bootloader I cloned the boot and system partitions to the SSD using KDE partition manager. After cloning I mounted the bootloader partition and did the following steps:

Code: Select all

zcat vmlinuz > vmlinux
nano config.txt
...and updated the [pi4] section accordingly.

I also added the auto decompress script but when I try to boot from SSD nothing happens. The pi does not boot up. I also tried a second raspberry pi 4. Unfortunately I do not have a suitable HDMI cable to check for outputs.

Any ideas are much welcomed.

nooz
Posts: 8
Joined: Sat Jul 25, 2020 1:45 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Nov 28, 2020 4:29 pm

On Ubuntu MATE 20.04 here and everything was great until just recently all video playback is severely impacted, youtube, twitter etc like <1 fps it's really bad. Have tried playing about with h264ify but no joy at all. System is fully updated. Any ideas?

edit: so changing to Chromium Version 87.0.4280.66 (Official Build) snap (64-bit) fixed the youtube video playback issues.

Anyone got bluetooth to pair and connect to any devices using this version? I'm having no joy at all. I will get the pairing request okay, but after that when I try to connect I get "Connection Failed: No route to host" :(

Deeco
Posts: 2
Joined: Fri Jan 02, 2015 12:18 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sun Dec 20, 2020 1:09 am

setuid wrote:
Mon Oct 19, 2020 10:48 pm
Just popping in here to state that these instructions do not/no longer work with 20.04 nor 20.10 images, arm64 nor armhf, for several reasons:

1.) The latest, upstream, Raspberry Pi firmware from September 3, 2020 absolutely requires an SD card to be inserted. It errors out on the very first splash screen unless you have one inserted.

2.) Ubuntu kernel images for arm64 are not compressed, they're MS-DOS executable format images (file(1) on the vmlinuz file will tell you this, and yes, that differs from a standard vmlinuz image on an x86_64 host)

3.) There's apparently no way to tell the firmware to ignore the missing SD card and jump to the attached USB SSD device.

I've tried multiple images, generations, Balena Etcher, dd and various other methods (dd works reliably the best and fastest). I can dd these same images to the SD card and they all boot perfectly fine without issue.

Does anyone have working instructions on how to get this working with just a USB SSD connected, and without a USB thumbdrive nor SD card inserted?
I'm facing the same issue, have you managed to find a solution?

hyst
Posts: 2
Joined: Fri May 19, 2017 6:19 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Tue Dec 22, 2020 2:31 am

Deeco wrote:
Sun Dec 20, 2020 1:09 am
setuid wrote:
Mon Oct 19, 2020 10:48 pm
Just popping in here to state that these instructions do not/no longer work with 20.04 nor 20.10 images, arm64 nor armhf, for several reasons:

1.) The latest, upstream, Raspberry Pi firmware from September 3, 2020 absolutely requires an SD card to be inserted. It errors out on the very first splash screen unless you have one inserted.

2.) Ubuntu kernel images for arm64 are not compressed, they're MS-DOS executable format images (file(1) on the vmlinuz file will tell you this, and yes, that differs from a standard vmlinuz image on an x86_64 host)

3.) There's apparently no way to tell the firmware to ignore the missing SD card and jump to the attached USB SSD device.

I've tried multiple images, generations, Balena Etcher, dd and various other methods (dd works reliably the best and fastest). I can dd these same images to the SD card and they all boot perfectly fine without issue.

Does anyone have working instructions on how to get this working with just a USB SSD connected, and without a USB thumbdrive nor SD card inserted?
I'm facing the same issue, have you managed to find a solution?
I assume you are talking about RPi4, have you updated the firmware today? I don't know if they changed it today or yesterday but I just updated as it's my first time trying USB boot with an adapter and an SSD and it works. I am having other problems but the SSD does at least boot, without any SD card or USB thumbdrive.

ncode
Posts: 4
Joined: Sun Dec 20, 2020 5:05 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Thu Dec 24, 2020 1:08 pm

I assume you are talking about RPi4, have you updated the firmware today? I don't know if they changed it today or yesterday but I just updated as it's my first time trying USB boot with an adapter and an SSD and it works. I am having other problems but the SSD does at least boot, without any SD card or USB thumbdrive.
Thanks for your reply, which firmware are you referring to exactly? My `rpi-eeprom` file is already up to date. Which method did you use to get it everything to work?

Perompak
Posts: 4
Joined: Wed Feb 26, 2014 5:36 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Fri Dec 25, 2020 3:59 pm

Folks. look here - the simple solution from Mumbai: https://www.youtube.com/watch?v=Mi_R8dGrNk0

thatchunkylad198966
Posts: 449
Joined: Thu Jul 04, 2019 10:21 am
Location: UK, Birmingham

Re: RPI4 Direct USB Boot Ubuntu 20.04

Fri Dec 25, 2020 5:02 pm

Perompak wrote:
Fri Dec 25, 2020 3:59 pm
Folks. look here - the simple solution from Mumbai: https://www.youtube.com/watch?v=Mi_R8dGrNk0
That's 20.10 which works with USB-BOOT out of the box.
This thread is about 20.04. ;)

goumas
Posts: 1
Joined: Sun Dec 27, 2020 3:04 am

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sun Dec 27, 2020 3:09 am

Hi guys. I need a usb boot ubuntu 18.04 asap pleaseee for raspb 4.
Thank you in advance.

ptTrR
Posts: 29
Joined: Thu Dec 03, 2020 11:44 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Tue Dec 29, 2020 12:17 pm

goumas wrote:
Sun Dec 27, 2020 3:09 am
Hi guys. I need a usb boot ubuntu 18.04 asap pleaseee for raspb 4.
Thank you in advance.
18.04 should be hard.
Checkout this image builder, there you can go with an 20.04.1 usb working version. prebuilt images also available.
viewtopic.php?f=131&t=296124

elsabio
Posts: 1
Joined: Sat Jan 02, 2021 12:15 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Sat Jan 02, 2021 1:30 pm

Confirming an earlier post, I have followed these instructions, but have not found them to be successful. Two immediate issues are that the vmlinux image is no longer compressed (and that is an easy fix) but more significant is that after updating the pi 4's EEPROM, booting will still only complete with an SD card in place, otherwise an 'SD: Card not found" error results. The updated firmware is from 11th December 2020, version 000138a1: are there different instructions for this firmware version?

I have posted this status and question on the canonical forum too, but that has seem no activity in the last 3 months as I post, so added here too.

swallow
Posts: 1
Joined: Mon Jan 18, 2021 9:14 pm

Re: RPI4 Direct USB Boot Ubuntu 20.04

Mon Jan 18, 2021 9:26 pm

I've been running ready-made image for a while successfully on USB-SSD (for about a month).
Today, I updated system (apt update/apt upgrade) which failed with following:

Code: Select all

Installing new w5500.dtbo.
dpkg: unrecoverable fatal error, aborting:
 unable to fsync updated status of 'initramfs-tools': Input/output error
Files have not changed, Decompression not needed
E: Sub-process /usr/bin/dpkg returned an error code (2)
I wasn't sure if I should run the update again or reboot and try and and decided on reboot.
And I got that beautiful rainbow screen of death.
Well, I call that as it's just stuck there and nothing else happens.
I went back to first post of this thread and decompressed vmlinuz (somehow on macOS Big Sur, I had to rename to add the extension Z otherwise I keep getting an error).
Still didn't boot.
It had vmlinux.bak so I renamed it to mvlinux and still no boot.
Can anyone give some pointers as to how I can revive it?

Return to “Ubuntu”