Running Raspbian from USB Devices : Made Easy

[RonR]

when I run sudo ./usb-boot, it immediately says "Aborted". No other prompts, just aborts and leaves me at the prompt.

I'm not familiar with OSMC. I just tried to spin up a copy on a Raspberry Pi 3B+, but it doesn't appear to be accessible headless via SSH. I tried to run it on a Raspberry Pi 4 where I have a keyboard and monitor, but it doesn't appear to boot properly.

usb-boot is intended for use on RaspiOS systems. "Aborted" is normally only displayed when you press the ESCAPE key on the keyboard.

It may be that OSMC is radically different from RaspiOS and therefore not compatible with usb-boot.

[SpiderT]

Given its application for Media Centers, I haven't had any reason to try accessing it via SSH when it was headless. OSMC was recently updated to Buster, but as far as I know, still isn't compatible with RPI4. I had assumed it was running on Raspbian, but I don't have sufficient expertise to evaluate that. Doing backups with your image-backup utility worked fine. I know that the /etc/os-release file identifies it as "OSMC" and that if I attempt to run raspi-config, I get a message that says in part "we do things a little differently" and points me to using My OSMC.

So... obviously some differences. Which ones might be incompatible with usb-boot, I have no idea. Does your program happen to examine the os-release file?

I know I tried to use your noobs2img utility on my initial NOOBS based installation. It indicated rsync wasn't installed and prompted me to do so, which I did. It appeared to complete normally then, with the exception of one error message: " ./noobs2img: line 178: ( - ) * 1024: syntax error: operand expected (error token is “) * 1024”)". An image file was created, but it wouldn't boot. So I just made a fresh download of OSMC (w/o NOOBS), bypassing the need for the noobs2img utility, and allowing compatibility with image-backup.

Thanks anyway for looking at this. Guess I'll have to manually make the modifications to my installation to get to booting off of the SD card and running off of the flash drive.

Dave

[RonR]

So... obviously some differences. Which ones might be incompatible with usb-boot, I have no idea. Does your program happen to examine the os-release file?

I moved my monitor and keyboard over to the Raspberry Pi 3 B+ and got OSMC up and running via WiFi and enabled SSH.

I managed to get usb-boot to run and successfully complete without errors by first running:

Code: Select all

apt update
apt install rsync whiptail parted
systemctl stop udisks-glue
systemctl disable udisks-glue.service

Unfortunately, OSMC does not appear to be happy trying to boot and run from a USB device.

It claims the filesystem is corrupt when, in fact, it is not.

[motomouse]

1. Download usb-boot.zip from the first post of this topic and unzip it to a location of your choice on your Raspberry Pi that is running Buster from an SD card

No need for a bootloader / firmware upgrade on a fresh out-of-the-box Pi4B ? Just got one and thinking about moving from SD as in my current Pi2B to USB SSD.

What is the lowest FW version required ?

[RonR]

1. Download usb-boot.zip from the first post of this topic and unzip it to a location of your choice on your Raspberry Pi that is running Buster from an SD card

No need for a bootloader / firmware upgrade on a fresh out-of-the-box Pi4B ? Just got one and thinking about moving from SD as in my current Pi2B to USB SSD.

Obviously, the system you intend to migrate must be capable of running on the target hardware. In order to run on a Raspberry Pi 4B, the system must be Buster and should have the latest updates installed (apt update && apt full-upgrade).

[wrigley418]

Another kudos on the pile from me; usb-boot took care of just about everything. My goal was to install the ubuntu-server image for raspberry pi on a 5TB hard drive, booting from the SD card. I had written the image to both devices, and usb-boot took care of booting to the HDD. The problem I have is that any kernel update packages aren't being picked up. Once the OS is running, /boot is mounted to the HDD; kernel images are written there, but the only kernel present on the SD card's /boot is the version from installation and that's what gets loaded.

I don't know if it's a compatibility with ubuntu-server; everything during the two executed steps (boot from SD to run from HDD, and GPT modification for >2TB disk) ran without errors. My /etc/fstab contains the following:

Code: Select all

$ cat /etc/fstab
PARTUUID=dca5f882-3528-47fd-93a9-101dd6b51719   /        ext4   defaults        0 0
LABEL=system-boot       /boot/firmware  vfat    defaults        0       1

And the system-boot label points to a partition on the HDD:

Code: Select all

$ ls -la /dev/disk/by-label/
total 0
drwxr-xr-x 2 root root  80 Sep 24 14:27 .
drwxr-xr-x 8 root root 160 Dec 31  1969 ..
lrwxrwxrwx 1 root root  10 Jan  4 15:03 system-boot -> ../../sda1
lrwxrwxrwx 1 root root  10 Jan  4 15:04 writable -> ../../sda2

The SD card is present with its partitions:

Code: Select all

$ sudo fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 7.43 GiB, 7973371904 bytes, 15572992 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x254a9658

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1 *      2048   526335   524288  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      526336 15572958 15046623  7.2G 83 Linux

When everything's running, should I see the SD card's /boot mounted (instead of the HDD)? Is that just a change to /etc/fstab & call it a day?

[RonR]

My goal was to install the ubuntu-server image for raspberry pi on a 5TB hard drive, booting from the SD card. I had written the image to both devices, and usb-boot took care of booting to the HDD. The problem I have is that any kernel update packages aren't being picked up. Once the OS is running, /boot is mounted to the HDD; kernel images are written there, but the only kernel present on the SD card's /boot is the version from installation and that's what gets loaded.

I don't know if it's a compatibility with ubuntu-server; everything during the two executed steps (boot from SD to run from HDD, and GPT modification for >2TB disk) ran without errors.

usb-boot is intended for use on RasPiOS-based systems.

Ubuntu has some major differences, especially in the boot area. Consequently, usb-boot is not suitable for use on Ubuntu.

[arkaium]

Hi all,

Is there a good one-spot guide for getting USB/SSD boot setup? I have a Pi 4, Argon One M.2, and 1TB M2 2280 drive I'm going to be setting up tonight. The smallest SD card I have is 16gb, I think, which feels like a waste but I don't have anything smaller!

I know I need to update the EEPROM in order to direct USB/SSD boot, so I'm thinking I need to flash Raspberry OS to the SD (which becomes inaccessible once the case is assembled btw), get it started, run the EEPROM tool, and then from what I'm seeing in this chain, I run USB-BOOT and it moves everything from the SD to the SSD and the next time I start, it direct USB boots?

Is there anything else I need to do to disable SD boot? I've done a few different searches here and online and I don't seem to find a concise one page guide because the feature is pretty fresh for Pi 4.

Thank you!

[RonR]

Is there a good one-spot guide for getting USB/SSD boot setup?

Get your Raspberry Pi 4 booting a fresh copy of RasPiOS from your USB device by starting here:: Booting a Pi-4B via USB

Once you can boot a fresh copy of RasPiOS from your USB device, boot your original SD card, update it (sudo apt update && sudo full-upgrade), and use usb-boot to replicate/migrate it to the USB device. When usb-boot completes, you should be able to power down, remove the SD card, boot the USB device, and be up and running your original system.

[arkaium]

Get your Raspberry Pi 4 booting a fresh copy of RasPiOS from your USB device by starting here:: Booting a Pi-4B via USB

Once you can boot a fresh copy of RasPiOS from your USB device, boot your original SD card, update it (sudo apt update && sudo full-upgrade), and use usb-boot to replicate/migrate it to the USB device. When usb-boot completes, you should be able to power down, remove the SD card, boot the USB device, and be up and running your original system.

Thank you! Am I able to use the SD card as extra storage without much hassle? Maybe I’ll put a larger one in if so, that’ll let me use it instead of trying to put the tiniest SD in, because I don’t have tiny ones and it feels a waste.

[RonR]

Am I able to use the SD card as extra storage without much hassle? Maybe I’ll put a larger one in if so, that’ll let me use it instead of trying to put the tiniest SD in, because I don’t have tiny ones and it feels a waste.

Yes. I set BOOT_ORDER=0xf14 which will boot the USB device even if a bootable SD card is present. To intentionally boot the SD card, it's simply necessary to unplug the USB device (my USB hub has switches, making it especially easy to disable USB devices). Having a bootable SD card present is a good fallback should something amiss happen to the USB device.

[poshbunny]

Hope someone can help.

Trying to copy SD to SSD.

Running Raspbian with XFCE, cannot get piclone user permissions to work - standard user can select source and target but error "permission denied", sudo and sudo su not able to select source or target.

So I ended up in this thread thinking usb-boot might be a saviour, but I am stuck.
Set usb-boot as executable and run it with sh, only to get the following output:

trap: SIGINT: bad trap
rsync version 3.1.3 protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
GPT fdisk (gdisk) version 1.0.3

usb-boot: 96: usb-boot: Syntax error: redirection unexpected

Any assistance - either on piclone or usb-boot - gratefully received.

Thank you

[RonR]

Set usb-boot as executable and run it with sh, only to get the following output:

usb-boot is a bash script and probably won't run under sh.

Simply run:

Code: Select all

sudo ./usb-boot

[poshbunny]

Set usb-boot as executable and run it with sh, only to get the following output:

usb-boot is a bash script and probably won't run under sh.

Simply run:

Code: Select all

sudo ./usb-boot

Thank you, but that didn't work either.

I did find a solution to run piclone though which is this command:

Code: Select all

sudo dbus-launch piclone

This got piclone running as needed and I have now moved my microSD across to SSD and running with no issues at all.

[RonR]

I did find a solution to run piclone though which is this command:

Code: Select all

sudo dbus-launch piclone

This got piclone running as needed and I have now moved my microSD across to SSD and running with no issues at all.

It's not clear what you're running or doing there. dbus-launch doesn't exist on a normal Raspberry Pi OS installation.

[rkger]

Hi. I tried to move my sd-card to an SSD with the usb-boot script. But I get an error

Code: Select all

mount: /tmp/usb-boot-mnt: can't read superblock on /dev/mmcblk0p2

Unable to mount ROOT partition

This is strange because this partition - mmcblk0p2 - seems to be a bit special.

When I do a fdisk -l I get

Code: Select all

sudo fdisk /dev/mmcblk0 -l
Disk /dev/mmcblk0: 14.4 GiB, 15485370368 bytes, 30244864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000e6819

Device         Boot   Start      End  Sectors  Size Id Type
/dev/mmcblk0p1         8192  5039062  5030871  2.4G  e W95 FAT16 (LBA)
/dev/mmcblk0p2      5039063 30244863 25205801   12G  5 Extended
/dev/mmcblk0p5      5046272  5111805    65534   32M 83 Linux
/dev/mmcblk0p6      5111808  5636093   524286  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p7      5636096 30244863 24608768 11.8G 83 Linux

When I then do a blkid I get

Code: Select all

sudo blkid
/dev/mmcblk0p1: LABEL_FATBOOT="RECOVERY" LABEL="RECOVERY" UUID="9B07-6027" TYPE="vfat" PARTUUID="000e6819-01"
/dev/mmcblk0p5: LABEL="SETTINGS" UUID="46acb6a5-**4432ab" TYPE="ext4" PARTUUID="000e6819-05"
/dev/mmcblk0p6: LABEL_FATBOOT="boot" LABEL="boot" UUID="06EB-CC87" TYPE="vfat" PARTUUID="000e6819-06"
/dev/mmcblk0p7: LABEL="root" UUID="5bbe724e-**5ac2" TYPE="ext4" PARTUUID="000e6819-07"
/dev/mmcblk0: PTUUID="000e6819" PTTYPE="dos"

As you can see "/dev/mmcblk0p2" is not listed. But when I do blkid with /dev/mmcblk0p2 I get

Code: Select all

sudo blkid /dev/mmcblk0p2
/dev/mmcblk0p2: PTTYPE="dos" PARTUUID="000e6819-02"

I have several copies of this SD card, which are also of different ages. The layout of the SD card is always identical. The Pi works and boots without errors. There are also no errors in the logs that indicate a problem with the file system. How do I solve the problem with "/dev/mmcblk0p2"?

My setup:

Code: Select all

pi@pi:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

pi@pi:~ $ cat /proc/device-tree/model
Raspberry Pi 3 Model B Plus Rev 1.3

[rkger]

Hi. I tried to move my sd-card to an SSD with the usb-boot script. But I get an error

Code: Select all

mount: /tmp/usb-boot-mnt: can't read superblock on /dev/mmcblk0p2

Unable to mount ROOT partition

This is strange because this partition - mmcblk0p2 - seems to be a bit special.

When I do a fdisk -l I get

Code: Select all

sudo fdisk /dev/mmcblk0 -l
Disk /dev/mmcblk0: 14.4 GiB, 15485370368 bytes, 30244864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000e6819

Device         Boot   Start      End  Sectors  Size Id Type
/dev/mmcblk0p1         8192  5039062  5030871  2.4G  e W95 FAT16 (LBA)
/dev/mmcblk0p2      5039063 30244863 25205801   12G  5 Extended
/dev/mmcblk0p5      5046272  5111805    65534   32M 83 Linux
/dev/mmcblk0p6      5111808  5636093   524286  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p7      5636096 30244863 24608768 11.8G 83 Linux

When I then do a blkid I get

Code: Select all

sudo blkid
/dev/mmcblk0p1: LABEL_FATBOOT="RECOVERY" LABEL="RECOVERY" UUID="9B07-6027" TYPE="vfat" PARTUUID="000e6819-01"
/dev/mmcblk0p5: LABEL="SETTINGS" UUID="46acb6a5-5fb1-4045-9960-59060e4432ab" TYPE="ext4" PARTUUID="000e6819-05"
/dev/mmcblk0p6: LABEL_FATBOOT="boot" LABEL="boot" UUID="06EB-CC87" TYPE="vfat" PARTUUID="000e6819-06"
/dev/mmcblk0p7: LABEL="root" UUID="5bbe724e-88e8-47ba-88d1-e77d7f715ac2" TYPE="ext4" PARTUUID="000e6819-07"
/dev/mmcblk0: PTUUID="000e6819" PTTYPE="dos"

As you can see "/dev/mmcblk0p2" is not listed. But when I do blkid with /dev/mmcblk0p2 I get

Code: Select all

sudo blkid /dev/mmcblk0p2
/dev/mmcblk0p2: PTTYPE="dos" PARTUUID="000e6819-02"

I have several copies of this SD card, which are also of different ages. The layout of the SD card is always identical. The Pi works and boots without errors. There are also no errors in the logs that indicate a problem with the file system. How do I solve the problem with "/dev/mmcblk0p2"?

My setup:

Code: Select all

pi@pi:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

pi@pi:~ $ cat /proc/device-tree/model
Raspberry Pi 3 Model B Plus Rev 1.3

[fbe]

Hi. I tried to move my sd-card to an SSD with the usb-boot script. But I get an error

Code: Select all

mount: /tmp/usb-boot-mnt: can't read superblock on /dev/mmcblk0p2

Unable to mount ROOT partition

This is strange because this partition - mmcblk0p2 - seems to be a bit special.
...

/dev/mmcblk0p2 is not a partition on your SSD but on your SD card. Actually there is no SSD or any other USB drive visible in the output of fdisk -l or blkid.

You are running an installation with NOOBS. /dev/mmcblk0p2 is an "extended" partition. This is a container that has no filesystem but allows the definition of partitions outside of the master boot record.

[rkger]

Yes that's right. It is also only about my SD card. The SSD is empty and is reinitialized with every attempt.

What do I do with the information? The script seems to try to mount something that it should not mount. Or do I see this wrong? Is my setup incompatible with the script or does it just need to be adjusted accordingly?

[RonR]

@rkger,

When Raspberry Pi OS is running in NOOBS, the BOOT partition is /dev/mcmblk0p6 and the ROOT partition is /dev/mcmblk0p7. usb-boot should not be attempting to access or mount any other partitions. Check /etc/fstab to verify these are the only partitions being referenced. Also run 'df' to verify no other partitions are mounted.

I would also insert the suspect SD card in a USB adapter, boot from another SD card, and check the ROOT filesystem by running:

Code: Select all

sudo fsck -f /dev/sda7

[rkger]

I tried all this. But I can't see any errors.

Code: Select all

cat /etc/fstab
proc            /proc           proc    defaults          0       0
/dev/mmcblk0p6  /boot           vfat    defaults          0       2
/dev/mmcblk0p7  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that


df
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/root       11979968 10494176    854192  93% /
devtmpfs          469544        0    469544   0% /dev
tmpfs             474152        0    474152   0% /dev/shm
tmpfs             474152    12440    461712   3% /run
tmpfs               5120        8      5112   1% /run/lock
tmpfs             474152        0    474152   0% /sys/fs/cgroup
/dev/mmcblk0p6    258094    53033    205061  21% /boot
tmpfs              94828        0     94828   0% /run/user/1000
tmpfs              94828        0     94828   0% /run/user/109

sudo fsck -f /dev/sda7
fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
root: 678909/796544 files (0.0% non-contiguous), 2690542/3185024 blocks

Maybe only one question: Why is / mounted on /dev/root and not /dev/mmcblk0p7?

[RonR]

Maybe only one question: Why is / mounted on /dev/root and not /dev/mmcblk0p7?

That's the way it's been since Jessie or Stretch.

You might try using image-backup (with the -x option) to extract Raspberry Pi OS to an image file and then write that image file to the SSD (NOOBS won't be altered). The end result will be the same.

They both use very similar logic, so you may or may not encounter the same problem.

[fbe]

When Raspberry Pi OS is running in NOOBS, the BOOT partition is /dev/mcmblk0p6 and the ROOT partition is /dev/mcmblk0p7. usb-boot should not be attempting to access or mount any other partitions.

... but it first mounts 2nd and 1st partition of the target USB device for replication of the running system and later, if "Use SD card to boot the USB device?" was confirmed (or for models that don't support booting without SD card), it attempts to mount 2nd and 1st partition of the SD card to adjust cmdline.txt on the SD card boot partition.

Code: Select all

...
MNTPATH="/tmp/usb-boot-mnt"

mntdev()
{
  MNTED=TRUE
  if [ ! -d "${MNTPATH}/" ]; then
    mkdir "${MNTPATH}/"
    if [ $? -ne 0 ]; then
      errexit "Unable to make ROOT partition mount point"
    fi
  fi
  mount "${1}2" "${MNTPATH}/"
  if [ $? -ne 0 ]; then
    errexit "Unable to mount ROOT partition"
  fi
  if [ ! -d "${MNTPATH}/boot/" ]; then
    mkdir -p "${MNTPATH}/boot/"
    if [ $? -ne 0 ]; then
      errexit "Unable to make BOOT partition mount point"
    fi
  fi
  mount "${1}1" "${MNTPATH}/boot/"
  if [ $? -ne 0 ]; then
    errexit "Unable to mount BOOT partition"
  fi
}
...
if [ "${USESDC}" = "TRUE" ]; then
  if [ -b /dev/mmcblk0 ]; then
    mntdev "/dev/mmcblk0p"
    sed -i "/^[[:space:]]*#/!s|^\(.*root=\)\S\+\(\s\+.*\)$|\1PARTUUID=${PARTUUID_2}\2|" "${MNTPATH}/boot/cmdline.txt"
    umntdev
  else
    errexit "No SD card present"
  fi
fi
...

The SSD is empty and is reinitialized with every attempt.

After review of usb-boot I would expect two partitions on the SSD with the contents of boot and root partitions of your SD card. But the SSD is not bootable (even if your Raspberry Pi supports booting from USB) because usb-boot aborted before the adjustments in cmdline.txt and /etc/fstab.

usb-boot may work with NOOBS if you anwer NO to "Use SD card to boot the USB device?". You are asked only if you have got a Raspberry Pi model that supports USB mass storage boot (https://www.raspberrypi.org/documentati ... des/msd.md).

[RonR]

When Raspberry Pi OS is running in NOOBS, the BOOT partition is /dev/mcmblk0p6 and the ROOT partition is /dev/mcmblk0p7. usb-boot should not be attempting to access or mount any other partitions.

... but it first mounts 2nd and 1st partition of the target USB device for replication of the running system and later, if "Use SD card to boot the USB device?" was confirmed (or for models that don't support booting without SD card), it attempts to mount 2nd and 1st partition of the SD card to adjust cmdline.txt on the SD card boot partition.

usb-boot was never really intended to be used with NOOBS. It turned out that it copies the running BOOT/ROOT partitions from NOOBS to a USB device just fine, but I didn't anticipate that someone would also try to use Hybrid booting in conjunction with the NOOBS SD card. I'll look into having usb-boot check if it's running on NOOBS and omit the Hybrid boot option if it is.

Answering 'No' to "Use SD card to boot the USB device?" will probably eliminate rkger's problem. If Hybrid booting is needed, an SD card with a normal Raspberry Pi OS image on it can be then be inserted and usb-boot run from it, answering 'No' to "Replicate BOOT/ROOT contents ...?".

[rkger]

Thanks for the review. I had also intended to do that, but due to time constraints have not yet managed. My Pi does not technically need hybrid-boot. I will try it with 'no'. And report afterwards.