BertilDator
Posts: 1
Joined: Wed Jan 09, 2019 9:37 pm

Re: Running Raspbian on USB Devices : Made Easy

Wed Jan 09, 2019 9:42 pm

Registered here only to thank you, so THANK YOU! :)

Couldn't get my OCZ Vertex 3 SSD boot via USB using Sabrent USB-SATA 3.0 cable (on RPi 3B), so this helpful script provided a suffient workaround for running my openHABianPi from SSD - and saved me a lot of time and money (testing different cables and SSDs).

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Sun Jul 14, 2019 2:45 am

Updated usb-boot to accommodate larger BOOT partition used on Raspbian Buster.

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Thu Jul 25, 2019 3:47 am

Quick startup:

Write the desired Raspbian image to an SD card and USB drive with Etcher.
If running headless, create an empty ssh (or ssh.txt) file in the boot partition of both the SD card and USB drive.
If running wireless, create a wpa_supplicant.conf file in the boot partition of both the SD card and USB drive.

Plug the SD card and USB drive into a Raspberry Pi and boot up.

Run usb-boot, answering 'No' to 'Replicate BOOT/ROOT contents from /dev/mmcblk0 to /dev/sdX?'.

Reboot (the USB drive will boot).
Last edited by RonR on Tue Jul 30, 2019 11:20 pm, edited 2 times in total.

geev03
Posts: 130
Joined: Thu Jun 07, 2012 12:40 pm
Location: London, UK

Re: Running Raspbian on USB Devices : Made Easy

Thu Jul 25, 2019 10:40 am

Raspbian on USB 3.1 stick ...

Code: Select all

[email protected]:/boot# ls
 bcm2708-rpi-b.dtb	    COPYING.linux   LICENCE.broadcom
 bcm2708-rpi-b-plus.dtb     fixup4cd.dat    overlays
 bcm2708-rpi-cm.dtb	    fixup4.dat	    start4cd.elf
 bcm2708-rpi-zero.dtb	    fixup4db.dat    start4db.elf
 bcm2708-rpi-zero-w.dtb     fixup4x.dat     start4.elf
 bcm2709-rpi-2-b.dtb	    fixup_cd.dat    start4x.elf
 bcm2710-rpi-3-b.dtb	    fixup.dat	    start_cd.elf
 bcm2710-rpi-3-b-plus.dtb   fixup_db.dat    start_db.elf
 bcm2710-rpi-cm3.dtb	    fixup_x.dat     start.elf
 bcm2711-rpi-4-b.dtb	    issue.txt	    start_x.elf
 bootcode.bin		    kernel7.img    'System Volume Information'
 cmdline.txt		    kernel7l.img    usb-boot
 config.txt		    kernel.img	    usb-boot.txt
[email protected]:/boot#
Attachments
boot_from_usb_pi4.jpg
boot_from_usb_pi4.jpg (148.86 KiB) Viewed 2670 times

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Thu Jul 25, 2019 5:20 pm

geev03 wrote:
Thu Jul 25, 2019 10:40 am
Raspbian on USB 3.1 stick ...
Congratulations!

Performance should be immensely better than running on an SD card.

Raspberry Pi 1/2/3 also show a very dramatic performance increase when run on a fast USB 3 device instead of an SD card (despite only having a USB 2 interface).

HankB
Posts: 125
Joined: Fri Jan 01, 2016 2:45 pm

Re: Running Raspbian on USB Devices : Made Easy

Thu Jul 25, 2019 8:18 pm

The script does not work so well if one has employed a seperate /home filesystem. It leaves /home on the SD card and fills the external drive with /. And it seems easy to make these unbootable when trying to resolve this, as I sit here staring at 4 raspbnerries on my screen.

I would suggest adding a test to the script tro bail if /home is not on / or at the least, put a warning in the instructions.

thanks,
hank

Edit.0: It turns out that using `gparted` to resize the root partition and add a home partition to the SSD altered the PARTUUIDs. I was able to resolve this by fixing the entries in /boot/cmdline.txt (on the SD card) and /etc/fstab on the SSD.
Last edited by HankB on Thu Jul 25, 2019 9:06 pm, edited 1 time in total.

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Thu Jul 25, 2019 8:55 pm

HankB wrote:
Thu Jul 25, 2019 8:18 pm
The script does not work so well if one has employed a seperate /home filesystem.
Is anyone other than you running a Raspberry Pi with /home on a separate filesystem? :)

usb-boot uses rsync to copy the contents of /dev/mmcblk0 to /dev/sdX (only when replicating the SD card to the USB device). If you know of an rsync option that will assist your situation, please let know.

Give me some details of what you're doing, why others might be doing the same thing, and any suggestions and I'll take a look.

HankB
Posts: 125
Joined: Fri Jan 01, 2016 2:45 pm

Re: Running Raspbian on USB Devices : Made Easy

Thu Jul 25, 2019 9:31 pm

RonR wrote:
Thu Jul 25, 2019 8:55 pm

Is anyone other than you running a Raspberry Pi with /home on a separate filesystem? :)
I may be the only one. ;)
usb-boot uses rsync to copy the contents of /dev/mmcblk0 to /dev/sdX (only when replicating the SD card to the USB device). If you know of an rsync option that will assist your situation, please let know.
AFAIK `rsync` limits its traversal to a single filesystem. To copy two filesystems, it would need to be run twice.
It's been a while since I modified a Pi to boot from USB (/boot on SD) but IIRC I used `dd` to copy the SD card to a larger USB drive, used `gparted` to adjust partition sizes and adjusted /boot/cmdline.txt and /etc/fstab for the new layout.
Give me some details of what you're doing, why others might be doing the same thing, and any suggestions and I'll take a look.
My process for standing up a Pi is presently:
  • copy downloaded image to micro-SD card.
  • create ssh and wpa_supplicant.conf if configuring a headless sytem that depends on WiFi
  • boot and go through initial configuration (resize, hostname, language, localization etc.)
  • remove micro-SD card and put in another PC.
  • use `gparted` to resize the root partition and create a home partition
  • move home directories to new filesystem.
  • adjust /etc/fstab to mount the new home partition
  • put micro-SD card back in the Pi and proceed with whatever I'm doing (and make sure I didn't forget anything.)
That's my initial setup. I had an idle SSD which seemed to work OK with the Pi (with UAS disabled) so I decided to move / and /home to it. I ran your script and it left me witrh a working system (Yay!) but with /home still on the micro-SD card. (boo.) I proceded to get this sorted and in the process learned a bit about `gparted` and how the Pi boots. ;) At present my setup is

Code: Select all

[email protected]:~ $ cat /etc/fstab
proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
PARTUUID=98408e28-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that


PARTUUID=98408e28-03  /home           ext4    defaults,noatime  0       1
oak:/export/share /mnt/share nfs noauto,x-systemd.automount,x-systemd.device-timeout=10,x-systemd.idle-timeout=1min 0 0

# previous /home and / mounts, set to noauto so they don't mount
/dev/mmcblk0p3 /mnt/sd-home ext4 defaults,noauto,noatime 0 1
/dev/mmcblk0p2 /mnt/sd-rootfs ext4 defaults,noauto,noatime 0 1
/dev/sda1 /mnt/ssd-boot vfat defaults,noauto 0 0

Code: Select all

[email protected]:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        19G  5.6G   12G  33% /
devtmpfs        1.8G     0  1.8G   0% /dev
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           2.0G  8.7M  1.9G   1% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/mmcblk0p1  253M   40M  214M  16% /boot
/dev/sda3       215G  682M  203G   1% /home
tmpfs           391M     0  391M   0% /run/user/1000
/dev/mmcblk0p3   18G  656M   16G   4% /mnt/sd-home
[email protected]:~ $ 
The "why" is just to partition system files from personal files. I've done that on most of my systems for decades now. (I don't always do that on single purpose systems such as those on Pi Zeroes, chromebooks and so on.)

Feel free to ask about this and/or point out anyuthing that youy think is going to give me problems.

cspan
Posts: 140
Joined: Sat Jun 10, 2017 1:03 pm
Location: Chattanooga, TN, USA

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 12:02 am

HankB wrote:
Thu Jul 25, 2019 9:31 pm
RonR wrote:
Thu Jul 25, 2019 8:55 pm

Is anyone other than you running a Raspberry Pi with /home on a separate filesystem? :)
The "why" is just to partition system files from personal files. I've done that on most of my systems for decades now. (I don't always do that on single purpose systems such as those on Pi Zeroes, chromebooks and so on.)
IANAAUY (I am not an advanced user yet) but I'd like to chime in and state that I do something similar with my RPD laptop, which I typically only use when I travel. Very little other than system files are on my laptop's hard drive, in case the laptop is stolen, broken, or dies. I keep my personal files on a USB stick, and only bring those that I need for the trip.

My desktop machine is simply backed up to external HDD, but again, I only back up my personal files, not the system. Perhaps if I was a corporate network manager I'd have a foolproof, er, generally reliable method to roll back/reinstall a system to a prior state. Once, long ago, I had occasion to need such a restoration, and it didn't work so well. More effective to do a clean reinstall, so I just backup my own documents, spreadsheets, and so forth, and bring them back in once everything else is back in place.

So HankB, I'm with you in spirit, at least.

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 1:02 am

HankB,

Thanks for the detailed explanation of how you're configuring your system.

The goal of usb-boot is to allow users to get standard Raspbian running on a USB device without the need to get involved with (or even have knowledge of) partitioning, fdisking, ptuuid's, creating filesystems, editing system files such as fstab and cmdline.txt, etc. To that end, I think asking it to handle rare and specialized cases is beyond its scope. While one could certainly automate your configuration process in a script, I don't think it belongs in usb-boot that tries to conform to normal Raspbian conventions. I wish I had a more accommodating response.

HankB
Posts: 125
Joined: Fri Jan 01, 2016 2:45 pm

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 1:23 am

RonR wrote:
Fri Jul 26, 2019 1:02 am
... To that end, I think asking it to handle rare and specialized cases is beyond its scope.
Hi RonR,
I agree 100%. I don't think it is worth anyone's effort to modify the script to accommodate non-standard partitioning. What I did suggest was
I would suggest adding a test to the script tro bail if /home is not on / or at the least, put a warning in the instructions.
(Emphasis added after the fact.)

best,
hank

User avatar
GavinW
Posts: 90
Joined: Tue Nov 01, 2011 8:11 pm
Location: UK
Contact: Website

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 12:39 pm

Please excuse a naive question on this thread. I have an Rpi4 with a USB(C) HDD attachable to it. Can I have Buster loaded from the HDD if it is attached, but loaded from the SSD card otherwise? Does this make sense?
otium negare negotium vanum

User avatar
rpdom
Posts: 15396
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 3:35 pm

GavinW wrote:
Fri Jul 26, 2019 12:39 pm
Please excuse a naive question on this thread. I have an Rpi4 with a USB(C) HDD attachable to it. Can I have Buster loaded from the HDD if it is attached, but loaded from the SSD card otherwise? Does this make sense?
Not yet. That is a feature that is planned for release in a month or two. It will require a firmware update to your Pi 4B with a special file on an SD card, but after that it will do as you requested.

The Pi 3B+ already has this feature hard-coded in.

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 5:41 pm

GavinW wrote:
Fri Jul 26, 2019 12:39 pm
Please excuse a naive question on this thread. I have an Rpi4 with a USB(C) HDD attachable to it. Can I have Buster loaded from the HDD if it is attached, but loaded from the SSD card otherwise? Does this make sense?
Using usb-boot, it's easy to switch loading Raspbian between the HDD and the SD card by simply moving a '#' in /boot/cmdline.txt:

Loading from HDD:

Code: Select all

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=12345678-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
# dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
Loading from SD card:

Code: Select all

# dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=PARTUUID=12345678-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
Last edited by RonR on Sun Jul 28, 2019 9:27 pm, edited 1 time in total.

User avatar
davidcoton
Posts: 4190
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 5:57 pm

rpdom wrote:
Fri Jul 26, 2019 3:35 pm
GavinW wrote:
Fri Jul 26, 2019 12:39 pm
Please excuse a naive question on this thread. I have an Rpi4 with a USB(C) HDD attachable to it. Can I have Buster loaded from the HDD if it is attached, but loaded from the SSD card otherwise? Does this make sense?
Not yet. That is a feature that is planned for release in a month or two. It will require a firmware update to your Pi 4B with a special file on an SD card, but after that it will do as you requested.

The Pi 3B+ already has this feature hard-coded in.
Not quite. The Pi3B+ will boot from the SDCard if it is present (and bootable), and will try USB devices next (IIRC).
Pi4 USB boot is not available yet, but expected before too long. It will likely be slightly different to the existing scheme because the controlling bits will be in EEPROM, not OTP memory.
Meanwhile, it is possible to put the boot partition on SDCard and transfer to root on a USB device. Search here, there are several suggestions from HawaiianPi and others.
Documentation here.
Signature retired

User avatar
rpdom
Posts: 15396
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 6:12 pm

davidcoton wrote:
Fri Jul 26, 2019 5:57 pm
Not quite.
I sit corrected :oops:
Yes, normally the Pi 3B+ (and 3B if the correct OTP bits are set) will try to boot from SD first, then USB.

It is possible to set other bits and tweak some gpios to make them boot in other orders.

Hopefully the upcoming options for the Pi 4 will have other boot sequence choices that can be changed in the eeprom.

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Fri Jul 26, 2019 10:35 pm

davidcoton wrote:
Fri Jul 26, 2019 5:57 pm
Meanwhile, it is possible to put the boot partition on SDCard and transfer to root on a USB device. Search here, there are several suggestions from HawaiianPi and others.
Or simply use the usb-boot script posted above to automatically set everything up without any partitioning, fdisking, editing, or anything else.

xlc
Posts: 26
Joined: Mon Jul 22, 2019 9:21 am

Re: Running Raspbian on USB Devices : Made Easy

Sat Jul 27, 2019 8:56 am

sorry Im a noob.
Where do I put the script?
does it run on boot?
can it be done headless?

Thanks

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Sat Jul 27, 2019 9:12 am

xlc wrote:
Sat Jul 27, 2019 8:56 am
Where do I put the script?
does it run on boot?
can it be done headless?
You simply place the script in your home directory ('pi') and run it manually:

Code: Select all

sudo ./usb-boot
From above:
RonR wrote:
Thu Jul 25, 2019 3:47 am
Quick startup:

Write the desired Raspbian image to an SD card and USB drive with Etcher.
If running headless, create an empty ssh (or ssh.txt) file in the boot partition of both the SD card and USB drive.
If running wireless, create a wpa_supplicant.conf file in the boot partition of both the SD card and USB drive.

Plug the SD card and USB drive into a Raspberry Pi and boot up.

Run usb-boot, answering 'No' to 'Replicate BOOT/ROOT contents from /dev/mmcblk0 to /dev/sdX?'.

Reboot (the USB drive will boot).
Last edited by RonR on Tue Jul 30, 2019 11:14 pm, edited 2 times in total.

xlc
Posts: 26
Joined: Mon Jul 22, 2019 9:21 am

Re: Running Raspbian on USB Devices : Made Easy

Sat Jul 27, 2019 10:06 am

ok so I have raspian already in my sd card.

I have a blank usb HDD.

I dont have to put an image onto the HDD? I can just run the script and select YES to copy over?

What about ssh? will it work this way?

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Sat Jul 27, 2019 6:07 pm

xlc wrote:
Sat Jul 27, 2019 10:06 am
ok so I have raspian already in my sd card.

I have a blank usb HDD.

I dont have to put an image onto the HDD? I can just run the script and select YES to copy over?

What about ssh? will it work this way?
If you already have an SD card running the way you want, you can simply run usb-boot and select 'Yes' to 'Replicate BOOT/ROOT contents from /dev/mmcblk0 to /dev/sdX?'. Your entire SD card will be copied to the USB drive. Then reboot and you'll be running from the USB drive.
Last edited by RonR on Sat Jul 27, 2019 7:58 pm, edited 1 time in total.

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Sat Jul 27, 2019 7:04 pm

xlc wrote:
Sat Jul 27, 2019 10:06 am
I can just run the script and select YES to copy over?
Yes.

I just wanted to add that the only change made to your SD card is in /boot/cmdline.txt:

root=/dev/mmcblk0p2

is changed to:

root=PARTUUID=12345678-02

(where 12345678 is the Partitiion Table UUID of your USB device)

If something runs amuck, simply edit cmdline.txt on your SD card, changing it back to:

root=/dev/mmcblk0p2

and all will be back to the way it was.

xlc
Posts: 26
Joined: Mon Jul 22, 2019 9:21 am

Re: Running Raspbian on USB Devices : Made Easy

Sun Jul 28, 2019 6:02 am

OK thanks.
Once Im done, how can I check that its actually booting from the usb and not the card anymore?

RonR
Posts: 552
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Running Raspbian on USB Devices : Made Easy

Sun Jul 28, 2019 6:24 am

xlc wrote:
Sun Jul 28, 2019 6:02 am
Once Im done, how can I check that its actually booting from the usb and not the card anymore?
Run:

Code: Select all

mount | grep " on / "
If running on the SD card, it will report:

Code: Select all

/dev/mmcblk0p2 on / type ext4 (rw,noatime)
If running on the USB drive, it will report:

Code: Select all

/dev/sda2 on / type ext4 (rw,noatime)

xlc
Posts: 26
Joined: Mon Jul 22, 2019 9:21 am

Re: Running Raspbian on USB Devices : Made Easy

Sun Jul 28, 2019 9:50 am

Ive placed it in my home directory and entered the run command but it says
sudo: ./usb-boot: command not found

Return to “Advanced users”