kvaruni
Posts: 2
Joined: Wed Feb 14, 2018 9:30 am

booting from a GPT mass storage device

Wed Feb 14, 2018 9:43 am

I'm trying to run a Raspberry Pi 3 directly from an external hard drive. The problem is that the drive is big (4TB), so MBR is not adequate. I can't seem to get GPT to work though.

So far, I have the Raspberry Pi 3 booting successfully from the hard drive by:
  • installing Raspbian Lite on it (no filesystem expansion, as it stretches to only 2TB);
  • setting up an initrd;
  • changing the boot options (fstab and cmdline.txt) to use UUID instead.
This all works fine and the Raspberry Pi 3 boots successfully from the HDD without a microSD card inserted.

The problems start when I try to convert the MBR to a GPT. To that end, I use gdisk, option r-f-y-w. The GPT gets nicely written, I can verify that the partitions are there and are correct, and I can still read the drive on Mac/Windows/Linux. However, the Raspberry Pi 3 won't boot anymore from this disk and I'm stuck on the rainbow screen. I have tried to add a hybrid MBR (though I'm not a fan), including only the first (boot) partition (set up once as not bootable and once as bootable), including the first and second partition (both bootable, only second one bootable), and including first, second, and free space (with identifier 0x04 to keep it working on MacOS). None of these seem to fix the issue.

Where am I going wrong here?

edit: I tried to add gpt to the end of cmdline.txt but that doesn't seem to make it work (i didn't try all versions of a hybrid MBR to see if that changed anything, mind). I also wired up a USB to UART Serial but that one remains blank, suggesting it cannot access /boot . The wiring for the serial was correct as I do see plenty of output when plugging in a microSD with a vanilla Raspbian Lite on there.

User avatar
thagrol
Posts: 2404
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: booting from a GPT mass storage device

Wed Feb 14, 2018 4:19 pm

This is speculation but I'd guess that the bootstrap code in the GPU doesn't support GPT disks, after all no-one makes an SD card big enought to need it and it's the GPU that handles the first stages of the boot sequence.

You might get away with using an sdcard with a boot partition containing only bootcode.bin, but more likely you'll need a full boot partition configured in such a way that it will use the HDD for "/".

Perhaps someone from RPF can confirm?
Attempts to contact me outside of these forums will be ignored unless signed in triplicate, sent in, sent back, queried, lost, found, subjected to public enquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters

kvaruni
Posts: 2
Joined: Wed Feb 14, 2018 9:30 am

Re: booting from a GPT mass storage device

Wed Feb 14, 2018 5:58 pm

I think that may indeed be the case, and I totally agree that this situation is an edge case. There are definitely other solutions out there, as you mention, such as using a tiny microsd and then mounting the external drive as /. At the moment, it's really about confirming whether or not it can be done.

There was one post in 2015 on the forums for GPT on a microSD but the question was never conclusively answered. The only bit of hope I have is https://github.com/raspberrypi/linux/issues/1060, but even if that one worked (I can't reproduce it) it may be that it doesn't work for booting directly from mass storage.

epoch1970
Posts: 4480
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: booting from a GPT mass storage device

Mon Nov 26, 2018 1:56 pm

FYI,

Code: Select all

gdisk ${IMG} <<EOF
r
h
1
n
04
y
n
m
w
y
EOF
works for me. "04" comes from here, I suppose you might prefer "06".

After boot if the SD resizes you get a complaint that the backup GPT header is not located after the last partition. You need to use gdisk again to fix that.

Code: Select all

gdisk ${BLKDEV} <<EOF
x
g
R
e
m
w
y
EOF
YMMV but for me resizing partitions with parted borked the hybrid GPT/MBR setup, resulting in failed reboot. Using gdisk to nuke/recreate partitions instead of using parted fixed that.
AFAIK GPT headers take 34 sectors each, beware of tools that fail to reserve enough room at each end of the block device.

Using hybrid GPT is a bit iffy overall, but it is a possibility.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

hdtodd
Posts: 37
Joined: Tue Mar 04, 2014 1:53 am

Re: booting from a GPT mass storage device

Sun Aug 04, 2019 4:32 am

I don't know if booting from a GPT mass storage device is still of interest and still an issue for others, but I set up such a system a few months ago and just had to rescue my prior install (after an upgrade from Stretch to Buster), and thought I'd share a sketch of what I did.

CAUTION: Everything I've read suggests that hybrid GPT/MBR drives could be unreliable, and I have no reason not to believe that. This has worked for me, in a non-critical application, but take care in using it for mission-critical systems.

I'm running a 3B+ booting off a 4TB WD USB3 drive. I used gdisk to set up the system using the instruction for creating a hybrid GPT/MBR drive from here: https://wiki.gentoo.org/wiki/Hybrid_partition_table. I created partitions to look like this:

Code: Select all

[email protected]:/home/hdtodd# gdisk /dev/sda
GPT fdisk (gdisk) version 1.0.3

Partition table scan:
  MBR: hybrid
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with hybrid MBR; using GPT.

Command (? for help): p
Disk /dev/sda: 7813969920 sectors, 3.6 TiB
Model: easystore 25FA  
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): 654D47AA-24BB-4FAC-A75D-22EAB4DF2C29
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7813969886
Partitions will be aligned on 2048-sector boundaries
Total free space is 10173 sectors (5.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            8192          507903   244.0 MiB   0700  Microsoft basic data
   2          507904        65634303   31.1 GiB    8300  Linux filesystem
   3        65634304        68706303   1.5 GiB     8200  
   4        68706304      7813967871   3.6 TiB     8300  

Command (? for help): q
In "recovery" mode, I created the MBR table with this:

Code: Select all

Recovery/transformation command (? for help): h

WARNING! Hybrid MBRs are flaky and dangerous! If you decide not to use one,
just hit the Enter key at the below prompt and your MBR partition table will
be untouched.

Type from one to three GPT partition numbers, separated by spaces, to be
added to the hybrid MBR, in sequence: 1
Place EFI GPT (0xEE) partition first in MBR (good for GRUB)? (Y/N): n

Creating entry for GPT partition #1 (MBR partition #1)
Enter an MBR hex code (default 07): 0c
Set the bootable flag? (Y/N): y

Unused partition space(s) found. Use one to protect more partitions? (Y/N): n

Recovery/transformation command (? for help): o

Disk size is 7813969920 sectors (3.6 TiB)
MBR disk identifier: 0x00000000
MBR partitions:

Number  Boot  Start Sector   End Sector   Status      Code
   1      *           8192       507903   primary     0x0C
   2                     1         8191   primary     0xEE
Note that I added just the first partition (FAT32) to the MBR tables. I didn't explore other options.

The upgrade from Stretch to Buster worked just fine, but the original install had only a 43MB partition for /BOOT, and the system warned me that it was nearly out of space. So I used gparted to shrink /dev/sda2 by 200MB on the front end and added that 200MB to /dev/sda1, and after that, the system wouldn't boot. It appears that using any partitioning tools other than gdisk to work with partitions destroys the hybrid tables, and you need to reconstruct them with the instructions above (the partition contents were preserved and booted just fine after the hybrid tables were reconstructed).

So I went through the process of creating the hybrid GPT/MBR partition tables as described above, and the system again boots just fine.

I've been using this Pi drive as an Apple Time Machine network backup system. Slow as molasses, but usable after the first save (on a 256GB system). Hoping that performance will improve when I attach the drive to a Pi-4, 4GB system this Fall. But at this point, at least, I have a system that has functioned for several months, and it has survived the Stretch --> Buster upgrade.
Last edited by hdtodd on Sun Aug 04, 2019 5:18 am, edited 1 time in total.

User avatar
rpdom
Posts: 16363
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: booting from a GPT mass storage device

Sun Aug 04, 2019 4:46 am

hdtodd wrote:
Sun Aug 04, 2019 4:32 am
So I used gparted to shrink /dev/sda2 by 200MB on the front end and added that 200MB to /dev/sda1, and after that, the system wouldn't boot.
I have heard that gparted has problems with resizing vfat partitions.

Code: Select all

Set the bootable flag? (Y/N): y
No need to set that flag. The Pi ignores it. It won't cause any problems, it just isn't used.

hdtodd
Posts: 37
Joined: Tue Mar 04, 2014 1:53 am

Re: booting from a GPT mass storage device

Sun Aug 04, 2019 5:28 am

I have heard that gparted has problems with resizing vfat partitions.
Yes, as I recall it did (I don't recall what all didn't work at that point). But I had made a backup "cp -a /boot nasboot/", so I reformatted and restored the resized VFAT partition when I wasn't sure what was causing the problem. That didn't solve the problem -- rebuilding the hybrid partition tables did -- but it would be a good precaution to back up that VFAT partition before resizing in any case.
Set the bootable flag? (Y/N): y

No need to set that flag. The Pi ignores it. It won't cause any problems, it just isn't used.
Thanks. I thought that might be the case, but I wasn't sure. It may not have done any good, but at least it didn't seem to do harm. And at the time, I was looking for any solution that worked. ;)

Return to “Advanced users”