dickon
Posts: 565
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Raspberry Pi 4 usb boot?

Sun Nov 10, 2019 3:55 pm

Two minutes of Googling found http://elm-chan.org/fsw/ff/00index_e.html which looks as if it'll do the trick, isn't GPL, and looks to be current. Bonus: designed for embedded systems, so minimally RO compiles to ~4k ARMv7, and requires a trivial set of IO primitives.

I'm sure there will be others out there. No need to write from scratch.

trejan
Posts: 885
Joined: Tue Jul 02, 2019 2:28 pm

Re: Raspberry Pi 4 usb boot?

Sun Nov 10, 2019 4:11 pm

dickon wrote:
Sun Nov 10, 2019 3:55 pm
Two minutes of Googling found http://elm-chan.org/fsw/ff/00index_e.html which looks as if it'll do the trick, isn't GPL, and looks to be current. Bonus: designed for embedded systems, so minimally RO compiles to ~4k ARMv7, and requires a trivial set of IO primitives.
Ah nice. I didn't know FatFs had been updated. I've used it before in a microcontroller and its a very handy library.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6084
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Raspberry Pi 4 usb boot?

Mon Nov 11, 2019 2:01 pm

AFAIK, there are no plans to support exFAT.

User avatar
PeterO
Posts: 5142
Joined: Sun Jul 22, 2012 4:14 pm

Re: Raspberry Pi 4 usb boot?

Mon Nov 11, 2019 3:50 pm

mahjongg wrote:
Sun Nov 10, 2019 2:21 pm
If the boot-loader will be revised, will it be able to boot from an exFAT partition? All previous RPI boot system were only capable of booting from a FAT32 partition, but with larger than 32GB SD-Cards getting cheaper all the time more and more cards will be formatted exFAT. I know that many people no longer use NOOBS, but wouldn't it be nice if exFAT booting could be implemented?

if booting from exFAT means we need to pay MS licences, then forget that I asked.
Surely the size of the SD card it irrelevant ? It's only the size of the boot partition that matters, and since that is quite small there is no need to support exFAT.

PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

User avatar
procount
Posts: 1847
Joined: Thu Jun 27, 2013 12:32 pm
Location: UK

Re: Raspberry Pi 4 usb boot?

Mon Nov 11, 2019 4:32 pm

PeterO wrote:
Mon Nov 11, 2019 3:50 pm
Surely the size of the SD card is irrelevant ? It's only the size of the boot partition that matters, and since that is quite small there is no need to support exFAT.
This is very true.

However, the instructions for NOOBS (since it was written when SD cards were all relatively small) state: "Format an SD card that is 8GB or greater in size as FAT32", which was ok when cards were small, but can cause issues now with SD cards >=64GB, as we know.

Actually, NOOBS only requires the first partition to be formatted as FAT32, and this need only be big enough to hold the NOOBS installation files. So you are absolutely right, but creating an initial boot partition of the correct size and then formatting it as FAT32 suddenly becomes a none-newbie experience.

Now that Etcher is available, my solution was to create an Etcher image for pinn-lite, which makes it as easy to install as any other OS, and avoids all the issues of large SD cards, special FAT32 formatting tools, and issues with SD cards not being partitioned properly with an MBR.
PINN - NOOBS with the extras... https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=142574

User avatar
PeterO
Posts: 5142
Joined: Sun Jul 22, 2012 4:14 pm

Re: Raspberry Pi 4 usb boot?

Mon Nov 11, 2019 5:48 pm

procount wrote:
Mon Nov 11, 2019 4:32 pm
Actually, NOOBS only requires the first partition to be formatted as FAT32, and this need only be big enough to hold the NOOBS installation files. So you are absolutely right, but creating an initial boot partition of the correct size and then formatting it as FAT32 suddenly becomes a none-newbie experience.
Ah, I had missed the NOOBS angle. I've never used it so I don't know how it works.
PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

vintozver
Posts: 32
Joined: Thu Jun 01, 2017 2:16 am
Location: Bellevue, WA, USA

Re: Raspberry Pi 4 usb boot?

Mon Nov 11, 2019 11:50 pm

trejan wrote:
Sun Nov 10, 2019 3:34 pm
It'll be the cost of writing an exFAT module for the VPU from scratch. The one in the Linux kernel can't be used as it is GPL. The initial patch for it was nearly 11,000 lines so it is a lot more complicated than FAT.
I don't see the point. boot partition is for boot. Just slice out the partition of the size you need.
I would rather ask for the GPT support ...

trejan
Posts: 885
Joined: Tue Jul 02, 2019 2:28 pm

Re: Raspberry Pi 4 usb boot?

Tue Nov 12, 2019 1:11 am

vintozver wrote:
Mon Nov 11, 2019 11:50 pm
I don't see the point. boot partition is for boot. Just slice out the partition of the size you need.
As procount mentioned above, it would be a good addition for NOOBS which is installed by extracting the contents of the zip into a FAT32 formatted SD card. The FAT32 requirement means needing to run a third party formatting tool which isn't very noob friendly.

User avatar
procount
Posts: 1847
Joined: Thu Jun 27, 2013 12:32 pm
Location: UK

Re: Raspberry Pi 4 usb boot?

Tue Nov 12, 2019 8:49 am

Sorry, I may not have been clear in my opinion.

I am not advocating adding exFAT as a boot option.
I was trying to point out WHY current instructions for installing NOOBS might make people think exFAT would be necessary now that large SD cards >= 64GB are more common.

But in fact, this is not needed. There are a couple of alternative options:

1. The noobs instructions could be modified to create a small recovery partition which can then be formatted as FAT32, instead of formatting the whole SD card. But this is not very newbie friendly, which is what NOOBS was all about.

2. Now that Etcher is here which makes writing images as easy as formatting/unzipping for newbies, NOOBS could be provided as an alternative image file, like pinn-lite.img.zip is.

Actually, requiring the whole SD card to be formatted as FAT32 first for NOOBS is wasteful in both time and SD writes, contributing to additional wear.

So NOOBS requirements can be made the same as any other OS (i.e. a small FAT32 partition). I think this is preferable to adding a load of exFAT bloat to the bootloader, which I'm sure RPF engineers won't be doing.
PINN - NOOBS with the extras... https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=142574

jj_0
Posts: 108
Joined: Wed Jul 11, 2012 7:07 am

Re: Raspberry Pi 4 usb boot?

Tue Nov 12, 2019 9:40 am

procount wrote:
Tue Nov 12, 2019 8:49 am
2. Now that Etcher is here which makes writing images as easy as formatting/unzipping for newbies, NOOBS could be provided as an alternative image file, like pinn-lite.img.zip is.
Yes, but at the moment Etcher is only installable on the Pi by getting the .deb package directly for @futurejones's github. This is also not easy for noobs. It might be easier if in addition of providing an image file the NOOBS instructions included installing gnome-disks. The benefit of this is that gnome-disks is part of the Raspbian distribution so can be installed via apt or via 'Add/ Remove Software' and also is maintained/updated.

User avatar
procount
Posts: 1847
Joined: Thu Jun 27, 2013 12:32 pm
Location: UK

Re: Raspberry Pi 4 usb boot?

Tue Nov 12, 2019 10:07 am

jj_0 wrote:
Tue Nov 12, 2019 9:40 am
Yes, but at the moment Etcher is only installable on the Pi by getting the .deb package directly for @futurejones's github.
Sorry, but I don't see the relevance of that comment.
You don't install Etcher on a RPI in order to install NOOBS (or any other OS) on an SD card in order to boot your RPI for the first time!
You run it on your Windows PC, your Mac, or maybe even your Linux PC (although Linux officianados may just use dd or cp or pkzip)

If you are using Etcher on a RPi, you have already overcome the hurdle of getting your first OS onto an SD card and booting it.
PINN - NOOBS with the extras... https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=142574

jj_0
Posts: 108
Joined: Wed Jul 11, 2012 7:07 am

Re: Raspberry Pi 4 usb boot?

Tue Nov 12, 2019 12:17 pm

procount wrote:
Tue Nov 12, 2019 10:07 am
jj_0 wrote:
Tue Nov 12, 2019 9:40 am
Yes, but at the moment Etcher is only installable on the Pi by getting the .deb package directly for @futurejones's github.
Sorry, but I don't see the relevance of that comment.
You don't install Etcher on a RPI in order to install NOOBS (or any other OS) on an SD card in order to boot your RPI for the first time!
You run it on your Windows PC, your Mac, or maybe even your Linux PC (although Linux officianados may just use dd or cp or pkzip)

If you are using Etcher on a RPi, you have already overcome the hurdle of getting your first OS onto an SD card and booting it.
You are right of course, I probably read too much into "now that Etcher is here"...

User avatar
ehem
Posts: 33
Joined: Mon Sep 09, 2019 1:17 am

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 1:05 am

procount wrote:
Tue Nov 12, 2019 8:49 am
I am not advocating adding exFAT as a boot option.
For the heck of it, I think the Raspberry PI should really support booting from UDF filesystems. UDF is an open standard unlike FAT. Every OS other than FreeBSD has support for reading and writing UDF, and UDF has a number of features FAT lacks (proper Unix filenames!).

HawaiianPi wrote:
Mon Jul 01, 2019 10:55 pm
I've been running a Minecraft SMP multiplayer server on a Pi 3B+ from a SanDisk Ultra Plus 64GB micro SD card since 2017, and it's still running fine. The server auto-saves the world every 3 minutes and I run weekly backups and monthly OS upgrades. Lots of writes and no trouble. I had a torrent server running for around a year from a micro SD card as well with no trouble (upgraded to SSD because I needed more room and SSD was cheaper).
My one concern about larger MicroSD cards is they tend to have very large erase sizes. I have a SanDisk Extreme 200GB (I must have gotten overstock since it wasn't that much) and the value for [/sys/block/mmcblk0/queue/] discard_granularity is 24MB. ext4's behavior of keeping files written at the same time together likely allows MicroSD cards to reduce the number of write and erase cycles. Yet that number is so large that I cannot help being concerned about write amplification causing major damage to the lifetime of such a card. For eMMC chips on phones the value is generally no more than 4096 (4KB).

As such I'm pondering changing my plan from using a large MicroSD card to using a USB3 connected SATA/M.2 SSD for main storage. My concern is strictly how long it will last. Anyone care to offer advice?

Sure, needing to use a MicroSD card for the initial boot stages is a downside. Yet I can readily live with that for now.

RossDv8
Posts: 243
Joined: Thu Jan 21, 2016 12:35 pm

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 5:34 am

Sure, needing to use a MicroSD card for the initial boot stages is a downside. Yet I can readily live with that for now.
Please don't take this as argumentative. It is just a question. What would be the downside of using a microSD card for the initial boot stages? Other than that with a purely SSD or eeprom > SSD boot, the microSD slot could possibly be used as removable storage.

Since setting up microSD boot > SSD operating, the Pi 4 is amazing. Once in a while I will etch something like Manjaro or some other OS to a microSD and just swap cards to have look at what I might like about other systems, but realistically, under normal circumstances (for me personally) I rarely think about the Pi 4 not booting from SSD except when I read a post about it.

That said, there must be a downside or two surely?
Remember, nobody is listening to you
until you fart ...

W. H. Heydt
Posts: 11091
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 6:30 am

RossDv8 wrote:
Sun Nov 24, 2019 5:34 am
That said, there must be a downside or two surely?
Or even just a preference to putting everything on one device and not having to concern oneself with where exactly the *correct* /boot partition is.

RonR
Posts: 642
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 6:39 am

RossDv8 wrote:
Sun Nov 24, 2019 5:34 am
What would be the downside of using a microSD card for the initial boot stages? Other than that with a purely SSD or eeprom > SSD boot, the microSD slot could possibly be used as removable storage.

IMO, using a microSD slot for removable storage is a horrible idea compared to using USB. An available microSD slot is likely to remain unused.
RossDv8 wrote:
Sun Nov 24, 2019 5:34 am
Once in a while I will etch something like Manjaro or some other OS to a microSD and just swap cards to have look at what I might like about other systems, but realistically, under normal circumstances (for me personally)

I almost never remove an SD card. I run multiple environments off of fast/inexpensive USB flash drives. Using only sdc-boot, I can quickly reboot to any of the multiple environments with a single command.

I seriously doubt the initial release of a USB boot capable eeprom will have the flexibility of what I've been using since the Raspberry Pi 1. If I'm wrong, all the better.

RonR
Posts: 642
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 6:48 am

W. H. Heydt wrote:
Sun Nov 24, 2019 6:30 am
RossDv8 wrote:
Sun Nov 24, 2019 5:34 am
That said, there must be a downside or two surely?
Or even just a preference to putting everything on one device and not having to concern oneself with where exactly the *correct* /boot partition is.

When booting via an SD card, the *correct* /boot partition is always located on the SD card. There's nothing to concern oneself with.

andrum99
Posts: 927
Joined: Fri Jul 20, 2012 2:41 pm

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 10:45 am

RonR wrote:
Sun Nov 24, 2019 6:48 am
W. H. Heydt wrote:
Sun Nov 24, 2019 6:30 am
RossDv8 wrote:
Sun Nov 24, 2019 5:34 am
That said, there must be a downside or two surely?
Or even just a preference to putting everything on one device and not having to concern oneself with where exactly the *correct* /boot partition is.

When booting via an SD card, the *correct* /boot partition is always located on the SD card. There's nothing to concern oneself with.
The way I handled figuring out which is the correct boot partition is to just have a single boot partition on my system. The way I did this was to image both the SD card and a hard disk with the Buster image, then delete the root partition from the SD card, and the boot partition from the hard disk.

RonR
Posts: 642
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 7:11 pm

andrum99 wrote:
Sun Nov 24, 2019 10:45 am
RonR wrote:
Sun Nov 24, 2019 6:48 am
W. H. Heydt wrote:
Sun Nov 24, 2019 6:30 am

Or even just a preference to putting everything on one device and not having to concern oneself with where exactly the *correct* /boot partition is.

When booting via an SD card, the *correct* /boot partition is always located on the SD card. There's nothing to concern oneself with.
The way I handled figuring out which is the correct boot partition is to just have a single boot partition on my system. The way I did this was to image both the SD card and a hard disk with the Buster image, then delete the root partition from the SD card, and the boot partition from the hard disk.

There's *nothing* to figure out and you're making this way more complicated than it needs to be. When booting via an SD card, there's only *one* relevant boot partition - the one on the SD card. The boot partition on the hard disk is never mounted and never accessed, so it's totally irrelevant. Deleting the boot partition from the hard disk and the root partition from the SD card serves no purpose whatsoever but to create totally non-standard Raspbian devices.

dickon
Posts: 565
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 7:17 pm

RonR wrote:
Sun Nov 24, 2019 6:48 am
When booting via an SD card, the *correct* /boot partition is always located on the SD card. There's nothing to concern oneself with.
The one problem with that approach is that it isn't true in the multiboot environment. The /boot partition also has implications for /lib/modules/ (which will be hosted on your root filesystem, whereever that happens to be), because it contains the kernel.

This why network booting in the way I have described is the One True Way[0] to boot a Pi. Anything else is Wrong[0].


[0] It isn't, of course. At all. It's just a very, very reliable mechanism which solves a lot of problems, and is extremely flexible. I do recommend it -- or I wouldn't do it myself -- but it isn't ideal in all circumstances. Just most of them.

RonR
Posts: 642
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 7:32 pm

dickon wrote:
Sun Nov 24, 2019 7:17 pm
RonR wrote:
Sun Nov 24, 2019 6:48 am
When booting via an SD card, the *correct* /boot partition is always located on the SD card. There's nothing to concern oneself with.
The one problem with that approach is that it isn't true in the multiboot environment. The /boot partition also has implications for /lib/modules/ (which will be hosted on your root filesystem, whereever that happens to be), because it contains the kernel.

I tend to think in terms of using the one RPF supported OS - Raspbian, for which there's no problem with kernels/modules. I've never proposed trying to boot other OS's via an SD card. It will be interesting to see if the USB boot capable eeprom code supports multiboot environments when it arrives.

dickon
Posts: 565
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 7:43 pm

Unfortunately, you didn't limit your comment to that assertion, so I thought it worth clarifying.

andrum99
Posts: 927
Joined: Fri Jul 20, 2012 2:41 pm

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 8:56 pm

RonR wrote:
Sun Nov 24, 2019 7:11 pm
andrum99 wrote:
Sun Nov 24, 2019 10:45 am
RonR wrote:
Sun Nov 24, 2019 6:48 am


When booting via an SD card, the *correct* /boot partition is always located on the SD card. There's nothing to concern oneself with.
The way I handled figuring out which is the correct boot partition is to just have a single boot partition on my system. The way I did this was to image both the SD card and a hard disk with the Buster image, then delete the root partition from the SD card, and the boot partition from the hard disk.

There's *nothing* to figure out and you're making this way more complicated than it needs to be. When booting via an SD card, there's only *one* relevant boot partition - the one on the SD card. The boot partition on the hard disk is never mounted and never accessed, so it's totally irrelevant. Deleting the boot partition from the hard disk and the root partition from the SD card serves no purpose whatsoever but to create totally non-standard Raspbian devices.
Root on hard disk is non-standard anyway on the Pi 4B (at the moment). This way I know for certain that the boot partition I'm looking at is the right one, because there only is one. I find it easier that way, and you're less likely to find your system won't boot because the wrong boot partition has been upgraded.

RonR
Posts: 642
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 9:14 pm

andrum99 wrote:
Sun Nov 24, 2019 8:56 pm
RonR wrote:
Sun Nov 24, 2019 7:11 pm
andrum99 wrote:
Sun Nov 24, 2019 10:45 am

The way I handled figuring out which is the correct boot partition is to just have a single boot partition on my system. The way I did this was to image both the SD card and a hard disk with the Buster image, then delete the root partition from the SD card, and the boot partition from the hard disk.

There's *nothing* to figure out and you're making this way more complicated than it needs to be. When booting via an SD card, there's only *one* relevant boot partition - the one on the SD card. The boot partition on the hard disk is never mounted and never accessed, so it's totally irrelevant. Deleting the boot partition from the hard disk and the root partition from the SD card serves no purpose whatsoever but to create totally non-standard Raspbian devices.
Root on hard disk is non-standard anyway on the Pi 4B (at the moment). This way I know for certain that the boot partition I'm looking at is the right one, because there only is one. I find it easier that way, and you're less likely to find your system won't boot because the wrong boot partition has been upgraded.

How is the Root on the hard disk non-standard on the Pi 4B (at the moment)? It's 100% the same as it would be if you put it on an SD card unless you go to great lengths to make it non-standard.

You can't look at or upgrade the wrong boot partition because the 'wrong' boot partition isn't mounted and therefore isn't accessible. There's no reason the bring the 'wrong' boot partition into the picture.

W. H. Heydt
Posts: 11091
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: Raspberry Pi 4 usb boot?

Sun Nov 24, 2019 11:07 pm

RonR wrote:
Sun Nov 24, 2019 9:14 pm
You can't look at or upgrade the wrong boot partition because the 'wrong' boot partition isn't mounted and therefore isn't accessible. There's no reason the bring the 'wrong' boot partition into the picture.
I won't claim to speak for anyone else, but at present I want there to be a boot partition on the HDD/SSD so that *when* a Pi4B can be booted directly from said HDD/SSD, at most all I have to do is copy the *contents* of the SD card /boot to the HDD/SSD...a process that would require having both of them mounted.

Return to “General discussion”