USB MSD bootloader EEPROM second update 2020-06-03 - BETA

[timg236]

The updated BETA bootloader version 2020-06-03 is now available via APT. No major changes in functionality although the start.elf files required for USB MSD boot are now available via APT.

If we don't find any regressions to the existing boot modes then I think this is a potential candidate for promotion to 'stable' and eventually 'production/critical' because a lot of people are using this and it's working well enough for feature-freeze with beta reverting back to bleeding edge.

Raspberry Pi Imager recovery release https://github.com/raspberrypi/rpi-eepr ... eleases.md

Release notes https://github.com/raspberrypi/rpi-eepr ... e-notes.md

• Resolve: Unable to boot from USB MSD - Seagate 5Tb HDD backup drive #139
• Increase USB MSD timeout from 10 to 20 seconds.
• Max retries now default to zero because the default BOOT_ORDER includes restart (0xf). Therefore, each boot-mode is now tried once before moving to the next mode. The retries mechanism is largely redudant now that the loop/restart mode exists.
• If TFTP fails and network boot retries > 0 then wait 5 seconds before retrying to avoid overloading a misconfigured TFTP server.
• Map undefined boot-modes in BOOT_ORDER to SD (0x1) instead of stopping.
• Add missing pieeprom-2020-05-28

Please avoid posting off-topic or general OS issues on this thread, the purpose is to find any issues which would block USB boot being enabled by default.

[jsimmonstx]

Where else can we get the appropriate elf/dat files?

I have a relatively new (2 weeks old) pihole instance running on a pi3 that I want to move to a pi4 that's got the 06-03 eeprom update. I'm assuming that I can just manually copy the appropriate elf/dat files to the usb drive and be good to go, right?

[andrum99]

Where else can we get the appropriate elf/dat files?
...

Update in the normal way - they are now in the main firmware package:

Code: Select all

sudo apt update && sudo apt full-upgrade

[jowage]

nice! have been playing around with this all day, I just can't get my external USB 3.0 SSD to boot from USB 3.0 port

works: external SSD as slave on USB 3.0 port
works: boot from USB 3.0 thumb drive on USB 3.0 port
works: boot from external SSD on normal USB port

does not work: boot from external SSD on USB 3.0 port (cable is verified 3.0 also)
any ideas?

awesome job anyway, looking forward to stable release!

[W. H. Heydt]

Logging into the GUI with a user ID other than 'pi' still causes the loss of the task bar.

[wildestpixel]

Logging into the GUI with a user ID other than 'pi' still causes the loss of the task bar.

But that's probably (or definitely) not attributable to the bootloader right ?

[W. H. Heydt]

Logging into the GUI with a user ID other than 'pi' still causes the loss of the task bar.

But that's probably (or definitely) not attributable to the bootloader right ?

Probably...but it may be related to other changes involved with the beta. It doesn't happen on my Pis that aren't running the USB MSD software, so there is at least a correlation, even if not causation.

[HawaiianPi]

Logging into the GUI with a user ID other than 'pi' still causes the loss of the task bar.

I had some annoying flickering of the taskbar, but I didn't lose it altogether (non pi user login).

Are you on the 5.4 kernel and firmware from rpi-update?

Since the other files needed for USB-MSD boot are now in apt, I restored from a pre-beta backup and performed a normal update/full-upgrade, so now I'm back to the 4.19.118 kernel. Might want to give that a try (fixed a hard crash I was having with tvservice).

Code: Select all

$ uname -r && sudo rpi-eeprom-update
4.19.118-v8+
BCM2711 detected
Dedicated VL805 EEPROM detected
BOOTLOADER: up-to-date
CURRENT: Wed 03 Jun 2020 12:53:47 PM UTC (1591188827)
 LATEST: Thu 16 Apr 2020 05:11:26 PM UTC (1587057086)
 FW DIR: /lib/firmware/raspberrypi/bootloader/stable
VL805: up-to-date
CURRENT: 000138a1
 LATEST: 000137ad

[W. H. Heydt]

Logging into the GUI with a user ID other than 'pi' still causes the loss of the task bar.

I had some annoying flickering of the taskbar, but I didn't lose it altogether (non pi user login).

Are you on the 5.4 kernel and firmware from rpi-update?

Since the other files needed for USB-MSD boot are now in apt, I restored from a pre-beta backup and performed a normal update/full-upgrade, so now I'm back to the 4.19.118 kernel. Might want to give that a try (fixed a hard crash I was having with tvservice).

Code: Select all

$ uname -r && sudo rpi-eeprom-update
4.19.118-v8+
BCM2711 detected
Dedicated VL805 EEPROM detected
BOOTLOADER: up-to-date
CURRENT: Wed 03 Jun 2020 12:53:47 PM UTC (1591188827)
 LATEST: Thu 16 Apr 2020 05:11:26 PM UTC (1587057086)
 FW DIR: /lib/firmware/raspberrypi/bootloader/stable
VL805: up-to-date
CURRENT: 000138a1
 LATEST: 000137ad

The Pi having the problem with non-pi logins is on 5.4 and rpi-update. I'm not so much trying to get it to work as posting notice that the beta software is showing the problem so that when the newer software goes live, it won't have the problem.

[PeterO]

I thought I would let the dust settle a bit before trying the Pi4 USB boot eerpom code as I've never had any trouble before with my Seagate USB drives,
but now I can't find the instructions on how to get this enabled and working.....

PeterO

[LTolledo]

I thought I would let the dust settle a bit before trying the Pi4 USB boot eerpom code as I've never had any trouble before with my Seagate USB drives,

I'd let the dust settle a bit....
me no hurry
system is up and running fine as it is...
but might try again if I feel like doing so....

just to confirm.... I need to do this?

Code: Select all

sudo rpi-eeprom-update

to enable MSD boot?

[HawaiianPi]

just to confirm.... I need to do this?

Code: Select all

sudo rpi-eeprom-update

to enable MSD boot?

No, you download the beta EEPROM recovery and make a recovery SD card to flash the boot-loader. Then it's recommended to start with a fresh Raspberry Pi OS on SD card and apt update/apt full-upgrade it (but I just did that with my current Raspbian Buster), after which you clone the SD card to a USB drive (SD Card Copier), and it should boot.

It's pretty simple now that you don't need to rpi-update or copy special *.elf & *.dat files any longer.

The command by itself just displays the current EEPROM information.

Code: Select all

$ sudo rpi-eeprom-update
BCM2711 detected
Dedicated VL805 EEPROM detected
BOOTLOADER: up-to-date
CURRENT: Wed 03 Jun 2020 12:53:47 PM UTC (1591188827)
 LATEST: Thu 16 Apr 2020 05:11:26 PM UTC (1587057086)
 FW DIR: /lib/firmware/raspberrypi/bootloader/stable
VL805: up-to-date
CURRENT: 000138a1
 LATEST: 000137ad

With additional options it will flash the EEPROM, or more accurately, it will place the needed files on the boot partition, and the flash will happen on the next boot. But the recovery SD card is probably simpler for most people (I flash mine manually because I alter a few config settings).

If you stick with the release kernel it's also quite stable, at least it has been for me (I did have some trouble with the rpi-update 5.4 kernel).

The "blue" Eluteng USB 3.0 to SATA-III adapter cable works with both UAS and TRIM (no quirks required).

I say, go for it! Test it first with a spare SD card and USB drive to get a feel for it if you are worried.

[PeterO]

Thanks HawaiianPi.

I grabbed an old 8Gb sdcard, and used that for the eerpom upgrader and put fresh PIOS64 image on a USB-HDD.

All worked fine first time. Now booting PIOS64 from a 1TB Seagate Backup Plus Portable Drive (USB3) that is a few years old.

PeterO

[hwissing]

Hello,
with the acutal release one can get a working MSB-system we have been waiting for such a long time. Its a comprehensive release with machting boot software. All tested USB drives - SD-card reader, LEXAR S47 32GB, SanDisk UltraFit 32 GB - worked fine with that release. For booting from the USB-3 ports with Hard Drives and SSDs one prerequisite is an ASMedia Technology chipset based USB-3 / SATA bride. This should be outlined in accompanying documentation.
All bridges based on Jmicron chipsets, which previously worked without complains with Pi 2, Pi 3, Pi 3+ and Windows PCs were detected, but failed somewhere during the boot process. That costs me a lot of time and a lot of money for two new bridges reported to work. There seems to be a workaround with "Quirk", wich may or may not help. But such a workaround cannot be the standard. All these bridges failing at USB-3 port work at USB-2 port of Pi4 and passed SD-Speed-Test. So I think this is rather an issue of current USB-3 driver software than chipsets, which should be solved in the near future.
In my setting not even the USB-3 hub of my iiyama monitor is detected at Pi4s USB-3 port, but at its USB-2. There it is reported as an additional "Genesys Logic, Inc. 4-port hub" with the same ID as the Pis.

Regards
hwissing

[hippy]

Logging into the GUI with a user ID other than 'pi' still causes the loss of the task bar.

But that's probably (or definitely) not attributable to the bootloader right ?

Probably...but it may be related to other changes involved with the beta. It doesn't happen on my Pis that aren't running the USB MSD software, so there is at least a correlation, even if not causation.

I noticed some odd behaviour with the task bar on whatever desktop Twister OS (Raspberry PI OS / Raspbian based) uses, appearing then disappearing before appearing again, which correlated with moving to boot that from MSD after upgrading to 5.4. That I believe is using the "pi" user. May be related. Maybe not.

[Kendek]

... For booting from the USB-3 ports with Hard Drives and SSDs one prerequisite is an ASMedia Technology chipset based USB-3 / SATA bride. This should be outlined in accompanying documentation.
...

But that's not the case. The only problem is the broken UAS with some chipsets or firmwares. Which could block the OS to work properly, after the Linux kernel is loaded and switched to UAS. This is off-topic here.

[HawaiianPi]

But that's not the case. The only problem is the broken UAS with some chipsets or firmwares. Which could block the OS to work properly, after the Linux kernel is loaded and switched to UAS. This is off-topic here.

Yup, the boot-loader works.The UAS problem is in Linux itself.

Hello,
with the acutal release one can get a working MSB-system we have been waiting for such a long time. Its a comprehensive release with machting boot software. All tested USB drives - SD-card reader, LEXAR S47 32GB, SanDisk UltraFit 32 GB - worked fine with that release.

There is no "actual release" yet. USB-MSD booting is still in beta.

For booting from the USB-3 ports with Hard Drives and SSDs one prerequisite is an ASMedia Technology chipset based USB-3 / SATA bride. This should be outlined in accompanying documentation.

That is not the only controller chip that works, and there is no official documentation yet, because it's still in beta. Also, not all ASMedia based adapters are fully functional, because the firmware in the adapter itself is a factor. There is no way every controller chip+firmware combination can be tested, and support for that is upstream in Debian anyway.

All bridges based on Jmicron chipsets, which previously worked without complains with Pi 2, Pi 3, Pi 3+ and Windows PCs were detected, but failed somewhere during the boot process. That costs me a lot of time and a lot of money for two new bridges reported to work.

This is a problem with UAS in Linux with JMicron controllers, which the Raspberry Pi Foundation has no control over (it's upstream). And the reason they worked on the Pi3 is that the UAS driver wasn't used. Also, they didn't work property even on the Pi3, because TRIM didn't work.

There seems to be a workaround with "Quirk", wich may or may not help. But such a workaround cannot be the standard.

If you want to use a JMicron controller, quirks are your only option.

All these bridges failing at USB-3 port work at USB-2 port of Pi4 and passed SD-Speed-Test. So I think this is rather an issue of current USB-3 driver software than chipsets, which should be solved in the near future.

Again, this is an upstream problem with Debian, not the boot-loader, and not anything the Raspberry Pi Foundation is doing. It has nothing to do with the beta boot-loader, or even Raspbian.

In my setting not even the USB-3 hub of my iiyama monitor is detected at Pi4s USB-3 port, but at its USB-2. There it is reported as an additional "Genesys Logic, Inc. 4-port hub" with the same ID as the Pis.

USB 3.0 hubs are another issue altogether, and yes, they can be problematic, but that has nothing to do with the boot-loader or Raspbian, it's a problem with the hub (some only work correctly with Windows or Mac OS).

[hwissing]

Don't split hairs!
We all know its still beta, but its available via APT, rather an official path and contemporaneous available "*.elf" files are matching. The question was, can it become "critical".

by timg236 » Fri Jun 05, 2020 3:06 pm
The updated BETA bootloader version 2020-06-03 is now available via APT. No major changes in functionality although the start.elf files required for USB MSD boot are now available via APT.

If we don't find any regressions to the existing boot modes then I think this is a potential candidate for promotion to 'stable' and eventually 'production/critical' because a lot of people are using this and it's working well enough for feature-freeze with beta reverting back to bleeding edge.

If it is, thousands of users will try to built a MSB system. After the hype on MSB with Pi 3 and 3+ many people set up such systems to get ride of SD-card trouble and it also works with Pi2, Pi B+ and Pi zero via bootcode.bin. Nobody understood why Pi4 came without and took a year for subsequent delivery. My humble advice, to achieve customer satisfaction and with respect to their spare-time / lifetime is, give them a clear note, that there are issues with USB-3 - SATA bridges.
This should be outlined in accompanying documentation.

[PeterO]

Don't split hairs!
We all know its still beta, but its available via APT, rather an official path and contemporaneous available "*.elf" files are matching. The question was, can it become "critical".

by timg236 » Fri Jun 05, 2020 3:06 pm
The updated BETA bootloader version 2020-06-03 is now available via APT. No major changes in functionality although the start.elf files required for USB MSD boot are now available via APT.

If we don't find any regressions to the existing boot modes then I think this is a potential candidate for promotion to 'stable' and eventually 'production/critical' because a lot of people are using this and it's working well enough for feature-freeze with beta reverting back to bleeding edge.

If it is, thousands of users will try to built a MSB system. After the hype on MSB with Pi 3 and 3+ many people set up such systems to get ride of SD-card trouble and it also works with Pi2, Pi B+ and Pi zero via bootcode.bin. Nobody understood why Pi4 came without and took a year for subsequent delivery. My humble advice, to achieve customer satisfaction and with respect to their spare-time / lifetime is, give them a clear note, that there are issues with USB-3 - SATA bridges.
This should be outlined in accompanying documentation.

Nobody understood why Pi4 came without and took a year for subsequent delivery.

You weren't paying attention then. It was explained that it would take a while becasue
1) Netboot was higher priority for educational users.
2) It required complete rewrite from earlier versions due to different hardware.
3) It was a non-trivial task.

Also lets not forget that the earlier than expected availability of the Pi4 SoC shortened the time available for these software developments. Maybe you would have preferred it if the Pi4 was only now being released ?

Telling the RPT guys how to do their jobs is generally frowned upon. Believe it or not they know far more about their customer base than you think you do !
PeterO

[knute]

If you want to use a JMicron controller, quirks are your only option.

I didn't know there was an issue with JMicron controllers so I just used it and it appears to work just fine?

Bus 002 Device 002: ID 152d:0583 JMicron Technology Corp. / JMicron USA Technology Corp.

[ 3.145447] usb 2-2: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 3.186415] usb 2-2: New USB device found, idVendor=152d, idProduct=0583, bcdDevice= 2.07
[ 3.197285] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3.207097] usb 2-2: Product: External
[ 3.213496] usb 2-2: Manufacturer: JMicron
[ 3.220238] usb 2-2: SerialNumber: DD564198855B9
[ 3.249448] scsi host0: uas
[ 3.256292] scsi 0:0:0:0: Direct-Access JMicron Tech 0207 PQ: 0 ANSI: 6
[ 3.268805] sd 0:0:0:0: [sda] Spinning up disk...

There are no usb-storage.quirks in cmdline.txt. Could it be somewhere else? Does the 5.44 kernel make a difference?

[HawaiianPi]

Don't split hairs!
We all know its still beta, but its available via APT...

Only if you manually switch to FIRMWARE_RELEASE_STATUS="beta" (which requires deliberate action by the user, with all associated risks).

Once again, the beta USB-MSD boot-loader will not be installed by a normal apt update/upgrade. You MUST make a deliberate change to the beta system for it to be installed. No one will install this by accident.

If it is, thousands of users will try to built a MSB system. After the hype on MSB with Pi 3 and 3+ many people set up such systems to get ride of SD-card trouble

And many of those people had trouble with the Pi3 USB boot as well. Some will tell you that finding a compatible USB-SATA adapter was akin to searching for the Holy Grail. You seem to be implying that USB booting on the Pi3 was trouble-free, which is not true.

My humble advice, to achieve customer satisfaction and with respect to their spare-time / lifetime is, give them a clear note, that there are issues with USB-3 - SATA bridges.

This is already well known to anyone who has done due diligence research. The documentation for Pi3 USB boot said it was not compatible with all USB-MSD devices, so I'm sure any new documentation for the Pi4 will say the same, when it is out of beta and officially supported.

If you are messing with beta software you should expect trouble, and be knowledgeable enough to fix it (which includes the aforementioned research). From my experience with USB booting the 3B/3B+ models, and SD+USB booting my 4B, I already had a pretty good idea which USB-SATA adapters would work, and which ones to expect trouble from. I had not yet messed with Quirks, but I knew where to get the info. I also made regular backups, and knew beforehand how to revert the boot-loader if something broke.

Beta-testing is not for new or inexperienced users!

I didn't know there was an issue with JMicron controllers so I just used it and it appears to work just fine?

Not all of them. My statement was a bit of a generalization (some of the most common ones don't work without quirks).

[RamaSpaceShip]

Nobody understood why Pi4 came without and took a year for subsequent delivery.

You didn't understand. Some like me did understand.
We didn't want to wait a year for a feature that can be worked around for some time., and where some Pi4features (USB3, separate Ethernet, and 2GB/4GB Ram) were solving several issues with the PI3.

I personnally thanks the RPF to have started to deliver the PI4 one year ago.

[donbrew]

My GenesysLogic 4 port powered USB 3 hub acts oddly; on initial boot the Pi does not blink the green light if the hub is plugged in; unplug it then replug boots right up dmesg reports a lot of disconnect/reconnect activity. On reboot everything goes smoothly. On reboot dmesg report a couple of "Timeout waiting for hardware cmd interrupt." then a bunch of sdhci register dump info.

It does not matter if the HD is plugged into the hub or the Pi. The Pi will not boot until the hub is unplugged. Then replug with HD on the hub or the Pi.

I am using a USB 2 adapter for now; I am hoping a 3.0 adapter will act differently.

Be sure to cut the red wire in the USB3 cable between the hub and the Pi. The power goes both ways and causes under voltage problems. After you cut it you cannot self power anything from it, you must use the power adapter. I found that my 640 GB laptop drive plugged into the hub before cutting the red wire caused low voltage and no/erratic boot.

[W. H. Heydt]

Be sure to cut the red wire in the USB3 cable between the hub and the Pi. The power goes both ways and causes under voltage problems. After you cut it you cannot self power anything from it, you must use the power adapter. I found that my 640 GB laptop drive plugged into the hub before cutting the red wire caused low voltage and no/erratic boot.

I have some USB 2 hubs that don't back power, without having to modify cables. At such time as I find that I need a USB 3 hub, I will look for a properly made model of those as well.

[tofflock]

An "interesting" side effect of releasing the SD socket for general use.

Now it's possible to have both boot & root NOT on an SD card, that frees up the SD card slot for other uses.

Buster will auto-mount an SD card, when it's inserted, by default - providing it has one or more acceptable file systems configured.
However, the eject command, issued from the command line, or from the eject button, barfs because it doesn't appear to like the device

Code: Select all

pi@raspberrypi:~ $ sudo eject /dev/mmcblk0
not an sg device, or old sg driver
eject: unable to eject, last error: Invalid argument

I could imagine that the same will also happen with a Pi4 that's been net-booted. (I have no way of testing a net-booted Pi4.)

The eject button (gui) appears to unmount any file systems on the SD card, but the eject command does not umount the filesystems.

PeterF

PS Posted here because this problem is only now (properly) observable with the new boot modes. Happy to post to a new thread if that's more appropriate.