Network Booting and Kernels

[gordanb]

I just managed to get my 3B to tftp boot, and as far as I can see the boot process always looks for, in order:

bootcode.bin
config.txt
start.elf
fixup.dat
config.txt
config.txt
cmdline.txt
bcm2710-rpi-3-b.dtb
config.txt
kernel7.img

I have a few questions regarding this:
1) Is there a way to specify which kernel to boot, other than kernel7.img? Is there a way to do this other than by symlinking the file?

2) What types of kernel images are supported for kernel7.img? All of my other aarch64 machines use UEFI and the vmlinuz images I use on that show up as, for example:
vmlinuz-4.9.196-1.el7.aarch64: MS-DOS executable, MZ for MS-DOS
The Raspbian images show up as:
kernel7.img: Linux kernel ARM boot executable zImage (little-endian)

Is there a way to use the former kernel directly on Raspberry Pi 3 / 4?

If not, is there a way to tftpboot Tianocore UEFI and have that tftp load the kernel? The goal is to be able to load the exact same kernel from both my UEFI aarch64 machines and the Pi 3 and Pi 4 machines.

[dickon]

You can change the kernel filename in config.txt, as for any other Pi and SD card booting. You're not going to get a vanilla aarch64 kernel to boot on the thing yet, as there are too many patches that haven't been upstreamed yet, I believe.

zImage is a precursor to vmlinuz, so if you're building your own from source, you should find you have one in arch/aarch64/boot/ (or similar).

[gordanb]

Thanks. It looks like the Pi 3 DTB is in 4.9.x upstream. Haven't checked for Pi 4 yet. I don't mind applying all the needed patches to a vanilla kernel as long as they don't break things for other targets.

I am hoping to use a kernel as packaged already for my other aarch64 machines. It would be very inconvenient to have to package both the zImage and vmlinuz forms.

[dickon]

The form is dependent on what your bootloaders will handle. U-Boot needs its own thing, the x86 BIOS needs something else, UEFI something else, and the Pi's GPU a vanilla zImage. I'd probably stick both versions into the same kernel package and let each system boot what it needs. It'll be the same kernel, just a different format.

[gordanb]

I'm more inclined to try use Tianocore as the 2nd stage bootloader and get that to take it from there.

[dickon]

Good luck with that. UEFI always seems to me to be a problem looking for a solution...

Let us know how you get on.

[gordanb]

The problem is lack of standards. Solution is having standards. UEFI is the only officially standardised way of booting aarch64.

I'll have a go and report back. I had the Pi 3 booting Tianocore off the memory card, but I'm not sure how well it will work with pure network booting.

[dickon]

Have you got a source for that claim? I can't see anything obvious with a reasonable google. Debian seem to support whatever works for whichever platform, which seems much more likely to me.

[gordanb]

The fact that you cannot find anything about a "standard" way of booting on aarch64 precisely proves my point that lack of standards compliance is indeed a huge problem.

RedHat/CentOS only support UEFI on aarch64. Debian apparently care less about standards, and more about supporting more devices.

Unfortunately, for those of us who care about using a multitude of devices to get things done instead of wasting a huge number of man-hours on nannying each different device into working in it's own special snowflake way is big and serious problem that detracts from genuinely productive work.

[dickon]

You asserted that 'UEFI is the only supported way of booting aarch64'. Now you're saying there isn't such a thing, and that this is a problem. I'd agree that this is a problem, but I'll disagree that UEFI is a good solution to this.

If Redhat and (by extension, as they're the same thing) Centos only support UEFI, then that's their lookout. Debian are clearly more flexible. The Pi's bootloader doesn't support it -- UEFI is a mess, and a nightmare to implement -- so you'll need to use something that it does support: that, I'm afraid, is zImage.

[gordanb]

You asserted that 'UEFI is the only supported way of booting aarch64'. Now you're saying there isn't such a thing, and that this is a problem.

Since you brought it up, the exact thing I said was: "UEFI is the only officially standardised way of booting aarch64." UEFI is a well defined standard. I am not aware of any other ARM booting way that is based on a similarly well defined standard.

If Redhat and (by extension, as they're the same thing) Centos only support UEFI, then that's their lookout. Debian are clearly more flexible. The Pi's bootloader doesn't support it -- UEFI is a mess, and a nightmare to implement -- so you'll need to use something that it does support: that, I'm afraid, is zImage.

Not really - there seems to be a perfectly well working Tianocore UEFI for Pi 3:
https://github.com/andreiw/RaspberryPiPkg

[dickon]

There certainly does. It isn't, however, supported.

[gordanb]

There certainly does. It isn't, however, supported.

What exactly does "supported" mean in this context?

[dickon]

You *might* get an answer out of the author of that port to anything you have a problem with, but certainly nothing official from the Foundation or (likely) even the developers of the upstream software. Booting a simple zImage, in 64b mode, OTOH, is supported by the Foundation, and if it doesn't work, a bug raised against the firmware will eventually be addressed (engineering time permitting).

zImage is a valid, and supported image format, supported by the Linux kernel and it's what the Foundation's bootloader boots. vmlinuz is another valid and supported image format, supported by the Linux kernel, but the Foundation's bootloader won't boot it. I'm having some difficulty seeing what your problem is, here. zImage, bootloader => supported. vmlinuz, bootloader => not supported. It isn't rocket science.

[gordanb]

So:
If there is an issue with booting a simple zImage, the Foundation may or may not do something about it.
If there is an issue with a different boot method, somebody else may or may not do something about it.

The kicker is that in my experience, community support is always far better in practically every way than any other support, often even paid commercial support.

My "issue" boils down to the fact that I cannot grab a generic distro installer or distro kernel and expect it to "just work", the way I can expect it to work with an UEFI based aarch64 dev board. Having a supported setup that works on one single modified distro is less valuable than having a setup that works on every standards compliant distro.

[dickon]

You have yet to supply me with an authoritative source that 'UEFI is the only officially standardised way of booting aarch64'. Until then, I'm afraid I simply reject that assertion. The kernel supports booting itself without it, and it doesn't require UEFI services. As far as the Linux kernel is concerned, UEFI is *not* the only way of booting it.

I cannot grab a generic distro installer or distro kernel and expect it to "just work", the way I can expect it to work with an UEFI based aarch64 dev board

Correct. The Pi is *not* a UEFI-based aarch64 dev board. So why you think you should be able to treat it as such is beyond me.

[gordanb]

You can reject it all you like, but the reasoning is very simple, and I explained it earlier.
1) UEFI is an industry standard
2) There is no other defined industry standard for ARM bootloaders. u-boot may be ubiquitous but it is nowhere nearly as standardised as UEFI.

So are you saying that you reject the assertion that UEFI is an industry standard, or are you saying that there are other well defined and widespread industry standards for this?

[dickon]

ARM-based systems usually boot using U-Boot. That's as much of an industry standard as anything else; certainly in embedded systems you won't find anything as ludicrous as UEFI.

I'm rejecting your assertion that UEFI is the only standardised way of booting an aarch64 kernel. That's simply wrong: lots of other platforms use U-Boot to do the same thing, without UEFI: I can name several SBCs which do this. UEFI is a standard that's most often used on x86 machines and the odd aarch64-based server. I have never met an ARM-based SBC which supports it. It's wild overkill. They all, with the exception of the Pi, use U-Boot.

[gordanb]

ARM-based systems usually boot using U-Boot. That's as much of an industry standard as anything else; certainly in embedded systems you won't find anything as ludicrous as UEFI.

And u-boot has implemented EFI for some years now.

I'm rejecting your assertion that UEFI is the only standardised way of booting an aarch64 kernel. That's simply wrong: lots of other platforms use U-Boot to do the same thing, without UEFI: I can name several SBCs which do this. UEFI is a standard that's most often used on x86 machines and the odd aarch64-based server. I have never met an ARM-based SBC which supports it. It's wild overkill. They all, with the exception of the Pi, use U-Boot.

u-boot has supported exposing EFI since at least 2016, and I accept that u-boot is a defacto standard. I have an armv5tel Dreamplug running u-boot in EFI mode and loading grub-efi, which takes over from there.

Even so, the point remains that whatever bootloader standards exist to make developers lives that little bit easier - pi doesn't ship with any of them out of the box.