Strange boot folder transformation in ubuntu 18.10!?

[gilius]

I wrote the Ubuntu 18.10 preinstall to Micro-SD for the Raspberry Pi 3 (mine is B+). Initially the /boot partition mounted as /mnt/boot included the following files:
*firmware folder
*raspi2 kernel? (wasn't in format .img or it might have been)
*There was a config raspi2.x.x.x.txt
*Inside the firmware folder was config.txt with a reference to uboot.bin
*There was a kernel8.img I added that I was trying to boot, but instead it booted the raspi2

Since I booted it a couple of times the whole structure of /boot in terms of the first partition being mounted to /mnt/boot has drastically changed:
*firmware folder has gone
*raspi2 references don't appear to be there anymore
*a new kernel8.img has been created in place of the one I compiled?
*The config.txt is now located under /boot and references kernel8.bin instead

What changed!?

I tried to overwrite my kernel8.img again, but now nothing boots - just rainbow screen:
sudo cp arch/arm64/boot/Image /mnt/boot/kernel8.img
sudo make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- INSTALL_MOD_PATH=/mnt modules_install

[gilius]

1.png (58.31 KiB) Viewed 4768 times

2.png (47.26 KiB) Viewed 4768 times

It seems one is the file system /boot vs. the boot loader partition /boot?

[gilius]

I think I figured out what the problem is... partition 1 was mounted as /mnt/boot before partition 2 was mounted as /mnt
Solution: mount partition 2 first followed by partition 1 so that the right boot folder gets mounted?
Still don't understand understand why there's 2 boot folders and why kernel8 is under partition 1 and raspi2 is under partition 2 - but should hopefully start to fill in more gaps through all this trial and error + reverse engineering in what feels like a horror movie. There's no proper information anywhere... just scraps I have to try to piece together... dead sea scrolls all over again!

[feelslikeautumn]

There's no proper information anywhere... just scraps I have to try to piece together... dead sea scrolls all over again!

Once again this is simply not true. /boot/firmware is documented on the wiki https://wiki.ubuntu.com/ARM/RaspberryPi ... i_packages

Arm64 is slightly different in that uboot.bin is called kernel8.img. I tried to argue that this was a stupid choice, but got ignored by the Ubuntu Devs. You can of course rename it yourself (with a corresponding change to config.txt).

[feelslikeautumn]

Due to how the raspi3 images were built there are files in /boot/firmware before it is mounted. This is a bug (one that I also pointed out, but again ignored).

[gilius]

This is a contradiction:

The GPU firmware partition is mounted at /boot/firmware. The files /boot/firmware/config.txt and /boot/firmware/cmdline.txt contain the system configuration and kernel command line options respectively.

Due to how the raspi3 images were built there are files in /boot/firmware before it is mounted. This is a bug (one that I also pointed out, but again ignored).

[feelslikeautumn]

Not at all. A directory can have different contents depending if/how it is mounted. That is what your thread is about.

[gilius]

Not at all. A directory can have different contents depending if/how it is mounted. That is what your thread is about.

OK, so does that filesystem partition 2 boot folder (including the firmware subfolder) do anything since the real kernel is meant to reside in boot partition 1?

[feelslikeautumn]

/boot is used to store the various kernels. It is not used directly by the pi firmware if that is what you mean. It would be used if you had grub2 installed (note typically a pi kernel is missing the necessary module (efi stub) to allow grub2 booting).

[gilius]

/boot is used to store the various kernels. It is not used directly by the pi firmware if that is what you mean. It would be used if you had grub2 installed (note typically a pi kernel is missing the necessary module (efi stub) to allow grub2 booting).

Are you saying that Kernels are stored under both partition 1 and partition 2 boot folders? Would that be simultaneously or alternatively?

[feelslikeautumn]

There is no boot folder on partition 1. In Raspbian p1 is mounted at /boot. In Ubuntu/Debian p1 is mounted at /boot/firmware.

All kernels are installed to /boot. Ubuntu copies the latest kernel from /boot to /boot/firmware (so p1) so that the GPU firmware can read it. It wouldn't have to do this if it used grub2 or a decent u-boot script.

(If using grub2 p1 would be mounted at /boot/efi)

[gilius]

feelslikeautumn, but the whole reason this topic exists is that there are two separate boot folders under partition 1 and partition 2, respectively. If you mount them both to /mnt/piboot and /mnt/piroot you will see. This is the case for Ubuntu RPi3 preinstalled. Partition 2 is usually EXT4 and Partition 1 is usually FAT32. Both have boot folders (compare screenshots above). This much I know despite the lack of much genuine help from the Linux community at large.

[feelslikeautumn]

If you don't like the answers you are getting, then maybe you should reflect on how you are asking the questions. You've already received a temporary ban for trolling on here. You've had a whole thread deleted (presumably due to a rude comment? at me maybe?).

I don't know how many more ways I can say the same thing. In Ubuntu there is no linux /boot folder on partition 1. Partition 1 contains the pi firmware. In Ubuntu p1 is labelled system-boot, which I agree adds a bit of confusion, but the 'system' has been added to distinguish it from the linux /boot directory which is on partition 2.

Good luck with your problem.

p.s. it does help if you say 'thank you' occasionally.

[code_exec]

Ubuntu images built using live-build use linux-raspi2 and flash-kernel. linux-raspi2 is an Ubuntu-style kernel with patches from the Raspbian kernel source so that it works better on the Raspberry Pi. The generic kernel can also be used with u-boot and GRUB2 although this kernel includes few optimizations for the Pi and you may encounter issues when using it.

flash-kernel is a script for ARM boards which detects the board in use and copies the kernel and initrd to the location specified by the database. On the Raspberry Pi, this is located at /boot/firmware. This is where the boot partition is located which includes the bootloader files mandatory to boot the Pi.

To sum it up, /boot is just a regular folder on the ext4 partition, as is /boot/firmware. /etc/fstab tells the kernel where to mount partitions. In Ubuntu's case, the boot partition is mounted at /boot/firmware. You can modify the fstab if you want so that the kernel mounts the boot partition at /boot instead, but then flash-kernel will fail when the kernel is updated.

[feelslikeautumn]

You can modify the fstab if you want so that the kernel mounts the boot partition at /boot instead, but then flash-kernel will fail when the kernel is updated.

In Ubuntu/Debian you can't do this. Or more precisely if you do the kernel packages will fail to install even before flash-kernel is triggered.

Personally I prefer referring to partition 1 as the firmware partition or system-boot partition. Boot partition is a raspbian thing and just adds to the confusion that we've already seen here.

[code_exec]

You can modify the fstab if you want so that the kernel mounts the boot partition at /boot instead, but then flash-kernel will fail when the kernel is updated.

In Ubuntu/Debian you can't do this. Or more precisely if you do the kernel packages will fail to install even before flash-kernel is triggered.

Personally I prefer referring to partition 1 as the firmware partition. Boot partition is a raspbian thing and just adds to the confusion that we've already seen here.

I'm referring to doing this on existing installations where the kernel is already installed and not in the live-build script.

[gilius]

If you don't like the answers you are getting, then maybe you should reflect on how you are asking the questions. You've already received a temporary ban for trolling on here. You've had a whole thread deleted (presumably due to a rude comment? at me maybe?).

No reason was given, but seeing how easy it was to achieve the challenge I set, I'm surprised nobody else came forward with a solution. So instead of losing money I lost about 5-6 hours of free time, cross-referencing 10 guides on how to replace the kernel - revealing that code_exec had indeed missed out some commands as I suspected, in his KVM tutorial.

I don't know how many more ways I can say the same thing. In Ubuntu there is no linux /boot folder on partition 1. Partition 1 contains the pi firmware. In Ubuntu p1 is labelled system-boot, which I agree adds a bit of confusion, but the 'system' has been added to distinguish it from the linux /boot directory which is on partition 2.

Good luck with your problem.

p.s. it does help if you say 'thank you' occasionally.

My bad - I got confused because code_exec suggested we mount partition 1 as /boot instead of sakaki's superior choice of piboot and piroot. So we have:
P1: system-boot AKA "firmware"
vs.
P2: /boot + /boot/firmware

You still haven't explained if and how kernels exist on both partitions, so you aren't yet deserved of a "thank you" IMO. You have yet to contribute any worthwhile information in response to my posts since I joined the forum just under a year ago; only interested in "splitting hairs", being deliberately vague and causing deliberate confusion so to speak (all forms of abuse in their own right). I'm not surprised the Ubuntu devs are ignoring you, either - not because you are to blame - but because the entire Linux world seems to be plagued with communication problems in general.

[gilius]

Ubuntu images built using live-build use linux-raspi2 and flash-kernel. linux-raspi2 is an Ubuntu-style kernel with patches from the Raspbian kernel source so that it works better on the Raspberry Pi. The generic kernel can also be used with u-boot and GRUB2 although this kernel includes few optimizations for the Pi and you may encounter issues when using it.

I don't think is accurate. Ubuntu "live" - being ISOs(?) - do not have the raspi2 kernel. Only the preinstalled img files have that raspi2. Anyhow, what is the difference between "flash-kernel", u-boot and GRUB? This guide claims flash-kernel vs. u-boot/EFI/GRUB:
https://wiki.ubuntu.com/ARM/RaspberryPi
But I don't think there's any EFI used in PI distros since nobody is utilizing this:
https://github.com/andreiw/RaspberryPiPkg
?
Which partitions/files/folders are included in a "flash-kernel" setup vs. the alternative?

flash-kernel is a script for ARM boards which detects the board in use and copies the kernel and initrd to the location specified by the database. On the Raspberry Pi, this is located at /boot/firmware. This is where the boot partition is located which includes the bootloader files mandatory to boot the Pi.

Too vague. What is the exact source and destination of the kernel copy process or where is the database that specifies this?

To sum it up, /boot is just a regular folder on the ext4 partition, as is /boot/firmware. /etc/fstab tells the kernel where to mount partitions. In Ubuntu's case, the boot partition is mounted at /boot/firmware. You can modify the fstab if you want so that the kernel mounts the boot partition at /boot instead, but then flash-kernel will fail when the kernel is updated.

Is the mount process the same as the copy process you spoke of in the previous paragraph? But when I checked /boot and /boot/firmware (see screenshots above) it wasn't mounted as the system wasn't live, so why did I find references to kernels on both partitions?

[gilius]

Personally I prefer referring to partition 1 as the firmware partition or system-boot partition. Boot partition is a raspbian thing and just adds to the confusion that we've already seen here.

All the distros I've seen have a boot partition in the general sense - not exclusive to Raspberry or ARM Linux. Usually it's like this:
P1: boot partition (EFI or GRUB)
P2: root partition (EXT4)
P3: swap

Not sure if Pi can use a swap partition.

Am not too familiar with GRUB/2, but I think it just had a folder called /boot/grub or maybe /firmware that contained some boot files. Were they .grub files? I know it's equivalent to Windows Legacy boot.
I know that EFI partitions are just a collection of .EFI files instead in a /Boot/efi folder. On Windows its under \EFI\Boot or \EFI\Microsoft\Boot

Again, I'm interested in what form a uboot-(efi)-grub takes compared to "flash-kernel". All this stuff is rumoured to be documented somewhere?

[gilius]

I'm referring to doing this on existing installations where the kernel is already installed and not in the live-build script.

Is the kernel actually "installed" or is it just read into memory from the IMG file placed on partition 1 - not to mention the copying of modules and DTBs?

[code_exec]

Ubuntu images built using live-build use linux-raspi2 and flash-kernel. linux-raspi2 is an Ubuntu-style kernel with patches from the Raspbian kernel source so that it works better on the Raspberry Pi. The generic kernel can also be used with u-boot and GRUB2 although this kernel includes few optimizations for the Pi and you may encounter issues when using it.

I don't think is accurate. Ubuntu "live" - being ISOs(?) - do not have the raspi2 kernel. Only the preinstalled img files have that raspi2.

live-build can also be used to create Raspberry Pi Ubuntu images which come in img format.

[code_exec]

If you don't like the answers you are getting, then maybe you should reflect on how you are asking the questions. You've already received a temporary ban for trolling on here. You've had a whole thread deleted (presumably due to a rude comment? at me maybe?).

No reason was given, but seeing how easy it was to achieve the challenge I set, I'm surprised nobody else came forward with a solution. So instead of losing money I lost about 5-6 hours of free time, cross-references 10 guides on how to replace the kernel - revealing that code_exec had indeed missed out some commands as I suspected, in his KVM tutorial.

I don't know how many more ways I can say the same thing. In Ubuntu there is no linux /boot folder on partition 1. Partition 1 contains the pi firmware. In Ubuntu p1 is labelled system-boot, which I agree adds a bit of confusion, but the 'system' has been added to distinguish it from the linux /boot directory which is on partition 2.

Good luck with your problem.

p.s. it does help if you say 'thank you' occasionally.

My bad - I got confused because code_exec suggested we mount partition 1 as /boot instead of sakaki's superior choice of piboot and piroot. So we have:
P1: system-boot AKA "firmware"
vs.
P2: /boot + /boot/firmware

You still haven't explained if and how kernels exist on both partitions, so you aren't yet deserved of a "thank you" IMO. You have yet to contribute any worthwhile information in response to my posts since I joined the forum just under a year ago; only interested in "splitting hairs", being deliberately vague and causing deliberate confusion so to speak (all forms of abuse in their own right). I'm not surprised the Ubuntu devs are ignoring you, either - not because you are to blame - but because the entire Linux world seems to be plagued with communication problems in general.

No offense, but if you think the entire Linux community uses technical wording that you cannot understand, why exactly are you using Ubuntu Server which is not a beginner's Linux distro?

There is only one boot partition, not two, and the boot partition is mounted at /boot/firmware which contains the bootloader files, the kernel and initrd, and config.txt and cmdline.txt files. flash-kernel is a script for ARM boards that runs upon every kernel update and copies the kernel and initrd to the location defined in the database for your board. If your board is missing from the database, flash-kernel will fail with the 'Unsupported platform' error. Each time a new Raspberry Pi comes out, linux-raspi2 and linux-firmware-raspi2 must be updated to support the hardware of the new board and flash-kernel must be updated to add the new board to its database. The Pi 3B+ and CM3/3+ have been added to the flash-kernel database in an update, but the Pi 3A+ is yet to be added.

[gilius]

No offense, but if you think the entire Linux community uses technical wording that you cannot understand, why exactly are you using Ubuntu Server which is not a beginner's Linux distro?

I didn't say it was due to technical wording, but due to communication problems.

There is only one boot partition, not two, and the boot partition is mounted at /boot/firmware which contains the bootloader files, the kernel and initrd, and config.txt and cmdline.txt files. flash-kernel is a script for ARM boards that runs upon every kernel update and copies the kernel and initrd to the location defined in the database for your board. If your board is missing from the database, flash-kernel will fail with the 'Unsupported platform' error. Each time a new Raspberry Pi comes out, linux-raspi2 and linux-firmware-raspi2 must be updated to support the hardware of the new board and flash-kernel must be updated to add the new board to its database. The Pi 3B+ and CM3/3+ have been added to the flash-kernel database in an update, but the Pi 3A+ is yet to be added.

Putting words into people's mouths is another type of communication problem. I did not say there were two boot partitions. When I mount partition 2 as the root filesystem it already has a /boot/firmware folder, so why would the firmware/bootloader, i.e. partition 1 then get mounted there? I can certainly check a live system to see if there's some kind of amalgamation that occurs between the system-boot and the /boot/firmware. Regarding the flash-kernel script you've repeated yourself for the most part, but without addressing my previous questions, i.e. taking on board new information, which would have led to further clarity. Again, this is another type of communication problem known as a "circular conversation". That then means I also have to repeat myself, and we never end up progressing:
A) This train ticket is only for London Zones 1-6
B) But I was told I could use it to go to Scotland!
A) Sorry, you can only use it within London. It says on the ticket, and you only paid £6 for it.
B) No, I'm sure it can be used for Scotland because that's what I was told.

[Paeryn]

When I mount partition 2 as the root filesystem it already has a /boot/firmware folder, so why would the firmware/bootloader, i.e. partition 1 then get mounted there?

Do you understand how Linux mounts work? Devices/partitions are mounted onto directories, the root file system has a directory /boot/firmware which can (and from what feelslikeautum has posted does) have other files and directories in it - it is an ordinary directory. When the first partition is mounted it needs a directory to be mounted on, which is /boot/firmware, from that point on (until it is unmounted) whenever you access /boot/firmware the kernel will be silently divert you to the base directory on partition 1. The actual directory on partition 2 will not be accessable through /boot/firmware until the mount is unmounted.

The rule is that the directory where you want a device to appear has to already exist on the target filesystem, if /boot/firmware didn't exist on partition 2 then partition 1 couldn't be mounted to /boot/firmware. Usually empty directories are used since the contents will be unaccessible for the duration of the mount (well, there are ways but not via that particular path) but it isn't mandatory.

[gilius]

When I mount partition 2 as the root filesystem it already has a /boot/firmware folder, so why would the firmware/bootloader, i.e. partition 1 then get mounted there?

Do you understand how Linux mounts work? Devices/partitions are mounted onto directories, the root file system has a directory /boot/firmware which can (and from what feelslikeautum has posted does) have other files and directories in it - it is an ordinary directory. When the first partition is mounted it needs a directory to be mounted on, which is /boot/firmware, from that point on (until it is unmounted) whenever you access /boot/firmware the kernel will be silently divert you to the base directory on partition 1. The actual directory on partition 2 will not be accessable through /boot/firmware until the mount is unmounted.

The rule is that the directory where you want a device to appear has to already exist on the target filesystem, if /boot/firmware didn't exist on partition 2 then partition 1 couldn't be mounted to /boot/firmware. Usually empty directories are used since the contents will be unaccessible for the duration of the mount (well, there are ways but not via that particular path) but it isn't mandatory.

Regarding your first sentence, I certainly understood Linux mounts based on your last sentence, i.e. devices are usually mounted to empty directories, hence the confusion. The fact that the /boot folder on partition 2 is not empty to start with should have been emphasized early on in this discussion - because that begs the question as to why it's pre-populated with files and other sub-folders if it's sole purpose is to be replaced by another mount point.

I am going to compare some different Ubuntu installations to see if I can figure out the remaining mysteries of how the kernel works with various bootloaders and filesystems.