User avatar
jojopi
Posts: 3085
Joined: Tue Oct 11, 2011 8:38 pm

Re: Raspbian with Read-only Root

Sat Dec 23, 2017 10:04 am

fromJPN wrote:
Fri Dec 22, 2017 9:32 am
About "00400041:" on the "DiskImageFile" read from a SD card...
That is byte 0x41 of the first sector of partition 1 (8192*512 + 0x41). dosfsck uses the "dirty bit" here to indicate whether the FAT32 file system needs to be checked.

It should be fine to mount the /boot filesystem readonly using "ro" in fstab, rather than going all the way to "noauto". You can temporarily switch a filesystem back to read-write using:

Code: Select all

sudo mount -o remount,rw /boot
I think it is better to have /boot mounted readonly rather than not at all. If you make / writeable to perform an upgrade, but forget about /boot, then you will get an error if a package tries to write to /boot. However, if / is writeable and /boot not mounted at all, any files written to /boot will go into the wrong filesystem and be ignored.
About "02E00578:" on the "DiskImageFile" read from a SD card...
That is byte 0x178 of the second block of partition 2 (94208*512 + 4096 + 0x178). It is the least significant byte of the ext4fs superblock field "Number of KiB written to this filesystem over its lifetime". (https://ext4.wiki.kernel.org/index.php/ ... uper_Block)

fromJPN
Posts: 14
Joined: Wed Dec 13, 2017 3:40 am

Re: Raspbian with Read-only Root

Mon Dec 25, 2017 4:15 am

to jojopi

Thanks for the reply.
This was a difficult task for me. And I also found places I should study.
This was a difficult task for me. I have also found a thema I should study at soon. :shock:
Thank you for teaching me. :D

burke3gd
Posts: 1
Joined: Fri Feb 09, 2018 9:31 am

Re: Raspbian with Read-only Root

Fri Feb 09, 2018 9:13 pm

I recently implemented the read-only root with overlay fs on my RPI3. A huge thanks to ejolson and all the contributors in this thread, I couldn't have figured it out without reading your posts.

During the process I did make a number of changes and simplifications that I would like to share with you.

I prefer to keep the files in /usr in their original state. So instead of modifying the scripts there, I did the following:

Code: Select all

echo "overlay" >/etc/initramfs-tools/modules
cp /usr/share/initramfs-tools/scripts/local /etc/initramfs-tools/scripts/overlay
The boot script (/etc/initramfs-tools/scripts/overlay) need only contain the following:

Code: Select all

# Local filesystem mounting                     -*- shell-script -*-

#
# This script overrides local_mount_root() in /scripts/local
# and mounts root as a read-only filesystem with a temporary (rw)
# overlay filesystem.
#

. /scripts/local

local_mount_root()
{
        local_top
        local_device_setup "${ROOT}" "root file system"
        ROOT="${DEV}"

        # Get the root filesystem type if not set
        if [ -z "${ROOTFSTYPE}" ]; then
                FSTYPE=$(get_fstype "${ROOT}")
        else
                FSTYPE=${ROOTFSTYPE}
        fi

        local_premount

        # CHANGES TO THE ORIGINAL FUNCTION BEGIN HERE
        # N.B. this code still lacks error checking

        modprobe ${FSTYPE}
        checkfs ${ROOT} root "${FSTYPE}"

        # Create directories for root and the overlay
        mkdir /lower /upper

        # Mount read-only root to /lower
        if [ "${FSTYPE}" != "unknown" ]; then
                mount -r -t ${FSTYPE} ${ROOTFLAGS} ${ROOT} /lower
        else
                mount -r ${ROOTFLAGS} ${ROOT} /lower
        fi

        modprobe overlay

        # Mount a tmpfs for the overlay in /upper
        mount -t tmpfs tmpfs /upper
        mkdir /upper/data /upper/work

        # Mount the final overlay-root in $rootmnt
        mount -t overlay \
            -olowerdir=/lower,upperdir=/upper/data,workdir=/upper/work \
            overlay ${rootmnt}
}
Perhaps worth noting is that the

Code: Select all

/scripts
directory referenced in the script is only present in the final initramfs image. After these two steps you can build the initramfs image using update-initramfs as before. Also worth noting is that I didn't have any problem using the default UUID for the root partition in cmdline.txt.

I've documented everything in more detail on my website and there you can also find a script to toggle the overlay on and off and a motd(5) script that prints the current state of the overlay when you log on.

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Sat Feb 10, 2018 7:19 am

burke3gd wrote:
Fri Feb 09, 2018 9:13 pm
During the process I did make a number of changes and simplifications that I would like to share with you.
I like your simplifications. Thanks for posting. When I created the original post, I was not aware of how /etc/initramfs-tools/modules worked. That is definitely a better way to create a customized initramfs. I have recently used this approach to add drivers to an initramfs needed for a Pi Zero to boot without an SD card using rpiboot and mount a full Raspbian system image over NFS using the Ethernet gadget driver and only one USB cable.

cloroxman
Posts: 3
Joined: Sun Mar 18, 2018 10:18 pm
Contact: Website

Re: Raspbian with Read-only Root

Sun Mar 18, 2018 10:26 pm

burke3gd wrote:
Fri Feb 09, 2018 9:13 pm
I recently implemented the read-only root with overlay fs on my RPI3. A huge thanks to ejolson and all the contributors in this thread, I couldn't have figured it out without reading your posts.

During the process I did make a number of changes and simplifications that I would like to share with you.

I prefer to keep the files in /usr in their original state. So instead of modifying the scripts there, I did the following:

Code: Select all

echo "overlay" >/etc/initramfs-tools/modules
cp /usr/share/initramfs-tools/scripts/local /etc/initramfs-tools/scripts/overlay
The boot script (/etc/initramfs-tools/scripts/overlay) need only contain the following:

Code: Select all

# Local filesystem mounting                     -*- shell-script -*-

#
# This script overrides local_mount_root() in /scripts/local
# and mounts root as a read-only filesystem with a temporary (rw)
# overlay filesystem.
#

. /scripts/local

local_mount_root()
{
        local_top
        local_device_setup "${ROOT}" "root file system"
        ROOT="${DEV}"

        # Get the root filesystem type if not set
        if [ -z "${ROOTFSTYPE}" ]; then
                FSTYPE=$(get_fstype "${ROOT}")
        else
                FSTYPE=${ROOTFSTYPE}
        fi

        local_premount

        # CHANGES TO THE ORIGINAL FUNCTION BEGIN HERE
        # N.B. this code still lacks error checking

        modprobe ${FSTYPE}
        checkfs ${ROOT} root "${FSTYPE}"

        # Create directories for root and the overlay
        mkdir /lower /upper

        # Mount read-only root to /lower
        if [ "${FSTYPE}" != "unknown" ]; then
                mount -r -t ${FSTYPE} ${ROOTFLAGS} ${ROOT} /lower
        else
                mount -r ${ROOTFLAGS} ${ROOT} /lower
        fi

        modprobe overlay

        # Mount a tmpfs for the overlay in /upper
        mount -t tmpfs tmpfs /upper
        mkdir /upper/data /upper/work

        # Mount the final overlay-root in $rootmnt
        mount -t overlay \
            -olowerdir=/lower,upperdir=/upper/data,workdir=/upper/work \
            overlay ${rootmnt}
}
Perhaps worth noting is that the

Code: Select all

/scripts
directory referenced in the script is only present in the final initramfs image. After these two steps you can build the initramfs image using update-initramfs as before. Also worth noting is that I didn't have any problem using the default UUID for the root partition in cmdline.txt.

I've documented everything in more detail on my website and there you can also find a script to toggle the overlay on and off and a motd(5) script that prints the current state of the overlay when you log on.

THANK YOU. Works amazing for my auto/dash project. Thank you so much for this.

derykmarl
Posts: 9
Joined: Wed Dec 13, 2017 3:33 pm

Re: Raspbian with Read-only Root

Wed Apr 18, 2018 10:20 am

Are people still successfully using this method with the latest kernel?

Just wondering as ever since the latest updates I'm having a kernel panic on boot about 9 out of 10 times as per this thread:
viewtopic.php?f=28&t=211321

It might be unrelated to this, however, but this is the one major modification I made to the installation.

Edit: I wonder if it's all the additional modules I shoved in there as per my long post on page 6. Some kind of race condition if the kernel and a module are trying to load the same function at the same time...

Edit 2: It wasn't the modules :/ but it seems like the initrd or the overlay makes it unstable.

User avatar
SlowBro
Posts: 164
Joined: Sat Feb 18, 2017 1:30 am

Re: Raspbian with Read-only Root

Wed Apr 18, 2018 11:35 am

What happens when you disable it?

derykmarl
Posts: 9
Joined: Wed Dec 13, 2017 3:33 pm

Re: Raspbian with Read-only Root

Wed Apr 18, 2018 3:10 pm

Removing the overlay makes no difference

The race seems to have flipped this afternoon so that 9 out of 10 boots work and only 1 in 10 fails, so it's hard to tell, but from what I can tell so far, removing the initrd line from config.txt resolves the boot issue. I've successfully booted at least a dozen times. So it appears to be an issue with the initrd image.

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Wed Apr 18, 2018 3:59 pm

derykmarl wrote:
Wed Apr 18, 2018 3:10 pm
from what I can tell so far, removing the initrd line from config.txt resolves the boot issue.
Some of the newer images specify the root partition using PARTUUIDs rather than partitions. This caused my initial RAM filesystem to have booting problems. Such may be the case for you. If so, edit /boot/cmdline.txt and change the "root=PARTUUID..." option to read "root=/dev/mmcblk0p2" instead.

From my point of view GNU/Linux boot and initialization has become complicated, opaque and impossible to understand. It would be nice if Raspbian did something to simplify things in a way that was more suitable to learning and IOT, but instead it closely follows Debian which has other goals.

User avatar
HawaiianPi
Posts: 4731
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Raspbian with Read-only Root

Wed Apr 18, 2018 7:25 pm

ejolson wrote:
Wed Apr 18, 2018 3:59 pm
From my point of view GNU/Linux boot and initialization has become complicated, opaque and impossible to understand.
While I agree that some things have become more complicated with Linux over the last decade or so, and what used to involve editing one configuration file now often involves editing multiple files, it's hardly opaque (most is well documented), and it's only impossible to understand if you don't try.
ejolson wrote:
Wed Apr 18, 2018 3:59 pm
It would be nice if Raspbian did something to simplify things in a way that was more suitable to learning and IOT, but instead it closely follows Debian which has other goals.
It follows Debian because Raspbian is Debian ARMhf with some Raspberry Pi specific optimizations. They are not going to reinvent the wheel to satisfy the needs of a small niche group of users.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

derykmarl
Posts: 9
Joined: Wed Dec 13, 2017 3:33 pm

Re: Raspbian with Read-only Root

Thu Apr 19, 2018 7:52 am

ejolson wrote:
Wed Apr 18, 2018 3:59 pm
derykmarl wrote:
Wed Apr 18, 2018 3:10 pm
from what I can tell so far, removing the initrd line from config.txt resolves the boot issue.
Some of the newer images specify the root partition using PARTUUIDs rather than partitions. This caused my initial RAM filesystem to have booting problems. Such may be the case for you. If so, edit /boot/cmdline.txt and change the "root=PARTUUID..." option to read "root=/dev/mmcblk0p2" instead.

From my point of view GNU/Linux boot and initialization has become complicated, opaque and impossible to understand. It would be nice if Raspbian did something to simplify things in a way that was more suitable to learning and IOT, but instead it closely follows Debian which has other goals.
Thanks for the idea... no luck unfortunately (did wonder as you'd expect a consistent error about mounting root in that situation)

Tend to agree with your sentiments on the obfuscation of Linux these days - if not this then udev, dot dee disease, selinux... can't help getting the feeling that with things like Rpi becoming popular the developers want to make sure things aren't too easy and attract too many newbies!

Managed to slow motion film the boot which allowed me to see more of the error message, specifically "bad mode in data abort handler" which is reported here https://github.com/raspberrypi/linux/issues/2450

derykmarl
Posts: 9
Joined: Wed Dec 13, 2017 3:33 pm

Re: Raspbian with Read-only Root

Thu Apr 19, 2018 10:11 am

Found a related thread: viewtopic.php?f=29&t=197689

Adding dwc_otg.fiq_enable=0 and dwc_otg.fiq_fsm_enable=0 to /boot/cmdline.txt as per paul433's comment there, appears to have either worked around the problem or at least made it a lot more stable.

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Thu Apr 19, 2018 2:42 pm

derykmarl wrote:
Thu Apr 19, 2018 10:11 am
Adding dwc_otg.fiq_enable=0 and dwc_otg.fiq_fsm_enable=0 to /boot/cmdline.txt
Thanks for posting this solution back here!

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Mon Apr 30, 2018 11:35 pm

Here is another thread about using an initramfs to set up a writable overlay on top of a read only root filesystem. The final link in that post points to an interesting script that contains significantly more code, error checking and commentary then the modifications suggested in the initial post here. Whether you use that script or not, it is definitely worth reading it through for the comments.

amirfh
Posts: 3
Joined: Sat Mar 30, 2013 11:33 pm

Re: Raspbian with Read-only Root

Fri May 25, 2018 3:54 am

raspbian desktop readonly with desktop : https://github.com/janztec/empc-arpi-linux-readonly

basicline
Posts: 7
Joined: Tue Apr 21, 2015 10:18 am

Re: Raspbian with Read-only Root

Fri Jul 06, 2018 4:22 pm

This option works well with the "Raspbian Stretch with Desktop (2017-09-07)" version but not with the new ones. Do you know if there is something that works with the new versions?

barsznica
Posts: 18
Joined: Thu Apr 14, 2016 12:38 pm

Re: Raspbian with Read-only Root

Wed Jul 25, 2018 5:17 am

Thanks!!! Worked for me on a PiZeroW, 4.14.52-v7+ #1123 SMP armv7l
I did NOT remove the "7"s

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Fri Jul 27, 2018 7:09 am

barsznica wrote:
Wed Jul 25, 2018 5:17 am
Thanks!!! Worked for me on a PiZeroW, 4.14.52-v7+ #1123 SMP armv7l
I did NOT remove the "7"s
I find it interesting that the ARMv7 kernel works on the Zero. Does the kernel without the 7's work better? Was there a reason you did not remove the 7's?

User avatar
TimG
Posts: 293
Joined: Tue Apr 03, 2012 12:15 am
Location: Switzerland

Re: Raspbian with Read-only Root

Fri Jul 27, 2018 11:11 pm

Surely if you don't remove the 7s the recipe has no effect on a Pi Zero? I just tried an ARMv7 kernel on my Zero and as expected it would not boot.

philippejadin
Posts: 1
Joined: Mon Oct 22, 2018 7:54 pm

Re: Raspbian with Read-only Root

Mon Oct 22, 2018 7:58 pm

For all those looking for a working version on latest raspbian, there is a more "established" script here : https://github.com/chesty/overlayroot

It seems there are also packages called fsprotect and bilibop-locks (can be installed with sudo apt install ... see the ease of use ?) but I didn't test them yet.

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Tue Oct 23, 2018 1:36 am

philippejadin wrote:
Mon Oct 22, 2018 7:58 pm
For all those looking for a working version on latest raspbian, there is a more "established" script here : https://github.com/chesty/overlayroot

It seems there are also packages called fsprotect and bilibop-locks (can be installed with sudo apt install ... see the ease of use ?) but I didn't test them yet.
Thanks for the link. It looks like a similar idea, though implemented slightly differently. I suspect both scripts share the fault that the initial RAM filesystem remains in memory after the system is booted.

From what I understand, the solution is to mount the overlay layers under /run on the initial RAM filesystem so memory can be freed after pivoting root. This improvement comes from my current project of mounting the root filesystem using loopback to an ext4 image accessed from a Microsoft Windows share. The same technique should save about 80MB of RAM here as well.

User avatar
SlowBro
Posts: 164
Joined: Sat Feb 18, 2017 1:30 am

Re: Raspbian with Read-only Root

Tue Oct 23, 2018 6:58 pm

philippejadin wrote:
Mon Oct 22, 2018 7:58 pm
It seems there are also packages called fsprotect and bilibop-locks (can be installed with sudo apt install ... see the ease of use ?) but I didn't test them yet.
Both of these require AUFS support in the kernel which I believe is not currently present. Ironically it would require more work to get those going than the scripts presented in this thread. Apt-get is only the first of many steps :-)

User avatar
SlowBro
Posts: 164
Joined: Sat Feb 18, 2017 1:30 am

Re: Raspbian with Read-only Root

Mon Nov 19, 2018 12:35 am

I finally got around to trying this (hello, username is SlowBro, that should tell you something) :D and an apt-get dist-upgrade with a new kernel caused the system after rebooting to be unable to load any modules. The reason is simple: No initrd for the new kernel.

At the moment I am running this to build the initrd. How might I capture the name of any new kernels that are being installed with apt and automagically build a new initrd? How are you all handling this condition?

Code: Select all

update-initramfs -c -k $(uname -r)

ejolson
Posts: 3682
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspbian with Read-only Root

Tue Nov 20, 2018 2:05 am

SlowBro wrote:
Mon Nov 19, 2018 12:35 am
I finally got around to trying this (hello, username is SlowBro, that should tell you something) :D and an apt-get dist-upgrade with a new kernel caused the system after rebooting to be unable to load any modules. The reason is simple: No initrd for the new kernel.

At the moment I am running this to build the initrd. How might I capture the name of any new kernels that are being installed with apt and automagically build a new initrd? How are you all handling this condition?

Code: Select all

update-initramfs -c -k $(uname -r)
When Raspbian updates the kernel it should check whether an initial RAM filesystem is being used and make a new one. As you have pointed out, this doesn't happen. Moreover, the default size of the boot partition is too small to store backup copies of previous initial RAM filesystems and it wouldn't help anyway because Raspbian automatically removes all traces of old kernels and their drivers from the root filesystem when updating to new kernel. I know of two possible solutions:

1. Pin the current kernel so it won't be automatically removed using something like

# echo XXXXXXX hold | dpkg --set-selections

where XXXXXXX is the package name of the currently installed kernel.

2. Watch carefully when doing update upgrade and regenerate the new initial RAM filesystem by hand if necessary.

I generally prefer the second option because it allows me to keep current; however, for the Pi Zero I prefer to stick with one of the older kernels that support jumbo packets over the USB Ethernet gadget.

User avatar
SlowBro
Posts: 164
Joined: Sat Feb 18, 2017 1:30 am

Re: Raspbian with Read-only Root

Tue Nov 20, 2018 4:29 am

Thank you for the suggested workarounds!
ejolson wrote:
Tue Nov 20, 2018 2:05 am
When Raspbian updates the kernel it should check whether an initial RAM filesystem is being used and make a new one. As you have pointed out, this doesn't happen.
Does it happen for you or am I the only one?

Return to “General discussion”