paulv
Posts: 558
Joined: Tue Jan 15, 2013 12:10 pm
Location: Netherlands

Forcing a file check on the root partition @ boot

Sat Jun 20, 2015 12:06 pm

Through some clever detective work, forum user "Joe Schmoe" found an undocumented feature (I could not find it) that allows you to force the kernel to do an SD card file check of both partitions (p1 and p2) during the boot sequence.

Note that this method does not seem to work on a NOOBS installation, see the details in the posts below!
Check a normal RASPBIAN installation with:
[email protected] ~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" LABEL="boot" UUID="15CD-3B79" TYPE="vfat"
/dev/mmcblk0p2: UUID="13d368bf-6dbf-4751-8ba1-88bed06bef77" TYPE="ext4"
This checking at boot time is particularly helpful if you have or suspect SD card issues due to power related problems, or if you have to yank the power connection to force a restart.

There are two possibilities:
1. Force the file check once:
Create a file in the root directory.

Code: Select all

sudo touch /forcefsck
At the next boot, the kernel will do the file check, but will remove the file afterwards.
This feature can be used by a program or shell, so you can do it at will.

2. Permanently do the file check during every boot:
Add the "forcefsck" command/keyword to the cmdline.txt file.
[email protected] ~ $ cat /boot/cmdline.txt
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait forcefsck
[email protected] ~ $
In both cases, the kernel does the file check right after the cmdline.txt file is read, so even before mounting /dev/mmcblk0p1. You can see this on the monitor during the boot process. Unfortunately, the file check does not show up in any of the regular logs. The only evidence can be found in /var/log/fsck:
[email protected] $ ll /var/log/fsck
total 8
-rw-r----- 1 root adm 210 Jun 20 10:31 checkfs
-rw-r----- 1 root adm 447 Jun 20 10:31 checkroot
checkfs shows the file check results with dosfsck on the mmcblk0p1 partition, while checkroot shows the file check with e2fsck of the mmcblk0p2 partition.
[email protected] ~ $ cat /var/log/fsck/checkfs
Log of fsck -C -R -A -y -f
Sun Jun 21 09:25:56 2015

fsck from util-linux 2.20.1
dosfsck 3.0.13, 30 Jun 2012, FAT32, LFN
/dev/mmcblk0p1: 56 files, 2475/7161 clusters

Sun Jun 21 09:25:56 2015
----------------
[email protected] $ cat /var/log/fsck/checkroot
Log of fsck -C -f -y -t ext4 /dev/mmcblk0p2
Sat Jun 20 10:30:52 2015

fsck from util-linux 2.20.1
e2fsck 1.42.5 (29-Jul-2012)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mmcblk0p2: 92748/482384 files (0.1% non-contiguous), 735882/1925120 blocks

Sat Jun 20 10:31:06 2015
----------------
[email protected] $
Note the -f option (force) in the command line above: Log of fsck -C -f -y -t ext4 /dev/mmcblk0p2
A normal boot shows this as a result:
[email protected] ~ $ cat /var/log/fsck/checkroot
Log of fsck -C -y -t ext4 /dev/mmcblk0p2
Sat Jun 20 13:39:11 2015

fsck from util-linux 2.20.1
e2fsck 1.42.5 (29-Jul-2012)
/dev/mmcblk0p2: clean, 92748/482384 files, 735997/1925120 blocks

Sat Jun 20 13:39:11 2015
----------------
[email protected] ~ $
Note the absence of the -f parameter

I hope this helps!
paulv
Last edited by paulv on Sun Jun 21, 2015 8:10 am, edited 2 times in total.

JimmyN
Posts: 1109
Joined: Wed Mar 18, 2015 7:05 pm
Location: Virginia, USA

Re: Forcing a file check on the root partition @ boot

Sat Jun 20, 2015 4:30 pm

It's not going to work for me, it doesn't check any vfat file systems, and that's where I have the problem after loss of power, the ext4 filesystems are always fine. I never see a problem with /root, it's /boot that gets corrupted. I pulled the power from the RPi which always results in a corrupted /boot and /sda (which is formatted FAT32).
I rebooted and dmesg shows it happened as expected, telling me to run fsck on mmcblk0p5 and sda because the volumes were not properly unmounted.

Added forcefsck to cmdline.txt per your example. Shutdown properly, then booted back up and dmesg still shows the same error for mmcblk0p5 and sda. Shutdown and booted again to make sure and here are the checkfs and checkroot files, the "-f" parameter is there.

Code: Select all

>$ cat /var/log/fsck/checkfs
Log of fsck -C -R -A -a -f 
Sat Jun 20 11:05:02 2015

fsck from util-linux 2.20.1

Sat Jun 20 11:05:02 2015
----------------

>$ cat /var/log/fsck/checkroot
Log of fsck -C -f -a -t ext4 /dev/mmcblk0p6 
Sat Jun 20 11:04:37 2015

fsck from util-linux 2.20.1
root: 89534/381264 files (0.1% non-contiguous), 889563/1524480 blocks

Sat Jun 20 11:04:54 2015
----------------
It checked mmcblk0p6 which is /root (ext4 filesystem), but it didn't check mmcblk0p5 which is /boot (vfat filesystem) and where the problem occurs. It didn't fsck the sda which is also vfat.

Code: Select all

>$ blkid
/dev/mmcblk0p1: LABEL="RECOVERY" UUID="3781-1729" TYPE="vfat" 
/dev/mmcblk0p3: LABEL="SETTINGS" UUID="17439bf6-ef15-4f71-ac62-9f0ff9487489" TYPE="ext4" 
/dev/mmcblk0p5: LABEL="BOOT" UUID="04F4-4098" TYPE="vfat" 
/dev/mmcblk0p6: LABEL="root" UUID="1e8cba98-4f87-4c4d-ac95-ae736a8e38fc" TYPE="ext4" 
/dev/sda: LABEL="SP UFD U3" UUID="11FC-1578" TYPE="vfat" 
So it doesn't seem to run fsck on any vfat partitions/drives, but those are the ones that are corrupted from improper shutdown, and adding the "forcefsck" to cmdline.txt increased the boot time so I've removed it. I used the method I posted in the other thread, then shutdown and re-booted and the file systems were then OK with no error messages.

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: Forcing a file check on the root partition @ boot

Sat Jun 20, 2015 6:50 pm

There is a problem of long-standing, where you always get that message on startup (if /boot ever gets shutdown improperly). It seems to be a bug in fsck.vfat - where it just doesn't clear the "dirty" flag (even though it did clean up the problem). They keep telling us that it will be fixed "any day now" - and, I guess it is fixed in "Jessie", so if you "upgrade" to "Jessie", you get the fixed fsck.vfat program.

Anyway, it sounds like you are experiencing this bug.
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

paulv
Posts: 558
Joined: Tue Jan 15, 2013 12:10 pm
Location: Netherlands

Re: Forcing a file check on the root partition @ boot

Sun Jun 21, 2015 7:04 am

Jimmy, I see a difference in the checkroot log I show, and the one you show.
From your blkid, it looks like you are using a NOOBS installation? I'm no expert on NOOBS or Linux, but it looks to me that this is the reason for the differences we have.

Here is my blkid, result from a simple installation of RASPBIAN:
[email protected] ~ $ sudo blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" LABEL="boot" UUID="15CD-3B79" TYPE="vfat"
/dev/mmcblk0p2: UUID="13d368bf-6dbf-4751-8ba1-88bed06bef77" TYPE="ext4"
My p1 shows msdos with vfat.
That partition get checked with dosfsck as follows:
[email protected] ~ $ cat /var/log/fsck/checkfs
Log of fsck -C -R -A -y -f
Sun Jun 21 09:25:56 2015

fsck from util-linux 2.20.1
dosfsck 3.0.13, 30 Jun 2012, FAT32, LFN
/dev/mmcblk0p1: 56 files, 2475/7161 clusters

Sun Jun 21 09:25:56 2015
----------------
My p2 (ext4) (should be the same as your p6) and that gets checked with e2fsck, but notice the difference in the command line. I show:
fsck -C -f -y -t ext4 /device
and you show:
fsck -C -f -a -t ext4 /device
according to man e2fsck(8), -y adds a "yes" to all questions, and -a acts like -p which repairs. This is confusing and I cannot explain this difference. Also in my report it shows the version information of the formatter, and the passes it took while running. This concurs with what I see on the console during the boot process.

In your report, the version of e2fsck is missing, and it also does not show the pass information. From the report, it looks like it did not run (completely?). What partitions do you actually see checked on the console during the boot process?

It looks to me that a NOOBS installation does not check the p5 ("BOOT" of type vfat), and maybe other partitions with the -f parameter, and that may be an issue. I will post a NOOBS warning in my original post to this effect.

Apart from this, I can verify what Joe is saying about the pesky /boot message, if you once had an issue, it stays with you, even though it was "seemingly" fixed.

JimmyN
Posts: 1109
Joined: Wed Mar 18, 2015 7:05 pm
Location: Virginia, USA

Re: Forcing a file check on the root partition @ boot

Tue Jun 23, 2015 5:59 pm

Sorry I missed your reply, don't know how that happened.

Yes that RPi is currently using a NOOBS install. After reading Joe Schmoe's post it dawned on me the he is right fsck.vfat didn't normally clear the dirty bit. So I installed "dosfstools 3.0.13-1_rpi1" from the repository and now it does, so fsck.vfat -ar clears the dirty bit and it's no longer a problem. But I don't know how you'd add fsck.vfat to the cmdline.txt file so it would check the vfat volumes as well. So I put it in a script and run that when the dirty bit appears again from a bad shutdown.

paulv
Posts: 558
Joined: Tue Jan 15, 2013 12:10 pm
Location: Netherlands

Re: Forcing a file check on the root partition @ boot

Fri Jun 26, 2015 7:04 am

Jimmy, it just dawned on me (duh) that you can automate the check by grepping the syslog file, and run the script (install in init.d) when there is a dirty bit detected.

squeekercat
Posts: 1
Joined: Wed Feb 04, 2015 3:26 pm

Re: Forcing a file check on the root partition @ boot

Sat Feb 20, 2016 4:42 pm

Joe Schmoe wrote:There is a problem of long-standing, where you always get that message on startup (if /boot ever gets shutdown improperly). It seems to be a bug in fsck.vfat - where it just doesn't clear the "dirty" flag (even though it did clean up the problem). They keep telling us that it will be fixed "any day now" - and, I guess it is fixed in "Jessie", so if you "upgrade" to "Jessie", you get the fixed fsck.vfat program.

Anyway, it sounds like you are experiencing this bug.
I have Jessie running on a Pi-zero, and can conform that thei issue is NOT resolved in 4.1.17+ #838 Tue Feb 9 12:57:10 GMT 2016 armv6l GNU/Linux

Since this affects the boot process, you'd think it would receive more attention.

corn
Posts: 1
Joined: Wed Apr 13, 2016 5:44 am

Re: Forcing a file check on the root partition @ boot

Wed Apr 13, 2016 5:59 am

Rasbian Jessie after 18th March 2016 release , The /boot/cmdline.txt have included the fsck.repair=yes. I'm guessing this is the patch for the issue.

Return to “Advanced users”