Read-only files system error - corrupt?


5 posts
by Germanunkol » Sat Jan 19, 2013 10:39 am
Hi,
I've been running a gameserver of a game I'm programming on my PI. Also, a apache2 and MySQL server are running on it and working relatively well. However, I've had filesystem problems with the pi.
I use the Pi only through ssh and I currently do not have physical access to it - which makes my Problem worse.

I have an Kingston SD (16Gb) and there was one time when the filesystem seemed to have gone corrupt - I cannot recall the exact symptoms, but I ended up setting up the Pi entirely.
Now it is happening again: This time, I get errors saying my filesystem was Read-only when I use TAB to auto-complete when using cd:
I entered "cd tr" then pressed TAB and get:
Code: Select all
cd tr-bash: cannot create temp file for here-document: Read-only file system
-bash: cannot create temp file for here-document: Read-only file system

It's not just cd though, when trying to use git pull origin in my project's directory I get:
Code: Select all
error: cannot open .git/FETCH_HEAD: Read-only file system

Then I tried fdisk -l and get an "command not found" message. Since I got this for multiple programs I checked "ls /usr/bin" and I get:
Code: Select all
ls: cannot access /usr/bin/mysqlshow: Input/output error
ls: cannot access /usr/bin/dbiprof: Input/output error
ls: cannot access /usr/bin/mysql_waitpid: Input/output error
ls: cannot access /usr/bin/mysqld_multi: Input/output error
ls: cannot access /usr/bin/innotop: Input/output error
ls: cannot access /usr/bin/mysqldump: Input/output error
ls: cannot access /usr/bin/sha384sum: Input/output error
ls: cannot access /usr/bin/mysqldumpslow: Input/output error
ls: cannot access /usr/bin/mysqlreport: Input/output error
ls: cannot access /usr/bin/pdftoppm: Input/output error
ls: cannot access /usr/bin/shuf: Input/output error
ls: cannot access /usr/bin/mysql_find_rows: Input/output error
ls: cannot access /usr/bin/mysqlrepair: Input/output error
ls: cannot access /usr/bin/setsid: Input/output error
ls: cannot access /usr/bin/mysql_fix_extensions: Input/output error
ls: cannot access /usr/bin/mysql_client_test: Input/output error
ls: cannot access /usr/bin/mysqlcheck: Input/output error
ls: cannot access /usr/bin/ucs2any: Input/output error
ls: cannot access /usr/bin/tsort: Input/output error
ls: cannot access /usr/bin/mysqloptimize: Input/output error
ls: cannot access /usr/bin/pdfunite: Input/output error
ls: cannot access /usr/bin/mysql_plugin: Input/output error

I suspected it might have something to do with the mysql server so I tired:
"/etc/init.d/mysqld stop" but, again, I get "command not found"
and mysql -u root -p gives me a segmentation fault.

Next thing was to check if the filesystems are full, because my gameserver timed out with an error about not being able to open a file:
df gives me:
Code: Select all
Filesystem     1K-blocks    Used Available Use% Mounted on
rootfs          12799120 1925948  10222932  16% /
/dev/root       12799120 1925948  10222932  16% /
devtmpfs          224436       0    224436   0% /dev
tmpfs              44900     212     44688   1% /run
tmpfs               5120       0      5120   0% /run/lock
tmpfs              89780       0     89780   0% /run/shm
/dev/mmcblk0p1     57288   16872     40416  30% /boot


I'm scared of rebooting the pi because right now, I at least still have access to it - who knows if it'll even boot again...
I need the pi to run for multiple months without reboot (webserver) - but I keep on getting these filesystem errors.
I'm running the official Debian distro and using the power cable supplied by the official Pi store where I got my Pi.
Help?
Posts: 4
Joined: Sat Jan 19, 2013 10:18 am
by Germanunkol » Sat Jan 19, 2013 11:32 am
Sorry, I forgot to add:
Because of the read-only error, I checked the "mount" command and I get:
Code: Select all
/dev/root on / type ext4 (ro,noatime,user_xattr,barrier=1,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=224436k,nr_inodes=56109,mode=755)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=44900k,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=89780k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
/dev/mmcblk0p1 on /boot type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=ascii,shortname=mixed,errors=remount-ro)

So /dev/root seems to be read-only after all... but how did this happen?
Afterwards I tried "mount -o remount,rw /" to try and change the root partition from ro to rw, but I get the error:
"cannot remount block device /dev/mmcblk0p2 read-write, is write-protected"
This sounds like the card's "lock" switch has been activated - but I haven't touched it (it's not even here) and it was working fine before. Also, the other filesystems are working in rw mode, and the lock would lock them all, right?

Sorry, lots of text, I know... but I hope this way someone can help me better.
Posts: 4
Joined: Sat Jan 19, 2013 10:18 am
by obcd » Sat Jan 19, 2013 12:06 pm
The file system became read only because something corrupted it.
I think it's fsck that actually sets it read only so that it check and repair it.
So, the question is, why did it became corrupt? There are a number of things possible.

Improper shutting down the Pi can cause corruption.
Overclocking can cause corruption.
Low Cost Chinese clone sd cards that look exactly like the originals can cause corruption.
.....

You can try to mount the sd card on another linux system and see if fsck can fix it.
Posts: 890
Joined: Sun Jul 29, 2012 9:06 pm
by seandoyle » Sun Jun 02, 2013 6:30 pm
Hi -

I just had an identical symptom and there was an easy fix. I had attached a crazyradio (for the crazyflie quadcopter) to the USB port and the Pi would not launch X; the basic problem was that the file system was read-only. "df -h" showed that my disk was about 36% full so there was plenty of room; the system had been rebooted by a 'sudo reboot' so the shutdown should have been gentle enough.

But - removing the crazyradio did the trick. My guess (only a guess) is that this drew too much power from the rest of the Pi and the results were then unpredictable. Next time I'll plug it into the powered USB hub.

Sean
Posts: 6
Joined: Mon Oct 22, 2012 2:26 am
by andig2 » Sat Jun 29, 2013 9:01 pm
I'm having the save problem now. Put my pi through apt-get update && upgrade, then shutdown -r now.

After that it won't come up- same Edimax WLAN adapter. The SD card also won't come up in another Pi. By won't come up I mean that the system boots but WLAN doesn't start. Hooking in via LAN shows that / is ro.

Any ideas how to fix that? Unfortunately I don't have another Linux box for fixing the card... sudo fsck / says:

fsck from util-linux 2.20.1
e2fsck 1.42.5 (29-Jul-2012)
/dev/mmcblk0p2: clean, 89052/974848 files, 799175/3874176 blocks
Posts: 24
Joined: Wed Oct 31, 2012 9:34 pm