markermoniker
Posts: 5
Joined: Tue Nov 19, 2013 2:22 am

SD Card Corruption

Tue Nov 19, 2013 3:38 am

I have been having an issue with my SD cards failing. Eventually, (after weeks or months of running fine) the running operating system will present strange behavior, including the root partition being read-only, (the lock tab was not moved) and many commands failing. (Shutdown doesn't work, for example) After manually powering off and on, it won't boot. Video output says that the filesystem has errors, that is it mounting as read only, and then continues to spout a list of bad blocks, and then hang.

I will toss the SD card into my laptop, and run fsck on it, which finds and corrects errors, but in the Pi, it presents the same behavior. When I finally give up on restoring it, I'll just dump the raspbian image onto it with dd, which gives me this error:

Code: Select all

dd: writing to `/dev/sdc': Input/output error
39953+0 records in
39952+0 records out
20455424 bytes (20 MB) copied, 41.7379 s, 490 kB/s
The bytes written is actually variable. Anywhere from 10MB to 200MB before it errors out. Once a card is corrupted, I have been unable to restore it.

This error has pursued me across two pis and four SD cards. The SD cards have been from different manufacturers, (Toshiba and Kingston) but all have been microSD converted through an adapter to standard size. I have not been overclocking.

Given that this is clearly not a piece of faulty hardware (there is no consistent element of hardware), I must conclude that the fault is in my practices. So, what am I doing wrong?

MoosePi
Posts: 112
Joined: Wed Nov 06, 2013 4:42 pm

Re: SD Card Corruption

Tue Nov 19, 2013 8:23 am

This is a subject which has been bought up many times, it would appear that the latest Kernel/Firmware has rectified many of the issues:

http://www.raspberrypi.org/phpBB3/searc ... corruption
Pi Forum Search and Google yields answers to many questions

obcd
Posts: 917
Joined: Sun Jul 29, 2012 9:06 pm

Re: SD Card Corruption

Tue Nov 19, 2013 9:56 am

The subject has indeed been brought up several times. The scary part about it is the fact that nobody seems to know what is causing the sd card corruption.

It might be interesting to see if you can fix the sd cards by reformatting them with the special sd card formatting tool.

A sd card can get damaged by exceeding it's maximum number of write cycles.
The 2 things we normally don't control in linux is the creation of the logfiles and the use of the swap file. Maybe a device connected to your Pi causes excessive generation of logfiles and that is killing your sd cards after a period of time?

I will also add some of the classic answers you will read considering sd card corrruption.

I am using my Pi for 6 months now and never had a sd card corruption problem. (Very helpfull isn't it?)
2 much overclocking can cause sd card corruption (I know you said you didn't overclock, just wanted the list to be complete)
some brand of sd card are worse for corruption than others. Try another brand of sd card. (You already tried 2 different brands)
You might have a bad power supply. As this might cause usb or network problems, those problems could be responsable for the creation of the logfiles.
Maybe it is the way you are using your Pi. In order to figure that out, you will need to give a bit more information.
What speed class are your sd cards?
What are you using the Pi's 4? What os is on the sd cards? Do you have some special software running?

obcd
Posts: 917
Joined: Sun Jul 29, 2012 9:06 pm

Re: SD Card Corruption

Tue Nov 19, 2013 10:09 am

One way to prevent sd card corruption is creating a read only rootfs. This basically means you should disable swap and create a tempfs in ram for the log files and other stuff that needs write access. Maybe changing the behavour of this 2 things, even without a read only rootfs might already help a lot against sd card corruption.

One thing I also forgot to mention is that some sd cards are cheap chinese clones of known good brands.
Your 2 euro Kingston card you bought on Ebay maybe isn't a Class 10 like it's written on the card, And some even don't have the size written on the card. Google for fake sd cards and you will find some interesting articles.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2239
Joined: Thu Jul 11, 2013 2:37 pm

Re: SD Card Corruption

Tue Nov 19, 2013 10:51 am

obcd wrote:The subject has indeed been brought up several times. The scary part about it is the fact that nobody seems to know what is causing the sd card corruption.
Incorrect.

https://github.com/raspberrypi/linux/issues/413

There are 3 currently known issues that can cause either a failure to boot or data corruption. The jackpot issue is #415 - found to be strongly correlated with clock-crossing as a result of core / emmc clock ratios.

Workarounds for all of these issues are in place in the latest firmware and kernel. Root-cause analysis on #415 - which is the only verified source of data corruption (ignoring dodgy cards, pulling the power or hotswapping the card while writing to it) - requires quite an in-depth analysis of what goes on inside the SoC. Currently all the reported feedback from the testing so far on the workaround has been positive.
Rockets are loud.
https://astro-pi.org

markermoniker
Posts: 5
Joined: Tue Nov 19, 2013 2:22 am

Re: SD Card Corruption

Wed Nov 20, 2013 8:14 am

First, thanks for all the leads. You guys have been very helpful.

To answer the investigatory questions: I have varied also in my selection of power supply, so it would be unlikely that those are the issue. I've been using the Pis largely as cheap servers. The most recent card to crash was in a VPN client and router to a virtual network interface. Before that, I have lost the cards doing things as innocuous as being a NAS, and as far as I know, I lost one before I it was really doing anything. I should be clear: not all of my cards have failed. I have two that are running just fine, and have been for months. One Pi, which I think is at something like four months of uptime, is using a card in all ways identical to one that failed (I even ordered them together).

Now, I have been using (recently, I was using other images before) the raspbian 2013-09-25 image available from http://www.raspberrypi.org/downloads. It is my understanding that a) the firmware for the system is contained in and read the first partition of that image, and that b) it contains the most up-to-date release of kernel and firmware available. If the former is incorrect, how can I go about upgrading the firmware? If the latter is incorrect, where can I find more up-to-date files to use? If I'm correct in assuming that this is all written into the image, I could pretty easily mount and modify an image I have to use newer firmware.

I'll pursue read only root for the moment, but this definitely seems like a non-optimal scenario. (Most of the point of a Raspberry Pi is tinkering on it!)

I appreciate all the help. Any links to point me in the right direction would be super useful. The github posting seemed to recommend modifying the source code for the a driver, and produced mixed results. Now, if this is the only way, I'll pursue that path and make it work, but I'm definitely under the impression that if this is a known issue, there is probably a simpler solution.

Again, thanks.

MoosePi
Posts: 112
Joined: Wed Nov 06, 2013 4:42 pm

Re: SD Card Corruption

Wed Nov 20, 2013 10:04 am

I will post verbatim from another thread how I achieved the latest beta/testing Kernel / Firmware:

If one starts with the latest download of Raspbian Wheezy "2013-09-25-wheezy-raspbian.zip"

Then at command line:

sudo apt-get update
sudo apt-get dist-upgrade
sudo reboot

sudo rpi-update
sudo reboot

This is what have at present:

/opt/vc/bin/vcgencmd version
Nov 12 2013 23:43:03
Copyright (c) 2012 Broadcom
version 4aee5454c7955e7bc0bbb152ca4c0e26e75376e1 (clean) (release)

uname -a
Linux raspberrypi 3.10.18+ #594 PREEMPT Wed Nov 13 17:59:34 GMT 2013 armv6l GNU/Linux

Note to self: Actually I must undertake another update there is a newer Kernel / Firmware
Pi Forum Search and Google yields answers to many questions

markermoniker
Posts: 5
Joined: Tue Nov 19, 2013 2:22 am

Re: SD Card Corruption

Wed Nov 20, 2013 10:09 am

Sweet, I'll try that, thanks!

MoosePi
Posts: 112
Joined: Wed Nov 06, 2013 4:42 pm

Re: SD Card Corruption

Wed Nov 20, 2013 10:13 am

markermoniker wrote:Sweet, I'll try that, thanks!
If you change the Board Preferences - My Board Style to prosilver via your User Control Panel you may find searching for previous information easier !
Pi Forum Search and Google yields answers to many questions

markermoniker
Posts: 5
Joined: Tue Nov 19, 2013 2:22 am

Re: SD Card Corruption

Wed Nov 20, 2013 10:35 am

That is certainly a more familiar board style. Thanks for the tip.

I had done quite a bit of searching before posting. I was disheartened by what appeared to be a consensus that overclocking was the cause, and so finally made the post. Thanks for being so helpful.

I'll let this thread die for now. If it comes to pass that a card becomes corrupted once again in the same manner after trying that upgrade, I'll come back to warn the similarly symptomed off. Until then, assume that this has worked! Thanks again!

MoosePi
Posts: 112
Joined: Wed Nov 06, 2013 4:42 pm

Re: SD Card Corruption

Wed Nov 20, 2013 12:27 pm

No problem, been there, done that and even have a Raspberry Pi T-Shirt !
Pi Forum Search and Google yields answers to many questions

Return to “Troubleshooting”