kaspencer
Posts: 79
Joined: Wed Mar 07, 2012 11:37 pm
Location: UK, England, Wiltshire

Re: Prevent SD-Card Corruption

Wed Mar 27, 2013 9:46 am

This is actually a very important topic. Especially, if (although probably not intended) people begin to use Raspberies Pi for serious applications.

I have posted several times elsewhere that I have 4 Raspberries Pi, and that three of them are being used for "serious" purposes. In my case, one is a webserver (108,000 hits and 14Gbyte served in 4 months), one is a Media Centre, and the third is a file-server and sort-of-PDC. I set these machines up to reduce the 24hr 7day power consumption of conventional PCs/Servers. (The fourth is not on except when being used).

None of my machines are overclocked. They all have approved power supplies rated at 1.5A. All the machines run very cool. They are very seldom indeed rebooted.

I have a great many SD cards, ranging from 4Gb Class 4 to 128Gb Class 10, and these include Transcend, Kingston, SanDisk, Intenso and HC. They are in both Micro and Standard SD packages.
And guess what? They have nearly all been currupted at some point or other, most several times. And it is virtually always the root partition that gets corrupted. So badly that fsck cannot recover it.

Now here is an interesting thing:
The Raspberry Pi that functions as a sort-of domain controller and fileserver does a backup nightly, of about 9.5Gbyte of key business data held with lots of other stuff on a 2TB USB disc. This backed-up data goes onto a separate ext4 partition on the SD Card, where it is held for four days and nights and is then deleted, to make way for the latest backup. This is a lot of reading and writing eh? But this partition has never been corrupted, notwithstanding all this activity. So the corruption of the SD Card, is clearly, in my view, something to do with what goes on in the root partition.

My conclusion is that if the Raspberry Pi is to have a broader future (which I want it to have), this issue of SD Card corruption must somehow be addressed.

Many thanks

Kenneth Spencer
2x256Mb + 2x512Mb RPi, Eth'netLAN+Win2k3R2 svr, 40MbpsFTTC.
RaspBMC: 128G SD, K400 wl Kb+TPd, HD32tv&Rem.
RW'y webserver: 64G SD, Y-RK49 wl Kb+M, HannsG W24" screen
RW'y PDC & fileserver: as above + 2TB disc.
+RiscOSPi on 32G uSD.

EdVoncken
Posts: 5
Joined: Sun Mar 17, 2013 5:00 pm

Re: Prevent SD-Card Corruption

Fri Mar 29, 2013 3:24 pm

Short update: the Pi seems stable with a Sandisk 4GB Class 4 SD card.
It's been running a fresh Raspbian 2013-02-09 for 7 days, no sign of corruption yet.

Next: see if installing all updates (apt-get upgrade; apt-get dist-upgrade) make a difference.

Things are starting to look good ;-)

kaspencer
Posts: 79
Joined: Wed Mar 07, 2012 11:37 pm
Location: UK, England, Wiltshire

Re: Prevent SD-Card Corruption

Fri Mar 29, 2013 4:53 pm

The thing is, Ed, that corruption is difficult to predict. You may have to wait weeks before the dreaded errors will appear, and stop your Raspberry Pi dead in its tracks.

You may be misled into a false sense of security in a 7-day only judgement.

Ken
2x256Mb + 2x512Mb RPi, Eth'netLAN+Win2k3R2 svr, 40MbpsFTTC.
RaspBMC: 128G SD, K400 wl Kb+TPd, HD32tv&Rem.
RW'y webserver: 64G SD, Y-RK49 wl Kb+M, HannsG W24" screen
RW'y PDC & fileserver: as above + 2TB disc.
+RiscOSPi on 32G uSD.

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: Prevent SD-Card Corruption

Fri Mar 29, 2013 5:03 pm

True, but I use a bunch of 8GB cass 4 Sandisk cards for about a year now (4 Pies, 7 SD cards), often 'pull the plug' and never had card corruption. So Ed may be on to something :D

EdVoncken
Posts: 5
Joined: Sun Mar 17, 2013 5:00 pm

Re: Prevent SD-Card Corruption

Fri Mar 29, 2013 11:55 pm

OK, just did the "apt-get update; apt-get dist-upgrade", rebooted. No errors.
Installed and ran rpi-update, followed instructions at https://github.com/Hexxeh/rpi-update. Rebooted again, and suddenly I see a few errors up in /var/log/messages:

Code: Select all

Mar 29 18:16:21 raspberrypi kernel: [    9.679809] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
Mar 29 18:16:21 raspberrypi kernel: [   14.952356] mmc0: missed completion of cmd 18 DMA (512/512 [1]/[1]) - ignoring it
Mar 29 18:16:21 raspberrypi kernel: [   14.962810] mmc0: DMA IRQ 6 ignored - results were reset
Note: I'm not claiming this is in any way related to rpi-update. I'll keep monitoring the syslog for signs of SD card related problems.

EdVoncken
Posts: 5
Joined: Sun Mar 17, 2013 5:00 pm

Re: Prevent SD-Card Corruption

Fri Mar 29, 2013 11:58 pm

kaspencer wrote:You may be misled into a false sense of security in a 7-day only judgement.
Ken
Yes, I fully agree. I'll have to do some more testing (putting load on the RPi) before making bold statements ;-)

seba
Posts: 5
Joined: Sat Mar 30, 2013 9:09 am

Re: Prevent SD-Card Corruption

Sat Mar 30, 2013 9:29 am

I've been having with SD card corruption. R.Pi is not and never was overclocked.

With the first card R.Pi worked for about 1 month, but it was never restarted. When it was, it didn't boot. Reflashed, but I was relatively quickly getting serious corruption. I've set up fsck to run every system boot and in rc.S to automatically fix errors. If I restarted many times in a short amount of time, it was ok no errors, but if I've waited a few days there were.
I did also 1 pass of memtest, no errors.
Flash drive tester for Windows showed errors in verify (read+write+verify) test.

I've bought a new card, before I've inserted into the R.Pi I did a read+write+verify test and it showed no errors. After 3 days it didn't boot anymore, before I rebooted I manually ran fsck and it showed errors.
Testing on windows and linux (badblocks -w) showed verify errors, badblocks read-only showed no errors.

It could be that the power supply is bad, I'll try changing it, also it could be that this is a coincidence and the SD card failed by chance.

There is a possibility however, that R.Pi is causing corruption of SD cards.

Power supply is a HTC charger, 1 A/5 V.
Card1: SP micro SD HC class 6, 4 GB
Card2: SDHC Sony SF-4B4 4 GB

seba
Posts: 5
Joined: Sat Mar 30, 2013 9:09 am

Re: Prevent SD-Card Corruption

Tue Apr 02, 2013 11:17 am

I've discovered that doing multiple low-level formatting with the tool found on sdcard.org or with the tool "HDD low level format" decreases the number of verify errors asymptotically. With the first card I've managed to almost have no errors, with the second I'm left with around 100 errors.

As for the R.Pi, I've switched to powering it with a computer PSU directly on the GPIO pins.
I have also added a 470 uF capacitor to the 3.3V rail of the R.Pi, that's around 10 times of what's there by default (47 uF + some 1-10 uF range caps + a lot of 100 nF ones).
Will see if this will work, I'm using now the first SD card.

Aydan
Posts: 560
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: Prevent SD-Card Corruption

Tue Apr 02, 2013 3:30 pm

kaspencer wrote: Now here is an interesting thing:
The Raspberry Pi that functions as a sort-of domain controller and fileserver does a backup nightly, of about 9.5Gbyte of key business data held with lots of other stuff on a 2TB USB disc. This backed-up data goes onto a separate ext4 partition on the SD Card, where it is held for four days and nights and is then deleted, to make way for the latest backup. This is a lot of reading and writing eh? But this partition has never been corrupted, notwithstanding all this activity. So the corruption of the SD Card, is clearly, in my view, something to do with what goes on in the root partition.
Due to how wear leveling works it doesn't matter on which partition data resides. Physical blocks are going to be assigned to all partitions on the card.
In your case I'd strongly encourage you to run fstrim on your backup partition after deleting the file to free up blocks for wear leveling.
Otherwise you'll deplete the wearleveling pool after a couple of backups.

Regards
Aydan

seba
Posts: 5
Joined: Sat Mar 30, 2013 9:09 am

Re: Prevent SD-Card Corruption

Wed Apr 10, 2013 1:09 pm

Didn't work, now I have ~5000 errors, unrepairable.

Maybe I should hook it up on an oscilloscope. hm.

kaspencer
Posts: 79
Joined: Wed Mar 07, 2012 11:37 pm
Location: UK, England, Wiltshire

Re: Prevent SD-Card Corruption

Mon Apr 15, 2013 5:16 pm

Hello Aydan and thanks for the comment.

I am reluctant to use fstrim in view of this:
http://www.raspberrypi.org/phpBB3/viewt ... 4&p=327161
How relevant do you see this?

Then I would add that some cards seem to be better than others in my various Raspberrys Pi. And then again some OS variants are also better than others with the different cards I have. I have a very wide range of cards (as reported elsewhere) but no cards seem to be exempt from corruption.

If, as seems to be the case, the SD Card technology is being used for a purpose for which it wasn't strictly intended and designed, perhaps this needs attention from the foundation, as it is obviously causing persistent, experienced and knowledgable computing persons quite alot of grief. I worry about the effects on those who come to computing for the first tome via the Raspberry Pi. It won't exactly encourage them.

Regards

Kenneth Spencer
2x256Mb + 2x512Mb RPi, Eth'netLAN+Win2k3R2 svr, 40MbpsFTTC.
RaspBMC: 128G SD, K400 wl Kb+TPd, HD32tv&Rem.
RW'y webserver: 64G SD, Y-RK49 wl Kb+M, HannsG W24" screen
RW'y PDC & fileserver: as above + 2TB disc.
+RiscOSPi on 32G uSD.

Aydan
Posts: 560
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: Prevent SD-Card Corruption

Mon Apr 15, 2013 9:39 pm

Hi Kenneth,
too bad fstrim doesn't work for you. One of my RasPis has been running since last july and I have it perform an fstrim nightly without any problems. It doesn't have too much data turnaround though.
In your case I'd recommend you use an extra drive (usbstick or some such) for your backup data and run fstrim on that, so fstrim doesn't corrupt your rootfs.
I'd also try to investigate why you're having problems.
Said RasPi uses an older 8GB Sandisk class 4 card. Maybe the cards you're using dont like the usage.
I'd recommend to not use class 10 cards since they seem to cause more problems than class 4 or class 6
If you don't want to use fstrim and don't want to use an extra drive, then you should probably leave at least 1GB unpartitioned, as a wear leveling reserve.

Regards
Aydan

kaspencer
Posts: 79
Joined: Wed Mar 07, 2012 11:37 pm
Location: UK, England, Wiltshire

Re: Prevent SD-Card Corruption

Mon Apr 15, 2013 10:51 pm

Thanks again, Aydan ...

I have Class 4, 6 and 10 cards ranging in size from 4GB to 128GB and they all have their tendency to corrupt. My solution is to have two cards always available for each of my three functional Raspberrys Pi. (One of my Pis is spare anyway and only used for investigations and program testing). When a card fails I can therefore easily swap a card over and have the machine back up and running is a few minutes.

But I do think that we have a serious little problem with SD Cards - just think some of us have worked with PCs for decades (35 years in my case and I first worked with Linux in 1992 - Yggdrasil from Stone Soup) and are not put off by little issues, but newcomers (that is, those at whom the machines are aimed) will easily get put off and give up.

All the best

Ken.
2x256Mb + 2x512Mb RPi, Eth'netLAN+Win2k3R2 svr, 40MbpsFTTC.
RaspBMC: 128G SD, K400 wl Kb+TPd, HD32tv&Rem.
RW'y webserver: 64G SD, Y-RK49 wl Kb+M, HannsG W24" screen
RW'y PDC & fileserver: as above + 2TB disc.
+RiscOSPi on 32G uSD.

rayperkins
Posts: 4
Joined: Fri Apr 19, 2013 8:53 am

Re: Prevent SD-Card Corruption

Fri Apr 19, 2013 9:05 am

Hi All
I have 4 very standard pis that corrupt their SD-Cards regularly. So I am in agreement with kaspencer that there is an issue that extends beyond card brands.
To be able to fix an issue we must first identify if there is one and the easiest way to do this would be to catch a corruption in the act of corrupting our poor cards.
Is anyone aware or suspects a method that causes the file system corruption under normal operation?
If we can repeatably cause the failures then we can identify what causes it.

Cheers
Ray

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 17886
Joined: Sat Jul 30, 2011 7:41 pm

Re: Prevent SD-Card Corruption

Fri Apr 19, 2013 10:43 am

Are you using the latest firmware/distro. I believe there was a change to the MMC/SD card stuff recently - whether that is related to the issue you are seeing I do not know.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

remsnet
Posts: 151
Joined: Wed Dec 19, 2012 7:32 pm
Location: Planet Gaia
Contact: Website Yahoo Messenger

Re: Prevent SD-Card Corruption

Fri Apr 19, 2013 11:24 am

dan_fritz wrote:I
..
..
What kernel release you have in use ?

currect rasppberry shuold be 3.6.11 at least.

remsnet
Posts: 151
Joined: Wed Dec 19, 2012 7:32 pm
Location: Planet Gaia
Contact: Website Yahoo Messenger

Re: Prevent SD-Card Corruption

Fri Apr 19, 2013 11:28 am

kaspencer wrote:Thanks again, Aydan ...

I have Class 4, 6 and 10 cards ranging in size from 4GB to 128GB and they all have their tendency to corrupt.
..
Ken.
class 6 and 10 wth size bigger the 32GB require at least 3.8.4 to run stable.

see my kenel hack post http://www.raspberrypi.org/phpBB3/viewt ... 87&t=40664

killkrt
Posts: 6
Joined: Fri Mar 29, 2013 12:59 pm

Re: Prevent SD-Card Corruption

Sun Apr 21, 2013 8:18 pm

Same problems here! :cry:

I have a 8GB Transcend SD card (bought on Amazon) class 10, I use a power supply bought on RS site (recommended for Raspberry) but I got corruption problems twice a month, even if I always shutdown correctly my Raspberry.

Sometimes it seems happen while is running! Just few minutes ago I was trying to open a file and suddenly I got corruption errors!

Maybe I'll try with a SD card class 4 (recommended brands?), but I'm quite pessimistic. :roll:

User avatar
hojnikb
Posts: 109
Joined: Mon Jun 04, 2012 3:59 pm
Location: @Home

Re: Prevent SD-Card Corruption

Mon Apr 22, 2013 9:27 pm

what about running rootfs off a usb stick ?
would that cause curroptions aswell ?
I had bunch of curroptions on my 16gb transcend card so i've moved my rootfs to usb stick. Hopefully this helps..
+°´°+,¸¸,+°´°~ Everyone should have a taste of UK Raspberry Pie =D ~°´°+,¸¸,+°´°+
Rasberry Pi, SoC @ 1225Mhz :o, 256MB Ram @ 550Mhz, 16GB SD-Card, Raspbian

benmadin
Posts: 2
Joined: Tue Mar 19, 2013 3:09 pm

Re: Prevent SD-Card Corruption

Thu Jun 06, 2013 4:56 am

rspitz wrote: Well, I bought the card together with my Pi directly from RS. I don't really believe they sell fake products.
The pre-installed card I bought from RS with the Pi didn't work when it arrived... which was disappointing.

However, I copied a newly downloaded Rasbpian image onto the same card (using dd on a Mac ;) ) and it has been working since (about four months now).

killkrt
Posts: 6
Joined: Fri Mar 29, 2013 12:59 pm

Re: Prevent SD-Card Corruption

Thu Jun 06, 2013 12:41 pm

I've fixed the problem buying, as already suggested by others, a SanDisk Class 4 SD card.
I made some tests with Class 6 SD card and I got many problems as with the Class 8 ones!

So I think if you want avoid problems you should buy Class 4 SD card.

JohanSA
Posts: 39
Joined: Mon Jun 17, 2013 8:08 am

Re: Prevent SD-Card Corruption

Sun Jun 23, 2013 9:46 pm

I use over 20 pis with the same software. Of those 20 i have had 4 corrupt SD
Cards. The SD cards are the same in all of them and they all have similar power supply..the corrupt SD cards is getting annoying.

The one card was working fine on the pi for three days. I did not touch it and power supply was consistent and yet is corrupted!

beruic
Posts: 28
Joined: Sun Jan 29, 2012 1:37 pm

Re: Prevent SD-Card Corruption

Thu Jun 27, 2013 11:26 am

I usually get corruption when updating Raspbian over SSH. Especially if I have two SSH connections open and have

Code: Select all

top
open in one and run the update in the other. I have two Pis that do this. One is rev. 1 and one is rev 2.

User avatar
RaTTuS
Posts: 9490
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Prevent SD-Card Corruption

Thu Jun 27, 2013 12:02 pm

Just to stick my ore in here
I've currently have 4 RPi's
one :-

Code: Select all

uprecords
     #               Uptime | System                                     Boot up
----------------------------+---------------------------------------------------
     1    71 days, 22:59:51 | Linux 3.6.11+             Tue Mar 12 08:37:44 2013
     2    30 days, 05:03:22 | Linux 3.2.27+             Wed Dec 12 12:26:21 2012
     3    27 days, 18:56:43 | Linux 3.6.11+             Thu May 23 17:35:40 2013
     4    22 days, 23:26:51 | Linux 3.2.27+             Sat Nov 17 17:55:38 2012
     5    18 days, 05:55:29 | Linux 3.1.9+              Fri Aug 10 11:23:26 2012
     6    17 days, 00:47:44 | Linux 3.6.11+             Mon Feb 11 16:24:05 2013
     7    13 days, 20:10:25 | Linux 3.2.27+             Thu Sep 20 16:53:12 2012
     8    11 days, 04:54:32 | Linux 3.2.27+             Fri Oct 19 08:39:20 2012
     9    11 days, 03:04:14 | Linux 3.2.27+             Thu Jan 24 13:13:37 2013
    10     9 days, 21:11:51 | Linux 3.2.27+             Fri Aug 31 14:48:28 2012
----------------------------+---------------------------------------------------
->  15     6 days, 19:08:31 | Linux 3.6.11+             Thu Jun 20 17:30:57 2013
----------------------------+---------------------------------------------------
1up in     0 days, 01:14:39 | at                        Thu Jun 27 13:54:06 2013
t10 in     3 days, 02:03:21 | at                        Sun Jun 30 14:42:48 2013
no1 in    65 days, 03:51:21 | at                        Sat Aug 31 16:30:48 2013
    up   326 days, 20:43:47 | since                     Tue Jul 17 12:46:07 2012
  down    18 days, 03:09:34 | since                     Tue Jul 17 12:46:07 2012
   %up               94.744 | since                     Tue Jul 17 12:46:07 2012
this is an old 256MB version with it's original 8GB Class 6 SD card run from a mins USB plug powering a texy LCD display and running a HDMI monitor in portrait mode
regularly updated - not as often rebooted
it will lock up every now and again - needing to be power cycled but I'ved never had it corrupt the SD card

another one

Code: Select all

 uprecords
     #               Uptime | System                                     Boot up
----------------------------+---------------------------------------------------
     1    72 days, 21:43:04 | Linux 3.6.11+             Fri Mar  1 12:24:00 2013
     2    20 days, 21:24:58 | Linux 3.2.27+             Thu Feb  7 12:16:50 2013
     3    13 days, 02:57:41 | Linux 3.6.11+             Thu Jun  6 09:30:02 2013
     4    12 days, 05:36:26 | Linux 3.6.11+             Fri May 17 10:31:54 2013
->   5     8 days, 00:15:13 | Linux 3.6.11+             Wed Jun 19 12:29:41 2013
     6     6 days, 04:43:05 | Linux 3.6.11+             Thu May 30 12:37:07 2013
     7     2 days, 03:19:35 | Linux 3.6.11+             Mon May 13 10:16:49 2013
     8     1 day , 19:55:38 | Linux 3.6.11+             Wed May 15 14:31:09 2013
     9     0 days, 20:13:58 | Linux 3.6.11+             Wed May 29 16:23:02 2013
    10     0 days, 00:10:04 | Linux 3.6.11+             Wed May 29 16:08:08 2013
----------------------------+---------------------------------------------------
1up in     4 days, 05:21:14 | at                        Mon Jul  1 18:06:06 2013
no1 in    64 days, 21:27:52 | at                        Sat Aug 31 10:12:44 2013
    up   138 days, 04:19:42 | since                     Thu Feb  7 12:16:50 2013
  down     1 day , 19:08:22 | since                     Thu Feb  7 12:16:50 2013
   %up               98.716 | since                     Thu Feb  7 12:16:50 2013
This one again is an original 256MB model and has an 8GB SD [class 10] card and a 64GB SSd with the rootfs on that - this is powered by a cheep USB hub which also powered the SSD again no SD card corruption this one has never locked up as far as I know
a third

Code: Select all

 uprecords
     #               Uptime | System                                     Boot up
----------------------------+---------------------------------------------------
     1    94 days, 18:18:26 | Linux 3.2.27+             Fri Feb  8 14:57:57 2013
     2    33 days, 19:22:26 | Linux 3.2.27+             Thu Dec 13 15:19:43 2012
     3    30 days, 18:04:22 | Linux 3.1.9+              Fri Jul 20 16:59:35 2012
     4    25 days, 04:38:26 | Linux 3.2.27+             Sun Nov 18 10:41:09 2012
     5    16 days, 17:37:40 | Linux 3.2.27+             Fri Aug 24 16:10:54 2012
     6    14 days, 20:13:22 | Linux 3.2.27+             Fri Nov  2 13:39:18 2012
     7    14 days, 04:09:16 | Linux 3.6.11+             Tue May 28 12:04:26 2013
     8    14 days, 01:01:47 | Linux 3.2.27+             Mon Oct  8 09:33:30 2012
     9    12 days, 23:20:15 | Linux 3.2.27+             Tue Sep 25 10:09:04 2012
    10    12 days, 00:33:04 | Linux 3.2.27+             Thu Jan 24 10:59:02 2013
----------------------------+---------------------------------------------------
->  13     7 days, 19:54:03 | Linux 3.6.11+             Wed Jun 19 16:56:25 2013
----------------------------+---------------------------------------------------
1up in     0 days, 00:02:18 | at                        Thu Jun 27 12:52:44 2013
t10 in     4 days, 04:39:02 | at                        Mon Jul  1 17:29:28 2013
no1 in    86 days, 22:24:24 | at                        Sun Sep 22 11:14:50 2013
    up   334 days, 00:42:10 | since                     Tue Jul 17 11:45:03 2012
  down    11 days, 00:23:15 | since                     Tue Jul 17 11:45:03 2012
   %up               96.807 | since                     Tue Jul 17 11:45:03 2012
is a 512MB version with 4GB class 4 SD card with a 640GB hard disk + RPI camera powered by the USB disk adapter , no issues with this one either
however I'm not 100% sure on the uprecords as this may of been a cloned SD card
the only time I've had sd card corruption is when we had some updates [rpi-update] that where responsible
mainly rebooted to move the camera etc...

the 4th is a model A + camera and Texy lcd shield + 16GB class 10 - no issues

I've got some 16GB , and 32Gb random class 4/ 6 / 10 cards from ebay that I may use to test if I get some time to play.
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

3dcircle
Posts: 11
Joined: Sun Jun 23, 2013 9:59 pm

Re: Prevent SD-Card Corruption

Sat Jun 29, 2013 3:59 pm

By observation of the green act light I notice that the Pi writes to the SD card for about 20 seconds after sudo halt enter. Removing power during this 20 seconds could result in SD corruption.

Return to “Troubleshooting”

Who is online

Users browsing this forum: baggyg, deady1000, quebec and 60 guests