johnspackman
Posts: 16
Joined: Tue Jul 16, 2013 9:29 am

Re: Prevent SD-Card Corruption

Tue Sep 08, 2015 11:21 am

ronald71 wrote:I have 3 Pi boards. Pi1 and Pi2. All develop SD card errors after some time.
One thing I'm strict about is not reusing cards; if a box is RMA'd for any reason (we do one month trials and sometimes they get returned) then the card is binned. IMHO if you've had corruption on a card then it's suspect because you don't know the exact cause, just that the net effect was that your filing system got corrupted, and while you have battery back up *now* and don't have power offs, that doesn't mean that the cards are undamaged.
ronald71 wrote:This is very good computer but for home use, learning and for the Linux fans but not for the production environment
It certainly needs some care in order to use it in a production environment but if you look around you'll find that quite a few people *do* use it production, successfully and reliably.

idecable
Posts: 1
Joined: Sat Dec 26, 2015 5:29 am

Re: Prevent SD-Card Corruption

Sat Dec 26, 2015 5:32 am

I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.

gkreidl
Posts: 5323
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Prevent SD-Card Corruption

Sat Dec 26, 2015 1:46 pm

idecable wrote:I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.
Get a small USB HDD. With a good power supply (and the max_usb_current=1 setting) you can connect it directly to the B+/Pi2. Move the OS to the HDD. Works like a charm and much faster than from SD - and no SD card corruption any more.
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

User avatar
SFX Group
Posts: 74
Joined: Thu Nov 01, 2012 3:54 am
Location: Cambridge, ON. Canada
Contact: Website

Re: Prevent SD-Card Corruption

Sat Dec 26, 2015 5:56 pm

idecable wrote:I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.

We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
Many Thanks
Ashley

© Copyright 2015 - www.ashleygriffin.ca

gkreidl
Posts: 5323
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Prevent SD-Card Corruption

Sat Dec 26, 2015 7:18 pm

SFX Group wrote:
We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
Are you using it for 5.6 years and running from SD card?
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

User avatar
SFX Group
Posts: 74
Joined: Thu Nov 01, 2012 3:54 am
Location: Cambridge, ON. Canada
Contact: Website

Re: Prevent SD-Card Corruption

Sun Dec 27, 2015 12:48 am

gkreidl wrote:
SFX Group wrote:
We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
Are you using it for 5.6 years and running from SD card?
Intel version has 32Gb of SSD built in, so there's no need for an SD card, with a MTBF of over 5 years it was a bit of an easy decision, we havnt had any issues as yet, it boots Windows 10 faster than the PI does (it has a faster CPU) and only uses between 1-4watts of power (configurable). We have some running KODI (XBMC) and it runs fine. There's a new version coming out first quarter of 2016 that has 4K HDMI as well (i think with 64 or 128Gb SSD build in).
Many Thanks
Ashley

© Copyright 2015 - www.ashleygriffin.ca

happnatious1
Posts: 1
Joined: Thu Apr 21, 2016 12:59 am

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 1:07 am

I use raspberry pi's for commercial signage because they are small and can be mounted to the back of a TV. But they are stand alone devices and power outages are a reality of life. Every few months I have to go re-burn an SD card that is corrupt somewhere. There has to be a way for these devices to handle unexpected power losses or they are doomed to the annals of history.

johnspackman
Posts: 16
Joined: Tue Jul 16, 2013 9:29 am

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 6:13 am

We do commercial signage also - I avoid the problem by having a read only root partition, a ramdisk for data and a read only partition for videos; periodically sync the ramdisk to a normally read-only data partition, and only make the media partition read/write when downloading new videos.

We used to use PCs runnings Windows XP for this, and with SD drives had a similar problem when users turned them off - power failures would corrupt the drive.

User avatar
SFX Group
Posts: 74
Joined: Thu Nov 01, 2012 3:54 am
Location: Cambridge, ON. Canada
Contact: Website

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 6:37 am

We no longer use PI for any hardware, as it was a great idea that didnt work in real life usage.

I notice HomeSeer are now using PI 2, they are having MAJOR issues with damaged SD cards, they have only been using them about 7 months, there is talk of moving away due to the SD card issue (it eats the card and they wont give out the OS so once its a year old, people will have to buy another one, this will damage there reputation.

There forums are full of problems with it, they have advised backing up the SD card which means taking the PI apart (as its inside), technically voids the warranty but they are aware there's a huge problem with this.
Many Thanks
Ashley

© Copyright 2015 - www.ashleygriffin.ca

rln
Posts: 141
Joined: Wed Apr 09, 2014 1:43 pm
Location: Sweden
Contact: Website

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 7:43 am

You may be interested in my project Nard SDK. It prevents SD card corruption by simply not using it! Nard runs the entire OS from RAM. After bootup you can even remove the SD card and the system will continue to operate just fine.
http://www.arbetsmyra.dyndns.org/nard/
Author of the robust Nard distro http://www.nard.se

johnspackman
Posts: 16
Joined: Tue Jul 16, 2013 9:29 am

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 7:52 am

Interesting - please can you expand on "Automatic repair of SD-card filesystems in the event of power loss during writing"? Have you had much success with this, I've tended to find that filing systems are largely unrecoverable; espcially in early Pi v1 days - specifically, something got corrupted on the boot partition

rln
Posts: 141
Joined: Wed Apr 09, 2014 1:43 pm
Location: Sweden
Contact: Website

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 8:38 am

johnspackman wrote:expand on "Automatic repair of SD-card filesystems in the event of power loss during writing"? Have you had much success with this
First of all Nard tries to prevent all SD card writes. It's only during an OS upgrade writes do occur and during this processes care is taken to write things in an order to minimize FAT corruption should a power cut happen. Then, at each boot, the filesystem is always checked for errors (fsck).

Naturally though it's not bullet prof. If your custom application writes huge amounts of data the SD card will eventually become corrupt anyway. HOWEVER, even in such a situation it possible to salvage. I've successfully been able to repair a remote unit where the SD card had broken down. Since Nard operates fine without SD card I could still login via SSH to the remote unit and re-write an entire new OS image to the SD-card. After reboot the unit worked fine.

Additionally Nard use F2FS instead of ext4.
Author of the robust Nard distro http://www.nard.se

User avatar
patrikg
Posts: 168
Joined: Sun Mar 18, 2012 10:19 pm

Re: Prevent SD-Card Corruption

Thu Apr 21, 2016 8:53 am

rln wrote:You may be interested in my project Nard SDK. It prevents SD card corruption by simply not using it! Nard runs the entire OS from RAM. After bootup you can even remove the SD card and the system will continue to operate just fine.
http://www.arbetsmyra.dyndns.org/nard/
Now we will discuss this again.
Have some old threads about this
viewtopic.php?f=29&t=142626
viewtopic.php?f=28&t=88403

But this is great, i love this.
I use two another ram based linux systems daily.
On pi's and another embedded based hardware.

OpenWRT https://openwrt.org/
Tiny Core Linux (PiCore) http://tinycorelinux.net/ports.html

MaxK1
Posts: 922
Joined: Sun Aug 26, 2012 11:34 pm

Re: Prevent SD-Card Corruption

Tue May 10, 2016 8:20 pm

Earlier today I ran into a specsheet for an embedded system that offered to ship with their "standard" SD card or an industrial grade card. The standard card was spec'ed at -25'C - +60'C while the industrial card was speced at -40'C - +80'C. Then I stumbled on this page:

https://www.sdcard.org/press/thoughtlea ... cards.html

Typically, when I buy an SD card, the operating temp. range isn't listed.
You are in a maze of twisty little passages, all alike.
When General Failure and Major Disaster get together, Private Parts usually suffers.

boyoh
Posts: 1046
Joined: Fri Nov 23, 2012 3:30 pm

Re: Prevent SD-Card Corruption

Tue May 10, 2016 10:01 pm

I think the method of removing the power from the Pi, could be cause problems
Removing the power from the AC in/put side of the adapter , could leave high
Inductive energy spikes flowing to the Pi ,, if the power adapter is not well filtered

When switching off on the AC side, you are switching at random of the AC cycle.
Might be best to to isolate on the DC side to the Pi, May be fitting a inline switch
Or removing the Micro USB lead. This is only a possibility,
BoyOh ( Selby, North Yorkshire.UK)
Some Times Right Some Times Wrong

i486
Posts: 131
Joined: Sun Aug 28, 2016 3:41 pm
Location: BG

Re: Prevent SD-Card Corruption

Wed Oct 26, 2016 1:42 pm

SFX Group wrote:...What we found is the most cards would pass a READ test, however where unable to WRITE to the card, we are actually thinking there is some hardware fault that is burning out the cards write capacity for what ever reason...
I had similar scenarios with SD cards and found a solution to "revive" it. Not sure whether it is temporary or stable, but the card become writable and can be formatted without problem. All you need to do is to issue ERASE command (from SDIO command set, this is CMD42 with 1 byte parameter 0x08 = ERASE). I use Transcend RDF5 card reader with its custom software RecoveRx (Security tab). Note that you have to set some password first (Lock) to be able to do Erase. I hope this may help to save many SD cards from their way to trash.

oionos75
Posts: 1
Joined: Fri Dec 30, 2016 1:57 pm

Re: Prevent SD-Card Corruption

Fri Dec 30, 2016 2:49 pm

I experience the same corrupt Disk problem which occurs extremely often.
As for my experience it is not an SD-Card only problem.
I have 3 Raspberry pi A+ and 3 Raspberry pi 3
I use all kinds of SD Cards but also disks but usually 32GB cards for all my pis
One of these Raspberry Pis also has a 500 GB USB HDD. This Raspberry Pi 3 has the problem more often than the rest and both the SD-Card and the HDD get corrupt.

The problem appears, as for my experience, only with the more fast Raspberry pis.
I have never experienced a Raspberry pi A+ SD-card corruption.
I started experiencing this problem after passing to Raspbian Jessie.

That is why I think that it is not a SD-Card related problem but a synchronisation problem of Raspbian

I have applied without success the prevent methods of Drscheme :

1) Check the file system every three reboots:

Code: Select all

tune2fs -c 3 /dev/mmcblkp02
2) Allow auto correction:

Code: Select all

nano /etc/default/rcS --> FSCKFIX=yes
I have tried some other methods such as:
regularly schedule/force a reboot and
schedule a sync
(both using cron)
The read-only filesystem method is not good for the use i intend for my Raspberry pis.

I've checked the power fluctuation issue but in my case it does not apply

So if anyone has an effective solution to prevent this disk corruption I'd like to test it

i486
Posts: 131
Joined: Sun Aug 28, 2016 3:41 pm
Location: BG

Re: Prevent SD-Card Corruption

Sat Dec 31, 2016 2:44 pm

SFX Group wrote:
idecable wrote:I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.

We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
35$ vs 150$ and this:
http://gizmodo.com/intel-compute-stick- ... 1699377058

johnspackman
Posts: 16
Joined: Tue Jul 16, 2013 9:29 am

Re: Prevent SD-Card Corruption

Tue Jan 03, 2017 10:14 am

oionos75 wrote:I experience the same corrupt Disk problem which occurs extremely often.
As for my experience it is not an SD-Card only problem.
I have 3 Raspberry pi A+ and 3 Raspberry pi 3
I use all kinds of SD Cards but also disks but usually 32GB cards for all my pis
One of these Raspberry Pis also has a 500 GB USB HDD. This Raspberry Pi 3 has the problem more often than the rest and both the SD-Card and the HDD get corrupt.

The problem appears, as for my experience, only with the more fast Raspberry pis.
I have never experienced a Raspberry pi A+ SD-card corruption.
I started experiencing this problem after passing to Raspbian Jessie.

That is why I think that it is not a SD-Card related problem but a synchronisation problem of Raspbian

I have applied without success the prevent methods of Drscheme :

1) Check the file system every three reboots:

Code: Select all

tune2fs -c 3 /dev/mmcblkp02
2) Allow auto correction:

Code: Select all

nano /etc/default/rcS --> FSCKFIX=yes
I have tried some other methods such as:
regularly schedule/force a reboot and
schedule a sync
(both using cron)
The read-only filesystem method is not good for the use i intend for my Raspberry pis.

I've checked the power fluctuation issue but in my case it does not apply

So if anyone has an effective solution to prevent this disk corruption I'd like to test it
AFAIK the only corruption issue ever reported is on SD cards, and that issue was due to the nature of how SD cards work - it categorically would not happen on a mechanical HDD.

Therefore if you are getting corruption on a mechanical hard drive then you have other problems, I would guess related to power issues. Have you built your own kernel, or used a filing system other than ext2/3/4?

gommaloano
Posts: 7
Joined: Wed Feb 17, 2016 9:20 am

Re: Prevent SD-Card Corruption

Fri Jul 21, 2017 9:52 am

Maybe it's worth considering f2fs that should decrease filesystem corruption issues
What are you kidding?

gommaloano
Posts: 7
Joined: Wed Feb 17, 2016 9:20 am

Re: Prevent SD-Card Corruption

Fri Jul 21, 2017 9:53 am

Maybe it's worth considering f2fs that should decrease filesystem corruption issues
What are you kidding?
malakai wrote:Corruption generally occurs when writing data to the card. The influencing factors are power loss and over clocking.

If you loose power when writing to the SD Card it can corrupt the card. This is typically due to cutting power during shut down, power on, etc.. When the Pi is up and running a lot is happening in memory and no writes are committed so power loss is not a guarantee of corruption when the Pi is running.

Extreme overclocking can corrupt the data as it's being written to the card if you experience corruption with over clocking try backing off or down your overclock settings.

One option is to move everything but the boot off the SD Card however if while writing a file to the other media both examples can still corrupt that file.

digger72
Posts: 63
Joined: Wed Feb 24, 2016 12:40 pm

Re: Prevent SD-Card Corruption

Fri Dec 01, 2017 11:01 pm

A Hardware solution will consist of a SuperCap, that will have enough juice to issue a shutdown command when pull chord is detected. See my latest blog about it here - http://referencedesigner.com/blog/raspb ... tion/6662/

User avatar
HawaiianPi
Posts: 1407
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Prevent SD-Card Corruption

Sat Dec 02, 2017 2:02 am

digger72 wrote:
Fri Dec 01, 2017 11:01 pm
A Hardware solution will consist of a SuperCap, that will have enough juice to issue a shutdown command when pull chord is detected. See my latest blog about it here - http://referencedesigner.com/blog/raspb ... tion/6662/
This seems like a whole lot of trouble over a problem that shouldn't even exist. Use good quality micro-USB cables that won't "wiggle loose" and don't pull the plug without a proper shutdown. File corruption due to power interruption is not unique to Raspberry Pi computers. It will happen with Windows and Mac computers as well. Treat your Pi like a computer instead of a toy and you'll have far fewer problems.
My password is the last 8 digits of Pi.

lost
Posts: 9
Joined: Tue Dec 05, 2017 9:38 am

Re: Prevent SD-Card Corruption

Tue Dec 05, 2017 10:06 am

HawaiianPi wrote:
Sat Dec 02, 2017 2:02 am
This seems like a whole lot of trouble over a problem that shouldn't even exist. Use good quality micro-USB cables that won't "wiggle loose" and don't pull the plug without a proper shutdown. File corruption due to power interruption is not unique to Raspberry Pi computers. It will happen with Windows and Mac computers as well. Treat your Pi like a computer instead of a toy and you'll have far fewer problems.
The problem with PI is the storage: Consumer grade SD storage controllers are crap, their firmware use dynamic wear leveling instead of static thus creating hot-spots & for MLC NAND storage with its underlying flash sectors only supporting ~3k erase cycles (was 100k for former SLC based flash), it's quickly weared. For SSD, with decent firmwares, you can get an oversized one for your needs to have a better endurance thanks to better leveling but this will be mostly inefficient for SD that are unable to distribute wear efficiently by explicitly moving static data.

PI fundation should have made a M.2 sata port or, if too costly, a eUSB one: eUSB storage is used for embedded devices (lot of telecom equipment use it) and provide better hardware (multi channel NAND to avoid buffering & loss of power issues + capacitors to be able to end on the fly transactions cleanly) and firmware (good wear levelling, data retention monitoring etc...).

And a way for everybody to be able to buy devices on a 1 by 1 basis, as this kind of devices are only available for professional customers & volume price.

You just cannot say this is a user problem. Learn a bit about how flash storage work & you'll change your mind.

I talked about eUSB, but some industry grade SD/uSD are also available (even if probably less reliable than eUSB). But the problem remains: How to buy them?

On top of that, to avoid cost of SLC based devices (mass flash production moved to MLC, so SLC is still the best but cost are high), some manufacturer provide so called pSLC (pseudi-SLC = MLC used as SLC ; using only one cell load level comparator => 30k erase cycles endurance due to better margin for a cost/bit only doubled & still benefit from MLC price).

See for instance swissbit S46u series.

So this can be fixed & having a bulky external HDD can be avoided. But Raspi foundation have to get involved in providing a way for individuals to buy devices that are made for 24/7 use.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 18206
Joined: Sat Jul 30, 2011 7:41 pm

Re: Prevent SD-Card Corruption

Tue Dec 05, 2017 12:08 pm

It's such a massive problem that we've only sold 17M devices.

All the suggestions above are attempting to fix a minor problem, and all cost a hat load of money.

SD card are cheap, and generally work fine.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

Return to “Troubleshooting”

Who is online

Users browsing this forum: No registered users and 39 guests