Page 8 of 10

Re: Prevent SD-Card Corruption

Posted: Tue Sep 08, 2015 11:21 am
by johnspackman
ronald71 wrote:I have 3 Pi boards. Pi1 and Pi2. All develop SD card errors after some time.
One thing I'm strict about is not reusing cards; if a box is RMA'd for any reason (we do one month trials and sometimes they get returned) then the card is binned. IMHO if you've had corruption on a card then it's suspect because you don't know the exact cause, just that the net effect was that your filing system got corrupted, and while you have battery back up *now* and don't have power offs, that doesn't mean that the cards are undamaged.
ronald71 wrote:This is very good computer but for home use, learning and for the Linux fans but not for the production environment
It certainly needs some care in order to use it in a production environment but if you look around you'll find that quite a few people *do* use it production, successfully and reliably.

Re: Prevent SD-Card Corruption

Posted: Sat Dec 26, 2015 5:32 am
by idecable
I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.

Re: Prevent SD-Card Corruption

Posted: Sat Dec 26, 2015 1:46 pm
by gkreidl
idecable wrote:I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.
Get a small USB HDD. With a good power supply (and the max_usb_current=1 setting) you can connect it directly to the B+/Pi2. Move the OS to the HDD. Works like a charm and much faster than from SD - and no SD card corruption any more.

Re: Prevent SD-Card Corruption

Posted: Sat Dec 26, 2015 5:56 pm
by SFX Group
idecable wrote:I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.

We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7

Re: Prevent SD-Card Corruption

Posted: Sat Dec 26, 2015 7:18 pm
by gkreidl
SFX Group wrote:
We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
Are you using it for 5.6 years and running from SD card?

Re: Prevent SD-Card Corruption

Posted: Sun Dec 27, 2015 12:48 am
by SFX Group
gkreidl wrote:
SFX Group wrote:
We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
Are you using it for 5.6 years and running from SD card?
Intel version has 32Gb of SSD built in, so there's no need for an SD card, with a MTBF of over 5 years it was a bit of an easy decision, we havnt had any issues as yet, it boots Windows 10 faster than the PI does (it has a faster CPU) and only uses between 1-4watts of power (configurable). We have some running KODI (XBMC) and it runs fine. There's a new version coming out first quarter of 2016 that has 4K HDMI as well (i think with 64 or 128Gb SSD build in).

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 1:07 am
by happnatious1
I use raspberry pi's for commercial signage because they are small and can be mounted to the back of a TV. But they are stand alone devices and power outages are a reality of life. Every few months I have to go re-burn an SD card that is corrupt somewhere. There has to be a way for these devices to handle unexpected power losses or they are doomed to the annals of history.

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 6:13 am
by johnspackman
We do commercial signage also - I avoid the problem by having a read only root partition, a ramdisk for data and a read only partition for videos; periodically sync the ramdisk to a normally read-only data partition, and only make the media partition read/write when downloading new videos.

We used to use PCs runnings Windows XP for this, and with SD drives had a similar problem when users turned them off - power failures would corrupt the drive.

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 6:37 am
by SFX Group
We no longer use PI for any hardware, as it was a great idea that didnt work in real life usage.

I notice HomeSeer are now using PI 2, they are having MAJOR issues with damaged SD cards, they have only been using them about 7 months, there is talk of moving away due to the SD card issue (it eats the card and they wont give out the OS so once its a year old, people will have to buy another one, this will damage there reputation.

There forums are full of problems with it, they have advised backing up the SD card which means taking the PI apart (as its inside), technically voids the warranty but they are aware there's a huge problem with this.

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 7:43 am
by rln
You may be interested in my project Nard SDK. It prevents SD card corruption by simply not using it! Nard runs the entire OS from RAM. After bootup you can even remove the SD card and the system will continue to operate just fine.
http://www.arbetsmyra.dyndns.org/nard/

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 7:52 am
by johnspackman
Interesting - please can you expand on "Automatic repair of SD-card filesystems in the event of power loss during writing"? Have you had much success with this, I've tended to find that filing systems are largely unrecoverable; espcially in early Pi v1 days - specifically, something got corrupted on the boot partition

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 8:38 am
by rln
johnspackman wrote:expand on "Automatic repair of SD-card filesystems in the event of power loss during writing"? Have you had much success with this
First of all Nard tries to prevent all SD card writes. It's only during an OS upgrade writes do occur and during this processes care is taken to write things in an order to minimize FAT corruption should a power cut happen. Then, at each boot, the filesystem is always checked for errors (fsck).

Naturally though it's not bullet prof. If your custom application writes huge amounts of data the SD card will eventually become corrupt anyway. HOWEVER, even in such a situation it possible to salvage. I've successfully been able to repair a remote unit where the SD card had broken down. Since Nard operates fine without SD card I could still login via SSH to the remote unit and re-write an entire new OS image to the SD-card. After reboot the unit worked fine.

Additionally Nard use F2FS instead of ext4.

Re: Prevent SD-Card Corruption

Posted: Thu Apr 21, 2016 8:53 am
by patrikg
rln wrote:You may be interested in my project Nard SDK. It prevents SD card corruption by simply not using it! Nard runs the entire OS from RAM. After bootup you can even remove the SD card and the system will continue to operate just fine.
http://www.arbetsmyra.dyndns.org/nard/
Now we will discuss this again.
Have some old threads about this
viewtopic.php?f=29&t=142626
viewtopic.php?f=28&t=88403

But this is great, i love this.
I use two another ram based linux systems daily.
On pi's and another embedded based hardware.

OpenWRT https://openwrt.org/
Tiny Core Linux (PiCore) http://tinycorelinux.net/ports.html

Re: Prevent SD-Card Corruption

Posted: Tue May 10, 2016 8:20 pm
by MaxK1
Earlier today I ran into a specsheet for an embedded system that offered to ship with their "standard" SD card or an industrial grade card. The standard card was spec'ed at -25'C - +60'C while the industrial card was speced at -40'C - +80'C. Then I stumbled on this page:

https://www.sdcard.org/press/thoughtlea ... cards.html

Typically, when I buy an SD card, the operating temp. range isn't listed.

Re: Prevent SD-Card Corruption

Posted: Tue May 10, 2016 10:01 pm
by boyoh
I think the method of removing the power from the Pi, could be cause problems
Removing the power from the AC in/put side of the adapter , could leave high
Inductive energy spikes flowing to the Pi ,, if the power adapter is not well filtered

When switching off on the AC side, you are switching at random of the AC cycle.
Might be best to to isolate on the DC side to the Pi, May be fitting a inline switch
Or removing the Micro USB lead. This is only a possibility,

Re: Prevent SD-Card Corruption

Posted: Wed Oct 26, 2016 1:42 pm
by i486
SFX Group wrote:...What we found is the most cards would pass a READ test, however where unable to WRITE to the card, we are actually thinking there is some hardware fault that is burning out the cards write capacity for what ever reason...
I had similar scenarios with SD cards and found a solution to "revive" it. Not sure whether it is temporary or stable, but the card become writable and can be formatted without problem. All you need to do is to issue ERASE command (from SDIO command set, this is CMD42 with 1 byte parameter 0x08 = ERASE). I use Transcend RDF5 card reader with its custom software RecoveRx (Security tab). Note that you have to set some password first (Lock) to be able to do Erase. I hope this may help to save many SD cards from their way to trash.

Re: Prevent SD-Card Corruption

Posted: Fri Dec 30, 2016 2:49 pm
by oionos75
I experience the same corrupt Disk problem which occurs extremely often.
As for my experience it is not an SD-Card only problem.
I have 3 Raspberry pi A+ and 3 Raspberry pi 3
I use all kinds of SD Cards but also disks but usually 32GB cards for all my pis
One of these Raspberry Pis also has a 500 GB USB HDD. This Raspberry Pi 3 has the problem more often than the rest and both the SD-Card and the HDD get corrupt.

The problem appears, as for my experience, only with the more fast Raspberry pis.
I have never experienced a Raspberry pi A+ SD-card corruption.
I started experiencing this problem after passing to Raspbian Jessie.

That is why I think that it is not a SD-Card related problem but a synchronisation problem of Raspbian

I have applied without success the prevent methods of Drscheme :

1) Check the file system every three reboots:

Code: Select all

tune2fs -c 3 /dev/mmcblkp02
2) Allow auto correction:

Code: Select all

nano /etc/default/rcS --> FSCKFIX=yes
I have tried some other methods such as:
regularly schedule/force a reboot and
schedule a sync
(both using cron)
The read-only filesystem method is not good for the use i intend for my Raspberry pis.

I've checked the power fluctuation issue but in my case it does not apply

So if anyone has an effective solution to prevent this disk corruption I'd like to test it

Re: Prevent SD-Card Corruption

Posted: Sat Dec 31, 2016 2:44 pm
by i486
SFX Group wrote:
idecable wrote:I have two rpi B+ and one rpi2. I've burnt over 5 sd cards this year. Tried different power supplies, it didn't changed anything.

I've only used genuine brandname sdcards. I'm running out of ideas.

We sold all Pi hardware and moved to the Intel Compute Stick... not had any problems, the MTBF of Intel is about 5.5 years at running 24/7
35$ vs 150$ and this:
http://gizmodo.com/intel-compute-stick- ... 1699377058

Re: Prevent SD-Card Corruption

Posted: Tue Jan 03, 2017 10:14 am
by johnspackman
oionos75 wrote:I experience the same corrupt Disk problem which occurs extremely often.
As for my experience it is not an SD-Card only problem.
I have 3 Raspberry pi A+ and 3 Raspberry pi 3
I use all kinds of SD Cards but also disks but usually 32GB cards for all my pis
One of these Raspberry Pis also has a 500 GB USB HDD. This Raspberry Pi 3 has the problem more often than the rest and both the SD-Card and the HDD get corrupt.

The problem appears, as for my experience, only with the more fast Raspberry pis.
I have never experienced a Raspberry pi A+ SD-card corruption.
I started experiencing this problem after passing to Raspbian Jessie.

That is why I think that it is not a SD-Card related problem but a synchronisation problem of Raspbian

I have applied without success the prevent methods of Drscheme :

1) Check the file system every three reboots:

Code: Select all

tune2fs -c 3 /dev/mmcblkp02
2) Allow auto correction:

Code: Select all

nano /etc/default/rcS --> FSCKFIX=yes
I have tried some other methods such as:
regularly schedule/force a reboot and
schedule a sync
(both using cron)
The read-only filesystem method is not good for the use i intend for my Raspberry pis.

I've checked the power fluctuation issue but in my case it does not apply

So if anyone has an effective solution to prevent this disk corruption I'd like to test it
AFAIK the only corruption issue ever reported is on SD cards, and that issue was due to the nature of how SD cards work - it categorically would not happen on a mechanical HDD.

Therefore if you are getting corruption on a mechanical hard drive then you have other problems, I would guess related to power issues. Have you built your own kernel, or used a filing system other than ext2/3/4?

Re: Prevent SD-Card Corruption

Posted: Fri Jul 21, 2017 9:52 am
by gommaloano
Maybe it's worth considering f2fs that should decrease filesystem corruption issues
What are you kidding?

Re: Prevent SD-Card Corruption

Posted: Fri Jul 21, 2017 9:53 am
by gommaloano
Maybe it's worth considering f2fs that should decrease filesystem corruption issues
What are you kidding?
malakai wrote:Corruption generally occurs when writing data to the card. The influencing factors are power loss and over clocking.

If you loose power when writing to the SD Card it can corrupt the card. This is typically due to cutting power during shut down, power on, etc.. When the Pi is up and running a lot is happening in memory and no writes are committed so power loss is not a guarantee of corruption when the Pi is running.

Extreme overclocking can corrupt the data as it's being written to the card if you experience corruption with over clocking try backing off or down your overclock settings.

One option is to move everything but the boot off the SD Card however if while writing a file to the other media both examples can still corrupt that file.

Re: Prevent SD-Card Corruption

Posted: Fri Dec 01, 2017 11:01 pm
by digger72
A Hardware solution will consist of a SuperCap, that will have enough juice to issue a shutdown command when pull chord is detected. See my latest blog about it here - http://referencedesigner.com/blog/raspb ... tion/6662/

Re: Prevent SD-Card Corruption

Posted: Sat Dec 02, 2017 2:02 am
by HawaiianPi
digger72 wrote:
Fri Dec 01, 2017 11:01 pm
A Hardware solution will consist of a SuperCap, that will have enough juice to issue a shutdown command when pull chord is detected. See my latest blog about it here - http://referencedesigner.com/blog/raspb ... tion/6662/
This seems like a whole lot of trouble over a problem that shouldn't even exist. Use good quality micro-USB cables that won't "wiggle loose" and don't pull the plug without a proper shutdown. File corruption due to power interruption is not unique to Raspberry Pi computers. It will happen with Windows and Mac computers as well. Treat your Pi like a computer instead of a toy and you'll have far fewer problems.

Re: Prevent SD-Card Corruption

Posted: Tue Dec 05, 2017 10:06 am
by lost
HawaiianPi wrote:
Sat Dec 02, 2017 2:02 am
This seems like a whole lot of trouble over a problem that shouldn't even exist. Use good quality micro-USB cables that won't "wiggle loose" and don't pull the plug without a proper shutdown. File corruption due to power interruption is not unique to Raspberry Pi computers. It will happen with Windows and Mac computers as well. Treat your Pi like a computer instead of a toy and you'll have far fewer problems.
The problem with PI is the storage: Consumer grade SD storage controllers are crap, their firmware use dynamic wear leveling instead of static thus creating hot-spots & for MLC NAND storage with its underlying flash sectors only supporting ~3k erase cycles (was 100k for former SLC based flash), it's quickly weared. For SSD, with decent firmwares, you can get an oversized one for your needs to have a better endurance thanks to better leveling but this will be mostly inefficient for SD that are unable to distribute wear efficiently by explicitly moving static data.

PI fundation should have made a M.2 sata port or, if too costly, a eUSB one: eUSB storage is used for embedded devices (lot of telecom equipment use it) and provide better hardware (multi channel NAND to avoid buffering & loss of power issues + capacitors to be able to end on the fly transactions cleanly) and firmware (good wear levelling, data retention monitoring etc...).

And a way for everybody to be able to buy devices on a 1 by 1 basis, as this kind of devices are only available for professional customers & volume price.

You just cannot say this is a user problem. Learn a bit about how flash storage work & you'll change your mind.

I talked about eUSB, but some industry grade SD/uSD are also available (even if probably less reliable than eUSB). But the problem remains: How to buy them?

On top of that, to avoid cost of SLC based devices (mass flash production moved to MLC, so SLC is still the best but cost are high), some manufacturer provide so called pSLC (pseudi-SLC = MLC used as SLC ; using only one cell load level comparator => 30k erase cycles endurance due to better margin for a cost/bit only doubled & still benefit from MLC price).

See for instance swissbit S46u series.

So this can be fixed & having a bulky external HDD can be avoided. But Raspi foundation have to get involved in providing a way for individuals to buy devices that are made for 24/7 use.

Re: Prevent SD-Card Corruption

Posted: Tue Dec 05, 2017 12:08 pm
by jamesh
It's such a massive problem that we've only sold 17M devices.

All the suggestions above are attempting to fix a minor problem, and all cost a hat load of money.

SD card are cheap, and generally work fine.