SD flash reliability


25 posts
by woodsy-al » Thu Dec 20, 2012 2:34 pm
Hi there. I got a question about the Flash SD card tha tthe Raspberry Pi boots from. With linux writing regularly to log files etc, or reflashing the system, how prone is the SD card to the flash starting to wear out and become less fast?

How long would it work or how often can you reflash it without normal use before it breaks and needs to be replaced ?
Posts: 4
Joined: Wed Dec 19, 2012 11:07 am
by SN » Thu Dec 20, 2012 3:00 pm
Well as an indicator I have a Kingston 4Gb card in my internet facing webserver which is capturing images off a wireless webcam and writing them onto the SD card and then using ImagicMagick to timestamp a second copy for display
This writes two 20kb to 40Kb files EVERY five seconds and has been running for 43 days and 15 hours - so that's one and a half million writes into the root filesystem without issue...
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?
User avatar
Posts: 1008
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
by texy » Thu Dec 20, 2012 3:06 pm
Reading/Writing to the SD card is not a problem. Continually plugging the card in and out however is. Since my first Pi delivery - May or June, I,ve broken 4 SD cards now. First they crack, then bits fall off, and eventually the Pi doesn't reconise that it's pluged in. Not the Pi's fault, more my heavy handedness ;)

Texy
"2.8inch TFT LCD + Touch screen" add-on boards for sale here :
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=93&t=65566
50p goes to the Foundation ;-)
Forum Moderator
Forum Moderator
Posts: 2479
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England
by SN » Thu Dec 20, 2012 3:09 pm
the answer to that is to use microSD cards in an adaptor in the SD card slot - don't 'think' this presents any issues with the Pi itself
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?
User avatar
Posts: 1008
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
by gordon77 » Thu Dec 20, 2012 3:18 pm
I struggled to find an answer to this when I wrote a programme that wrote to the card every few seconds, but then found out how to write to the ram !

http://www.maclife.com/article/howtos/a ... e_worth_it

This article says 100,000 but then there is wear leveliing on top of that ?

I found another artucle that tested a flash stick that failed after 90million writes but still could be read.

Gordon77
Posts: 850
Joined: Sun Aug 05, 2012 3:12 pm
by woodsy-al » Thu Dec 20, 2012 3:28 pm
What about putting just teh bootloader and /boot partition on the flash and have the root filesystem on an external hard disk, would that be possible ?
Posts: 4
Joined: Wed Dec 19, 2012 11:07 am
by texy » Thu Dec 20, 2012 3:35 pm
SN wrote:the answer to that is to use microSD cards in an adaptor in the SD card slot - don't 'think' this presents any issues with the Pi itself



I do now have a couple of 4gig micro's with adapters, but in the early days the micro's were not recommended, although that may of been the faster variety.
Texy
"2.8inch TFT LCD + Touch screen" add-on boards for sale here :
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=93&t=65566
50p goes to the Foundation ;-)
Forum Moderator
Forum Moderator
Posts: 2479
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England
by hojnikb » Thu Dec 20, 2012 4:33 pm
gordon77 wrote:I struggled to find an answer to this when I wrote a programme that wrote to the card every few seconds, but then found out how to write to the ram !

http://www.maclife.com/article/howtos/a ... e_worth_it

This article says 100,000 but then there is wear leveliing on top of that ?

I found another artucle that tested a flash stick that failed after 90million writes but still could be read.

Gordon77

well 100k rewrites on todays sd cards is just untrue.. Newer cards are rated from 1k up to about 5k at most, becouse flash gemetry is getting smaller and smaller.You can get more p/e out of them, but that all depends on luck. (most of them dont just die when they hit their rated p/e)

But this is nothing to worry about. Any decent brand name sd card should have somekind of wearlevelling, so it should wear out evenly. Also unless you write to your card like crazy, wearout of card is the last thing to worry about. Even if it does happen, data will still be there, so it can always be cloned to a new card..
+°´°+,¸¸,+°´°~ Everyone should have a taste of UK Raspberry Pie =D ~°´°+,¸¸,+°´°+
Rasberry Pi, SoC @ 1225Mhz :o, 256MB Ram @ 550Mhz, 16GB SD-Card, Raspbian
User avatar
Posts: 112
Joined: Mon Jun 04, 2012 3:59 pm
Location: @Home
by SN » Thu Dec 20, 2012 4:50 pm
hojnikb wrote:
gordon77 wrote:I struggled to find an answer to this when I wrote a programme that wrote to the card every few seconds, but then found out how to write to the ram !

http://www.maclife.com/article/howtos/a ... e_worth_it

This article says 100,000 but then there is wear leveliing on top of that ?

I found another artucle that tested a flash stick that failed after 90million writes but still could be read.

Gordon77

well 100k rewrites on todays sd cards is just untrue.. Newer cards are rated from 1k up to about 5k at most, becouse flash gemetry is getting smaller and smaller.You can get more p/e out of them, but that all depends on luck. (most of them dont just die when they hit their rated p/e)

But this is nothing to worry about. Any decent brand name sd card should have somekind of wearlevelling, so it should wear out evenly. Also unless you write to your card like crazy, wearout of card is the last thing to worry about. Even if it does happen, data will still be there, so it can always be cloned to a new card..

So how on my 4Gb card, am I still running with 1,5 million writes? and these are overwriting the same pair of files over and over again... :?

EDIT - I just checked the script, its actually over 2.2 million writes as there's a 'wget', a 'convert' and then a 'cp' every 5 seconds for the last 43 days
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?
User avatar
Posts: 1008
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
by toysareforboys » Thu Dec 20, 2012 11:25 pm
I'm running an e-mail server on the Pi, with 30 e-mail domains, webmail, pop3/imap/smtp, ad well as Pancake + PHP for the web server (I have php doing a lot of file WRITES too, not just reads).

I too was worried about the SD card wearing out, so I moved the operating system onto SSD just to be a bit safer. Runs way faster too ;)

Image
LICK FOR HIGH RES


Image
LICK FOR HIGH RES!


If you're worried, move your operating system onto USB HDD, then you'll be set :)

http://tafb.yi.org:8000/

-Jamie M.
Seagate GoFlex Home, 1.2GHz ARM (kirkwood), 128MB RAM, Gigabit Ethernet, SATA2. Sandisk Extreme 120GB SSD running Arch ARM Linux 3.6.11-0. nginx + php-fpm = LIVE STATUS hosted right on the SGFH!! http://tafb.yi.org
User avatar
Posts: 136
Joined: Thu Dec 06, 2012 11:01 pm
by Sleep Mode zZ » Fri Dec 21, 2012 2:04 am
SN wrote:So how on my 4Gb card, am I still running with 1,5 million writes? and these are overwriting the same pair of files over and over again... :?

EDIT - I just checked the script, its actually over 2.2 million writes as there's a 'wget', a 'convert' and then a 'cp' every 5 seconds for the last 43 days

You are writing the same pair of files over and over again but you are not writing them to the same physical location on the card. Wear leveling is saving your card.

It seems that SD cards have dynamic wear leveling that writes evenly to the free segments on the card. No invidual segment is written over and over again if there are other free segments.
Posts: 200
Joined: Sun Aug 19, 2012 5:56 am
Location: Finland
by wayner » Fri Dec 21, 2012 4:15 am
toysareforboys wrote:I too was worried about the SD card wearing out, so I moved the operating system onto SSD just to be a bit safer. Runs way faster too ;)
.
I was wondering about what is the fastest media storage for the RPi is it:

1) SD Card
2) SSD connected via USB
3) USB flash drive
4) Mechanical hard drive connected via USB

I have seen your posts on the SSD and I may give that a try but is the bottlenect the USB interface of the SSD/ hard drive? If it is the latter then does it matter if you are using a SSD or traditional mechanical hard drive?
Posts: 18
Joined: Thu Oct 25, 2012 1:40 am
Location: Toronto, Ontario, Canada
by toysareforboys » Fri Dec 21, 2012 4:43 am
wayner wrote:I was wondering about what is the fastest media storage for the RPi is it:
The order from slowest to fastest would be:

1) SD Card
2) USB flash drive (a good one, i.e. http://www.chaintech.com.tw/a511_newsre ... p?serno=72 )
3) Mechanical hard drive connected via USB (all of them preform about the same)
4) SSD connected via USB (even the cheapest crappiest SSD smokes all the rest).

Obviously the USB interface to the SSD is not ideal, you're getting 30mb/sec write speeds instead of the 500mb/sec it's capable of ;)

Any usb mechanical hard drive will give you great performance on the Pi :)

-Jamie M.
Seagate GoFlex Home, 1.2GHz ARM (kirkwood), 128MB RAM, Gigabit Ethernet, SATA2. Sandisk Extreme 120GB SSD running Arch ARM Linux 3.6.11-0. nginx + php-fpm = LIVE STATUS hosted right on the SGFH!! http://tafb.yi.org
User avatar
Posts: 136
Joined: Thu Dec 06, 2012 11:01 pm
by wayner » Fri Dec 21, 2012 4:47 am
Why would the SSD be faster than the mechanical hard drive - I guess this implies that random writes to a mechanical hard drive are lower than 30 MB/s, correct? So the bottleneck is the random write (or read) speed of the hard drive and not the USB 2.0 port?
Posts: 18
Joined: Thu Oct 25, 2012 1:40 am
Location: Toronto, Ontario, Canada
by toysareforboys » Fri Dec 21, 2012 4:55 am
wayner wrote:Why would the SSD be faster than the mechanical hard drive - I guess this implies that random writes to a mechanical hard drive are lower than 30 MB/s, correct? So the bottleneck is the random write (or read) speed of the hard drive and not the USB 2.0 port?
It all depends on what you're using it for. Sequential read and write speeds will be almost identical between SSD and physical hard discs on usb 2.0. The big difference comes in small block (4k) random read and write times. The mechanical hard drive just can't keep up with SSD.

If you're using your pi as a server (web, email, whatever) then 4k random performance is important. If you're using it to stream 1080p videos, it's not ;)

-Jamie M.
Seagate GoFlex Home, 1.2GHz ARM (kirkwood), 128MB RAM, Gigabit Ethernet, SATA2. Sandisk Extreme 120GB SSD running Arch ARM Linux 3.6.11-0. nginx + php-fpm = LIVE STATUS hosted right on the SGFH!! http://tafb.yi.org
User avatar
Posts: 136
Joined: Thu Dec 06, 2012 11:01 pm
by jackokring » Fri Dec 21, 2012 7:08 am
I would assume (though not tested), that any failure to verify a write places the block in a bad block list, and the SD shrinks in size as bad blocks accumulate. Further processing by adapting the file system could use all the unfailed bits in any bad block, by block triplet majority reads.

It is also important to know the state a bit fails in. Tis could be used to utilize blocks which have minor failures, up to 3 bit loss per failed bit if bits always fail in one bit state i.e. 1.

Failure mainly occurs due to over current in the bit cell charging. Modern SD uses a current regulation charge, as opposed to a fixed voltage inrush current (fail).

Would it be possible to use block compression to extend SD lifespan? Probably. Is it done now? Probably not.
Pi=B256R0USB CL4SD8GB Raspbian Stock. https://sites.google.com/site/rubikcompression/strictly-long https://dl.dropboxusercontent.com/u/1615413/Own%20Work/Leptronics.pdf https://groups.google.com/forum/#!topic/comp.compression/t22ct_BKi9w
User avatar
Posts: 784
Joined: Tue Jul 31, 2012 8:27 am
Location: London, UK
by hojnikb » Fri Dec 21, 2012 3:22 pm
Sleep Mode zZ wrote:
SN wrote:So how on my 4Gb card, am I still running with 1,5 million writes? and these are overwriting the same pair of files over and over again... :?

EDIT - I just checked the script, its actually over 2.2 million writes as there's a 'wget', a 'convert' and then a 'cp' every 5 seconds for the last 43 days

You are writing the same pair of files over and over again but you are not writing them to the same physical location on the card. Wear leveling is saving your card.

It seems that SD cards have dynamic wear leveling that writes evenly to the free segments on the card. No invidual segment is written over and over again if there are other free segments.

Yeah, you dont actually overwrite your whole card 2m times, as it would surly die (unless you have a really really good card with SLC flash)..
+°´°+,¸¸,+°´°~ Everyone should have a taste of UK Raspberry Pie =D ~°´°+,¸¸,+°´°+
Rasberry Pi, SoC @ 1225Mhz :o, 256MB Ram @ 550Mhz, 16GB SD-Card, Raspbian
User avatar
Posts: 112
Joined: Mon Jun 04, 2012 3:59 pm
Location: @Home
by wbp » Wed May 01, 2013 5:15 am
I realize this is a fairly old thread, but this topic is of interest to me, as I'm setting up a couple of Pi's to collect weather data and serve it to the web. One of them got really slow due to poor SD card performance. I replaced the older class 4 card with a new class 10, and now it runs so fast it almost flies.

Given "wear leveling", it occurs to me that one should use a larger card than needed - wouldn't this have a significant impact (reduction) on the wear? My little servers had 4 GB cards and were running at better than 50% free space, but 16 GB Class 10 cards are so cheap now that's what I got. It seems to me that this gives the wear leveling algorithm 4 times the space to use before it has to rewrite any given block, which should be big improvement in reliability, no?

William
Posts: 16
Joined: Sat Mar 30, 2013 12:34 am
by SN » Sun May 26, 2013 10:30 pm
you're probably right there.

my little 4Gb SD card driven web server passed the 200 day mark today - hanging on in there 8-)
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?
User avatar
Posts: 1008
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
by Heater » Mon May 27, 2013 2:37 pm
wbp,
It seems to me that this gives the wear leveling algorithm 4 times the space to use before it has to rewrite any given block.

I would not be so sure. I faintly recall reading somewhere that SD cards were divided up into regions for the purposes of wear levelling such that wear levelling was done on each region independently. In which case having a bigger card with more regions does not help. Sorry I can't find any links to confirm that and actually getting any clues as to how manufacturers implement wear levelling is pretty hard.
Posts: 1319
Joined: Tue Jul 17, 2012 3:02 pm
by wbp » Mon May 27, 2013 6:14 pm
I cannot find anything that says a card might be divided into regions for wear leveling. If you have something solid about this I'd like to hear it. Are you sure you aren't confusing this with allocation groups? https://wiki.linaro.org/WorkingGroups/K ... CardSurvey

It's important to remember that this technology is evolving pretty quickly - older SD cards are very basic, but there is a lot of competition, and the manufacturers see the need for faster and more reliable cards. A Class 10 card is a huge improvement over the Class 4 and Class 6 cards of just a couple of years ago.

I have several Pi's running different things. Two are on 24/7 collecting data from weather stations and posting it to the web. On one I had used an old 4 GB Class 4 card I had lying around, and it seemed to work fine at first, but I started to notice it getting really slow. Things that should have taken less than 1 second were taking 10 or even 20 seconds. I started collecting and graphing data from /proc/stat and discovered it was spending a large percent of the time in IO Wait. I switched to a new 16 GB Class 10 card and the IO Wait times went to near zero, and everything runs fast again. That was about 2 months ago, certainly not a long term test. I am still collecting and graphing /proc/stat numbers, so if the card starts to go bad I should be able to see changes in the IO Wait time.
Posts: 16
Joined: Sat Mar 30, 2013 12:34 am
by Heater » Mon May 27, 2013 7:31 pm
I have to ask. Why am I so unlucky?

Over the past two years I have been through a pile of SD cards. All kind of sizes and all kind of brands. I am down to just four or five.

What happens is that even if they have not been used for any length of time all sorts of block errors occur. File systems get corrupted. This is not an ext3/4 corruption due to yanking the power or other unclean shutdown. No, it's the blocks on the cards not working.

I have tried reformating under Windows. I have tried reformatting with the SD card association tools. Sometimes that turns up an error about being "write protected". Well I never write protected anything and there seems to be no way to undo it.

I have tried writing with dd. No go. Read back, with dd, what I have written and it's not the same.

Rather than waste time these cards they go into the bin.

Guess what I'm left with? Four of my five surviving cards are Transend 8GB Class 10.

All in all, from here, it looks as if an SD card is about as reliable as writing on tissue paper and then putting out in the rain.
Posts: 1319
Joined: Tue Jul 17, 2012 3:02 pm
by remsnet » Mon May 27, 2013 7:58 pm
Heater wrote:I have to ask. Why am I so unlucky?



Guess what I'm left with? Four of my five surviving cards are Transend 8GB Class 10.

All in all, from here, it looks as if an SD card is about as reliable as writing on tissue paper and then putting out in the rain.



Read this , for help : https://github.com/raspberrypi/linux/issues/280

i found an workarround that mades it more stable .

i am use Transend 16/32/64GB Class 10 cards
Posts: 151
Joined: Wed Dec 19, 2012 7:32 pm
Location: Planet Gaia
by boltun » Tue Nov 05, 2013 8:00 pm
I've been using RPI as SVN server for few month when I ran into this issue:
https://github.com/raspberrypi/linux/issues/280
Now sitting on ddrescue for a week already trying to recover important data (yes, I was stupid enough not to make backups).
I am considering splitting an 8Gb card into two partition of 4Gb each and doing a software RAID. Anyone tried that ? Is it even going to help ?
Posts: 1
Joined: Tue Nov 05, 2013 7:50 pm
by wbp » Tue Nov 05, 2013 11:54 pm
Splitting the card and doing s/w RAID would be interesting, but seems like a lot of work, and if the card fails it might not work. Why not just get a small external drive and back up frequently to that? Or backup to USB solid state drive - it won't get written that often.

My 2 Pi's with 16GB Samsung Class 10 cards are still going, 24/7, with no sign of problems after several months. Common OS image, all user data backed up twice a day to a NAS. I can rebuild if I have to in about an hour. 16 GB Samsung class 10 costs $20 now.
Posts: 16
Joined: Sat Mar 30, 2013 12:34 am