SD flash reliability

[woodsy-al]

Hi there. I got a question about the Flash SD card tha tthe Raspberry Pi boots from. With linux writing regularly to log files etc, or reflashing the system, how prone is the SD card to the flash starting to wear out and become less fast?

How long would it work or how often can you reflash it without normal use before it breaks and needs to be replaced ?

[SN]

Well as an indicator I have a Kingston 4Gb card in my internet facing webserver which is capturing images off a wireless webcam and writing them onto the SD card and then using ImagicMagick to timestamp a second copy for display
This writes two 20kb to 40Kb files EVERY five seconds and has been running for 43 days and 15 hours - so that's one and a half million writes into the root filesystem without issue...

[texy]

Reading/Writing to the SD card is not a problem. Continually plugging the card in and out however is. Since my first Pi delivery - May or June, I,ve broken 4 SD cards now. First they crack, then bits fall off, and eventually the Pi doesn't reconise that it's pluged in. Not the Pi's fault, more my heavy handedness

Texy

[SN]

the answer to that is to use microSD cards in an adaptor in the SD card slot - don't 'think' this presents any issues with the Pi itself

[gordon77]

I struggled to find an answer to this when I wrote a programme that wrote to the card every few seconds, but then found out how to write to the ram !

http://www.maclife.com/article/howtos/a ... e_worth_it

This article says 100,000 but then there is wear leveliing on top of that ?

I found another artucle that tested a flash stick that failed after 90million writes but still could be read.

Gordon77

[woodsy-al]

What about putting just teh bootloader and /boot partition on the flash and have the root filesystem on an external hard disk, would that be possible ?

[texy]

the answer to that is to use microSD cards in an adaptor in the SD card slot - don't 'think' this presents any issues with the Pi itself

I do now have a couple of 4gig micro's with adapters, but in the early days the micro's were not recommended, although that may of been the faster variety.
Texy

[hojnikb]

I struggled to find an answer to this when I wrote a programme that wrote to the card every few seconds, but then found out how to write to the ram !

http://www.maclife.com/article/howtos/a ... e_worth_it

This article says 100,000 but then there is wear leveliing on top of that ?

I found another artucle that tested a flash stick that failed after 90million writes but still could be read.

Gordon77

well 100k rewrites on todays sd cards is just untrue.. Newer cards are rated from 1k up to about 5k at most, becouse flash gemetry is getting smaller and smaller.You can get more p/e out of them, but that all depends on luck. (most of them dont just die when they hit their rated p/e)

But this is nothing to worry about. Any decent brand name sd card should have somekind of wearlevelling, so it should wear out evenly. Also unless you write to your card like crazy, wearout of card is the last thing to worry about. Even if it does happen, data will still be there, so it can always be cloned to a new card..

[SN]

I struggled to find an answer to this when I wrote a programme that wrote to the card every few seconds, but then found out how to write to the ram !

http://www.maclife.com/article/howtos/a ... e_worth_it

This article says 100,000 but then there is wear leveliing on top of that ?

I found another artucle that tested a flash stick that failed after 90million writes but still could be read.

Gordon77

well 100k rewrites on todays sd cards is just untrue.. Newer cards are rated from 1k up to about 5k at most, becouse flash gemetry is getting smaller and smaller.You can get more p/e out of them, but that all depends on luck. (most of them dont just die when they hit their rated p/e)

But this is nothing to worry about. Any decent brand name sd card should have somekind of wearlevelling, so it should wear out evenly. Also unless you write to your card like crazy, wearout of card is the last thing to worry about. Even if it does happen, data will still be there, so it can always be cloned to a new card..

So how on my 4Gb card, am I still running with 1,5 million writes? and these are overwriting the same pair of files over and over again...

EDIT - I just checked the script, its actually over 2.2 million writes as there's a 'wget', a 'convert' and then a 'cp' every 5 seconds for the last 43 days

[toysareforboys]

I'm running an e-mail server on the Pi, with 30 e-mail domains, webmail, pop3/imap/smtp, ad well as Pancake + PHP for the web server (I have php doing a lot of file WRITES too, not just reads).

I too was worried about the SD card wearing out, so I moved the operating system onto SSD just to be a bit safer. Runs way faster too


LICK FOR HIGH RES


LICK FOR HIGH RES!

If you're worried, move your operating system onto USB HDD, then you'll be set

http://tafb.yi.org:8000/

-Jamie M.

[Sleep Mode zZ]

So how on my 4Gb card, am I still running with 1,5 million writes? and these are overwriting the same pair of files over and over again...

EDIT - I just checked the script, its actually over 2.2 million writes as there's a 'wget', a 'convert' and then a 'cp' every 5 seconds for the last 43 days

You are writing the same pair of files over and over again but you are not writing them to the same physical location on the card. Wear leveling is saving your card.

It seems that SD cards have dynamic wear leveling that writes evenly to the free segments on the card. No invidual segment is written over and over again if there are other free segments.

[wayner]

I too was worried about the SD card wearing out, so I moved the operating system onto SSD just to be a bit safer. Runs way faster too
.

I was wondering about what is the fastest media storage for the RPi is it:

1) SD Card
2) SSD connected via USB
3) USB flash drive
4) Mechanical hard drive connected via USB

I have seen your posts on the SSD and I may give that a try but is the bottlenect the USB interface of the SSD/ hard drive? If it is the latter then does it matter if you are using a SSD or traditional mechanical hard drive?

[toysareforboys]

I was wondering about what is the fastest media storage for the RPi is it:

The order from slowest to fastest would be:

1) SD Card
2) USB flash drive (a good one, i.e. http://www.chaintech.com.tw/a511_newsre ... p?serno=72 )
3) Mechanical hard drive connected via USB (all of them preform about the same)
4) SSD connected via USB (even the cheapest crappiest SSD smokes all the rest).

Obviously the USB interface to the SSD is not ideal, you're getting 30mb/sec write speeds instead of the 500mb/sec it's capable of

Any usb mechanical hard drive will give you great performance on the Pi

-Jamie M.

[wayner]

Why would the SSD be faster than the mechanical hard drive - I guess this implies that random writes to a mechanical hard drive are lower than 30 MB/s, correct? So the bottleneck is the random write (or read) speed of the hard drive and not the USB 2.0 port?

[toysareforboys]

Why would the SSD be faster than the mechanical hard drive - I guess this implies that random writes to a mechanical hard drive are lower than 30 MB/s, correct? So the bottleneck is the random write (or read) speed of the hard drive and not the USB 2.0 port?

It all depends on what you're using it for. Sequential read and write speeds will be almost identical between SSD and physical hard discs on usb 2.0. The big difference comes in small block (4k) random read and write times. The mechanical hard drive just can't keep up with SSD.

If you're using your pi as a server (web, email, whatever) then 4k random performance is important. If you're using it to stream 1080p videos, it's not

-Jamie M.

[jackokring]

I would assume (though not tested), that any failure to verify a write places the block in a bad block list, and the SD shrinks in size as bad blocks accumulate. Further processing by adapting the file system could use all the unfailed bits in any bad block, by block triplet majority reads.

It is also important to know the state a bit fails in. Tis could be used to utilize blocks which have minor failures, up to 3 bit loss per failed bit if bits always fail in one bit state i.e. 1.

Failure mainly occurs due to over current in the bit cell charging. Modern SD uses a current regulation charge, as opposed to a fixed voltage inrush current (fail).

Would it be possible to use block compression to extend SD lifespan? Probably. Is it done now? Probably not.

[hojnikb]

So how on my 4Gb card, am I still running with 1,5 million writes? and these are overwriting the same pair of files over and over again...

EDIT - I just checked the script, its actually over 2.2 million writes as there's a 'wget', a 'convert' and then a 'cp' every 5 seconds for the last 43 days

You are writing the same pair of files over and over again but you are not writing them to the same physical location on the card. Wear leveling is saving your card.

It seems that SD cards have dynamic wear leveling that writes evenly to the free segments on the card. No invidual segment is written over and over again if there are other free segments.

Yeah, you dont actually overwrite your whole card 2m times, as it would surly die (unless you have a really really good card with SLC flash)..

[wbp]

I realize this is a fairly old thread, but this topic is of interest to me, as I'm setting up a couple of Pi's to collect weather data and serve it to the web. One of them got really slow due to poor SD card performance. I replaced the older class 4 card with a new class 10, and now it runs so fast it almost flies.

Given "wear leveling", it occurs to me that one should use a larger card than needed - wouldn't this have a significant impact (reduction) on the wear? My little servers had 4 GB cards and were running at better than 50% free space, but 16 GB Class 10 cards are so cheap now that's what I got. It seems to me that this gives the wear leveling algorithm 4 times the space to use before it has to rewrite any given block, which should be big improvement in reliability, no?

William

[SN]

you're probably right there.

my little 4Gb SD card driven web server passed the 200 day mark today - hanging on in there

[Heater]

wbp,

It seems to me that this gives the wear leveling algorithm 4 times the space to use before it has to rewrite any given block.

I would not be so sure. I faintly recall reading somewhere that SD cards were divided up into regions for the purposes of wear levelling such that wear levelling was done on each region independently. In which case having a bigger card with more regions does not help. Sorry I can't find any links to confirm that and actually getting any clues as to how manufacturers implement wear levelling is pretty hard.

[wbp]

I cannot find anything that says a card might be divided into regions for wear leveling. If you have something solid about this I'd like to hear it. Are you sure you aren't confusing this with allocation groups? https://wiki.linaro.org/WorkingGroups/K ... CardSurvey

It's important to remember that this technology is evolving pretty quickly - older SD cards are very basic, but there is a lot of competition, and the manufacturers see the need for faster and more reliable cards. A Class 10 card is a huge improvement over the Class 4 and Class 6 cards of just a couple of years ago.

I have several Pi's running different things. Two are on 24/7 collecting data from weather stations and posting it to the web. On one I had used an old 4 GB Class 4 card I had lying around, and it seemed to work fine at first, but I started to notice it getting really slow. Things that should have taken less than 1 second were taking 10 or even 20 seconds. I started collecting and graphing data from /proc/stat and discovered it was spending a large percent of the time in IO Wait. I switched to a new 16 GB Class 10 card and the IO Wait times went to near zero, and everything runs fast again. That was about 2 months ago, certainly not a long term test. I am still collecting and graphing /proc/stat numbers, so if the card starts to go bad I should be able to see changes in the IO Wait time.

[Heater]

I have to ask. Why am I so unlucky?

Over the past two years I have been through a pile of SD cards. All kind of sizes and all kind of brands. I am down to just four or five.

What happens is that even if they have not been used for any length of time all sorts of block errors occur. File systems get corrupted. This is not an ext3/4 corruption due to yanking the power or other unclean shutdown. No, it's the blocks on the cards not working.

I have tried reformating under Windows. I have tried reformatting with the SD card association tools. Sometimes that turns up an error about being "write protected". Well I never write protected anything and there seems to be no way to undo it.

I have tried writing with dd. No go. Read back, with dd, what I have written and it's not the same.

Rather than waste time these cards they go into the bin.

Guess what I'm left with? Four of my five surviving cards are Transend 8GB Class 10.

All in all, from here, it looks as if an SD card is about as reliable as writing on tissue paper and then putting out in the rain.

[remsnet]

I have to ask. Why am I so unlucky?



Guess what I'm left with? Four of my five surviving cards are Transend 8GB Class 10.

All in all, from here, it looks as if an SD card is about as reliable as writing on tissue paper and then putting out in the rain.

Read this , for help : https://github.com/raspberrypi/linux/issues/280

i found an workarround that mades it more stable .

i am use Transend 16/32/64GB Class 10 cards

[boltun]

I've been using RPI as SVN server for few month when I ran into this issue:
https://github.com/raspberrypi/linux/issues/280
Now sitting on ddrescue for a week already trying to recover important data (yes, I was stupid enough not to make backups).
I am considering splitting an 8Gb card into two partition of 4Gb each and doing a software RAID. Anyone tried that ? Is it even going to help ?

[wbp]

Splitting the card and doing s/w RAID would be interesting, but seems like a lot of work, and if the card fails it might not work. Why not just get a small external drive and back up frequently to that? Or backup to USB solid state drive - it won't get written that often.

My 2 Pi's with 16GB Samsung Class 10 cards are still going, 24/7, with no sign of problems after several months. Common OS image, all user data backed up twice a day to a NAS. I can rebuild if I have to in about an hour. 16 GB Samsung class 10 costs $20 now.