MuntyScruntfundle
Posts: 223
Joined: Fri Oct 27, 2017 11:14 pm

SD card write count?

Wed Oct 17, 2018 3:11 pm

Hi folks.

I'm working on s bit of monitoring code for a pi cluster, part of the data I want to poll for is the SD card write count. As far as I can work out (with Python anyway (using psutil)) this number is the only data I can gather about the SD health.

The more writes (not really true) the more life has been sucked out of the card. But this is reported on different web sites wildly differently. Modern firmware/hardware sends blocks of data all over the disk so the 'beginning' doesn't wear out faster than the 'end', but there doesn't appear to be any data I can gather on this. I have an old 3 here running my mysql server (the data is on an external raid drive) and it's write count is over 700,000. According to some web sites I've found this card should have been buried about a year ago, but it's only 13% full so there's loads of apace for data to be spattered around.

I always use genuine good quality cards and as yet I haven't had any fail, apart from one that arrived dead. But it's going to happen sooner or later, it would be good to have some kind of early warning system.

So, has anyone else out there gathered any useful information on this topic? Or should I keep all my logs and post them here next October?

Many thanks.

W. H. Heydt
Posts: 10733
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: SD card write count?

Wed Oct 17, 2018 5:23 pm

It isn't that simple. There is an internal processor in the SD card that will attempt to even out the number of write cycles (look up "wear leveling") across the blocks of flash memory. This also means that a larger/less full card, will wear slower than an small/full card.

Heater
Posts: 13058
Joined: Tue Jul 17, 2012 3:02 pm

Re: SD card write count?

Wed Oct 17, 2018 8:12 pm

No SD card manufacture will specify a write lifetime of their product because they have no idea. It all depends on what you are doing.

Every time you update a file your operating system is rewriting some blocks of data on the SD.

But the SD will write those blocks wherever it likes in it's FLASH store so as to try and minimize writes to any particular FLASH cells. It will map those real, physical, FLASH blocks to whatever blocks the OS thinks it is writing to.

This is the so called "wear leveling" that the computers in the SD cards do.

Of course every SD card manufacturer has it's own wear leveling algorithm and they are not about to tell us how they work.

Actually testing this might be a problem. Basically one would have to overwrite the entire card, read the data back to check it's correct, over and over until it fails. Could take a long time.

Has anyone attempted that?

kaksi
Posts: 92
Joined: Tue Mar 10, 2015 6:19 am

Re: SD card write count?

Fri Oct 19, 2018 7:26 am

Heater wrote:
Wed Oct 17, 2018 8:12 pm
Basically one would have to overwrite the entire card, read the data back to check it's correct, over and over until it fails. Could take a long time.

Has anyone attempted that?
The german c't magazine performed such a test on SSDs some months ago. The article was quite long, so I skipped to the conclusion: in normal use, they should last for years.

Anyway, I think it is a good idea to move seldom used files elsewhere.

jahboater
Posts: 4596
Joined: Wed Feb 04, 2015 6:38 pm

Re: SD card write count?

Fri Oct 19, 2018 8:33 am

I thought most SD cards did not do wear leveling?
Its normal for SSD's of course.

For me, I would take a couple of basic steps to reduce the writes and then forget about it.
1) In /etc/fstab make sure the / mount has "noatime" in the options and add ",commit=600" to those options.
2) add a couple of ram disks (virtual memory actually).

The last three entries in fstab should look something like this:

Code: Select all

PARTUUID=1486ef1d-02  /  ext4  defaults,noatime,commit=600,errors=remount-ro  0 1
tmpfs /tmp tmpfs defaults,noatime,nosuid 0 0
tmpfs /var/log tmpfs defaults,noatime,nosuid,size=64m 0 0
Obviously your PARTUUID will be different.

"noatime" is the default and means the super block is not written to every time a file is read.

"commit=600" means flush the disk cache every 10 minutes instead of the default which is every 5 seconds.
This will also improve performance, but only use it if your site is not prone to random power cuts, and you are careful to shut the Pi down cleanly.

The two "tmpfs" disks just save writes in the usual way. Note they are virtual memory and unlike true ram disks only take as much memory as the files on them need (plus a tiny overhead).

I have been using Pi's since day one and never had an SD card wear out.

Heater
Posts: 13058
Joined: Tue Jul 17, 2012 3:02 pm

Re: SD card write count?

Fri Oct 19, 2018 8:42 am

jahboater,
I thought most SD cards did not do wear leveling?
Google tells me that all modern SD cards do. For example:

https://www.rs-online.com/designspark/a ... ht-sd-card

jahboater
Posts: 4596
Joined: Wed Feb 04, 2015 6:38 pm

Re: SD card write count?

Fri Oct 19, 2018 8:48 am

The table in your link says that "industrial" grade cards have wear leveling, but "consumer" grade cards do not.
Edit: it actually says "advanced wear leveling", maybe the consumer cards have something basic.

http://www.mouser.com/ds/2/669/SanDisk_ ... 805940.pdf

If wear leveling is in use then obviously buy a much larger card than needed.
I see the new Sandisk A2 class cards come in a minimum size of 64GB, so for most uses that may be plenty.

Heater
Posts: 13058
Joined: Tue Jul 17, 2012 3:02 pm

Re: SD card write count?

Fri Oct 19, 2018 10:06 am

jahboater,
If wear leveling is in use then obviously buy a much larger card than needed.
That sounds intuitively true. But is it?

Let's consider a very simple case:

We have a storage media with only two blocks A and B.

We write something to block A that we never want to change again.

We proceed to repeatedly overwrite block B with some ever changing data.

Eventually block B will reach some write limit count, say a million. Meanwhile block A still has a write count of only 1.

So now, a clever wear leveling algorithm could swap the content of A and B and remap them as B and A.

Now we have 999999 more writes we can do to our block B, which now actually block A.

Potentially if you have millions of blocks that never get rewritten and only a few that do, then such swapping between "active" and static blocks would amplify your write limit millions of times.

I have no idea if any of this happens. SD card manufacturers don't say much about what goes on in their controllers.

MuntyScruntfundle
Posts: 223
Joined: Fri Oct 27, 2017 11:14 pm

Re: SD card write count?

Fri Oct 19, 2018 10:26 am

Yes, I mentioned levelling.

Paul Hutch
Posts: 378
Joined: Fri Aug 25, 2017 2:58 pm
Location: Blackstone River Valley, MA, USA
Contact: Website

Re: SD card write count?

Fri Oct 19, 2018 10:37 am

jahboater wrote:
Fri Oct 19, 2018 8:48 am
The table in your link says that "industrial" grade cards have wear leveling, but "consumer" grade cards do not.
Edit: it actually says "advanced wear leveling", maybe the consumer cards have something basic.
My understanding from professionally designing with FLASH for decades, especially raw FLASH in embedded systems, is that without basic wear leveling FLASH memory cards would be nearly useless (dead after a thousand or so saves of a file). Some of the earliest FLASH I worked with in the late 1990's had a life spec of only 100 writes. That was OK for replacing PROM for application storage but obviously wasn't useful for much else.

User avatar
Mortimer
Posts: 923
Joined: Sun Jun 10, 2012 3:57 pm

Re: SD card write count?

Fri Oct 19, 2018 11:01 am

LOL! reminds me of the time way back when I was working in the avionics industry, we got in a bubble memory device to try out. Someone thought it a good idea to try it out as a hard-drive in a computer. I think it lasted less than an hour!
--------------
The purpose of a little toe is to ensure you keep your furniture in the right place.

jahboater
Posts: 4596
Joined: Wed Feb 04, 2015 6:38 pm

Re: SD card write count?

Fri Oct 19, 2018 12:42 pm

Heater wrote:
Fri Oct 19, 2018 10:06 am
jahboater,
If wear leveling is in use then obviously buy a much larger card than needed.
That sounds intuitively true. But is it?
Your suggestion is interesting - perhaps that's one thing "advanced" wear leveling does.
But it has to keep track of write counts to blocks in use.

SD cards know which blocks are in use and which are not - that's what TRIM does (the "discard" option in /etc/fstab).
Therefore I suspect buying an oversize card makes it very simple for the controller.

W. H. Heydt
Posts: 10733
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: SD card write count?

Fri Oct 19, 2018 1:38 pm

kaksi wrote:
Fri Oct 19, 2018 7:26 am
The german c't magazine performed such a test on SSDs some months ago. The article was quite long, so I skipped to the conclusion: in normal use, they should last for years.
But under what conditions? Using an SD card as a computer system mass storage device doesn't usually fall under "normal use".

Heater
Posts: 13058
Joined: Tue Jul 17, 2012 3:02 pm

Re: SD card write count?

Fri Oct 19, 2018 5:42 pm

W. H. Heydt,
But under what conditions?
Yes, perhaps someone who reads c't magazine could let us know?

The test I had in mind goes something like this:

1) Seed a pseudo random number generator and use it's output to fill all the blocks of an SD card.

2) Starting from the same seed, read all the blocks back from the SD and check they match the PRNG output.

3) If there is no error choose a different seed and start again from 1).

At the end of all this we have a count of block writes until failure and it might be informative to log the time each iteration takes.

I imagine all this might take more time than I'm willing to invest.

W. H. Heydt
Posts: 10733
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: SD card write count?

Fri Oct 19, 2018 7:43 pm

I was looking at a short "Wow...look at that" blurb in MaximumPC today. They were talking about a PNY 512GB microSD card. It has a "lifetime" guarantee. It also costs $360. Makes one wonder how long it would really last if used in a Pi (and if they would actually honor that "lifetime" if used that way).

kaksi
Posts: 92
Joined: Tue Mar 10, 2015 6:19 am

Re: SD card write count?

Sat Oct 20, 2018 11:31 am

Heater wrote:
Fri Oct 19, 2018 5:42 pm
W. H. Heydt,
But under what conditions?
Yes, perhaps someone who reads c't magazine could let us know?
I suppose the article is copyrighted, so I am cautious to quote. The beginning is here:

https://www.heise.de/ct/ausgabe/2017-21 ... 39820.html

but you do not learn anything w/o paying 1.49 E.

Return to “General discussion”