User avatar
WillBoxOwO
Posts: 122
Joined: Wed Jun 27, 2018 10:24 pm

Why does SD cards do that?

Fri Nov 02, 2018 12:06 am

Why does USB drives, MicroSD cards and so on get corrupted?

And most of all, when they get corrupted, why does the storage support becomes read-only?

This concept seems to me really wierd, lol.

Cheers.
I like cats

W. H. Heydt
Posts: 12648
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: Why does SD cards do that?

Fri Nov 02, 2018 1:06 am

It has to do with the way flash memory works. The first thing to understand is that flash memory (of all types) can only tolerate being written to a relatively few number of times (typically in the hundreds to a few thousands for any given location). The second thing is that when you read or write flash memory, it's on a whole block basis (and not by the individual byte). Third, writing to flash is a fairly slow process, comparatively.

So, combining those characteristics, every flash device has a small processor (and the less space and cost involved, the smaller and less capable that processor is). It tries to keep the number of write actions as even as possible across all available blocks. As a result, if you read part of a block, the internal processor reads the whole block to some temporary storage and writes it back...somewhere else. If you write part of a block, the device reads the whole block, updates the part you're writing and writes it all back out to flash...somewhere else.

Therefore, if the power is removed in the middle of a read-rewrite process, the file system is in an undetermined state, aka "corrupted". The faster the device, and the better the internal processor, the less likely it is to get caught in this situation. Some microSD cards are easier to corrupt than USB drives, and USB drives are easier to corrupt than SSDs (which are built to handle frequent, small reads and writes). Going to "read only" is simply a method of allowing you to retrieve any data that can be retrieved without taking the risk of doing more rewrite actions and making the corruption worse.

Hence the very strong recommendation to always tell the Pi to shut down, and then wait for the activity light to stop blinking before removing the power.

Caveat...if you know that your Pi is idle, and has been for at least a minute, then it is *probably* safe to just pull the power. But if that corrupts an SD card, either you were wrong about it being inactive, or your timing was bad.

User avatar
WillBoxOwO
Posts: 122
Joined: Wed Jun 27, 2018 10:24 pm

Re: Why does SD cards do that?

Fri Nov 02, 2018 1:57 am

Thanks for your anwser, I understand now. :)
I like cats

User avatar
Gavinmc42
Posts: 4526
Joined: Wed Aug 28, 2013 3:31 am

Re: Why does SD cards do that?

Fri Nov 02, 2018 2:44 am

Flash memory wears out. it only has a limited number of write cycles.
The number of writes can be as low as 1000 for MLC, multi layer cells.

https://en.wikipedia.org/wiki/Flash_memory
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Burngate
Posts: 6302
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: Why does SD cards do that?

Fri Nov 02, 2018 10:09 am

W. H. Heydt wrote:
Fri Nov 02, 2018 1:06 am
... As a result, if you read part of a block, the internal processor reads the whole block to some temporary storage and writes it back...somewhere else. If you write part of a block, the device reads the whole block, updates the part you're writing and writes it all back out to flash...somewhere else.
Writing, yes but reading?
If it's only reading, and nothing's changed, why should it write it back ... somewhere else?

User avatar
RaTTuS
Posts: 10559
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Why does SD cards do that?

Fri Nov 02, 2018 10:43 am

linux files syytems have last accessed time stamps so reading can write to the drive unless you switch of the Atime ....

see adding
noatime,nodiratime to your fstab line for the drive
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

User avatar
DougieLawson
Posts: 39121
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Why does SD cards do that?

Fri Nov 02, 2018 11:23 am

Also since secure digital was a step on from MMC primarily for cameras. When the flash memory goes bad you get a few effects 1) you can't store a new photo so you know the card is toast, 2) you can't format the card so you know it's toast, but 3) you don't lose any of the photos saved before the card toasted itself.

In general that's a good thing. With a Raspberry that's a pain in the tail, but not losing any old stuff means you may be able to clone it to a good card and carry on from where you left off.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

hippy
Posts: 7733
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Why does SD cards do that?

Fri Nov 02, 2018 2:32 pm

W. H. Heydt wrote:
Fri Nov 02, 2018 1:06 am
Going to "read only" is simply a method of allowing you to retrieve any data that can be retrieved without taking the risk of doing more rewrite actions and making the corruption worse.
I am not convinced that's right, that any SD card is deliberately forcing itself permanently read only or that any Linux or other system would instruct it to go permanently read only when it determined file system corruption was present.

I can understand a particular block being taken out of circulation when there is an unrecoverable problem but going permanently read only is an extreme action for an SD Card to take when it only deals in data blocks and has no care what the data within those blocks mean.

My belief is that power being pulled causes the SD Card controller to malfunction, misbehave or misinterpret commands sent to it, and somehow the read only flag ends up being permanently set rather than it being a deliberate and intended action.

It could alternatively be that power being pulled causes the SoC to send an incorrect or corrupted command to the SD Card which cause it to set the read only bit.
Last edited by hippy on Fri Nov 02, 2018 2:41 pm, edited 1 time in total.

User avatar
Paeryn
Posts: 2966
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: Why does SD cards do that?

Fri Nov 02, 2018 2:36 pm

Burngate wrote:
Fri Nov 02, 2018 10:09 am
W. H. Heydt wrote:
Fri Nov 02, 2018 1:06 am
... As a result, if you read part of a block, the internal processor reads the whole block to some temporary storage and writes it back...somewhere else. If you write part of a block, the device reads the whole block, updates the part you're writing and writes it all back out to flash...somewhere else.
Writing, yes but reading?
If it's only reading, and nothing's changed, why should it write it back ... somewhere else?
Generally it doesn't but it can happen that constantly reading a cell can alter an adjacent cell in the same block. Some controllers may keep tabs on how many times a block has been read since last written and decide to move it. Also this helps where, for example if 90% of the memory is used but only read from. Without moving the read only blocks would mean 10% of the memory is always used for writing, that would wear out quickly whilst the rest of the memory would still be good for a lot more cycles. Moving a block that is normally only read from to a block which is running low on usable write cycles gives you a longer working life.
She who travels light — forgot something.

W. H. Heydt
Posts: 12648
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: Why does SD cards do that?

Fri Nov 02, 2018 4:26 pm

hippy wrote:
Fri Nov 02, 2018 2:32 pm
W. H. Heydt wrote:
Fri Nov 02, 2018 1:06 am
Going to "read only" is simply a method of allowing you to retrieve any data that can be retrieved without taking the risk of doing more rewrite actions and making the corruption worse.
I am not convinced that's right, that any SD card is deliberately forcing itself permanently read only or that any Linux or other system would instruct it to go permanently read only when it determined file system corruption was present.
I agree that a Linux system won't do that. *nix systems will simply throw errors and suggest running fsck to try fix the file system.

For flash memory, it's a "fail safe" operation.

Return to “Off topic discussion”