Raspberry Pi Forums

Why does USB drives, MicroSD cards and so on get corrupted?

And most of all, when they get corrupted, why does the storage support becomes read-only?

This concept seems to me really wierd, lol.

Cheers.

It has to do with the way flash memory works. The first thing to understand is that flash memory (of all types) can only tolerate being written to a relatively few number of times (typically in the hundreds to a few thousands for any given location). The second thing is that when you read or write flash memory, it's on a whole block basis (and not by the individual byte). Third, writing to flash is a fairly slow process, comparatively.

So, combining those characteristics, every flash device has a small processor (and the less space and cost involved, the smaller and less capable that processor is). It tries to keep the number of write actions as even as possible across all available blocks. As a result, if you read part of a block, the internal processor reads the whole block to some temporary storage and writes it back...somewhere else. If you write part of a block, the device reads the whole block, updates the part you're writing and writes it all back out to flash...somewhere else.

Therefore, if the power is removed in the middle of a read-rewrite process, the file system is in an undetermined state, aka "corrupted". The faster the device, and the better the internal processor, the less likely it is to get caught in this situation. Some microSD cards are easier to corrupt than USB drives, and USB drives are easier to corrupt than SSDs (which are built to handle frequent, small reads and writes). Going to "read only" is simply a method of allowing you to retrieve any data that can be retrieved without taking the risk of doing more rewrite actions and making the corruption worse.

Hence the very strong recommendation to always tell the Pi to shut down, and then wait for the activity light to stop blinking before removing the power.

Caveat...if you know that your Pi is idle, and has been for at least a minute, then it is *probably* safe to just pull the power. But if that corrupts an SD card, either you were wrong about it being inactive, or your timing was bad.

Thanks for your anwser, I understand now.

Flash memory wears out. it only has a limited number of write cycles.
The number of writes can be as low as 1000 for MLC, multi layer cells.

https://en.wikipedia.org/wiki/Flash_memory

W. H. Heydt wrote: ↑

Fri Nov 02, 2018 1:06 am

... As a result, if you read part of a block, the internal processor reads the whole block to some temporary storage and writes it back...somewhere else. If you write part of a block, the device reads the whole block, updates the part you're writing and writes it all back out to flash...somewhere else.

Writing, yes but reading?
If it's only reading, and nothing's changed, why should it write it back ... somewhere else?

linux files syytems have last accessed time stamps so reading can write to the drive unless you switch of the Atime ....

see adding
noatime,nodiratime to your fstab line for the drive

Also since secure digital was a step on from MMC primarily for cameras. When the flash memory goes bad you get a few effects 1) you can't store a new photo so you know the card is toast, 2) you can't format the card so you know it's toast, but 3) you don't lose any of the photos saved before the card toasted itself.

In general that's a good thing. With a Raspberry that's a pain in the tail, but not losing any old stuff means you may be able to clone it to a good card and carry on from where you left off.

W. H. Heydt wrote: ↑

Fri Nov 02, 2018 1:06 am

Going to "read only" is simply a method of allowing you to retrieve any data that can be retrieved without taking the risk of doing more rewrite actions and making the corruption worse.

I am not convinced that's right, that any SD card is deliberately forcing itself permanently read only or that any Linux or other system would instruct it to go permanently read only when it determined file system corruption was present.

I can understand a particular block being taken out of circulation when there is an unrecoverable problem but going permanently read only is an extreme action for an SD Card to take when it only deals in data blocks and has no care what the data within those blocks mean.

My belief is that power being pulled causes the SD Card controller to malfunction, misbehave or misinterpret commands sent to it, and somehow the read only flag ends up being permanently set rather than it being a deliberate and intended action.

It could alternatively be that power being pulled causes the SoC to send an incorrect or corrupted command to the SD Card which cause it to set the read only bit.

Burngate wrote: ↑

Fri Nov 02, 2018 10:09 am

W. H. Heydt wrote: ↑

Fri Nov 02, 2018 1:06 am

... As a result, if you read part of a block, the internal processor reads the whole block to some temporary storage and writes it back...somewhere else. If you write part of a block, the device reads the whole block, updates the part you're writing and writes it all back out to flash...somewhere else.

Writing, yes but reading?
If it's only reading, and nothing's changed, why should it write it back ... somewhere else?

Generally it doesn't but it can happen that constantly reading a cell can alter an adjacent cell in the same block. Some controllers may keep tabs on how many times a block has been read since last written and decide to move it. Also this helps where, for example if 90% of the memory is used but only read from. Without moving the read only blocks would mean 10% of the memory is always used for writing, that would wear out quickly whilst the rest of the memory would still be good for a lot more cycles. Moving a block that is normally only read from to a block which is running low on usable write cycles gives you a longer working life.

hippy wrote: ↑

Fri Nov 02, 2018 2:32 pm

W. H. Heydt wrote: ↑

Fri Nov 02, 2018 1:06 am

Going to "read only" is simply a method of allowing you to retrieve any data that can be retrieved without taking the risk of doing more rewrite actions and making the corruption worse.

I am not convinced that's right, that any SD card is deliberately forcing itself permanently read only or that any Linux or other system would instruct it to go permanently read only when it determined file system corruption was present.

I agree that a Linux system won't do that. *nix systems will simply throw errors and suggest running fsck to try fix the file system.

For flash memory, it's a "fail safe" operation.