Finally found out why SD cards were so unreliable on early Pi

[Heater]

In the early days I used to have a lot of trouble with SD card corruption on the Pi. Requiring a reinstall.

Recently, nothing like so much.

I often suspected there was some kernel problem back in that day but nobody ever hinted what that might have been.

Now, I think I found it. A problem identified and fixed by Rob Landley and his team.

See 30:39 into this video: "Optimizing a new processor architecture": https://www.youtube.com/watch?v=o0milqmt4ao

[ejolson]

In the early days I used to have a lot of trouble with SD card corruption on the Pi. Requiring a reinstall.

Recently, nothing like so much.

I often suspected there was some kernel problem back in that day but nobody ever hinted what that might have been.

Now, I think I found it. A problem identified and fixed by Rob Landley and his team.

See 30:39 into this video: "Optimizing a new processor architecture": https://www.youtube.com/watch?v=o0milqmt4ao

He gets a lot of laughs from the statement

They didn't expect it to be reliable.

which really does reflect the mindset some people have about the Pi.

Fortunately, I've been quite lucky and with the notable exception of a 3B and a couple Zeros all the Pi computers I've had were able to run for months at a time at standard clock speeds. Moreover, the 3B was easily fixed by reducing the clock speed. I haven't tried the same trick with the Zero computers, because I've been busy with the Dell desktop in my office which seems to crash every week. Since the memory and processor pass rigorous stress testing, the problem appears to be either with the graphics-card driver or something going to sleep and then not waking up. I think many people don't expect their office computers to be reliable either.

[6by9]

Nice find by them, although it doesn't quite match up. He references SPI MMC controllers infrastructure - the Pi controller is not over SPI. I can't see an obvious commit around this area to see if this was just a slip of the tongue.

IIRC 2014 there were two employees of Raspberry Pi - Liz and a hardware guy (Rob Bishop). At the time all support was from Broadcom employees, and was quite limited. This sort of core stuff you do expect to just work.
Same as the recent change that broke hardware checksum offload on a bundle of boards including the Pi - it took 2-3 man weeks of effort on our side identifying the cause, and that was with a fairly contained area to investigate.
It's near impossible to be an expert in all areas of the kernel as it is just so large these days. If you end up in the quieter backwaters of the kernel, then things are less tested, and that's just the way it is.

[Heater]

6by9,

...the Pi controller is not over SPI.

Ah well. It was an interesting theory.

If I understand correctly his argument was that card detect was accessing the cards asynchronously without any locking, so it would trample over other read/write access of the card going on.

Is it possible this is managed at some higher level and is not SPI specific?

I can well understand the problem with the size of the kernel and the traps waiting in those unwalked paths.

[6by9]

6by9,

...the Pi controller is not over SPI.

Ah well. It was an interesting theory.

If I understand correctly his argument was that card detect was accessing the cards asynchronously without any locking, so it would trample over other read/write access of the card going on.

Is it possible this is managed at some higher level and is not SPI specific?

I can well understand the problem with the size of the kernel and the traps waiting in those unwalked paths.

I wonder if he had a slip of the tongue and meant SDIO (not SPI) MMC controller. That would match. Interesting if so.
Then again his notes say quite clearly SPI.

[jdb]

I wonder if he had a slip of the tongue and meant SDIO (not SPI) MMC controller. That would match. Interesting if so.
Then again his notes say quite clearly SPI.

SD cards can be accessed over SPI. It part of specification compliance - cards must implement this mode. Microcontrollers frequently use SPI to access SD storage as a SPI peripheral is far simpler than an SD host peripheral.

[6by9]

I wonder if he had a slip of the tongue and meant SDIO (not SPI) MMC controller. That would match. Interesting if so.
Then again his notes say quite clearly SPI.

SD cards can be accessed over SPI. It part of specification compliance - cards must implement this mode. Microcontrollers frequently use SPI to access SD storage as a SPI peripheral is far simpler than an SD host peripheral.

Ah, I learn something new.
I knew they had several compatibility modes, but I wasn't aware they'd drop all the way back to SPI.