jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Unreliable Unreliable Unreliable

Thu Sep 05, 2013 1:29 pm

I have invested months designing hardware around the raspberry Pi, wasted time I think.

The Pi board itself is unusable. It is simply the most unstable single board ARM computer I have ever used.

I produced a number of products with other ARM boards using USB keys for the file-systems and all are more reiable than the Pi board.

Why the constant SD corruption ?
Has anyone put any time into figuring out why ?

I have tried my digital scope, the PSU is nice and stable - no voltage drops even under high current load, yet the Pi still corrupts its card OVER AND OVER AGAIN !!!

I have 12 Pi main boards - over 15 YES 15 ! sdcards of differing makes, 4 PSU including some I have bult from 2Amp buck converters powered by 12v 4Amp power adapters, so its not current issue.

WIth no other boot mechanism than the SDcard I have no choice but to offer the whole lot for sale on ebay and consider the past 5 months wasted effort.

Unless anyone can offer a REAL solution?

User avatar
mahjongg
Forum Moderator
Forum Moderator
Posts: 12347
Joined: Sun Mar 11, 2012 12:19 am
Location: South Holland, The Netherlands

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 1:34 pm

how are you powering down? did you use over clocking? did you try another brand of SD-card?

Dutch_Master
Posts: 362
Joined: Sat Jul 27, 2013 11:36 am

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 1:40 pm

Now that you've blown off steam, go back to the basics: one RPi, one 1 Amp micro-USB power supply, a simple mouse+keyboard and HDMI monitor. Load Raspbian on a 4 GB SD-card and use common sense. Don't remove it from its slot while the RPi is on. Don't expect too much performance from a device that's never intended as a high-performance platform but a robust teaching tool for youngsters. If it corrupts the card, find out why it's corrupting the card. Is the card itself OK? If you offer a defective SD card to the RPi, it's quite unfair to blame that device for corrupting the card, isn't it?

User avatar
mahjongg
Forum Moderator
Forum Moderator
Posts: 12347
Joined: Sun Mar 11, 2012 12:19 am
Location: South Holland, The Netherlands

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 1:47 pm

WIth no other boot mechanism than the SDcard
Although the PI must have an SD-card to read boot code from, you can divert the booting to another device, and boot from that, for example from a USB thumb drive.
Examples of how to do it can be found.

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 1:50 pm

I'd like to offer you a dollar each for the 12 Pis.

I think that is a good deal for you, since they are otherwise useless to you.

We can work out shipping details privately. Feel free to PM me.
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23912
Joined: Sat Jul 30, 2011 7:41 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 3:33 pm

I've said this before, but its worth repeating, I have never had an SD card corruption. So, we need to get to the bottom of why the OP is seeing so many problems, over multiple Raspi's and SD cards.

So, some questions.

What revision of Raspi do you have? (approx serial numbers might help)
What SD cards are you using and what size?
What HW are you attaching to? (You said you had been designing HW)
How are you shutting them down?
What OS are you using? And is it up to date?
What environments are they running in?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 3:56 pm

So, we need to get to the bottom of why the OP is seeing so many problems, over multiple Raspi's and SD cards
Or not. I think it is abundantly clear at this point that the OP's ship has sailed. Nothing in his post indicates that he is asking for, looking for, or in any way interested in help. Instead, he seems intent on cutting his losses, and I don't think we'd be doing him any favors trying to disuade him from this path.

My entirely generous offer still stands, although I can't really guarantee it beyond today. After midnight (EDT), the offer may need to be renegociated. So, I'd act soon if I were you.
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23912
Joined: Sat Jul 30, 2011 7:41 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 3:59 pm

Joe Schmoe wrote:
So, we need to get to the bottom of why the OP is seeing so many problems, over multiple Raspi's and SD cards
Or not. I think it is abundantly clear at this point that the OP's ship has sailed. Nothing in his post indicates that he is asking for, looking for, or in any way interested in help. Instead, he seems intent on cutting his losses, and I don't think we'd be doing him any favors trying to disuade him from this path.

My entirely generous offer still stands, although I can't really guarantee it beyond today. After midnight (EDT), the offer may need to be renegociated. So, I'd act soon if I were you.
It may not help the OP, but it would help in debugging what the problem is...
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
Speedwell68
Posts: 178
Joined: Sun May 12, 2013 5:21 am
Location: Cornwall, UK

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 5:10 pm

I have to say in nearly a year of running an RPI I haven't suffered a single corruption and it is over clocked.

User avatar
pluggy
Posts: 3635
Joined: Thu May 31, 2012 3:52 pm
Location: Barnoldswick, Lancashire,UK
Contact: Website

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 5:16 pm

We could do with getting to the bottom of this James. It isn't universal or you and I would be plagued by SD card corruption as well. The don't pull the plug without stopping the Pi first and don't overclock are sounding very tired. I break these 'rules' quite often and don't have any issues. 1 corrupted card in over a year with 5 Pis and that was the result of a thunderstorm. I don't use anything over 8GB, Class 4 and usually buy Sandisk if its any help. I buy my power supplies and SD cards off ebay....
Don't judge Linux by the Pi.......
I must not tread on too many sacred cows......

User avatar
DeeJay
Posts: 2027
Joined: Tue Jan 01, 2013 9:33 pm
Location: East Midlands, UK

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 5:22 pm

Speedwell68 wrote:I have to say in nearly a year of running an RPI I haven't suffered a single corruption and it is over clocked.
If @jonshouse is still listening, that's why his ability to apparently 'kill' SD cards on such a regular basis would be of value to people like @jamesh at the Foundation, so that they can attempt to work out why there is such a divergence of user experience.

So @jonshouse - I understand your frustration, but please would you consider responding to the queries that @jamesh posed earlier today?
How To Ask Questions The Smart Way: http://www.catb.org/~esr/faqs/smart-questions.html
How to Report Bugs Effectively: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

User avatar
rikas
Posts: 17
Joined: Sun Jul 31, 2011 1:28 pm
Location: Austria y Argentina

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 5:36 pm

Maybe, its the wrong place to suggest (admin will you move my message in this case?), but ever and ever again reading about damaged SD-cards:

I would suggest, to offer an alternate kernel with "aufs" (or unionfs)

* sure, none of the two is in the mainline kernel,
* I know about all the discussions regarding overlay FS in the kernel
* sure, it is more work: integrate in building process, patching, compiling and offering the alternate kernel
* but wouldn't this newcomers save a lot of trouble?

The basic idea is: provide a third partition, along with "shutdown" automagically save the overlay to that partition.
Power fail = changes lost (ok, that one of course will not change, but the SD is not in danger) - moreover, it could be a good lesson!
As an example see here: http://www.raspberrypi.org/phpBB3/viewt ... 29&t=24794 - micerinos and MatF's solution

Building a kernel for sure is nothing for newbies (still I do have my kind of errrrrrrrors - mostly in layer 8 !)
Definitely I am not a newbie - but power fail happens as a matter of fact ( and frequently I could bang my head ... obviously also not a kernel hacker).
Honestly, updating, redo the patches, reconfigure, recompile cost a lot of time (here talks laziness: I have to do that during my development quite frequently)

I happily offer to cleanup my notes (which are a big mess at the moment) on this subject and provide a guide on using an overlay FS (along with some supporting scripts) - which should be helpful for newcomers!

Raspberry and Raspbian in my opinion have done and do a terrific job to educate!
Maybe, it's worth to think about my suggestion to even further improve the "Raspberry Pi experience".
( just my 2 cent )


Richard
-- rikas

pjc123
Posts: 913
Joined: Thu Mar 29, 2012 3:37 pm
Contact: Website

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 5:47 pm

I will go $1.25 for each pi. Seriously, a year has gone by and not a single corrupted card on each of my two pis. However, I run my pis off of UPS's or directly off of batteries, and always do a proper shutdown, just like I do with my many Linux desktop computers. I will tell you that I can hear the UPS go off several times a day, and that the power glitches (whether they are dips or surges) are undetectable by the naked eye, meaning I do not see the lights flicker when this happens, so without the UPS I would never know that the power coming into the house was that unreliable.
Last edited by pjc123 on Thu Sep 05, 2013 6:03 pm, edited 1 time in total.
My Raspberry Pi Project Page:
https://www.flaminghellmet.com/launch/

User avatar
FTrevorGowen
Forum Moderator
Forum Moderator
Posts: 5113
Joined: Mon Mar 04, 2013 6:12 pm
Location: Bristol, U.K.
Contact: Website

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 5:52 pm

pluggy wrote:We could do with getting to the bottom of this James. It isn't universal or you and I would be plagued by SD card corruption as well. The don't pull the plug without stopping the Pi first and don't overclock are sounding very tired. I break these 'rules' quite often and don't have any issues. 1 corrupted card in over a year with 5 Pis and that was the result of a thunderstorm. I don't use anything over 8GB, Class 4 and usually buy Sandisk if its any help. I buy my power supplies and SD cards off ebay....
This "struck a chord" - I have a similar number of Pi's and also mainly use 4Gb or 8Gb cards, a mix of "true" SDHC and micro-SDHC's + adapter. Most are class 4's, a few class 10 and one is a 16Gb class 4 with the root filesystem expanded to use about half of the card and an extra FAT32 partition using the remainder. So far only one card has possibly been corrupted, but, not necessarily by a Pi, since it was "ex-mobile phone". Re-formatting with the SD association's tool seems to have got it working again.
Although I've had my first (B1) Pi for just over a year I've only been involved here for a few months but, w.r.t. SD card "corruption" issues I've noticed that it appears to be "worse" (more likely) for larger cards (16Gb, 32Gb, 64Gb). Of course it could just be that the owner's of these, more expensive cards are more likely to complain.
I strongly agree with jamesh's requests to the O.P. - although now retired, nearly 40 years in R & D taught me that you have to ask questions (that sometimes might seem stupid, inappropriate ...) in order to solve a problem (which often turned out to be something else entirely).
Trev.
Still running Raspbian Jessie or Stretch on some older Pi's (an A, B1, B2, B+, P2B, 3xP0, P0W, 2xP3A+, P3B+, P3B, B+, A+ and a B2) but Buster on the P4B's. See: https://www.cpmspectrepi.uk/raspberry_pi/raspiidx.htm

User avatar
pluggy
Posts: 3635
Joined: Thu May 31, 2012 3:52 pm
Location: Barnoldswick, Lancashire,UK
Contact: Website

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 6:04 pm

I've just spent the last half hour trying to corrupt an SD card in one of mine. Overclocking, pulling the plug during busy times and SD card writes and even part way through the boot up. No go, the SD card is still booting up like nothing happened. Maybe I'll buy a big fast card and see if that will corrupt.
Don't judge Linux by the Pi.......
I must not tread on too many sacred cows......

Tzarls
Authorised Reseller
Authorised Reseller
Posts: 225
Joined: Tue Feb 26, 2013 6:59 am

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 6:43 pm

pluggy wrote:I've just spent the last half hour trying to corrupt an SD card in one of mine. Overclocking, pulling the plug during busy times and SD card writes and even part way through the boot up. No go, the SD card is still booting up like nothing happened....
Maybe you Pi is faulty..... :lol:

Sorry, couldn´t resist...

Now, during the time I´ve had my Pi I´ve only got 1 corruption. I´ve used a 4Gb class 2 card (can´t remember brand) and an 8Gb class 6 Sandisk card. The corruption happened with the first card when I overclocked the Pi to 1Ghz and booted Openelec. After thatI´ve stayed at "stock" clock speed and a few times had to pull the plug without shutting correctly but never had a corruption.

jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 7:39 pm

mahjongg wrote:how are you powering down?
Why would this matter - seriously.... this is touted as "the reason it corrupts" but its not based on fact or reason.

A proper disk device driver will only write the sectors it has been told to by the filesystem code, unless you are modifying the boot loader or the kernel then WHY would it matter how you shut it down. It does a periodic sync, with ext4 it *supposed* to recover the files that have journel corruption - entirely different from card corruption.

I see a lot of card corruption, files that are not being written to since install getting dammaged - kernel modules, O/S files etc - seemingly at random. Much like noise causing random writes ?
did you use over clocking?
Mostly no. XBMC over clocked the machine slightly, that card lasted 1 month. The experiminal O/S card I keep up to date with Raspian lasted less than two months. A lot of the cards I write seem to corrupt and trash themselves into the ground on the second or third boot.

I have tried two different usb card writers. I also did a dd of the card image to the card, then back to a file. Then a binary compare of the two files to veryify that the PC card reader was working and that the card was not fake/remapped. Worked fine, I also did the same testing using md5sum just for luck.
did you try another brand of SD-card?
4 brands. Some Kingston, some "elite pro" - whatever that is , some Sony. ALL eventually corrupt, some faster than others.

If the hardware is correct at the SDcard lines are properly terminated then ALL sd cards should work. My camera will use all cards, as will my phone - so why the is the Pi so special ?

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1442
Joined: Sat Sep 10, 2011 11:43 am

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 7:52 pm

OK,

So to solve this problem what you need to do is this:

1) Show that it is reproducible within a short period when working correctly with the latest raspbian release.

If you can't reproduce it on Raspbian and instead only on Arch Linux or RaspBMC etc that's fine, but it needs to be the latest release.

2) Document the problem and try to reduce it down to identify which piece of hardware is causing the problem...

3) Send me the combination that guarantee's the problem gets seen! In general I always send the stuff back or send better stuff in its place!

Then I can actually diagnose the problem, this is how I fixed the first issue with SD card corruption last month

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 7:56 pm

jamesh wrote:I've said this before, but its worth repeating, I have never had an SD card corruption. So, we need to get to the bottom of why the OP is seeing so many problems, over multiple Raspi's and SD cards.
So, some questions.

What revision of Raspi do you have? (approx serial numbers might help)
Wow ...... I have more than 12 of the things, purchased in three batches. Strangely the original 4 model B cards I got the month after release have not corrupted -ever .... hmmm ...... Getting serial numbers from the all would take a while as they are scattered in various devices and its a real job to move everything round and power them up. I can do if it is important.

What SD cards are you using and what size?
4 GB mostly. "Elite Pro", "Kingston" and "Sony" are the main ones.

What HW are you attaching to? (You said you had been designing HW)
I have two of these units built, so far one has corrupted once in 6 months, better than average.
http://www.youtube.com/watch?v=zHSGtInrsDY

I have another single (the original 256MB ModelB) in this:
It has never corrupted. I would have to undo a lot of screws to find what card is in it.
http://www.youtube.com/watch?v=S3ahT1oHhyE

I have 7 units built with this front panel board. Most, but not all, show corruption, some minor - some major.
http://www.youtube.com/watch?v=qbrIKQTly_s

I was hoping to sell the boards as a kit to make desktop players, but that seems impossible if the user puts the card into the machine and it promptly fails.
http://www.youtube.com/watch?v=a6VpvDC5Cs4

I dont have much documentation on my projects at the moment. The front panel board is nothing very clever. It has an SPI driven O-LED display and PIC microntroller connected to the Pi via its serial pins. It does power the Pi via its GPIO header using a 5v buck converter - BUT the machines with their own USB power supply suffer if anything more corruption. I have also measured the noise and its around 10mv, bad but less bad than the typical USB 5v power adapter.

I also have two XBMC machines, each using a standard USB power adapter.
These are a mess, neither of the two now run, one lasted 2 weeks the other only a few hours before it was un-bootable. I dont have time to look at them so they are sitting unused.

How are you shutting them down?
I pull the plug. Most the devices have no keyboard, some have no input device at all.

What OS are you using?
Raspian.

And is it up to date?
Mostly... some are right up to date. The most reliable is more than a year out of date.

What environments are they running in?
Normal domestic , nothing clever.

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1442
Joined: Sat Sep 10, 2011 11:43 am

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 8:00 pm

So you have a card that will reliably corrupt its filesystem after only a couple of reboots?

I assume this is with the latest Raspbian release?

Have you tried it in different Raspberry Pi's?

If so then send that card to me at

Gordon Hollingworth
Raspberry Pi
Mount Pleasant House
Mount Pleasant
Cambridge
CB3 0RN


P.S The reason it matters is that the SD card is not like a hard drive... When you are sat there doing nothing it is happily moving data around on the flash... If you pull the plug whilst its doing something then it will corrupt itself... We've been told by Samsung / Kingston engineers that you do have to be careful to make sure it has finished processing whilst running in the background. That's one of the things that happens when you sync...
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 8:06 pm

[quote="gsh"]OK,

So to solve this problem what you need to do is this:

1) Show that it is reproducible within a short period when working correctly with the latest raspbian release.
Ok, as best I can. Its all a little random.

If you can't reproduce it on Raspbian and instead only on Arch Linux or RaspBMC etc that's fine, but it needs to be the latest release.
Ok, its all generic Raspian so far.

2) Document the problem and try to reduce it down to identify which piece of hardware is causing the problem...
Not being funny here, but the Pi board ! I've tried several makes of PSU, does not seem to impact on the problem (best I can tell), the make of card seem to make the problem better or worse but all suffer in the end. My work style involves a lot of compiling on the Pi board, but it seems to be the XBMC machines with an entirely generic display/PSU/keyboard setup that corrupt the fastest.

3) Send me the combination that guarantee's the problem gets seen! In general I always send the stuff back or send better stuff in its place!
Ok, I can do that.

Then I can actually diagnose the problem, this is how I fixed the first issue with SD card corruption last month
Ok. Can you point me at the fix, do I need a specific firmware version to cure the problem ?

Does the Pi SDcard driver have a write sector, read sector, verify option. DOS used to call if "verify on write". IE each sector is written the read and compared the original write buffer, if so then card corruption could be detected much more quickly ?

Thanks,
Jon

pjc123
Posts: 913
Joined: Thu Mar 29, 2012 3:37 pm
Contact: Website

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 8:13 pm

jonshouse wrote: How are you shutting them down?
I pull the plug.
Say no more.
My Raspberry Pi Project Page:
https://www.flaminghellmet.com/launch/

User avatar
DeeJay
Posts: 2027
Joined: Tue Jan 01, 2013 9:33 pm
Location: East Midlands, UK

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 8:16 pm

pjc123 wrote:
jonshouse wrote: How are you shutting them down?
I pull the plug.
Say no more.
We've got a developer and a user talking to each other. I suggest we quietly let them get on with it...
How To Ask Questions The Smart Way: http://www.catb.org/~esr/faqs/smart-questions.html
How to Report Bugs Effectively: http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1442
Joined: Sat Sep 10, 2011 11:43 am

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 8:16 pm

OK,

Sounds like you've got some specific Pi's that cause the problem, thats a great step forward (believe it or not!) Send it to me at the adddress and I'll get a good look through... If you've got a specific card that seems to corrupt quicker than others send that as well (timing is really important with this stuff...)

Doing an rpi-update on your unit will get you the latest (head) release. This includes the fix we've already made... Try this first just to make sure...

No Linux doesn't have a verify on write type option (that I know about anyway!) in fact what I've seen so far is that it ignores errors that occur on writes... So doesn't retry if it goes wrong...



Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Thu Sep 05, 2013 8:17 pm

pjc123 wrote:
jonshouse wrote: How are you shutting them down?
I pull the plug.
Say no more.
Oh wait, I will walk over to each device in turn and plug in a keyboard and display and type shutdown then ? How many embedded devices have you designed?

Return to “General discussion”