jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Fri Sep 06, 2013 6:26 pm

1: Where were the problem SD Cards purchased? (Asking simply due to the high instances of false-capacity counterfeits on the likes of eBay, dhgate, and so on, over the last decade or so. Further info is available here, for anyone interested.)

2: Have the SD Cards been tested as genuine using a dedicated tool for this specific purpose? Specifically, F3 on Linux/Mac OS X, or H2TestW on Microsoft Windows.
You missed the part where I said I used dd, binary compare and md5sum to test. IE each and every block of the SDcard has had some random contents copied onto it - then is verified in read. This is all the windows test software does.

From a command line of a linux PC with a USB card reader (device /dev/sdb on my PC) it looks something like this:

Produce 4GB (decimal) of random data
# dd if=/dev/random of=randomdata bs=4000M count=1
Copy data onto card
# dd if=randomdata of=/dev/sdb
<wait for completion plus a long pause for cache to clean, then unplug and re-plug usb device)
# md5sum /dev/sdb
<a result>
# md5sum randomdata
<a result>

If the two results match then you can be pretty sure that 1) every sector copied ok 2) the card is not a fake (remapped) as this would drastically change the file hash. This is one of several tests I did but is the most informative.

Jon
PS this isnt exactly what I typed but close enough to show the idea, I wrote and read the actual number of blocks the device reported instead ....

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1570
Joined: Sat Sep 10, 2011 11:43 am

Re: Unreliable Unreliable Unreliable

Fri Sep 06, 2013 6:38 pm

jonshouse wrote:
If so then send that card to me at
Ok, I found a card - tested it on a couple of Pi boards. This card frequently corrupts on Pi boards but verifies as good when tested on a PC.

It will go in the first post tommorow so you should get it sometime during the week.

Thanks,
Jon
When you say "tested it on a couple of Pi boards" do you mean you reproduced the corruption twice? If so what is the current state of the card? Is it in a corrupted state?

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Fri Sep 06, 2013 6:48 pm

gsh wrote:
jonshouse wrote:
If so then send that card to me at
Ok, I found a card - tested it on a couple of Pi boards. This card frequently corrupts on Pi boards but verifies as good when tested on a PC.

It will go in the first post tommorow so you should get it sometime during the week.

Thanks,
Jon
When you say "tested it on a couple of Pi boards" do you mean you reproduced the corruption twice?
I cant guarantee that this card has been corrupted more than once, but this brand of card suffers on the Pi. I *think* this card has been used several times and always corrupts, but its tricky to tell as I have several of this type. It is a good canidate to test/debug the RPI marginal sd card interface - or at very leat update the supported card list to say "doesnt work"
If so what is the current state of the card? Is it in a corrupted state?
Yes, it currently has corrupted copy of Raspian on it. Now you mention it probably not an up to date one though.

Nobody did mention if the firmware with the sd card fix was rolled into the Raspian installer? If not then the card would probably corrupt before it could be apt-get upgraded?

Do you want me to test again from scratch with newer raspian (the installer I have on my local disk has the same md5 hash as the one currently linked to on the download page - this worries me slightly - see above ....)

Jon

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1570
Joined: Sat Sep 10, 2011 11:43 am

Re: Unreliable Unreliable Unreliable

Fri Sep 06, 2013 7:28 pm

The current Raspbian version on the downloads page contains the timeout fix. Be good to just run through the process of imaging, booting it a couple of times and it getting corrupted. Just note down the number of reboots it took in each case just so I know how many times to reboot before giving up!

It's unlikely but if it is an environmental problem I need to know the exact sequence of things you did to reproduce the problem otherwise I may not see the same problem.

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

jonshouse
Posts: 44
Joined: Sat Aug 11, 2012 7:10 pm

Re: Unreliable Unreliable Unreliable

Fri Sep 06, 2013 8:20 pm

I need to know the exact sequence of things you did to reproduce the problem otherwise I may not see the same problem.
Ok, I will do that tommorow. I will download the installer again just in case and start from scratch making notes just to make sure.

Anyone else reading this please feel free to join in and try it .......

On the hardware front.

Looking at the Pi schematic you are assuming the SOC internal pullup is <=50k, I expect Broadcom claim it is but that does not make it so... Also it assumes the impediance is flat with voltage, maybe - maybe not..... personally I would try adding real 47k external pullups and turn the internal ones off in the firmware as my first test.... just me 2c worth. If I get some free time I will try adding some external pullups but the REAL test would be like I say, external pullups and turn the internal ones off.

Thanks,
Jon

User avatar
LemmeFatale
Posts: 253
Joined: Fri Feb 01, 2013 8:47 pm
Location: UK

Re: Unreliable Unreliable Unreliable

Sat Sep 07, 2013 12:29 am

jonshouse wrote:
1: Where were the problem SD Cards purchased? (Asking simply due to the high instances of false-capacity counterfeits on the likes of eBay, dhgate, and so on, over the last decade or so. Further info is available here, for anyone interested.)

2: Have the SD Cards been tested as genuine using a dedicated tool for this specific purpose? Specifically, F3 on Linux/Mac OS X, or H2TestW on Microsoft Windows.
You missed the part where I said I used dd, binary compare and md5sum to test. IE each and every block of the SDcard has had some random contents copied onto it - then is verified in read. This is all the windows test software does.

From a command line of a linux PC with a USB card reader (device /dev/sdb on my PC) it looks something like this:

Produce 4GB (decimal) of random data
# dd if=/dev/random of=randomdata bs=4000M count=1
Copy data onto card
# dd if=randomdata of=/dev/sdb
<wait for completion plus a long pause for cache to clean, then unplug and re-plug usb device)
# md5sum /dev/sdb
<a result>
# md5sum randomdata
<a result>

If the two results match then you can be pretty sure that 1) every sector copied ok 2) the card is not a fake (remapped) as this would drastically change the file hash. This is one of several tests I did but is the most informative.

Jon
PS this isnt exactly what I typed but close enough to show the idea, I wrote and read the actual number of blocks the device reported instead ....
Ah, ok - thankyou!

Whilst I didn't miss it, I wasn't 100% sure if the manual process differed any from what the automated software for Linux, Mac OS X, and Microsoft Windows does.

I would still be interested to know where the cards came from, but honestly I'm not sure quite why - I'm sure you know where to avoid. :)
Classic - Raspberry Pi Model B (512MB) with Motorola Atrix Lapdock
Lemcon-One - Raspberry Pi Model B (256MB) PiMAME TV-Box

User avatar
pluggy
Posts: 3635
Joined: Thu May 31, 2012 3:52 pm
Location: Barnoldswick, Lancashire,UK
Contact: Website

Re: Unreliable Unreliable Unreliable

Sat Sep 07, 2013 10:08 am

Since hard and fast information about SD cards is in very short supply in this thread. I'd hazard a guess that those having issues with corrupting SD cards are using large fast expensive SD cards, whilst us cheapskates using small cheap slower cards aren't.
Don't judge Linux by the Pi.......
I must not tread on too many sacred cows......

User avatar
Mortimer
Posts: 925
Joined: Sun Jun 10, 2012 3:57 pm

Re: Unreliable Unreliable Unreliable

Sat Sep 07, 2013 10:35 am

I've been using a couple of orange, 32GB, Class 10, Samsung SDHC cards.

I've had them since March and one in particular is in the Raspberry Pi that I attach to my Atrix Lapdock, so it is shutdown after each session. So is in effect rebooted several times a day. I haven't had any problems with either car yet.

If it helps, the detail on the back of the card:

SAMSUNG
MB-SP8GC
MBSPBGVEQAFA-XH
D HJ8226GF 310
--------------
The purpose of a little toe is to ensure you keep your furniture in the right place.

lenkf
Posts: 19
Joined: Wed Jul 10, 2013 10:02 pm
Location: S.California

Re: Unreliable Unreliable Unreliable

Sat Sep 07, 2013 7:57 pm

I have had numerous SD card corruptions. Some were caused by removing power on a stuck headless, wireless, minimally powered pi far from the router/hub/switch (outside aviarycam). Some were just strange glitches probably attributed to me learning the pi. You know its corrupted when the pi boots with fsck (file system check) errors. I learned to make backups with Win32DiskImager if I made several changes or installed/setup new apps.

I also learned NOT to unplug power immediately when the screen goes to black on shutdown, but to wait until the green LED on the board stops blinking 10 times - what's the blinking for? I quit using the power strip switch to turn on/off the pi, and removed the usb cable from the power supply.

pjc123
Posts: 922
Joined: Thu Mar 29, 2012 3:37 pm
Contact: Website

Re: Unreliable Unreliable Unreliable

Sat Sep 07, 2013 8:20 pm

wouterel wrote:
jonshouse wrote: Oh wait, I will walk over to each device in turn and plug in a keyboard and display and type shutdown then ? How many embedded devices have you designed?
No. You put a button on your embedded device and use GPIO with an interrupt handler to monitor when it gets pressed and have the interrupt routine call shutdown or poweroff.
Hummmm, you mean like the one I am using here to insure the shutdown is complete and the file systems have finished un-mounting before I kill the power, because my raspberry pi is headless. It is also useful for the case where I do have a wifi dongle plugged in but I am out of range of the signal or lose network connectivity for whatever other reasons:

https://www.youtube.com/watch?v=55IqoxoDdTc
My Raspberry Pi Project Page:

https://www.flaminghellmet.com/launch/

wouterel
Posts: 52
Joined: Thu Feb 14, 2013 3:09 pm

Re: Unreliable Unreliable Unreliable

Sat Sep 07, 2013 9:09 pm

@pjc: for example. But really any idea that uses GPIO to make an off switch would be fine with me ;-). There are quite a few around (with or without a relay that automatically cuts power after shutdown).

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2415
Joined: Thu Jul 11, 2013 2:37 pm

Re: Unreliable Unreliable Unreliable

Sun Sep 08, 2013 5:59 pm

lenkf wrote: I also learned NOT to unplug power immediately when the screen goes to black on shutdown, but to wait until the green LED on the board stops blinking 10 times - what's the blinking for? I quit using the power strip switch to turn on/off the pi, and removed the usb cable from the power supply.
The Pi cannot switch itself off. It can only be reset (via software or hardware).

When you do a sudo shutdown or sudo halt, what actually happens is that the OS will go through its shutdown routine and then tell Linux to "poweroff" or "reboot". In the Poweroff case, we have some special code that sets a magic number in the watchdog's registers, which persists across a reset. The bootcode will then read this and recognise halt/poweroff from the value left in the watchdog's registers and then stop booting.

The flash sequence is there to tell you that the bootcode has halted bootup and it's safe to turn off the power. It's a pity we can't have the orange splash screen that win95 used to have...
Rockets are loud.
https://astro-pi.org

Narf03
Posts: 243
Joined: Mon Jun 11, 2012 3:44 pm
Location: Malaysia

Re: Unreliable Unreliable Unreliable

Sun Sep 08, 2013 6:35 pm

I have tried many brand and model of SD cards, they ALL can corrupt if you just pull power, if you claims there is a brand that wont corrupt means u didnt did enough testing.

my testing is simple, have few power extensions, a timer power switch, and adapters(min 1.5 amp). with the setup, I can cycle power on and off every 15 mins, u can cycle power 48 times per pi per day, with 10 pi and run for 10 days, u can cycle power 4800 times, each pi just need to get corrupted once and they are dead forever. and let them run 24/7 for days, normally within 4-5 days, all pi's are dead.

my only solution is make the root file system read-only, and if you want some data to be stored(read/write), create another partition for your data, and that can only protect the root file system, but at least u can get the pi started up to a point that u can do something. and at that point u can repair your data partition with fsck.

WebKarnage
Posts: 25
Joined: Fri Aug 30, 2013 8:45 pm

Re: Unreliable Unreliable Unreliable

Sun Sep 08, 2013 7:04 pm

Narf03 wrote:I have tried many brand and model of SD cards, they ALL can corrupt if you just pull power, if you claims there is a brand that wont corrupt means u didnt did enough testing.
It's true you can't just power off as many have already said and the reasons have been explained, it is indeed nothing to do with make of card.

Shut down (halt) properly first and you'll be able to keep the system running fine (assuming no existing issues with the card or adapter as it turned out I had).

Narf03
Posts: 243
Joined: Mon Jun 11, 2012 3:44 pm
Location: Malaysia

Re: Unreliable Unreliable Unreliable

Sun Sep 08, 2013 7:30 pm

WebKarnage wrote:
Narf03 wrote:I have tried many brand and model of SD cards, they ALL can corrupt if you just pull power, if you claims there is a brand that wont corrupt means u didnt did enough testing.
It's true you can't just power off as many have already said and the reasons have been explained, it is indeed nothing to do with make of card.

Shut down (halt) properly first and you'll be able to keep the system running fine (assuming no existing issues with the card or adapter as it turned out I had).
yes, i just want to point out that, accept the fact, all SD card can corrupt with pi, dont blame adapter, faulty SD card, low voltage of tp1 to tp2. and yes those can cause corruption too, but even you fix all other problems, a pi can still corrupt if you simply cut power.

User avatar
LemmeFatale
Posts: 253
Joined: Fri Feb 01, 2013 8:47 pm
Location: UK

Re: Unreliable Unreliable Unreliable

Sun Sep 08, 2013 8:24 pm

Narf03 wrote:yes, i just want to point out that, accept the fact, all SD card can corrupt with pi, dont blame adapter, faulty SD card, low voltage of tp1 to tp2. and yes those can cause corruption too, but even you fix all other problems, a pi can still corrupt if you simply cut power.
And so can any magnetic hard disk drive (and, I would imagine, solid state drive, and whatever other type of storage one can imagine, as well), on any type of computer, with any processor architecture, and any brand or charity's name on it... ;) I don't think anyone said that wasn't so - we all accept it because it's a fact of computing.
Classic - Raspberry Pi Model B (512MB) with Motorola Atrix Lapdock
Lemcon-One - Raspberry Pi Model B (256MB) PiMAME TV-Box

australisblue
Posts: 5
Joined: Mon Sep 09, 2013 7:20 am

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 10:12 am

I am extremely interested to see where this thread goes and I hope some good discoveries and hopefully solutions come out of this to help get things much more reliable and the chances of any corruption small. I'm feeling a little nervous about sending some of my raspberry pi's out to the field and having corruptions while I can't easily access them.

Anyway, thanks to those who are looking at this and I hope this thread stays production :D

David

Oakham
Posts: 366
Joined: Tue Aug 20, 2013 9:11 pm

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 11:37 am

..are Industrial CF Cards any more robust ?

Transcend's CF300 cards are high-speed industrial CF cards offering impressive 300X transfer rates. With ultimate performance and durability, the CF300 CF cards are perfect for POS and embedded systems that require both industrial reliability and ultra high speed data transfer.

Transcend's CFast cards integrate the rapid, more robust SATA interface into the existing CF card form factor, offering improved performance and convenience for POS and embedded systems. Featuring enhanced speed and robust portability, the CFast cards are designed to perform like SSDs, and as such are extremely capable when used as bootable drives.

http://uk.transcend-info.com/industry/p ... unc2No=171
Searching is easy, most questions have been asked before !

technoducky
Posts: 62
Joined: Sat Jan 26, 2013 7:29 pm
Contact: Website

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 12:22 pm

jonshouse wrote:
pjc123 wrote:
jonshouse wrote: How are you shutting them down?
I pull the plug.
Say no more.
Oh wait, I will walk over to each device in turn and plug in a keyboard and display and type shutdown then ? How many embedded devices have you designed?
The Pi was never designed to be used in production environments, it is supposed to be an educational tool.
If you use it expecting it to be a perfect 100% reliable embedded computing platform then you clearly did not know what you were buying when you bought 12 of them.

australisblue
Posts: 5
Joined: Mon Sep 09, 2013 7:20 am

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 12:46 pm

technoducky wrote: The Pi was never designed to be used in production environments, it is supposed to be an educational tool.
If you use it expecting it to be a perfect 100% reliable embedded computing platform then you clearly did not know what you were buying when you bought 12 of them.
I would love to see this thread work towards solutions/improvements regardless of the original design intent of the device. I think it benefits everyone along with the Raspberry Pi cause in general to further improve something that's already pretty awesome :) I do think that in a lot of situations, shutting down the Pi cleanly is not always entirely practical. Whether this is even the problem having read through all these posts, I'm not sure. It sounds like the OP is having a worse time than most have experienced so it seems a good chance to learn what might be happening.

User avatar
rpdom
Posts: 17400
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 1:13 pm

LemmeFatale wrote:
Narf03 wrote:yes, i just want to point out that, accept the fact, all SD card can corrupt with pi, dont blame adapter, faulty SD card, low voltage of tp1 to tp2. and yes those can cause corruption too, but even you fix all other problems, a pi can still corrupt if you simply cut power.
And so can any magnetic hard disk drive (and, I would imagine, solid state drive, and whatever other type of storage one can imagine, as well), on any type of computer, with any processor architecture, and any brand or charity's name on it... ;) I don't think anyone said that wasn't so - we all accept it because it's a fact of computing.
A modern magnetic hard disk is less likely to get corrupted, as they contain certain safeguards for sudden power loss. Firstly they can have some capacitors that store just enough power to finish writing the contents of their buffer safely to the disk, then they either use capacitor power, or use the spindle motor and rotation of the disk as a generator to produce enough power for the fraction of a second it takes to move the head back to a safely parked position. SSDs probably have capacitors on board to complete the writing stage, but they have no spin or heads to park.

omega1
Posts: 113
Joined: Mon Jul 02, 2012 3:10 pm
Location: UK
Contact: Website

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 2:13 pm

I haven't read the whole thread but...

I have three Pi's, all different models/releases, all different OS's and all different SD cards, all performing different tasks...

They are battered with poor PSU's and the power unplugged randomly.

I have NEVER had an issue with any of them, the only problem I ever has was poor analogue audio which was due to a dodgy cheap ebay PSU.

Either the OP has been very, very, very unlucky with all 12 Pi's, all PSU's and all SD cards or... Well, I'm not sure what!
Get your Pi from here! http://bit.ly/18blVup

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26825
Joined: Sat Jul 30, 2011 7:41 pm

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 3:07 pm

omega1 wrote:I haven't read the whole thread but...

I have three Pi's, all different models/releases, all different OS's and all different SD cards, all performing different tasks...

They are battered with poor PSU's and the power unplugged randomly.

I have NEVER had an issue with any of them, the only problem I ever has was poor analogue audio which was due to a dodgy cheap ebay PSU.

Either the OP has been very, very, very unlucky with all 12 Pi's, all PSU's and all SD cards or... Well, I'm not sure what!
And that is why its important to find out what, because his experience is really really weird.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
LemmeFatale
Posts: 253
Joined: Fri Feb 01, 2013 8:47 pm
Location: UK

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 4:54 pm

rpdom wrote:
LemmeFatale wrote:
Narf03 wrote:yes, i just want to point out that, accept the fact, all SD card can corrupt with pi, dont blame adapter, faulty SD card, low voltage of tp1 to tp2. and yes those can cause corruption too, but even you fix all other problems, a pi can still corrupt if you simply cut power.
And so can any magnetic hard disk drive (and, I would imagine, solid state drive, and whatever other type of storage one can imagine, as well), on any type of computer, with any processor architecture, and any brand or charity's name on it... ;) I don't think anyone said that wasn't so - we all accept it because it's a fact of computing.
A modern magnetic hard disk is less likely to get corrupted, as they contain certain safeguards for sudden power loss. Firstly they can have some capacitors that store just enough power to finish writing the contents of their buffer safely to the disk, then they either use capacitor power, or use the spindle motor and rotation of the disk as a generator to produce enough power for the fraction of a second it takes to move the head back to a safely parked position. SSDs probably have capacitors on board to complete the writing stage, but they have no spin or heads to park.
I'm aware of that. But I was simply pointing out that the bad things that "can" happen if the power is suddenly cut are not unique to the Raspberry Pi, at all, as the wording of the post I was responding to appeared to imply. :P

Anyway, I really don't want to drag this thread off-topic - I'm finding it interesting to follow the productive bits!
Classic - Raspberry Pi Model B (512MB) with Motorola Atrix Lapdock
Lemcon-One - Raspberry Pi Model B (256MB) PiMAME TV-Box

User avatar
rpdom
Posts: 17400
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Unreliable Unreliable Unreliable

Mon Sep 09, 2013 5:45 pm

LemmeFatale wrote:Anyway, I really don't want to drag this thread off-topic - I'm finding it interesting to follow the productive bits!
Indeed. Apologies for the distraction. This thread is interesting and I would like to know what is causing the OP's issues too.

Return to “General discussion”