christopher.landress
Posts: 5
Joined: Sat Aug 25, 2012 7:03 am

USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 3:46 am

I have a sophisticated setup that uses two rpis that work in conjunction to provide GPS and 3G services. The 3G modem I use is a Huawei e220 which happens to have a small amount of on-board memory that contains drivers for various operating systems. Since the partition is formatted as CDFs; linux recognizes it as a virtual cd-rom and tries to mount it. The real problem comes in when the modem draws a large amount of current (like it does when it tries to enumerate the bus) and then does the unthinkable.... it overwrites the OS SD card with the mounted memory from the modem. Also, because the partition on the modem is CDFs (read-only by design), it renders the SD card completely useless once its overwritten... not completely useless to the machine, but also physically useless as well since you cant remove a read-only partition from the SD card (even with a low-level reformat tool). Therefore ruining the SD card permanently. I can reproduce this problem; and have the SD cards to prove it. Can anyone offer any ideas on what is happening? I find it incredible that usb mounted memory can overwrite the system partition.

User avatar
Jim JKla
Posts: 2218
Joined: Sun Jan 29, 2012 11:15 pm
Location: Newcastle upon Tyne UK

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 9:24 am

Have you tried using the format in a camera I have actually salvaged SD Cards from beyind the pail with this method when all others have failed.

And I know it sounds silly but make sure you have not flipped the lock switch on the side of the SD card, this is not actually read or even sensed by the RaspberryPi so the card could be changed in your RPi and blind to any other reader.

Oh and white space in the post would help. Man that's intense text. :D

I've added a bit to the Noob wiki page on this.
Last edited by Jim JKla on Thu Aug 30, 2012 9:36 am, edited 1 time in total.
Noob is not derogatory the noob is just the lower end of the noob--geek spectrum being a noob is just your first step towards being an uber-geek ;)

If you find a solution please post it in the wiki the forum dies too quick

User avatar
redhawk
Posts: 3465
Joined: Sun Mar 04, 2012 2:13 pm
Location: ::1

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 9:36 am

I have a Huawei E160 and it does not overwrite the system partition I would have thought this was impossible anyway.

Are you using a powered USB hub or is your dongle connected directly to the PI??

Also check the voltage output at TP1/TP2 if the PI has insufficient power this could be a likely cause for your memory corruption.

Richard S.

dms75
Posts: 25
Joined: Mon Aug 06, 2012 1:41 pm

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 9:43 am

Hi,

From The SDCARD Spec (the non simplified old one V1.9 after that they only published simplified specs wich left out timing information and writeprotection information:

"1.5.9.10. Write Protection
Two-card level write protection options are available: permanent and temporary. Both can be set using the PROGRAM_CSD command (see below). The permanent write protect bit, once set, cannot be cleared. This feature is implemented in the SD Card controller firmware and not with a physical OTP cell. "

So in theory you can permanently write protect a SDCARD and so render it useless for future use. But for this to happen the Linux SDCARD Driver would have to send a PROGRAM_CSD command (CMD27) to the SDCARD with the appropriate writeprotection bits set. I cannot imagine the SDCARD driver even sending a PROGRAM_CSD command. (for normal operation you'd only read it during initialisation to get info about the card). It might be bossible for low power situations that a voltage drob might flip a Bit to zero on the bus i suppose, but the original command must be almost identical to the resulting PROGRAM_CSD command for that to occur.
The CSD register of the SDCARD is a 128 Bit data structure (Bit 127 to Bit 0) which holds the permanent write protection bit at position 13 (one time programmable) and the
temporary write protection bit at position 12 (programmable several times)
It can be read with the SEND_CSD command (CMD 9).
These Bits affect the whole card.

optionally there are also the commands
CMD28 SET_WRITE_PROT
CMD29 CLR_WRITE_PROT
CMD30 SEND_WRITE_PROT
These commands aply to writegroups on the SDCARD but not all SDCARDS implement those commands.

Either way if the card is not permanently write protected it can be erased using a force erase, wich will work even if the card was locked with a password like in some Windows Phones which could render cards useless for other devices. So to recover your SDCARDS you'll want to do a force erase.
To do a force erase you have to send a CMD 42 (while the card is in transfer state) to the card with Bit_3 set to 1 and Bit_2, Bit_1 and Bit_0 set to 0.
This shold remove all temporary write protection bits from the CSD register and Group Write protections from the commands 28-30.

The tricky bit will be sending those commands to the card. You could try to connect the card via SPI Bus and to talk to the card via spi protocol. If you search the forums someone already wrote a SDCARD driver for the SPI bus for a second SDCARD slot on the RPi i think. It might be worth looking at that. You'd have to initialize the card and get it into transfer state before you could send the CMD 42. I suppose some programming with the progamming language of your choice will be required to do so. Luckily the RPi does have a SPI interface on the GPIO Header that you could use since the SDCARD slot will have to hold the OS you'll be using - your chance to learn something about the SDCARD standard, and learning / teaching is what this device is all about, right? ;-)

So if you want to investigate, first I'd check via CMD_9 (SEND_CSD) wether your card is really permanently write protected, wich I doubt very much.
(If it really is, it might be interesting to see from where it got those commands but for that you'd probably want to attach some SPI protocoll analyzer to the bus or some digital Oscilloscope with SPI decoding / triggering functionallity. And yes devices like that are not cheap, if you're lucky you have access at them at work i suppose...)
And if those faulty commands result from insuficient powersupply then a solution might be to use a good quality Powered USB Hub wich does not draw power from the RPi, or to use a usb cable with the 5V power connection cut to connect the powered hub. You might also solve the problem by modding your RPi (replace the USB Polyfuses with 1/2 Ohm Resistors and place a large Capacity behind them to provide energy for the USB device when hotplugging). I think this is also extensively covered in other threads on the forum.

So if you are curious wether you can recover your SDCARDs I'd reccomend you to have a look at these documents here:
http://www.senita.de/downloads/SD-Card- ... rdv1.9.pdf
https://www.sdcard.org/downloads/pls/si ... 100518.pdf

The first one is the old spec before there were SDHC cards, but with a lot of information that has been left out in later simplified versions.
The second one is the new spec, but in simplified form (Full spec can only be obtained by SD Card Association members). It lacks timing information but you can get that from the old doc and it lacks info on some other commands like write protection and security features. On the other hand it documents most of the new commands and datastructures that came with SDHC and SDXC.
So taking those 2 documents together there is all the information you'll ever need to write a SDCARD driver or to interface a your RPi to a SDCARD.


Good luck in investigating

Dirk

User avatar
Jim JKla
Posts: 2218
Joined: Sun Jan 29, 2012 11:15 pm
Location: Newcastle upon Tyne UK

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 9:46 am

Or you could check the lock switch. :D
Noob is not derogatory the noob is just the lower end of the noob--geek spectrum being a noob is just your first step towards being an uber-geek ;)

If you find a solution please post it in the wiki the forum dies too quick

dms75
Posts: 25
Joined: Mon Aug 06, 2012 1:41 pm

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 9:51 am

If the RPi would care about the lock switch...
It is in the responsibility of the SDCARD-reader to check for the lock swich (The card itself cannot sense its position) and afaik the SDCARD-slot on the RPi cannot sense the lock switch position either because that signal is not connected.

Edit: The format of a camera might actually do you the favor of sending a force_erase to the SDCARD, so that's worth a try. A device wich is known to be able to send a force_erase is a Nokia N8 for example. Some cameras might also do that prior to formating.

Dirk

User avatar
Jim JKla
Posts: 2218
Joined: Sun Jan 29, 2012 11:15 pm
Location: Newcastle upon Tyne UK

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 10:18 am

:lol: It's the responsibility of the user to check the lock switch :lol:
Noob is not derogatory the noob is just the lower end of the noob--geek spectrum being a noob is just your first step towards being an uber-geek ;)

If you find a solution please post it in the wiki the forum dies too quick

christopher.landress
Posts: 5
Joined: Sat Aug 25, 2012 7:03 am

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 10:39 am

I tried all the n00b stuff :) It's definitely not the SD card lock. I don't even think the pi would boot with a read only root partition. I would test this now, but all my SD cards have an un-deleteable CDFs partition. I was able to reproduce this behavior 3 times (and would be willing to send the cards to anyone interested for lower level investigation). I even removed one of the card's lock switches with my teeth and taped up the slot with scotch tape to simulate a permanently unlocked card and subjected one of the others to every sd-card "fixer" I could find.

Dirk, is there an easy way for me to check the burn lock bit without mounting the drive or having external equipment? I'm a software guy at heart, though I am learning the electrical side slowly (one IC at a time). I think the foundation would be on rev 2 before I read, understood, and applied the material you provided.

I don't really care about "saving" the cards per-say, but I am concerned that there is a serious problem with the USB driver. Both pi's are individually powered by 3.8 A 5.5 volt power supplies and are completely headless (no kb, mouse, monitor)... just built in eth0, sshd, and one USB device (GPS and GSM respectively) per pi.

I'll be replacing the polyfuses before putting it back together and also wiping the modems onboard memory.

Aydan
Posts: 697
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 11:30 am

Hi Christopher,
I have an extra SD slot wired up to my pi, and a python script for data transfers.
I could check out the SD cards if you want and if possible wipe them.

Regards
Aydan

dms75
Posts: 25
Joined: Mon Aug 06, 2012 1:41 pm

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 11:43 am

Without further hardware it might be possible to boot a kernel with a small initramfs form a working SDCARD. After booting you could remove the SDCARD, unload the SDCARD driver and insert the not working one.
After reloading the SDCARD driver it should be possible to have a look at the CSD register by doing something like:
cat /sys/class/mmc_host/mmc0/mmc*/csd
Afaik the CSD register from the card is read during initialisation and the loading of the driver/kernelmodule should do that.

Unfortunately USB based SD-card readers don't give you the CSD information (anyone correct me if I'm wrong please) so you have to use the builtin one to get it.

The commands and kernelmodules needed would have to be in a filesystem in RAM for this to work since you will be removing the SDCARD with the OS. Therefore a linux with a small initramfs might do the trick.

But unfortunately I'm still waiting for my RPi (Yes i ordered from RS... perhaps it'll come in november...) so I can't test it on real hardware.

Maybe someone with real hardware can comment on this approach?

Dirk

oddball
Posts: 3
Joined: Thu Aug 30, 2012 1:27 pm

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 1:42 pm

Hi Have you tried to use Sandisk Cruzer U3 remover ?

Link = http://mp3support.sandisk.com/downloads ... emoval.exe

I used to have the same problem with their USB-Sticks when i wanted to make them Bootable, i

think it is worth a try :-)

oddball
Posts: 3
Joined: Thu Aug 30, 2012 1:27 pm

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 3:35 pm

And offcourse this requier that you have a Windows Computer with Internal/external

sd-card reader.

Then you can allways use it on your Usb-Sticks ( to remove this bloat ) so it won't happend again.

Even if the programs gives you an error, close the program, remove the card, insert the card again

and if windows says that you need to reformat the card, then you have succeded :D

dms75
Posts: 25
Joined: Mon Aug 06, 2012 1:41 pm

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 3:48 pm

After some searching through the forums I found that the official raspbian net installer comes with a initramfs and you can open a new terminal with [Alt]+F2 or [Alt]+F3.
Since this installer will download and write stuff to your sdcard it obviously has the required Kernelmodules for the sdcard interface.
So you could try booting that, and then open a new terminal session with [Alt]+F2 and after doing so swap the sdcards to check the CSD register of the write protected card.

How to reload the sdcard driver so it will reinitialize / reread the CSD register of the sdcard you want to test i cannot say (does the RPi detect card insertion and automatically reinitialize new cards?), as I'm not that familiar with linux (if its a seperate kernel module a rmmod followed by a modprobe of the module would probably do the trick, if its compiled into the kernel I don't know how to reinitialize the sdcard driver), but I'm pretty sure someone else on this forum can help out with that.

Anyhow if you manage to read the CSD from the non working cards I'd be interested what they contain.

Dirk

PS.: here is the link to the raspbian net installer:
http://archive.raspbian.org/installer/r ... -19-12.zip

fredjam
Posts: 83
Joined: Thu Jul 19, 2012 3:19 am
Location: London UK

Re: USB Device Overwrites SD Card (permanently)

Thu Aug 30, 2012 11:17 pm

To work a sd card needs to have a partition table and at least one valid partition.
When a pi crashes it can sometimes destroy the sd card partition table. This will make it
look like a dead card, You won't be able to reformat it or read from it.

you need to look at the card on a linux computer using the 'gparted' partition editor
This will enable you to write a new (msdos) partition table to the card and to create a
new partition which you can then format again and use for loading the pi operating
system.

If 'gparted' can't resurrect the card then it probably really is dead.

Aydan
Posts: 697
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: USB Device Overwrites SD Card (permanently)

Fri Aug 31, 2012 6:50 am

fredjam wrote:To work a sd card needs to have a partition table and at least one valid partition.
That's not all true. The SD card has a "partitionless" mode where the filesystem is the top level.
This is the flag "FILE_FORMAT" in the CSD register (bits 11:10).
This seems to be only valid for CSD version 1.

Regards

fredjam
Posts: 83
Joined: Thu Jul 19, 2012 3:19 am
Location: London UK

Re: USB Device Overwrites SD Card (permanently)

Sat Sep 01, 2012 12:55 am

A wonderful academic post but perfectly useless in solving the practical problem
these people have..

gritz
Posts: 449
Joined: Sat Jan 28, 2012 2:33 am

Re: USB Device Overwrites SD Card (permanently)

Sat Sep 01, 2012 2:23 am

Solving practical issues is a world away from quoting Wikipedia. ;)

Wikipedia never stopped me from being showered in ceramic slurry at my last job, no matter how "unlikely" the problem was.

christopher.landress
Posts: 5
Joined: Sat Aug 25, 2012 7:03 am

Re: USB Device Overwrites SD Card (permanently)

Sat Sep 01, 2012 4:09 am

I sent the card along with one of the pis to an old engineering co-worker that has much more experience with hardware than I. I'll update the thread once he has some additional information. I read all the relevant information on parted and fdisk and was able to confirm that block device itself is locked as read-only. Sorry I can't provide any further information.

I did however buy two new sdcards along with one of the pi certified powered 7 port hubs and it appears to have fixed all my current draw issues (this hub allows full power across all 7 ports). Now I can stuff both the GPS and GSM on the same pi with no issues whatsoever.

dms75
Posts: 25
Joined: Mon Aug 06, 2012 1:41 pm

Re: USB Device Overwrites SD Card (permanently)

Sat Sep 01, 2012 9:00 am

Well, since it seems to be narrowed down to a powersupply related problem reading the sdcards CSD register will only serve in determining wether the card can be recovered or not, and of course to satisfy our curiosity. The time spent to read the CSD register might not be worth the cost of the cards, so it boils down to curiosity ;-)
Frankly it's the first time i have encountered a writeprotected sdcard (using the internal sdcard features not the sliding switch on the card that is) in the wild and I find it surprising that voltage drops on the bus might result in this condition.
Does the RPi onboard sdrard slot use the proprietary 4 bit sd-mode for communication?
If so the default settings of an sdcard are to protect all commands and all datapackets with a CRC and this should reduce the chance of accidentially getting a different comand greatly. However if the interface works in SPI Mode the default is for non CRC transmission iirc. Still unlikely for a corrupted command to make sense but not as unlikely.
Anyhow I'm glad the new hub fixed your problem and wish you good luck with your GPS and 3G services project on your RPi.

Dirk

oddball
Posts: 3
Joined: Thu Aug 30, 2012 1:27 pm

Re: USB Device Overwrites SD Card (permanently)

Wed Sep 05, 2012 8:28 am

I wrote this earlier:

Hi Have you tried to use Sandisk Cruzer U3 remover ?

Link for download = http://mp3support.sandisk.com/downloads ... emoval.exe

Link to site = http://u3.sandisk.com/launchpadremoval.htm

I used to have the same problem with their USB-Sticks when i wanted to make them Bootable, i

think it is worth a try :-)

Okey, i have tested this now and i know it works.

You have to have a computer with Xp or Win7 on it and a usb sd-card reader.

It might say it was unsuccesfull removing, but eject and reinsert the card, the Windows will tell

you that the card is unfomated and ask you if you want to format it.

Return to “General discussion”