mmkw43
Posts: 597
Joined: Tue Dec 24, 2013 6:18 pm

Re: Corrupt SD card -- guide

Tue Dec 01, 2015 2:28 pm

Looking back on this I think the help page or video (whatever) I was looking for initially was what Doug ended up telling me to do. (thanks again Doug).

Obviously, a corrupt SD card CAN be complicated but from what I'm reading more often than not FSCK is probably going to get you out of it? (I see that in many posts). So, whether that will work or not, at least it's a good starting point and it's simple to do.

So why not have a troubleshooting section somewhere on this site or maybe just an obvious section somewhere saying? --

"Pi won't boot ? Corrupt SD card? -- try this"

(1) Make another Raspian SD boot card
(2) Put bad SD card in USB card reader
(3) Boot PI with new card
(4) In terminal, run FDISK -l
(5) Make note of the bad cards partitions ie /dev/SDA1 etc
(6) In terminal run For i in a1 etc etc (the command Doug gave me)
(7) Shut down PI
(8) Put faulty card in PI and attempt to boot
(9) If boot fails (give it plenty of time), shut down PI
(10) Put new SD card back in PI and the faulty card in USB card reader
(11) Boot PI -- go to file manager and cross your fingers

I'm just saying that if that had been listed somewhere, (like I mentioned in the first post) I could have been in and out in 15 minutes.
I'm guessing if the above doesn't work, chances are you're screwed.

Anyway, my next learning curve will be a better understanding of why an SD card can do this to begin with -- my impression has been that electronic memory (non moving part non magnetic) is stable. Once you "Save" you're in good shape. But I'll find out in an electronics design group I follow. I kept blowing off backing up because I had the PI B ports all utilized. Never again obviously.

So weird -- I had literally just done my last bit of code on something probably 5 or 6 pages long and the power went off.

God screwing with me again.

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: Corrupt SD card -- guide

Tue Dec 01, 2015 3:06 pm

Yes, I think that's a good summary. One more point.

If you absolutely know that some particular text is there, but you can't find it any of the files recovered by fsck (or if fsck just doesn't work), then there is one more thing to try. This is specifically when you are looking for one particular block of text - which was the case when I needed to do this.

My situation was that I had been running Raspbian off of a USB flash drive, and the flash drive went bad (while the system was running). After shutting down and doing all the usual things to try to retrieve one particular file, I ended up doing (something like):

# less < /dev/sdb

and just pawing through it (which was long and lots of bad junk - since the flash drive was 8G), but I did eventually find the text I needed.
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

Heater
Posts: 16869
Joined: Tue Jul 17, 2012 3:02 pm

Re: Corrupt SD card -- guide

Tue Dec 01, 2015 3:51 pm

Oh yeah, trawling though disk blocks trying to find lost data. Fun, fun fun...

Don't forget the "strings" command. Helps to get the textual data out of a lot of binary gunk. For example:

Code: Select all

$ strings /dev/sdb | less
Memory in C++ is a leaky abstraction .

Joe Schmoe
Posts: 4277
Joined: Sun Jan 15, 2012 1:11 pm

Re: Corrupt SD card -- guide

Tue Dec 01, 2015 3:57 pm

Heater wrote:Oh yeah, trawling though disk blocks trying to find lost data. Fun, fun fun...

Don't forget the "strings" command. Helps to get the textual data out of a lot of binary gunk. For example:

Code: Select all

$ strings /dev/sdb | less
Yeah, I think that is actually what I did (using strings to filter out the obvious garbage). I couldn't quite remember which command I ended up using.

Note, BTW that you do have to run this as root in order to have direct access to the device.
So, your prompt should be "#".
And some folks need to stop being fanboys and see the forest behind the trees.

(One of the best lines I've seen on this board lately)

vivelibre
Posts: 1
Joined: Thu Aug 03, 2017 2:42 pm

Re: Corrupt SD card -- guide

Thu Aug 03, 2017 2:48 pm

Hello guys I have to work with a lot of raspberryPi and I found something use full for those cases that when SD card can start in Rbpi I just have to plug into my SD card slot for my computer system (Windows operative system) then just open the SD as files and finally remove safely once that I finish this I just insert the SD card into the rbpi again and start working as usual

User avatar
jahboater
Posts: 6301
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: Corrupt SD card -- guide

Thu Aug 03, 2017 3:57 pm

ame wrote:
Tue Dec 01, 2015 2:44 am
mmkw43 wrote:Now I'm really worried -- months of code written in Python.
You only have one copy of months of work?

If it was important you'd have several copies in several different places.
Decades ago, a PHD student came to me with a problem, his thesis which he had spent 3 years working on, was on a computer - and the hard disk failed! No backups, nothing.
ame wrote:
Tue Dec 01, 2015 2:44 am
If it was important you'd have several copies in several different places.
Definitely. Consider the problems if a fire destroyed the building and so on.
The cloud helps with that apart from the obvious privacy issues. A USB drive with the backup on, saved with a friend or relative works too.

Gadgetguy
Posts: 166
Joined: Fri Aug 15, 2014 2:55 am

Re: Corrupt SD card -- guide

Thu Aug 03, 2017 6:36 pm

A very quick gui-based solution for file system corruption that prevents rebooting and which worked for me is to use gparted, which is in the Raspbian repository. After many many months of using a very nice super fast sandisk extreme sd card -during which time I had regularly unplugged my pi whenever it froze( usually too many tabs in web browser) and never had had a problem with it not rebooting( although the boot screen often indicated it was checking and fixing the file system) today I was dismayed to find my pi simply would't reboot. I didn't have too much stuff on it that I hadn't backed up but I did have some. Moreover it was fully up to date.

On one of my windows computers I have a couple of linux file reading utilities that will let me read and copy linux files but this computer was temporarily unavailble to me. So I rebooted the pi with another sd card that was running Raspbian, and stuck the problem sd card into a usb adapter. I was unfotunately unable to read any of the files on the problem sd card's ext4 system which was reported as being corrupt. I then loaded gparted , navigated to the problem device in the drop down menu in the top right corner. I noted the ext4 partition was flagged as corrupt. In the right-click context menu there is a command called check which checks and attempts to repair the file system. I clicked it and applied the command in the edit menu. I ejected the sd card and reloaded it in the pi and voila it rebooted. It is probably a bit of a leap of faith and it would be best to copy beforehand any files off the corrupt sd if this is possible. However this solution worked for me and may prove useful to others.

No doubt this makes me a linux heretic but I prefer if possible to use a gui based solution to problems rather than using the terminal to type in cryptic and dangerously powerful linux commands I may or may not understand.

ranjo2721762
Posts: 1
Joined: Thu Dec 07, 2017 7:40 am

Re: Corrupt SD card -- guide

Thu Dec 07, 2017 7:44 am

If you have a digital camera use it's format sd card feature and it will format it and solves the problem. I did it several times and you don't have to buy a program to fix your sd card.

Heater
Posts: 16869
Joined: Tue Jul 17, 2012 3:02 pm

Re: Corrupt SD card -- guide

Thu Dec 07, 2017 11:24 am

Yes, but that totally obliterates any data you had on the SD card.

It does not fix the file system corruption, at least well enough to boot from the card or salvage some files by mounting it on a running Linux system.

Also note that if your card is working properly, it's only you file system(s) that are scrambled, then it is not necessary to format it before writing a new Raspbian or other image to it. That operation does not care about whatever format is on there beforehand.

Also, I and many others have experienced card failures where they become wholly or partially write protected. At which point no amount of formatting by anything will fix it.
Memory in C++ is a leaky abstraction .

divya.bhargava
Posts: 2
Joined: Fri May 11, 2018 10:03 am

Re: Corrupt SD card -- guide

Tue Jul 24, 2018 10:08 am

Hi All, Lets make it simple, insert your SD card in any linux based os machine with GUI like 7 centos . and you can browse through the folders and copy/ backup your data.

n67
Posts: 938
Joined: Mon Oct 30, 2017 4:55 pm

Re: Corrupt SD card -- guide

Tue Jul 24, 2018 10:28 am

Also note that if your card is working properly, it's only you file system(s) that are scrambled, then it is not necessary to format it before writing a new Raspbian or other image to it. That operation does not care about whatever format is on there beforehand.
The "digital camera" advice - or, really, any method of formatting the SD card back to "normal" (aka, "Factory Default") - is generally given in the context of NOOBS (i.e., PINN), where it is important to format the card properly before putting the software (NOOBS/PINN) on it.

If you are using the advanced "raw image" method, then, of course, it isn't necessary.
"L'enfer, c'est les autres"

G fytc hsqr rum umpbq rm qyw rm rfc kmbq md rfgq dmpsk:

Epmu Sn!

J lnacjrw njbruh-carppnanm vxm rb mnuncrwp vh yxbcb!

Heater
Posts: 16869
Joined: Tue Jul 17, 2012 3:02 pm

Re: Corrupt SD card -- guide

Tue Jul 24, 2018 10:58 am

I always advise against using NOOBS. Judging by the questions that pop up here all the time NOOBS seems to cause more confusion and problems than it fixes.

There is nothing "advanced" about copying an image to SD card. Especially now that we have Etcher.
Memory in C++ is a leaky abstraction .

NDemberger
Posts: 3
Joined: Thu Aug 09, 2018 4:57 pm

Re: Corrupt SD card -- guide

Sun Mar 31, 2019 6:44 pm

Hello Everyone,

My 2 Cents

I was just running into this issue again. It's happened a few times and at first, I didn't realize what caused the issue. But I eventually realized when my Pi's lost power unexpectedly they may not boot back up. I just did this on purpose for testing and found this to be the cause of my Pi's shitting the bed. To correct this I just booted my PC with the latest Ubuntu Desktop(Ubuntu 18.04.2 LTS) install disk, then inserted the "bad/corrupt" Pi's SD card, and browsed around the file structure, and then Ejected it. The SD card was then plugged back into the Pi, and now it works again.

This was tested with two Pi's running OctoPrint, tied to 3D printers.

FIX QUICK STEPS
1. Boot another PC with the Ubuntu 18.04.2 LTS install media (Install not needed)
2. Plugin the Corrupt/Bad SD card (With Pi install)
3. Browse to the SD card.
4. Eject SD card.
5. Plug into PI.


I hope this helps someone in the future.
-Nick

svavar
Posts: 1
Joined: Sun Dec 08, 2019 6:29 pm

Re: Corrupt SD card -- guide

Sun Dec 08, 2019 6:37 pm

I was able to do this on a Windows laptop using VirtualBox and a Ubuntu VM. You can create a VMDK file to mount your SD card inside a VM.

I opened a command line as administrator and ran the following command:

Code: Select all

cd \Program Files\Oracle\VirtualBox\
VBoxManage internalcommands createrawvmdk -filename "C:\Users\<username>\VirtualBox VMs\<Machine Folder>\raspi1.vmdk" -rawdisk \\.\PhysicalDrive1
You many need to change the filename to suit your VM location. If you have more than one hard drive, use the disk manager to get the drive number for the PhysicalDrive parameter.

Once you have the VMDK file. Go into the settings for your VM in VirtualBox and add it as a hard drive before starting the VM. You may need to run VirtualBox as administrator to get this to work.

User avatar
LDighera
Posts: 46
Joined: Wed Aug 29, 2012 1:04 am
Location: Santa Barbara, California, USA

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 1:07 am

While updating/upgrading Raspberian, power was interrupted resulting in the following when attempting to reboot:

Ubuntu mounted the SD card, and I found the root of sda1 with lots of kernel images, etc. The boot directory was empty.

I ran fack on the SD card partitions. sda1 fscked without issue. sda2 indicated a good deal of corruption in the rfs.

I presume copying the contents of the boot directory from the raw image may be a logical first step?

Any clues on how to proceed in restoring the card is appreciated.

Larry
Attachments
RaspberryPiSD_CardCorrupt33-66_20201030_151029.jpg
RaspberryPiSD_CardCorrupt33-66_20201030_151029.jpg (140.27 KiB) Viewed 460 times
There is no expedient to which a man will not resort
to avoid the real labor of thinking.
-- Sir Joshua Reynolds

ejolson
Posts: 6039
Joined: Tue Mar 18, 2014 11:47 am

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 2:37 am

LDighera wrote:
Sat Oct 31, 2020 1:07 am
While updating/upgrading Raspberian, power was interrupted resulting in the following when attempting to reboot:

Ubuntu mounted the SD card, and I found the root of sda1 with lots of kernel images, etc. The boot directory was empty.

I ran fack on the SD card partitions. sda1 fscked without issue. sda2 indicated a good deal of corruption in the rfs.

I presume copying the contents of the boot directory from the raw image may be a logical first step?

Any clues on how to proceed in restoring the card is appreciated.

Larry
You've discovered while upgrading a kernel that Raspberry Pi OS moves everything out of boot and copies what it needs back at the end. This is an obviously fragile technique that results in a non-bootable system if power is interrupted during the upgrade. I think you should be able to copy the old boot directory back in place and then proceeded with the upgrade. I would back up any important data on the SD card at this time if you haven't already done so.

User avatar
Greg Erskine
Posts: 162
Joined: Sat Sep 15, 2012 4:20 am

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 8:30 am

Heater wrote:
Tue Dec 01, 2015 5:39 am
....
If you have 4 months of coding in any language you might like to think about keeping it in a version control system like github or bitbucket or even just use git on your local machines.

git is not a backup solution by itself but it makes it dead easy to "clone" your code to multiple machines and keep track of all your revisions. It gives me great peace of mind know that it''s almost impossible for me to lose even one line of code even if I delete it from file myself!

Having all my revisions neatly numbered and commented means I can easily go back and see what I was doing and where I might have broken a feature.
Great advise Heater. Git is a must for programmers.

What do you mean by this "git is not a backup solution by itself "?

At least at the project level, don't you think git meets the requirements of a backup system?

I don't backup anything that I stored in git on a remote system. Am I missing something?
* Raspberry Pi is a trademark of the Raspberry Pi Foundation

User avatar
rpdom
Posts: 17736
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 8:54 am

Greg Erskine wrote:
Sat Oct 31, 2020 8:30 am
I don't backup anything that I stored in git on a remote system. Am I missing something?
What happens when the git system (however unlikely) suffers a major problem (like gitlab did)?
Unreadable squiggle

Heater
Posts: 16869
Joined: Tue Jul 17, 2012 3:02 pm

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 10:10 am

Greg Erskine wrote:
Sat Oct 31, 2020 8:30 am
What do you mean by this "git is not a backup solution by itself "?
Many things I guess.

1) A typical scenario is to turn whatever directory one is developing code in into a git repo, "git init", so as to keep a change history, be able to revert to old versions, maintain branches of different bits of development etc. Usual source code management stuff. That mb itself is no backup solution. You only have one copy of your code there.

2) Things get better if we push our code to some other git repository. Preferably at at a different location. Now we have two copies. We can start to think of that as a backup. But note, that is not particularly git that is managing your back up solution, it is you and your procedures.

3) OK. Now we are happy, we have a backup solution that happens to use git. Say our "backup" is on github. OOPs my PC has just burned down and I find GitHub has deleted my repository. Does happen: https://arstechnica.com/tech-policy/202 ... iaa-claim/ We had better do better.

As it happens I do use git as a back up solution, or at least part of the solution. I have my projects in git repos all around the place. One my dev PC, on PC's in the office, on our cloud server instances, in github and bitbucket. It would take quite a world wide catastrophe to loose stuff.

I prefer to not think of this as "backup". Backup are static, dead, stale, old, copies stashed away in vaults. Only to be dug out in emergencies. Quite often people are not even sure if their backups will even be readable when the crunch comes or exactly what is in there (when their are many versions of files scattered all over the place)

I prefer to think of all my git repos as being "live" versions of the project scattered around. Which indeed they are, they get used to build code on different architectures and operating system wherever they are. I, or others, can hack on and update them from anywhere. They all get sync'ed up to the latest versions as and when required. They all contain a complete history of what has happened.

"backups" are such an old fashioned idea. From the time we only had one computer, a mainframe say, and needed to save whatever it had to tapes and such incase of failure. The world is not like that anymore. Our actual live, in progress, data can be distributed all around, if any instance of it gets destroyed it is not a problem. No old fashioned backups required.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 6039
Joined: Tue Mar 18, 2014 11:47 am

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 2:02 pm

Heater wrote:
Sat Oct 31, 2020 10:10 am
Greg Erskine wrote:
Sat Oct 31, 2020 8:30 am
What do you mean by this "git is not a backup solution by itself "?
They all get sync'ed up to the latest versions as and when required. They all contain a complete history of what has happened.

"backups" are such an old fashioned idea. From the time we only had one computer, a mainframe say, and needed to save whatever it had to tapes and such incase of failure. The world is not like that anymore. Our actual live, in progress, data can be distributed all around, if any instance of it gets destroyed it is not a problem. No old fashioned backups required.
The reason backups are offline is to avoid problems with user error and malware that might delete everything that is online. While it's possible that GitHub may delete your code from their end to satisfy a third party beyond your control and it's also possible to delete things oneself out of confusion, another danger is some malware gets your git keys and deletes the entire repository using your own access before you notice what's going on.

Third factor authentication can help, but I think it is it is also possible to set up GitHub so the credentials you use to update your own repository don't have the authority to delete it. Did you do that? Then only the credentials with delete authority need to be offline.

User avatar
Greg Erskine
Posts: 162
Joined: Sat Sep 15, 2012 4:20 am

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 8:05 pm

Thanks guys,

I agree, git is not a backup solution but can be used as a integral component of your backup solution.

I can now see the remote repository could be an issue, but in most circumstances the remote repository can be rebuilt from a local repository to a different remote git server.

So for an active project using git with multiple coders, it is almost impossible to lose a significant amount of work, probably comparable to a normal backup solution.

But it is not a good idea to rely on this system for long term historical backups of non-active projects as something may happen to the remote repository that may not be noticed before all the local repositories have disappeared (due to computer upgrades).

regards
Greg
* Raspberry Pi is a trademark of the Raspberry Pi Foundation

Heater
Posts: 16869
Joined: Tue Jul 17, 2012 3:02 pm

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 8:49 pm

ejolson wrote:
Sat Oct 31, 2020 2:02 pm
The reason backups are offline is to avoid problems with ... another danger is some malware gets your git keys and deletes the entire repository using your own access before you notice what's going on.
That is a very good point.

Luckily I don't use git keys. If anyone gets into a server or dev machine, I have much bigger problems than they delete the git repos on there. Those repos exist elsewhere, many elsewheres, as well.
ejolson wrote:
Sat Oct 31, 2020 2:02 pm
Third factor authentication can help, but I think it is it is also possible to set up GitHub so the credentials you use to update your own repository don't have the authority to delete it. Did you do that? Then only the credentials with delete authority need to be offline.
No I did not. Sounds like a very good idea. Thanks for the heads up on that.

Sounds like time to review security and procedures again...
Memory in C++ is a leaky abstraction .

HvdW
Posts: 163
Joined: Tue Jun 17, 2014 12:41 pm

Re: Corrupt SD card -- guide

Sat Oct 31, 2020 9:39 pm

I've been reading this thread with pleasure.
Some good tips.

Just want to say it all starts here.
Who knows knows
Who doesn't doesn't

User avatar
LDighera
Posts: 46
Joined: Wed Aug 29, 2012 1:04 am
Location: Santa Barbara, California, USA

Re: Corrupt SD card -- guide

Sun Nov 01, 2020 11:29 pm

ejolson wrote:
Sat Oct 31, 2020 2:37 am
You've discovered while upgrading a kernel that Raspberry Pi OS moves everything out of boot and copies what it needs back at the end. This is an obviously fragile technique that results in a non-bootable system if power is interrupted during the upgrade. I think you should be able to copy the old boot directory back in place and then proceeded with the upgrade. I would back up any important data on the SD card at this time if you haven't already done so.
Thank you very much for the clue.

After fscking the card, I found the missing /boot files located in /usr/share/kernelhack, and rsync -avh ed them to /boot for a successful boot from my old SD card. You're a life saver. Very much appreciated.

I have that system configured just the way I want it, and would have had to spend many hours to configure a new image to match it.

Here's perspicacious information about making backups: https://www.jetsonhacks.com/2020/07/21/lets-backup/ for we who need it. :oops: Jim is a remarkable intellect...

Best regards,
Larry
There is no expedient to which a man will not resort
to avoid the real labor of thinking.
-- Sir Joshua Reynolds

Return to “General discussion”