Warning re. using the SD card copier

[gtho]

I am not 100% sure, but I think I just erased 2 tera bytes from a hard drive using the "SD card copier" in the menu (introduced a couple of months ago). I thought I had carefully selected an SD card from the dropdown as the destination, but unfortunately not. It must have been the hard drive.

Someone should look into the possibility of having a warning or confirmation function in the SD card copier. I dare not use it again without detaching my hard drives, and that is difficult when you operate remotely. This is a dangerous tool.

[HawaiianPi]

I know you are probably just upset and venting, but seriously? You pick the wrong destination and somehow this is the software's fault?

It sucks that you lost a bunch of data, but you have no one but yourself to blame. You picked the wrong drive, and you didn't have a backup of your data. Learn from your mistakes and move on.

[liudr]

You pick the wrong destination and somehow this is the software's fault?

Well, it is supposed to be made for beginners to learn computers with. At the very least the program should warn the user that the target is a hard drive or has a very very large volume size for an SD card. It's the programmer's fault to think that everyone thinks like him and guess what, those kids using this program will think that it is the norm that you should feel stupid for not already knowing everything you're supposed to learn. I felt sorry for the OP. I feel the same frustration you have sometimes.

[mattmiller]

We've all done something like that at one stage (or come very very close!)

Your right in that it would be very nice of the software to do a size check and issue a warning.

Contact Simon Long at the Foundation as he is the author of the program and make a suggestion to him
(Word it as a suggestion and not a complaint)

Matthew

[gregeric]

Also, the source code is here https://github.com/raspberrypi/piclone for those wanting to contribute enhancements.

[RaTTuS]

see also testdisk ,
you may be able to recover stuff
there will always be edge cases where things are not covered

[mattmiller]

Also, the source code is here

IIRC that is not the current source code - it is the code Simon forked to produce his version

His version is not openly available

[SonOfAMotherlessGoat]

It would take quite a while to actually delete the full 2TB data. You may have a long road ahead, but tools like Recuva are helpful, as typically it's the File Allocation Table (or it's equivalent) that is erased. If you think of the Hard Drive as a book, you haven't erased all the pages, you've just erased the Index in the back. Some file structures also keep a duplicate of the FAT (or equivalent) that can help with recovery.

It's a very hard lesson to learn, and I've lost data myself, but it's a good time to look at your backup approach as well. If you have data that can be easily replaced, then it really makes sense to backup as much as you can. Things are going to go wrong at times, and it's easy to get lulled into the fact that nothing bad has happened prior so why bother with the expense in time and money for backups. Now you see why.

My best to you in recovering as much of your lost data as possible.

[bensimmo]

Agreed it would be good, I wouldn't want to wipe out a lot of data if other drives are connected.
Always good to bring these user problems up.

Hopefully someone who can alter the source or knows the one used in Raspian can give it a nudge.
It is software in the main distibutions menu after all, not a hidden 'power tool'

[swampdog]

I haven't used the "SD card copier" but I would imagine it has only scribbled over the first few Gb of your disk. If your big disk was formatted with a unix filesystem then there will have been copies of the superblock scattered all over it as part of the formatting process.

If you are serious in attempting to recover data then you need to adopt a forensic approach. Pull the big disk and make it "readonly" in another device then 'dd' a copy to another device. You can then experiment on the copy to see how much can be recovered.

[liudr]

Agreed it would be good, I wouldn't want to wipe out a lot of data if other drives are connected.
Always good to bring these user problems up.

Hopefully someone who can alter the source or knows the one used in Raspian can give it a nudge.
It is software in the main distibutions menu after all, not a hidden 'power tool'

Simon Long talked a lot about this little tool since we've been waiting for 4 years for a beginner solution of backup and restore. Does he hang out here by any chance?

[mattmiller]

Very rarely - I suggest contact him direct at Foundation

[HawaiianPi]

Well, it is supposed to be made for beginners to learn computers with. At the very least the program should warn the user that the target is a hard drive or has a very very large volume size for an SD card. It's the programmer's fault to think that everyone thinks like him and guess what, those kids using this program will think that it is the norm that you should feel stupid for not already knowing everything you're supposed to learn. I felt sorry for the OP. I feel the same frustration you have sometimes.

This is utter nonsense. What if you have a large SD card and a similar size HDD?

The program DOES give you a warning that it will erase all data on the destination drive, and asks if you are sure. If you answer yes to that warning then it is YOUR fault, period.

A warning that says you are about to erase all data on something should be ominous enough to get you to double-check your selection, and if you still end up erasing your hard drive, then you should "feel stupid" as you said, because that's a boneheaded mistake. And I know that from experience, I've made a few of them myself, but at least I accept responsibility for my mistakes.

It's a very hard lesson to learn, and I've lost data myself, but it's a good time to look at your backup approach as well. If you have data that can (not) be easily replaced, then it really makes sense to backup as much as you can. Things are going to go wrong at times, and it's easy to get lulled into the fact that nothing bad has happened prior so why bother with the expense in time and money for backups. Now you see why.

My best to you in recovering as much of your lost data as possible.

Couldn't have said it better myself. Thanks for that.

[liudr]

Well, it is supposed to be made for beginners to learn computers with. At the very least the program should warn the user that the target is a hard drive or has a very very large volume size for an SD card. It's the programmer's fault to think that everyone thinks like him and guess what, those kids using this program will think that it is the norm that you should feel stupid for not already knowing everything you're supposed to learn. I felt sorry for the OP. I feel the same frustration you have sometimes.

This is utter nonsense. What if you have a large SD card and a similar size HDD?

The program DOES give you a warning that it will erase all data on the destination drive, and asks if you are sure. If you answer yes to that warning then it is YOUR fault, period.

A warning that says you are about to erase all data on something should be ominous enough to get you to double-check your selection, and if you still end up erasing your hard drive, then you should "feel stupid" as you said, because that's a boneheaded mistake. And I know that from experience, I've made a few of them myself, but at least I accept responsibility for my mistakes.

It's a very hard lesson to learn, and I've lost data myself, but it's a good time to look at your backup approach as well. If you have data that can (not) be easily replaced, then it really makes sense to backup as much as you can. Things are going to go wrong at times, and it's easy to get lulled into the fact that nothing bad has happened prior so why bother with the expense in time and money for backups. Now you see why.

My best to you in recovering as much of your lost data as possible.

Couldn't have said it better myself. Thanks for that.

You want to fan a flame, you go ahead. I'm not with you on this issue and never will. Thanks for quoting me and calling me names.

[bensimmo]

I can't see why the hostility to the Original Poster?

Poster points out something that can happen.
At no point was said it wasn't their fault.
It was just a suggestion, but if the software can be modified as suggested, then it improves the software (windows can tell if it's SD or something else, I assume a debian based system should be able too?)

It was just a suggestion from the op.

Hopefully the new poster doesn't think, sod that it's not friendly in here...

[DougieLawson]

The trouble with asking any number of "Are you sure?" or "Are you a complete idiot?" or "How sure are you that you're a complete idiot?" questions is that it doesn't achieve any more protection than not bothering to ask at all. Everyone has been pre-programmed by those pointless End-User Licence Agreements (EULAs) to always click through any form of "How much would you like to destroy your system?" question. How many questions do you ask before you carry out the risky operation.

We've all done it, we've all had the "Oh no second" (that's the measure of time between pressing enter and realising that wasn't really what you wanted the system to do). We've all crashed things, damaged things, fouled things up with unintended consequences. The best solution is the "four eyes" principle, you have a colleague watching. That's not practical for the home hobbyist with his small RPi system. So for the home user, take a backup, when that backup is complete test it. Read the words on the screen, go and fetch a cup of tea, read the words on the screen again and only then press enter. Understand the risk of everything you do and take an extra backup if needed.

It gets interesting when you're playing with large commercial mainframe installations the ones that get a news item on http://news.bbc.co.uk when the customer facing systems go west. Foul up a clearing bank and you'll get a phone call from the Bank of England before the day is done. It's one of the reasons I carry professional indemnity insurance (which costs my limited company nearly £400 a year).

[CarlRJ]

I've used Unix and Unix-like systems for a very long time, and I'm quite familiar with the Unix approach to things, and I still think it'd be quite reasonable to have a dialog that pops up if the destination is over some size that makes sense for SD cards (say, 64GB or 128GB), saying, "This utility is intended for writing to SD cards and will erase the current contents of the destination, and your selected destination looks awfully large for an SD card - are you _sure_ you want to overwrite your selected destination, erasing the current contents?"

It'd only trigger for a few SD cards (sure, you could use one that large, but I'd wager the vast majority of Pi's are running on cards 32GB and under), and it would warn for pretty much every modern hard drive (attempting to talk the user out of messing up their drive - especially if the default answer is set to no/cancel). Seems like a good idea.

[stderr]

I am not 100% sure, but I think I just erased 2 tera bytes from a hard drive using the "SD card copier"I dare not use it again without detaching my hard drives, and that is difficult when you operate remotely. This is a dangerous tool.

When I use dd, I use it almost exclusively on a dedicated or at least semi-dedicated machine, one that doesn't have server drives attached to it. Also, I note that the sdcard /dev name isn't the same format as regular hard drives. This certainly helps. When using gparted, I also always, I make it a rule, check what drives are visible and which one I'm operating on, does the choice make sense in the complete context of what I see and what I should be able to see? Of course anyone can make a mistake which is why I like the dedicated machine approach for dd at least.

I haven't got any traction with this idea, but I think that a version of R. jessie should be made available that has the choice of booting into jessie or into a fairly well fleshed out ram only linux such as tiny core. What this would mean is that anyone wishing to use dd in a safe manner, ok a safer manner, could simply boot to tiny core and then remove the sdcard leaving the slot open for whatever they wish to copy to. Of course some means of getting the image to the system would be needed, via network or more likely flash drive. Yes, this all could be easily achieved with a second sdcard for tiny core, the problem is that people just don't do that yet they have need for mounting cards and copying them that are currently being met often in a disappointing manner.

[stderr]

I still think it'd be quite reasonable to have a dialog that pops up if the destination is over some size that makes sense for SD cards (say, 64GB or 128GB), saying, "This utility is intended for writing to SD cards and will erase the current contents of the destination, and your selected destination looks awfully large for an SD card - are you _sure_ you want to overwrite your selected destination, erasing the current contents?"

It could check to see if the drive is "removable". Isn't that how Microsoft decides whether or not to prevent you from seeing more than the initial partition on sdcards and usb flash drives? The claim that people just click past stuff they see each time is exactly true, so by *not* complaining unless there's a clear need to complain, one might get a sane response from the user.

[CarlRJ]

The trouble with asking any number of "Are you sure?" or "Are you a complete idiot?" or "How sure are you that you're a complete idiot?" questions is that it doesn't achieve any more protection than not bothering to ask at all. Everyone has been pre-programmed by those pointless End-User Licence Agreements (EULAs) to always click through any form of "How much would you like to destroy your system?" question. How many questions do you ask before you carry out the risky operation.

Don't ask "Are you sure you want to destroy your system? >>YES<</no" Experienced users will hit Enter out of habit. Inexperienced users will cheerfully choose the default assuming it to be "right". Make the answer default to NOT destroying things. It's not a panacea, but it may save some folks. And if you only ask when it's likely to be a concern (disk is very large), it doesn't present any extra burden in the vast majority of cases. If you save even, say, one in five of the folks unintentionally trying to destroy their systems, without inconveniencing the 98% of users writing to small SD cards, that's a win. For software likely to be used by inexperienced users, I think this is reasonable.

We once halfway seriously considered asking "Are you sure?" "Are you really sure?" "On a scale of 4 to 37 how sure are you?" "Okay, enter the entire Gettysburg Address to continue."

[DougieLawson]

The trouble is you can't tell that a removable /dev/sdx device isn't the legitimate target.

Code: Select all

dougie@apollo:~$ sudo fdisk -l /dev/sda
[sudo] password for dougie:

Disk /dev/sda: 149.1 GiB, 160041885696 bytes, 312581808 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x2f3846c8

Device     Boot Start       End   Sectors   Size Id Type
/dev/sda1        2048 312581807 312579760 149.1G 83 Linux

dougie@apollo:~$

That's a 150GB USB hard drive, you can ask me a hundred times "Do you want to destroy the ext4 file system on device /dev/sda with 150GB of space?" and because I'm a complete idiot, programmed to always click "YES" that's what will happen.

You can do the maddening thing of "You answered too soon, you can't have read and digested that instruction" if the click comes in 1 or 2 ms after the question is presented.

I quite like the way z/OS does it when you try to vary a system offline.

Code: Select all

VARY XCF,FRED,OFFLINE
57 IXC371D CONFIRM REQUEST TO VARY SYSTEM FRED OFFLINE, REPLY SYSNAME=FRED TO REMOVE FRED OR C TO CANCEL
R 57,SYSNAME=FRED

that makes you confirm you've got the right system and might prevent finger trouble.

[B.Goode]

It is possible to do the same sort of irreversible damage to a hard drive using Win32DiskImager, which has been the utility of choice for copying SD card images since the RPi was announced over 4 years ago.

Ask me how I know...

(I'm still convinced that I used the drive letter for the removable sd card, not the c: drive containing the system, but there we are - I must have got it wrong.)

[SonOfAMotherlessGoat]

It is possible to do the same sort of irreversible damage to a hard drive using Win32DiskImager, which has been the utility of choice for copying SD card images since the RPi was announced over 4 years ago.

Ask me how I know...

(I'm still convinced that I used the drive letter for the removable sd card, not the c: drive containing the system, but there we are - I must have got it wrong.)

You too eh? 500GB Removable WD Passport. "Ohhhhhhhh Nooooooooo!"

[ejolson]

Well, it is supposed to be made for beginners to learn computers with. At the very least the program should warn the user that the target is a hard drive or has a very very large volume size for an SD card.

Reporting the "Model Number" and "Serial Number" strings as given by hdparm -I before copying would help cut down on errors.

[DougieLawson]

Reporting the "Model Number" and "Serial Number" strings as given by hdparm -I before copying would help cut down on errors.

hdparm -I doesn't work at all well for an SDCard in my USB reader.

Code: Select all

root@pi-server:~ # hdparm -I /dev/sda

/dev/sda:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Unknown device type:
        bits 15&14 of general configuration word 0 both set to 1.