Can not restore backup.img

[vguerrao]

Please help.

When restoring from a backup.img generated from an 32 Gb SD to a new one of the same size, imagemaker for windows won´t go because there are missing few sectors.

This is the message i get.

>>>>>> Not enough space on disk. Size 615....... sectors. Available 614..... <<<<<

[Davespice]

Hi there!
Imagemaker for windows? Do you mean Win32DiskImager?

[vguerrao]

Yes i use the latest version of Win32DiskImager

Thanks]

[rpdom]

This is a common problem. Not all SD cards are the same size. The size on the label is only approximate and various things can affect the actual usable size of the card, like manufacturing defects - a number of bad blocks on the card are acceptable and won't be used. How many depends on the individual card. There is no way of telling what this number will be when you buy the card.

Some old versions of diskimager will still write the image to the card, but fail when they run out of space. This will be OK if you haven't expanded the filesystem to fill the card, but may cause issues if you have. Firstly, any data in the last few blocks of the drive will be lost. Secondly, Linux will still assume that space is available for use as that will be the filesystem size.

It's a real problem, which is why I don't like image backups unless they are going back on the same card.

[SirLagz]

This is a common problem. Not all SD cards are the same size. The size on the label is only approximate and various things can affect the actual usable size of the card, like manufacturing defects - a number of bad blocks on the card are acceptable and won't be used. How many depends on the individual card. There is no way of telling what this number will be when you buy the card.

Some old versions of diskimager will still write the image to the card, but fail when they run out of space. This will be OK if you haven't expanded the filesystem to fill the card, but may cause issues if you have. Firstly, any data in the last few blocks of the drive will be lost. Secondly, Linux will still assume that space is available for use as that will be the filesystem size.

It's a real problem, which is why I don't like image backups unless they are going back on the same card.

Really this issue only exists for Windows users.
If someone is using Linux and is aware of this issue, then they could simply not expand the image to the full size of the card, and when they are backing it up, only backup the amount of the card that is used.

[rpdom]

It's a real problem, which is why I don't like image backups unless they are going back on the same card.

Really this issue only exists for Windows users.
If someone is using Linux and is aware of this issue, then they could simply not expand the image to the full size of the card, and when they are backing it up, only backup the amount of the card that is used.

If using Linux, why bother with an image backup, when a tar or rsync backup would take less space and be faster? I'd just do a image dump of the vfat partition (due to it being so small), then tar any ext[234] partitions to compressed archives.

I've also been experimenting with various other arcane backup methods, but I'll not go into that here

[SirLagz]

It's a real problem, which is why I don't like image backups unless they are going back on the same card.

Really this issue only exists for Windows users.
If someone is using Linux and is aware of this issue, then they could simply not expand the image to the full size of the card, and when they are backing it up, only backup the amount of the card that is used.

If using Linux, why bother with an image backup, when a tar or rsync backup would take less space and be faster? I'd just do a image dump of the vfat partition (due to it being so small), then tar any ext[234] partitions to compressed archives.

I've also been experimenting with various other arcane backup methods, but I'll not go into that here

Got a point there...but I still prefer doing image backups.
Then I can mount the image, check things out, change things, etc without having to untar/retar.

[vguerrao]

Thanks to everybody. It´s now clear that if other physical storage devices have bad sectors removed and blocked, this should also happen in SD cards. Really thanks for pointing that out.

I´m trying to backup some other way, using the same Raspberry, but i´m not proficient enough and the Internet instructions are to complicated and leave some issues to my technical capacities.

For example: everybody assumes that i can find the name of a newly mounted card in /dev, ussualy is called mmcblk0, but whether the card is mounted or not the same mmcblkxxxx remains in the /dev directory. Of course the in the /media directory it appears, for example, as KINGSTONxx, but how to relate this with mmcblk0 is magic.

Please refer me to places where i can learn these things, and thanks again.

[pirandr]

For example: everybody assumes that i can find the name of a newly mounted card in /dev

Actually, plugged in and mounted don't mean the same thing. Usually when I plug in a (usb) device or SD card, I can see it in /dev/ but before it is mounted, I can't see it in /media. You might have some kind of automount on (so it tries to mount new things plugged in).

But then even if you unmount it manually, it will still be in /dev/ (until its unplugged and this is detected). To see your computer's reaction to new things plugged in, you can type

Code: Select all

dmesg

and look at the last few lines for things that look relevant to your device (knowing what's relevant might be hard at the start but you'll start learning some keywords soon enough). To know which device is mounted where, you can type

Code: Select all

mount

and look for the line containing /dev/mmblk0 (or whatever device you are looking for) and a bit more to the right, you'll see something like /media/KINGSTON telling you where the device is mounted.

You could also type something like

Code: Select all

mount | grep /dev/mmblk0

to have your computer help you single out the line you are looking for.

[rpdom]

The card in the built-in slot in the Pi will show up as /dev/mmcblk0 and the partitions as /dev/mmcblk0p1,2,3...
Cards in USB adaptors in the Pi will show as /dev/sda, /dev/sdb etc and the partitions as a 1, 2, 3 etc on the end.

If you run the command "dmesg | tail -20l" a few seconds after plugging in a card (on any Linux system), you should see a message about a new storage device and it will list the name of the device under /dev for the whole device and all the partitions.

Example on my Linux laptop with a Raspbian card in a USB adaptor:

Code: Select all

rpdom@oz:~$ dmesg | tail -20l
[43677.899116] usb 4-1: not running at top speed; connect to a high speed hub
[43677.922109] usb 4-1: New USB device found, idVendor=05e3, idProduct=0736
[43677.922116] usb 4-1: New USB device strings: Mfr=3, Product=4, SerialNumber=2
[43677.922137] usb 4-1: Product: USB Storage
[43677.922140] usb 4-1: Manufacturer: Generic
[43677.922144] usb 4-1: SerialNumber: 000000000272
[43677.932232] scsi3 : usb-storage 4-1:1.0
[43678.943044] scsi 3:0:0:0: Direct-Access     Generic  STORAGE DEVICE   0272 PQ: 0 ANSI: 0
[43678.948598] sd 3:0:0:0: Attached scsi generic sg1 type 0
[43679.055986] sd 3:0:0:0: [sdb] 3970048 512-byte logical blocks: (2.03 GB/1.89 GiB)
[43679.061960] sd 3:0:0:0: [sdb] Write Protect is off
[43679.061985] sd 3:0:0:0: [sdb] Mode Sense: 0b 00 00 08
[43679.066951] sd 3:0:0:0: [sdb] No Caching mode page present
[43679.066975] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[43679.090936] sd 3:0:0:0: [sdb] No Caching mode page present
[43679.090944] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[43679.097130]  sdb: sdb1 sdb2
[43679.118942] sd 3:0:0:0: [sdb] No Caching mode page present
[43679.118966] sd 3:0:0:0: [sdb] Assuming drive cache: write through
[43679.118972] sd 3:0:0:0: [sdb] Attached SCSI removable disk
rpdom@oz:~$ 

Here you can see that the whole card is /dev/sdb and the partitions are /dev/sdb1 and /dev/sdb2

[ramstrong]

I have that trouble, too, copying 16 GB to 16 GB results in "not enough space". I tried to repartition the SD card to smaller size, but for some reason, even though I can see 4 partition in /dev, I only see one big unallocated chunk in gparted. What gives?

How do I resize the partition so it'll be slightly smaller as to fit the 16 GB to 16 GB dump? (1)

Once I have two cards mirrored, then I can use rsync for back up. But I need to have that image transfer done right,first.

Also, I have a 64 GB flashcard I can use for tarball. But it stopped writing after 4 GB. Am I correct in assuming that using tape size --tape-length 500000 will cause it to save in multiple files of 500M ? And can you give me example on how to use the commands for both archive and unarchive a partition?

TIA
(1) Probably a good idea to change NOOBS behavior so it backs off a few megabytes off the limits so this doesn't happen too often.

[rpdom]

Once I have two cards mirrored, then I can use rsync for back up. But I need to have that image transfer done right,first.

You don't need to mirror the cards first, just partition the new one so that each partition is big enough for the contents of each partition on the old card, format the filesystems and then use rsync to copy the contents. With any vfat partitions it can be easier just to make them exactly the same size and dd the whole partition across, as they aren't usually huge and it'll keep the format exactly the same.

I haven't tried this on NOOBS yet, I've run out of spare cards.

Also, I have a 64 GB flashcard I can use for tarball. But it stopped writing after 4 GB

I'm guessing that your card is formatted as FAT32 (vfat), which has a file size limit of 4GB. Unless you plan to use the card with a non-linux computer, you are better off reformatting it as ext4 which doesn't have that restriction.

[ramstrong]

I have a 32 GB flashdrive I can reformat to ext4 for tarball, no problem. I tried rsync, and it works for /home directory, but for / directory, lots of problems. In the end, I rsync these directories
/boot /home /usr /bin /etc /sbin /opt /lib
The system hangs when trying to rsync /sys. So, that's not everything.

Can you tell me what format the four partitions are so I can manually format them using gparted? Partition size, file format, and whatever else I need. Then I'll try to tarball into them, and see what happens.
Thanks

[rpdom]

I have a 32 GB flashdrive I can reformat to ext4 for tarball, no problem. I tried rsync, and it works for /home directory, but for / directory, lots of problems. In the end, I rsync these directories
/boot /home /usr /bin /etc /sbin /opt /lib
The system hangs when trying to rsync /sys. So, that's not everything.

You should only need to rsync / and /boot, making sure you use the "-x" option to prevent it trying to copy things from other partitions or filesystems.

You don't need to backup /sys at all. It doesn't exist on the card, it is just a "virtual" filesystem that exists inside the Linux kernel. It gets created from scratch every time you reboot. The same goes for /proc and a few others like /dev and /var/run. Look at the output of the "mount" command and you can ignore anything that shows as "tmpfs".

Can you tell me what format the four partitions are so I can manually format them using gparted? Partition size, file format, and whatever else I need. Then I'll try to tarball into them, and see what happens.
Thanks

I can't, because I don't have them.

The "mount" command will show the filesystem format of all mounted partitions, so you can work that out easily enough.
"sudo fdisk -l" will list the start and end sector of each partition on the card, and the sizes in KB.

[ramstrong]

Thanks for the help. Once more into the breach, I guess.

I'll do an rsync -axv / /media/flashcardwhatever, then write it back out to a another SD card.

Will try to do that partition some other time. I think all this trouble comes from the fact that gparted does not read the existing partition correctly. I thought, all I have to do is to resize mmcblk0p6 slightly less, and be done with it. Well, can't do it on the current card due to unallocated status. It's a good thing I have lots of cards and flashdrives lying around.

EDIT:
I don't think the rsync method will work since the ownership of files goes to "pi" despite my using "root" account. THere's an error changing ownership on the flashdrive. In the end, I gave up.

I reformatted my flashdrive to ext4, and do tarballs on it for boot and root.
tar --one-file-system -psP -cvf /media/Lexar/boot.tar /boot

I checked it using
tar -tvf /media/Lexar/boot.tar
File ownership looks intact. I just didn't think it would take hours to copy. I ended up sleeping as it backed it up overnight.

Now, all I have to do is do a restore to the SD card, except, I don't have the partition set up correctly. It really is annoying to not be able to just resize the last partition slightly to fit my other SD card. So, I will create a new SD noobs card, and restore the files to there.

I hope this will work, because if not, then I'm out of option.

EDIT
This seems to work. Unfortunately, the file systems points to boot partition. So, in order to restore the files, I have to boot using the backup SD card. Backing up to tarball? Easy. Checking it? Turn off Pi, replace card and reboot. Highly inconvenient.

Does anybody have a more convenient way to check it?