Backup SD


63 posts   Page 3 of 3   1, 2, 3
by Noel Hibbard » Thu Jun 18, 2015 2:30 pm
In the past I would image my 16gb card using the method detailed in this thread. But now I am using dd for windows which I sort of like better because I'm not backing up a filesystem that is active.

dd for windows does stdin/stdout too so I can pipe the input/output to and from 7z. So my 1.6gb image comes out to 500mb.

You can download dd for windows here:
http://www.chrysocome.net/dd

Here is a sample backup command line:
dd if=\\?\Device\Harddisk2\Partition0 bs=512 count=3236351 --progress | "C:\Program Files\7-Zip\7z.exe" a -siBackup.img Backup.7z

To find the disk number simply run diskpart in windows and then type "list disk".

Here is an example for restoring:
"C:\Program Files\7-Zip\7z.exe" x Backup.7z -so | dd if=- of=\\?\Device\Harddisk2\Partition0 --progress

Note that before you can restore you have to clean the disk in diskpart. To do this, run diskpart then type "list disk" to find the correct disk. Then type "sel disk 2" (using the right disk number) and then type "clean". If you don't do this first then dd will give you an access denied message.

"Partition0" isn't really a partition. It actually represents the whole disk.

Hopfully this will be helpful for windows users.
Posts: 4
Joined: Mon Sep 29, 2014 3:29 pm
by gkovacsp » Mon Dec 28, 2015 10:44 am
I've been struggleing with the same problem:
-back up my 32 GB Raspbian SD card
-make sure backup works (don't do it on a running system)
-reduce backup size
I came up with the following solution:
I've installed Raspbian Lite on a 2GB SD card, then put this SD card to an USB SD adapter and plugged it into PI. (This way it became /dev/sda after booting Raspbian).
I've booted from the 32GB Raspbian SD card, modified the /boot/cmdline.txt file to boot from the 2GB SD card in the USB adapter:
Code: Select all
dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/sda2 rootfstype=ext4 elevator=deadline rootwait

Reboot again, now it will boot the 2GB SD with Raspbian Lite.
Some packages must be installed to make sure it works nicely:
-pv
-pigz
-zerofill
Once it is running it is pretty easy to create a fully working backup using DD.
I pipe it into zip compression which is able to reduce the 32GB SD card to about 5GB. I use pigz instead of gzip to use all four cores of the PI.
Code: Select all
dd bs=32M conv=sync,noerror iflag=fullblock | pigz > image.img.gz

I use the pv package to have some visual feedback during the operation
Code: Select all
pv -tpreb /dev/mmcblk0 -s 32026656768 | dd bs=32M conv=sync,noerror iflag=fullblock | pigz > image.img.gz

Use blockdev to determine your SD card's size (needed by the pv package)
Code: Select all
blockdev --getsize64 /dev/mmcblk0
32026656768

The 32 GB SD card backs up in about 80-90 minutes.
In order to improve the compression zerofill can be used on the SD card to fill unused cells with zeros which will make compression a lot more effective. (This will take quite some time...)
Code: Select all
zerofill -v /dev/mmcblk0p2

Once it competes I restore the original /boot/cmdline.txt and reboot back to the main PI OS. The IMG file can be restored a similar way or using win32diskimager from windows.

This process can be automated as well to do it once a week/month, but that is a different topic.
I hope this helps.
Posts: 7
Joined: Fri Jun 08, 2012 6:50 pm
by warwolf7 » Tue Aug 30, 2016 4:51 am
Noel Hibbard wrote:In the past I would image my 16gb card using the method detailed in this thread. But now I am using dd for windows which I sort of like better because I'm not backing up a filesystem that is active.

dd for windows does stdin/stdout too so I can pipe the input/output to and from 7z. So my 1.6gb image comes out to 500mb.

You can download dd for windows here:
http://www.chrysocome.net/dd

Here is a sample backup command line:
dd if=\\?\Device\Harddisk2\Partition0 bs=512 count=3236351 --progress | "C:\Program Files\7-Zip\7z.exe" a -siBackup.img Backup.7z

To find the disk number simply run diskpart in windows and then type "list disk".

Here is an example for restoring:
"C:\Program Files\7-Zip\7z.exe" x Backup.7z -so | dd if=- of=\\?\Device\Harddisk2\Partition0 --progress

Note that before you can restore you have to clean the disk in diskpart. To do this, run diskpart then type "list disk" to find the correct disk. Then type "sel disk 2" (using the right disk number) and then type "clean". If you don't do this first then dd will give you an access denied message.

"Partition0" isn't really a partition. It actually represents the whole disk.

Hopfully this will be helpful for windows users.


Wow! Thank you soo much!! This is wonderful. Really, It's perfect.
I changed the bs to 512k and the backup was a lot faster. But I havent been able to make it work properly (its boots to NOOBS) but doesn't see the raspbian
When i comes to restore, It took forever. I decided to stop it and unzip it and use win32diskimager, that was a lot faster, but it gave me the problem above


I do have a question about the size, you decided to make your image 1.6Gb, is it because that how big it is in your Rpi?
Posts: 6
Joined: Wed Dec 23, 2015 8:41 pm
by nl3prc » Tue Aug 30, 2016 9:14 am
diskpart is not recognize my sd card or any of my hdd's thats connected to the pc with usb adaptor

I have a 64gb image data used on it is about 12 gb howe do i burn the image on a 32gb card
User avatar
Posts: 152
Joined: Sun Jul 24, 2016 12:39 pm
Location: Den Helder the netherlands
by nl3prc » Wed Aug 31, 2016 11:36 am
I just found a script that works perfectly on my raspberrry pi with debian jessie full desktop version

the debian is on a 64 gb sd card and about 12 gb is used i now made a workin copy ofr the sd card on a 32 gb sd card
So as you kan see downsize does work
https://github.com/billw2/rpi-clone
Attachments
rpi-clone-master.zip
(6.65 KiB) Downloaded 70 times
User avatar
Posts: 152
Joined: Sun Jul 24, 2016 12:39 pm
Location: Den Helder the netherlands
by romney_yw » Wed Nov 02, 2016 10:47 pm
While this is quite an old thread, there have been some fairly recent posts, so it's perhaps worth remembering that since May 2016 versions of Raspbian Jessie have included the SD Card Copier. This tool simply and easily copies the image from the card running in the Pi SD slot to another card, typically in an external card reader plugged into a USB socket. (For perhaps obvious reasons, you cannot use the tool to restore a backup by copying from the external card to the Pi SD slot...... :shock: )

The card copier works very well indeed and I use it routinely every month or so (and more frequently as required) to make a backup of my running Pi system. I then use the USB Image Tool (see http://www.alexpage.de/) to save the image from the backup card to a compressed image file (.img.gz) on my PC.

see here https://www.raspberrypi.org/blog/another-update-raspbian/ for more details.
Posts: 12
Joined: Fri Apr 01, 2016 11:31 am
by mattW15 » Mon Mar 20, 2017 1:39 pm
If you are a lazy windows user.... like me :) You can use this tool...... http://www.minitool.com/free-tools/mini ... ecopy.html to copy your "large" card to your "small" card, works well.
Regards


Matt :)
Posts: 2
Joined: Mon Mar 20, 2017 1:29 pm
by Ernst » Mon Mar 20, 2017 3:56 pm
mattW15 wrote:If you are a lazy windows user.... like me :) You can use this tool...... http://www.minitool.com/free-tools/mini ... ecopy.html to copy your "large" card to your "small" card, works well.


Does not look usable, according the the specification it does not support "non-microsoft" file systems.
Code: Select all
Operating system: Windows XP/Vista/7/8/2000 Professional, ect.
File system: FAT12/FAT16/FAT32/VFAT/NTFS/NTFS 5, and so on.
Hard disk: IDE hard disk, SATA hard disk, SCSI hard disk, and other mobile hard disks.


Question: have you done a successful clone of a Pi SD card ?
My first computer was an ICT1500
Posts: 197
Joined: Sat Feb 04, 2017 9:39 am
Location: Germany
by JustDucky » Mon Mar 20, 2017 10:33 pm
I am new to Pi, (Christmas 2016) but, I don't get the copy/backup problem or complex solutions. What am I missing? I have several SD cards for several configurations and for my favorites, I just do SD copy as mentioned by Romney_yw above on 11/2/2016. I copy the entire SD onto one I've stuck in the USB adapter. It doesn't seem to matter that my original is 32G and, since not full, I can copy it onto an 8g SD. I then shutdown and swap SDs. I can boot the entire configuration from the 8G so I shutdown and stash it away. If I need to restore, I dig it up and boot from the 8g, SD copy to the new 16g/32g in a USB adapter, and I don't need to mess with images or PCs or external software. Did I miss the point, or do I just not like CLI as much as most. Speaking of which, Since some configs are on lite, without GUI, is there a command line for the SD copy routine, or can I copy it or apt-get it?
Posts: 2
Joined: Mon Mar 20, 2017 10:12 pm
by KLL » Tue Mar 21, 2017 1:53 am
JustDucky wrote: I copy the entire SD onto one I've stuck in the USB adapter...
I can boot the entire configuration from the 8G so I shutdown and stash it away.
If I need to restore, I dig it up and boot from the 8g, *** SD copy to the new 16g/32g in a USB adapter

*** would be a good idea to do a UPDATE UPGRADE at that point, so the MASTER does not get OLD...
Posts: 583
Joined: Wed Jan 09, 2013 3:05 pm
Location: thailand
by JustDucky » Tue Mar 21, 2017 3:13 am
Good point. Old or new, a backup is a snapshot in time. It will eventually be aged. 42 years in the backup business (early mainframe to modern server) has taught me, anytime you restore, there will probably be newer pieces which are missing. Remember that your backup, whatever method and however current at that time, is probably not perfectly current at time of recovery.
Posts: 2
Joined: Mon Mar 20, 2017 10:12 pm
by KLL » Tue Mar 21, 2017 7:05 am
yes, and if your backup is on a 8'' floppy you can wrap the RPI in it!

but i took your prior post as a good idea about creating a MASTER SD ( custom OS setup for RPI )
to save 1 ..2 hours download and first installation...
using the SD copier tool to make YOUR new 'work' system SD in about 15min.
not so much for backup...
Posts: 583
Joined: Wed Jan 09, 2013 3:05 pm
Location: thailand
by mattW15 » Tue Mar 21, 2017 3:43 pm
Ernst wrote:
mattW15 wrote:If you are a lazy windows user.... like me :) You can use this tool...... http://www.minitool.com/free-tools/mini ... ecopy.html to copy your "large" card to your "small" card, works well.


Does not look usable, according the the specification it does not support "non-microsoft" file systems.
Code: Select all
Operating system: Windows XP/Vista/7/8/2000 Professional, ect.
File system: FAT12/FAT16/FAT32/VFAT/NTFS/NTFS 5, and so on.
Hard disk: IDE hard disk, SATA hard disk, SCSI hard disk, and other mobile hard disks.


Question: have you done a successful clone of a Pi SD card ?


Yes, I cloned a retropi card yesterday :) It worked 1st time, and showed the partitions on the card (being copied), however that is all I have tested it on :)
Regards


Matt :)
Posts: 2
Joined: Mon Mar 20, 2017 1:29 pm