gunflame
Posts: 35
Joined: Mon May 14, 2012 2:15 am

SD Card Backup and Custom Image...

Sun Oct 28, 2012 7:25 pm

Here's the thing... I want to keep playing around with different software and configurations on my RPI... but I have already highly customized my Raspbian distribution.

I would like to make a backup (or an image) that I can use to reflash my SDcard in case everything goes wrong. I actually have 3 SD cards so I would also like to port the changes without having to do all the configurating from scratch.

So I would like to:
- Backup my SD card
- Restore (copy) to other SD Cards freely

Can anyone point me to a guide on how to do this? I don't even know what to write on a search bar in google/forum to get an answer.

tdaemon
Posts: 4
Joined: Sun Oct 28, 2012 9:43 pm

Re: SD Card Backup and Custom Image...

Sun Oct 28, 2012 9:48 pm

You can use dd or Win32DiskImager, which are recommended to write SD images from download secton. With both tools you can also create image from SD card...

billw
Posts: 363
Joined: Tue Sep 18, 2012 8:23 pm

Re: SD Card Backup and Custom Image...

Mon Oct 29, 2012 2:22 am

If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS

smallgod
Posts: 10
Joined: Fri Jan 04, 2013 12:19 am

Re: SD Card Backup and Custom Image...

Sun Jan 13, 2013 8:27 am

billw wrote:If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS
Thanks for this script!

billw
Posts: 363
Joined: Tue Sep 18, 2012 8:23 pm

Re: SD Card Backup and Custom Image...

Mon Jan 14, 2013 1:45 am

I made a small change to rpi-clone to avoid a possible error message when rsync tries and fails to copy a gnome .gvfs directory. If anyone has been getting this error, the error is harmless since .gvfs is a fuse mount point which should not be copied anyway, but I'm now explicitly excluding it to keep rsync quiet. If you don't want to re download rpi-clone, you can just edit it with this patch:

Code: Select all

--- rpi-clone.old0	2013-01-13 19:10:15.220909998 -0600
+++ rpi-clone	2013-01-13 19:09:40.174680138 -0600
@@ -311,10 +311,14 @@
 
 START_TIME=`date '+%H:%M:%S'`
 
+# Exclude fuse mountpoint .gvfs, various other mount points, and tmpfs
+# file systems from the rsync.
+#
 sync
 echo "Starting the filesystem rsync to $DST_DISK"
 echo -n "(This may take several minutes)..."
 rsync $RSYNC_OPTIONS --delete \
+		--exclude '.gvfs' \
 		--exclude '/dev' \
 		--exclude '/media' \
 		--exclude '/mnt' \
In case anyone wants to try it, you can do the local edit of your rpi-clone file using patch. If you create a patch file (say rpi-clone.patch) containing the above patch text, you can cd to
the bin directory where you installed rpi-clone, and apply the patch with:

Code: Select all

patch -p0 -l rpi-clone < rpi-clone.patch
Last edited by billw on Mon Jan 14, 2013 1:55 am, edited 1 time in total.

kumarnarain
Posts: 7
Joined: Sat May 19, 2012 1:27 pm

Re: SD Card Backup and Custom Image...

Tue Jan 15, 2013 1:25 pm

billw wrote:If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS
Hi
The script throws an error on Line 26 'Function not found' What is missing possibly? I have a 16 gb card inserted for back up purposes. Tried looking at the log ......

[email protected] /var/log $ tail rpiclone.log
tail: cannot open `rpiclone.log' for reading: No such file or directory

Regards
Kumar

billw
Posts: 363
Joined: Tue Sep 18, 2012 8:23 pm

Re: SD Card Backup and Custom Image...

Tue Jan 15, 2013 4:34 pm

Do you have the bash shell installed? You could get that error if for some reason dash is being used instead of bash. If bash is installed, try running bash explicitly:

Code: Select all

bash /pathto/rpi-clone
If that still fails, add the -x flag to bash to get a trace.

If you don't have bash installed, one thing you can try is edit rpi-clone and change line 26 from:

Code: Select all

function usage
to

Code: Select all

usage()
Regardless, that's a change I should probably make anyway.

kumarnarain
Posts: 7
Joined: Sat May 19, 2012 1:27 pm

Re: SD Card Backup and Custom Image...

Wed Jan 16, 2013 5:32 am

Thank you. Will try that and post back. Did not know it needed bash.
Regards
Kumar

sqrt3
Posts: 9
Joined: Thu Jan 17, 2013 2:52 pm

Re: SD Card Backup and Custom Image...

Thu Jan 17, 2013 3:19 pm

Thanks for this script from me too.
Since i'm running a different locale where the decimal seperator is ',' the following lines did not work for me:

Code: Select all

PART2_START=$(parted /dev/mmcblk0 -ms unit MB p | grep "^2" \
				| cut -f 2 -d: | sed s/MB// | cut -f 1 -d.)
I changed it to:

Code: Select all

PART2_START=$(parted /dev/mmcblk0 -ms unit MB p | grep "^2" \
				| cut -f 2 -d: | cut -f 1 -d. | cut -f 1 -d,)
which worked.

kumarnarain
Posts: 7
Joined: Sat May 19, 2012 1:27 pm

Re: SD Card Backup and Custom Image...

Thu Jan 17, 2013 3:48 pm

Which locale is this? Just curious..

sqrt3
Posts: 9
Joined: Thu Jan 17, 2013 2:52 pm

Re: SD Card Backup and Custom Image...

Thu Jan 17, 2013 4:58 pm

It's german ',' and '.' are interchanged in decimal numbers. BTW i omitted the

Code: Select all

sed s/MB//
but it's probably better to keep that in case of numbers without decimals.

billw
Posts: 363
Joined: Tue Sep 18, 2012 8:23 pm

Re: SD Card Backup and Custom Image...

Thu Jan 17, 2013 6:39 pm

Thanks for letting me know about that locale problem and I've put a fix for it into the rpi-clone script. Also, I've now put rpi-clone on github and you can get it with:

Code: Select all

git clone https://github.com/billw2/rpi-clone.git 
For now, it's still on that pastebin link, but I think github is a better place for it.

1HzCoder
Posts: 31
Joined: Thu Jul 12, 2012 1:15 am
Location: Lower Alabama

Re: SD Card Backup and Custom Image...

Sat Jan 19, 2013 11:56 pm

Billw
Downloaded manually and put in proper folder, git doesn't seem to work on Wheezy, ran it and after trying to use the drive,
got it to copy sda1 successfully, but when it gets to

"starting the file system rsync to sda
(This may take several minutes) .../user/local/sbin/rpi-clone: line 323: rsync: command not found
../user/local/sbin/rpi-clone: line 371 /mnt/clone//var/log/rpi-clone.log: no such file or directory

***Done with clone to /dev/sda ***
started:16:24:56 Finished: 16:25:01"

Do I need to create the log file, and should I redownload the zip file?
Thanks for the script, it's been years since I messed with Linux and it pretty much guided me through all my screw-ups.
TomJ
Einstein once said you don't really understand anything until you can explain it to your Grandmother

billw
Posts: 363
Joined: Tue Sep 18, 2012 8:23 pm

Re: SD Card Backup and Custom Image...

Sun Jan 20, 2013 2:19 am

1HzCoder wrote:Billw
Downloaded manually and put in proper folder, git doesn't seem to work on Wheezy, ran it and after trying to use the drive,
got it to copy sda1 successfully, but when it gets to

"starting the file system rsync to sda
(This may take several minutes) .../user/local/sbin/rpi-clone: line 323: rsync: command not found
../user/local/sbin/rpi-clone: line 371 /mnt/clone//var/log/rpi-clone.log: no such file or directory

***Done with clone to /dev/sda ***
started:16:24:56 Finished: 16:25:01"

Do I need to create the log file, and should I redownload the zip file?
Thanks for the script, it's been years since I messed with Linux and it pretty much guided me through all my screw-ups.
TomJ
The first error you got was because the script tried to run the rsync command but could not find it because it looks like rsync is not installed by default on the Raspbian image. I did not remember that I must have installed it myself at some point and should have made a point of mentioning that rsync needs to be installed ... sorry about that.

Anyway, you need to install rsync:

Code: Select all

apt-get update
apt-get install rsync
The second error you got about the log file was because rsync did not run and create the necessary directory structure needed to write the log file. So just get rsync installed and try running rpi-clone again. The script was running so I'm pretty sure you downloaded and installed it OK.

1HzCoder
Posts: 31
Joined: Thu Jul 12, 2012 1:15 am
Location: Lower Alabama

Re: SD Card Backup and Custom Image...

Sun Jan 20, 2013 3:13 am

Thanks Bill,
Will check that in the morning,there are So-o-o many little things like that that I don't remember.
:roll:
TomJ
Einstein once said you don't really understand anything until you can explain it to your Grandmother

dallas1951
Posts: 1
Joined: Sun Feb 24, 2013 4:59 am

Re: SD Card Backup and Custom Image...

Sun Feb 24, 2013 7:12 am

I tried this today and the script failed.
[email protected]:~# cd /usr/local/sbin/rpi-clone/
[email protected]:/usr/local/sbin/rpi-clone# ./rpi-clone sda -v

Clone destination disk : sda
Clone destination rootfs : /dev/sda2 (no label) on /mnt/clone
Clone destination bootfs : /dev/sda1 on /mnt/clone/boot
Verbose mode : on
===============================
Final check, is it Ok to proceed with the clone (yes/no)?: y
=> Mounting /dev/sda2 (no label) on /mnt/clone
mount: special device /dev/sda2 does not exist
Mount failure of /dev/sda2, aborting!

Prior to running the script cat /prog/partitions gave me this.
[email protected]:~# cat /proc/partitions
major minor #blocks name

179 0 7761920 mmcblk0
179 1 76800 mmcblk0p1
179 2 7683072 mmcblk0p2
8 0 7782400 sda
8 1 76800 sda1
8 2 7703552 sda2
After it failed I got this.
[email protected]:~# cat /proc/partitions
major minor #blocks name

179 0 7761920 mmcblk0
179 1 76800 mmcblk0p1
179 2 7683072 mmcblk0p2

As you can see I'm running incredible pbx which is raspian wheezy December 2012.
To get the sda partitions back I have to power down the pi and restart - that seems odd.

Does anyone have any suggestions?

gridrun
Posts: 46
Joined: Mon Feb 18, 2013 12:26 pm
Contact: Website

Re: SD Card Backup and Custom Image...

Sun Feb 24, 2013 2:24 pm

Windows users might like HDDGURU's free 'HDD Raw Copy Tool' to fully clone and/or image their SD Cards. Most other Windows utilities work with partitions and can't see anything non-Windows, while this one works on the device layer and does not care about partitions nor filesystems.

You need to take special care of what devices you actually select for 'source' and 'destination' as to not accidentally overwrite the wrong drive. But the tool works flawlessly and does exactly as advertized. File sources (image files) are supported as either 'source' or 'destination', letting you create and restore images of an entire SD card's content at once.

URL: http://hddguru.com/software/HDD-Raw-Copy-Tool/
Find more info on Raspberry Pi, Virtualization and all things cloudy on my blog: http://niston.wordpress.com

mcgyver83
Posts: 358
Joined: Fri Oct 05, 2012 11:49 am

Re: SD Card Backup and Custom Image...

Thu Feb 28, 2013 2:50 pm

I'm interested in this tool to have a always up to date SD backup.
I plug in the usb hub a sd card reader with a card.
I run ./rpi-clone but this is the output:

Code: Select all

sudo ./rpi-clone sdb -v
A destination partition is busy (mounted).  Mount status:
    /dev/sdb2:  /media/usb1
    /dev/sdb1:  /media/usb0
Aborting!
Ok, it is clear :D
But my usb stuff is automounted so I can use also my external usb hdd with xbmc.
How can I edit the github script to umount these devices before run the backup?

Second question: I left 6GB of empty (not partitioned) space at the end of the usb disk, how can I use this space to backup my running rasp?In this way I can schedule a backup each night forgetting sd wearing problem.

billw
Posts: 363
Joined: Tue Sep 18, 2012 8:23 pm

Re: SD Card Backup and Custom Image...

Thu Feb 28, 2013 8:04 pm

Ok, I updated rpi-clone to try unmounting mounted destination partitions instead of just aborting. But of course it will still abort if the unmount fails.
Above I said to get rpi-clone you can do a:

Code: Select all

git clone https://github.com/billw2/rpi-clone.git
But you can also go to: https://github.com/billw2/rpi-clone and download the zip file.

keble
Posts: 36
Joined: Tue Apr 03, 2012 4:46 pm

Re: SD Card Backup and Custom Image...

Thu Feb 28, 2013 10:27 pm

Many thanks for that Bill. I have just stumbled across the same problem.
The updated script is running now.
Cheers!

mcgyver83
Posts: 358
Joined: Fri Oct 05, 2012 11:49 am

Re: SD Card Backup and Custom Image...

Mon Mar 04, 2013 12:05 pm

Nice!
Many thanks!
:D :D :D

Radics
Posts: 1
Joined: Thu Jan 31, 2013 3:48 pm

Re: SD Card Backup and Custom Image...

Mon Mar 18, 2013 2:00 pm

The script on the git site is working for me it is running right now. My question is do I have to restore this somewhere or will this card now boot up like the one it is a clone of?

spamdan
Posts: 6
Joined: Sun Jul 07, 2013 3:38 am

Re: SD Card Backup and Custom Image...

Sun Jul 07, 2013 3:46 am

Isn't it true that using dd to copy an active filesystem can lead to data corruption? I've read this in several places and it seems to make sense as some of the files will be open at the time (and possibly being written to). This tool looks great but I want to make sure I'm backing up my Pi in a way that won't cause data corruption.

chorlton2080
Posts: 124
Joined: Sun Dec 23, 2012 9:44 pm

Re: SD Card Backup and Custom Image...

Thu Jan 02, 2014 12:24 am

billw wrote:If you have a USB card reader to plug your SD cards into, you can try
a script I wrote that does backups entirely on the Pi. It backs up
a booted Pi SD card to a card plugged into a Pi USB port.
I use it to maintain backups of several SD cards I have and
the backup SD cards can be a different size from the booted SD card.

The script is rpi-clone and I have it on pastebin:
http://pastebin.com/48fr9BAS
Bill: this is a superb tool and much more efficient/flexible than using Win32DiskImager. Thanks for putting the time and effort into creating and sharing it.

Eradicatore
Posts: 23
Joined: Sun Jul 29, 2012 1:21 am

Re: SD Card Backup and Custom Image...

Fri Jan 03, 2014 9:50 pm

Yes, I second chorlton2080's comments!! Thank you so much Bill for sharing your script and putting the time into this! I just started reading the linux man pages on "parted" and partitioning in general, and then I finally stumbled on this thread. It was hard to find for some reason. All the noise out there about using the windows tool or dd....

I see your comment about being able to back up a booted RPI. That's what I was always worried about with using "dd". It sounds like you're saying this script is safe to run on a booted/mounted drive. Is that right? I'll try to answer that question for myself too. Thanks!

Justin

Return to “Advanced users”

Who is online

Users browsing this forum: asavah and 11 guests