pi-man-uk
Posts: 62
Joined: Tue May 08, 2012 1:41 pm

make backup dd image while pi is live

Mon Dec 09, 2013 4:42 pm

I like to backup the whole SD card from time to time using DD. I know it's inefficient, but I do find it convenient, one command to grab a whole card, and I like the fact it's a snapshot at a point in time that I can recreate from in one hit if necessary. I shutdown the system, put the card in my macbook and then dd.

Just wondered is there a way to do this across the network while the machine is running... i can imagine it may cause problems as the SD card could be being written to at the time and be in a transient state. Would be great if I could have the Pi image itself to my NAS via cron every once in a while. Anyone else do this?

User avatar
DougieLawson
Posts: 30168
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: make backup dd image while pi is live

Mon Dec 09, 2013 7:18 pm

You can use a Fuji SDCard Reader in the USB on a RPi. That's been the best £8 I've spent.

dd if=/dev/mmcblk0 of=/dev/sdb bs=1M

it's a "fuzzy" copy so may need a fsck when you boot that cloned card. If you 'touch /boot/forcefsck' before doing that copy then when it boots Raspbian will do an automagic fsck. Use rm /boot/forcefsck when you're done with the copy.

I've also done a dd if=/dev/mmcblk0 of=/mnt/NAS/RaspPi/raspbian.ddmmyyy.img bs=1M and left that running overnight (because it's slow but my machine is dormant (less fuzzy)).

Once you have an *.img then http://www.raspberrypi.org/phpBB3/viewt ... 91&t=58069 lets you resize it down to the minimum size needed (in my case using DeanC's script made 32GB shrink to 5GB).
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

Since 2012: 1B*5, 2B*2, B+, A+, Zero*2, 3B*3

Please post ALL technical questions on the forum. Do not send private messages.

pi-man-uk
Posts: 62
Joined: Tue May 08, 2012 1:41 pm

Re: make backup dd image while pi is live

Tue Dec 10, 2013 12:48 pm

That's interesting, thanks. You've described two ways to do the job on a "live" system. 1) Copy "card to card", presumably I need the SD cards to be the same size, sounds like this is the fastest way. 2) Copy the filesystem as an "image" to an external location, sounds like this is slower.

I'll have to try this out. Did I broadly understand it correctly?

User avatar
DougieLawson
Posts: 30168
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: make backup dd image while pi is live

Tue Dec 10, 2013 12:56 pm

If you are doing a straight if=/dev/mmcblk0 of=/dev/sdb copy the the output card needs to be the same size or larger.

With the img you can shrink it.

You can grow your ext4 filesystem using raspi-config.
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

Since 2012: 1B*5, 2B*2, B+, A+, Zero*2, 3B*3

Please post ALL technical questions on the forum. Do not send private messages.

vob
Posts: 5
Joined: Tue Dec 10, 2013 2:34 pm

Re: make backup dd image while pi is live

Tue Dec 10, 2013 3:13 pm

Well, if your NAS is based on linux and supports SSH you could use dd over ssh. Alternatively you could use netcat nc if encryption and security is not required.

Or just dd on a smb or nfs share. Maybe a good option would be to compress (gzip, bzip2, whatever) the stream before sending/storing, by piping it.

Code: Select all

dd if=/dev/sdx | gzip -1 - | dd of=/somewhere/sdx.img.gz

User avatar
RaTTuS
Posts: 9569
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: make backup dd image while pi is live

Tue Dec 10, 2013 3:51 pm

or rsync the filesystem ...
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

ShaneH
Posts: 5
Joined: Sun Jan 13, 2013 11:20 am

Re: make backup dd image while pi is live

Mon Dec 16, 2013 3:11 am

pi-man-uk wrote:2) Copy the filesystem as an "image" to an external location, sounds like this is slower.

I'll have to try this out. Did I broadly understand it correctly?
Will be considerably quicker if you pipe the output through gzip with max compression (gzip -9), since it will squash unallocated space down to pretty much nothing. DD will read the entire card, (which it will do no matter what method you use), but the output file will be very compressed.

If you have been really busy on the SD card, which will leave file fragments in unallocated space, you can 'wash' the unallocated space with:
dd if=/dev/zero of=bob.file
This will create a file filled with zeros until there is no unallocated space left on the disk, and dd just halts with a "No space left on device" message. Simply delete "bob.file" and that file becomes unallocated.

To be honest though, you would be better off chucking the card into a reader on your computer and dumping the entire card out to a file. If you keep working on the Pi whilst it is imaging, (or some other process/daemon is really active) you could end up corrupting the resulting output file, since activity will be changing the card contents as it's imaging. This includes any swap space if you plan to image the entire card at the block device level (/dev/sda vs /dev/sda1)

User avatar
RaTTuS
Posts: 9569
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: make backup dd image while pi is live

Mon Dec 16, 2013 9:50 am

if you are worried about life of sd cards then dd'ing 0's to the filesystem is not good

rsync is a much better option if you have a suitable remote server or even a SD card reader plugged into the RPi
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

Hoagie
Posts: 119
Joined: Sun Sep 23, 2012 7:19 pm

Re: make backup dd image while pi is live

Sat Jan 03, 2015 4:05 pm

An interesting (well I like it) side effect of saving an image to a network location, is that you can mount it and read files from it, almost as if it was a real card plugged into the pi. It's easier to do if you only image the partitions and not the whole card, but it's still possible with a card image.

Return to “Advanced users”

Who is online

Users browsing this forum: No registered users and 18 guests