How do I make clone/backup image of my SD card?

[LucidEye]

OK... so I've been experimenting a LOT with Raspbian and Pidora and have gotten most of the programming and functionality for my projects working and now I want to make full clones and or backup images of my SD cards just in case I break something or have a card get corrupted for some reason. This would save me hours (maybe even days) of having to reinstall and setup all the packages, config file edits, and scripts I have done.

What software is available that can make full backup image files of Linux partitions from an SD card, and be able to write them back to another SD card? Or just do direct cloning from SD card to SD card?

I used to be a Windows user and I would have used Acronis to do this kind of backup... but I am now exclusively a Linux user and no longer have a windows machine... and I am having trouble finding any kind of backup application with the capabilities of Acronis.

Can someone please offer any suggestions for a good, easy to use, and robust backup image and cloning app for Linux?

Thank you.
J

[SirLagz]

dd
just like when you flashed the card originally, you can also create images of the card.
Plug the card into a computer and run this as root

Code: Select all

dd if=/dev/sdb of=raspi.img bs=1M

Replace /dev/sdb with whichever device your SD Card is at
Rename raspi.img to whatever you want to call the image.

[pluggy]

Like SirLagz says, good ol' "disk destroyer" will work. Not to say the windows option isn't capable of achieving the exactly the same negative result.

Beware putting it back onto a different card that is slightly smaller than the original. There a topic all on it's own (come up many times, as has this one).

[Heater]

I would be inclined to find a more organized way to back up your work that just a snapshot of your current disk image.

1) packages - if they are regular packages that come with Raspian or whatever I would not worry about them. They are just an apt-get away.
If they are things you have downloaded and compiled perhaps you can backup the original down load and if the compiling was complex or took a long time back up the results of that as well. This could be as simple as:

$ tar -cvzf someProgramsDirectory someProgram.tgz

and the copy that over to you PC in a safe place.

2) config file edits - You should at least keep a record of what you have changed and why. Some people will back up their entire /etc directory using tar as above. Some people even keep there etc directory in a source code repository like git.

3) and scripts - This is basically code you have created. As such it should be kept in a source code repository. Like git.

See www,github.com. That is a good place to throw stuff like this. It's free.

The nice thing about sorting your self out like this is:

1) You don't end up keeping multiple versions of huge image files around. Which is slow to create and restore anyway.

2) Using a source code repo it's easy to undo changes when you make a mistake. You don't have to copy the whole image which may or may not be recent at the time.

3) When you want to share all your hard work with friends it's much easier to give them those repos and tarballs than having them have to adopt your entire OS image.

Another approach is to use a program called unison, which is available on Linux and Windows and can synchronize files and directories across a network. It's kind of cool because it won't just automatically sync stuff, which be a problem if you accidentally delete something one day. It will ask you which way you want changes to propagate. There is a nice GUI for unison as well.

[RaTTuS]

see also
rsync

[LucidEye]

Awesome!
Thank you all for the suggestions
I think the ability to re-save my current setup as a complete .img file is the best option for me as I can save copies of it anywhere I want or make a backup card to keep handy should I do something to corrupt the card I'm using in the Pi at the moment... which is a big possibility since I will be writing thousands of pictures to it for time lapse movies.

Heater,
I will definitely check out UNISON also... sounds like it could be handy for doing rollbacks if I need that option on more complicated projects as well as for backing up my laptops... but mostly I have a "history" of all the things I've installed and edited in my browser bookmarks and a few notes I have that are all backed up. I don't really need all the detail you suggested for my Pi projects as once I have something working reliably I tend to just leave it alone... I am a strong believer in the kiss principle. The projects I'm working on are mostly for time-lapse photography and now that I have them working I just want to have a backup of the whole OS and all the changes I made for emergency recovery in the field should I need it... fastest way to do that is with a copy of the SD card I can just pop in should the main one fail.

Thanks again everyone! This gives me peace of mind and less fear of breaking something
J

[LucidEye]

So from what I've read, the "dd" command-line app does not support the EXT4 file system... or at least not very well. And most of the other Linux drive imaging apps out there do not support EXT4 either... which I just don't understand.... why have a file system that can not be backed up through full drive imaging or cloning?

Full drive imaging is the best way to backup any system... in the event of a crash you can have your whole system up and running again in minutes with maybe only a couple of updates to run depending on how often you make backup images. Most of the Linux "backup solutions" are only FILE backups... file backups are great for backing up USER DATA like photos or documents... but they are not good for backing up the system partition which your OS boots from as they usually do not backup boot files, boot sectors, or MBRs... nor can they backup any files that are in use by the system. Nor can file backups preserve drive partition information. So these are not what I consider full disaster recovery backup solutions, and will not work for what I need to do here. In the event of a crash you would have to do a full re-install of your whole OS, and then reinstall and configure all your apps, and then copy all of your backed-up files and directories to the drive and hope that it all works... that process could take hours, days, or longer... rather than the 10 to 20 minutes it takes to simply clone a whole drive image to a new drive or SD card and be up and running again like nothing happened.

As far as I am aware, all of the current R-Pi distros are set up by default using the ext4 file system... which means that most of the imaging backup apps out there will not work. However, Pi distros also have a FAT32 partition which I assume is the boot partition needed to start the Pi and load the firmware for the ARM/GPU.

Does anyone know if this FAT32 partition is needed or can the Pi boot directly from an ext3 or ext4 bootable partition? Did they have to do it this way because there is no BIOS stored on the Pi? Can I expand the FAT32 partition and move all my Linux files there and just run my whole distro from one partition instead of 2? It would be nice if I could run it all from one FAT32 partition since there are many drive imaging apps that fully support FAT32.

Either way, I am still in search of a drive imaging/cloning app for Linux that supports the EXT4 file system... and preferably a GUI based one since I really don't like messing with CLI apps that can irreparably damage or erase an entire drive with just one simple typo.
What apps are these companies using that offer preloaded Pi SD cards? What apps are the developers releasing the Pi distros using to create the .img files?

[SirLagz]

So from what I've read, the "dd" command-line app does not support the EXT4 file system... or at least not very well. And most of the other Linux drive imaging apps out there do not support EXT4 either... which I just don't understand.... why have a file system that can not be backed up through full drive imaging or cloning?

....what ????
dd is filesystem agnostic, it just makes a bit for bit copy of the device in question.
I'd like to see what you've read if that's what they say.

Full drive imaging is the best way to backup any system... in the event of a crash you can have your whole system up and running again in minutes with maybe only a couple of updates to run depending on how often you make backup images. Most of the Linux "backup solutions" are only FILE backups... file backups are great for backing up USER DATA like photos or documents... but they are not good for backing up the system partition which your OS boots from as they usually do not backup boot files, boot sectors, or MBRs... nor can they backup any files that are in use by the system. Nor can file backups preserve drive partition information. So these are not what I consider full disaster recovery backup solutions, and will not work for what I need to do here. In the event of a crash you would have to do a full re-install of your whole OS, and then reinstall and configure all your apps, and then copy all of your backed-up files and directories to the drive and hope that it all works... that process could take hours, days, or longer... rather than the 10 to 20 minutes it takes to simply clone a whole drive image to a new drive or SD card and be up and running again like nothing happened.

As far as I am aware, all of the current R-Pi distros are set up by default using the ext4 file system... which means that most of the imaging backup apps out there will not work. However, Pi distros also have a FAT32 partition which I assume is the boot partition needed to start the Pi and load the firmware for the ARM/GPU.

Does anyone know if this FAT32 partition is needed or can the Pi boot directly from an ext3 or ext4 bootable partition? Did they have to do it this way because there is no BIOS stored on the Pi? Can I expand the FAT32 partition and move all my Linux files there and just run my whole distro from one partition instead of 2? It would be nice if I could run it all from one FAT32 partition since there are many drive imaging apps that fully support FAT32.

Either way, I am still in search of a drive imaging/cloning app for Linux that supports the EXT4 file system... and preferably a GUI based one since I really don't like messing with CLI apps that can irreparably damage or erase an entire drive with just one simple typo.
What apps are these companies using that offer preloaded Pi SD cards? What apps are the developers releasing the Pi distros using to create the .img files?

Have you had a look at clonezilla ?

[RaTTuS]

Several things here are just ...

So from what I've read, the "dd" command-line app does not support the EXT4 file system... or at least not very well.

wrong ,
dd just does basic low level stuff it does not know or care about any filesystem

And most of the other Linux drive imaging apps out there do not support EXT4 either... which I just don't understand.... why have a file system that can not be backed up through full drive imaging or cloning?

dunno where you get this info from can you URL me a source

Full drive imaging is the best way to backup any system... in the event of a crash you can have your whole system up and running again in minutes with maybe only a couple of updates to run depending on how often you make backup images.

it depends - and not in mins if you have several hundred Gig's worth of data
what you may want to do is have the OS and data on separate drives [raid 5 for starters]

Most of the Linux "backup solutions" are only FILE backups... file backups are great for backing up USER DATA like photos or documents... but they are not good for backing up the system partition which your OS boots from as they usually do not backup boot files, boot sectors, or MBRs... nor can they backup any files that are in use by the system. Nor can file backups preserve drive partition information. So these are not what I consider full disaster recovery backup solutions, and will not work for what I need to do here. In the event of a crash you would have to do a full re-install of your whole OS, and then reinstall and configure all your apps, and then copy all of your backed-up files and directories to the drive and hope that it all works... that process could take hours, days, or longer... rather than the 10 to 20 minutes it takes to simply clone a whole drive image to a new drive or SD card and be up and running again like nothing happened.

file backups are fine for for the most part

the backup of the OS drive will backup all installed programs and settings
if you want to start again from a fresh system
all you have to do is partition up the new drive how you want it and then dump the file info onto that partition, in fact this is better than a dis image as you can have a completely new file system regime [moved from riserfs to ext4 or other] and everything is happy

the partition info is immaterial to the os

As far as I am aware, all of the current R-Pi distros are set up by default using the ext4 file system... which means that most of the imaging backup apps out there will not work. However, Pi distros also have a FAT32 partition which I assume is the boot partition needed to start the Pi and load the firmware for the ARM/GPU.

Does anyone know if this FAT32 partition is needed or can the Pi boot directly from an ext3 or ext4 bootable partition? Did they have to do it this way because there is no BIOS stored on the Pi? Can I expand the FAT32 partition and move all my Linux files there and just run my whole distro from one partition instead of 2? It would be nice if I could run it all from one FAT32 partition since there are many drive imaging apps that fully support FAT32.

nothing wrong with ext4 - some may argue that it is perhaps not the best for a flash filesystem but I'm not really convinced [and may people out there in linux land also say this]
the RPi needs a fat32 partition as that is what the SOC can read via the GPU -
this can be backed up with a simple copy the size is not important

you can run from an image stored in a file on a fat32 image but it will be limited to sized constrained to fat32 ..
and also adds another level of obscurity -
linux really needs a file system has extended file attributes on

Either way, I am still in search of a drive imaging/cloning app for Linux that supports the EXT4 file system... and preferably a GUI based one since I really don't like messing with CLI apps that can irreparably damage or erase an entire drive with just one simple typo.
What apps are these companies using that offer preloaded Pi SD cards? What apps are the developers releasing the Pi distros using to create the .img files?

gui is just as damaging as the commandline

if you want to backup your system [on the RPi] then dd will do the job you want there is probably a gui version but the raspberry is best served by the command line as it empowers you to know what is happening,
you can take out the sd card and use windows or a mac and copy the entire disk
or even dd from a live system on the RPi but I would use rsync personally

the distros will use dd to create the image
make a small 2GB system and then dd and limit the dd to be just after the end of the filesystem
then it uses parted to expand the filesystem on 1st boot