User avatar
pnelsonsr
Posts: 20
Joined: Sat Mar 30, 2013 6:40 pm

SD Backup Process

Tue Jul 29, 2014 8:56 pm

I am seeking a backup process for my 16GB SD card. What I have been doing (it does work...) is:

1. halt the rpi and on another linux system insert the rpi's SD card
2. use gparted to reduce the size of the second partition on the SD card down to 6GB (based on the data on the partition is always about 5GB).
3. use dd to create image of the SD and gzip it
4. use gparted to increase the size of the second partition on the SD card up to max allowable.
5. move the SD card back to the rpi.

The reason to reduce the partition size is because a call like 'dd if=/dev/sdg of=myrpi.img bs=4M' creates a image that can not be put onto another 16GB SD card. So I reduce the size of partition 2 and then I can restore the image to a new 16GB SD card and have it work in a rpi.

So... I was going to try having partition 2 not take up the max space left on the SD card and make it be 1GB less. Hoping that the restore would create a working SD card. But I thought I would ask if anyone has a better idea or maybe I just doing something wrong. Any ideas?

User avatar
mikronauts
Posts: 2726
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: SD Backup Process

Tue Jul 29, 2014 9:08 pm

I've always just restored the saved image (without resizing) - however I make sure the destination card is at least as big as the source card.

Your method has the advantage of working on slightly smaller destination cards :)

FYI, I've had good results with Kingston and ADATA cards, and some issues with Patriot SD cards (which I now use in my cameras)
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

User avatar
DougieLawson
Posts: 36302
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: SD Backup Process

Tue Jul 29, 2014 9:13 pm

It's not difficult to resize the image of a card.
http://www.raspberrypi.org/phpBB3/viewt ... 91&t=58069
http://www.raspberrypi.org/phpBB3/viewt ... 91&t=60918

I normally do my backups on a running RPi set it going at midnight and it chuggs away overnight.

sudo dd if=/dev/mmcblk0 of=/shared/Raspi/Raspbian.ddmmyyy.img bs=50M &

My /shared directory is on my NAS device and mounted as an NFS share.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
pnelsonsr
Posts: 20
Joined: Sat Mar 30, 2013 6:40 pm

Re: SD Backup Process

Tue Jul 29, 2014 11:08 pm

DougieLawson wrote:It's not difficult to resize the image of a card.
Well I think I like the sound of that...

Do you run the perl resize script? I looked at it and I was trying to figure out how it gets done. It looks like:

1. the variable $image is set from command line input
2. use 'parted -m $image unit B print | grep ext4' which the first 4 fields get populated into $num,$start,$old,and $dummy.
3. use 'losetup -f --show -o $start $image' ouput to $loopback
4. use 'e2fsck -p -f $loopback' to check and preen the partition
5. use `resize2fs -P $loopback 2>&1` which the first 2 fields are set to $dummy and $size and this size is the min size of the fs
6. add 1024 to $size
7. use 'resize2fs -p $loopback $size 2>&1` to resize the partition (showing progress)
8. use 'losetup -d $loopback' to detach the image file

--->OK so that kind of makes sense... But I find the rest confusing:<---

9. multiply $size by 4096 and add it to $start
<-not sure why
10. use 'parted $image rm $start'
<-not sure what this does?
11. use `parted -s $image unit B mkpart primary $start $size`
<-as with the previous, not sure what this does?
12. add 58720257 to $size
<-not sure what 58720257 is?
13. use 'truncate -s $size $image to reset the image size based on the changes

do you understand what these last 4 steps (9-12)?

User avatar
AndrewS
Posts: 3625
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
Contact: Website

Re: SD Backup Process

Tue Jul 29, 2014 11:19 pm

pnelsonsr wrote:do you understand what these last 4 steps (9-12)?
It's too late in the day for me to dive into it in detail, but it looks like they're deleting the first (FAT) partition, and then creating a new (empty) 56MB first partition? :?

User avatar
DougieLawson
Posts: 36302
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: SD Backup Process

Tue Jul 29, 2014 11:33 pm

The reason for starting on a 4K boundary is for SDCard performance reasons. Split blocks aren't delivered as quickly as contiguous blocks on a 4K boundary. That's why there's some juggling done to create the new partitions on a sensible boundary.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
pnelsonsr
Posts: 20
Joined: Sat Mar 30, 2013 6:40 pm

Re: SD Backup Process

Wed Jul 30, 2014 1:25 am

DougieLawson wrote:The reason for starting on a 4K boundary is for SDCard performance reasons. Split blocks aren't delivered as quickly as contiguous blocks on a 4K boundary. That's why there's some juggling done to create the new partitions on a sensible boundary.
is that for step 9 or step 12?
9. multiply $size by 4096 and add it to $start
<-not sure why
10. use 'parted $image rm $start'
<-not sure what this does?
11. use `parted -s $image unit B mkpart primary $start $size`
<-as with the previous, not sure what this does?
12. add 58720257 to $size

User avatar
AndrewS
Posts: 3625
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
Contact: Website

Re: SD Backup Process

Wed Jul 30, 2014 1:31 am

FYI: 56 MB = 57344 KB = 58720256 bytes ;)

User avatar
rpdom
Posts: 15353
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: SD Backup Process

Wed Jul 30, 2014 5:16 am

pnelsonsr wrote:9. multiply $size by 4096 and add it to $start
<-not sure why
10. use 'parted $image rm $start'
<-not sure what this does?
11. use `parted -s $image unit B mkpart primary $start $size`
<-as with the previous, not sure what this does?
12. add 58720257 to $size
<-not sure what 58720257 is?
13. use 'truncate -s $size $image to reset the image size based on the changes

do you understand what these last 4 steps (9-12)?
I believe I see what is happening here.
The previous steps have resized the / filesystem, but not changed the size of the partition it is in.
Here goes:
9. We need the new filesystem size in bytes, so convert from 4KB blocks to Bytes by multiplying by 4096 (4K).

10. Use parted to delete the old partition parameters.

11. Use parted to create the new partition parameters to match the new filesystem data (same $start, new $size).

12. Add the size of the VFAT filesystem and the MBR to the size of the new partition to get the size of the new image file in bytes.

13. Reduce the size of the image file to match that value.

User avatar
AndrewS
Posts: 3625
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
Contact: Website

Re: SD Backup Process

Wed Jul 30, 2014 8:03 am

...ahhhhh. In which case the script has a hard-coded assumption about the size of the existing VFAT partition, which obviously may not be correct! :(

User avatar
rpdom
Posts: 15353
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: SD Backup Process

Wed Jul 30, 2014 8:53 am

That's what it looks like to me, but I haven't examined the whole script, just the notes above.

What I would probably do is to add the size of the second partition to the start of the partition, then that size in bytes would be the required size of the file.

User avatar
pnelsonsr
Posts: 20
Joined: Sat Mar 30, 2013 6:40 pm

Re: SD Backup Process

Wed Jul 30, 2014 7:18 pm

rpdom wrote:I believe I see what is happening here.
The previous steps have resized the / filesystem, but not changed the size of the partition it is in.
Here goes:
9. We need the new filesystem size in bytes, so convert from 4KB blocks to Bytes by multiplying by 4096 (4K).

10. Use parted to delete the old partition parameters.

11. Use parted to create the new partition parameters to match the new filesystem data (same $start, new $size).

12. Add the size of the VFAT filesystem and the MBR to the size of the new partition to get the size of the new image file in bytes.

13. Reduce the size of the image file to match that value.
Thanks for the step by step description... Helps me get what is going on!

User avatar
pnelsonsr
Posts: 20
Joined: Sat Mar 30, 2013 6:40 pm

Re: SD Backup Process

Wed Jul 30, 2014 7:30 pm

AndrewS wrote:...ahhhhh. In which case the script has a hard-coded assumption about the size of the existing VFAT partition, which obviously may not be correct! :(
So how would one get the actual size of the VFAT?
rpdom wrote:That's what it looks like to me, but I haven't examined the whole script, just the notes above.
What I would probably do is to add the size of the second partition to the start of the partition, then that size in bytes would be the required size of the file.
So, for the file size (ie step 13 above) you need to add the size of the second partition to the start of the second partition -> is that (2nd Partition Size + 1st Partition size)?

User avatar
AndrewS
Posts: 3625
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
Contact: Website

Re: SD Backup Process

Wed Jul 30, 2014 7:37 pm

pnelsonsr wrote:So how would one get the actual size of the VFAT?
The same way as you get the size of the ext4?
So, for the file size (ie step 13 above) you need to add the size of the second partition to the start of the second partition -> is that (2nd Partition Size + 1st Partition size)?
It might be. But you need to take into account that (depending on how exactly the disk was partitioned) there might be unused space on the disk before the first partition, between the first and second partitions, and after the second partition. :geek: So the end of the first partition might not be the same as the size of the first partition, and the end of the first partition might not be the same as the start of the second partition. :?

User avatar
pnelsonsr
Posts: 20
Joined: Sat Mar 30, 2013 6:40 pm

Re: SD Backup Process

Wed Jul 30, 2014 8:02 pm

AndrewS wrote:It might be. But you need to take into account that (depending on how exactly the disk was partitioned) there might be unused space on the disk before the first partition, between the first and second partitions, and after the second partition. :geek: So the end of the first partition might not be the same as the size of the first partition, and the end of the first partition might not be the same as the start of the second partition. :?
well that is exactly the situation! There is unused space between the 1st and 2nd partition and unused space at the end. How to account for that?

User avatar
AndrewS
Posts: 3625
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
Contact: Website

Re: SD Backup Process

Thu Jul 31, 2014 3:27 pm

pnelsonsr wrote:well that is exactly the situation! There is unused space between the 1st and 2nd partition and unused space at the end. How to account for that?
By repeating everything that's done to get info about the ext4 partition, but grepping for 'fat32' instead of 'ext4' ?

Return to “Advanced users”