SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Transfer working image to smaller card?

Tue Jan 22, 2013 6:50 pm

Hi,

My current Raspbian installation is working nicely, just the way I want it to with numerous edits and adjustments that I have made (such as the ones needed to reclaim the UART from the system, additions to the Bash path, and so on).

Trouble is, I originally flashed this image to a 32GB card which had been given to me because it was starting to fall apart and could no longer be safely inserted into/removed from the camera it was being used in, in case it started to leave bits of itself inside the camera. No such worry with the Pi, which has a wide open card slot.

However, it has now started to disintegrate to such an extent that it no longer sits reliably in the Pi's card holder, so I need to transfer the image to a new card.

On the face of it, the simplest way to do this is to copy the image of the existing card (All 32GB of it) to, say, a HDD and then copy that image to a new 32GB card. The problems with this are

1) The waste / time involved in copying a largely empty very large image to any kind of storage,

2) The possibility that the image from this 32GB card may not actually fit onto a new 32GB card, since different cards by different manufacturers seem to have capacities which differ by a few KB or so. If the new card is a few bytes smaller than the original card, the image will not fit onto it.

So: How can I reverse the raspi-config process which originally expanded the 2GB image to fit the 32GB card, and bring it back down to, say, just under 4GB (You can assume that all significant info on the card will still fit into 4GB for the purposes of this question).

And then, having created a 4GB Raspbian on the 32GB card, how do I clone that to another 4GB card?

Suggestions using Linux are welcome - I have a Linux (Debian Squeeze) PC which I can use for any Linux heavy-lifting that might be required. I can guess that I need to downsize the largest partition on the existing card, and then clone all significant content on it onto another card - but I don't really know how to do that in Linux.

Any help would be appreciated.

sdjf
Posts: 1395
Joined: Fri Mar 16, 2012 5:20 am
Location: California
Contact: Website

Re: Transfer working image to smaller card?

Tue Jan 22, 2013 7:46 pm

I would trying tarring up the contents of both partition 1 (boot) and partition 2 (the rootfs, as in, /) separately, 2 tar files. Then reflash a fresh image onto your 4 GB card, then put yourself at /boot to unpack any boot sector revisions, and at / to unpack the tar for partition 2.

This is not step by step, you will have to carefully choose where to do the tarring from, so you do not get extra directory levels you do not want. But this is what I would do. And what I have done for specific directories within partition 2 when having to reflash a fresh image and wanting to restore what was in, for example, /etc and /root, for which I created separate tar files as I had not installed enough additional software to make it worth tarring up the other directories. That is another approach, just tar up the contents of the directories you know have personalized tweaks, trouble with doing it that way, is there may be modifications in other directories that you may not know about.
FORUM TIP: To view someone's posting history, sign in, click on their user name, then on "Search User's Posts." || Running ArchLinuxArm on Model 2B and 512MB Model B

warwound
Posts: 5
Joined: Thu Jan 03, 2013 7:30 pm
Location: Norfolk, UK

Re: Transfer working image to smaller card?

Tue Jan 22, 2013 10:22 pm

I think i'd boot a pc using a PartedMagic ISO (ISO transferred to a spare USB stick) and use GParted to resize the ~32GB partition down to ~4GB.

Then you can use Clonezilla to backup the entire (4GB) SD card to a file or clone the SD card directly to a new SD card (space permitting).
(In fact i think the first thing i'd do is backup the original 32GB SD card to a file just in case anything goes wrong.)
Any backup files created with Clonezilla can be transferred back to a blank SD card at a later date.

Clonezilla applies compression to it's backup images so if you backup the original 32GB card then the backup will NOT be 32GBs in size.

PartedMagic includes both GParted and Clonezilla by the way - PartedMagic is all you need and it's free: http://partedmagic.com/doku.php#.UP8QJmfV-Uk

Martin.

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Tue Jan 22, 2013 10:45 pm

Thanks for the response -

After posting initially I looked through various Linux magazine coverdisks that I have accumulated and found one which had both Partition Magic and Clonezilla (Which I have never used before) on it. Bear in mind that I did this before I had read your suggestions, but I'll relate what I tried anyway.

On a PC, I first ran Partition Magic and resized the main partition on the flaky card to comfortably under 8GB - I had found an 8GB card and decided to use that as the destination for the copy. I left enough room for the other small (boot) partition to be able to fit into the 8GB as well.

I then tried using Clonezilla in its simplest mode to clone the flaky 32GB card to the 8GB card. As I had expected, Clonezilla picked up on the fact that the physical size of the destination device was much smaller than the physical size of the source, and refused to proceed. I had hoped it would notice that the actual partitions were small enough to fit on the destination card and do some intelligent juggling, but sadly not.

So I started again and this time, told Clonezilla to work with images rather than device to device, and when I got to the relevant part of the process I used the SaveParts feature of the program to save an image file of the partitions to the PC's hard drive, and verify the saved image as well. That went OK.

I then started Clonezilla again, worked through similar steps until I got to the point where I could choose 'RestoreParts' from the image I'd already saved and wrote the two partitions to the new smaller card. That all seemed to go OK and the process completed satisfactorily.

However, on attempting to boot from the 8GB card, trouble looms when it starts to say things like:

Code: Select all

VFS: Cannot open root device "mmcblk0p2" or unknown-block (179,2)
Please append a correct "root=" boot option; here are the available partitions:
b300 7846912 mmcblk0 driver: mmcblk
b301 7839688 mmcblk0p1 00000000-0000-0000-0000-000000000000
Kernel Panic -not syncing: VFS : Unable to mount root fs on unknown-block(179,2)
..It all pretty much goes to hell after that. From this, I assume that it wasn't sufficient just to clone the two partitions across onto the new card, but that the card also has to have some other low level stuff on it (like the boot sector on a HDD) which was not reinstated or created by the cloning process.

Your method would seem to be a more gentle way of doing things since it obviously isn't going to disturb whatever it is on the newly imaged card which needs to be there for Raspbian to boot: It just replaces some default stuff with my changed versions -so I'll give that a go. Part of the problem, though, is remembering just exactly what I have changed.

It would still be useful to know how to clone a card (any size) onto a smaller card (as long as the used area does not exceed the capacity of the destination card) as I can imagine that being something which people may want to do quite often.

Incidentally, the 32GB card still boots even after I resized the main partition from 29GB ish to less than 8GB - so damage to that partition does not appear to be the problem.

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Tue Jan 22, 2013 11:02 pm

warwound wrote:I think i'd boot a pc using a PartedMagic ISO (ISO transferred to a spare USB stick) and use GParted to resize the ~32GB partition down to ~4GB.

Then you can use Clonezilla to backup the entire (4GB) SD card to a file or clone the SD card directly to a new SD card (space permitting).
(In fact i think the first thing i'd do is backup the original 32GB SD card to a file just in case anything goes wrong.)
Any backup files created with Clonezilla can be transferred back to a blank SD card at a later date.

Clonezilla applies compression to it's backup images so if you backup the original 32GB card then the backup will NOT be 32GBs in size.

PartedMagic includes both GParted and Clonezilla by the way - PartedMagic is all you need and it's free: http://partedmagic.com/doku.php#.UP8QJmfV-Uk

Martin.
I did try this: Specifically,

Resized main partition on the original card so that it, plus the boot partition, were easily small enough to fit on an 8GB card.

Tried to clone big card to small card directly: Failed because Clonezilla knew the total size of the source was bigger than the destination.

Saved the partitions (boot, and the resized main partitions) to the PC's hard drive as a file. Restored them to the destination card. Process appeared OK but the card will not get more than a few lines into boot process (see one of my other posts in this thread for the error message).

You're saying that if I clone the big card to a HDD disc image, then clone it from there to the small card, Clonezilla will adjust it to fit the physical size of the destination, even though smaller?

User avatar
dhardingham
Posts: 28
Joined: Mon Jan 02, 2012 12:45 pm
Location: Stevenage, UK
Contact: Website

Re: Transfer working image to smaller card?

Wed Jan 23, 2013 12:00 am

I use a combination of gparted, Win32DiskImager and gsplit3, all runing on a Windows PC. I use gparted to reduce the size of the largest partiion, so that I have everything I need at the begining of the large SD card. I then use Win32DiskImager to copy the entire contents of the large SD card to an image file.

I then use gsplit3 to split the large image file into several smaller files. I make sure that the split size is large enough to cover all of the partitions that want to copy from the large image file to the first of the small image files.

Finally, I write the first small image file to a small SD card, using Win32DiskImager.
David Hardingham

sdjf
Posts: 1395
Joined: Fri Mar 16, 2012 5:20 am
Location: California
Contact: Website

Re: Transfer working image to smaller card?

Wed Jan 23, 2013 12:10 am

I think you missed the power of my suggesting you use the tar command. I could not take the time to walk you through earlier, but I am sure it would work.

1. dd a fresh image to your 8 GB card.

2. from command line on a linux box or on the pi (maybe the fresh one on the 8 GB card), do

cd PATH-TO-THE-32-GB-CARD-PARTITI0N-1/boot
tar cf PATH--TO-WHERE-TAR-FILE-GOES/boot-partition.tar *

3. from command line, as above, from a linux box (or your pi running the temporary 8 GB image). If mounted as /dev/sda2, then you would "cd /dev/sda2" instead of cd PATH-TO/

cd PATH-TO-THE-32-GB-CARD-PARTITION-2/
tar cf PATH--TO-WHERE-TAR-FILE-GOES/rootfs-partition.tar

Then you have 2 tar files, one for each partition.

4. use gparted or some other similar tool to expand the second partition on the 8 GB card to full capacity.

5. Again, still on your main Linux box:

cd PATH-TO-BOOT-PARTITION-ON-8GB-CARD/
tar xf PATH-TO-FIRST-TARFILE/boot-partition.tar

cd PATH-TO-SECOND-PARTITION-ON-8GB-CARD/
tar xf PATH-TO-SECOND-TARFILE/rootfs-partition.tar

6. I think that is it, but requires doing it all on a Linux box, as Windoze/MAC cannot read the second partition.
Last edited by sdjf on Wed Jan 23, 2013 12:23 am, edited 2 times in total.
FORUM TIP: To view someone's posting history, sign in, click on their user name, then on "Search User's Posts." || Running ArchLinuxArm on Model 2B and 512MB Model B

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Wed Jan 23, 2013 12:19 am

dhardingham wrote:I use a combination of gparted, Win32DiskImager and gsplit3, all runing on a Windows PC. I use gparted to reduce the size of the largest partiion, so that I have everything I need at the begining of the large SD card. I then use Win32DiskImager to copy the entire contents of the large SD card to an image file.

I then use gsplit3 to split the large image file into several smaller files. I make sure that the split size is large enough to cover all of the partitions that want to copy from the large image file to the first of the small image files.

Finally, I write the first small image file to a small SD card, using Win32DiskImager.
That sounds pretty workable: The main thing I baulk at is the prospect of having to make an uncompressed image of a 32GB card onto something else when I know that it is mostly empty space. But if it works, well...

I have to admit this solution would not have occurred to me because I would assume that the OS would start spreading its files all over the available physical area of the SD card, so that after a while there would be little bits of stuff scattered all over it - the idea that you can just saw off a quarter of the raw image and hope that everything you want is safely cocooned within that first quarter seems fantastic, but since you've obviously tried it then I'll be happy to give it a go.

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Wed Jan 23, 2013 12:30 am

sdjf wrote:I think you missed the power of my suggesting you use the tar command. I could not take the time to walk you through earlier, but I am sure it would work.

1. dd a fresh image to your 8 GB card.

2. from command line on a linux box or on the pi (maybe the fresh one on the 8 GB card), do

cd PATH-TO/boot
tar cf PATH--TO-WHERE-TAR-FILE-GOES/boot-partition.tar *

3. from command line, as above, from a linux box (or your pi running the temporary 8 GB image). If mounted as /dev/sda2, then you would "cd /dev/sda2" instead of cd PATH-TO/

cd PATH-TO/
tar cf PATH--TO-WHERE-TAR-FILE-GOES/rootfs-partition.tar

Then you have 2 tar files, one for each partition.

4. use gparted or some other similar tool to expand the second partition on the 8 GB card to full capacity.

5. Again, still on your main Linux box:

cd PATH-TO-BOOT-PARTITION-ON-8GB-CARD/
tar xf PATH-TO-FIRST-TARFILE/boot-partition.tar

cd PATH-TO-SECOND-PARTITION-ON-8GB-CARD/
tar xf PATH-TO-SECOND-TARFILE/rootfs-partition.tar

6. I think that is it, but requires doing it all on a Linux box, as Windoze/MAC cannot read the second partition.
OK - If I understand you correctly you are (I think) actually suggesting tarring up the ENTIRE contents of those two partitions from the original card, and then overwriting the default contents of those partitions on a newly imaged card with the untarred contents of those two archives? That's why I don't need to try to remember what to include in the tar - because I'm including everything-?

I'll give that a try.

sdjf
Posts: 1395
Joined: Fri Mar 16, 2012 5:20 am
Location: California
Contact: Website

Re: Transfer working image to smaller card?

Wed Jan 23, 2013 2:33 am

Yup, you got the idea. If you wanted to mess around with fdisk and mkfs to create the right size partitions and file system types, you would not need to even bother dd'ing the fresh image to your 8 GB card, but that is harder to learn to do. I keep trying to add partitions to a card and keep messing that up, which is why I did not suggest it. But dd'ing an image is pretty fast, a lot easier than learning two more commands you will not need to use much if at all, otherwise.
FORUM TIP: To view someone's posting history, sign in, click on their user name, then on "Search User's Posts." || Running ArchLinuxArm on Model 2B and 512MB Model B

1HzCoder
Posts: 31
Joined: Thu Jul 12, 2012 1:15 am
Location: Lower Alabama

Re: Transfer working image to smaller card?

Thu Jan 24, 2013 11:56 pm

Check out this thread http://www.raspberrypi.org/phpBB3/viewt ... 29&t=21342,
describes a small script for doing it all atomatically, rpi-clone, worked like a champ to backup/clone
my 16Gb to a 4Gb.
TomJ
Einstein once said you don't really understand anything until you can explain it to your Grandmother

warwound
Posts: 5
Joined: Thu Jan 03, 2013 7:30 pm
Location: Norfolk, UK

Re: Transfer working image to smaller card?

Fri Jan 25, 2013 12:13 am

Well i've just done something very similar to this - i had to copy a working ubuntu install from a 250GB hard drive to a 10GB (virtual) hard drive. The ubuntu was using about 9.3GBs of the 250GB hard drive, i wanted to copy it to a virtual hard drive so i could use it in a virtual machine.

So i'm not talking about SD cards but the technique should still apply and be successful.

First i did what i suggested previously - shrunk the parition to be copied using Gparted.
(I booted using Parted Magic from a USB stick).

So i now had a 9.5GB parition i wanted to copy to a 10GB virtual hard drive.

I used CloneZilla - not expecting it to work!
BUT within the CloneZilla options it asks 'do you want to do a disk to disk clone or a parition to partition clone?' - i choose partition to partition and let it clone the 9.5GB hard drive partition to the 10GB virtual hard drive.

Next i ran GrubDoctor - to install Grub on the virtual drive - and rebooted.
Moved the virtual drive to a virtual machine and it booted perfectly.

So to copy a working Pi SD card to a smaller SD card i'd say boot your PC with Parted Magic, use Gparted to shrink the working SD card partition to a smaller size and then use CloneZilla to do a partition to partition copy.
(No need for GrubDoctor of course).

I also found out that you can copy/paste using GParted. Simply right click a partition and there's the options to copy and paste. I didn't test this but presumably (lol!) you could copy n paste instead of using CloneZilla.

Martin.

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Fri Jan 25, 2013 1:09 am

warwound wrote:W

So to copy a working Pi SD card to a smaller SD card i'd say boot your PC with Parted Magic, use Gparted to shrink the working SD card partition to a smaller size and then use CloneZilla to do a partition to partition copy.
(No need for GrubDoctor of course).

Martin.
I did exactly that, but sadly the card so constructed would not boot. I can only assume there must be something else on a legitimate Pi SD card image besides those two partitions - something which is the equivalent of a HDD boot sector or fulfils the same function.

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Fri Jan 25, 2013 1:12 am

1HzCoder wrote:Check out this thread http://www.raspberrypi.org/phpBB3/viewt ... 29&t=21342,
describes a small script for doing it all atomatically, rpi-clone, worked like a champ to backup/clone
my 16Gb to a 4Gb.
TomJ
Thanks for that, I've bookmarked that thread and will take a proper look over the weekend.

SiriusHardware
Posts: 520
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: Transfer working image to smaller card?

Tue Jan 29, 2013 12:06 am

1HzCoder wrote:Check out this thread http://www.raspberrypi.org/phpBB3/viewt ... 29&t=21342,
describes a small script for doing it all atomatically, rpi-clone, worked like a champ to backup/clone
my 16Gb to a 4Gb.
TomJ
I tried this one first as it seemed like the path of least resistance and it worked second time: It needs 'rsync' to be installed first, a fact I would have divined had I just read far enough down that thread.

The script successfully cloned my working tweaked Raspbian from its original home on a 32GB card to an 8GB card without any problems - each now runs as well as the other with my various tweaks and additions intact.

Kudos to the original author of that script, and my thanks to 1HzCoder for bringing it to my attention.

Thanks to everyone else for suggesting all the equally interesting, alternative ways of solving the problem. This will all be very valuable to anyone else who comes looking for ways to do the same thing at some point in the future.

Return to “Beginners”