User avatar
RaTTuS
Posts: 10574
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: SD image too big for another SD Card

Tue Mar 05, 2013 7:25 am

azeam wrote:
simplesi wrote:Now I come to think about it, can we not simply plug a £1 shop USB SD card reader into an RPi and get it to do the DD ing itself?

Simon
"dd:ing" an image of a running os is not a good idea at all, better to unmount the sd card and do it externally.
switch to single user and you can do it
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

SirLagz
Posts: 1705
Joined: Mon Feb 20, 2012 8:53 am
Location: Perth, Australia
Contact: Website

Re: SD image too big for another SD Card

Tue Mar 05, 2013 7:26 am

Colly wrote:
pluggy wrote:You're aware that SD card sizes vary anyway ?. They can't manufacture them all perfect so they size the cards afterwards and block out any bits that don't work. Some will have more bits blocked out than others so the size varies slightly. Theres nothing wrong with the resize option in raspi-config its just a shell script if you care to look and you can work out what it does for yourself. Your kernel panic problems are almost certainly corrupted card problems. I reckon many of your problems would disappear if you lost the Class 10 fixation.

The Pi has always exhibited more tendency to corrupt higher class cards. Lower class cards will often read as fast or faster than higher class cards and reading is what matters. I have 5 Pis and a bag full of SD cards, for use in them. All class 4, never had a corrupted card, and I often do things like depriving them of power without shutting them down first. Its fairly easy to resize a partition before you make an image of it and then truncate the .img file so it doesn't give an error when you restore it to a smaller card.
:) yup - NOW I'm aware they vary. I should have realized before, but it was brought home by the mischance that the one good web server image I made, got expanded to the largest of all my SDcards so now I can't restore its image to another card. I'll just have to create a fresh one all over again and when I made the first one, my build responded to some of the steps a little differently than the example in the Tinkernut video on how to do it - I had to work around those differences, but got it working - and don't quite remember what I did for the workarounds (more the fool because I didn't take notes while I was doing it - I will THIS time though.)
As for corrupted cards - I've never had any corrupted if they ever booted up once. The kernel panics were all resulting from the fact that the image restores never had enough bytes to restore to in the slightly smaller cards, so their files systems simply failed to mount when they hit the "bad block" (exceeded the physical space on the too-small sdcards.) ALL the images I've built on this brand of class 10 cards has worked through many power ups and shutdowns. I usually never unplug them, but on the rare instances when I have, I simply made sure the system wasn't actually doing anything when I did so. My RaspBMC images work fine. My Wheezy images work fine. And I'm very careful to either tell RaspBMC to "Power off system" or else I login with SSH (on the headless RPi's) to issue: "sudo shutdown -h -P -t 3 now" which seems to safely shut everything down and power off the RPi. I wait then until the little activity LED goes out before unplugging. I never overclock beyond the medium 800Mhz, though someday I may risk toasting one of my RPi's if I ever buy some heatsinks to put on it first. My RiData SDHC "Lightning Series" Class 10 cards were not only some of the least expensive I could find online, they've all worked quite reliably. I did try a couple of class 4 16GB sd cards I got at Big Lots for a good price, but they seemed noticeably slower, though I haven't done any actual benchmarking tests. So far, I have 7 RPi's - 3 with RaspBMC being used on monitors (7", 19", and 50"), one (#4) (my original and only revision 1 board) being dedicated as a headless web server, one (#5) given to my wife to build a server on, one (#6) on a Pi Plate to fiddle with the RTC and 16x2 LCD, one (#7) in a clear box case, used for showing at the PC club meetings. I dedicated the revision 1 board as my web server because it will never need the MPG2/WVC1 decode keys.
*
I think from here on, when I go to build a new image, I will NOT use the expand2fs option until AFTER I create the first backup image of the working build. I've also discovered how fast you can eat up space on even terabyte drives by making backups of 16gb images that are mostly empty diskspace! Silly, now that I think of it. I will create my working images, configured the way I like them with the added stuff I want, and then back those up while they are working correctly, but still small, so the images won't waste space on my main desktop. I've been cutting adhesive mailing labels into 1/4" by 3/4" strips to label my SDcards so I don't get them mixed up. I stick the labels on the back of the sdcard (same side as the contacts, but opposite end) and I can easily read them while they're inserted in the RPi's even in the VESA mounts. I've also numbered each of my RPi's on the top of the Ethernet socket with a permanent marker, so I know which RPi gets which mpeg keys, in case I eventually get over the 6 or 7 key limit. So far, I've only bought 4 sets of keys, so I just have them all strung together on each card with RaspBMC.

I love my Raspberry Pi's and I love Linux, where the only true limits are the imagination of their users. :)
That's a nice setup you've gotten there haha
Though you can resize the partitions in the images themselves, saves you the effort of having to re-flash, re-install, then reconfigure.
Have a look on the forums or on my blog for the guide :)
My Blog - http://www.sirlagz.net
Visit my blog for Tips, Tricks, Guides and More !
WiFi Issues ? Have a look at this post ! http://www.raspberrypi.org/phpBB3/viewtopic.php?f=28&t=44044

Schorschi
Posts: 239
Joined: Thu Nov 22, 2012 9:38 pm

Re: SD image too big for another SD Card

Sun Jun 30, 2013 8:31 pm

RomanG, I am interested! I guess I am surprised there is not a way to repair the MBR from Linux? Maybe just not in this case? Windows OS has had such a tool for ages?

RomanG
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada

Re: SD image too big for another SD Card

Mon Jul 01, 2013 5:02 am

Schorschi wrote:RomanG, I am interested! I guess I am surprised there is not a way to repair the MBR from Linux? Maybe just not in this case? Windows OS has had such a tool for ages?
Schorschi,
I am not sure if this can be classified as a repair, it is more a modification rather than repair.

OK, here is more detailed description of what I did:

Checking original SD card, we got:

Code: Select all

sudo fdisk -l /dev/loop1

Disk /dev/loop1: 3965 MB, 3965190144 bytes
255 heads, 63 sectors/track, 482 cylinders, total 7744512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9

      Device Boot      Start         End      Blocks   Id  System
/dev/loop1p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop1p2          122880     7744511     3810816   83  Linux
Using dd command, MBR can be extracted from the card/image (see previous post for details).

According to Wikipedia article linked previously, 2nd partition entry in MBR starts at +1CEh address and is 16 bytes long:

Code: Select all

..........
000001c0	03 00 0C A5 1E 07 00 20 00 00 00 C0 01 00 00 00		...¥... ...À....
000001d0	C1 80 83 03 10 AF 00 E0 01 00 00 4C 74 00 00 00		Á€ƒ..¯.à...Lt...
1st byte indicates partition status (00h)
2nd to 4th byte indicates CHS (cylinder, head, sector) address of the first absolute sector in the partition (00h C1h 80h)
5th byte indicates partition type (83h)

next part is the important one for this modification:
6th to 8th byte indicates CHS (cylinder, head, sector) address of the last absolute sector in the partition (03 10 AF)
6th - 03h (03 dec) is the head
7th - 10h indicates sector in bits 5–0; bits 7–6 are high bits of cylinder, 10h is 00010000b and in bits 5-0 we have 10000b which is 16 dec
8th - AFh indicates low bits of cylinder, AF7h is 10101111b, adding two high bits from previous address we have 0010101111b which is 175 dec

so the last sector is at cylinder 175, sector 16 head 3

LBA of the first absolute sector is in next 4 bytes, 9th to 12th
and number of sectors in partition is in the last 4 bytes,
13th to 16th, these are in little endian notation, so we have 00744C00h, which is 7621632d sector.

fdisk above shown us that second partition starts at sector 122880 and ends at sector 7744511, which is 7744511-122880=7621631 sectors.

But we know that the target SD card has only 7741440 sectors and that the second partition starts at sector 122880 so we will need to adjust MBR record accordingly,
7741440-122880=7618560 sectors which is 744000h, so the the last 4 bytes of the 2nd partition record will be 00 40 74 00 (little endian, least significant bit first).

Now we know the number of sectors is 7741440 and last sector of the 2nd partition is 7741439 which is also LBA of the last sector and we just need to convert it to CHS notation.
This Wikipedia article describes how it is done http://en.wikipedia.org/wiki/Logical_block_addressing
We have 255 heads and 63 sectors/track, so:
LBA=7741439
HPC=255
SPT=63
C=481 => 0111100001b, high two bits 01b=01h, low eight bits 11100001b=E1h
H=224 => 11100000b => E0h
S=63 => 111111b => 3Fh
6th byte will be E0h
7th byte will be 01111111b => 7Fh
8th byte will be E1h

Now we can reconstruct new row at address 000001d0 of MBR record:
000001d0 C1 80 83 E0 7F E1 00 E0 01 00 00 40 74 00 00 00

Using your favorite hex editor, modify the MBR and then write it back to the SD card, using dd command (as described in previous post).

Let me know if something is not clear

Roman

spl147
Posts: 2
Joined: Thu Sep 12, 2013 2:24 pm

Re: SD image too big for another SD Card

Thu Sep 12, 2013 2:28 pm

RomanG wrote:
Schorschi wrote:RomanG, I am interested! I guess I am surprised there is not a way to repair the MBR from Linux? Maybe just not in this case? Windows OS has had such a tool for ages?
Schorschi,
I am not sure if this can be classified as a repair, it is more a modification rather than repair.

OK, here is more detailed description of what I did:

Checking original SD card, we got:

Code: Select all

sudo fdisk -l /dev/loop1

Disk /dev/loop1: 3965 MB, 3965190144 bytes
255 heads, 63 sectors/track, 482 cylinders, total 7744512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9

      Device Boot      Start         End      Blocks   Id  System
/dev/loop1p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop1p2          122880     7744511     3810816   83  Linux
Using dd command, MBR can be extracted from the card/image (see previous post for details).

According to Wikipedia article linked previously, 2nd partition entry in MBR starts at +1CEh address and is 16 bytes long:

Code: Select all

..........
000001c0	03 00 0C A5 1E 07 00 20 00 00 00 C0 01 00 00 00		...¥... ...À....
000001d0	C1 80 83 03 10 AF 00 E0 01 00 00 4C 74 00 00 00		Á€ƒ..¯.à...Lt...
1st byte indicates partition status (00h)
2nd to 4th byte indicates CHS (cylinder, head, sector) address of the first absolute sector in the partition (00h C1h 80h)
5th byte indicates partition type (83h)

next part is the important one for this modification:
6th to 8th byte indicates CHS (cylinder, head, sector) address of the last absolute sector in the partition (03 10 AF)
6th - 03h (03 dec) is the head
7th - 10h indicates sector in bits 5–0; bits 7–6 are high bits of cylinder, 10h is 00010000b and in bits 5-0 we have 10000b which is 16 dec
8th - AFh indicates low bits of cylinder, AF7h is 10101111b, adding two high bits from previous address we have 0010101111b which is 175 dec

so the last sector is at cylinder 175, sector 16 head 3

LBA of the first absolute sector is in next 4 bytes, 9th to 12th
and number of sectors in partition is in the last 4 bytes,
13th to 16th, these are in little endian notation, so we have 00744C00h, which is 7621632d sector.

fdisk above shown us that second partition starts at sector 122880 and ends at sector 7744511, which is 7744511-122880=7621631 sectors.

But we know that the target SD card has only 7741440 sectors and that the second partition starts at sector 122880 so we will need to adjust MBR record accordingly,
7741440-122880=7618560 sectors which is 744000h, so the the last 4 bytes of the 2nd partition record will be 00 40 74 00 (little endian, least significant bit first).

Now we know the number of sectors is 7741440 and last sector of the 2nd partition is 7741439 which is also LBA of the last sector and we just need to convert it to CHS notation.
This Wikipedia article describes how it is done http://en.wikipedia.org/wiki/Logical_block_addressing
We have 255 heads and 63 sectors/track, so:
LBA=7741439
HPC=255
SPT=63
C=481 => 0111100001b, high two bits 01b=01h, low eight bits 11100001b=E1h
H=224 => 11100000b => E0h
S=63 => 111111b => 3Fh
6th byte will be E0h
7th byte will be 01111111b => 7Fh
8th byte will be E1h

Now we can reconstruct new row at address 000001d0 of MBR record:
000001d0 C1 80 83 E0 7F E1 00 E0 01 00 00 40 74 00 00 00

Using your favorite hex editor, modify the MBR and then write it back to the SD card, using dd command (as described in previous post).

Let me know if something is not clear

Roman
i got all the way to this part, editing the MBR just doesnt make sence to me, i extracted the MBR from the card after writing it back, i cannot find the address you are talking about!
nor can i figure out how you got these values....

RomanG
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada

Re: SD image too big for another SD Card

Thu Sep 12, 2013 10:32 pm

Well, let's have a look together.
Can you post what you have?
Your MBR?

Thanks,
Roman

JonesThePi
Posts: 21
Joined: Tue Oct 23, 2012 1:08 pm

Re: SD image too big for another SD Card

Thu Sep 19, 2013 5:41 pm

Hi Roman,

Thanks for this guide, it has saved much frustration after buying 2 new SD cards only to discover they were smaller than the originals. Who would have thought I would be that lucky to use the larger ones first :lol:

I have managed to resize my SD image and now have a working backup but have a question about repairing the MBR.
How did you get from S=3Fh to the 7th byte being 7Fh? Not quite sure where 01111111b comes from.

Thanks

Bob

RomanG
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada

Re: SD image too big for another SD Card

Thu Sep 19, 2013 7:06 pm

I know, it is a bit confusing if you do not do it every day ;)

6th byte -> head
7th byte -> bits 0 to 5 of the 7th byte hold the sector and bits 7 and 8 are high bits of the cylinder
8th byte -> low bits of the cylinder

So...after doing translation to CHS notation we have 481 cylinders which is 0111100001b, now this number is a 10 bit number and as such needs to be split between two bytes, as mentioned above two high bits are in the bits 6 and 7 of the 7th byte and remaining 8 low bits are in the 8th byte.
We take two high bits from 0111100001b, which is 01b and remainder is 8bit number 11100001b, which is E1h in hexadecimal format,. That's our 8th byte.
From CHS notation, we also have 63 sectors, which is, when expressed in binary 111111b, or 3Fh in hexadecimal. But, that's only 5 low bits of the 7th byte.
Then we compose the whole 7th byte taking 2 high bits 01b (leftower from number 481) and 6 low bits 111111b, we will get 01111111b, which is 7Fh in hexadecimal, that's our whole 7th byte.

Still confusing?

Roman

JonesThePi
Posts: 21
Joined: Tue Oct 23, 2012 1:08 pm

Re: SD image too big for another SD Card

Thu Sep 19, 2013 7:35 pm

Hi Roman,

Not confused at all now, a very clear explanation.
So glad I asked though, I would never have got there in a month of Sundays :D

Bob

RomanG
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada

Re: SD image too big for another SD Card

Thu Sep 19, 2013 7:48 pm

Great!
When I modified mine, I did everything on the paper first to wrap my head around all the conversions.
But since I was working with a backup of the image, there was no real pressure, I knew if I mess up something I can start over again.

Roman

Remco
Posts: 5
Joined: Tue Oct 01, 2013 3:59 pm

Re: SD image too big for another SD Card

Thu Oct 31, 2013 12:19 pm

I read this thread diagonally, and here is how I solved this issue.

Remco

User avatar
pluggy
Posts: 3635
Joined: Thu May 31, 2012 3:52 pm
Location: Barnoldswick, Lancashire,UK
Contact: Website

Re: SD image too big for another SD Card

Thu Oct 31, 2013 1:29 pm

Probably the most involved of the dozen or so ways of doing it I've seen.

Shrink the main partition, before you start, dd it from the old card to an image, dd it from the image to the new card, Ignore the error it gives. It doesn't matter then because what matters has been transferred in its entirety, the error occurs in the 'null' after the partitions have been written. If you really can't live with the error, use the linux truncate command to shrink the image size slightly. the shrinking the partition using gparted/parted/whatever before you start bit is crucial.

Or use rsync...
Don't judge Linux by the Pi.......
I must not tread on too many sacred cows......

RomanG
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada

Re: SD image too big for another SD Card

Thu Oct 31, 2013 5:06 pm

Not sure which method you are commenting on.
pluggy wrote:Probably the most involved of the dozen or so ways of doing it I've seen.

Shrink the main partition, before you start, dd it from the old card to an image...
Yes, that would be the simplest way of doing it, but that would assume that you have your old card with good data/image on it.
But if your card failed and/or is corrupted, then you can't do that as your new shrank partition will be garbage.

Procedure above describes a situation when you are working with a backup image that does not fit on to your new card and there is no old card that you can start with.

Roman

SiriusHardware
Posts: 510
Joined: Thu Aug 02, 2012 9:09 pm
Location: UK

Re: SD image too big for another SD Card

Thu Oct 31, 2013 7:18 pm

This subject has been covered before in this thread:

http://www.raspberrypi.org/phpBB3/viewt ... 29&t=21342

In particular, look for the 'rpi-clone' script discussed in that thread - yes, it images a running system to a smaller card plugged into a card reader on the Pi, as long as the destination card has enough room for all the significant information that the larger card contains. No, I haven't had any problems with corruption so far.

In my experience the card copied TO works immediately, just like the card it was copied FROM.

I do agree that it would be helpful if the 'expand to maximum' feature in Raspi-config stopped short of filling the card entirely so that an image from any given (say) 4GB card would be likely to fit on any other 4GB card.

User avatar
Offcenter
Posts: 183
Joined: Wed Jul 31, 2013 4:57 pm
Location: Northwestern New Jersey USA

Re: SD image too big for another SD Card

Thu Oct 31, 2013 8:21 pm

This fellow has a script that expands the file system but leaves
a little bit of "wiggle room".
http://rpi.tnet.com/project/scripts/rpi-wiggle
I used it to expand my file system. I haven't tried
to make a backup copy from it yet though.
George in New Jersey.
(learning a little bit every day.)
(and darned confused most of the time!)

skipdup
Posts: 1
Joined: Fri Jan 17, 2014 10:22 pm
Location: San Antonio, TX

Re: SD image too big for another SD Card

Fri Jan 17, 2014 10:30 pm

This is surely a silly question (I know nothing about any of this and just got my first Pi)...
I was hoping there was an "anti" expand_rootfs (like in the config) out there, or some automatic way of shrinking the root back down and not using all the free space on the 8GB card. I assume there's a reason I haven't found this?

emilio507
Posts: 1
Joined: Fri Jul 17, 2015 5:30 pm

Re: SD image too big for another SD Card

Fri Jul 17, 2015 5:37 pm

RomanG wrote:I run in to similar problem, I had a backup image with older kernel and needed to restore it so that I could troubleshoot ALSA issues.
The problem was that the other 4GB SD cards that I have were slightly smaller than the one from which the image backup was made.

The original card had size of 3965MB and spare cards that I have were 3963MB or 3904MB respectively.
So clearly, writing the backup image to either one of these cards would result in a corrupted file system.

Here is how I solved this problem.

First, create a copy of your backup image to play with
In my case:

Code: Select all

cp pi_30112012.img test.img
Searching google I found that image files can be mounted as a block devices using losetup command:

Code: Select all

sudo losetup -f --show test.img
/dev/loop1
Let's see what we got:

Code: Select all

sudo fdisk -l /dev/loop1

Disk /dev/loop1: 3965 MB, 3965190144 bytes
255 heads, 63 sectors/track, 482 cylinders, total 7744512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9

      Device Boot      Start         End      Blocks   Id  System
/dev/loop1p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/loop1p2          122880     7744511     3810816   83  Linux
As you can see, this image is 3965MB and 7744512 sectors (512b x 7744512 = 3965190144b)
Also geometry of the card is important to note, 255 heads and 63 sectors.
The last sector of the last partition is Total sectors - 1, so there is no spare room left on the card.

Now, let's have a look at the target SD card

Code: Select all

sudo fdisk -l /dev/mmcblk0

Disk /dev/mmcblk0: 3963 MB, 3963617280 bytes
128 heads, 63 sectors/track, 960 cylinders, total 7741440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192     7741439     3866624    6  FAT16
The target SD card is 3963MB and 7741440 sectors, so it is smaller than the image by 3072 sectors or 1572864b. Also note the geometry of the card showing 128 heads and 63 sector. For some reason this is incorrect as this card has 255 heads and 63 sectors. For some reason fdisk does not show this always correctly. At another occasion, it showed this card with 4 heads and 16 sectors. Not sure why...

We need to shrink second partition of the image, otherwise if we attempt to write image to the card, it will result in a corrupted file system
let's mount the second partition of the image as a loopback device so that we can resize it
image starts at sector 122880, which is 512*122880=62914560b

Now that we have the offset for mounting the second partition of the image,
we can now use with losetup’s -o argument to set an offset

Code: Select all

sudo losetup -f --show -o 62914560 test.img
/dev/loop2
scan for errors before resizing the image

Code: Select all

sudo e2fsck -f /dev/loop2

e2fsck 1.42 (29-Nov-2011)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop2: 80300/245760 files (0.2% non-contiguous), 604445/952704 blocks
using resize2fs we can shrink the partition, using -p option to show progress and specifing new size which is
slightly smaller than the new SD card

-p Prints out a percentage completion bars for each resize2fs operation, so that the user can keep track of what the program is doing.

size parameter may be suffixed by one of the following the units designators: 's', 'K', 'M', or 'G', for 512 byte sectors, kilobytes, megabytes, or gigabytes, respectively.

We need to scale dovwn the image by 3072 sectors, lets go with 4096 sectors just to be sure. Image size is End - Start = 7744512 - 122880 = 7621632 sectors, substracting 4096 sectors, the new size is 7621632 - 4096 = 7617536 sectors

Code: Select all

sudo resize2fs -p /dev/loop2 7617536s

Resizing the filesystem on /dev/loop2 to 952192 (4k) blocks.
The filesystem on /dev/loop2 is now 952192 blocks long.
Now that the filesystem was resized, loopback devices are no needed anymore:

Code: Select all

sudo losetup -d /dev/loop1 /dev/loop2
To write the modified image to the SD card, use command:
(this may take a while, depending on the speed of your card, in my case it took more than 30 minutes)

Code: Select all

sudo dd if=test.img of=/dev/mmcblk0

dd: writing to `/dev/mmcblk0': No space left on device
7741441+0 records in
7741440+0 records out
3963617280 bytes (4.0 GB) copied, 1908.13 s, 2.1 MB/s
As you can see, there is an error message 'No space left on the device', but because we re-sized the file system, card will be useable and Raspberry Pi will boot up without any issues.
However, if you attempt to look at the partition(s) on the card with a tool such us parted or gparted, it will end up with an error "...partition extends beyond end of device..."
To correct this, we need to modify MBR (Master Boot Record) of the SD card.
First, we will extract the first sector from the card which contains MBR

Code: Select all

dd if=/dev/mmcblk0 of=sd.mbr bs=512 count=1
This will create a file sd.mbr which we will need to edit with a hex editor (I did this on my laptop using Ubuntu 12.04LTS Live CD and tried GHex and Bless Hex editors, I liked Bless Hex better, but that's just my preference).
To play with the MBR in the hex editor, you need to know what values at which address need to be modified, Wikipedia has very good reference material about MBR.
http://en.wikipedia.org/wiki/Master_boot_record
From there you can see that Partition entry #1 starts at address 0x1BE and Partition entry #2 starts at address 0x1CE.
Layout of the partition entry is described here http://en.wikipedia.org/wiki/Master_boot_record#PTE
Values for the CHS address of last absolute sector in partition, LBA of first absolute sector in the partition and number of sectors in partition need to be calculated and changed.
I can explain this in more detail if someone is interested.

Once MBR was modified, it can be written back to the sd card

Code: Select all

dd if=sd.mbr of=/dev/mmcblk0 bs=512 count=1
running:

Code: Select all

sudo fdisk -l /dev/mmcblk0

Disk /dev/mmcblk0: 3963 MB, 3963617280 bytes
4 heads, 16 sectors/track, 120960 cylinders, total 7741440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000714e9

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/mmcblk0p2          122880     7741439     3809280   83  Linux
We can see that the end sector of last partition is less than total number of sectors.
If we run now parted or gparted, the card partitions will be recognized and can be further manipulated by using these tools.

I hope this helps to someone

Cheers,
Roman
I was wondering if you could send me a link or some more information on editing the MBR. The method you posted here works perfectly but I need to figure out the MBR portion. Thanks in advance.

Emilio

W. H. Heydt
Posts: 13345
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: SD image too big for another SD Card

Fri Jul 17, 2015 9:13 pm

skipdup wrote:This is surely a silly question (I know nothing about any of this and just got my first Pi)...
I was hoping there was an "anti" expand_rootfs (like in the config) out there, or some automatic way of shrinking the root back down and not using all the free space on the 8GB card. I assume there's a reason I haven't found this?
The problem is not so much one of shrinking the file system. That can be easily done with gparted. The problem is that the typical means of making a backup copy of an SD card creates a file that is all the blocks of the the card, and if you restore that to another card that is a bit (or a lot) smaller, it will throw fits, even if the filesystem will be okay and not be hurt by missing a few empty blocks.

What I think is needed is a slightly smarter version of SDImager coupled with restricting the raspi-config partition expansion to around 98% of the card, rather than all of it (or, most likely better, an option in raspi-config to select some unallocated space and say how much).

thodoris85
Posts: 1
Joined: Sun Feb 21, 2016 9:04 pm

Re: SD image too big for another SD Card

Mon Feb 22, 2016 6:35 am

RomanG's reply was what worked for me (even though I had a working SD card in hand).

I just read the first page of the replies (don't know why but I didn't notice the second page). So, of course I stuck on the last part (editing the MBR). However after a bit of internet search I came across this:
http://blog.creativeitp.com/posts-and-a ... -workshop/.
Which made things clear (specially on noting that "bytes stored in little-endian")
Also used an online hex-decimal-hex converter
(like this one http://www.binaryhexconverter.com/decim ... -converter).

I just registered in order to say a big thank you to Roman for his step by step tut, and the I realized that there was a second page with replies, and Roman explained how to edit the MBR! :)

Thanks again!
Theo

P.S For some reason, when I use my RPi 2 to build an image I cannot use gparted to make the image file any smaller. With the RPi B, I never had any issues

treii28
Posts: 97
Joined: Fri May 10, 2013 4:52 pm

Re: SD image too big for another SD Card

Sun Apr 16, 2017 5:26 am

I know this is an old thread and a few people commented on the method but I didn't see anyone spelling it out. If anyone runs into the need to do this, check this post out: https://softwarebakery.com/shrinking-images-on-linux

There are also some updated methods of using losetup for tweaking the image: http://stackoverflow.com/questions/1419 ... ns-a-parti

i.e. you can now mount the entire dd image of your card:

Code: Select all

losetup --show -f -P myimage.img

fruitoftheloom
Posts: 24084
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: SD image too big for another SD Card

Sun Apr 16, 2017 5:47 am

treii28 wrote:I know this is an old thread and a few people commented on the method but I didn't see anyone spelling it out. If anyone runs into the need to do this, check this post out: https://softwarebakery.com/shrinking-images-on-linux

There are also some updated methods of using losetup for tweaking the image: http://stackoverflow.com/questions/1419 ... ns-a-parti

i.e. you can now mount the entire dd image of your card:

Code: Select all

losetup --show -f -P myimage.img
PiClone which may be of interest as it backs up to a SD Card: https://github.com/raspberrypi-ui/piclo ... ter/README
Thinking outside the box is better than burying your head in the sand...

spannernick
Posts: 15
Joined: Sun Oct 12, 2014 11:50 am

Re: SD image too big for another SD Card

Thu May 24, 2018 8:18 pm

I was wondering can you copy one SD card to another,so they are both 32 GB but one is 29.7 GB and the other one to copy to is 28.7 GB..? that way you just need to copy the img file to the old 29.7 SD card then copy from the 29.7 to 28.7 GB SD card,like copying hard drive to hard drive..??

They could easily sort this out by fixing Win32 disk imager and making it copy to all SD cards no mater what size they are.

User avatar
B.Goode
Posts: 10568
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: SD image too big for another SD Card

Thu May 24, 2018 8:26 pm

spannernick wrote:
Thu May 24, 2018 8:18 pm
I was wondering can you copy one SD card to another,so they are both 32 GB but one is 29.7 GB and the other one to copy to is 28.7 GB..? that way you just need to copy the img file to the old 29.7 SD card then copy from the 29.7 to 28.7 GB SD card,like copying hard drive to hard drive..??
No. Can't be done. Not with a simple 'dd' copy. The capacity of the destination always has to be the same as or larger than the source, even if their nominal size (what it says on the packing) is the same.

But the two posts prior to yours in this thread give two alternatives....

LTolledo
Posts: 3922
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: SD image too big for another SD Card

Thu May 24, 2018 9:41 pm

Ever since encountering this problem with Win32DiskImager I've used Raspbian Stretch's SD Card Copier.

Copied from 32GB to 32GB, 32GB to 16GB, 32GB to 8GB, 32GB to 128GB, 16GB to 1TB, SD card to USB pendrive, USB pendrive to USB HDD, without any problems.

Used rsync command as well but SD Card Copier made it all easier
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

Return to “Beginners”