Use dd to write into extended partition

[kusti8]

The title pretty much says it all, although I'll elaborate a little. So I want to write the Raspbian image alongside an already present working system (which is loaded into RAM and has two partition, the boot and another for extensions - its piCore). I want to write that image into an extended partition that I created at /dev/mmcblk0p3
Maybe I don't understand partitions well enough, but I've tried writing to mmcblk0p3, but apparently that's only the "container" so I made a partition, wrote into that and it complained about an invalid partition scheme and I've also tried the skip option which didn't work.

So how do I write an image file with two partitions to the end of two already present partitions?

Thanks!

[rpdom]

How about you post your partition table (as shown by fdisk) and how you'd like it to look and we may be able to advise you.

[kusti8]

Whoops, forgot about that.

Code: Select all

tc@box:~$ fdisk -l

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders
Units = cylinders of 64 * 512 = 32768 bytes

        Device Boot      Start         End      Blocks  Id System
/dev/mmcblk0p1             129         768       20480   c Win95 FAT32 (LBA)
Partition 1 does not end on cylinder boundary
/dev/mmcblk0p2             769        1056        9216  83 Linux
Partition 2 does not end on cylinder boundary

[rpdom]

Tricky, but can be done.

I'm tired and would probably make mistakes right now, otherwise I'd give you step-by-step instructions, but it's not too hard to do.

Getting the Pi to boot from the new partitions may be a different matter.

Is there any reason you're not using NOOBS or Berryboot or something else to handle more than one OS on a card?

[kusti8]

Tricky, but can be done.

I'm tired and would probably make mistakes right now, otherwise I'd give you step-by-step instructions, but it's not too hard to do.

Getting the Pi to boot from the new partitions may be a different matter.

Is there any reason you're not using NOOBS or Berryboot or something else to handle more than one OS on a card?

Ah, you're in the UK. What time is it there? Its only 2:30 here.

I was sifting through the NOOBS source and found some /sys file that it writes to and that sets a flag in the chip for that to the be boot partition.

I'm planning to make something like NOOBS, but with some more features that require it to be built from scratch, where I want to use .img files instead of .tar files and I'm using Tiny Core as the base to try and make it as small as possible.

[rpdom]

Ah, you're in the UK. What time is it there? Its only 2:30 here.

It's only 8pm now, but I've had a tiring day, sorry

I was sifting through the NOOBS source and found some /sys file that it writes to and that sets a flag in the chip for that to the be boot partition.

Good, that sounds like it will work just fine

I'm planning to make something like NOOBS, but with some more features that require it to be built from scratch, where I want to use .img files instead of .tar files and I'm using Tiny Core as the base to try and make it as small as possible.

I think what you will need to do is create partitions at suitable sizes, then extract the data from the image files into those partitions. It is possible, but not easy, to copy the image files directly onto the card if not planning to overwrite the whole card.

[kusti8]

I'm planning to make something like NOOBS, but with some more features that require it to be built from scratch, where I want to use .img files instead of .tar files and I'm using Tiny Core as the base to try and make it as small as possible.

I think what you will need to do is create partitions at suitable sizes, then extract the data from the image files into those partitions. It is possible, but not easy, to copy the image files directly onto the card if not planning to overwrite the whole card.

The problem is that Tiny Core is run from RAM, so I can't download the entire compressed iamge file (990MB) and mount it. My current solution is to pipe wget to gunzip and finally to dd, which works for overwriting the card for one OS, but for multiboot, I would need to have the Tiny Core to act as the switcher and somehow write the image file after the Tiny Core partitions.

I won't be writing over the entire card, just "appending" the Raspbian partitions to the end of the already present partitions. I'm not sure if dd can even do this?

[rpdom]

The problem is that Tiny Core is run from RAM, so I can't download the entire compressed iamge file (990MB) and mount it. My current solution is to pipe wget to gunzip and finally to dd, which works for overwriting the card for one OS, but for multiboot, I would need to have the Tiny Core to act as the switcher and somehow write the image file after the Tiny Core partitions.

I won't be writing over the entire card, just "appending" the Raspbian partitions to the end of the already present partitions. I'm not sure if dd can even do this?

I understand. I will think about ways to do this and if I find anything I will update.

[rpdom]

My initial thought is to create one large partition, install the image file into it, then delete the partition and create two new ones at the correct locations to use the contents of the file. There will be a small amount of space wasted, but not much.

[kusti8]

My initial thought is to create one large partition, install the image file into it, then delete the partition and create two new ones at the correct locations to use the contents of the file. There will be a small amount of space wasted, but not much.

Interesting. I'm going to try that.

[rpdom]

Let me know how you get on

[kusti8]

I've written the image to /dev/mmcblk0p5 and have deleted it, but now I'm having trouble with the sizes of the partitions. I can't seem to get the units sorted out. I created a large partition:

Code: Select all

Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders
Units = cylinders of 64 * 512 = 32768 bytes

        Device Boot      Start         End      Blocks  Id System
/dev/mmcblk0p1             129         768       20480   c Win95 FAT32 (LBA)
Partition 1 does not end on cylinder boundary
/dev/mmcblk0p2             769        1056        9216  83 Linux
Partition 2 does not end on cylinder boundary
/dev/mmcblk0p3            1057      242560     7728128   5 Extended
/dev/mmcblk0p5            1057      242560     7728120  83 Linux

Which has the image written into it.

But the units are different and I can't seem to get the right partition sizes:

Code: Select all

Disk raspbian.img: 3276 MB, 3276800000 bytes
255 heads, 63 sectors/track, 398 cylinders, total 6400000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xa6202af7

       Device Boot      Start         End      Blocks   Id  System
raspbian.img1            8192      122879       57344    c  W95 FAT32 (LBA)
raspbian.img2          122880     6399999     3138560   83  Linux

What should I put into when creating the partitions, which has Units = cylinders of 64 * 512 = 32768 bytes?

I don't know what cylinders and sectors are also, so some enlightenment could be helpful as well?

[rpdom]

A sector is the smallest block of data that can be written to a storage device. Traditionally on disks that has been 512 (0.5KB) bytes, but now disks are getting bigger it can be 4096 (4KB) bytes or more.

A cylinder has no real meaning nowadays, but it used to be an important part of the disk geometry. A cylinder was basically one track (ring) on the disk unit. It is sometimes used as a low-level measurement to align data to the optimal position for read/write speeds.

fdisk should have command line options to set the units used for display, but you can see that one cylinder = 64 sectors.

Try deleting p5, then creating an new p5 starting at 1057 + (8192 / 64) = 1185, with length 57344 / 64 = 896.
The create p6 starting at 1057 + (122880 / 64) = 2977, and use the rest of the space, so you can expand it as needed.

Double check the units that are asked when creating the partitions. Don't forget that writing the partition table doesn't touch any data, so if you get it wrong first time you can recreate the table and try again.

[jojopi]

In that rather old version of fdisk, you will want to try the "-u" option to switch units to sectors, and possibly the "-c" option to disable MS-DOS compatibility warnings, both of which are the default in newer versions.

[kusti8]

Sorry if I'm asking dumb questions.

So, I used jojopi's suggestion and ran fdisk with -u so now the units are the same. I created /dev/mmcblk0p5 with start 75776 and end 190463 with 57344 blocks, which should be right, but when I try to create the second partition with start 190464, it says that its already allocated and suggests the default to be 190480, which would mean that I would loose data, right?

Thanks for all the help so far rpdom and jojopi...

[FM81]

Do you know, that logical drives inside an extended partition are chained in an other way? It is not the same, as with the four (possible) primary ones in the MBR-table?

Mounting the IMG-file as loop and simply copying the contents is not an option for you?

Greetings, FM_81

[kusti8]

Do you know, that logical drives inside an extended partition are chained in an other way? It is not the same, as with the four (possible) primary ones in the MBR-table?

Mounting the IMG-file as loop and simply copying the contents is not an option for you?

Greetings, FM_81

Ah, that makes sense.

No, because I'm running in RAM and can't download the entire file and store it somewhere, so I'm piping it to dd.

I guess I'll have to find another way.

[kusti8]

I've decided to simply use tarballs.

I'll keep on thinking about this issue, but will put it aside.

[rpdom]

I'll keep on thinking about this issue, but will put it aside.

Me, too.

Do you know, that logical drives inside an extended partition are chained in an other way? It is not the same, as with the four (possible) primary ones in the MBR-table?

Yes, but that should not be a problem. We're ignoring the table that is on the card and setting the start/length of the partitions to match.

[kusti8]

What's interesting is that even with some part cut of (I think?), I could sucessfully mount /dev/mmcblk0p6 and browse its files. When I tried to boot it, setting the boot partition to 5, it flashed 7 times, meaning it couldn't find kernel.img and I was unable to mount it, although I don't know if that's a problem with the partition or my system's lack of FAT mounting features.

I have a backup (the tarballs), but I'm going to continue experimenting and post my progress (if I have any )

Edit: It seems I was missing dosfstools, I'm going to try and mount it now.

[Aydan]

What about this:
1. get the partiton geometry from the image via fdisk (if the image is compressed you'd just need to copy the first few blocks of the image into a temp file for this to work)
2. DD the individual partitions (use bs, skip and count for this) from the image to the existing partitions. This should work with a compressed image as well.

Regards
Aydan

PS: Where is the source image stored?

[kusti8]

What about this:
1. get the partiton geometry from the image via fdisk (if the image is compressed you'd just need to copy the first few blocks of the image into a temp file for this to work)
2. DD the individual partitions (use bs, skip and count for this) from the image to the existing partitions. This should work with a compressed image as well.

Regards
Aydan

PS: Where is the source image stored?

I looked into that already and for some reason it didn't work. I'll try it again.

The image is stored on Github, so it can wget it, pipe it to gunzip and then write it.

The script that I've written on another pi makes the compressed image, I could have it make a file stating the partition table.

But I just got it to work with tarballs, so for the moment I'm happy.