User avatar
radilly
Posts: 4
Joined: Sat Oct 20, 2018 6:35 pm
Location: Pittsburgh, PA USA

Re: How can I prevent root to expand at first boot?

Sun Oct 21, 2018 8:06 pm

scruss wrote:
Sun Oct 21, 2018 5:07 pm
if you leave off count, dd will read all the blocks it can. This will typically be the full size of the filesystem.
Here the terminology, perhaps, trips me up a little. In the fdisk output above, it shows that the "disk" is (nominally) 4 GB - an intentionally small SD card. But only ~ 1.3 GB is allocated to partitions / filesystems (whichever is the correct term). That's the only part I believe I need to image to get the smallest snapshot; 1.3 GB. The structure may be easier to see with lsblk

Code: Select all

$ lsblk /dev/sdc
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdc      8:32   1  3.8G  0 disk
+-sdc1   8:33   1 41.7M  0 part
+-sdc2   8:34   1  1.2G  0 part
So you are right, when I imaged sdc, without count I got a 4 GB image file - the whole thing. When I did sdc1 and sdc2 without count I got 2 separate images totaling about 1.3 GB. But 2 image files are messier to work with than 1, so I think I just want the first part of the disk that has been allocated to partitions / filesystems. Restoring a 1.3 GB image is likely going to be considerably faster than 4 GB irrespective of the blocksize. At least that is my thinking.

Technically I think there are some unalloacted sectors within that 1.3 GB, but it's a rather small percentage. I believe dd'ing the first 1,304,671,232 bytes in this example is enough to copy what Etcher created to another SD card cleanly. Of course, as I think you plan, I want to do some initial configuration and may need to expand sdc2 a little to get the starting point I want work from.

When I searched for information on the use of dd for this purpose I either got very specific examples, or seemingly complicated discussions. I'm hoping this approach is a balance between simplicity and enough generalization to work for many common cases, i.e. the combination of
  • fdisk -l
  • dd (with bs and count numbers copied from the fdisk output)
I guess I could test it, but as @jahboater suggested, a smaller blocksize might keep one from having to try the dd multiple times. And using bs=512 (the default I think) means one can just copy the bs and count values from the fdisk output - without risk of making a math error! ;)

Writing this out helped me clarify my thinking - and hopefully may help others who want to image a (partially allocated) SD card. (Assuming the resize2fs_once process is bypassed as above.)

Cheers, and Thanks!

User avatar
scruss
Posts: 2059
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: How can I prevent root to expand at first boot?

Mon Oct 22, 2018 4:15 pm

radilly wrote:
Sun Oct 21, 2018 8:06 pm
… Restoring a 1.3 GB image is likely going to be considerably faster than 4 GB irrespective of the blocksize. At least that is my thinking.
You're right. It will be. If you want the speed benefit of the 1M block size while ensuring all the filesystem is copied, you could script something like this:

Code: Select all

sudo fdisk -l /dev/sdc | awk '/^Units/ {u=$(NF-1);} END{print int(1 + ($3 * u)/(1024*1024));}'
That gives you the count= value in MB, suitably rounded up.

For me, though, even on my smallest Raspberry Pi setups, this is kind of a moot point. I can't buy 4 GB cards any more (and don't think I ever saw a Class 10 one). USB 3 card readers are fast and cheap.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

dgerman
Posts: 11
Joined: Sat May 04, 2013 7:27 pm

Re: How can I prevent root to expand at first boot?

Sun Dec 16, 2018 1:59 am

Excellent question.

Why does the installation force users to have only one partition?
It's easy to make the root partition larger, making it smaller not so easy ( I haven't figured out how to do it yet!)

The 'Filesystem Hierarchy Standard' states that /var shall contain
"Variable files—files whose content is expected to continually change during normal operation of the system—such as logs, spool files, and temporary e-mail files."

A much better scheme would be to have most of the available space used for /var

Filling /var if it is a separate partition is much more recoverable than if /var resides in the root partition.

DirkS
Posts: 9734
Joined: Tue Jun 19, 2012 9:46 pm
Location: Essex, UK

Re: How can I prevent root to expand at first boot?

Sun Dec 16, 2018 10:50 am

dgerman wrote:
Sun Dec 16, 2018 1:59 am
Why does the installation force users to have only one partition?
I would suggest you read up on the target audience for the Pi, etc.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 21534
Joined: Sat Jul 30, 2011 7:41 pm

Re: How can I prevent root to expand at first boot?

Sun Dec 16, 2018 11:49 am

dgerman wrote:
Sun Dec 16, 2018 1:59 am
Excellent question.

Why does the installation force users to have only one partition?
It's easy to make the root partition larger, making it smaller not so easy ( I haven't figured out how to do it yet!)

The 'Filesystem Hierarchy Standard' states that /var shall contain
"Variable files—files whose content is expected to continually change during normal operation of the system—such as logs, spool files, and temporary e-mail files."

A much better scheme would be to have most of the available space used for /var

Filling /var if it is a separate partition is much more recoverable than if /var resides in the root partition.
Because for 99.999% of users that's the best option.

Not sure var having all the space is a good idea. Would be more useful assigned to /home. Or just do the default, and use the entire SD card for the whole system.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

code_exec
Posts: 260
Joined: Sun Sep 30, 2018 12:25 pm

Re: How can I prevent root to expand at first boot?

Sun Dec 16, 2018 12:16 pm

I'm sure that the Raspbian images have a cmdline.txt argument which causes the root filesystem to expand upon first boot. I haven't used Raspbian on any of my Pi's in ages (I'm using Debian ARM64 at the moment).

I suggest looking in cmdline.txt to see if there is anything relating to raspi-config and if you see anything, comment it out. Doing so may stop the filesystem from expanding on first boot.

But why would you want to?
Ubuntu 18.04 LTS desktop images for the Raspberry Pi 3.

https://github.com/CodeExecution/Ubuntu-ARM64-RPi

pfletch101
Posts: 368
Joined: Sat Feb 24, 2018 4:09 am

Re: How can I prevent root to expand at first boot?

Sun Dec 16, 2018 10:24 pm

code_exec wrote:
Sun Dec 16, 2018 12:16 pm
I suggest looking in cmdline.txt to see if there is anything relating to raspi-config and if you see anything, comment it out. Doing so may stop the filesystem from expanding on first boot.

But why would you want to?
The OP wanted to be able to partition a (larger) SD manually after first boot to regulate the size of the working partitions and include additional partition(s) of his choice. This does not seem entirely unreasonable. :)

User avatar
radilly
Posts: 4
Joined: Sat Oct 20, 2018 6:35 pm
Location: Pittsburgh, PA USA

Re: How can I prevent root to expand at first boot?

Sun Dec 16, 2018 10:46 pm

pfletch101 wrote:
Sun Dec 16, 2018 10:24 pm
code_exec wrote:
Sun Dec 16, 2018 12:16 pm
I suggest looking in cmdline.txt to see if there is anything relating to raspi-config and if you see anything, comment it out. Doing so may stop the filesystem from expanding on first boot.

But why would you want to?
The OP wanted to be able to partition a (larger) SD manually after first boot to regulate the size of the working partitions and include additional partition(s) of his choice. This does not seem entirely unreasonable. :)
Take a look at viewtopic.php?f=66&t=148492

connectable
Posts: 13
Joined: Tue Sep 01, 2015 5:34 am
Location: Central Portugal
Contact: Website

Re: How can I prevent root to expand at first boot?

Tue Dec 18, 2018 8:16 am

I’ve just spent two days, trawling through the web, trying various means of making a WORKING copy of my Raspbian SD card. From the relatively simple, to the ridiculously complex, I’ve so far succeeded in absolutely nothing whatsoever. The nearest I got was a card, which when booted, warned me of a bad partition, and did I want to format it? Ignoring this it proceeded to re-install, from the web, a new copy of NOOBS.

We all know, at least now, that all SD cards are not born the same – the default trash trotted out to copy a working image using WinDiskImager, dd, et al, DOES NOT WORK in almost every case I have used these.

Reading through the various contributions, the DEFAULT expansion of the file system now adopted, instead of a CONTROLLED one – a choice made by the compilers of the distribution, has caused endless problems for a whole host of users when they wish to simply make a copy of their current installation. Mindless answers such as ‘helps you get to know the system’ are simply not good enough. For myself, I just want to get on with developing my applications without being bogged down by a decision made to solve a problem, that with care at installation time, would simply not exist.

So, this morning I copied my 16Gb card image, with a mostly empty root filesystem, onto a 32Gb card. It works fine.

Raspberry Pi Distribution Implementors – Wake up and hear the music. Fix this wantonly stupid installation, and revert to giving a choice to expand the system TO A SIZE THAT WILL RELIABLY FIT ALL iterations of a given-sized card, or better a size chosen by the user.

Roken
Posts: 261
Joined: Sun Dec 31, 2017 4:35 pm

Re: How can I prevent root to expand at first boot?

Tue Dec 18, 2018 7:41 pm

Saamir wrote:
Tue May 17, 2016 6:35 pm
Has the OP question been answered? I see a lot of discussion about why you don't need to disable it, but what if there is one reason for disabling it that I consider valid? /rant
It simply fails here, but a valid reason is if you are booting from a USB drive (with rootfs on that drive) and the remainder of the drive has varied directories etc. that you want preserved.

For example, I have Pi No 1 as an NFS server, booting rootfs straight from the drive that is mounted as an NFS drive. The other 3 Pis NFS boot from directories on that drive, and the remainder of the drive is stuff I want to share via the NFS.

In my configuration, the last thing I would want is the boot pi expanding the filesystem. Like I said, it simply fails here, which is exactly what I want, but it is a valid reason not to have it automatically expanded.
Headless PI. OMG, someone cut it's head off. Oh, hang on. it didn't have one to start with.

User avatar
radilly
Posts: 4
Joined: Sat Oct 20, 2018 6:35 pm
Location: Pittsburgh, PA USA

Re: How can I prevent root to expand at first boot?

Tue Dec 18, 2018 7:52 pm

FYI- A while back I took a quick look at the bash script that does the expansion. If I recall, if the "disk" doesn't have exactly 2 partitions, the resize aborts.

I was thinking that was another way to bypass the expansion; add another small (perhaps temp) partition before booting.

I have not yet tried this, however....

User avatar
Greg Erskine
Posts: 94
Joined: Sat Sep 15, 2012 4:20 am

Re: How can I prevent root to expand at first boot?

Tue Dec 18, 2018 8:01 pm

connectable wrote:
Tue Dec 18, 2018 8:16 am
I’ve just spent two days, trawling through the web, trying various means of making a WORKING copy of my Raspbian SD card. From the relatively simple, to the ridiculously complex, I’ve so far succeeded in absolutely nothing whatsoever. The nearest I got was a card, which when booted, warned me of a bad partition, and did I want to format it? Ignoring this it proceeded to re-install, from the web, a new copy of NOOBS.

We all know, at least now, that all SD cards are not born the same – the default trash trotted out to copy a working image using WinDiskImager, dd, et al, DOES NOT WORK in almost every case I have used these.

Reading through the various contributions, the DEFAULT expansion of the file system now adopted, instead of a CONTROLLED one – a choice made by the compilers of the distribution, has caused endless problems for a whole host of users when they wish to simply make a copy of their current installation. Mindless answers such as ‘helps you get to know the system’ are simply not good enough. For myself, I just want to get on with developing my applications without being bogged down by a decision made to solve a problem, that with care at installation time, would simply not exist.

So, this morning I copied my 16Gb card image, with a mostly empty root filesystem, onto a 32Gb card. It works fine.

Raspberry Pi Distribution Implementors – Wake up and hear the music. Fix this wantonly stupid installation, and revert to giving a choice to expand the system TO A SIZE THAT WILL RELIABLY FIT ALL iterations of a given-sized card, or better a size chosen by the user.
hi connectable,

Why not just pick another Linux distribution or get rid your Raspberry Pi's, for something more suitable to your requirements. Simple!

regards
Greg
* Raspberry Pi is a trademark of the Raspberry Pi Foundation

User avatar
scruss
Posts: 2059
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: How can I prevent root to expand at first boot?

Tue Dec 18, 2018 8:24 pm

connectable wrote:
Tue Dec 18, 2018 8:16 am
Fix this wantonly stupid installation, and revert to giving a choice to expand the system TO A SIZE THAT WILL RELIABLY FIT ALL iterations of a given-sized card, or better a size chosen by the user.
That might indicate that your card isn't as large as it claims to be. This is very common with knock-off cards, which can sometimes come through even official distribution channels. If I suspect a card is dodgy, this is what I do:
  • format it using the SD Association's SD Memory Card Formatter using the slower “Overwrite” option. This tool is Windows and Mac only, unfortunately.
  • Test the card's capacity with f3write/f3read (or if it works for you, f3probe).
It's a fairly reasonable approach that the Foundation takes that it will fill the card to its stated capacity. Sure, it's frustrating when you get a bad card, but most users outside this thread don't want to mess with setting filesystem sizes.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

dgerman
Posts: 11
Joined: Sat May 04, 2013 7:27 pm

Re: How can I prevent root to expand at first boot?

Wed Jan 02, 2019 2:22 am

Can the init_resize.sh script be moved from the root partition to the boot partition

Details at
viewtopic.php?f=63&t=225791&p=1406467#p1406467

Thank you.

DirkS
Posts: 9734
Joined: Tue Jun 19, 2012 9:46 pm
Location: Essex, UK

Re: How can I prevent root to expand at first boot?

Wed Jan 02, 2019 10:53 am

dgerman wrote:
Wed Jan 02, 2019 2:22 am
Can the init_resize.sh script be moved from the root partition to the boot partition

Details at
viewtopic.php?f=63&t=225791&p=1406467#p1406467

Thank you.
For requests like this an issue should be opened at https://github.com/RPi-Distro/pi-gen/issues
You will have to add reasons you would like the change, though

Return to “Raspbian”