How to make turn a custom raspberry pi install into a image?

[Milliways]

Ok, I have learnt you can't really make an img from a running install.

Of course you CAN - I do this routinely.

There is minimal risk that a file may change during the backup.
Prudence dictates you wouldn't backup during an upgrade etc but if the OS is idle it is OK.

Of course this assumes you are doing a file based backup - using dd on a running OS is bound to fail.

[kerry_s]

I don't know, im just guessing as no one seems to be helping, as usual Kerry_S is being helpful and I am trying the zenity GUI script they sent

just letting you know, that dd is slow & if you have a large disk, it takes to long. also i had a problem putting the img.gz back to a disk.
for now i have reverted to spare sd cards & the raspberry clone. i found i can clone other os's (tested ubuntu) from device to device, in this case i cloned ubuntu on ssd to a sd card & the sd card booted & worked perfectly.

currently taking advantage of prime day to shop for spare sd cards & other stuff.

[RaspbianUser1]

I don't know, im just guessing as no one seems to be helping, as usual Kerry_S is being helpful and I am trying the zenity GUI script they sent

I was under the impression you were having success using image-backup. It definitely creates image files from a running install.

Ok, well I don't particularly understand how to use it, I don't care about all the other features such as shrink, all I want is backup

I am able to use it after putting it in /usr/bin but I don't understand the questions the program asks, could you answer them/explain what they do.
Thanks in advance for your effort

[RaspbianUser1]

I don't know, im just guessing as no one seems to be helping, as usual Kerry_S is being helpful and I am trying the zenity GUI script they sent

just letting you know, that dd is slow & if you have a large disk, it takes to long. also i had a problem putting the img.gz back to a disk.
for now i have reverted to spare sd cards & the raspberry clone. i found i can clone other os's (tested ubuntu) from device to device, in this case i cloned ubuntu on ssd to a sd card & the sd card booted & worked perfectly.

currently taking advantage of prime day to shop for spare sd cards & other stuff.

I gave it a try using a usb adaptor of a install and it seems to either only do 1gb in an hour or do more than was on the actual card. Disk space usage on it was 10gb for the gz file was 11.3gb and the img file inside was only 1gb ????

[RonR]

I don't particularly understand how to use it, I don't care about all the other features such as shrink, all I want is backup

Just run image-backup with no parameters:

Code: Select all

sudo image-backup

All you need to specify is a path and filename for the image file to be created. The path must begin with /media or /mnt. Normally, you would have an external USB storage device mounted there, although you can create the image file directly in /media or /mnt if you have sufficient space available (creating the image file directly in /media or /mnt is likely to be much slower than backing up to a USB storage device mounted there).

Accept the default values (shown inside [...]) for the remaining questions by simply pressing the RETURN key.

That's all there is to it. It's very simple to use.

[HPCguy]

I tried the SD card copier to a USB flash drive and the USB drive wouldn't boot. I set up raspi-config for the appropriate failover if an SD card wasn't present, so that can't be the problem. I am really struggling with trying to make a USB flash drive image for my model 4B.

[RonR]

I tried the SD card copier to a USB flash drive and the USB drive wouldn't boot. I set up raspi-config for the appropriate failover if an SD card wasn't present, so that can't be the problem. I am really struggling with trying to make a USB flash drive image for my model 4B.

1. Disconnect the USB drive.

2. Make sure your SD card is updated with 'sudo apt update && sudo apt full-upgrade'.

3. Run Raspberry Pi EEPROM Manager, update the bootloader and VL805 (USB) EEPROM's to the latest version, if needed, and verify that the bootloader config has 'BOOT_ORDER=0xf41'.

4. Then connect the USB drive and run usb-boot to copy the SD card to the USB drive.

5. Shutdown, remove the SD card, power up, and the USB drive should boot.

[Botspot]

Is there a simple GUI program?

Yes! At least, there is a simple GUI program to make your own image. It's Pi-Power-Tools.
img-mode.png

I did look at your software, does it allow you to copy the current raspberry pi install to an img file?

OK, I've recently had several requests for this exact feature from different people, so I've decided to add this feature as of today.
Now, when you launch Pi-Power-Tools IMG Mode, you will see a new button:

snapshot.png (28.19 KiB) Viewed 403 times

It will ask where to save the img, then will begin copying files. Yes it's incremental, similar to RonR's 'image-backup' approach, but much more versatile and straightforward.
There shouldn't be any bugs, but if there are, please let me know.

[RaspbianUser1]

Thank you RonR for your reply
I will try again and give you an update

Thank you also Botspot
You have gone out of your way help primarily me from my question, I will try your new part as well and give an update!

Thank you everyone I will try these and see how it goes!

Edit for Botspot: You might want to add to your readme.md file to show the new pictures of sd to ___

[RaspbianUser1]

@Botspot

I have some comments about your update.

I am currently using RPI-OS MATE desktop (as in my sig)
As I am using MATE, lxterminal isn't installed by default instead i uses mate-terminal

Maybe either quote you must install lxterminal or change the code to use another terminal

Maybe install something like guake or terminator (make sure to say that it does this) then replace all instances of lxterminal to the new terminal.

Now this is probably a me issue but I can't get vdesktop to run my new img file, the button doesn't seem to run anything and manually running the script in ./vdesktop doesn't seem to run anything.

vdesktop sounds very useful for me as I intented to make lots of imgs and don't want to have to reboot and switch sd cards.

Also another question, what should happen when pressing shrink as nothing seems to happen as the size doesn't get smaller


Overall though, the updates you have made have made me install your software and I try to keep my new installs minimal, great job

[Botspot]

As I am using MATE, lxterminal isn't installed by default instead i uses mate-terminal
Maybe either quote you must install lxterminal or change the code to use another terminal

It's interesting you would mention that. I've been wondering what the best way would be to make PPT flexible, not calling terminal directly.
When you run the below command, does a terminal pop up? If so, which one? Or do you get a 'command not found' error?

Code: Select all

x-terminal-emulator

Now this is probably a me issue but I can't get vdesktop to run my new img file, the button doesn't seem to run anything and manually running the script in ./vdesktop doesn't seem to run anything.
vdesktop sounds very useful for me as I intented to make lots of imgs and don't want to have to reboot and switch sd cards.

Hmm. The version of vdesktop on Github is rather old.
About two weeks ago, I collaborated with kreal (a genius guy) and he was able to make vdesktop fully hardware accelerated. I also fixed a truckload of bugs.

Now, I want to push this new vdesktop version to Github, but before I can do that, I need some people to test it first. I posted a topic about it a week ago, but no responses.
So try installing that one, and see if it works better for you.

Also another question, what should happen when pressing shrink as nothing seems to happen as the size doesn't get smaller

That's likely another terminal issue. A terminal should pop up and do the image shrink stuff. But of course, if lxterminal isn't installed, then nothing would happen when you click that button.

Overall though, the updates you have made have made me install your software and I try to keep my new installs minimal, great job

[RaspbianUser1]

Sorry to ruin your praise but I have tried to use the img to another sd card and all was fine until booting it.

I get errors saying their is no firmware, I am able to boot using a different card though

The boot partitions files were in a boot folder so I moved all the files from the boot folder into just the boot partition and I get part the uart but just get stuck at the 4 raspberries along with it saying about the 4 usb hubs or something.

Its creating images, not working ones though

Also your program seems to have broken my wifi manager.

Should rootfs boot folder be empty as well?

[RaspbianUser1]

Update

I think its my fault

I just thought I didn't run your program as root... so it couldn't copy the boot folder

Also /dev is empty in it so....

I will retry

also the wifi thing was also my fault, a fix to a previous thing broke it

[RaspbianUser1]

@Botspot

Your program doesn't work, I am sorry to say

It doesn't copy /run /dev /boot

I have tried 3 times for it to work and it doesn't

I thank you for your effort though of trying.

[Botspot]

@Botspot
Your program doesn't work, I am sorry to say
I have tried 3 times for it to work and it doesn't
I thank you for your effort though of trying.

Don't lose hope! This is the design cycle:

design cycle.png (9.98 KiB) Viewed 276 times

Bug reports keep software moving forward. Thank you so much for pointing out ways PPT could be improved, I really appreciate it.

It doesn't copy /run /dev /boot

The directories /run, /dev, /sys, and others should not be copied: they are filled with device files, sockets, and other non-files. (RonR's image-backup doesn't copy these directories either BTW)
Boot should absolutely be copied! If it isn't copied, maybe try updating Pi-Power-Tools. (I made several bug fixes since bumping the version to 4)
BTW: updating PPT is really easy, just run this command:

Code: Select all

wget -O - https://raw.githubusercontent.com/Botspot/Pi-Power-Tools/master/update | bash

[RaspbianUser1]

Hi thanks for the reply

I wasn't aware of this

However I notice that on my current install which works, on the rootfs, boot is empty while boot isn't on the boot partition

I know the sd card is fine as its from amazon and I bought 2 of the same cards ( I was backing from the same card to the other one I bought)

Sandisk Endurance cards

Just a question, could you turn PiClone into making images as I like that program and it works

I don't know but it could work better

[RaspbianUser1]

I was already up to date and I ended up using RonRs software which worked with flying colours and already flashed it and it works fine no problems

Botspot, if it helps, the program didn’t seem to be able to write key files even with sudo privilege

[Botspot]

Just a question, could you turn PiClone into making images as I like that program and it works

I don't know but it could work better

The thing with taking & modifying other people's stuff (like piclone or image-utils) is you run the risk of really annoying someone. (in that case, RonR was annoyed and so I had to remove image-utils from PPT)
Also, PPT's IMG-from-SD feature is intended to be better, faster, and more reliable than image-backup or PiClone. (It uses a unique command sequence, that, once it works, will be superior than other alternatives)

Hi thanks for the reply

I wasn't aware of this

However I notice that on my current install which works, on the rootfs, boot is empty while boot isn't on the boot partition

OK, I've done several hours-worth of extensive testing on PPT, and have fixed a bunch more bugs a few minutes ago.
It turns out that in some scenarios, the boot partition isn't copied. Please run the PPT update command again.

[RaspbianUser1]

I must have said it wrong.

My install has the 2 standard partitions boot and rootfs

My rootfs boot folder is for some reason empty
And my boot partition, when copied using your software, instead of dumping all the boot files, it seems to create a folder called boot and put the files in that

Its easily fixed by running a file manager as root then moving all the files from the boot folder (in boot partition) to just in boot partition

Also I don’t know whats causing the next issue but it seems to get to the part when it’s registered the USB hubs and devices then stops entirely, when the programs start to turn on ( the bit with the green on text followed by what started)

[Botspot]

I must have said it wrong.

My install has the 2 standard partitions boot and rootfs

My rootfs boot folder is for some reason empty
And my boot partition, when copied using your software, instead of dumping all the boot files, it seems to create a folder called boot and put the files in that

Its easily fixed by running a file manager as root then moving all the files from the boot folder (in boot partition) to just in boot partition

Also I don’t know whats causing the next issue but it seems to get to the part when it’s registered the USB hubs and devices then stops entirely, when the programs start to turn on ( the bit with the green on text followed by what started)

Back to the books then. I'll do more fixing until it works reliably. Now that you have the image though, the rest of PPT's features may come in handy.
This afternoon I switched PPT from lxterminal to x-terminal-emulator. Please let me know if that successfully opens a terminal on your install.