sn2019
Posts: 16
Joined: Thu Nov 29, 2018 10:06 pm

Re: Image File Utilities

Tue Sep 01, 2020 3:09 pm

RonR wrote:
Tue Sep 01, 2020 12:11 am

If any errors occur during image-backup, they will be reported. If you want full details, use:
...

Code: Select all

image-backup --options --progress <image file>

Thanks. I am planning to run it as a cron job.

sandra6
Posts: 8
Joined: Sun Apr 24, 2016 10:35 am

Re: Image File Utilities

Sat Sep 26, 2020 8:15 am

I have used your great backup utility before on my rpi4 without problem..

Now I boot from SSD on the USB I'm getting an error that was reported a couple of pages back, so I tried the solution suggested but still getting this error. This is a full, not an incremental backup with latest zip from page1

Code: Select all

sudo /home/pi/image-utils/image-backup /media/rpi4
losetup: /media/rpi4: Warning: file is smaller than 512 bytes; the loop device may be useless or invisible for system tools.
mount: /tmp/img-backup-mnt: special device /dev/loop0p2 does not exist.

Unable to mount image ROOT partition
Running...

Code: Select all

 sudo losetup -D
Doesn't help and gives the same result

Is there anything else I can try, or give you more info to help?

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sat Sep 26, 2020 8:22 am

sandra6 wrote:
Sat Sep 26, 2020 8:15 am
Is there anything else I can try, or give you more info to help?

Please post the output of 'lsblk' and 'fdisk -l /dev/sdX' (where sdX is the SSD you're running on).

sandra6
Posts: 8
Joined: Sun Apr 24, 2016 10:35 am

Re: Image File Utilities

Sat Sep 26, 2020 9:25 am

Thank you, here we are...

Code: Select all

 lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 223.6G  0 disk
├─sda1   8:1    0   256M  0 part /boot
└─sda2   8:2    0 223.3G  0 part /

Code: Select all

 sudo fdisk -l /dev/sda
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Disk model:
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Disklabel type: dos
Disk identifier: 0x73311f5e

Device     Boot  Start       End   Sectors   Size Id Type
/dev/sda1         8192    532479    524288   256M  c W95 FAT32 (LBA)
/dev/sda2       532480 468862127 468329648 223.3G 83 Linux

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sat Sep 26, 2020 6:05 pm

sandra6 wrote:
Sat Sep 26, 2020 8:15 am
Now I boot from SSD on the USB I'm getting an error that was reported a couple of pages back, so I tried the solution suggested but still getting this error. This is a full, not an incremental backup with latest zip from page1

Code: Select all

sudo /home/pi/image-utils/image-backup /media/rpi4

For a full backup, you should not be specifying a filename or path on the command line.

To do a full backup interactively, simply run image-backup with no parameters:

Code: Select all

sudo /home/pi/image-utils/image-backup

You may specify full backup parameters on the command line instead of interactively by using a -i or --initial option:

Code: Select all

sudo /home/pi/image-utils/image-backup --initial pathto/filename of image file [,inital size MB [,added space for incremental MB]]

If 'inital size MB' or 'added space for incremental MB' is not given, they will be the default values.

sandra6
Posts: 8
Joined: Sun Apr 24, 2016 10:35 am

Re: Image File Utilities

Sun Sep 27, 2020 7:21 am

Thank you.

I'd forgotten how to start with a full backup and included the file name, then thought moving to USB boot was my problem!

Followed your instructions and all good now, apologies for the mix up.

Help and guidance was much appreciated.
Last edited by sandra6 on Sun Sep 27, 2020 9:16 am, edited 1 time in total.

sandra6
Posts: 8
Joined: Sun Apr 24, 2016 10:35 am

Re: Image File Utilities

Sun Sep 27, 2020 9:15 am

Apologies, backup completed now without error

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sun Sep 27, 2020 5:47 pm

sandra6 wrote:
Sun Sep 27, 2020 9:15 am
Apologies, backup completed now without error

Thanks for the feedback. Good to hear you got it going.

Bosse_B
Posts: 1074
Joined: Thu Jan 30, 2014 9:53 am

Re: Image File Utilities

Sun Oct 04, 2020 9:40 am

I wonder if these utilities work also on Ubuntu?
I have a massive img file created on an RPi3 by using dd towards an NFS share on Ubuntu Server 18.
I want to shrink the image down to allow writing to a smaller sd card.
So in order to use the higher power of the Ubuntu system it would be good if I could use the shrink option directly on Ubuntu rather than to do it on an RPi via the network.
Bo Berglund
Sweden

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sun Oct 04, 2020 6:10 pm

Bosse_B wrote:
Sun Oct 04, 2020 9:40 am
I wonder if these utilities work also on Ubuntu?

I would think so, but I've never run them on Ubuntu. I can't think of any danger in trying them. Work on a copy of the image file rather than the original, just in case.

User avatar
felgy
Posts: 1
Joined: Sat Oct 10, 2020 4:44 pm

Re: Image File Utilities

Sat Oct 10, 2020 5:06 pm

Hi RonR,
Just used your script for the 1st time and wanted to thank you for the awesome piece of code!!
It'll help a lot the boring task of keeping my Pis backed up.
Cheers
felgy
Raspberry Pi 2 Model B Rev 1.1
Raspberry Pi Zero W Rev 1.1
Raspberry Pi 3 Model B Rev 1.2
Raspberry Pi 3 Model B Plus Rev 1.3
Raspberry Pi 4 Model B Rev 1.2 [4GB]

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sat Oct 10, 2020 8:36 pm

felgy wrote:
Sat Oct 10, 2020 5:06 pm
Just used your script for the 1st time and wanted to thank you for the awesome piece of code!!
It'll help a lot the boring task of keeping my Pis backed up.

Thanks for the feedback!

GuideThomas
Posts: 2
Joined: Fri Oct 16, 2020 12:24 pm

Re: Image File Utilities

Fri Oct 16, 2020 12:36 pm

Hi friends,

i have got a really stupid question:

image-backup and all the other scripts have to be started with

Code: Select all

sh image-backup ...
or not?

'cause when i try to call without sh there's the error Command not found. Of course i did

Code: Select all

chmod +x
to all files before.

But nevertheless, after i call

Code: Select all

sh image-backup /path-to-image/image-name.img
i got an error on line 210:

/opt/image-utils-master/image-backup: 210: /opt/image-utils-master/image-backup: Syntax error: redirection unexpected

Any suggestions?

GuideThomas
Posts: 2
Joined: Fri Oct 16, 2020 12:24 pm

Re: Image File Utilities

Fri Oct 16, 2020 12:53 pm

GuideThomas wrote:
Fri Oct 16, 2020 12:36 pm
Hi friends,

i have got a really stupid question:

...

Sorry, yet found the solution by myself:

In my case, the call has to be made by

Code: Select all

./image-backup --initial /path-to-image/image-name.img
... then it's working

PS: --progress doesn't seem to be supported!?

PSS: I also tried --noexpand but then i got the error:

rsync: --noexpand: unknown option
rsync error: syntax or usage error (code 1) at main.c(1596) [client=3.1.3]

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Fri Oct 16, 2020 5:50 pm

GuideThomas wrote:
Fri Oct 16, 2020 12:53 pm
PS: --progress doesn't seem to be supported!?

PSS: I also tried --noexpand but then i got the error:

rsync: --noexpand: unknown option
rsync error: syntax or usage error (code 1) at main.c(1596) [client=3.1.3]

--progress is an rsync option and must be introduced with a -o or --options image-backup option:

image-backup --options --progress

-n or --noexpand is simply an image-backup option:

image-backup --noexpand

Everything appears to be working as intended and expected.

User avatar
McMelloW
Posts: 7
Joined: Mon Oct 12, 2020 10:23 am

Re: Image File Utilities

Mon Oct 26, 2020 5:30 pm

Dear RonR,

thanks for all the work and effort you have done on this IFU. Is there an example file with some "How to's" for the use of this IFU.
It is a bit time consuming to search this whole topic.
Greetings McMelloW

ivanpop
Posts: 5
Joined: Thu Jun 21, 2018 6:05 am

Re: Image File Utilities

Wed Nov 11, 2020 11:08 am

How do I run this?
image-backup -h
bash: image-backup: command not found

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Wed Nov 11, 2020 6:49 pm

ivanpop wrote:
Wed Nov 11, 2020 11:08 am
How do I run this?
image-backup -h
bash: image-backup: command not found

What directory did you unzip image-utils.zip to?

Does an 'ls' of that directory show image-backup?

Did you make image-backup executable with 'chmod +x image-backup'?

Are you running image-backup with sudo?

timdenning
Posts: 2
Joined: Fri Nov 13, 2020 1:26 pm

Re: Image File Utilities

Fri Nov 13, 2020 1:37 pm

has the raspberry pi imager no plans for creating images from your sd card to be able to make backups ?
Im using linux mint, and win32 disk imager wont work, the dd commands almost work, why aint the raspberry pi imager got the option to clone or backup your microsd card in a simple way like it has to write images to sd cards ?

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Fri Nov 13, 2020 7:41 pm

timdenning wrote:
Fri Nov 13, 2020 1:37 pm
has the raspberry pi imager no plans for creating images from your sd card to be able to make backups ?
Im using linux mint, and win32 disk imager wont work, the dd commands almost work, why aint the raspberry pi imager got the option to clone or backup your microsd card in a simple way like it has to write images to sd cards ?

You are posting your frustration and questions in the wrong topic The Image File Utilities posted and discussed in this topic are unrelated to the Raspberry Pi Imager and Linux Mint. You would be better served if you start a new topic to address your support requests pertaining to the Raspberry Pi Imager and Linux Mint.

0setdetnet-0
Posts: 2
Joined: Sat Nov 14, 2020 3:50 pm

Re: Image File Utilities

Sat Nov 14, 2020 7:02 pm

Sorry to interject with an unhelpful (to the author) post. But, I just cannot get how overcomplicated this is.
This is somewhat flawed through:
* It's over complicated - far too much overcooking of the egg.
* Having to guess at space needed to later discover "out of space" errors.
* Shrinking images 3 times
* Errors thrown relating to legacy OS but the author refuses to support
* Not very robust to errors

Folks, may I introduce you to SquashFU
Developed by @falconindy (not my work)
But I do have a small repo you can download a single file to get up and runninng
https://github.com/setdetnet/pi-backup

*Very simple but very flexible
*Creates "Bins" that can hold full or incremental backups.
*Bins can store a min number of full or incremental backups
*Bins can rotate
*When incremental backups reach a predefined number, squashfu will create a new full backup and increment that one.
*Has a config file at /etc/squashfu.conf
*Dead easy to run:
squashfu -B


It does it all for you.

Sory Rob, this image-backup is faaaar to convoluted. But i recognise sensible users will prefer yours and the way it works. That's fine. For n00bs and easy use, headover to squashfu

Lipown
Posts: 157
Joined: Sun Oct 13, 2019 8:32 am

Re: Image File Utilities

Sat Nov 14, 2020 11:15 pm

Thank you for this. This is what I was looking for! Why it is not into official repository?

tinker2much
Posts: 195
Joined: Wed Jun 20, 2018 12:38 am

Re: Image File Utilities

Tue Nov 17, 2020 12:53 am

I frequently use these utilities to incrementally back up (and occasionally restore) many RPIs of various types and vintages, using a 500GB mobile had drive. Works great!

I am also an enthusiastic user of "Debian Buster with Raspberry Pi Desktop" on my Acer ex-Chromebook. A familiar and productive software environment.

Can I combine these? I need a backup solution for the laptop and would like to use the tools I already know.

I know that the disk format may not be the same:

Code: Select all

pi@piacer:~/.pwsafe $ df -H
Filesystem      Size  Used Avail Use% Mounted on
udev            968M     0  968M   0% /dev
tmpfs           201M  5.9M  195M   3% /run
/dev/mmcblk0p2   13G  7.9G  4.4G  65% /
tmpfs           1.1G   30M  973M   3% /dev/shm
tmpfs           5.3M  4.1k  5.3M   1% /run/lock
tmpfs           1.1G     0  1.1G   0% /sys/fs/cgroup
/dev/mmcblk0p1  536M  4.3M  532M   1% /boot/efi
tmpfs           201M     0  201M   0% /run/user/1000
I also know that backing up is only half the puzzle - I also need a tool to do restores, and I don't know whether the usual RPI imaging tools (things like RPI imager, etcher, etc) could be made to work, because RPD is normally installed using what looks like the typical Debian installation process, not by writing images.

But one can hope...

Thoughts?

JMK8
Posts: 50
Joined: Tue Apr 28, 2015 10:00 am

Re: Image File Utilities - no longer runs without errors

Fri Nov 20, 2020 11:52 am

For some reason image-backup, that was OK, now fails every time. What have I broken :cry:

Code: Select all

john@V4-1:~/sdimage $ dir
total 60K
drwxr-xr-x 2 john john 4.0K Nov 19 11:08 .
drwxr-xr-x 9 john john 4.0K Nov 19 12:54 ..
-rwxr-xr-x 1 john john  13K Nov 19 11:08 image-backup
-rwxr-xr-x 1 john john 1.5K Nov 19 11:08 image-check
-rwxr-xr-x 1 john john 2.8K Nov 19 11:08 image-compare
-rwxr-xr-x 1 john john 2.7K Nov 19 11:08 image-info
-rwxr-xr-x 1 john john 1.7K Nov 19 11:08 image-mount
-rwxr-xr-x 1 john john 5.6K Nov 19 11:08 image-set-partuuid
-rwxr-xr-x 1 john john 4.1K Nov 19 11:08 image-shrink
-rw-r--r-- 1 john john 3.4K Nov 19 11:08 README.txt
john@V4-1:~/sdimage $ sudo ./image-backup -i /mnt/backup/image20112020

Starting full backup (for incremental backups, run: ./image-backup /mnt/backup/image20112020)

e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0p2: 47416/93888 files (0.9% non-contiguous), 327263/375296 blocks

resize2fs 1.44.5 (15-Dec-2018)
The filesystem is already 375296 (4k) blocks long.  Nothing to do!

resize2fs 1.44.5 (15-Dec-2018)
The filesystem is already 375296 (4k) blocks long.  Nothing to do!

resize2fs 1.44.5 (15-Dec-2018)
The filesystem is already 375296 (4k) blocks long.  Nothing to do!

e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0p2: 47416/93888 files (0.9% non-contiguous), 327263/375296 blocks
mount: /tmp/img-backup-mnt: wrong fs type, bad option, bad superblock on /dev/loop0p2, missing codepage or helper program, or other error.

Unable to mount image ROOT partition

john@V4-1:~/sdimage $ ./image-check /mnt/backup/image20112020

./image-check must be run as root user

john@V4-1:~/sdimage $ sudo ./image-check /mnt/backup/image20112020

Check /mnt/backup/image20112020 (y/n)? y

e2fsck 1.44.5 (15-Dec-2018)
The filesystem size (according to the superblock) is 375296 blocks
The physical size of the device is 309504 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? yes

Code: Select all

john@V4-1:~/sdimage $ uname -a
Linux V4-1 5.4.72-v7l+ #1356 SMP Thu Oct 22 13:57:51 BST 2020 armv7l GNU/Linux
john@V4-1:~/sdimage $

RonR
Posts: 1706
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities - no longer runs without errors

Fri Nov 20, 2020 7:58 pm

JMK8 wrote:
Fri Nov 20, 2020 11:52 am
john@V4-1:~/sdimage $ sudo ./image-check /mnt/backup/image20112020

Check /mnt/backup/image20112020 (y/n)? y

e2fsck 1.44.5 (15-Dec-2018)
The filesystem size (according to the superblock) is 375296 blocks
The physical size of the device is 309504 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? yes

Any chance the device mounted on /mnt is too small or defective?:
JMK8 wrote:
Fri Nov 20, 2020 11:52 am
e2fsck 1.44.5 (15-Dec-2018)
The filesystem size (according to the superblock) is 375296 blocks
The physical size of the device is 309504 blocks

The filesystem size is somehow larger than the device size.

Return to “Advanced users”