RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sun Mar 01, 2020 4:35 am

Jamescoop wrote:
Sun Mar 01, 2020 4:23 am
For those following along, a standard USB-drive plugged in to a Pi appears in /media/pi/[drive name]

So when prompted ""Image file to create" enter: /media/pi/[USB Drive name]/[Desired file name]

That should do it.
Jamescoop wrote:
Sun Mar 01, 2020 4:23 am
Do I need to specify anything when prompted "Added space for incremental updates" <--- what does this do?

If "Added space for incremental updates" is zero (default), then no additional space is included for incremental updates. The resulting initial/full backup will be shrunk to the smallest size possible with little to no free space for expansion. If you plan to incrementally update the image file, then you need to have some additional space added to the image file for this growth.

User avatar
Botspot
Posts: 1631
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas! :)
Contact: Website

Re: Image File Utilities

Sun Mar 01, 2020 1:40 pm

RonR wrote:
Sun Mar 01, 2020 4:10 am
I would prefer that you simply include a link to the first post of this topic (not the attachment URL itself as it changes when the attachment is updated) so there's not multiple sources, some of which will surely become outdated over time.
The scripts themselves will have to be shipped along with the data of the rest of the tool. I have modified a the scripts to wait until Enter is pressed.
The GUI looks like this:
Screenshot from 2020-03-01 07-38-00.png
Screenshot from 2020-03-01 07-38-00.png (29.93 KiB) Viewed 2194 times
So, may I have your permission to include your scripts as part of my app?
Last edited by Botspot on Sun Mar 01, 2020 7:37 pm, edited 1 time in total.
Want to run Minecraft Java on your RPi? Easiest way is with Pi-Apps - just click Install.
https://github.com/Botspot/pi-apps
Pi-Apps also includes Zoom, Visual Studio, Tor browser, Windows 10 theme, and 33 more.
Over 50,000 users!

Jamescoop
Posts: 15
Joined: Tue May 26, 2015 8:06 pm

Re: Image File Utilities

Sun Mar 01, 2020 4:23 pm


If "Added space for incremental updates" is zero (default), then no additional space is included for incremental updates. The resulting initial/full backup will be shrunk to the smallest size possible with little to no free space for expansion. If you plan to incrementally update the image file, then you need to have some additional space added to the image file for this growth.
So just take a guess as how much I will need for the next backups? Or is this only for the first incremental backup, then I specify again during that process?

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Sun Mar 01, 2020 7:35 pm

Jamescoop wrote:
Sun Mar 01, 2020 4:23 pm
So just take a guess as how much I will need for the next backups? Or is this only for the first incremental backup, then I specify again during that process?

Once the initial/full backup is created, its size cannot be changed. If you expect to be doing incremental backups that would require a larger image file, you must allow for that by specifying some additional space to hold any future expansion. Worst case, if you exceed the available space in an image file during an incremental backup, you will get a 'Not enough space available' error during that backup and you will simply have to start over with a fresh initial/full backup.

Jamescoop
Posts: 15
Joined: Tue May 26, 2015 8:06 pm

Re: Image File Utilities

Mon Mar 02, 2020 3:29 am

Thank you for your help.

Now that I have the backup on USB stick, I stuck it in my PC to check it with a new SD card...running in to some poblems -- etcher does not recognize the file.

Do I need to rename the file or something? Your initial post says it is recognized by etcher -- what have I missed?

Thank you in advance.

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Mon Mar 02, 2020 3:45 am

Jamescoop wrote:
Mon Mar 02, 2020 3:29 am
Now that I have the backup on USB stick, I stuck it in my PC to check it with a new SD card...running in to some poblems -- etcher does not recognize the file.

Do I need to rename the file or something? Your initial post says it is recognized by etcher -- what have I missed?

Windows programs typically recognize file types by the extension portion of their filename (*.txt = a text file, *.exe = an executable file, etc.) Did you give the backup image filename an extension that Etcher recognizes in its Open dialog box (most commonly, *.img)? If not, rename it to something.img and Etcher will recognize it as an image file.

SteveI
Posts: 2
Joined: Wed Mar 04, 2020 3:21 pm

Re: Image File Utilities

Wed Mar 04, 2020 3:30 pm

I would like to use image-backup within a bash script I create that updates /etc/motd.... mounts/unmounts my device on /media, etc. then calls
image-backup

I've tried: cat <<EOF | sudo image-backup
/media/images/newbackup.img

50
y
EOF

with in my bash script..but can't seem to get it to work.
Any chance on adding command line options to image-backup to provide , filename, BOOT size, Added Space for idumps, etc

.thanks

..Steve

User avatar
Botspot
Posts: 1631
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas! :)
Contact: Website

Re: Image File Utilities

Wed Mar 04, 2020 8:57 pm

I'll be brief, just wanted to mention this here: Pi Power Tools has just been released TODAY. If you find Image File Utilities useful, then boy will you love Pi Power Tools! :D
It does all sorts of stuff (see my signature below), and I promised to keep this brief so here's just one screenshot:
main3.png
main3.png (29.71 KiB) Viewed 2040 times
Want to run Minecraft Java on your RPi? Easiest way is with Pi-Apps - just click Install.
https://github.com/Botspot/pi-apps
Pi-Apps also includes Zoom, Visual Studio, Tor browser, Windows 10 theme, and 33 more.
Over 50,000 users!

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Wed Mar 04, 2020 10:22 pm

SteveI wrote:
Wed Mar 04, 2020 3:30 pm
I've tried: cat <<EOF | sudo image-backup
/media/images/newbackup.img

50
y
EOF

with in my bash script..but can't seem to get it to work.

Steve,

The correct syntax is:

Code: Select all

sudo image-backup <<EOF
/media/images/newbackup.img

50
y
EOF

This will create newbackup.img, but the shrinking of the image will fail because e2fsck and resize2fs are not intended to be run non-interactively.

If you're willing to forego safety checks, you can make the following changes to image-backup (please do not distribute the modified version).

Add '-n' (4 places):

Code: Select all

e2fsck -f -n "${LOOP2}"

Add '-f' (4 places):

Code: Select all

resize2fs -f -M "${LOOP2}"

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Wed Mar 04, 2020 11:12 pm

Botspot wrote:
Wed Mar 04, 2020 8:57 pm
If you find Image File Utilities useful, then boy will you love Pi Power Tools! :D

Botspot,

You should let users know that you are distributing outdated and modified versions of my Image File Utilities in opposition to my request and against my wishes.

SteveI
Posts: 2
Joined: Wed Mar 04, 2020 3:21 pm

Re: Image File Utilities

Wed Mar 04, 2020 11:52 pm

RonR,
The correct syntax is:

Code: Select all

sudo image-backup <<EOF
/media/images/newbackup.img

50
y
EOF
Thanks.. I'll give it a try.. I didn't want to modify your code!

Regards,
Steve

User avatar
Botspot
Posts: 1631
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas! :)
Contact: Website

Re: Image File Utilities

Thu Mar 05, 2020 12:02 am

RonR wrote:
Wed Mar 04, 2020 11:12 pm
Botspot,

You should let users know that you are distributing outdated and modified versions of my Image File Utilities in opposition to my request and against my wishes.
Sorry, I must have misunderstood your request...
RonR wrote:
Sun Mar 01, 2020 4:10 am
I would prefer that you simply include a link to the first post of this topic (not the attachment URL itself as it changes when the attachment is updated) so there's not multiple sources, some of which will surely become outdated over time.
...to mean you wanted me to give credit using a link to the homepage. :oops:

What will satisfy you:
  • Completely remove image-utils from pi power tools?
  • Give you credit and add a link to the page? (That's how I interpreted your request)
  • Occasionally manually update image-utils, and add the modifications each time?
  • Or, somehow have the script download image-utils directly from your link each time a user installs it? I'm not sure this is even possible. Edit: This is exactly what I did: whenever a user downloads Pi-Power-Tools, it will download the latest version of image-utils directly from your download link.

FYI, the modification is extremely minimal. At the end of a couple of the scripts, I added two lines to wait until Enter is pressed. Is that OK with you, or do you insist I leave them completely stock? Edit: Found a workaround. From now on, image-utils will not be modified at all.
Last edited by Botspot on Thu Mar 05, 2020 2:46 am, edited 1 time in total.
Want to run Minecraft Java on your RPi? Easiest way is with Pi-Apps - just click Install.
https://github.com/Botspot/pi-apps
Pi-Apps also includes Zoom, Visual Studio, Tor browser, Windows 10 theme, and 33 more.
Over 50,000 users!

User avatar
Botspot
Posts: 1631
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas! :)
Contact: Website

Re: Image File Utilities

Thu Mar 05, 2020 2:16 am

Update: I figured out how to have the installation script download your image-utils directly from your provided link.
It parses the html file to determine the download link, so this will continue to work the next time you update the .zip file.

Also, I removed the requirement for any modifications to be made, so your scripts will be kept completely stock.

This update has been pushed to GitHub, so please delete your harsh comment from my Pi-Power-Tools homepage.
Want to run Minecraft Java on your RPi? Easiest way is with Pi-Apps - just click Install.
https://github.com/Botspot/pi-apps
Pi-Apps also includes Zoom, Visual Studio, Tor browser, Windows 10 theme, and 33 more.
Over 50,000 users!

dsyleixa123
Posts: 1117
Joined: Mon Jun 11, 2018 11:22 am

Re: Image File Utilities

Thu Mar 05, 2020 9:15 am

Botspot wrote:
Thu Mar 05, 2020 2:16 am
Update: I figured out how to have the installation script download your image-utils directly from your provided link.
It parses the html file to determine the download link, so this will continue to work the next time you update the .zip file.

Also, I removed the requirement for any modifications to be made, so your scripts will be kept completely stock.

This update has been pushed to GitHub, so please delete your harsh comment from my Pi-Power-Tools homepage.
will this GUI window menu version also work with Stretch Lite without GUI (e.g., Retropie) ?
Or will it be possible to generate a pure-text-window-menu (e.g., like Raspi-config) to work also with Stretch-Lite?

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Thu Mar 05, 2020 9:47 am

dsyleixa123 wrote:
Thu Mar 05, 2020 9:15 am
will this GUI window menu version also work with Stretch Lite without GUI (e.g., Retropie) ?
Or will it be possible to generate a pure-text-window-menu (e.g., like Raspi-config) to work also with Stretch-Lite?

Image File Utilities are intended to be run from the command line and should run under Stretch-Lite (I haven't run Stretch lately, but the Image File Utilities were originally developed back in the Wheezy and Jessie days).

dsyleixa123
Posts: 1117
Joined: Mon Jun 11, 2018 11:22 am

Re: Image File Utilities

Thu Mar 05, 2020 10:24 am

yes, thank you, but tbh: the dozens of command line parameters are really confusing me, so I actually would prefer a menu-guided interactive start, feat. media-auto-detection for selection ;)

User avatar
Botspot
Posts: 1631
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas! :)
Contact: Website

Re: Image File Utilities

Thu Mar 05, 2020 1:12 pm

RonR,

Can you confirm that my above solution to Pi-Power-Tools has satisfied your requirement of not "distributing outdated and modified versions" of Image-utils?
Want to run Minecraft Java on your RPi? Easiest way is with Pi-Apps - just click Install.
https://github.com/Botspot/pi-apps
Pi-Apps also includes Zoom, Visual Studio, Tor browser, Windows 10 theme, and 33 more.
Over 50,000 users!

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Thu Mar 05, 2020 8:39 pm

dsyleixa123 wrote:
Thu Mar 05, 2020 10:24 am
the dozens of command line parameters are really confusing me, so I actually would prefer a menu-guided interactive start, feat. media-auto-detection for selection ;)

As you are already familiar with working from the command line, simply execute the command with no parameters to see the appropriate usage:

---

root@FreePBX:~# ./image-check

Usage: ./image-check imagefile [W95|Linux]

---

root@FreePBX:~# ./image-compare

Image file to compare?

---

root@FreePBX:~# ./image-mount

Usage: ./image-mount imagefile mountpoint [W95|Linux]

---

root@FreePBX:~# ./image-set-ptuuid

Usage: ./image-set-ptuuid imagefile ptuuid

---

root@FreePBX:~# ./image-shrink

Usage: ./image-shrink imagefile [Additional MB]

---

dsyleixa123
Posts: 1117
Joined: Mon Jun 11, 2018 11:22 am

Re: Image File Utilities

Thu Mar 05, 2020 9:02 pm

I actually hardly use the commandline (neither for Retropie's Stretch Lite nor even for my 2nd SD card feat. full GUI Stretch), the letters are far too small on my small screen, I have no internet browser on my Stretch Lite to copy and paste, and I am always getting annoying typos, that's why I am using GUI menus as far as possible (like in MS Windows). So a menu window like for raspi-config etc would be far easier to use!
Unfortunately I also didn't manage yet to upgrade the Stretch-Lite to the full GUI Stretch so far to use Botspot's GUI window wrapper.

holle75
Posts: 8
Joined: Thu Mar 05, 2020 9:27 pm

Re: Image File Utilities

Thu Mar 05, 2020 9:34 pm

Dear All, i created a fritzbox share and am able to read and write. Raspberry 3 with buster.

When trying to create an backup image i am getting following error.

Code: Select all

pi@rasp4fhem:~ $ sudo ./image-utils/image-backup

Image file to create? /media/fritzbox/rasp4fhem.img

Initial image file ROOT filesystem size (MB) [3983]? 

Added space for incremental updates after shrinking (MB) [0]? 

Create /media/fritzbox/rasp4fhem.img (y/n)? y
fdisk: cannot open /media/fritzbox/rasp4fhem.img: Unpassender IOCTL (I/O-Control) für das Gerät
sfdisk: cannot open /media/fritzbox/rasp4fhem.img: Unpassender IOCTL (I/O-Control) für das Gerät
./image-utils/image-backup: Zeile 23: * 512: Syntax Fehler: Operator erwartet. (Fehlerverursachendes Zeichen ist \"* 512\").

Unable to create BOOT loop device
could anyone lead me in the right direction how to solve this problem?

best
H.

User avatar
Botspot
Posts: 1631
Joined: Thu Jan 17, 2019 9:47 pm
Location: Texas! :)
Contact: Website

Re: Image File Utilities

Thu Mar 05, 2020 10:08 pm

RonR,
I have not seen a reply from you yet.

Pi Power Tools no longer ships with image-utils from GitHub. Instead the installer downloads directly from your link. So,
Can you confirm that my use of your image-utils is alright with you?
Want to run Minecraft Java on your RPi? Easiest way is with Pi-Apps - just click Install.
https://github.com/Botspot/pi-apps
Pi-Apps also includes Zoom, Visual Studio, Tor browser, Windows 10 theme, and 33 more.
Over 50,000 users!

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Thu Mar 05, 2020 10:54 pm

holle75 wrote:
Thu Mar 05, 2020 9:34 pm
i created a fritzbox share and am able to read and write. Raspberry 3 with buster.

When trying to create an backup image i am getting following error.

Code: Select all

pi@rasp4fhem:~ $ sudo ./image-utils/image-backup

Image file to create? /media/fritzbox/rasp4fhem.img

Initial image file ROOT filesystem size (MB) [3983]? 

Added space for incremental updates after shrinking (MB) [0]? 

Create /media/fritzbox/rasp4fhem.img (y/n)? y
fdisk: cannot open /media/fritzbox/rasp4fhem.img: Unpassender IOCTL (I/O-Control) für das Gerät
sfdisk: cannot open /media/fritzbox/rasp4fhem.img: Unpassender IOCTL (I/O-Control) für das Gerät
./image-utils/image-backup: Zeile 23: * 512: Syntax Fehler: Operator erwartet. (Fehlerverursachendes Zeichen ist \"* 512\").

Unable to create BOOT loop device
could anyone lead me in the right direction how to solve this problem?

Is a /media/fritzbox/rasp4fhem.img file being created? If so, what is its size (you specified 3983 MB, so you need 4GB of free space)?

Maybe the fritzbox o/s doesn't have the IOCTL support needed by fdisk and sfdisk?: Inappropriate IOCTL (I/O-Control) for the device

holle75
Posts: 8
Joined: Thu Mar 05, 2020 9:27 pm

Re: Image File Utilities

Thu Mar 05, 2020 11:37 pm

Hello RonR, the image file is created with 0kb. On the attached HD on the fritzbox is sufficient space (around 170GB).
RonR wrote: Maybe the fritzbox o/s doesn't have the IOCTL support needed by fdisk and sfdisk?: Inappropriate IOCTL (I/O-Control) for the device
To be honest, i have no clue. i wasn´t aware that the receiving media/mounted share needs anything besides being writable. What a pitty. I really liked the idea to use the fritzbox share for scheduled backups.

RonR
Posts: 1884
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Image File Utilities

Fri Mar 06, 2020 12:08 am

holle75 wrote:
Thu Mar 05, 2020 11:37 pm
the image file is created with 0kb. On the attached HD on the fritzbox is sufficient space (around 170GB).

The first thing image-backup does (in the case you presented) is run:

Code: Select all

truncate -s 4176478208 /media/fritzbox/rasp4fhem.img

truncate apparently reported success (there was no "Unable to create image file" message), yet the file created was 0kb instead of:

Code: Select all

root@FreePBX:~# truncate -s 4176478208 /media/fritzbox/rasp4fhem.img
root@FreePBX:~# ls -l /media/fritzbox
total 0
-rw-r--r-- 1 root root 4176478208 Mar  5 17:55 rasp4fhem.img

So even a simple truncate command is not working properly on the fritzbox.
holle75 wrote:
Thu Mar 05, 2020 11:37 pm
i wasn´t aware that the receiving media/mounted share needs anything besides being writable. What a pitty. I really liked the idea to use the fritzbox share for scheduled backups.

What are you using to mount the fritzbox share? I mount Windows shares on Raspbian using '-t cifs' and have no problems using any of the
Image File Utilities
.

holle75
Posts: 8
Joined: Thu Mar 05, 2020 9:27 pm

Re: Image File Utilities

Fri Mar 06, 2020 12:24 am

i also mount the fritzbox share with -t cifs:

Code: Select all

sudo mount -t cifs -o username=xx,password=yyyyyyy,vers=1.0 //fritz.nas/xxxx/TOSHIBA_EXT/rasp4fhem /media/fritzbox
Anyways i´ve seen a lot of discussions about AVM´s buggy/simple implementation of the fritzbox´s nas functionality. I´ll try a normal windows share the next days.

Thank you RonR

Return to “Advanced users”