Paulaner
Posts: 13
Joined: Tue Aug 25, 2020 1:26 pm

Re: USB image writer

Sun Sep 20, 2020 5:33 pm

Hi,

thanks for developing a new promising tool.

Is reading from USB and writing to backup image file in the Windows version already implemented or does it only support to write existing images to USB so far?

When I select my USB stick and click on "read" in Win 10 the tool crashes on my computer...

Best
Paulaner

petecal
Posts: 69
Joined: Fri Feb 15, 2019 10:19 pm
Location: Nashville TN

Re: USB image writer Access Denied

Sun Sep 20, 2020 9:24 pm

Just downloaded usbimager_1.0.1-i686-win-gdi.zip
Run as Administrator.
"Access is denied.
Unable to unmount volumes on device."

First SD card is from my RPi, FAT32
Second SD card is FAT32 written with Windows 8.1

Running Windows 8.1 on desktop.

Pardon me if there was a solution published somewhere in the 8 pages of replyies.
I'm not going to live long enough to do all I want to do.

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Mon Sep 21, 2020 1:30 pm

Paulaner wrote:
Sun Sep 20, 2020 5:33 pm
thanks for developing a new promising tool.
Welcome!
Paulaner wrote:
Sun Sep 20, 2020 5:33 pm
Is reading from USB and writing to backup image file in the Windows version already implemented or does it only support to write existing images to USB so far?
Yes it is. all features are implemented in all versions.
Paulaner wrote:
Sun Sep 20, 2020 5:33 pm
When I select my USB stick and click on "read" in Win 10 the tool crashes on my computer...
That's a new! Please run with the command line flag "-v". A new window should appear with detailed logs. Please open an issue on gitlab and copy'n'paste those messages there. Or you could copy them here. I'll able to tell more when I see the logs.

Cheers,
bzt

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer Access Denied

Mon Sep 21, 2020 1:40 pm

petecal wrote:
Sun Sep 20, 2020 9:24 pm
Just downloaded usbimager_1.0.1-i686-win-gdi.zip
Run as Administrator.
"Access is denied.
Unable to unmount volumes on device."
You're not running it properly as Administrator maybe? You should not see "Access denied" message when running as Administrator.

My other guess would be that another application is actively using the device, that's why the WIN32 API can't dismount and lock the device. Or another application could have already locked the device.

EDIT: I've just noticed, why did you download the 1.0.1??? Download the latest, 1.0.5! If the problem still exists with the latest version, then run "USBImager.exe -v", open an issue on gitlab and paste the output there. I need to see the logs to tell you more.

Cheers,
bzt

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Mon Sep 21, 2020 5:07 pm

Paulaner wrote:
Sun Sep 20, 2020 5:33 pm
When I select my USB stick and click on "read" in Win 10 the tool crashes on my computer...
I've found the problem! Believe it or not, this is actually a gcc bug! I've updated to the latest MSYS2 before I compiled 1.0.5 for Win, and I did not notice something went wrong (there was no warning during compilation).

I've narrowed the problem down and located that the crash was actually happening in stream.c at this line:

Code: Select all

wsprintfW((wchar_t*)str, L"%6" PRIu64 " %s / %" PRIu64 " %s%s%s", (ctx->readSize >> 20), lang[L_MIB], ...
Under WIN32, PRIu64 should be defined as "llu" by the POSIX headers. As it turned out wsprintfW with "llu" formating crashes! That's because gcc 10.2.0 incorrectly concatenates strings. I've replaced that line with

Code: Select all

wsprintfW((wchar_t*)str, L"%6u %s / %u %s%s%s", (unsigned int)(ctx->readSize >> 20), lang[L_MIB], ...
and now it compiles correctly with mingw-gcc 10.2.0. This workaround has a drawback though, the status line will overflow under Win with images above 4 Petabytes. Hopefully never gonna happen with common OS images (and even then only the displayed status text will overflow, the disk still will be written correctly.)

I made sure of it, that all "l" and "ll" formatting and PRIx string concatenation have been removed from wsprintfW arguments, and recompiled the Windows binary. Please download the zip again.

I've checked, the Linux version wasn't affected (however I've experienced other problems with gcc 10.2.0 under Linux, where it "forgot" to add the offset to the base address).

Cheers,
bzt

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Tue Sep 22, 2020 7:43 pm

Dear All,

USBImager 1.0.5 is out! Not much changed.

Changelog
- Added support for ZStandard compressed disk images
- Added an initial version of ARM64 (AArch64) Linux binary
- No MacOS nor Ubuntu deb binary sadly

Unfortunately my machine that I used to run my VMs on proved again that computers work with smoke. When the smoke gets out, the computer stops working. I've tried to recompile 1.0.5 for all supported platforms, and it was almost successful: only Ubuntu deb and MacOS zip missing, I managed to compile for all the other platforms.

A fix for @Paulaner's problem also included, no crash under Windows any more.

Cheers,
bzt

petecal
Posts: 69
Joined: Fri Feb 15, 2019 10:19 pm
Location: Nashville TN

Re: USB image writer Access Denied

Tue Sep 22, 2020 8:21 pm

bzt wrote:
Mon Sep 21, 2020 1:40 pm

EDIT: I've just noticed, why did you download the 1.0.1???
Because that was the first one I saw on gitlab.
I just got the correct one and will try but I hit something else that I need to experiment with.
I think you are right about another application.
It could be a problem with my Windows installation because if I just plug in an SD Card, even one with windows files on it I sometimes get a message about something is wrong with the card.
If I look at properties, tools, Check, it says no problems.
But I can tell it to scan anyway, it says Windows can't repair the drive while it is in use. But I haven't started anything that would be using the Card I just plugged in.
I need to do some more testing.

Thanks
I'm not going to live long enough to do all I want to do.

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer Access Denied

Wed Sep 23, 2020 10:54 am

petecal wrote:
Tue Sep 22, 2020 8:21 pm
Because that was the first one I saw on gitlab.
Okay, I've double checked, the link on README now points to 1.0.5 for sure.
petecal wrote:
Tue Sep 22, 2020 8:21 pm
I just got the correct one and will try but I hit something else that I need to experiment with.
I think you are right about another application.
It could be a problem with my Windows installation because if I just plug in an SD Card, even one with windows files on it I sometimes get a message about something is wrong with the card.
If I look at properties, tools, Check, it says no problems.
But I can tell it to scan anyway, it says Windows can't repair the drive while it is in use. But I haven't started anything that would be using the Card I just plugged in.
If it were a Linux, I'd say use lsof. Not sure about Windows, but maybe Process Monitor can show you which process has open files on the device. It has a utility called Filemon (Just a guess, I don't use Windows.)

Cheers,
bzt

GlowInTheDark
Posts: 1229
Joined: Sat Nov 09, 2019 12:14 pm

Re: USB image writer

Wed Sep 23, 2020 11:43 am

I use ProcExp (process explorer) for that (checking which process has a file open - that is keeping me from deleting that file).

ProcExp is also in the SysInternals suite. Very handy program.
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

Loves Linux; loves to dance.

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Wed Sep 23, 2020 6:45 pm

Hi @Anitapatel,

I'm not exactly sure what was your intention with quoting that old post. But you should have used [ quote ] tags, btw.

Just to be clear, time flies by and things have changed since:

1. Although Raspberry Pi still recommends balenaEtcher on the homepage, they also have created Raspberry Pi Imager since that post. That is the official tool now. If you do not want to create backups and write custom images, just to create a bootable RaspiOS card, then that's the best tool for you.

2. USBImager got udisks2 support since, so it is handling privileged disk access exactly the same way as Pi Imager, balenaEtcher and Gnome Disks (the deb versions compiled with that at least, the zip version is an extremely minimal, dependency-free version that uses disk group permissions and requires nothing else than libX11).

And yes, USBImager is still UNIXy, does one job, and one job only: write/read compressed disk images. It doesn't require net connection in order to do that (paranoid users can relax, no ads nor telemetry possible without net).

And yes, USBImager is still the smallest of all, being as slim as possible. Size varies on platforms, but it's always less than 192K. For example under Windows you'll need to download:
- balenaEtcher installer is ca. 142 Megabytes (and this is just the installer!),
- Pi Imager is ca. 19 Megabytes, and
- USBImager is just 142 Kilobytes (about 134 times smaller than Pi Imager, and 1024 times smaller than etcher installer).

Cheers,
bzt

Paulaner
Posts: 13
Joined: Tue Aug 25, 2020 1:26 pm

Re: USB image writer

Tue Sep 29, 2020 3:01 pm

bzt wrote:
Mon Sep 21, 2020 1:30 pm
That's a new! Please run with the command line flag "-v". A new window should appear with detailed logs. Please open an issue on gitlab and copy'n'paste those messages there. Or you could copy them here. I'll able to tell more when I see the logs.

Cheers,
bzt
I have no explanation, but backup is working now. Maybe I had to many devices on the USB hub...

Thank you very much

Edit: I just saw that you updated to a newer version. It's working :)

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Wed Nov 25, 2020 8:05 pm

Hi,

Good news, the USBImager repo just reached 100 stars! I'd like to say thanks to all of its users, for all the precious feedback and all the support! Special thanks to all the contributors involved, I couldn't have done USBImager without you!

Next goal: have 500 stars and then take on the world!!! :-D :-D :-D
Image

Cheers,
bzt

User avatar
CaptainMidnight
Posts: 188
Joined: Sun Nov 03, 2019 4:32 pm
Location: UK

Re: USB image writer

Wed Nov 25, 2020 10:16 pm

bzt wrote:
Wed Nov 25, 2020 8:05 pm
Hi,

Good news, the USBImager repo just reached 100 stars! I'd like to say thanks to all of its users, for all the precious feedback and all the support!

...... thank you, for creating a multiplatform imaging/backup tool and being open minded to potential project development requests.
"Never get out of the boat." Absolutely goddamn right!
Unless you were goin' all the way...

DarkElvenAngel
Posts: 1228
Joined: Tue Mar 20, 2018 9:53 pm

Re: USB image writer

Thu Nov 26, 2020 2:50 am

bzt wrote:
Wed Nov 25, 2020 8:05 pm
Hi,

Good news, the USBImager repo just reached 100 stars! I'd like to say thanks to all of its users, for all the precious feedback and all the support! Special thanks to all the contributors involved, I couldn't have done USBImager without you!

Next goal: have 500 stars and then take on the world!!! :-D :-D :-D
Image

Cheers,
bzt
Awesome! Great to hear, you already have my star. Btw nice choice on Purple Tentacle now I have to dig out and play Day of the Tentacle.

User avatar
kerry_s
Posts: 1734
Joined: Thu Jan 30, 2020 7:14 pm

Re: USB image writer

Thu Nov 26, 2020 3:06 am

no ubuntu desktop version yet. :D
i normally just use disks, but it would be nice to have a dedicated app.
Attachments
Screenshot from 2020-11-25 17-05-37.resized.png
Screenshot from 2020-11-25 17-05-37.resized.png (183.22 KiB) Viewed 1290 times

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Thu Nov 26, 2020 1:04 pm

DarkElvenAngel wrote:
Thu Nov 26, 2020 2:50 am
Awesome! Great to hear, you already have my star. Btw nice choice on Purple Tentacle now I have to dig out and play Day of the Tentacle.
You're welcome and thanks! Do that, it's a really awesome game! I played it many many times when I was young, and I've recently played it through again with my boy, that's what gave me the idea. :-) I wish there were more ScummVM games like DoTT!
kerry_s wrote:
Thu Nov 26, 2020 3:06 am
no ubuntu desktop version yet. :D
i normally just use disks, but it would be nice to have a dedicated app.
There's a deb version for 1.0.4.

Sadly my machine that I used to run VMs to compile packages went out of order, and since then I can't compile all variants. Although I'm able to compile the deb on my current PC, but as it's not debian based, I have considerably newer versions of the shared libs meaning it's not possible to install the resulting deb on current Ubuntu (glibc is too old for one). That's why I haven't uploaded the deb variant. But the only difference between 1.0.4 and 1.0.5 is the zstd support, everything else is the same.

If someone could compile a 1.0.5 deb version for me under Ubuntu, I would be glad to add it to the binaries repo.

Code: Select all

$ USE_LIBUI=yes USE_UDISKS2=yes make all deb

Cheers,
bzt

donmerch
Posts: 36
Joined: Mon Feb 15, 2016 8:47 pm
Contact: Website

Re: USB image writer

Fri Nov 27, 2020 4:03 pm

I have a 16GB sd card from my Raspberry Pi 4B that I want to archive and save as a file to be used later to make a new card in case I crash my present card. I've installed your version 1.0.5 on my Pi4B. I put the card I want to read into an sd card reader and plugged into the USB jack. I then opened your program and selected the device "sda [15.8 GiB] Multi Flash Reader". I then click on Read and I get this error: "GDBus.Error:org.freedesktop.UDisks2.Error.NotAuthorizedCanObtain: Not authorized to perform operation" so I don't know how to proceed with that.

I also have a MacBook Pro with macOS version Catalina 10.15.7. I downloaded the mac version of your software and got by the security issue of running the program and got it to run. I select "disk2 Generic USB3.0 Card Reader" and then click on Read. I get the message "Permission Denied". I followed your suggestion and added the USBImager to the Full Disk Access list but still get this error message. I then tried your other suggestion to run it in terminal:

Code: Select all

donmerchant@Dons-MacBook-Pro ~ % sudo /Applications/USBImager.app/Contents/MacOS/usbimager
Password:
2020-11-27 09:34:00.052 usbimager[8678:363309] get 0x7fff87a6f148
It then just sits there showing that bottom line. Is it doing anything? I waited a while but did not see anything happening so I pressed control c and got back to the prompt.

Help!!

donmerch
Posts: 36
Joined: Mon Feb 15, 2016 8:47 pm
Contact: Website

Re: USB image writer

Fri Nov 27, 2020 5:00 pm

I was using remote desktop on my mac to the Pi4B. I just figured out that if I run the program on the Pi4B itself that it does seem to work. I'll keep you posted as to the result of that. Still curious as to why it does not work on my mac though.

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Sat Nov 28, 2020 6:13 pm

Hi,
donmerch wrote:
Fri Nov 27, 2020 4:03 pm
I have a 16GB sd card from my Raspberry Pi 4B that I want to archive and save as a file to be used later to make a new card in case I crash my present card. I've installed your version 1.0.5 on my Pi4B. I put the card I want to read into an sd card reader and plugged into the USB jack. I then opened your program and selected the device "sda [15.8 GiB] Multi Flash Reader". I then click on Read and I get this error: "GDBus.Error:org.freedesktop.UDisks2.Error.NotAuthorizedCanObtain: Not authorized to perform operation" so I don't know how to proceed with that.
That UDisks2 error is interesting, because in lack of privilege UDisks2 should prompt you for authorization instead of showing an error message. Try the X11 version of USBImager. That has setgid bit set for disk group, meaning you should be able to access any disk without additional authorization.
donmerch wrote:
Fri Nov 27, 2020 4:03 pm
I also have a MacBook Pro with macOS version Catalina 10.15.7. I downloaded the mac version of your software and got by the security issue of running the program and got it to run. I select "disk2 Generic USB3.0 Card Reader" and then click on Read. I get the message "Permission Denied". I followed your suggestion and added the USBImager to the Full Disk Access list but still get this error message.
That's strange. Are you sure you have added Full Disk Access properly? Have you tried to reboot and see if USBImager is still in the list? Sorry, I'm just guessing here, because I don't have Catalina.
donmerch wrote:
Fri Nov 27, 2020 4:03 pm
I then tried your other suggestion to run it in terminal:

Code: Select all

donmerchant@Dons-MacBook-Pro ~ % sudo /Applications/USBImager.app/Contents/MacOS/usbimager
Password:
2020-11-27 09:34:00.052 usbimager[8678:363309] get 0x7fff87a6f148
It then just sits there showing that bottom line. Is it doing anything? I waited a while but did not see anything happening so I pressed control c and got back to the prompt.
Sadly this is a known issue. I've tried to fix it many times, but the problem is, one of the core functions, CFRunLoopRun() freezes and never returns. I've asked people to try out some alternatives, you can find those here, but I got no response in that matter, so I don't know if they worked. If you could give those alternatives a try, that would be awesome.

I know it's inconvenient, but there's a workaround: manually umount (but don't eject) the device from a Terminal using "diskutil umountDisk /dev/diskX", after that USBImager will work perfectly.

ps.: hopefully within a few months I'll be able to buy a new machine and restore my VMs, so that I'll be able to compile and test MacOS versions again. In the meantime I'll help you any way I can. Please don't hesitate to open an issue on gitlab!

Cheers,
bzt

rrobinet
Posts: 15
Joined: Wed Oct 09, 2019 11:29 pm

Re: USB image writer

Mon Dec 14, 2020 5:00 pm

Thanks so much for this great program!
But I can't run on Mac OSX Big Sur 11.0.1:

rob@Robs-MBP-New:/Users/rob> sudo diskutil unmountDisk /dev/disk2
Unmount of all volumes on disk2 was successful
rob@Robs-MBP-New:/Users/rob> sudo /Applications/USBImager.app/Contents/MacOS/usbimager
2020-12-14 08:57:44.195 usbimager[950:11499] get 0x7fff80445a08
2020-12-14 08:57:54.286 usbimager[950:11499] Warning: Expected min height of view: (<NSButton: 0x7f88ea8798e0>) to be less than or equal to 30 but got a height of 32.000000. This error will be logged once per view in violation.

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Tue Dec 15, 2020 6:52 pm

Thanks for checking out!

It looks like this is a bug in the libui library, as it calculates incorrect size for the button. However it's reported as a warning only (so it should work regardless). I'm afraid you'll have to wait a little until I can get a new machine. Once I get a MacOS build environment again, I'm planning to rewrite the interface in native ObjC to solve some other issues as well, this should be fixed with that too. One of those issues is that sometimes the window is opened in the background, and you can't see it unless you press a Cmd+Tab combo.

Cheers,
bzt

rrobinet
Posts: 15
Joined: Wed Oct 09, 2019 11:29 pm

Re: USB image writer

Tue Dec 15, 2020 7:34 pm

I can use my one Windows 10 laptop until you get that fixed.

In addition, it would be great if you could add an option to automatically shrink the partition size while creating an image file. I create a customized version of Raspian for a group of users by installing and configuring the standard distro on a 16+ GB microSD. While the compressed image you create is small, the expanded version mirrors the size of my 32 GB SD.
I am working through the steps to manually run gparted and dd to shrink the image before creating the img file, but it would be a very useful (and I think unique) feature if your program offered that as an automated option.

tookys
Posts: 1
Joined: Tue Dec 15, 2020 8:36 pm

Re: USB image writer

Tue Dec 15, 2020 8:45 pm

by rrobinet » Tue Dec 15, 2020 7:34 pm
I can use my one Windows 10 laptop until you get that fixed.

In addition, it would be great if you could add an option to automatically shrink the partition size while creating an image file. I create a customized version of Raspian for a group of users by installing and configuring the standard distro on a 16+ GB microSD. While the compressed image you create is small, the expanded version mirrors the size of my 32 GB SD.
I am working through the steps to manually run gparted and dd to shrink the image before creating the img file, but it would be a very useful (and I think unique) feature if your program offered that as an automated option.
@rrobinet I came across this project on github called pi-shrink that is able to reduce the images down to something more usable. Has worked pretty well for the custom images Ive been making.
https://github.com/Drewsif/PiShrink

You have have to run it on a linux machine is the only shortcoming i have with it.




Not sure if this is the right place to put this, but as a feature request i would like to see the imager give the option of a remote-resetable image.
Another project i came across on github is the "RaspberryPi-Factory-Reset"
https://github.com/shivasiddharth/Raspb ... tory-Reset
It creates 2 copies of a provided OS, and merges it into one image. Once booted it allows you to run a 1 line command to tell the pi to reboot itself. I've combined it with pi-shrink to help keep the images small.

This works well for me, I have pi's I keep at a remote location that I like to remote into and use. The custom image I use has ssh enabled, password changes, wifi log-in info so it reboots ready to go.


Current issues is that sometimes when re-flashing (pi4) the wifi module will stop working, but otherwise it works fine as long as its hard wired.

bzt
Posts: 603
Joined: Sat Oct 14, 2017 9:57 pm

Re: USB image writer

Sat Dec 19, 2020 5:05 pm

rrobinet wrote:
Tue Dec 15, 2020 7:34 pm
I can use my one Windows 10 laptop until you get that fixed.
Now two users reported that with build 12 the umount is working perfectly now (on High Sierra and Catalina). You should give it a try too!
rrobinet wrote:
Tue Dec 15, 2020 7:34 pm
In addition, it would be great if you could add an option to automatically shrink the partition size while creating an image file.
Sorry, but that's not an option. Shrinking the partition means first you have to shrink the filesystem. Adding file system drivers is a huge task and goes against the UNIX philosophy. USBImager (just like dd) does not interpret the image at all, it's only job is to read and write images to drives.
rrobinet wrote:
Tue Dec 15, 2020 7:34 pm
I am working through the steps to manually run gparted and dd to shrink the image
This way you're going to create a corrupted file system. You also must shrink the file system on the partition, see resize2fs (only works with ext file system family), and fatresize (only FAT file systems), ntfsresize (NTFS only) etc. There's no common tool that I know of, only proprietary solutions (like Partition Magic for example). You could try the tool @tookys recommended as well.

Here's a step by step tutorial that might be useful to you. It uses resize2fs and fdisk (note: "cat cmds.txt | fdisk" works perfectly, so it is easily scriptable).

Cheers,
bzt

rrobinet
Posts: 15
Joined: Wed Oct 09, 2019 11:29 pm

Re: USB image writer

Sat Dec 19, 2020 7:04 pm

I found https://github.com/Drewsif/PiShrink, a script which shrinks the partitions of a pi microSD and sets it up to resize itself upon first run.
Combined with your script, I now can generate the custom Pi microSDs I want. Thanks

Return to “General discussion”