GlowInTheDark
Posts: 390
Joined: Sat Nov 09, 2019 12:14 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 2:02 pm

incognitum wrote:
Sat Feb 22, 2020 1:48 pm
GlowInTheDark wrote:
Sat Feb 22, 2020 1:14 am
I think people are upset about the fact that it (apparently) requires a network connection to run - and can be seen to be doing network communication while running (if you use something like Wireshark or tcpdump on it).
And does some "analytics", like phoning home the path of the image you are writing: https://github.com/balena-io/etcher/blo ... er.ts#L263

Like "c:\users\your name\Downloads\image.img"
Ouch!

Yeah, I'd say the bloom is off the rose on this "etcher" thing.
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 3:12 pm

GlowInTheDark wrote:
Sat Feb 22, 2020 12:31 pm
Note that I'm never quite sure what is and isn't installed-by-default in Raspbian. Several perfectly obvious, for all practical purposes essential, utilities are not installed-by-default (I always have to manually install them when setting up a new system). So I'm not sure if "unzip" is always there. gzip, of course, will always be there.
unzip is included in Raspbian Lite
The command "dpkg -l" lists all the installed packages.

Code: Select all

$ dpkg -l | grep unzip
ii  unzip       6.0-23+deb10u1         armhf        De-archiver for .zip files
$ 
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 3:30 pm

pica200 wrote:
Sat Feb 22, 2020 12:35 pm
I prefer dd over cp/cat because you can control the block size. NAND flash has large pages these days so it's a good idea to write in blocks a multiple of the page size. It's faster and causes less wear.
What block size do you suggest?

cp and cat use 128KB blocks which seem reasonable to me.

I avoid very large blocks because you want the writes to proceed at the time as the reads (for the best speed).
Clearly the first write cannot start until the first block is read.
If you set the block size to be close to the size of the file, then it is terribly slow!

I suspect that the 128KB was chosen because it is related to the amount of kernel prefetch.
Which is doubled because the cp command sets FADV_SEQUENTIAL.
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

pica200
Posts: 216
Joined: Tue Aug 06, 2019 10:27 am

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 4:17 pm

That doesn't really matter. It will pre-read blocks in advance as long as you don't tell it not to. So it will keep the destination device busy. You can try it yourself. You will see there is not a huge difference between 128 KiB and 1 or 4 MiB ;) The later is what i usually use for block sizes and it works great. I can easily get close to the advertised speed of some microSD cards using a PCIe based card reader (or USB 3.0). The real archiveable speed of course depends on the card.

Another advantage of big block sizes is less system call overhead but it only really shows with weak CPUs. Flash page sizes are around 128 KiB or more these days. With a PCIe based reader you can read out what the microSD reports.

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 4:48 pm

pica200 wrote:
Sat Feb 22, 2020 4:17 pm
That doesn't really matter. It will pre-read blocks in advance as long as you don't tell it not to. So it will keep the destination device busy. You can try it yourself. You will see there is not a huge difference between 128 KiB and 1 or 4 MiB ;) The later is what i usually use for block sizes and it works great. I can easily get close to the advertised speed of some microSD cards using a PCIe based card reader (or USB 3.0). The real archiveable speed of course depends on the card.

Another advantage of big block sizes is less system call overhead but it only really shows with weak CPUs. Flash page sizes are around 128 KiB or more these days. With a PCIe based reader you can read out what the microSD reports.
Interesting.
I have had bs=4M fail in the past whereas bs=1M worked fine.
1MiB sounds good. I guess it should be bigger than any flash page size, and shouldn't take too long to read the first block.
and 1M is quicker to type than 128K :)

Just for interest, you can see what the tools do with strace.
For example:

strace cp bigfile /dev/null

Note that cp does:

fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL)

Its supposed to double the readahead window size.
dd doesn't do this.
It is the readahead (prefetch) that allows reading and writing to proceed in parallel.
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

incognitum
Posts: 453
Joined: Tue Oct 30, 2018 3:34 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 5:07 pm

jahboater wrote:
Sat Feb 22, 2020 4:48 pm
I have had bs=4M fail in the past whereas bs=1M worked fine.
The image file size probably wasn't dividable by 4 MB then.
dd doesn't like partial blocks.

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 5:51 pm

This is all very interesting.
I used blockdev to get the info for the Pi's SD card (64GB Sandisk A1 class)

// block size in bytes
[email protected]:~ $ sudo blockdev --getbsz /dev/mmcblk0
4096
// physical block (sector) size
[email protected]:~ $ sudo blockdev --getpbsz /dev/mmcblk0
512
// logical sector size in bytes
[email protected]:~ $ sudo blockdev --getss /dev/mmcblk0
512
// readahead (in 512-byte sectors)
[email protected]:~ $ sudo blockdev --getfra /dev/mmcblk0
256
// minimum I/O size
[email protected]:~ $ sudo blockdev --getiomin /dev/mmcblk0
512
// optimal I/O size
[email protected]:~ $ sudo blockdev --getioopt /dev/mmcblk0
0
// full report
[email protected]:~ $ sudo blockdev --report /dev/mmcblk0
RO RA SSZ BSZ StartSec Size Device
rw 256 512 4096 0 63864569856 /dev/mmcblk0
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 5:59 pm

Earlier I speculated that cp's choice of 128KB for its block size was somehow related to the kernels readahead.

// readahead (in 512-byte sectors)
[email protected]:~ $ sudo blockdev --getfra /dev/mmcblk0
256

That is 128KB !! cool.

You learn something every day here.
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

incognitum
Posts: 453
Joined: Tue Oct 30, 2018 3:34 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 6:45 pm

jahboater wrote:
Sat Feb 22, 2020 5:51 pm
This is all very interesting.
I used blockdev to get the info for the Pi's SD card (64GB Sandisk A1 class)
For writes it may be more interesting to see if your card reports the erase block size, e.g.:

Code: Select all

$ cat /sys/block/mmcblk0/device/preferred_erase_size 
4194304

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 7:03 pm

incognitum wrote:
Sat Feb 22, 2020 6:45 pm
jahboater wrote:
Sat Feb 22, 2020 5:51 pm
This is all very interesting.
I used blockdev to get the info for the Pi's SD card (64GB Sandisk A1 class)
For writes it may be more interesting to see if your card reports the erase block size, e.g.:

Code: Select all

$ cat /sys/block/mmcblk0/device/preferred_erase_size 
4194304
Its the same as yours, 4MiB

Does that mean the optimal write block size is 4M ?

I hope the writing starts at the beginning of the disk, so its aligned with the erase blocks.
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

incognitum
Posts: 453
Joined: Tue Oct 30, 2018 3:34 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 7:16 pm

jahboater wrote:
Sat Feb 22, 2020 7:03 pm
Does that mean the optimal write block size is 4M ?
If you are using unbuffered IO (e.g. oflag=direct with dd) probably, yes.
If not, keep in mind that the kernel will buffer your writes, so will likely already write much larger chunks to storage than the block size you specified in your program.
So in practice the difference you experience will not be so impressive.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25462
Joined: Sat Jul 30, 2011 7:41 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 8:00 pm

I use balenaEtcher. The fact the people are writing multiple posts above about block sizes when using dd should be a good indicator that there is a market for an easy to use application!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

fruitoftheloom
Posts: 22069
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 8:23 pm

jamesh wrote:
Sat Feb 22, 2020 8:00 pm
I use balenaEtcher. The fact the people are writing multiple posts above about block sizes when using dd should be a good indicator that there is a market for an easy to use application!

+1 Etcher does what is says it does, do I care about Privacy, no more than when using various Closed Source Software.
Retired disgracefully.....
......to an uncomplicated life !

Rather than negativity think outside the box !

pica200
Posts: 216
Joined: Tue Aug 06, 2019 10:27 am

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 9:03 pm

That's your choice. Reminds me of "I have nothing to hide.".

I don't think they need most of the data it phones home. The ads are not so much of a problem.

bzt
Posts: 518
Joined: Sat Oct 14, 2017 9:57 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sat Feb 22, 2020 9:53 pm

jamesh wrote:
Sat Feb 22, 2020 8:00 pm
I use balenaEtcher. The fact the people are writing multiple posts above about block sizes when using dd should be a good indicator that there is a market for an easy to use application!
Why do you use it? Do they pay you or something? I'm pretty sure you are more than capable of correctly specifying the dd utility's arguments. If we put aside that etcher violates GDPR, and collects personal information it is still a bloody webapp, and HUNDREDS of Megabytes! That app is bigger than 90% of the OS images I write on USB sticks! Who knows what malware is hidden in those megabytes? There's already at least one spyware we know about...

About an easy to use alternative, USBImager is just 128K, has native GUI on Windows, MacOSX and also on Linux, does not need a network connection, and does not spy on you. It can read and write compressed images, and it handles block sizes correctly. Just for the records it uses 1M blocks, and it rounds the last block up to be multiple of 512 bytes (that's a Linux restriction), filling the area with zeros. So all these things in ca. 128K! Furthermore, it is a Free and Open Source project, if in doubt, you can make sure of it that it's malware and spyware free.

Cheers,
bzt

fruitoftheloom
Posts: 22069
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 7:54 am

bzt wrote:
Sat Feb 22, 2020 9:53 pm
jamesh wrote:
Sat Feb 22, 2020 8:00 pm
I use balenaEtcher. The fact the people are writing multiple posts above about block sizes when using dd should be a good indicator that there is a market for an easy to use application!
Why do you use it? Do they pay you or something? I'm pretty sure you are more than capable of correctly specifying the dd utility's arguments. If we put aside that etcher violates GDPR, and collects personal information it is still a bloody webapp, and HUNDREDS of Megabytes! That app is bigger than 90% of the OS images I write on USB sticks! Who knows what malware is hidden in those megabytes? There's already at least one spyware we know about...

About an easy to use alternative, USBImager is just 128K, has native GUI on Windows, MacOSX and also on Linux, does not need a network connection, and does not spy on you. It can read and write compressed images, and it handles block sizes correctly. Just for the records it uses 1M blocks, and it rounds the last block up to be multiple of 512 bytes (that's a Linux restriction), filling the area with zeros. So all these things in ca. 128K! Furthermore, it is a Free and Open Source project, if in doubt, you can make sure of it that it's malware and spyware free.

Cheers,
bzt

Until a Lawyer is involved, there can not be a definitive answer, so far all the concerns are people's perspectives.

If it was such a "big issue" there would be more Web Searches flagging this up ??

It is great we have alternatives, but deriding a product to promote an alternative leaves a sour taste.


Maybe Google's ChromeOS SD Card Creator is a good alternative ?

https://www.raspberrypi.org/documentati ... hromeos.md

Err no that would send Analytics to Alphabet !!! but at least it should be available in Chromium based Browsers ???
Retired disgracefully.....
......to an uncomplicated life !

Rather than negativity think outside the box !

jahboater
Posts: 5216
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 8:13 am

fruitoftheloom wrote:
Sun Feb 23, 2020 7:54 am
It is great we have alternatives, but deriding a product to promote an alternative leaves a sour taste.


Maybe Google's ChromeOS SD Card Creator is a good alternative ?

https://www.raspberrypi.org/documentati ... hromeos.md

Err no that would send Analytics to Alphabet !!! but at least it should be available in Chromium based Browsers ???
Whats wrong with @bzt's https://gitlab.com/bztsrc/usbimager ? If you need a GUI it looks very good.

If you don't need a GUI then the standard Linux tools (for example "unzip -p") need no installation, are simple enough, don't require an internet connection to write home, and are open source. Upgrade to dd if you want to fine tune the performance.
"We are in the beginning of a mass extinction, and all you can talk
about is money and fairy tales of eternal economic growth."
- Greta Thunberg

fruitoftheloom
Posts: 22069
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 8:40 am

jahboater wrote:
Sun Feb 23, 2020 8:13 am
fruitoftheloom wrote:
Sun Feb 23, 2020 7:54 am
It is great we have alternatives, but deriding a product to promote an alternative leaves a sour taste.


Maybe Google's ChromeOS SD Card Creator is a good alternative ?

https://www.raspberrypi.org/documentati ... hromeos.md

Err no that would send Analytics to Alphabet !!! but at least it should be available in Chromium based Browsers ???
Whats wrong with @bzt's https://gitlab.com/bztsrc/usbimager ? If you need a GUI it looks very good.

If you don't need a GUI then the standard Linux tools (for example "unzip -p") need no installation, are simple enough, don't require an internet connection to write home, and are open source. Upgrade to dd if you want to fine tune the performance.

There is as far as I can tell nothing wrong with bzt's software, just do not like the way he uses a supposed breach of privacy in Etcher to promote.


MacOS actually has an alternative to Etcher in Apple Pi-Baker:

https://www.tweaking4all.com/software/m ... -baker-v2/


So alternatives galore if one believes Balena is a baddy :roll:

https://www.raspberrypi.org/documentati ... /README.md
Retired disgracefully.....
......to an uncomplicated life !

Rather than negativity think outside the box !

User avatar
B.Goode
Posts: 9611
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 9:24 am

jahboater wrote:
Sun Feb 23, 2020 8:13 am

Whats wrong with @bzt's https://gitlab.com/bztsrc/usbimager ? If you need a GUI it looks very good.


At present usbimager is the work of one lone developer, and is stated by that developer to have 'beta' status. Perhaps a little premature to be promoting it as a panacea?

fruitoftheloom
Posts: 22069
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Google ChromeOS Recovery alternative to Etcher

Sun Feb 23, 2020 9:31 am

ChromeOS devices have a SD Card Creation Utility, but it can also be used to create Bootable SD Cards.


Though it is a Google Product and may send analytics to Alphabet (can be turned off in permissions), it does work really well in Google Chrome and Chromium based Web Browsers (except the Blink based Microsoft Edge Browser) as an Extension available from Google Web Store.


I used the Brave Web Browser https://brave.com in Windows 10 x86-64 Professional using the steps outlined here:

https://www.raspberrypi.org/documentati ... hromeos.md


Like Etcher it can use the .zip file and verifies, but as an extra this Software also has an Erase SD Card Option.
Retired disgracefully.....
......to an uncomplicated life !

Rather than negativity think outside the box !

mikerr
Posts: 2814
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 11:27 am

You don't even need a computer to prepare an SD card for the pi, just your android phone:

viewtopic.php?f=63&t=170856
Android app - Raspi Card Imager - download and image SD cards - No PC required !

GlowInTheDark
Posts: 390
Joined: Sat Nov 09, 2019 12:14 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 12:09 pm

just your? android phone:
What android phone???
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

bzt
Posts: 518
Joined: Sat Oct 14, 2017 9:57 pm

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 12:10 pm

fruitoftheloom wrote:
Sun Feb 23, 2020 7:54 am
Until a Lawyer is involved, there can not be a definitive answer, so far all the concerns are people's perspectives.
That's wrong. No need for a lawyer to tell it is sending data about its users without their consent, which is something that GDPR explicitly forbids. It is the other way around, they need a lawyer to prove they haven't broke any laws on purpose (which would be difficult as etcher does send these even if the user explicitly turned this feature off, that's an open ticket as we speak. It is hard to believe to be a simple bug when it generates profit for the company).
fruitoftheloom wrote:If it was such a "big issue" there would be more Web Searches flagging this up ??

It is great we have alternatives, but deriding a product to promote an alternative leaves a sour taste.


Maybe Google's ChromeOS SD Card Creator is a good alternative ?

https://www.raspberrypi.org/documentati ... hromeos.md

Err no that would send Analytics to Alphabet !!! but at least it should be available in Chromium based Browsers ???
Anything that reads LOCAL files and writes to LOCAL devices and needs internet for that is just bad. Period.
B.Goode wrote:At present usbimager is the work of one lone developer, and is stated by that developer to have 'beta' status. Perhaps a little premature to be promoting it as a panacea?
We probably have different idea about the 'beta' status. Let me phrase it this way: it is fully functional, it works all the times for its author, and it needs to be tested on wider range of machines so that all possible bugs can be located. Therefore it is at the stage where it should be promoted; so that users became aware of it, and can test it out for themselves. But if this bothers you, call the current version release candidate.

About one developer, true, but a very talented and experienced one (everybody says that). Furthermore the entire source is available, it is Open Source, it is not tied to anybody nor to any company. If it happens that tomorrow the tram cuts me in half, all the information accessible and anyone could pick up the project, there's absolutely no disadvantage in currently having one developer only.

Cheers,
bzt

GlowInTheDark
Posts: 390
Joined: Sat Nov 09, 2019 12:14 pm

GDPR?

Sun Feb 23, 2020 12:15 pm

No need for a lawyer to tell it is sending data about its users without their consent, which is something that GDPR explicitly forbids.
If this is true, then won't basically all Internet services (all the things we know and love) going to become illegal in the EU?

They all do that (communicate with the mother ship). Everybody knows that.

People like their Internet services. They are not going to stand for them all being made illegal.

And, just to be clear on this, all Internet services have these attributes:

1) Your only recourse is not to use the service. In the case under discussion, this means that everyone is free to not use "Etcher". Sensible people do exactly that (refuse to use it).

2) There is an EULA somewhere (that you effectively "signed") - that says that. That the service is free to do whatever they want and that you are free not to use the service.

It is as simple as that.
GitD's list of things that are not ready for prime time:
1) IPv6
2) 64 bit OSes
3) USB 3
4) Bluetooth

User avatar
B.Goode
Posts: 9611
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: What is a good replacement for Etcher (ads and ignores privacy)

Sun Feb 23, 2020 12:26 pm

bzt wrote:
Sun Feb 23, 2020 12:10 pm


B.Goode wrote:At present usbimager is the work of one lone developer, and is stated by that developer to have 'beta' status. Perhaps a little premature to be promoting it as a panacea?
We probably have different idea about the 'beta' status. Let me phrase it this way: it is fully functional, it works all the times for its author, and it needs to be tested on wider range of machines so that all possible bugs can be located. Therefore it is at the stage where it should be promoted; so that users became aware of it, and can test it out for themselves. But if this bothers you, call the current version release candidate.



I thought that @bzt wrote this:
bzt wrote:
Sun Feb 23, 2020 4:07 am

DarkElvenAngel wrote:One thing I would add is a version number somewhere.
You mean visible? Somewhere on the interface? I can certainly do that. Just keep in mind until I got the first stable release it'll be 0.0.1 (suggests beta).


Return to “General discussion”