OS download is rather useless

[hippy]

Use apt-cacher-ng so that rebuilding images and/or updating already-running Pi systems only hits the internet for the first download of each package.

That's a good suggestion for someone who already has a Pi and is likely installing the same thing on every additional Pi they acquire. I had a mixed experience when I used that. It was fine when there were few or small updates, but I had major problems at times with 'apt' failing someway through and needing a reboot to get it to succeed. I never fully investigated so that may have been a configuration issue, network problems, might have just been my bad luck, but I ended up uninstalling it as more trouble than it was worth. YMMV.

[PeterO]

Just downloaded the latest RPI OS, which wasn't too bad (6 or 7 gig file). However the pi installer was taking forever, literally to write to the SD card (like it would be hour++...why why)...so I aborted & used Balena etcher --took maybe 7 minutes to write & 5 to verify.

Now the rpi is running , but starts off downloading the updates...WHICH IS TAKING LONGER THAN THE OS I just downloaded!!!. If the OS is so out of date that the updates are more than the OS download, it seem like the download section is being allowed to be kept woefully out of date.
This is worse than downloading the OS twice... why bother if it is all going to be replaced with update?

Download OS and program sdcard with rpi-imager on a PI4 : 10 mins
Update new image on sdcard : 5 mins

There is not a problem with the process, but there may be with your internet access !

PeterO

[bls]

Use apt-cacher-ng so that rebuilding images and/or updating already-running Pi systems only hits the internet for the first download of each package.

That's a good suggestion for someone who already has a Pi and is likely installing the same thing on every additional Pi they acquire. I had a mixed experience when I used that. It was fine when there were few or small updates, but I had major problems at times with 'apt' failing someway through and needing a reboot to get it to succeed. I never fully investigated so that may have been a configuration issue, network problems, might have just been my bad luck, but I ended up uninstalling it as more trouble than it was worth. YMMV.

My experience has been exactly the opposite. I started running it earlier this year, and have NEVER had a problem with it. I just checked the stats page, and it claims to have served 2.2GB, while only downloading 370MB. These are not lifetime stats...when I have gone through periods of multiple SD Card rebuilds I've seen well over 6GB served with only a few hundred MB downloaded.

[hippy]

But also, this exists - http://downloads.raspberrypi.org/nightlies/

I don't believe anyone is asking for the latest leading-edge of everything; just a download which reflects what they'd have after installing the published download and updating, so, if those are that, it seems the perfect solution.

Perhaps the existence of nightlies need more promotion ?

[jamesh]

I am just updating an almost brand new Mac Mini to BigSur. It's a 12GB download.

The amount of GB's downloaded to update recently installed Pi images is tiny compared with the amount of GB downloaded for iPlayer or Netflix or Amazon Prime.

[jamesh]

But also, this exists - http://downloads.raspberrypi.org/nightlies/

I don't believe anyone is asking for the latest leading-edge of everything; just a download which reflects what they'd have after installing the published download and updating, so, if those are that, it seems the perfect solution.

Perhaps the existence of nightlies need more promotion ?

They are effectively beta's so should be treated as such - promoting too heavily means people who should NOT be using them end up using them, which can end up detrimental overall.

[rpdom]

I don't see anything wrong with the current situation. It's pretty much the same as if you run a Debian or Ubuntu system. Download and install the latest image, then apply updates to the current state.

I'm not sure how often they release pre-built images these days, but it didn't use to be more than every few months.

Updating the image automatically every day (the daily builds) isn't a problem, but checking that it all worked and won't cause errors is.

FWIW I expect other OSen have a similar situation.

[ejolson]

I am just updating an almost brand new Mac Mini to BigSur. It's a 12GB download.

The amount of GB's downloaded to update recently installed Pi images is tiny compared with the amount of GB downloaded for iPlayer or Netflix or Amazon Prime.

If that's one of those ARM-based Apples, would you mind running the Pi pie chart program from the thread

viewtopic.php?f=63&t=227177

and posting results there? I think it would be interesting to use the equivalent of taskset to restrict the parallel runs to

• the 4 big cores
• the 4 little cores
• all of them

for comparison.

Back on topic, rather than looking back over one's shoulder to compare how others are doing, a race may be easier to win by always focusing on improvement and looking forward. Maybe that's why Apple marketing compared the M1 to other Macs and not the Pi.

[trejan]

If that's one of those ARM-based Apples, would you mind running the Pi pie chart program from the thread and posting results there?

jamesh has an Intel Mac Mini if he's updating it to Big Sur since the Apple silicon ones are only supported starting with that version.

[ejolson]

If that's one of those ARM-based Apples, would you mind running the Pi pie chart program from the thread and posting results there?

jamesh has an Intel Mac Mini if he's updating it to Big Sur since the Apple silicon ones are only supported starting with that version.

Just like the Buster version of Raspberry Pi OS is still Buster after it has updated, my understanding is that the Mac Mini came with Big Sur and that Big Sur also needed a significant amount of updating.

The difference here is that Big Sur came pre installed on the SSD, whereas the Raspberry Pi image is downloaded on demand from the network. In the latter case, there are no inventory and logistics considerations that prevent starting with an up to date image in the first place.

[Heater]

It does seem you live a charmed life and rarely experience the problems and issues others do.

My life has been anything but "charmed", I'll spare you the details, except as far as Raspi downloads it seems.

I have a cable internet connection which is fast enough, but the whole process does feel like it takes forever at times. Probably it's not so much the download as the installation time, especially when it involves a kernel upgrade with its large number of diversions and removals. Even without it can still be a lengthy process.

Perhaps we should quantify "lengthy process"? A minute, an hour, a day, a week.... How long is too long?

Ten minutes is enough time to get the kettle on and have tea ready when it is done. Some hours is time to have a good nights sleep and wake up to a working system in the morning.

On the other hand, when Win 10 on a laptop decides it has to update now, when I'm working outdoors in the freezing cold and time is money that is a major pain and expense.

I do appreciate that such delays are multiplied by the number of users trying to do the thing. It add up to multiple human lifetimes wasted.

On the other hand, I'm sure we have all spent countless hours trying to get this and that to work in our ever churning software world. Actual download times pale into insignificance by comparison.

[swampdog]

I've got a 100Mb internet connection. I'm actually going to do this on an rpi4 to a usb3 thumb drive (plugged into blue usb3 connector). You can look at the timings and figure out where your problem lies. First culprit would be the sdcard. Second, the sdcard dongle. Third, windows - in which case you could boot your PC off a linux live image but I'm getting sidetracked.

First. Download the image..

Code: Select all

foo@pi18:/wrk/P $ wget https://downloads.raspberrypi.org/raspios_armhf/images/raspios_armhf-2020-08-24/2020-08-20-raspios-buster-armhf.zip
--2020-11-27 18:43:18--  https://downloads.raspberrypi.org/raspios_armhf/images/raspios_armhf-2020-08-24/2020-08-20-raspios-buster-armhf.zip
Resolving downloads.raspberrypi.org (downloads.raspberrypi.org)... 46.235.230.122, 176.126.240.84, 46.235.231.151, ...
Connecting to downloads.raspberrypi.org (downloads.raspberrypi.org)|46.235.230.122|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1187281347 (1.1G) [application/zip]
Saving to: ‘2020-08-20-raspios-buster-armhf.zip’

2020-08-20-raspios- 100%[===================>]   1.11G  4.79MB/s    in 4m 3s   

2020-11-27 18:47:21 (4.66 MB/s) - ‘2020-08-20-raspios-buster-armhf.zip’ saved [1187281347/1187281347]

Uncompress it..

Code: Select all

foo@pi18:/wrk/P $ time unzip 2020-08-20-raspios-buster-armhf.zip 
Archive:  2020-08-20-raspios-buster-armhf.zip
  inflating: 2020-08-20-raspios-buster-armhf.img  

real	1m12.848s
user	0m55.316s
sys	0m16.506s

Write it..

Code: Select all

foo@pi18:/wrk/P $ sudo su
root@pi18:/wrk/P# time dd if=./2020-08-20-raspios-buster-armhf.img of=/dev/sdb progress=status
dd: unrecognized operand ‘progress=status’
Try 'dd --help' for more information.

real	0m0.011s
user	0m0.008s
sys	0m0.002s
root@pi18:/wrk/P# time dd if=./2020-08-20-raspios-buster-armhf.img of=/dev/sdb status=progress
3813735424 bytes (3.8 GB, 3.6 GiB) copied, 329 s, 11.6 MB/s
7462912+0 records in
7462912+0 records out
3821010944 bytes (3.8 GB, 3.6 GiB) copied, 331.109 s, 11.5 MB/s

real	5m31.117s
user	0m6.387s
sys	1m24.380s
root@pi18:/wrk/P# time sync

real	0m0.015s
user	0m0.008s
sys	0m0.002s

Code: Select all

root@pi18:/wrk/P# mount /dev/sdb1 /mnt/tmp
root@pi18:/wrk/P# touch /mnt/tmp/ssh
root@pi18:/wrk/P# umount /mnt/tmp
root@pi18:/wrk/P# exit
foo@pi18:/wrk/P $

Now I'm removing the thumb drive and putting into another rpi4 and booting off it Takes a couple of minutes to resize, reboot, then bring up the gui. Now I'll ssh into it..

Code: Select all

foo@sdu ~/usr/src/rpi/rpinew $ sshi pi@192.168.1.41
Warning: Permanently added '192.168.1.41' (ECDSA) to the list of known hosts.
pi@192.168.1.41's password: 
Linux raspberrypi 5.4.51-v7l+ #1333 SMP Mon Aug 10 16:51:40 BST 2020 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug 20 11:54:29 2020

SSH is enabled and the default password for the 'pi' user has not been changed.
This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.


Wi-Fi is currently blocked by rfkill.
Use raspi-config to set the country before use.

pi@raspberrypi:~ $ uptime
 19:07:52 up 2 min,  3 users,  load average: 2.20, 1.61, 0.66
pi@raspberrypi:~ $ df -hP
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        58G  2.9G   53G   6% /
devtmpfs        1.8G     0  1.8G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G   17M  1.9G   1% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1       253M   54M  199M  22% /boot
tmpfs           383M     0  383M   0% /run/user/1000

Grab the update info..

Code: Select all

pi@raspberrypi:~ $ time sudo apt-get update 2>&1 | tee /tmp/z
Get:1 http://archive.raspberrypi.org/debian buster InRelease [32.6 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [336 kB]
Get:5 http://raspbian.raspberrypi.org/raspbian buster/contrib armhf Packages [58.7 kB]
Fetched 13.4 MB in 13s (1,045 kB/s)
Reading package lists...

real	0m16.298s
user	0m7.876s
sys	0m1.851s

Do the updates..

Code: Select all

pi@raspberrypi:~ $ time sudo apt-get full-upgrade 2>&1 | tee /tmp/zz
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
The following packages will be upgraded:
  agnostics alacarte arandr base-files bind9-host bluez-firmware
  chromium-browser chromium-browser-l10n chromium-codecs-ffmpeg-extra
  dphys-swapfile firmware-atheros firmware-brcm80211 firmware-libertas
  firmware-misc-nonfree firmware-realtek libbind9-161 libdns-export1104
  libdns1104 libegl-mesa0 libexif12 libfm-data libfm-extra4 libfm-gtk-data
  libfm-gtk4 libfm-modules libfm4 libfreetype6 libfreetype6-dev libgbm1
  libgl1-mesa-dri libglapi-mesa libgles2-mesa libglx-mesa0 libgs9
  libgs9-common libgssapi-krb5-2 libgssdp-1.0-3 libgupnp-1.0-4
  libisc-export1100 libisc1100 libisccc161 libisccfg163
  libjavascriptcoregtk-4.0-18 libk5crypto3 libkrb5-3 libkrb5support0
  libldap-2.4-2 libldap-common liblirc-client0 liblwres161 libqt5concurrent5
  libqt5core5a libqt5dbus5 libqt5gui5 libqt5network5 libqt5printsupport5
  libqt5sql5 libqt5sql5-sqlite libqt5widgets5 libqt5xml5 libraspberrypi-bin
  libraspberrypi-dev libraspberrypi-doc libraspberrypi0 libvlc-bin libvlc5
  libvlccore9 libwebkit2gtk-4.0-37 libx11-6 libx11-data libx11-xcb1 libzmq5
  lxpanel lxpanel-data lxplug-bluetooth lxplug-cputemp lxplug-ejecter
  lxplug-magnifier lxplug-network lxplug-ptbatt lxplug-volume mesa-va-drivers
  mesa-vdpau-drivers pcmanfm pipanel pishutdown pprompt qt5-gtk-platformtheme
  raspberrypi-bootloader raspberrypi-kernel raspberrypi-sys-mods raspi-config
  rc-gui rp-bookshelf rp-prefapps rpi-chromium-mods rpi-eeprom thonny tzdata
  vlc vlc-bin vlc-data vlc-l10n vlc-plugin-base vlc-plugin-notify
  vlc-plugin-qt vlc-plugin-samba vlc-plugin-skins2 vlc-plugin-video-output
  vlc-plugin-video-splitter vlc-plugin-visualization xserver-common
  xserver-xorg-core
113 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 306 MB of archives.
After this operation, 25.6 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://archive.raspberrypi.org/debian buster/main armhf chromium-browser-l10n all 84.0.4147.105-rpt3 [3,419 kB]
Get:2 http://archive-bm.raspbian.org/raspbian buster/main armhf base-files armhf 10.3+rpi1+deb10u6 [70.1 kB]
[snip]
Processing triggers for install-info (6.5.0.dfsg.1-4+b1) ...
Processing triggers for desktop-file-utils (0.23-4) ...
Processing triggers for initramfs-tools (0.133+deb10u1) ...
Processing triggers for libvlc-bin:armhf (3.0.11-0+deb10u1+rpt3) ...

real	16m30.881s
user	2m49.681s
sys	1m12.748s

pi@raspberrypi:~ $ sudo reboot
pi@raspberrypi:~ $ Connection to 192.168.1.41 closed by remote host.
Connection to 192.168.1.41 closed.
foo@sdu ~/usr/src/rpi/rpinew $

Now the rpi is ready to use. I'm too lazy to total up all the times manually!

[hippy]

I don't see anything wrong with the current situation. It's pretty much the same as if you run a Debian or Ubuntu system.

I would suggest that simply shows they are just as bad and even worse. I've never had much truck with 'we're no worse than someone else' when one can be better. That doesn't invalidate any complaint or desire for improvement.

[hippy]

pi@raspberrypi:~ $ time sudo apt-get full-upgrade 2>&1 | tee /tmp/zz

real 16m30.881s

And how much slower would that have been on a Pi Zero W ?

About four times in my experience, so that's an hour-plus.

[Heater]

I would suggest that simply shows they are just as bad and even worse. I've never had much truck with 'we're no worse than someone else' when one can be better. That doesn't invalidate any complaint or desire for improvement.

I'm not sure what it is you are getting at there.

Downloading, installing, updating Debian or Ubuntu takes time. My experience of downloading, updating, installing other operating systems has been very similar. Including various versions of Windows. I have no experience of Mac.

Whilst improvement is always welcome, I'm not sure what you are suggesting the creators of such OS distributions should actually do. From the shoe string budget orgs like the Pi Foundation to the multi-billion corporations like MS, none of them seem to have been able to provide what you want, despite decades of work on such issues.

What do you suggest?

[swampdog]

pi@raspberrypi:~ $ time sudo apt-get full-upgrade 2>&1 | tee /tmp/zz

real 16m30.881s

And how much slower would that have been on a Pi Zero W ?

About four times in my experience, so that's an hour-plus.

I've no need to stick anything but "lite" on a zero. I only have an old zero to hand so it'll have to be ethernet and I'm not doing it manually so here's my "script" writing the sdcard..

Code: Select all

foo@sdu ~/usr/src/rpi/rpinew $ time ./go --all sdd 2020-08-20-raspios-buster-armhf-lite.zip pi99
Archive:  2020-08-20-raspios-buster-armhf-lite.zip
  inflating: 2020-08-20-raspios-buster-armhf-lite.img  
'ssh' -> '/mnt/loop/p1/ssh'
'wpa_supplicant.conf' -> '/mnt/loop/p1/wpa_supplicant.conf'
'/mnt/loop/p2/etc/bash.bashrc' -> '/mnt/loop/p2/etc/bash.bashrc.ORIGINAL'
##
alias lc="ls -l"
alias la="lc -a"
alias pu="pushd 1>/dev/null"
alias po="popd 1>/dev/null"
alias j="jobs"
alias LS="ls --color=never"
alias LC="LS -l"
alias LA="LC -a"
export HISTTIMEFORMAT='%F %T ' HISTFILESIZE=2500
##
'000_foo-noreboot' -> '/mnt/loop/p2/etc/sudoers.d/000_foo-noreboot'
'010_foo-nopasswd' -> '/mnt/loop/p2/etc/sudoers.d/010_foo-nopasswd'
'010_foo-nosecpath' -> '/mnt/loop/p2/etc/sudoers.d/010_foo-nosecpath'
'/mnt/loop/p2/etc/hostname' -> '/mnt/loop/p2/etc/hostname.ORIGINAL'
pi99
'/mnt/loop/p2/etc/hosts' -> '/mnt/loop/p2/etc/hosts.ORIGINAL'
127.0.0.1	localhost
::1		localhost ip6-localhost ip6-loopback
ff02::1		ip6-allnodes
ff02::2		ip6-allrouters

127.0.1.1		pi99
'/mnt/loop/p2/etc/systemd/timesyncd.conf' -> '/mnt/loop/p2/etc/systemd/timesyncd.conf.ORIGINAL'
NTP=dns.swampdog
FallbackNTP=0.pool.ntp.org 1.pool.ntp.org
'rinit.sh' -> '/mnt/loop/p2/wrk//rinit.sh'
'finit.sh' -> '/mnt/loop/p2/wrk//finit.sh'
'fipaddr.sh' -> '/mnt/loop/p2/wrk//fipaddr.sh'
'dns.pub' -> '/mnt/loop/p2/wrk//dns.pub'
'sdu.pub' -> '/mnt/loop/p2/wrk//sdu.pub'
'wrk.tar' -> '/mnt/loop/p2/wrk/wrk.tar'
'rcl.sh' -> '/mnt/loop/p2/wrk//rcl.sh'
'/mnt/loop/p2/etc/rc.local' -> '/mnt/loop/p2/etc/rc.local.ORIGINAL'
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
  printf "My IP address is %s\n" "$_IP"
fi

/wrk/rcl.sh

exit 0
enable_uart=1
Writing image.. (wait for it to flush and emit +OKI)..
3604480+0 records in4MiB/s] [               <=>                                ]
3604480+0 records out
1845493760 bytes (1.8 GB, 1.7 GiB) copied, 485.085 s, 3.8 MB/s
1.72GiB 0:08:05 [3.63MiB/s] [              <=>                                 ]
3604480+0 records in
3604480+0 records out
1845493760 bytes (1.8 GB, 1.7 GiB) copied, 648.648 s, 2.8 MB/s
MODEL                     REVISION  SERIAL               DEVICE
--------------------------------------------------------------------------
ST1000DM010-2EP102        CC43      W9AD25NP             sda     
WDC WD5000AAKX-00ERMA0    15.01H15  WD-WMC2E9047773      sdb     
WDC WD5000AAKX-00ERMA0    15.01H15  WD-WCC2EU466989      sdc     
HL-DT-ST DVDRAM GP57EW40  PF00      KD6Z5KN0919          sr0     
Generic MassStorageClass  1536      000000001536         sdd     
+OKI (you can pull the sdcard now)
removed '2020-08-20-raspios-buster-armhf-lite.img'

real	11m28.240s
user	0m24.661s
sys	1m10.517s

..which of course has has collided with an existing "pi99" in dns(*) so we'll have to use its ip address below..

Code: Select all

foo@sdu ~/usr/src/rpi/rpinew $ time ./go --post 192.168.1.43 2>&1 | tee /wrk/z
[snip]
real	12m40.577s
user	0m0.168s
sys	0m0.370s

..the above spends the majority of its time performing apt-get update/upgrade.

Now we're ready.

Code: Select all

foo@sdu ~/usr/src/rpi/rpinew $ ./go --ssh 192.168.1.43

foo@pi99:~ $ cat /proc/cpuinfo 
processor	: 0
model name	: ARMv6-compatible processor rev 7 (v6l)
BogoMIPS	: 997.08
Features	: half thumb fastmult vfp edsp java tls 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xb76
CPU revision	: 7

Hardware	: BCM2835
Revision	: 900093
Serial		: 00000000017ff4d1
Model		: Raspberry Pi Zero Rev 1.3

No idea how old the above pi is!

I'm not that surprised as the update cycle essentially uses a single core.

(*) My dns/dhcp is a mess.

[hippy]

From the shoe string budget orgs like the Pi Foundation ...

Over it's lifetime RPF has had income of £145 million and had £30 million in end of year funds at the end of 2019, RPT has seen £118 million turnover, £77 million gross profit, £36 million profit, £29 million gift-aided to RPF - I don't see how that can in any way be described as "shoestring budget".

What do you suggest?

Only as has been suggested; that more regularly updated released images would reduce the amount of update necessary after installing such an image.

[cspan]

Not sure where the OP is located, but IIRC there was a big internet outage affecting the eastern half of the USA recently. If the OP's updates were caught up in that mess, that would be one explanation.

https://www.theverge.com/2020/11/25/217 ... n-internet

[picandies]

from jamesh:

No way could it be done for the 1st of each month - takes longer than a month to test all the changes. Which is one reason why it's about every 6 months.

Not sure I'd completely buy that---so if a brand new user downloads the rpi OS & then the first thing the pi says-- it is going to turn around & download/install all the updates---you are saying they are then using untested software?

By this, you are saying we can't make a new download, because something is untested, but that is indeed what the user will have after the updates. No newbie is going to buy an rpi , set it up , and say NO, I don''t want today's software, I want last year's, last summer, whatever instead.

Whether tested or untested, that is what the newbie user is going to have anyway, since of course they will demand the updates when they unwrap their new goody. Just give all of that the first time in one download, perhaps call it the all-in-one beta.

The update took at least 1/2 hour to 40 minutes, on top of the original install, and that was with a 38 meg/sec rpi connection & 200 meg/sec home connection & using a quality a sandisk SD card.

Think of the bandwidth resources it will save & thus speed download access for everyone, especially around Christmas morning.

none of them seem to have been able to provide what you want, despite decades of work on such issues.
What do you suggest?

Well, I'm glad none of them were trying to get to the moon Instead of downloading "A" then replacincing 80% of it with "B" to end up with "C", just download "C" in the first place....seems like "C" could be automatically generated at the factory...even if it took 24hours to do so...but then everyone can use it.

[DougieLawson]

No way could it be done for the 1st of each month - takes longer than a month to test all the changes. Which is one reason why it's about every 6 months.

Not sure I'd completely buy that---so if a brand new user downloads the rpi OS & then the first thing the pi says-- it is going to turn around & download/install all the updates---you are saying they are then using untested software?

Having worked for a very large software and hardware company (one letter off from the HAL9000) between 1994 and 2014 you are not discussing anything new or strange. Most software is tested in the field. The old adage of "never take a xx.0 version" is an absolute truth. Just about every fix we ever shipped to a customer had been unit tested (as in it assembles or compiles OK) in the development lab but nothing much more than that.

BTW, can you learn how to use [quote="JamesH"] ... quoted text here ... [/quote] tags. Use that "Quote" button (which looks like a pair of inverted commas) in the posting you want to quote and it does some magic stuff for you.

[swampdog]

I think we lost the OP but to round off the discussion there are so many methods implemented over the years and no matter which method is the flavour of day, someone is going to be annoyed. Cache it locally and folk like me will clean it because I don't want the space wasted on the client. Then there were redhat diffs (dunno if redhat they invented them) which used to wind people up because "reasons". Mirrors in case the primary is down and/or overloaded and folk get miffed because the mirror may not be in sync.

Anyone who has the misfortune to maintain a windows update server (aka WSUS) has the worst of all worlds. It can take my WSUS (win 2008 server) over a day to even notice it has updates for *itself*. I will speak no more of WSUS or I'll start swearing!

Historically my redhat/centos KVM hypervisors absolutely had to be identical. I rsync various mirrors to my local PXE server and point yum/dnf to that local PXE. With retirement, not so much because it is a full time job. They still are because I've been too lazy to do anything about it!

When it comes down to it, what we have is "good enough" which I mean in a good way, as in the top of the bell curve.

[picandies]

Having worked for a very large software and hardware company (one letter off from the HAL9000) between 1994 and 2014 you are not discussing anything new or strange. Most software is tested in the field. The old adage of "never take a xx.0 version" is an absolute truth. Just about every fix we ever shipped to a customer had been unit tested (as in it assembles or compiles OK) in the development lab but nothing much more than that.

I am glad you agree with the proposal.. the new user will end up with those codes after the update, so they may as well get it the first time.

[DougieLawson]

I am glad you agree with the proposal.. the new user will end up with those codes after the update, so they may as well get it the first time.

Unfortunately that is completely alien to "The DebIan Way". Their software management is to sit behind the leading edge and let the folks running Mint or Ubuntu test things which they might include in the next version some time within the next two years. That's completely at odds with agile software development and rapid deployment.

The DebIan Way is a constant source of frustration for folks on this forum who want python 3.latest not 3.7 because the latest python release has the latest foobar widget support they need for their project. The DebIan way is also at odds with new hardware and tends to push the hardware development folks to release co-incident with a DebIan version (which means you're running shiny new untested hardware with an untested version of DebIan (including ancient versions of packages)).

[jamesh]

From the shoe string budget orgs like the Pi Foundation ...

Over it's lifetime RPF has had income of £145 million and had £30 million in end of year funds at the end of 2019, RPT has seen £118 million turnover, £77 million gross profit, £36 million profit, £29 million gift-aided to RPF - I don't see how that can in any way be described as "shoestring budget".

What do you suggest?

Only as has been suggested; that more regularly updated released images would reduce the amount of update necessary after installing such an image.

RPM funds irrelevant, they don't fund trading. We even have to buy services from them!

Trading, despite having a relatively small team, is expensive to run. For many reasons I won't go into. And most of the profit goes upstream. We are also careful with money, avoiding overly fast expansion etc. We don't just throw money at problems and hope. So really, considering what we have achieved, its fairly shoestring.

[swampdog]

Having worked for a very large software and hardware company (one letter off from the HAL9000) between 1994 and 2014 you are not discussing anything new or strange. Most software is tested in the field. The old adage of "never take a xx.0 version" is an absolute truth. Just about every fix we ever shipped to a customer had been unit tested (as in it assembles or compiles OK) in the development lab but nothing much more than that.

I am glad you agree with the proposal.. the new user will end up with those codes after the update, so they may as well get it the first time.

From a logistics aspect it could be conceived to be useful to make a "2020-12-23" image containing everything since "2020-08-20" simply to reduce bandwidth and server load but alternatively if the servers can cope, why bother? The rpi is an educational tool so does it matter too much if the update can't be done there & then? I still have a pi running "wheezy".

The best documentation I have ever read were manuals for the BBC micro. Things didn't change so much back then on account of having to purchase new ROMs. In my first ever paid work I wrote some stock control software which (eventually) all worked fine on the school BBC - the one with a floppy drive.

The target machine had older BBC Basic ROMs. No random access to floppies. Frantic couple of weeks with machine level debugger (?was it xmon?) but fortunately had the same "other ROM's" (sorry, memory - been a long time) so with FX calls from excellent manual and embedding some asm, made it work. It would have been so much easier to "apt-get update/upgrade".

Downside is, today, docs cannot keep up. Swings & roundabouts.