tjlusco
Posts: 10
Joined: Sun Feb 12, 2017 3:32 am

Busting for Buster? Try it today!

Wed Feb 13, 2019 3:34 pm

By now I imagine by now there must be a fair few people out there who literally can't wait for the next version of Debian/Raspbian and all of the juicy updated packages it will bring. Just imagine:

* Python 3.5 -> 3.7
* NodeJS 4.8.2 -> 10.15.1
* GCC 6 -> 8
* CLang 4 -> 7
* FFMpeg 3.2 -> 4.1
* GStreamer 1.10.4 -> 1.14.4

And 1000's more, see https://packages.debian.org/stretch/allpackages for the current list. NodeJS in particular boggles my mind, I've literally built node to bake into an image only to have a new "LTS" released the next day. Thankfully the velocity of NodeJS updates has slowed to the point I can finally keep up with! Raspbian also now packages a more up to date version thankfully, not that I was thankful at the time when it broke whole bunch of modules in an update..

So, literally can't wait? Try it now!

A few caveats. Virtually every single Raspbian specifically packaged software is currently unavailable. This is everything that makes Raspbian Raspbian. For this reason, I wouldn't attempt any of the below on a desktop image, but whether this really effects you depends on whether the package of interest is on the list https://packages.debian.org/stretch/allpackages (OK) OR http://archive.raspberrypi.org/debian/d ... f/Packages (You'll need to wait). I'm sure the Raspbian build servers will be raring to go once Buster is officially released, but I wouldn't go bugging the Pi people until then!

Second caveat, there will be a kernel upgrade with this release (4.14 -> 4.19) but the old kernel is currently the one packaged, but you also don't need to wait to get a new kernel/firmware: https://github.com/Hexxeh/rpi-update#branch.

Code: Select all

sudo BRANCH=next rpi-update
And don't blame me if your system breaks :) But what I would say is if you have any specific kernel built flags you'd like turned, speak up now! Buster may be your only chance for a few years to get any big (potentially setup breaking) changes made. Given the huge amount of kernel mainlining efforts that have gone on with RPi, I have no doubt there will already be some big changes that will make life easier for everyone in the long run. I'd love to see a switch to the mainline USB (dwc2) and WIFI drivers (rtl8xxxu, untested devices enabled please :))

So, why wait? But, it's the year of the pig, lets go the whole hog. A full from scratch Raspbian Buster image. For those who weren't aware or have always wondered, here is the tool that the Raspbian images are built from. https://github.com/RPi-Distro/pi-gen

Its essentially debootstrap, plus some wrapping up the qemu/chroot tricky and some elegent scripts so pull in all the RPi packages and nitty-gritty setup. Having written a very similar tool for a project, I am slapping myself for not trying this sooner. This tool is quick, and can do everything you ever wanted to do in a setup.sh, I couldn't recommend it highly enough. For reference, I had a Stretch lite built in 20mins, and the desktop image took closer to an hour, but only due to slow internet. Much faster than running directly on a Pi, and its reproducible!

So what do you have to do? Clone the tool, change every instance of the word "stretch" to "buster", and remove a few packages that aren't currently available, and scripts which arent' working. Simple. I also didn't try building the desktop image, as most of the packages haven't been rebuilt (there a lots of Raspbian specific stuff in the desktop image), but the light image was easy enough.

Keep an eye out for when the Raspbian packages start going up and then you'll be able to have some real fun :)

Here is my diff:

Code: Select all

diff --git a/export-image/01-set-sources/00-patches/0-sources.diff b/export-image/01-set-sources/00-patches/0-sources.diff
deleted file mode 100644
index 17badff..0000000
--- a/export-image/01-set-sources/00-patches/0-sources.diff
+++ /dev/null
@@ -1,9 +0,0 @@
-Index: export-jessie/rootfs/etc/apt/sources.list.d/raspi.list
-===================================================================
---- export-jessie.orig/rootfs/etc/apt/sources.list.d/raspi.list
-+++ export-jessie/rootfs/etc/apt/sources.list.d/raspi.list
[email protected]@ -1,3 +1,3 @@
--deb http://archive.raspberrypi.org/debian/ stretch main ui staging
-+deb http://archive.raspberrypi.org/debian/ stretch main ui
- # Uncomment line below then 'apt-get update' to enable 'apt-get source'
- #deb-src http://archive.raspberrypi.org/debian/ stretch main ui
diff --git a/export-image/01-set-sources/00-patches/series b/export-image/01-set-sources/00-patches/series
deleted file mode 100644
index 49ccdbc..0000000
--- a/export-image/01-set-sources/00-patches/series
+++ /dev/null
@@ -1 +0,0 @@
-0-sources.diff
diff --git a/export-image/01-set-sources/01-run.sh b/export-image/01-set-sources/01-run.sh
deleted file mode 100755
index 86dea2d..0000000
--- a/export-image/01-set-sources/01-run.sh
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash -e
-
-on_chroot << EOF
-apt-get update
-apt-get -y dist-upgrade
-apt-get clean
-EOF
diff --git a/export-noobs/00-release/files/os.json b/export-noobs/00-release/files/os.json
index a37e266..f694932 100644
--- a/export-noobs/00-release/files/os.json
+++ b/export-noobs/00-release/files/os.json
@@ -15,5 +15,5 @@
     ],
     "url": "http://www.raspbian.org/",
     "username": "pi",
-    "version": "stretch"
+    "version": "buster"
 }
diff --git a/stage0/00-configure-apt/files/raspi.list b/stage0/00-configure-apt/files/raspi.list
index 656ab48..70b5fd6 100644
--- a/stage0/00-configure-apt/files/raspi.list
+++ b/stage0/00-configure-apt/files/raspi.list
@@ -1,3 +1,3 @@
-deb http://archive.raspberrypi.org/debian/ stretch main ui staging
+deb http://archive.raspberrypi.org/debian/ buster main ui staging
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://archive.raspberrypi.org/debian/ stretch main ui
+#deb-src http://archive.raspberrypi.org/debian/ buster main ui
diff --git a/stage0/00-configure-apt/files/sources.list b/stage0/00-configure-apt/files/sources.list
index 45e5210..61820ac 100644
--- a/stage0/00-configure-apt/files/sources.list
+++ b/stage0/00-configure-apt/files/sources.list
@@ -1,3 +1,3 @@
-deb http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi
+deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi
 # Uncomment line below then 'apt-get update' to enable 'apt-get source'
-#deb-src http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi
+#deb-src http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi
diff --git a/stage0/prerun.sh b/stage0/prerun.sh
index 7c09b02..9ce3e02 100755
--- a/stage0/prerun.sh
+++ b/stage0/prerun.sh
@@ -1,5 +1,5 @@
 #!/bin/bash -e
 
 if [ ! -d "${ROOTFS_DIR}" ]; then
-	bootstrap stretch "${ROOTFS_DIR}" http://raspbian.raspberrypi.org/raspbian/
+	bootstrap buster "${ROOTFS_DIR}" http://raspbian.raspberrypi.org/raspbian/
 fi
diff --git a/stage1/03-install-packages/00-packages b/stage1/03-install-packages/00-packages
index e8c148c..73edcc9 100644
--- a/stage1/03-install-packages/00-packages
+++ b/stage1/03-install-packages/00-packages
@@ -1 +1 @@
-libraspberrypi-bin libraspberrypi0 raspi-config
+libraspberrypi-bin libraspberrypi0
diff --git a/stage2/00-copies-and-fills/01-packages b/stage2/00-copies-and-fills/01-packages
deleted file mode 100644
index 283c290..0000000
--- a/stage2/00-copies-and-fills/01-packages
+++ /dev/null
@@ -1 +0,0 @@
-raspi-copies-and-fills
diff --git a/stage2/01-sys-tweaks/00-packages b/stage2/01-sys-tweaks/00-packages
index f058ffb..bd1914d 100644
--- a/stage2/01-sys-tweaks/00-packages
+++ b/stage2/01-sys-tweaks/00-packages
@@ -10,13 +10,9 @@ fake-hwclock nfs-common usbutils
 libraspberrypi-dev libraspberrypi-doc libfreetype6-dev
 dosfstools
 dphys-swapfile
-raspberrypi-sys-mods
-pi-bluetooth
 apt-listchanges
 usb-modeswitch
 apt-transport-https
-libpam-chksshpwd
-rpi-update
 libmtp-runtime
 rsync
 htop
diff --git a/stage2/01-sys-tweaks/00-patches/07-resize-init.diff b/stage2/01-sys-tweaks/00-patches/07-resize-init.diff
deleted file mode 100644
index 0701641..0000000
--- a/stage2/01-sys-tweaks/00-patches/07-resize-init.diff
+++ /dev/null
@@ -1,5 +0,0 @@
---- a/rootfs/boot/cmdline.txt
-+++ b/rootfs/boot/cmdline.txt
[email protected]@ -1 +1 @@
--dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
-+dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh
diff --git a/stage2/01-sys-tweaks/00-patches/series b/stage2/01-sys-tweaks/00-patches/series
index aee0402..19b04f5 100644
--- a/stage2/01-sys-tweaks/00-patches/series
+++ b/stage2/01-sys-tweaks/00-patches/series
@@ -2,4 +2,3 @@
 02-swap.diff
 04-inputrc.diff
 05-path.diff
-07-resize-init.diff
diff --git a/stage2/01-sys-tweaks/01-run.sh b/stage2/01-sys-tweaks/01-run.sh
index 8d28adc..5aa0d63 100755
--- a/stage2/01-sys-tweaks/01-run.sh
+++ b/stage2/01-sys-tweaks/01-run.sh
@@ -20,7 +20,6 @@ if [ "${ENABLE_SSH}" == "1" ]; then
 else
 	systemctl disable ssh
 fi
-systemctl enable regenerate_ssh_host_keys
 EOF
 
 if [ "${USE_QEMU}" = "1" ]; then
diff --git a/stage2/02-net-tweaks/00-packages b/stage2/02-net-tweaks/00-packages
index cc4a68e..13150b4 100644
--- a/stage2/02-net-tweaks/00-packages
+++ b/stage2/02-net-tweaks/00-packages
@@ -1,4 +1,3 @@
 wpasupplicant wireless-tools firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek
-raspberrypi-net-mods
 dhcpcd5
 net-tools

So, what doesn't work? Networking. Its almost completely broken, but it would just need tweaking of the setup. Not too dissimilar to the pain of wheezy -> jessie -> stretch. I haven't used buster enough to know what is the "right way" to do it yet, I used ifconfig and wpa_supplicant directly to get it all up and running. Under stretch you had dhcpcd5, dhcp, network/interfaces ipup/down, systemd/networkd, dbus managed wpa_supplicant vs manual configuration, and all of the many ways each of these components interact and break in there own fun ways. I still haven't been able to figure out how dhcpcd5 tears down and pulls back up loopback interface at startup, but boy that is fun to debug with network services which mysteriously break because they couldn't hit 127.0.0.1. If only dhcpcd5 wasn't the least worst option. I also couldn't bring up the internal wifi on a RPI3, I think the driver is in one of those missing packages (or it isn't working, did see some messages in the log), but external wifi devices work.

I managed to pull down a bunch of packages to try out, and it all works just works in an almost boring Debian style way, no surprises. I wonder if the Raspbian version will spice things up :)

Have fun!

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5717
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: Busting for Buster? Try it today!

Wed Feb 13, 2019 3:40 pm

The public buster repo for archive.raspberrypi.org is not ready yet.

https://wiki.debian.org/DontBreakDebian ... nkenDebian

incognitum
Posts: 280
Joined: Tue Oct 30, 2018 3:34 pm

Re: Busting for Buster? Try it today!

Wed Feb 13, 2019 4:48 pm

tjlusco wrote:
Wed Feb 13, 2019 3:34 pm
But what I would say is if you have any specific kernel built flags you'd like turned, speak up now! Buster may be your only chance for a few years to get any big (potentially setup breaking) changes made.
Wishlist:

- CONFIG_INPUT_EVDEV=y

Building evdev statically into the kernel instead of as module would have the advantage that you have working keyboard and mouse in X, even if modules fail to load.
E.g. when updates go wrong, and you end up in a situation in which you do have a newer kernel, but not the matching modules, it would be nice if you have working keyboard so can retry update.


- Ramoops support

https://www.raspberrypi.org/forums/view ... p?t=199047
Stores kernel panic log information at a known location in RAM.
Allowing you to recover the information after reboot by doing: cat /sys/fs/pstore/dmesg-ramoops-0

Ideally the location specified should be a range that is guaranteed not to be overwritten by boot firmware on boot either.


- Let kernel handle bluetooth, instead of using the current legacy userspace hciattach helpers that no longer exist in newer bluez versions.

Recall the following changes were needed for that.

* Enable the right entries In kernel configuration so that uart slave entries in device-tree files are handled properly:

Code: Select all

CONFIG_SERIAL_DEV_BUS=y
CONFIG_SERIAL_DEV_CTRL_TTYPORT=y
(NOT as module)

* Add proper device-tree entries for BT. Similar to what upstream already has:

Code: Select all

/* uart0 communicates with the BT module */
&uart0 {
	pinctrl-names = "default";
	pinctrl-0 = <&uart0_ctsrts_gpio30 &uart0_gpio32 &gpclk2_gpio43>;
	status = "okay";

	bluetooth {
		compatible = "brcm,bcm43438-bt";
		max-speed = <2000000>;
		shutdown-gpios = <&expgpio 0 GPIO_ACTIVE_HIGH>;
	};
};
* Get your Bluetooth firmware files fixed so they setup MAC address properly.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22752
Joined: Sat Jul 30, 2011 7:41 pm

Re: Busting for Buster? Try it today!

Wed Feb 13, 2019 4:58 pm

incognitum wrote:
Wed Feb 13, 2019 4:48 pm
- CONFIG_INPUT_EVDEV=y

Building evdev statically into the kernel instead of as module would have the advantage that you have working keyboard and mouse in X, even if modules fail to load.
E.g. when updates go wrong, and you end up in a situation in which you do have a newer kernel, but not the matching modules, it would be nice if you have working keyboard so can retry update.
I like this idea.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Aydan
Posts: 670
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: Busting for Buster? Try it today!

Thu Feb 14, 2019 1:09 pm

jamesh wrote:
Wed Feb 13, 2019 4:58 pm
incognitum wrote:
Wed Feb 13, 2019 4:48 pm
- CONFIG_INPUT_EVDEV=y

Building evdev statically into the kernel instead of as module would have the advantage that you have working keyboard and mouse in X, even if modules fail to load.
E.g. when updates go wrong, and you end up in a situation in which you do have a newer kernel, but not the matching modules, it would be nice if you have working keyboard so can retry update.
I like this idea.
Will the USB hub part of the smsc95xx work without the proper kernel module? Or is it baked into the kernel already?

Regads
Aydan

incognitum
Posts: 280
Joined: Tue Oct 30, 2018 3:34 pm

Re: Busting for Buster? Try it today!

Thu Feb 14, 2019 1:16 pm

Aydan wrote:
Thu Feb 14, 2019 1:09 pm
Will the USB hub part of the smsc95xx work without the proper kernel module? Or is it baked into the kernel already?
Already in the kernel.

And so is standard USB HID.
Only if you have a special keyboard that needs tweaks you do need extra modules.
(And you might consider building all those in the kernel as well, as they are very tiny, and Linux normally has them built-in by default on kernels that do not specify CONFIG_EMBEDDED=y to micro manage options).

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22752
Joined: Sat Jul 30, 2011 7:41 pm

Re: Busting for Buster? Try it today!

Thu Feb 14, 2019 1:59 pm

incognitum wrote:
Thu Feb 14, 2019 1:16 pm
Aydan wrote:
Thu Feb 14, 2019 1:09 pm
Will the USB hub part of the smsc95xx work without the proper kernel module? Or is it baked into the kernel already?
Already in the kernel.

And so is standard USB HID.
Only if you have a special keyboard that needs tweaks you do need extra modules.
(And you might consider building all those in the kernel as well, as they are very tiny, and Linux normally has them built-in by default on kernels that do not specify CONFIG_EMBEDDED=y to micro manage options).
Standarad USB HID/mouse is a module, so if the modules don't match, you get the symptoms described above. Making it part of the kernel would probably mean that even with mismatched modules, the mouse would still work.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

incognitum
Posts: 280
Joined: Tue Oct 30, 2018 3:34 pm

Re: Busting for Buster? Try it today!

Thu Feb 14, 2019 2:37 pm

jamesh wrote:
Thu Feb 14, 2019 1:59 pm
Standarad USB HID/mouse is a module, so if the modules don't match, you get the symptoms described above.
EVDEV which is one of the methods of exposing the input events to userspace, is currently a module.
X11 happens to use that method to read key input. So on a default Raspbian install -which boots to X- a mismatch in kernel modules results in that for all practical purposes your keyboard is unusable.
So I am suggesting to build that into the kernel.

Aydan was questioning if the other bits and pieces to have working input devices do are present, and in response to that I mentioned they are.
Standard USB HID does is already built into the kernel.
If you are booting to console instead of X, you already have a functional keyboard even without modules, because that does not uses evdev but reads the key input from a tty device instead.
(Unless you have a keyboard that does not speak proper HID but needs tweaks. E.g. some Apple, Logitech unifying wireless and Microsoft keyboards currently do need modules. But like I said, you could consider turning those on by default as well.)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22752
Joined: Sat Jul 30, 2011 7:41 pm

Re: Busting for Buster? Try it today!

Thu Feb 14, 2019 4:07 pm

incognitum wrote:
Thu Feb 14, 2019 2:37 pm
jamesh wrote:
Thu Feb 14, 2019 1:59 pm
Standarad USB HID/mouse is a module, so if the modules don't match, you get the symptoms described above.
EVDEV which is one of the methods of exposing the input events to userspace, is currently a module.
X11 happens to use that method to read key input. So on a default Raspbian install -which boots to X- a mismatch in kernel modules results in that for all practical purposes your keyboard is unusable.
So I am suggesting to build that into the kernel.

Aydan was questioning if the other bits and pieces to have working input devices do are present, and in response to that I mentioned they are.
Standard USB HID does is already built into the kernel.
If you are booting to console instead of X, you already have a functional keyboard even without modules, because that does not uses evdev but reads the key input from a tty device instead.
(Unless you have a keyboard that does not speak proper HID but needs tweaks. E.g. some Apple, Logitech unifying wireless and Microsoft keyboards currently do need modules. But like I said, you could consider turning those on by default as well.)
I'll suggest it to those that deal with the kernel configuration.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

p00ya
Posts: 1
Joined: Sun Jun 23, 2019 4:13 pm

Re: Busting for Buster? Try it today!

Sun Jun 23, 2019 4:25 pm

Upgraded to Buster this weekend... other than some breakages with raspi-copies-and-fills (had to roll my own) and lack of an nf_tables module (fixed with latest rpi-update) things seem to be going well.

Hope I'm not too late to suggest kernel config changes... would love:

Code: Select all

CONFIG_NET_SCH_CAKE=m
sch_cake is a low-config packet scheduler that provides stream isolation, framing compensation, and AQM (like codel). Using my Raspberry Pi as a router with sch_cake made sharing a 6mbit ADSL link with my household much less frustrating.

I've been compiling it out-of-tree for years, but it would be a great opportunity to get it in!

User avatar
DougieLawson
Posts: 35381
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Busting for Buster? Try it today!

Mon Jun 24, 2019 5:57 am

Don't try it on a Raspberry with an AIY hat. I had to rebuild back to Stretch to get that working.
Note: Having anything remotely humorous in your signature is completely banned on this forum.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

Return to “Advanced users”