gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 1:29 pm

I can't get it to work. I wrote the following to an empty micro-SD card using Etcher on Ubuntu 16.04 amd64 host:
http://cdimage.ubuntu.com/ubuntu/releas ... pi3.img.xz

I then proceeded with these commands:
linuxpc ~ # mkdir -pv /mnt/piroot
linuxpc ~ # mount -v /dev/mmcblk0p2 /mnt/piroot
linuxpc ~ # mount -v /dev/mmcblk0p1 /mnt/piroot/boot

This command threw up some errors regarding change of permissions, but still seemed to copy the files across:
linuxpc ~ # wget -cO- https://github.com/sakaki-/bcm2711-kern ... 724.tar.xz | tar -xJf- -C /mnt/piroot/

linuxpc ~ # nano -w /mnt/piroot/boot/config.txt
Modify the [pi4] section of this file (it appears near the end of the file), so it reads as follows:

Since this is not Raspbian there is no [pi4] section, so I created a whole new section by copying and pasting sakaki's lines below the existing:
[pi4]
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
# memory must be clamped at 1GiB for current 64-bit Pi4 kernels
# restriction will hopefully be removed shortly
total_mem=1024
arm_64bit=1
enable_gic=1
armstub=armstub8-gic.bin
# differentiate from Pi3 64-bit kernels
kernel=kernel8-p4.img

I then had trouble unmounting - was busy - so I just physically removed the sd-card and placed in the pi4.

I tried booting from each HDMI socket - but nothing came up on the screen.

m][sko
Posts: 106
Joined: Fri Jul 20, 2012 6:37 am
Location: Slovakia

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 2:51 pm

gilius2k15 wrote:
Sun Jul 28, 2019 1:29 pm
I can't get it to work. I wrote the following to an empty micro-SD card using Etcher on Ubuntu 16.04 amd64 host:
http://cdimage.ubuntu.com/ubuntu/releas ... pi3.img.xz

I then proceeded with these commands:
linuxpc ~ # mkdir -pv /mnt/piroot
linuxpc ~ # mount -v /dev/mmcblk0p2 /mnt/piroot
linuxpc ~ # mount -v /dev/mmcblk0p1 /mnt/piroot/boot

This command threw up some errors regarding change of permissions, but still seemed to copy the files across:
linuxpc ~ # wget -cO- https://github.com/sakaki-/bcm2711-kern ... 724.tar.xz | tar -xJf- -C /mnt/piroot/

linuxpc ~ # nano -w /mnt/piroot/boot/config.txt
Modify the [pi4] section of this file (it appears near the end of the file), so it reads as follows:

Since this is not Raspbian there is no [pi4] section, so I created a whole new section by copying and pasting sakaki's lines below the existing:
[pi4]
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
# memory must be clamped at 1GiB for current 64-bit Pi4 kernels
# restriction will hopefully be removed shortly
total_mem=1024
arm_64bit=1
enable_gic=1
armstub=armstub8-gic.bin
# differentiate from Pi3 64-bit kernels
kernel=kernel8-p4.img

I then had trouble unmounting - was busy - so I just physically removed the sd-card and placed in the pi4.

I tried booting from each HDMI socket - but nothing came up on the screen.

you still need some files from buster /boot

fixup*
bootcode.bin
armstub*
that files boot up gpu part
that aren't part os sakaki kernel package

User avatar
sakaki
Posts: 405
Joined: Sun Jul 16, 2017 1:11 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 4:12 pm

gilius2k15 wrote: I can't get it to work. I wrote the following to an empty micro-SD card using Etcher on Ubuntu 16.04 amd64 host:
...
As m][sko notes, the ubuntu image doesn't contain sufficiently modern boot firmware in /boot, so you'll need to copy that over too (easiest place to take it from is my modified 64-bit kernel Buster image, above). I actually did note this point in my instructions earlier, but probably didn't make the directions sufficiently explicit, sorry:
sakaki wrote:
Thu Jul 25, 2019 8:05 pm
...
If Ubuntu server has an image for the Pi3, you can adapt it using the instructions in my first post (both the Pi3 and Pi4 are ARMv8-A systems, so their userlands should be compatible, whether 32-bit or 64-bit). You will need bang-up-to-date firmware in /boot though - copy from this image if in doubt.
...
(emphasis added)

In other words, copy over the various *4*.{dat,elf} files from a modern copy of Raspbian (or the full bootable image I provided above) into /boot on your target image. You actually don't need to worry about bootcode.bin - on an RPi4, that is ignored (onboard EEPROM used instead), nor about armstub8-gic.bin, as that actually is part of the kernel tarball (for the moment, at least, completely up-to-date firmware includes it already, so I will drop this at some future point).

I just tried making these mods, out of interest, and got the Ubuntu 18.04.2 LTS system you referenced to boot in 64-bit mode on an RPi4. After doing some initial setup via console, I could then log in over ssh. Screenshot:

Image

Best,

sakaki

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 6:23 pm

Strange!? I copied these files across the to the FAT boot partition:
Image

I now get a Rainbow screen and the monitor stays on - but the screen is black. I've waited at least 10 minutes and nothing happens - from either HDMI.

I will try one more time using a different host: a newer Ubuntu on WSL2 and see if that works.

BTW, why would you use SSH/console in this context? Is there a problem outputting via HDMI or something?

User avatar
sakaki
Posts: 405
Joined: Sun Jul 16, 2017 1:11 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 7:33 pm

gilius2k15 wrote:
Sun Jul 28, 2019 6:23 pm
Strange!? I copied these files across the to the FAT boot partition:
...

I now get a Rainbow screen and the monitor stays on - but the screen is black. I've waited at least 10 minutes and nothing happens - from either HDMI.

I will try one more time using a different host: a newer Ubuntu on WSL2 and see if that works.

BTW, why would you use SSH/console in this context? Is there a problem outputting via HDMI or something?
The image you specified is a console-based system so I assumed getting to the point you can ssh into it would be a reasonable target. HDMI0 output works fine also, that was how I initially logged in.

Something is probably awry with your /boot contents; please enter that directory and run "ls -l" (ell) and post the results here.

Best, sakaki

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 8:23 pm

The only file I've edited was config.txt to add the stuff above the [pi4] part to see if that would trigger it to boot properly as per the modified Raspbian Buster image.

-rwxrwxrwx 1 root root 423 Jul 24 08:15 COPYING.linux
drwxrwxrwx 1 root root 512 Jul 28 18:45 'System Volume Information'
-rwxrwxrwx 1 root root 2878947 Jul 24 08:15 System-p4.map
-rwxrwxrwx 1 root root 512 Jul 24 08:15 armstub8-gic.bin
-rwxrwxrwx 1 root root 25469 Jul 28 07:05 bcm2710-rpi-3-b-plus.dtb
-rwxrwxrwx 1 root root 24850 Jul 28 07:05 bcm2710-rpi-3-b.dtb
-rwxrwxrwx 1 root root 23558 Jul 28 07:05 bcm2710-rpi-cm3.dtb
-rwxrwxrwx 1 root root 40284 Jul 24 08:15 bcm2711-rpi-4-b.dtb
-rwxrwxrwx 1 root root 19689 Jul 28 07:05 bcm2837-rpi-3-b-plus.dtb
-rwxrwxrwx 1 root root 18833 Jul 28 07:05 bcm2837-rpi-3-b.dtb
-rwxrwxrwx 1 root root 18758 Jul 28 07:05 bcm2837-rpi-cm3-io3.dtb
-rwxrwxrwx 1 root root 412 Jul 28 07:05 boot.scr
-rwxrwxrwx 1 root root 52296 Jul 28 07:05 bootcode.bin
drwxrwxrwx 1 root root 512 Jul 28 07:05 broadcom
-rwxrwxrwx 1 root root 134 Jul 28 07:05 cmdline.txt
-rwxrwxrwx 1 root root 170919 Jul 24 08:15 config-p4
-rwxrwxrwx 1 root root 1974 Jul 28 19:28 config.txt
-rwxrwxrwx 1 root root 6694 Jul 28 07:05 fixup.dat
-rwxrwxrwx 1 root root 6068 Jul 9 14:07 fixup4.dat
-rwxrwxrwx 1 root root 3030 Jul 9 14:07 fixup4cd.dat
-rwxrwxrwx 1 root root 9146 Jul 9 14:07 fixup4db.dat
-rwxrwxrwx 1 root root 9148 Jul 9 14:07 fixup4x.dat
-rwxrwxrwx 1 root root 2623 Jul 28 07:05 fixup_cd.dat
-rwxrwxrwx 1 root root 9876 Jul 28 07:05 fixup_db.dat
-rwxrwxrwx 1 root root 9872 Jul 28 07:05 fixup_x.dat
-rwxrwxrwx 1 root root 25832311 Jul 28 07:05 initrd.img
-rwxrwxrwx 1 root root 12982784 Jul 24 08:15 kernel8-p4.img
-rwxrwxrwx 1 root root 492032 Jul 28 07:05 kernel8.img
-rwxrwxrwx 1 root root 25 Jul 28 07:05 meta-data
-rwxrwxrwx 1 root root 75 Jul 28 07:05 network-config
drwxrwxrwx 1 root root 512 Jul 24 08:15 overlays
-rwxrwxrwx 1 root root 2869348 Jul 28 07:05 start.elf
-rwxrwxrwx 1 root root 2759172 Jul 9 14:07 start4.elf
-rwxrwxrwx 1 root root 762880 Jul 9 14:07 start4cd.elf
-rwxrwxrwx 1 root root 4716552 Jul 9 14:07 start4db.elf
-rwxrwxrwx 1 root root 3672712 Jul 9 14:07 start4x.elf
-rwxrwxrwx 1 root root 683172 Jul 28 07:05 start_cd.elf
-rwxrwxrwx 1 root root 5016516 Jul 28 07:05 start_db.elf
-rwxrwxrwx 1 root root 3956164 Jul 28 07:05 start_x.elf
-rwxrwxrwx 1 root root 65 Jul 28 07:05 user-data
-rwxrwxrwx 1 root root 21666304 Jul 28 07:05 vmlinuz

User avatar
sakaki
Posts: 405
Joined: Sun Jul 16, 2017 1:11 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Jul 28, 2019 9:33 pm

OK, the sizes of the fixup*.elf and start*.elf files look fine, but the kernel symbol table (System-p4.map) and bootable kernel (kernel8-p4.img) files are both too small (yours: 2878947 vs mine: 3202696, and yours: 12982784 vs mine: 14936576, respectively).

These files are both ~~90% of the correct length on your system, suggesting that the tarball download / unpack process did not complete successfully (or the microSD card was removed without a proper eject / sync or similar). In any event, if these are incorrect, then the kernel's module set is probably also suspect (under /lib/modules/<kernel-release-name>/...).

So, I'd recommend re-mounting the image, then downloading and unpacking the kernel tarball again, but in two steps this time, since this may be less error prone. Insert the microSD card into your PC, and note the path: I'll assume /dev/mmcblk0 in the following, but adapt the instructions as appropriate (if it shows up as /dev/sdc or something). I'll assume you're working as root, for simplicity.

Issue:

Code: Select all

linuxpc ~ # mkdir -pv /mnt/piroot
linuxpc ~ # mount -v /dev/mmcblk0p2 /mnt/piroot
linuxpc ~ # mount -v /dev/mmcblk0p1 /mnt/piroot/boot
linuxpc ~ # wget -c https://github.com/sakaki-/bcm2711-kernel-bis/releases/download/4.19.59.20190724/bcm2711-kernel-bis-4.19.59.20190724.tar.xz
linuxpc ~ # md5sum bcm2711-kernel-bis-4.19.59.20190724.tar.xz
3c1c7863fc913f3d51cebbe5425b4a16  bcm2711-kernel-bis-4.19.59.20190724.tar.xz
Make sure the md5sum you get matches that shown above. Then, unpack the tarball, which will overwrite the bootable kernel etc. with (hopefully) correct versions:

Code: Select all

linuxpc ~ # tar -xJf bcm2711-kernel-bis-4.19.59.20190724.tar.xz -C /mnt/piroot/
linuxpc ~ # sync
Do "ls -l" again, on /mnt/piroot/boot and check the lengths of System-p4.map and kernel8-p4.img match mine this time. If so then:

Code: Select all

linuxpc ~ # umount -v /mnt/piroot{/boot,}
You can now try removing the microSD card and booting it in a Pi4 again. Be sure to use the HDMI0 port (the one nearer the USB-C power connector) to connect your monitor, for initial boot.

Any better luck this time?

sakaki

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Mon Jul 29, 2019 12:46 am

Yeah it worked this time - but initially the kernel files got written to the filesystem boot folder instead of the system boot partition - you might consider going back to piroot and piboot as separate mounts; so I simply extracted FAT partition 1 from the tar.xz file using 7-Zip and it booted!!! :ugeek:

Have just done a sudo apt update and upgrade to see if it has any errors as per before, and I fear it might well have the same issue:
dpkg: error processing package flash-kernel (--configure):
installed flash-kernel package post-installation script subprocess returned error exit status 1
e: Sub-process /usr/bin/dpkg returned as error code (1)

That's not a good sign, so maybe the GUI will fail to work now - unless I install it under the Pi3 first before attempting the above - but then certain packages may fail too - with or without the GUI?

EDIT: less errors compared to before - but again just a black screen with mouse cursor. Also at the very start of the boot there's a message in red above the green writing stating that kernel modules failed to load.
EDIT: seems the kernel error only appeared once.

User avatar
sakaki
Posts: 405
Joined: Sun Jul 16, 2017 1:11 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Mon Jul 29, 2019 12:01 pm

gilius2k15 wrote:
Mon Jul 29, 2019 12:46 am
Yeah it worked this time - but initially the kernel files got written to the filesystem boot folder instead of the system boot partition - you might consider going back to piroot and piboot as separate mounts; so I simply extracted FAT partition 1 from the tar.xz file using 7-Zip and it booted!!! :ugeek:
No idea why the files went to the wrong place. I tested both this remedial install and the original install instructions verbatim prior to posting them (and indeed went so far as to create a bootable Ubuntu image using the original set, as described - with screenshot - above). Perhaps you mounted the partitions incorrectly (Ubuntu image's rootfs should go first to/mnt/piroot, then its bootfs to /mnt/piroot/boot), or perhaps you omitted or mistyped the "-C /mnt/piroot/" part of the tar -xJf command? In any case, in the absence of a copy and paste of the actual commands you issued at the terminal and results output, I can't really offer any further insight.

However, I would note again that the original install /boot files appeared to be truncated somewhat (prior to you fixing these, by extracting them via 7-zip and re-copying them), and that the following may indicate the cause (again, as you have provided no actual console capture, I can't really tell, but your second point below is certainly suggestive of an incomplete write):
gilius2k15 wrote:
Sun Jul 28, 2019 1:29 pm
...
This command threw up some errors regarding change of permissions, but still seemed to copy the files across:
...
I then had trouble unmounting - was busy - so I just physically removed the sd-card and placed in the pi4.
...
(emphasis yours)
Anyway, whatever the reason, I pointed out above that in such a case:
sakaki wrote:
Sun Jul 28, 2019 9:33 pm
... the kernel's module set is probably also suspect (under /lib/modules/<kernel-release-name>/...).
As such, while your 7-Zip approach has fixed up the files in the bootfs, it may well have left a partial / incomplete set of kernel modules in the rootfs (in /lib/modules/4.19.59-v8-689fc28a5af0-p4-bis+/). Which would possibly explain:
gilius2k15 wrote:
Mon Jul 29, 2019 12:46 am
... Also at the very start of the boot there's a message in red above the green writing stating that kernel modules failed to load.
and maybe some of the other problems that you are seeing also. (Again, in the absence a provided dmesg or journalctl trace from yourself, it is difficult for me to work out exactly, but this is a fair guess.)

To address this, extract this folder (/lib/modules/4.19.59-v8-689fc28a5af0-p4-bis+/) from the bcm2711-kernel-bis-4.19.59.20190724.tar.xz tarball using your preferred method, and copy it into the /lib/modules/ folder (as root, to preserve permissions) on your image. This should ensure you have the full module set present. You should have the following folders in /lib/modules/ once done:

Code: Select all

4.15.0-1031-raspi2  4.19.59-v8-689fc28a5af0-p4-bis+
If you boot the image, and then login as the ubuntu user, run "lsmod" you should get something like the following (obviously the exact details will be different, but the number of kernel modules should be similar):

Code: Select all

[email protected]:~$ lsmod
Module                  Size  Used by
squashfs               45056  0
fuse                  118784  3
bcm2835_v4l2           49152  0
bcm2835_mmal_vchiq     32768  1 bcm2835_v4l2
v4l2_common            16384  1 bcm2835_v4l2
videobuf2_vmalloc      16384  1 bcm2835_v4l2
videobuf2_memops       16384  1 videobuf2_vmalloc
videobuf2_v4l2         28672  1 bcm2835_v4l2
videobuf2_common       49152  2 videobuf2_v4l2,bcm2835_v4l2
videodev              249856  4 v4l2_common,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common
brcmfmac              319488  0
vc4                   200704  0
media                  40960  1 videodev
drm_kms_helper        200704  2 vc4
vc_sm_cma              36864  1 bcm2835_mmal_vchiq
spidev                 20480  0
brcmutil               16384  1 brcmfmac
drm                   503808  3 drm_kms_helper,vc4
sha256_generic         20480  0
drm_panel_orientation_quirks    20480  1 drm
cfg80211              757760  1 brcmfmac
snd_soc_core          217088  1 vc4
snd_compress           20480  1 snd_soc_core
joydev                 24576  0
snd_pcm_dmaengine      16384  1 snd_soc_core
evdev                  24576  3
snd_pcm               126976  3 vc4,snd_soc_core,snd_pcm_dmaengine
snd_timer              40960  1 snd_pcm
snd                    86016  4 snd_timer,snd_compress,snd_soc_core,snd_pcm
rfkill                 32768  2 cfg80211
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
fb_sys_fops            16384  1 drm_kms_helper
i2c_bcm2835            16384  0
spi_bcm2835            20480  0
vchiq                 335872  2 vc_sm_cma,bcm2835_mmal_vchiq
uio_pdrv_genirq        16384  0
fixed                  16384  0
uio                    20480  1 uio_pdrv_genirq
sch_fq_codel           20480  6
iscsi_tcp              20480  0
libiscsi_tcp           28672  1 iscsi_tcp
libiscsi               57344  2 libiscsi_tcp,iscsi_tcp
ip_tables              28672  0
x_tables               40960  1 ip_tables
ipv6                  507904  20
gilius2k15 wrote:
Mon Jul 29, 2019 12:46 am
Have just done a sudo apt update and upgrade to see if it has any errors as per before, and I fear it might well have the same issue:
dpkg: error processing package flash-kernel (--configure):
installed flash-kernel package post-installation script subprocess returned error exit status 1
e: Sub-process /usr/bin/dpkg returned as error code (1)

That's not a good sign, so maybe the GUI will fail to work now - unless I install it under the Pi3 first before attempting the above - but then certain packages may fail too - with or without the GUI?
Again, you have not provided much in the way of console output, so I have tried this myself: the aarch64 Ubuntu update / upgrade seemed to go through OK (mostly):

Code: Select all

[email protected]:~$ sudo apt-get update
Hit:1 http://ports.ubuntu.com/ubuntu-ports bionic InRelease 
Get:4 http://ports.ubuntu.com/ubuntu-ports bionic-security InRelease [88.7 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic-updates InRelease [88.7 kB]                                                      
...
Get:32 http://ports.ubuntu.com/ubuntu-ports bionic-backports/universe arm64 Packages [3732 B]                                      
Get:33 http://ports.ubuntu.com/ubuntu-ports bionic-backports/universe Translation-en [1696 B]                                      
Fetched 4196 kB in 1min 49s (38.6 kB/s)                                                                                            
Reading package lists... Done
[email protected]:~$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  linux-headers-raspi2 linux-image-raspi2 linux-raspi2 u-boot-tools
The following packages will be upgraded:
  apport apt apt-utils base-files bash bind9-host busybox-initramfs busybox-static bzip2 cloud-init console-setup
  console-setup-linux curl dbus debconf debconf-i18n distro-info-data dmeventd dmsetup dnsutils file flash-kernel
  friendly-recovery gcc-8-base gettext-base initramfs-tools initramfs-tools-bin initramfs-tools-core iputils-ping
  iputils-tracepath isc-dhcp-client isc-dhcp-common keyboard-configuration landscape-common language-selector-common
  libapt-inst2.0 libapt-pkg5.0 libbind9-160 libbz2-1.0 libcurl3-gnutls libcurl4 libdb5.3 libdbus-1-3 libdevmapper-event1.02.1
  libdevmapper1.02.1 libdns-export1100 libdns1100 libdrm-common libdrm2 libelf1 libexpat1 libgcc1 libglib2.0-0 libglib2.0-data
  libgnutls30 libidn11 libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 libldap-2.4-2 libldap-common liblvm2app2.2
  liblvm2cmd2.02 liblwres160 libmagic-mgc libmagic1 libnss-systemd libntfs-3g88 libnuma1 libpam-modules libpam-modules-bin
  libpam-runtime libpam-systemd libpam0g libparted2 libpci3 libplymouth4 libpng16-16 libpolkit-agent-1-0 libpolkit-backend-1-0
  libpolkit-gobject-1-0 libpython3.6 libpython3.6-minimal libpython3.6-stdlib libseccomp2 libsqlite3-0 libssl1.0.0 libssl1.1
  libstdc++6 libsystemd0 libudev1 libunistring2 libx11-6 libx11-data libxcb1 linux-firmware login lvm2 mdadm netplan.io nplan
  ntfs-3g open-iscsi openssh-client openssh-server openssh-sftp-server openssl parted passwd patch pciutils plymouth
  plymouth-theme-ubuntu-text policykit-1 python-apt-common python3-apport python3-apt python3-cryptography python3-debconf
  python3-distro-info python3-distupgrade python3-gdbm python3-gi python3-httplib2 python3-jinja2 python3-problem-report
  python3-software-properties python3-update-manager python3-urllib3 python3.6 python3.6-minimal snapd software-properties-common
  systemd systemd-sysv tmux tzdata u-boot-rpi ubuntu-minimal ubuntu-release-upgrader-core ubuntu-server ubuntu-standard udev ufw
  uidmap unattended-upgrades update-manager-core update-notifier-common ureadahead vim vim-common vim-runtime vim-tiny wget
  wpasupplicant xxd
158 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
Need to get 125 MB of archives.
After this operation, 35.3 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 base-files arm64 10.1ubuntu2.5 [60.0 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 bash arm64 4.4.18-2ubuntu1.2 [546 kB]
...
Get:157 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 wpasupplicant arm64 2:2.6-15ubuntu2.3 [807 kB]              
Get:158 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 cloud-init all 19.1-1-gbaa47854-0ubuntu1~18.04.1 [393 kB]   
Fetched 125 MB in 54min 8s (38.4 kB/s)                                                                                             
Extracting templates from packages: 100%
Preconfiguring packages ...
(Reading database ... 61834 files and directories currently installed.)
Preparing to unpack .../base-files_10.1ubuntu2.5_arm64.deb ...
...
Setting up dmeventd (2:1.02.145-4.1ubuntu3.18.04.1) ...
dm-event.service is a disabled or a static unit not running, not starting it.
Setting up lvm2 (2.02.176-4.1ubuntu3.18.04.1) ...
update-initramfs: deferring update (trigger activated)
Setting up ubuntu-release-upgrader-core (1:18.04.34) ...
Setting up update-manager-core (1:18.04.11.10) ...
Setting up update-notifier-common (3.192.1.7) ...
Processing triggers for initramfs-tools (0.130ubuntu3.8) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-1031-raspi2
Unsupported platform.
run-parts: /etc/initramfs/post-update.d//flash-kernel exited with return code 1
dpkg: error processing package initramfs-tools (--configure):
 installed initramfs-tools package post-installation script subprocess returned error exit status 1
Setting up ubuntu-server (1.417.2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Errors were encountered while processing:
 initramfs-tools
E: Sub-process /usr/bin/dpkg returned an error code (1)
Trying the upgrade again, to isolate that last issue for ease of reference:

Code: Select all

[email protected]:~$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  linux-headers-raspi2 linux-image-raspi2 linux-raspi2 u-boot-tools
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] 
Setting up initramfs-tools (0.130ubuntu3.8) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for initramfs-tools (0.130ubuntu3.8) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-1031-raspi2
Unsupported platform.
run-parts: /etc/initramfs/post-update.d//flash-kernel exited with return code 1
dpkg: error processing package initramfs-tools (--configure):
 installed initramfs-tools package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 initramfs-tools
E: Sub-process /usr/bin/dpkg returned an error code (1)
The key error seems to be "unsupported platform" on initramfs-tools; however when using my 64-bit kernel, no initramfs is used anyhow (the kernel mounts the rootfs directly) so I can't imagine this will cause a problem.

However Ubuntu's not really my thing, so perhaps others on here can help (perhaps post a thread in the Operating System Distributions -> Other subforum)?

FWIW, post-upgrade, the aarch64 Ubuntu image still booted fine on my RPi4, as expected. I'm pretty sure I could now install e.g. a graphical desktop etc. on this image if required - but unfortunately I can't really spend any more time on this at this point; I have a ton of work pending for the forthcoming Gentoo 64-bit RPi4/RPi3 release. Hope this has been of some use to you in any case!

PS one last thing. I did a quick web search for this issue, and found a blog about bringing up Ubuntu LTS 18.04.2 in 64 bits mode (using an older bootfs/rootfs bcm2711 tarball I provided, strangely enough!). The author suggests:
James A Chambers wrote: Fix apt-get update

If you try to apt-get update now it will try to update your firmware with older firmware from the Ubuntu repository. The workaround for now is to remove that package so it keeps your existing firmware. Make a note to remember you did this step as later on we will want to reenable updates from the repository once support has been added.

Code: Select all

sudo apt remove flash-kernel initramfs-tools
You may now run sudo apt-get update && sudo apt-get upgrade but don’t use dist-upgrade yet because the kernels in the repository it will update you to don’t support the Pi 4 yet. But this should get you all up to date on the packages!
So maybe you should try that - others will be in a better position to advise than I will (note that doing an apt-get upgrade / update without removing flash-kernel or initramfs-tools does still allow the image to reboot fine, pace what the author states above, but I guess removing them will at least prevent errors).

Best,

sakaki

jdonald
Posts: 417
Joined: Fri Nov 03, 2017 4:36 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Tue Jul 30, 2019 6:10 am

sakaki wrote:
Thu Jul 25, 2019 8:05 pm
I have now created a bootable 32-bit Raspbian Buster with desktop userland / 64-bit kernel image (per instructions above, no other changes have been made) for the RPi4, and uploaded it to my isshoni.org server.

To download, on your Linux box, issue (you may need to be root, or use sudo, for the following, hence the '#' prompt):

Code: Select all

linuxpc ~ # wget -c https://isshoni.org/downloads/raspbian_buster_64bit_kernel.img.xz
linuxpc ~ # wget -c https://isshoni.org/downloads/raspbian_buster_64bit_kernel.img.xz.asc
Thanks so much for packaging it up and sharing, sakaki.

I know that it's all detailed across GitHub, but what's the high-level view of regressions specific to running under the 64-bit kernel with fkms? Not interested in every single issue, just the ones Raspbian users would care about. Such as:
* Accelerated video decode: does vlc --vout mmal_vout, omxplayer, or kodi (32-bit) work now?
* Dispmanx alone (32-bit)? e.g. /opt/vc/src/hello_pi/hello_dispmanx
* SoC camera captures with raspivid/raspistill (32-bit)? Able to run the bcm2835_v4l2/bcm2835_unicam (64-bit) kernel modules?
* What about the thing with FIQ being deprecated on aarch64? Less reliable USB connectivity or did that get fixed?

Getting just a couple of those aspects working well would be a major step forward on usability compared to any 64-bit Pi OS from the past.

And good to hear the 1 GB limit will be fixed soon enough.

User avatar
sakaki
Posts: 405
Joined: Sun Jul 16, 2017 1:11 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Tue Jul 30, 2019 10:17 pm

jdonald wrote:
Tue Jul 30, 2019 6:10 am
I know that it's all detailed across GitHub, but what's the high-level view of regressions specific to running under the 64-bit kernel with fkms? Not interested in every single issue, just the ones Raspbian users would care about. Such as:
* Accelerated video decode: does vlc --vout mmal_vout, omxplayer, or kodi (32-bit) work now?
* Dispmanx alone (32-bit)? e.g. /opt/vc/src/hello_pi/hello_dispmanx
* SoC camera captures with raspivid/raspistill (32-bit)? Able to run the bcm2835_v4l2/bcm2835_unicam (64-bit) kernel modules?
* What about the thing with FIQ being deprecated on aarch64? Less reliable USB connectivity or did that get fixed?

Getting just a couple of those aspects working well would be a major step forward on usability compared to any 64-bit Pi OS from the past.

And good to hear the 1 GB limit will be fixed soon enough.
Will come back with a more detailed answer here once I have the full 64-bit Gentoo image together. However, iirc from the 64-bit kernel / 32-bit Raspbian userland image downloadable above, the minimal (!) testing I did showed (under fkms):
  • total_mem > 3072 breaks USB (but system still usable-ish if rootfs on microSD) with latest kernels; bounce-buffers apparently needed but not yet implemented in aarch64 (see e.g. this comment, and this one);
  • I'm having trouble getting HDMI audio to work with the most recent 64-bit kernels; but that may just be something in my local setup;
  • vlc --vout mmal_vout and omxplayer don't work, ton of mmal errors from the former; didn't try kodi;
  • dispmanx works (at least things like hello_dispmanx do);
  • no sign of the camera or v4l2 interfaces, even under the -bis kernel that supposedly turns them on (and even when booted with the appropriate firmware), but I may just have missed a setting there; this is on my todo list;
  • FIQ - don't know the correct answer here, @6by9? Also see below quote from rst.
rst wrote:
Sat Jul 13, 2019 11:28 am
Interrupt controller

There is a new GIC-400 interrupt controller available, which implements the GICv2 architecture (not GICv3!). It is not known, if the BCM2835 interrupt controller is still available and can be used to generate interrupts. The Distibutor window of the GIC-400 is at 0xFF841000 (unique) and the CPU Interface window at 0xFF842000 (each core "sees its own CPU Interface window at this address). Interrupt numbers can be taken from the device tree sources from Linux. 32 has to be added to SPI interrupt numbers and 16 to PPI numbers from these files.

Unfortunately it is difficult to use the FIQ with the GIC-400 without "kernel_old=1", because the required register settings can only be done from Secure Mode. A Secure Monitor has to be implemented to support this, which is possible in AArch32, but probably not in AArch64. Maybe one has to move to the "kernel_old=1" setting in config.txt to use the FIQ.
More later when I get time (currently in the final stages of rebuilding all the userland binary packages for mtune=cortex-a72 ^-^)

hth, sakaki

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Wed Jul 31, 2019 8:02 am

This command seemed to help Ubuntu with sakaki's kernel:
sudo apt remove flash-kernel initramfs-tools

It certainly feels quite stable in server mode and could not only update and upgrade - but could install quite a large set of packages in the form of sudo apt install ubuntukylin-desktop

Unfortunately, the GUI once again only displays with a black screen and mouse cursor. This is the same pi4/ubuntu problem we had under another custom kernel (see my two references on page 1).

That red message about not loading kernel modules happened once again incidentally, but this time I can confirm all files were written using sakaki's original guide without any truncation, etc.

I will try installing the GUI first on the Pi3 before switching the Kernel to see if that helps.

BTW, a dev was testing his UEFI bootloader on the Pi4 and encountered the same 1GB RAM limit - so it could be deliberate in terms of the manufacturer's system boot files - and isn't ingrained in the kernel, thankfully. I guess that's the same with USB boot - no Pi3 distros seem to boot USB by default except Ubuntu Mate and sakaki's Gentoo, and pi4 USB boot seems held back completely right now.

User avatar
sakaki
Posts: 405
Joined: Sun Jul 16, 2017 1:11 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Wed Jul 31, 2019 3:50 pm

Actually, for this modified Ubuntu image (and indeed the Debian buster variant linked above) you should be safe to set:

Code: Select all

total_mem=3072
in /boot/config.txt (mounted at /boot/firmware/config.txt when the Ubuntu image is running, although just /boot/config.txt in Raspbian). Having 3GiB (on a 4GiB Pi4) is a lot better than just 1GiB! Setting total_mem any higher (or omitting it from config.txt) prevents the USB ports working properly with the current state of the 64-bit kernel (see trejan's post above; I have also confirmed this personally).

Also, if you comment out the vc4 driver overlay line in config.txt, so it reads:

Code: Select all

#dtoverlay=vc4-fkms-v3d
you may have more luck getting a desktop running (as this will just then just be using basic framebuffer support). Both these changes require a reboot to take effect.

As to the remaining missing kernel module, that's just due to a mismatch between the bcm2711_defconfig and what Ubuntu's default kernel config uses (module ib_iser omitted) Incidentally, you can review (just) the errors from the last boot when logged in at the console, by using:

Code: Select all

journalctl -p3 -xb
which can be useful when narrowing things down. Nothing major in this case, and easy to fix in the bcm2711-kernel-bis project, which has a tweak file for exactly this kind of thing. I'll put it on the list. In the meantime you can simply comment out ib_iser from /lib/modules-load.d/open-iscsi.conf, as suggested here.

hth, sakaki

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Wed Jul 31, 2019 10:22 pm

Wow, looks nice - thanks sakaki for all your help!!!
Image
Splendid, splendid... 8-) :ugeek:

Now I can finally start to test the Pi4! :lol:

User avatar
Gavinmc42
Posts: 4020
Joined: Wed Aug 28, 2013 3:31 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 6:03 am

Minecraft first?

I like that look.
Getting close to looking like the mainstream OS's.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

webbsmurfen
Posts: 59
Joined: Fri Jun 27, 2014 12:22 pm
Location: Sweden

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 11:51 am

sakaki wrote:
Wed Jul 31, 2019 3:50 pm
Having 3GiB (on a 4GiB Pi4) is a lot better than just 1GiB! Setting total_mem any higher (or omitting it from config.txt) prevents the USB ports working properly with the current state of the 64-bit kernel (see trejan's post above; I have also confirmed this personally).

hth, sakaki
That sounds a bit strange. Right now, USB3 feels more of a problem than a blessing. Do you know why this phenomen accures, It seems ONLY (what i know of) to affect 64 bit systems?! I will try to set memory in a different 64bit OS tonight to see what happeneds. Anything more than 3072 and USB will get issues, correct?

trejan
Posts: 862
Joined: Tue Jul 02, 2019 2:28 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 11:57 am

webbsmurfen wrote:
Thu Aug 01, 2019 11:51 am
Right now, USB3 feels more of a problem than a blessing. Do you know why this phenomen accures, It seems ONLY (what i know of) to affect 64 bit systems?!
It is a problem with the PCIe interface that the USB controller is attached to. Read https://github.com/raspberrypi/linux/issues/3093 for the technical details.
webbsmurfen wrote:
Thu Aug 01, 2019 11:51 am
Anything more than 3072 and USB will get issues, correct?
Yes.

webbsmurfen
Posts: 59
Joined: Fri Jun 27, 2014 12:22 pm
Location: Sweden

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 12:27 pm

trejan wrote:
Thu Aug 01, 2019 11:57 am
webbsmurfen wrote:
Thu Aug 01, 2019 11:51 am
Right now, USB3 feels more of a problem than a blessing. Do you know why this phenomen accures, It seems ONLY (what i know of) to affect 64 bit systems?!
It is a problem with the PCIe interface that the USB controller is attached to. Read https://github.com/raspberrypi/linux/issues/3093 for the technical details.
webbsmurfen wrote:
Thu Aug 01, 2019 11:51 am
Anything more than 3072 and USB will get issues, correct?
Yes.
Oh i see... Thats no easy task..

A preformance question though
32 bits are limited to max 4Gb memory, which we have in our RPI atm.. What else is the gain to move to a 64bit OS. I know the CPU/GPU is a 64 bit, forced to work in 32bits, but how much of a preformance gain is there between 64 and 32 bit OS

User avatar
rpdom
Posts: 15572
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 12:51 pm

Someone did some comparisons of running a 32 bit OS and running a 64 bit OS on a Pi 4B. The performance was pretty much the same, but the 64 bit code used more memory.

ejolson
Posts: 3800
Joined: Tue Mar 18, 2014 11:47 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 1:34 pm

rpdom wrote:
Thu Aug 01, 2019 12:51 pm
Someone did some comparisons of running a 32 bit OS and running a 64 bit OS on a Pi 4B. The performance was pretty much the same, but the 64 bit code used more memory.
This isn't the right thread to be talking about the advantages and disadvantages of 64-bit architectures. However, since the Pi runs Linux it is interesting to hear what Linus has to say about the PAE extensions used in 32-bit machines with 4GB of memory.

https://www.realworldtech.com/forum/?th ... stid=76973

With that out of the way, it seems that real progress is being made with the 64-bit kernel. Being able to address 3GB is much better than only one. Thanks for all the effort in figuring out how to make things work.

bobmon
Posts: 1
Joined: Sat Mar 19, 2016 8:51 pm

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Thu Aug 01, 2019 4:30 pm

Hi,
I've got your (sakaki's) July 24 kernel installed, with the 64-bit Devuan distro. It boots on a Raspberry Pi 4, but networking seems a bit flaky --- I have a configuration to make the system headless, it depends on using dnsmasq as a simple dhcp server. This works great on a 3B and 3B+ (with pure Devuan 64-bit distro), but it seems to struggle with assigning an IP address when I connect my laptop. Unplugging and replugging the Ethernet cable seems to fix the problem.

But a bigger issue is that the system doesn't find the wifi adapter. I'm using the command "nmcli device" to probe for network connections and all I see is eth0 and lo. I've also plugged in a USB-Ethernet adapter (to give me two Ethernet ports) and "nmcli device" sees the adapter as well. But no "wlan0". Again, this all works fine with pure Devuan on a 3B+.

Finally, a little blip --- when I connect to my ISP through the USB adapter, it gets two IP addresses. Unplug the cable and plug it into the 4B's Ethernet port, it gets one IP address as expected. Plug the cable back into the adapter, two IP addresses on the adapter again.

I'm not familiar with gentoo, but is there a full OS package lying around that is known to work?

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Aug 04, 2019 10:32 am

Omitting the VC4 seems quite problematic for using KVM with an Ubuntu GUI, so I think it's important to know which distro GUIs can work with VC4 and which cannot.

I'm going to try Raspbian next to see if KVM works better on that despite the userland being 32-bit.

EDIT: So far QEMU is building ok on 64-bit Raspbian with 32-bit userland.
sudo apt install glib2.0 libpixman-1-dev gcc libsdl2-dev bison flex libvirt-daemon-system
wget https://download.qemu.org/qemu-4.1.0-rc3.tar.xz
tar xvJf qemu-4.1.0-rc3.tar.xz
cd qemu-4.1.0-rc3
mkdir build
cd build
sudo ../configure --target-list=aarch64-softmmu
sudo make
sudo make install


Just over 2GB of RAM free for a VM if it actually runs.

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Aug 04, 2019 12:20 pm

Nah - failed to run VM with KVM acceleration - so needs a 64-bit userland it seems besides the 64-bit kernel. :evil:

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Sun Aug 04, 2019 7:20 pm

I don't think Ubuntu 19.04 and 19.10 are working with these kernels - just a black screen after the rainbow screen. Somebody created an Ubuntu docker image builder especially for Ubuntu 19.10 - but without knowing his environment I don't think it's possible to use?

gilius2k15
Posts: 55
Joined: Thu Jul 18, 2019 8:42 am

Re: 64-bit weekly kernel autobuilds for RPi4 released (bcm2711_defconfig)

Mon Aug 05, 2019 1:34 pm

Raspex failed.

Ubuntu 18.04 upgraded to 19.04 - but the KVM would no longer initialize display. Turning VC4 back on resulted in a messed up display instead of the usual black screen and white mouse cursor.

Will try Gentoo next.

Return to “General discussion”