Ubuntu Server 18.04 arm64 on RPi3B+

[snookypi]

Hello,

I am standing at the end of the road of my knowledge. I am trying to setup a raspi 3b+ with Ubuntu Server 18.04 arm64. I downloaded the arm64 iso from http://cdimage.ubuntu.com/ubuntu/releas ... -arm64.iso and followed the instructions on https://wiki.ubuntu.com/ARM/RaspberryPi ... ISO_images.

But I do not have success to get Ubuntu Server to run. What do I have to do to achieve a successful installation?

Your help is highly appreciated.

Cheers,
Christoph

[feelslikeautumn]

Hi, I'm sorry those instructions didn't work for you. Could you provide more details about what is the problem you are having?

[feelslikeautumn]

Some info here that might be useful https://askubuntu.com/questions/1041853 ... ntu-arm64/

[snookypi]

The main problem is that the pi does not boot into ubuntu server. And I am not sure which files exactly have to be copied to the boot partition of the sd card.

[feelslikeautumn]

I've never installed from the server ISO. I'll download it later and have a play.

[feelslikeautumn]

When you say it doesn't boot into Ubuntu server, do you mean you can't get the installed system to run, or do you mean you can't get the installer to run? I wrote the instructions you are following so it would really help if you could be a bit more descriptive in what you've done and what you are seeing on the screen.

I've just run the server installer. I hit two problems. The first is uboot doesn't like my USB drive so it stalls at uboot. Using an Sd card gets over that. The second is when the installer scans for the cdrom it doesn't find it because it is on an sd card. Inserting the USB drive back in at this points overcomes that.

I have a 3B. You'll face more obstacles with a 3B+.

[minusInfinite]

I'm having issues with the instructions of getting the installer to boot off SD card after following what instructions I can.

As my main computer is a Windows PC I understand I'm missing a great deal of tools to do some tasks like compile the u-boot image but I don't have a system I'm prepped to install linux on other than a RPI.

I'm just not sure of our the SD partitions should look when I go to insert it into the PI as the instructions expect me to have another linux computer to use to setup the SD card.

[feelslikeautumn]

Is there a particular reason why you need a 64bit operating system? Ubuntu provides an official 32bit version that is installed just like raspbian.

If you really must have a 64 bit operating system, then it is an easier to install xubuntu - https://ubuntu-mate.community/t/aarch64 ... b-3b/16853

If you want to have a go at the server instructions then you can do it all from a live/desktop Ubuntu ISO running on your PC. Or you can set it up from raspbian/32 bit Ubuntu etc running on the pi.

[minusInfinite]

Is there a particular reason why you need a 64bit operating system? Ubuntu provides an official 32bit version that is installed just like raspbian.

Other than I now have the hardware to do so? Not particularly

If you really must have a 64 bit operating system, then it is an easier to install xubuntu - https://ubuntu-mate.community/t/aarch64 ... b-3b/16853

I'm not really after a desktop option as it's meant to replace my current RPI server once I get it all working with everything I want. I do see there was an option to uninstall the desktop someone had issues afterwards. I'll give it a go though

If you want to have a go at the server instructions then you can do it all from a live/desktop Ubuntu ISO running on your PC. Or you can set it up from raspbian/32 bit Ubuntu etc running on the pi.

Are you even able to download additional software when running from live CD? As I would need to be able to get a cross compiler among other things I would likely need to do it via a USB boot on my main desktop. Haven't done much with the live desktop option in a while.

[feelslikeautumn]

Are you even able to download additional software when running from live CD?

Yep, no probs.

[FalcoGer]

I got it to boot manually, but I'm having trouble with configuring grub. maybe those steps here help in some way or another.

but as I said it's broke.
viewtopic.php?f=56&t=217703


if you don't want to install ubuntu natively, you can use something like vmware or virtual box to install ubuntu there and then pass the sd card through to it. just make sure windows doesn't get it's hand on the fat32 boot partition, so always pass through the sd card reader before inserting the card, or it'll create 'System Volume Information'

[cyclic]

When you say it doesn't boot into Ubuntu server, do you mean you can't get the installed system to run, or do you mean you can't get the installer to run? I wrote the instructions you are following so it would really help if you could be a bit more descriptive in what you've done and what you are seeing on the screen.

On pi3b+, booting from USB, I am able to run Ubuntu server iso arm64 and make arm64 installation.
I pre format my target usb as msdos part 1 fat32 and part 2 ext4.
grub install fails near the end (as seems to be expected) but everything else seems to complete ok.
After install part 1 is still empty and part 2 contains what appears to be good filesystem.
fstab looks good with mounts for / and /boot/firmware by UUID.

However per the other previous post I cannot set this up to boot from the fat32 part1..
I have tried a very large number of creative ideas for the part 1 fat32 including every post I have found on the subject many of which are divergent or contradictory. config.txt, cmdline.txt, cloning images from /boot into the fat32 partition 1, compiled uboots etc.
The wiki.ubuntu.com page is helpful but also confusing as it covers in the same document merged instructions for many diferent dated ubuntus, many different dated pis and many different arch and builds so it is difficult to know what is the magic combination for bionic/usb/pi3b+.
I believe that the kit and setup is good and and along the way I have been able to see it setup the new lan ok, request dhcp, receive and set the correct ip, try and retry to boot from tftp but I do not want to boot from the net.
I need a stand-alone usb bootable configuration - My knowledge of grub is not especially good.

Is there a "known to work" step by step for Ubuntu server 18.04.1 to allow boot (pi3b+/usb/arm64) of an installed system from part 2?

[feelslikeautumn]

The wiki includes an example of how to boot the system using the installers grub. I don't have a 3b+ so somebody with one will have to edit the wiki with any particular instructions for that.

The following script should create an installer using the linux-raspi2 kernel. This should work on the 3b+ (ethernet and wifi working). I wrote it a while ago, not sure if I fully tested it. It has a massively long wait at the rainbow screen due to a huge initramfs.

Code: Select all

#!/bin/sh
# Construct a arm64+raspi2 installer.

set -e

# Download the iso
release=bionic
iso_name="ubuntu-18.04-server-arm64.iso"
wget "http://cdimage.ubuntu.com/ubuntu/releases/$release/release/$iso_name"

# Extract the files from the iso to a directory server-raspi2
7z x -oserver-raspi2 "$iso_name"

# Download the latest pi bootloader files
wget http://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/raspberrypi-bootloader_1.20180417-1_armhf.deb
dpkg-deb -x raspberrypi-bootloader_1.20180417-1_armhf.deb /tmp/pi-bootloader

cd server-raspi2

# Copy the bootloader files
cp /tmp/pi-bootloader/boot/* .

# The cmdline.txt (adjust as necessary)
cat <<'EOF' > cmdline.txt
file=/cdrom/preseed/pi.seed --- quiet net.ifnames=0 dwc_otg.lpm_enable=0 modprobe.blacklist=sdhci_iproc
EOF

# The config.txt
cat <<'EOF' > config.txt
# For more options and information see:
# https://www.raspberrypi.org/documentation/configuration/config-txt/README.md

# Kernel and initramfs
kernel=install/vmlinuz-raspi2
initramfs install/initrd-raspi2.gz followkernel

# boot in AArch64 (64-bit) mode
arm_64bit=1

[pi2]
# there is no dtb file for the pi2; using the pi3's file seems to be the done thing
device_tree=bcm2710-rpi-3-b.dtb
[all]

# Please note: 
#   To use overlays, the overlays folder must be copied from 
#   /lib/firmware/4.XX.X-XXXX-raspi2/device-tree/
#   !This is not automatically updated by the flash-kernel pacakge!

# Enable audio (loads snd_bcm2835)
dtparam=audio=on
EOF

# Run 'unix2dos config.txt' if you want it readable in windows

# Create preseed files
mkdir -p preseed
cd preseed

cat <<'EOF' > pi.seed 
# Ignore "no kernel modules were found for this kernel"
d-i anna/no_kernel_modules boolean true

# The partitioning scheme to be used
d-i partman-partitioning/default_label select msdos
d-i partman-auto/expert_recipe_file string /cdrom/preseed/raspberry/pi_recipe

# Maximum size of swapfile in MiB
d-i partman-swapfile/size string 1024

# Maximum percentage of free space to use for swapfile
d-i partman-swapfile/percentage text 10

# Kernel options to be used for the insalled system (added to those used with the installer)
d-i debian-installer/add-kernel-opts string elevator=deadline rootwait ro

# We'll use the pi's own bootloader
d-i grub-installer/skip boolean true

# Don't pause for the "No boot loader installed" message
d-i nobootloader/confirmation_common note

# Setup fat firmware partition (replicates the work of flash-kernel-installer)
d-i preseed/late_command string /cdrom/preseed/raspberry/pi_late_command

# Adjust installer options for the pi
d-i partman/early_command string /cdrom/preseed/raspberry/pi_partman_command
EOF

mkdir raspberry
cd raspberry

cat <<'EOF' > pi_recipe 
raspberrypi ::

128 128 128 fat32
	$primary{ }
	$bootable{ }
	method{ format }
	format{ }
	use_filesystem{ }
	filesystem{ fat32 }
	label{ pi-boot }
	mountpoint{ /boot/firmware } .

900 10000 -1 $default_filesystem
	$lvmok{ }
	$primary{ }
	method{ format }
	format{ }
	use_filesystem{ }
	$default_filesystem{ }
	label{ pi-rootfs }
	mountpoint{ / } .
EOF

cat <<'EOF' > pi_partman_command
#!/bin/sh
# Remove partman options that aren't suitable for the pi bootloader

set -e

# Replace option creates duplicate boot partitions so just get rid 
rm -r /lib/partman/automatically_partition/25replace || true

# We don't have the ability to select an os like you can with grub2 etc
rm -r /lib/partman/automatically_partition/10resize_use_free || true

# Disable flash-kernel-installer
echo "#!/bin/sh" > /var/lib/dpkg/info/flash-kernel-installer.isinstallable
echo "exit 1" >> /var/lib/dpkg/info/flash-kernel-installer.isinstallable
chmod +x /var/lib/dpkg/info/flash-kernel-installer.isinstallable
EOF

cat <<'STOP' > pi_late_command 
#!/bin/sh
# Install Raspberry Pi firmware.  Written by Adam Smith
#
# (Some bits are borrowed from flash-kernel-installer)

set -e

findfs () {
	mount | grep "on /target${1%/} " | tail -n1 | cut -d' ' -f1
}

# Check if the installer diverted update-initramfs, revert before we move on
if [ -e /target/usr/sbin/update-initramfs.flash-kernel-diverted ];then
	logger -t late_command "Removing update-initramfs diversion"
	rm -f /target/usr/sbin/update-initramfs
	in-target dpkg-divert --remove --local --rename /usr/sbin/update-initramfs
fi

logger -t late_command "Installing raspi2 kernel"

# We don't want flash-kernel run before the database is updated
chroot /target dpkg-divert --divert /usr/sbin/flash-kernel.real --rename /usr/sbin/flash-kernel

# Install the raspi2 kernel
mount -o bind /dev /target/dev
if ! apt-install linux-raspi2; then
	# Fallback to cdrom debs
	debs=""
	for i in /cdrom/preseed/raspberry/kernel-debs/*.deb ; do
		debs="$debs /media/$i"
	done
	in-target dpkg -i $debs 
fi
umount /target/dev || true

logger -t late_command "Removing generic kernel"

old_kernel="$(readlink /target/boot/vmlinuz.old | sed 's,^vmlinuz-,,g')"
in-target apt-get --autoremove --yes purge linux-generic linux-image-generic linux-headers-generic linux-image-$old_kernel linux-headers-$old_kernel linux-modules-$old_kernel linux-modules-extra-$old_kernel

logger -t success_command "Copying wifi firmware"

# Copy wifi-firmware and ensure it is not overwritten
# !Users will have to manually update!
for i in /cdrom/preseed/raspberry/wifi-firmware/*sdio* ; do
	[ -f "$i" ] || continue
	chroot /target dpkg-divert --divert "/lib/firmware/brcm/$(basename "$i").bak" --rename "/lib/firmware/brcm/$(basename "$i")" 
	cp "$i" /target/lib/firmware/brcm/
	chmod 644 "/target/lib/firmware/brcm/$(basename "$i")"
done

# The Pi doesn't have a parallel port
if [ -e  /target/etc/modules-load.d/cups-filters.conf ];then
	logger -t late_command "Removing cups-filters.conf"
	rm /target/etc/modules-load.d/cups-filters.conf
fi

logger -t late_command "Copying bootloader files"

# If the correct mount points are used there should already be a firmware directory
mkdir -p /target/boot/firmware/

# Copy bootloader files
cp /cdrom/bootcode.bin /target/boot/firmware/
cp /cdrom/fixup*.dat /target/boot/firmware/
cp /cdrom/start*.elf /target/boot/firmware/
cp /cdrom/config.txt /target/boot/firmware/

# Update config.txt
sed -i 's/^kernel=.*/kernel=vmlinuz/g' /target/boot/firmware/config.txt
sed -i 's/^initramfs .*/initramfs initrd.img followkernel/g' /target/boot/firmware/config.txt

# Create cmdline.txt file
user_params="$(echo $(user-params))" || true
vol_id=$(blkid -o value -s UUID $(findfs /))
echo "root=UUID=$vol_id $user_params" > /target/boot/firmware/cmdline.txt

logger -t late_command "Running flash-kernel ..."

# Remove flash-kernel diversion
chroot /target dpkg-divert --rename --remove /usr/sbin/flash-kernel

# Install flash-kernel if it hasn't already been
if ! apt-install flash-kernel u-boot-tools; then
	logger -t late_command "error: apt-install flash-kernel u-boot-tools failed"
	exit 1
fi

# Check flash-kernel databases and add entry if necessary
machine="$(cat /proc/device-tree/model)"
if ! { grep -q "$machine" /target/usr/share/flash-kernel/db/all.db || grep -q "$machine" /target/etc/flash-kernel/db; } || \
	echo "$machine" | grep -q "Raspberry Pi 2"; then
		# Make a guess about the dtb file to use
		case "$machine" in
			'Raspberry Pi 2 Model B'*)
				# In arm64 there is no dtb file for the pi2; using the pi3's file seems to be the done thing
				dtb="bcm2710-rpi-3-b.dtb"
			;;
			'Raspberry Pi 3 Model B Plus'*)
				dtb="bcm2710-rpi-3-b-plus.dtb"
			;;
			'Raspberry Pi 3 Model B'*)
				dtb="bcm2710-rpi-3-b.dtb"
			;;
			*)
				logger -t late_command "error: unknown model"
				exit 1
		esac

		cat <<EOF >> /target/etc/flash-kernel/db

# Automatically added by the installer
Machine: $machine
DTB-Id: $dtb
Boot-DTB-Path: /boot/firmware/$dtb
Boot-Kernel-Path: /boot/firmware/vmlinuz
Boot-Initrd-Path: /boot/firmware/initrd.img
EOF
fi

# Use flash-kernel to copy kernel, initrd.img, dtb file, etc to the boot partition
mount -o bind /dev /target/dev
if ! in-target flash-kernel; then
	# Run out of space on boot partition or missing dtb file?
	logger -t late_command "error: flash-kernel failed"
	umount /target/dev || true
	exit 1
fi
umount /target/dev || true

logger -t late_command "flash-kernel successful"
STOP

# Download wifi firmware
mkdir wifi-firmware
cd wifi-firmware
# Pi 3B
wget https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm/brcmfmac43430-sdio.bin
wget https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm/brcmfmac43430-sdio.txt
# Pi 3B+
wget https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm/brcmfmac43455-sdio.bin
wget https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm/brcmfmac43455-sdio.clm_blob
wget https://github.com/RPi-Distro/firmware-nonfree/raw/master/brcm/brcmfmac43455-sdio.txt 

# Download raspi2 kernel debs (these probably should be saved in pool)
cd ..
mkdir kernel-debs
cd kernel-debs
wget https://launchpad.net/ubuntu/+source/linux-raspi2/4.15.0-1010.11/+build/14791518/+files/linux-image-4.15.0-1010-raspi2_4.15.0-1010.11_arm64.deb
wget https://launchpad.net/ubuntu/+source/linux-raspi2/4.15.0-1010.11/+build/14791518/+files/linux-modules-4.15.0-1010-raspi2_4.15.0-1010.11_arm64.deb
wget https://launchpad.net/ubuntu/+source/linux-meta-raspi2/4.15.0.1010.9/+build/14796074/+files/linux-image-raspi2_4.15.0.1010.9_arm64.deb

# Extract vmlinuz-raspi2 and dtb files
dpkg-deb -x linux-image-[0-9]*.deb /tmp/pi-kernel
dpkg-deb -x linux-modules-[0-9]*.deb /tmp/pi-kernel
cd ../../../
version="$(ls /tmp/pi-kernel/boot/vmlinuz* | sed 's,^\/tmp\/pi-kernel\/boot\/vmlinuz-,,g')"
cp /tmp/pi-kernel/lib/firmware/$version/device-tree/broadcom/bcm2710*.dtb .
cp /tmp/pi-kernel/boot/vmlinuz-$version install/vmlinuz-raspi2

# Create raspi2 initrd
zcat -k install/initrd.gz > install/initrd-raspi2

# Prepare to include wifi-firmware in initrd
mkdir -p /tmp/pi-kernel/lib/firmware/brcm/
chmod 755 /tmp/pi-kernel/lib/firmware/brcm/
cp preseed/raspberry/wifi-firmware/*sdio* /tmp/pi-kernel/lib/firmware/brcm/
chmod 644 /tmp/pi-kernel/lib/firmware/brcm/*

# evbug spams the logs, get rid
rm /tmp/pi-kernel/lib/modules/$version/kernel/drivers/input/evbug.ko || true

# Generate modules.dep etc files
depmod -a -b /tmp/pi-kernel -F /tmp/pi-kernel/boot/System.map-$version $version

# Remove unneeded files
rm -r /tmp/pi-kernel/lib/modules/$version/initrd || true
rm -r /tmp/pi-kernel/boot
rm -r /tmp/pi-kernel/usr
rm -r /tmp/pi-kernel/lib/firmware/$version

# Add raspi2 modules/firmware to the initrd and compress
find /tmp/pi-kernel/.  | sed 's,^/tmp/pi-kernel\/,,g'| cpio -D /tmp/pi-kernel -R +0:+0 -H newc -o -A -F install/initrd-raspi2
gzip install/initrd-raspi2

echo "Finished!"
echo "Copy all files (including the hidden .disk folder) in the server-raspi2 folder to a fat formatted usb drive."

[cyclic]

Thank you for the reply.
I have not found the examples for booting from that wiki actually work with the ISO-install/18.04.1/arm64/pi3b+ combination.

Unfortunately the rest of the reply relates to raspi2 which is armhf.
and I am trying to boot an arm64 installation.

[feelslikeautumn]

There are arm64 and armhf raspi2 kernels.

What are the error messages you get when booting? For an installation on a USB drive it would be something like:

Code: Select all

ls
set root=(hd1,msdos2)
ls /
linux /boot/vmlinuz root=/dev/sda2
initrd /boot/initrd.img
boot

[cyclic]

Using ubuntu server 18.04... ISO "CD" Installer booted through grub menu
Using the interactive installer (which cannot find net and fails grub install...)
By lengthy manual grub tinkering I am able to get USB boot to proceed:
linux seems to load and script scrolls
initrd seems to load and script scrolls
final scrolling message is:
"fb: switching to vc44drmfb from EFI VGA"
followed by blank screen
followed by occasional momentary flash of another script log (in response to keys?)
with green stuff down the left hand side
with occasional plausible disk (usb) activity.

My assumption was that maybe the kernel was not correctly set up for the bcm video
and that it may have proceeded headless afterwards...

If any one at Canonical Land is working to produce a working install Ub.Se-arm64-pi3b+
then why not share, so we can feed back progress?

[lorensimon]

I have managed to install Ubuntu Server 18.04 arm64 on RPI3B but I am still struggling to be able to reboot from the SD card. I have a couple of additional steps to try; if I am successful I will send some instructions

[jxdking]

If you follow the guide of https://wiki.ubuntu.com/ARM/RaspberryPi ... ISO_images, I guess you are able to manually boot into Ubuntu Server, but get stuck when running:

Code: Select all

sudo grub-install --no-nvram

From my experience, I got this error "grub-install: error: cannot find EFI directory".
The solution is here https://askubuntu.com/questions/853497/ ... -directory
To make it bootable without installer media, you have to copy start*.elf, fixup*.dat, bootcode.bin, *.dtb and kernel8,img from install media to your boot partition.

I have successfully made it boot into Ubuntu Server by sd card itself. However, I am not able to make network working. I guess Ubuntu lacks a proper driver for lan and wlan of Raspberry Pi 3 B+

[lowkey]

If you follow the guide of https://wiki.ubuntu.com/ARM/RaspberryPi ... ISO_images, I guess you are able to manually boot into Ubuntu Server, but get stuck when running:

Code: Select all

sudo grub-install --no-nvram

From my experience, I got this error "grub-install: error: cannot find EFI directory".
The solution is here https://askubuntu.com/questions/853497/ ... -directory
To make it bootable without installer media, you have to copy start*.elf, fixup*.dat, bootcode.bin, *.dtb and kernel8,img from install media to your boot partition.

I have successfully made it boot into Ubuntu Server by sd card itself. However, I am not able to make network working. I guess Ubuntu lacks a proper driver for lan and wlan of Raspberry Pi 3 B+

I was able to get rid of the grub install EFI error thanks to your post but I can't boot to ubuntu using my boot partition (with raspberry firmware/bootloader & u-boot files on it).
Can you post your grub-install command and the content of your boot partition please ? That's would be awesome

[jxdking]

Can you post your grub-install command and the content of your boot partition please ? That's would be awesome

I have abandoned the idea of putting Ubuntu 64bit directly on Raspberry pi, so I don't have those files on any of my SD card now. Since your install media can boot into u-boot without any problem, if you copy all necessary files from install media to the boot partition of your SD card, it should work.
Make sure your SD card is partition with MBR, not GPT. When you do "ls" command under u-boot, partitions should be listed like (hd1,msdos2), "msdos*" means it is MBR disk.


There is another way to run 64bit applications on Raspberry Pi. Compile your own 64bit raspbian kernel. Keep userland 32bit. In this way, you don't have to deal with any driver incompatibility issue. Then, you can "chroot" a full 64bit debian/ubuntu distro. You can find lots of tutorials regarding how to cross-compile 64bit raspberrypi kernel.

[sakaki]

In case it is of interest, I have just posted some instructions here for booting aarch64 Ubuntu Server 18.04 under qemu/kvm on 64-bit Gentoo (using an Ubuntu cloud image, and tianocore EFI bios). Screenshot:



best, sakaki

[sakaki]

Hello,

FYI, I have just updated the mini-tutorial, to cover running an xfce4 desktop on the 18.04 guest image (here). Screenshot:



Best, sakaki

[z3n]

My post on this forum may help you:
viewtopic.php?t=215724