Electron752
Posts: 142
Joined: Mon Mar 02, 2015 7:09 pm

Refresh of 64 bit linux kernel for RPI 3

Wed Aug 10, 2016 2:40 am

I just wanted to let everybody know that I put out a refresh of the 64 bit linux port for RPI 3 that I've been tinkering with.

https://github.com/Electron752/linux

I am only providing source code for now since I still need to think about the implications of distributing a binary.

BUILDING - I do this from a 64 bit chroot on debian amd64 with the aarch64 cross compilers installed:
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 bcmrpi3_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 modules_install

This installs the modules in the /lib/modules directory of the chroot.

INSTALLING - Assuming the RPI 3 media is mounted at /mnt and /mnt/boot
cp arch/arm64/boot/Image /mnt/boot/kernel8.img
cp arch/arm64/boot/dts/broadcom/bcm2710-rpi-3-b.dtb /mnt/boot
cp -rv /lib/modules/* /mnt/lib/modules/

If you have a relatively new version of the firmware, no config.txt changes should be needed. It may be necessary to delete the other *.img and vmlinuz images. Of course, if you are using a custom kernel or initrd that will need to changed or removed.

WHAT'S NEW:
1. Built off the foundation 4.6 linux tree.
2. Uses dwc_otg driver for USB(More features and performance)
3. DMA support for unaccelerated video(Improves performance)
4. PI Foundation touch screen support
5. HDMI audio
6. Builtin wifi(Needs /lib/firmware/brcm/* files from the raspbian image)
7. Builtin bluetooth(Needs /lib/firmware/brcm/* files + bluez source patches from raspbian source code)
8. Several other misc drivers added back in that are included in raspbian(GPIO, SPI, I2C etc..)

MAJOR KNOWN ISSUES:
1. HDMI audio has serious performance issues but it does work.
2. VCHIQ driver needs a 32 bit compatible layer for direct calls from 32bit userland.
3. Newer versions of systemd as distributed by debian arm64 sid needs some of the security features disabled. The symptom is systemd-logind.service and a few other services may not start.

User avatar
TehWoomiestRPI
Posts: 12
Joined: Thu Apr 07, 2016 7:39 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Thu Aug 11, 2016 9:07 pm

THANK YOU SO MUCH FOR YOUR HARD WORK!

You have no idea how much I TRULY appreciate all you do for porting a 64bit kernel for us!

Do you have any idea on how often you will update the kernel?

Electron752
Posts: 142
Joined: Mon Mar 02, 2015 7:09 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Thu Aug 11, 2016 10:44 pm

It's a matter of the amount of interest and if anybody else gets involved.

Right now there isn't much advantage to aarch64 from an end user point of view. It has very few performance benefits except for the ability to have larger swap files. I actually did some some "real world" benchmarks such as ffmpeg and everything was very comparable. I did see a few things were the larger swap files makes a difference though.

It may make a difference in the future though as these arm based devices need more ram. My understanding is that 1 or 2 GB is reaching the limits of 32 bit arm due to architectural limits.

Heater
Posts: 13700
Joined: Tue Jul 17, 2012 3:02 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Fri Aug 12, 2016 4:38 am

64 bits means that some databases can handle more than 2GB of data. For example Mongodb. Yes, not all that data is in memory at the same time, but as they use memory mapped files 32 bit pointers is a limitation.

Anyway, I just hate to think of all those extra bits in the 64 bit ARM architecture going to waste :)
Memory in C++ is a leaky abstraction .

Electron752
Posts: 142
Joined: Mon Mar 02, 2015 7:09 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Fri Aug 12, 2016 5:54 am

I agree, I should have added memory mapped files with the restriction that the database itself is recomplied for aarch64.

The swap files can benefit 32 arm applications that aren't recompiled for whatever reason. Even though each process itself can't access more then 32 bits, the total from all the 32 bit processes can be greater then 32 bit.

User avatar
TehWoomiestRPI
Posts: 12
Joined: Thu Apr 07, 2016 7:39 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Fri Aug 12, 2016 10:26 pm

Actually I have had performance improvements.

I use a RPI3 as a Homebridge Server and a Minecraft server just for home.

The System boots 4 seconds faster(6 seconds vs 10) and Ive noticed complete performance improvements on both the homebridge server and the Minecraft server

This is using a 64bit Kernel and 64bit Arch Linux user land; Along with -O3 optimizations and -march=native compile flags.

User avatar
TehWoomiestRPI
Posts: 12
Joined: Thu Apr 07, 2016 7:39 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Fri Aug 12, 2016 10:27 pm

Electron752 wrote:I agree, I should have added memory mapped files with the restriction that the database itself is recomplied for aarch64.

The swap files can benefit 32 arm applications that aren't recompiled for whatever reason. Even though each process itself can't access more then 32 bits, the total from all the 32 bit processes can be greater then 32 bit.

Well it would benefit from the kernel being able to process the information 2 32-bit pages at a time, wouldn't it? Along with the Armv8 CRC acceleration the RPI3 Supports.

Electron752
Posts: 142
Joined: Mon Mar 02, 2015 7:09 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Sat Aug 13, 2016 12:37 am

I agree. Somethings will certainly be faster especially since the size of compiled code didn't change much from 32 bit to 64 bit while still adding more bits to the registers.

In the tests I did some things are faster and some things are slower. So for now, I think it's just a matter of what the specific application is.

User avatar
TehWoomiestRPI
Posts: 12
Joined: Thu Apr 07, 2016 7:39 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Sat Aug 13, 2016 12:56 am

Electron752 wrote:I agree. Somethings will certainly be faster especially since the size of compiled code didn't change much from 32 bit to 64 bit while still adding more bits to the registers.

In the tests I did some things are faster and some things are slower. So for now, I think it's just a matter of what the specific application is.

Exactly, that's why it's so nice that you got this kernel to run flawlessly on a headless system!

I've been wanting the performance boost so bad, you have no idea how thankful I am!

Electron752
Posts: 142
Joined: Mon Mar 02, 2015 7:09 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Sat Aug 13, 2016 3:14 am

Your quite welcome and I'm glad you find it useful.

I couldn't have done it without the help of many others. My main contribution to the headless system was getting USB and networking to work on 64bit. Trying to debug though a serial terminal is quite painful. It took some effort, but in the end the number of needed changes was actually quite small.

User avatar
TehWoomiestRPI
Posts: 12
Joined: Thu Apr 07, 2016 7:39 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Sat Aug 13, 2016 8:18 pm

Electron752 wrote:Your quite welcome and I'm glad you find it useful.

I couldn't have done it without the help of many others. My main contribution to the headless system was getting USB and networking to work on 64bit. Trying to debug though a serial terminal is quite painful. It took some effort, but in the end the number of needed changes was actually quite small.

Yeah... Serial Debugging is AWFUL, Serial Terminals in General are just a PAIN, but your labour COMPLETELY paid off in the end though, allowing a flawless 64bit headless system to be born.

n0w4y
Posts: 5
Joined: Mon Apr 11, 2016 3:55 am

Re: Refresh of 64 bit linux kernel for RPI 3

Tue Aug 16, 2016 3:57 am

Electron752 wrote:I just wanted to let everybody know that I put out a refresh of the 64 bit linux port for RPI 3 that I've been tinkering with.

... snip ...
With CONFIG_COMPAT for AARCH64 (arm64 build) enabled this kernel can run either 64 or 32 bit EL0 runtime environments.
Ie, elf32 and/or elf64 , provided that the required dynamic loader .so is present. If a binary is built for AARCH64 then so long as its "runtime library closure" is present it will run -- in the midst of an otherwise V7a/32 or AARCH32 runtime.

Stock Raspbian 32-bit rootfs along with kernel modules from this kernel (build) work fine. The kernel grumbles about deprecated instructions with Raspbian/32 runtime because it is compiled to V7a rather than AARCH32 instruction set but nothing fails as a result.

Happy to confirm that onboard wifi works in both 32 (Raspbian) and 64 bit (Linaro Vivid) runtimes.

ebagdasa
Posts: 2
Joined: Thu Oct 06, 2016 9:55 pm

Re: Refresh of 64 bit linux kernel for RPI 3

Thu Oct 06, 2016 9:58 pm

Does your kernel support KVM? I am not sure if it is correct to ask this, I think I would need this instead: https://git.kernel.org/cgit/linux/kerne ... vmarm.git/ . Do you think I can compile this kernel with your bcmrpi3_defconfig?

n0w4y
Posts: 5
Joined: Mon Apr 11, 2016 3:55 am

Re: Refresh of 64 bit linux kernel for RPI 3

Mon Oct 24, 2016 4:23 am

No, the kernel does not support KVM.

Short summary, KVM ideally requires an ARM GIC (interrupt controller). The Pi3 has a proprietary
interrupt controller that is at best documented by source code, reverse engineering and guesswork.

If you want to work with KVM on an ARMv8 SoC then use something other than a Pi.

Return to “Off topic discussion”