Raspbian Jessie (64bit) for RPi3?

[jahboater]

Where would 64 bit help?

I know its artificial, but I just ran sysbench on 32 and 64 bit cortex-a53's (sudo apt install sysbench).
The Pi3 CPU is a cortex-a53.

Code: Select all

sysbench --test=cpu --num-threads=4 --cpu-max-prime=20000 run

Adjusted for different clock speeds:
32-bit takes 86.07 seconds
64-bit takes 5.55 seconds

Quite a large improvement, normally I see around 10 - 15 %.

Compiling the same program with the same compiler on a few machines gives the following code sizes:

84k armv6 32-bit
83k armv8 32-bit
80k armv8 64-bit
76k intel 64-bit

The new A64 ISA was redesigned over six years to get rid of the historical cruft and streamline it for modern hardware. Its much more uniform and easy to decode. The new out-of-order arm processors need it.

[peterlite]

Impressive difference.

[jahboater]

The fact that it takes so long implies that its doing division/remainder in a loop and not the sieve method.

I just checked for interest and the sysbench executable on the Pi Zero is identical to that on the Pi3. The Pi Zero doesn't have a divide instruction and so makes a library call. That means its making the library call on the Pi3 too.

Aarch64 always has a divide instruction, so it goes a lot faster.

[DrMTR]

I have just received my RPi3 and am imaging the SDcard with the new Jessie from Feb 26.

Now up and running including the built-in WiFi! Seems to be working fine.
Just don't know how to benchmark it...

I will make a benchmark for Wifi, only to get my Rpi3 tommorow. Its arrived into my local post.

[ebagdasa]

where did you get 64-bit version of Raspbian? I am just curious. Haven't found anything

[jahboater]

where did you get 64-bit version of Raspbian? I am just curious. Haven't found anything

Its not yet available.
You can run a cut down 64-bit Fedora from here:
https://www.kraxel.org/repos/rpi2/images/
They are minimal to say the least; the Fedora one (arm64-rpi3-f24...) is best as it has a package installer included.
I did "yum install gcc", and got gcc 6.1.1.
The .raw file is the same as a .img file, so you dd it in the normal way.

Alternatively for a full 64-bit OS (Ubuntu, Debian or Android), you will need another board such as the Odroid-C2 or the Pine 64. These both have Cortex-a53 CPU's the same as the Pi3. I prefer the C2, its very fast.

[technix]

So far I have sorted out a technical solution to port Raspbian to 64-bit with minimal effort:

• The kernel itself is a full-blown AArch64 Linux kernel, with one concept and one flag on mmap(2) ported over from AMD64 architecture.
• No change needed to VC code
• Minimal change to VC driver stack to solve the ARM-VC pointer passing problem

The concept ported from AMD64 is "canonical address". Despite otherwise being a 64-bit processor, AMD64 doesn't have 64 address lines yet (they have 48 or 56, based on the exact processor model.) So AMD introduced the concept of canonical addresses - the highest bits of a pointer, where no address line exist, must be same to the highest bit of the address lines. This allows pointers to be shortened with their highest bits cropped off when transmitting on the address bus, and sign extended back to full 64 bits when passed back.

In the AMD64 kernel there is also a flag, MAP_32BIT for mmap(2), which results in the address block allocated by this call being guaranteed to sit within the first 2GB of virtual memory address, making sure all highest 33 bits of the pointer is zero.

AArch64 does have full 64-bit address lines, but the VC core doesn't. So the 32-bit canonical address is used when interfacing the GPU. When the kernel boots up it need to relocate itself to the last 2GB of virtual memory addresses (with all highest 33 bits of all kernel addresses being 1.) This results in all kernel address, and all addresses allocated from mmap(2) with MAP_32BIT, have all their 33 bits being the same, within the 32-bit canonical address range that can be passed to VC and back losslessly.

So by porting the MAP_32BIT flag on mmap(2) from AMD64 to AArch64, and adding code to relocate the kernel to the last 2GB of virtual memory space, this largely vanilla AArch64 kernel is able to talk to VC without any change on VC side - cropping off high 32-bit when sending a pointer out, and sign extend it back to 64-bit when receiving from VC. Userland code that uses VC have to be modified though, by making sure all pointers that would go to the VC are allocated using mmap(2) with MAP_32BIT.

[jojopi]

The fact that it takes so long implies that its doing division/remainder in a loop and not the sieve method.

Each thread is effectively:

Code: Select all

#include <stdio.h>
#include <math.h>

typedef unsigned long long integer;
#define max_prime 20000

int main()
{
  integer c, l, t, n=0;
  int rep;
  for (rep=0; rep < 2500; rep++) {
    for (c=0; c < max_prime; c++) {
      t=sqrt(c);
      for (l=2; l <= t; l++)
        if (c % l == 0) break;
      if (l > t) n++;
    }
  }
  printf("%lld\n", (long long)n);
  return 0;
}

The main issue is the use of a 64bit arithmetic, when actually all of the values would fit easily in 32bits. Times running on the same AArch64 kernel at 600MHz:

15.25s: compiled in Debian arm64
204.09s: compiled in Raspbian
68.61s: compiled in Raspbian with typedef changed to unsigned long
15.68s: compiled in Raspbian with unsigned long and -mcpu=cortex-a53

(Just setting the CPU target without changing the typedef does not help, because GCC cannot use 32bit UDIV to implement a 64bit modulo.)

[jahboater]

Yes indeed.
I looked at the man page for sysbench:

cpu
The cpu is one of the most simple benchmarks in SysBench. In this mode each request
consists in calculation of prime numbers up to a value specified by the
--cpu-max-primes option. All calculations are performed using 64-bit integers.

as you say, its all in 64-bits. Probably because sysbench is intended for databases.

"bl __aeabi_uldivmod" will never compete with udiv/msub that the 64-bit mode will do.

Setting the cpu type correctly makes a large difference because the cortex-a53 always has the udiv instruction whereas other 32-bit arm cpu's might not and gcc will use a slow library call which will always work.

[nizx]

I personally think it is a good option, I wanted to do some tests with mongodb and found out that the only version that runs is 2.x, the last 32bit version of it. Having a 64bit OS would theoretically allow me to run that and finish my tests.

Other reason to do it is because the Pi 3 can do it and that should be enough.

[nastybuttler322]

A lot of opinions about this. I do want to see 64bit Armv8 processing being used as 64bit in Raspbian, or even another distro if it can be made to require as little resources as raspbian has been able to pull off. I just want to see it out of curiosity, because I have no idea how that would effect performance one way or another, and I am currently taking Operating Systems in college as part of my programming and analysis degree. The reason I know that I do not know one way or the other, is because if and when I do, I will either have the skill to start writing the kernel and distro to be able to use the 64bit architecture, or I will have already completed it and have ran the tests.

If you think you know one way or the other, why not quit wasting time and get to the structure theorem and build it yourself? This distro ain't gonna write itself lol

[jamesh]

People have been working on porting parts of the kernel to 64 bits and getting it upstreamed - keep an eye on the raspberry pi github kernel page for more information.

[django013]

I run sysbench on my Pi3 with these results:

Code: Select all

time sysbench --test=cpu --num-threads=4 --cpu-max-prime=20000 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          185.2612s
    total number of events:              10000
    total time taken by event execution: 740.8878
    per-request statistics:
         min:                                 73.84ms
         avg:                                 74.09ms
         max:                                124.05ms
         approx.  95 percentile:              74.38ms

Threads fairness:
    events (avg/stddev):           2500.0000/6.36
    execution time (avg/stddev):   185.2219/0.02


real    3m5.283s
user    12m20.660s
sys     0m0.000s

3 minutes and 5 seconds is way slower than the 86 seconds from jahboater.

What am I doing wrong?

System: raspbian stretch

Code: Select all

cat /proc/version 
Linux version 4.4.50-v7+ ([email protected]) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611) ) #970 SMP Mon Feb 20 19:18:29 GMT 2017

Code: Select all

cat /proc/device-tree/model
Raspberry Pi 3 Model B Rev 1.2

[bstrobl]

People have been working on porting parts of the kernel to 64 bits and getting it upstreamed - keep an eye on the raspberry pi github kernel page for more information.

Once things work well, I hope to see the bcm2835 replaced with a new low power 64 bit chip (single core Cortex A35 maybe), that way we can start moving things to an all 64 bit Pi family and optimise for ARMv8 only. Aren't 512MB RAM Modules getting scarce right about now

[jamesh]

People have been working on porting parts of the kernel to 64 bits and getting it upstreamed - keep an eye on the raspberry pi github kernel page for more information.

Once things work well, I hope to see the bcm2835 replaced with a new low power 64 bit chip (single core Cortex A35 maybe), that way we can start moving things to an all 64 bit Pi family and optimise for ARMv8 only. Aren't 512MB RAM Modules getting scarce right about now

With 12M installed base, remember it's important to keep those people up to date as well. I also seriously doubt there will be a A35 based SoC. People want more power, not less.

[bstrobl]

With 12M installed base, remember it's important to keep those people up to date as well. I also seriously doubt there will be a A35 based SoC. People want more power, not less.

Its not about power(some people want/need a low power SoC to run from batteries etc), but instruction set. We are all still using the ARMv6 userspace without the additional benefits such as NEON unless we compile ourselves. It would help greatly to have everyone on the same Platform, especially once 64 bit is well functioning. Unless the plan is to keep a 32 bit image around for a very long time(not sure how that's going to work when everyone else is slowly moving to 64 bit, in some cases dropping 32).

[ShiftPlusOne]

Just a heads up that NEON IS being used where it makes a difference.

https://wiki.debian.org/Multiarch/LibraryPathOverview

[bstrobl]

Just a heads up that NEON IS being used where it makes a difference.

https://wiki.debian.org/Multiarch/LibraryPathOverview

Ah ok, stand corrected on that one. Heard that the armv6/armv7 kernel can be combined to one as well, is that likely?

[jamesh]

With 12M installed base, remember it's important to keep those people up to date as well. I also seriously doubt there will be a A35 based SoC. People want more power, not less.

Its not about power(some people want/need a low power SoC to run from batteries etc), but instruction set. We are all still using the ARMv6 userspace without the additional benefits such as NEON unless we compile ourselves. It would help greatly to have everyone on the same Platform, especially once 64 bit is well functioning. Unless the plan is to keep a 32 bit image around for a very long time(not sure how that's going to work when everyone else is slowly moving to 64 bit, in some cases dropping 32).

The point is that everyone ISN'T on the same platform, but still need to be supported. There are at least 6M non-64 bit boards out there, so 32 bit is not going away.

[bstrobl]

The point is that everyone ISN'T on the same platform, but still need to be supported. There are at least 6M non-64 bit boards out there, so 32 bit is not going away.

I understand that backward compatibility is very important, but am not sure how continuing to sell 32 bit only hardware will reduce the eventual support burden. If another 6M older Pis are sold without 64 bit support, its more likely that support will have to be kept for a rather long time. While 6M units for example may only constitute 25% of Pis 5 years in the future, selling another 6M 32 bit units would give it a 50% market share, making an eventual phase out of old software that much harder, especially with ARMv6 being as old as it is right now (no Ubuntu support etc). Not that I am complaining though, my Pi 2 in the datacenter needs updates for as long as possible

[rplantz]

My interest in 64-bit is in the classroom. We use the RPi for teaching introduction to computer organization at our university (bob.cs.sonoma.edu). The subsequent class uses the Arm Edition of Patterson & Hennessy's "Computer Organization and Design." This book uses a subset of AARCH64 on an emulator. P&H use AARCH64 because it is more regular.

Although I say a few words about AARCH64 in my book, I would love to write an AARCH64 version. I know that other boards exist, and that there are other 64-bit OSs for the RPi. But Raspbian on the RPi is stable and a very nice setup for students. RPi3 has enough power for them to read my book online while doing the programming exercises in a terminal window. This allows them to concentrate on the course work and not get distracted by trying to solve OS bugs, etc. This makes life much easier for the instructor.

[cryptohunter]

on other platforms there is no speed advantage in 64 bit code

Oh, never mind. Then I must live in some parallel universe

Anyway, just found article relevant to this topic
http://www.cnx-software.com/2016/03/01/ ... tructions/

A good point made delicately.... nice post fanoush.

[Gavinmc42]

[quote]I understand that backward compatibility is very important[/quote]

Only to those people who looks backwards.
To people who look forwards 32bits is not the future.

An A35 Armv8 with NEON in Zero or A+ form is forward thinking.
It makes sense for those teaching ARMv8 64bit and NEON.
It also runs at half the power? 90mw at 1GHz? < 40ma, that Arduino country.

Looking backwards from 10 years when 80%? of Pi's are 64bit and forced to run 32bit OS's.
Ignoring the 13Million installed base?
Well how many are 64 bit now and could run faster with 64bit OS?
3-4-5-6-7Million Pi3's are being ignored? Pi2 is 64bit now.

Actually I don't need a 64bit OS as I am moving to baremetal ARMv8 + NEON.
I just need ARMv8 Zero's, A+, B+
Single or even dual core would be ok.

The industry is well on it's way to doing mainstream 64Bit OS's for Pi's anyway.
While a 64bit Raspbian would be nice for developing AArch64 stuff, there are painful ways around it.
It will happen anyway.
RPF may have their fingers in the dike at the moment.
but probably in the background they are running around with sandbags or just swapping the wellies for waders

Backwards compatibility for Education?
Well how much penetration does RPF have in the Education market?
Mainly UK and early adopters in other countries. <0.1% of world schools?
How many of UK schools are using Pi and how many actually care about 32 or 64 bits?

http://www.eweek.com/mobile/arm-unveils ... martphones
[quote]A four-core SoC would hit 1.0GHz and consume 90 milliwatts.[/quote]
What 4 cores for 40ma
Single core A35 Zero 10ma?
Nah, give us 4 core Zeros,A+,B+

Die size A35 single core is 0.4mm sq, that's small..
Small die, cheaper/higher yields, more cpu's per wafer
Down side, software guys stay busy.

BCM2835 with 28nm A35 quad core, cheaper?
I see numbers that match 28, 35, it's destiny

[AntonAV]

I understand that backward compatibility is very important

Only to those people who looks backwards.
To people who look forwards 32bits is not the future.

Well how much penetration does RPF have in the Education market?
Mainly UK and early adopters in other countries. <0.1% of world schools?
How many of UK schools are using Pi and how many actually care about 32 or 64 bits?

Thanks for your estimate of the RPFs < 0.1% penetration in the world education market.

It is true that most schools don´t care about 32 / 64 bits but they do care about smooth and timely OS updates: if 64 bits help there than that is a huge plus.

[cryptohunter]

where did you get 64-bit version of Raspbian? I am just curious. Haven't found anything

Its not yet available.
You can run a cut down 64-bit Fedora from here:
https://www.kraxel.org/repos/rpi2/images/
They are minimal to say the least; the Fedora one (arm64-rpi3-f24...) is best as it has a package installer included.
I did "yum install gcc", and got gcc 6.1.1.
The .raw file is the same as a .img file, so you dd it in the normal way.

Alternatively for a full 64-bit OS (Ubuntu, Debian or Android), you will need another board such as the Odroid-C2 or the Pine 64. These both have Cortex-a53 CPU's the same as the Pi3. I prefer the C2, its very fast.

Thanks for this post