Actual memory bandwidth of raspberry pi4?

[cleverca22]

A72 utilizes ARM's crypto extension included in ARMv8 and the AES performance tested on Geekbench 3 is about 1G/(s * 2.3GHz)https://www.anandtech.com/show/9878/the ... 8-review/3 so for RPi 4 3.5G/s AES performance is expected when OC to 2GHz which is enough to make memory bottleneck

the pi4 core lacks the crypto extensions

[Gnyueh]

At any rate, the block diagram showing how the cache and memory controller are laid out reminded me that I've been planning to run the same program on the Raspberry Pi 4B. I'll post results soon.

Here is a graph showing cache memory performance for the Pi 4B.



Each consecutive dot represents a buffer size that has been increased by a factor of two. The graphs seem to imply the 3B has more level 1 cache than the 4B but that the 4B has more level 2 cache than the 3B. Is that actually true?

Since the 4B and Zero graphs were made using gcc version 10.1 running the current version of Raspberry Pi OS while the graphs for the 3B were made using a gcc 5.x and a much older Raspian, could it be possible the cache footprint of the new kernel and systemd make it appear the 4B has less level 1 cache?

That is true.
A53 L1-D 64K L2 None (VC4 L2 is not included)
A72 L1-D 32K L2 1M

[Gnyueh]

A72 utilizes ARM's crypto extension included in ARMv8 and the AES performance tested on Geekbench 3 is about 1G/(s * 2.3GHz)https://www.anandtech.com/show/9878/the ... 8-review/3 so for RPi 4 3.5G/s AES performance is expected when OC to 2GHz which is enough to make memory bottleneck

the pi4 core lacks the crypto extensions

I didn't even realize it. I thought Crypto extension is required in an A72 core. Thanks a lot for your information. Maybe I will switch my VPN to the x86 based board computer for better performance or simply use chacha.

[cleverca22]

A53 L1-D 64K L2 None (VC4 L2 is not included)
A72 L1-D 32K L2 1M

do you have more information on how much L1/L2 every model of pi has?

[Paeryn]

A53 L1-D 64K L2 None (VC4 L2 is not included)
A72 L1-D 32K L2 1M

I thought the A53s in the RPi3 have 512KB of L2 cache and 32K of L1 (16K Instruction & 16K Data) and only the single core RPi0 & 1 don't have a dedicated L2 cache (they share the VC4's L2 cache).

[jamesh]

Just realised that the processor pages in the docs need a little more love in the cache department!

[ejolson]

Just realised that the processor pages in the docs need a little more love in the cache department!

Is this

https://www.raspberrypi.org/documentati ... spberrypi/

the page that will be updated? When it's ready, where would I find documentation that contains the cache size?

[jamesh]

I guess in the subpages specifically about the processors. Not really thought about it yet.

[Gnyueh]

I guess in the subpages specifically about the processors. Not really thought about it yet.

In that page supported memory speed is reported as 2400MHz while according to some discussion in the forum the speed is 3200MHz......
viewtopic.php?f=63&t=263562&p=1604269&h ... 0#p1604269
Also adding some block diagrams like this would be easier to understand just like these:

[ejolson]

I guess in the subpages specifically about the processors. Not really thought about it yet.

In that page supported memory speed is reported as 2400MHz while according to some discussion in the forum the speed is 3200MHz......

I think the chip is rated for 3200 MHz but is only run at 2400 MHz, possibly due to limitations in the SOC or routing on the circuit board. Indeed, the memory bus on the 4B is noticeably slower than the same on many other computers. At the same time, the price is noticeably cheaper.

[Gnyueh]

I guess in the subpages specifically about the processors. Not really thought about it yet.

In that page supported memory speed is reported as 2400MHz while according to some discussion in the forum the speed is 3200MHz......

I think the chip is rated for 3200 MHz but is only run at 2400 MHz, possibly due to limitations in the SOC or routing on the circuit board. Indeed, the memory bus on the 4B is noticeably slower than the same on many other computers. At the same time, the price is noticeably cheaper.

Actually it is rated for 3733MHz https://www.micron.com/products/dram/lp ... dt-053-aat (abbreviation of the code is printed on memory chip), and that is why I am a bit confused. Maybe there is some inconsistency.
I have compared many single board PC and of course RPi 4 is one with high cost performance dispite not so satisfying memory performance.

[cleverca22]

I think the chip is rated for 3200 MHz but is only run at 2400 MHz, possibly due to limitations in the SOC or routing on the circuit board. Indeed, the memory bus on the 4B is noticeably slower than the same on many other computers. At the same time, the price is noticeably cheaper.

https://gist.github.com/cleverca22/84b7 ... 73f895601f

this is the logs from BOOT_UART=1 in bootconf.txt with a 4gig pi4, of note is this line:

Code: Select all

Initialising SDRAM 'Micron' 16Gb x2 total-size: 32 Gbit 3200

i believe the 3200 here is the clock speed its running the dram at

[Gnyueh]

I think the chip is rated for 3200 MHz but is only run at 2400 MHz, possibly due to limitations in the SOC or routing on the circuit board. Indeed, the memory bus on the 4B is noticeably slower than the same on many other computers. At the same time, the price is noticeably cheaper.

https://gist.github.com/cleverca22/84b7 ... 73f895601f

this is the logs from BOOT_UART=1 in bootconf.txt with a 4gig pi4, of note is this line:

Code: Select all

Initialising SDRAM 'Micron' 16Gb x2 total-size: 32 Gbit 3200

i believe the 3200 here is the clock speed its running the dram at

Thanks a lot for your update. Maybe there is some inconsistency in this page https://www.raspberrypi.org/documentati ... /README.md , this page is also not updated for the 8G model (max capacity).

[ejolson]

I think the chip is rated for 3200 MHz but is only run at 2400 MHz, possibly due to limitations in the SOC or routing on the circuit board. Indeed, the memory bus on the 4B is noticeably slower than the same on many other computers. At the same time, the price is noticeably cheaper.

https://gist.github.com/cleverca22/84b7 ... 73f895601f

this is the logs from BOOT_UART=1 in bootconf.txt with a 4gig pi4, of note is this line:

Code: Select all

Initialising SDRAM 'Micron' 16Gb x2 total-size: 32 Gbit 3200

i believe the 3200 here is the clock speed its running the dram at

Thanks a lot for your update. Maybe there is some inconsistency in this page https://www.raspberrypi.org/documentati ... /README.md , this page is also not updated for the 8G model (max capacity).

While the memory is running slower than the part rating, it looks like I was wrong about the numbers. Maybe it's time to rerun the memory bandwidth test to see if the new kernel changed anything.

[Gnyueh]

https://gist.github.com/cleverca22/84b7 ... 73f895601f

this is the logs from BOOT_UART=1 in bootconf.txt with a 4gig pi4, of note is this line:

Code: Select all

Initialising SDRAM 'Micron' 16Gb x2 total-size: 32 Gbit 3200

i believe the 3200 here is the clock speed its running the dram at

Thanks a lot for your update. Maybe there is some inconsistency in this page https://www.raspberrypi.org/documentati ... /README.md , this page is also not updated for the 8G model (max capacity).

While the memory is running slower than the part rating, it looks like I was wrong about the numbers. Maybe it's time to rerun the memory bandwidth test to see if the new kernel changed anything.

Welp I think 3200MHz is correct, the slow speed and bad scaling with increasing core number probably result from the memory controller inside VC6(the single thread memory performance is fine but it decrease when thread number increases). I am looking forward to your result updates on new kernel.

[cleverca22]

another factor i heard about on irc, is that the dram cant transfer at the full rate you would assume, from just bus_width * clock

because a certain percentage of the time is taken up by dram refresh cycles
and there is a certain number of clock cycles you must wait for, every time you open a new row in the dram

so each burst of transfer may move at the full width*clock rate, but the dead times between transfers enforce a lower average rate

[jamesh]

I cannot be particularly detailed (NDA), but it worth noting that everyday memory bandwidth usage will, on average, be higher than pathological test case memory tests seen here. We would probably expect anywhere between 20 and 40% better performance in real world usage than that show in some memory test systems. It would certainly be possible, given what we know about how everything works, to 'game' these benchmarks and get much higher speeds reported, along the lines of that 20-40% figure, but we don't do that. I cannot be sure about other manufacturers ethics on this.

That fact that the vast majority of people don't even notice stuff like that, means that it works pretty well.

We know all this because we tested all this before the Pi4 release.

[Gnyueh]

another factor i heard about on irc, is that the dram cant transfer at the full rate you would assume, from just bus_width * clock

because a certain percentage of the time is taken up by dram refresh cycles
and there is a certain number of clock cycles you must wait for, every time you open a new row in the dram

so each burst of transfer may move at the full width*clock rate, but the dead times between transfers enforce a lower average rate

Refresh usually doesnt take too much time. I have uploaded results of my icelake laptop with LPDDR4 3733 memory in an earlier post in this discussion and 80% theoratical BW is achieved according to calculation.

[Gnyueh]

I cannot be particularly detailed (NDA), but it worth noting that everyday memory bandwidth usage will, on average, be higher than pathological test case memory tests seen here. We would probably expect anywhere between 20 and 40% better performance in real world usage than that show in some memory test systems. It would certainly be possible, given what we know about how everything works, to 'game' these benchmarks and get much higher speeds reported, along the lines of that 20-40% figure, but we don't do that. I cannot be sure about other manufacturers ethics on this.

That fact that the vast majority of people don't even notice stuff like that, means that it works pretty well.

We know all this because we tested all this before the Pi4 release.

Welp that is interesting, the BW test is of course not that real world but it indicates the performace of memory controller in the specific dimension, which is meaningful, and how about the BW scaling with thread increasing in internal test? I wonder rather believe these disappointing results are not real.

[jamesh]

I cannot be particularly detailed (NDA), but it worth noting that everyday memory bandwidth usage will, on average, be higher than pathological test case memory tests seen here. We would probably expect anywhere between 20 and 40% better performance in real world usage than that show in some memory test systems. It would certainly be possible, given what we know about how everything works, to 'game' these benchmarks and get much higher speeds reported, along the lines of that 20-40% figure, but we don't do that. I cannot be sure about other manufacturers ethics on this.

That fact that the vast majority of people don't even notice stuff like that, means that it works pretty well.

We know all this because we tested all this before the Pi4 release.

Welp that is interesting, the BW test is of course not that real world but it indicates the performace of memory controller in the specific dimension, which is meaningful, and how about the BW scaling with thread increasing in internal test? I wonder rather believe these disappointing results are not real.

The problem with most benchmarks is that the only thing they really benchmark is the benchmark.

[ejolson]

I cannot be particularly detailed (NDA), but it worth noting that everyday memory bandwidth usage will, on average, be higher than pathological test case memory tests seen here. We would probably expect anywhere between 20 and 40% better performance in real world usage than that show in some memory test systems. It would certainly be possible, given what we know about how everything works, to 'game' these benchmarks and get much higher speeds reported, along the lines of that 20-40% figure, but we don't do that. I cannot be sure about other manufacturers ethics on this.

Welp that is interesting, the BW test is of course not that real world but it indicates the performace of memory controller in the specific dimension, which is meaningful, and how about the BW scaling with thread increasing in internal test? I wonder rather believe these disappointing results are not real.

From reading the thread

viewtopic.php?t=271121

it is clear that different people obtained and confirmed the same result, so there is no error in the sense of reproducibility. Instructions are also given in that thread, if you want to run the test yourself.

As I described where

While stream is definitely a synthetic RAM benchmark, in my opinion copying memory around is not uncommon as are sections of code that add and rescale vectors of numbers. The stream kernels were originally developed to represent the kinds of vector operations used in a real-world ocean-modelling program and explain why the Cray supercomputers performed so well when performing such calculations. More information is available at

http://www.cs.virginia.edu/~mccalpin/ST ... -01-25.pdf

which appears in

viewtopic.php?p=1647290#p1647290

When performing similar operations in practical code, it becomes important to understand the hardware well enough to know whether memory contention will result in the elapsed time of the parallel computation being more or less.

Note that scheduling is not a problem as these are trivially parallel operations. The difficulty is that even though there are many cores, they all share the same memory bus. Moreover, the nature of the stream vector kernels allows a single fast core to max out available bandwidth on the 4B's memory bus.

This is one reason the extra memory channels on high core-count EPYC and Power9 processors are attractive for parallel scaling. It is also why the high-bandwidth memory of the Fujitsu AFX64 led to a machine that absolutely trounces other systems in per-core efficiency on problems similar to the high-performance conjugate gradient. The advantage of such designs become most obvious when compared to the memory controllers used in the ShenWei SW26010 and, of course, the Raspberry Pi.

[Gnyueh]

I cannot be particularly detailed (NDA), but it worth noting that everyday memory bandwidth usage will, on average, be higher than pathological test case memory tests seen here. We would probably expect anywhere between 20 and 40% better performance in real world usage than that show in some memory test systems. It would certainly be possible, given what we know about how everything works, to 'game' these benchmarks and get much higher speeds reported, along the lines of that 20-40% figure, but we don't do that. I cannot be sure about other manufacturers ethics on this.

Welp that is interesting, the BW test is of course not that real world but it indicates the performace of memory controller in the specific dimension, which is meaningful, and how about the BW scaling with thread increasing in internal test? I wonder rather believe these disappointing results are not real.

From reading the thread

viewtopic.php?t=271121

it is clear that different people obtained and confirmed the same result, so there is no error in the sense of reproducibility. Instructions are also given in that thread, if you want to run the test yourself.

As described where

While stream is definitely a synthetic RAM benchmark, in my opinion copying memory around is not uncommon as are sections of code that add and rescale vectors of numbers. The stream kernels were originally developed to represent the kinds of vector operations used in a real-world ocean-modelling program and explain why the Cray supercomputers performed so well when performing such calculations. More information is available at

http://www.cs.virginia.edu/~mccalpin/ST ... -01-25.pdf

which appears in

viewtopic.php?p=1647290#p1647290

When performing similar calculations in practical code, it becomes important to understand the hardware well enough to know whether memory contention will result in the elapsed time of the parallel code being more or less.

Note that scheduling is not a problem as these are trivially parallel operations. The difficulty is that even though there are many cores, they all share the same memory bus. Moreover, the nature of the stream vector kernels allows a single fast core to max out available bandwidth on the memory bus.

This is one reason the extra memory channels on high core-count EPYC and Power9 processors are attractive for parallel scaling in high-performance computing. It is also why the high-bandwidth memory of the Fujitsu AFX64 led to a machine that absolutely trounces other systems in per-core efficiency on problems similar to the high-performance conjugate gradient. The advantage of such designs become most obvious when compared to the memory controllers used in the ShenWei SW26010 and, of course, the Raspberry Pi.

These numbers are just too bad to be true......(They are true after all) I friend of mine found memory latency increasing with threads numbers, which coincides the BW test results

Fujitsu AFX64 is the only processor that utilizes HBM memory and that makes it a performance monster.

For RPi4 I think it is basically VC6's fault.

[jamesh]

For RPi4 I think it is basically VC6's fault.

You may think that. We don't.

[Gnyueh]

For RPi4 I think it is basically VC6's fault.

You may think that. We don't.

Whatever, you are under NDA and cannot say anything more anyway.

[Heater]

I wonder rather believe these disappointing results are not real.
...
The problem with most benchmarks is that the only thing they really benchmark is the benchmark.

Better believe it.

We all know that all these little benchmarks are pretty useless on their own.

However whilst ejolson's OpenMP memory test may be extreme we now have loads of examples that scaling with number of cores is far less effective on the Pi than Intel x86, Jetson Nano and so on.

My examples include the "Million Digit Fibonacci challenge" in C++ https://github.com/ZiCog/fibo_4784969
and the a convolution over 20 million elements: https://github.com/ZiCog/rust_convolution

Feel free to try them out.