Inconsistency in the performance of RPi 3 B+

[mrdone]

While trying to overclock my new RPi 3B+ I noted the following inconsistency in its performance measured by sysbench. For this experiment I removed all overclocking options from config.txt, fully updated Stretch system with apt-get and did rpi-update.

Code: Select all

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.14.68-v7+ #1140 SMP Wed Sep 5 17:17:58 BST 2018 armv7l GNU/Linux

The RPi doesn't have heatsink or any fan on it. The PSU has 5V and 3A output, which I bought together with RPi 3B+. First run is after some period of time with switched off power, two consequent runs are with 1 minute delay and the last one I did without any delay.

Code: Select all

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          81.7408s
    total number of events:              10000
    total time taken by event execution: 326.8867
    per-request statistics:
         min:                                 31.61ms
         avg:                                 32.69ms
         max:                                 81.15ms
         approx.  95 percentile:              37.76ms

Threads fairness:
    events (avg/stddev):           2500.0000/43.93
    execution time (avg/stddev):   81.7217/0.01

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          89.0788s
    total number of events:              10000
    total time taken by event execution: 356.2378
    per-request statistics:
         min:                                 31.62ms
         avg:                                 35.62ms
         max:                                 91.25ms
         approx.  95 percentile:              42.79ms

Threads fairness:
    events (avg/stddev):           2500.0000/50.05
    execution time (avg/stddev):   89.0594/0.01

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          93.9490s
    total number of events:              10000
    total time taken by event execution: 375.7129
    per-request statistics:
         min:                                 31.62ms
         avg:                                 37.57ms
         max:                                105.34ms
         approx.  95 percentile:              44.17ms

Threads fairness:
    events (avg/stddev):           2500.0000/35.90
    execution time (avg/stddev):   93.9282/0.02

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          95.4428s
    total number of events:              10000
    total time taken by event execution: 381.6713
    per-request statistics:
         min:                                 31.62ms
         avg:                                 38.17ms
         max:                                 87.14ms
         approx.  95 percentile:              45.12ms

Threads fairness:
    events (avg/stddev):           2500.0000/40.42
    execution time (avg/stddev):   95.4178/0.01

As you can see from the output the performance went down from avg 32.69ms to avg 38.17ms, while the temperature barely reached 65 degrees celsius. I repeated the same exercise twice shutting down RPi and waiting and the results were quite similar to the above. According to this video https://www.youtube.com/watch?v=SWl0n-uNdLc from EC the temperature starts to affect RPi 3B+ performance only when it is above 70C. So my question is whether I have such strangely behaving RPi sample or it is how all RPi 3B+ behave and then what is really affecting the performance of RPi 3B+?

PS: Just to intrigue you more I did the same exercise after switching off power for quite some period and venting the RPi manually with a piece of paper (I do not have a proper fan). Guess what, the avg time was, it was around 79ms

[jahboater]

It starts (soft) throttling at 60C with the latest Raspbian (the clock speed drops from 1400MHz to 1200MHz)

Try this in /boot/config.txt

temp_soft_limit=70

[mrdone]

Sorry, it seems that it didn't copy a new line characters after the command. The correct command line is:

Code: Select all

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run

Following you advice, I have added temp_soft_limit=70 into /boot/config.txt and it seems to work, although the first run produced very strange results with avg 147ms the consequent run gave avg 32.71ms shooting temp to almost 70C very fast.

Code: Select all

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          368.3268s
    total number of events:              10000
    total time taken by event execution: 1473.2409
    per-request statistics:
         min:                                 31.62ms
         avg:                                147.32ms
         max:                             286678.20ms
         approx.  95 percentile:              37.18ms

Threads fairness:
    events (avg/stddev):           2500.0000/56.72
    execution time (avg/stddev):   368.3102/0.01

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          81.7914s
    total number of events:              10000
    total time taken by event execution: 327.0815
    per-request statistics:
         min:                                 31.62ms
         avg:                                 32.71ms
         max:                                 74.83ms
         approx.  95 percentile:              37.11ms

Threads fairness:
    events (avg/stddev):           2500.0000/4.64
    execution time (avg/stddev):   81.7704/0.01

Do you think it is safe to raise temp_soft_limit to 65 and do you know why did they lowered it so much at first place?

[jamesh]

Sorry, it seems that it didn't copy a new line characters after the command. The correct command line is:

Code: Select all

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run

Following you advice, I have added temp_soft_limit=70 into /boot/config.txt and it seems to work, although the first run produced very strange results with avg 147ms the consequent run gave avg 32.71ms shooting temp to almost 70C very fast.

Code: Select all

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          368.3268s
    total number of events:              10000
    total time taken by event execution: 1473.2409
    per-request statistics:
         min:                                 31.62ms
         avg:                                147.32ms
         max:                             286678.20ms
         approx.  95 percentile:              37.18ms

Threads fairness:
    events (avg/stddev):           2500.0000/56.72
    execution time (avg/stddev):   368.3102/0.01

pi@raspberrypi:~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          81.7914s
    total number of events:              10000
    total time taken by event execution: 327.0815
    per-request statistics:
         min:                                 31.62ms
         avg:                                 32.71ms
         max:                                 74.83ms
         approx.  95 percentile:              37.11ms

Threads fairness:
    events (avg/stddev):           2500.0000/4.64
    execution time (avg/stddev):   81.7704/0.01

Do you think it is safe to raise temp_soft_limit to 65 and do you know why did they lowered it so much at first place?

It was lowered because some of the original Pi3B+ SoC were not stable under some circumstances. Setting a soft limit traded off peak vs average performance - more time at 1200 is better than peaking at 1400 and dropping to 800 as things got hot.

Most 3B+ should be fine at 70, but it would be worth running your tests and finding out which gives the best average performance at different limits. Adding a heatsink would also improve performance although its not necessary for a stability Pov.

[HawaiianPi]

While trying to overclock my new RPi 3B+ I noted the following inconsistency in its performance measured by sysbench. For this experiment I removed all overclocking options from config.txt, fully updated Stretch system with apt-get and did rpi-update.

The 3B+ downclocks to the 3B speed of 1200MHz at 60C. It used to be 70C but was reduced in recent versions of Raspbian to address issues found in some early 3B+ models. Supposedly those are not common and most 3B+ computers may run fine at the original 70C setting, so it's probably okay to use the setting recommended above, but if you want to be sure a heatsink wouldn't hurt.

If it reaches 80C it behaves the same as the 3B model, which is to say it will throttle even further, and at 85C it will drop to its minimum speed of 600MHz (and I believe it even disables some cores).

Also note that the 3B+ uses the system board ground plane and ports to help dissipate heat, so a Pi3B+ that has many ports in use (HDMI, Ethernet, USB) should run cooler than a headless unit.

I have most of my Pi3B and 3B+ computers in Flirc Gen2 aluminum alloy cases. The case itself acts as a huge heatsink to keep my Pi3B(+) computers running cool, without the noise, dust and additional power consumption of fans (and they look really cool as well).

[mrdone]

Thanks for a quick reply! I use conky widget to monitor frequency, but during the test it always shows 1.4Ghz , although there certainly were drops in frequency. Given that sysbench sometimes produces very strange results and conky is not very reliable can you recommend a better tools to test RPi's performance?

[jahboater]

My Pi3B+ consistently takes 31.6ms for the "avg" time.
There is no throttling because it has a heatsink.
Here is the log:-

Code: Select all

Time      Temp    CPU        Throttle       Vcore
10:09:11 41.3'C  600MHz 0000000000000000000 1.2V
10:09:21 42.9'C 1400MHz 0000000000000000000 1.3250V
10:09:32 43.5'C 1400MHz 0000000000000000000 1.3250V
10:09:42 45.1'C 1400MHz 0000000000000000000 1.3250V
10:09:52 45.1'C 1400MHz 0000000000000000000 1.3250V
10:10:03 45.1'C 1400MHz 0000000000000000000 1.3250V
10:10:13 46.2'C 1400MHz 0000000000000000000 1.3250V
10:10:23 46.2'C 1400MHz 0000000000000000000 1.3250V
10:10:34 47.2'C 1400MHz 0000000000000000000 1.3250V
10:10:44 45.1'C  600MHz 0000000000000000000 1.2V
10:10:54 44.0'C  600MHz 0000000000000000000 1.2V

[jahboater]

Following you advice, I have added temp_soft_limit=70 into /boot/config.txt and it seems to work, although the first run produced very strange results with avg 147ms the consequent run gave avg 32.71ms shooting temp to almost 70C very fast.

I think you should look into the cooling.
If you look at the temperature column in my previous post you can see it is very stable and only rises by 6 degrees during the sysbench run.

[mrdone]

The case looks great, thanx for sharing! Unfortunately, it is not available in my country

[HawaiianPi]

The case looks great, thanx for sharing! Unfortunately, it is not available in my country

Where are you located (you can put your location in your forum profile). Perhaps someone here could suggest an alternative. There are also Pi vendors who will ship almost anywhere.

[mrdone]

Readings from your RPi made me jealous I've just installed a set of heatsinks that I bought earlier, but even during light browsing temp stays around 55C. What heatsink do you use? And can your share the script you used to measure temp, freq, and voltage?

[jahboater]

HawaiianPi,

If you have some spare time, please could you run this on a 3B+ in a FLIRC case, noting the ambient temp.
I am interested in how it compares to the Kintaro heatsink.
The log in my prior post was run at 24C ambient, giving a delta-T of 23C at the peak temp during the run.
You may need "sudo apt install sysbench" if you dont already have it.

Code: Select all

sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run

My log script:

Code: Select all

#!/bin/bash
Counter=14
DisplayHeader="Time      Temp    CPU        Throttle       Vcore"
while true ; do
  let Counter++
  if [ ${Counter} -eq 15 ]; then
    echo -e "${DisplayHeader}"
    Counter=0
  fi
  Health=$(perl -e "printf \"%19b\n\", $(vcgencmd get_throttled | cut -f2 -d=)")
  Temp=$(vcgencmd measure_temp | cut -f2 -d=)
  Clockspeed=$(vcgencmd measure_clock arm | awk -F"=" '{printf ("%0.0f",$2/1000000); }' )
  CoreVoltage=$(vcgencmd measure_volts | cut -f2 -d= | sed 's/000//')
  echo -e "$(date '+%H:%M:%S') ${Temp} $(printf '%4s' ${Clockspeed})MHz $(printf '%019d' ${Health}) ${CoreVoltage}"
  sleep 10
done

[jahboater]

Readings from your RPi made me jealous I've just installed a set of heatsinks that I bought earlier, but even during light browsing temp stays around 55C. What heatsink do you use? And can your share the script you used to measure temp, freq, and voltage?

Please see the script in my last post.

I use the Kintaro heatsink
https://www.kintaro.co/products/kintaro-custom-heatsink
You need the 3B+ version (sadly Amazon appear to have sold out, and only have 2B/3B ones).

Looking at your three, I think the blue one should give the best results, but make sure the fins are aligned vertically (and the Pi is on edge) for best performance. And please make sure the sticky tape is proper thermal tape (usually 3M).
The Kintaro heatsink uses decent thermal paste TIM because it is bolted on (much better).

[RaTTuS]

https://www.aliexpress.com/item/Raspber ... 15484.html will probably ship to where ever you are .. there are other options you can use

[jahboater]

Readings from your RPi made me jealous I've just installed a set of heatsinks that I bought earlier, but even during light browsing temp stays around 55C. What heatsink do you use? And can your share the script you used to measure temp, freq, and voltage?

If you want to test a heat sink, this little program uses NEON on all four cores and will heat the Pi up much faster than sysbench.

Code: Select all

wget https://raw.githubusercontent.com/ssvb/cpuburn-arm/master/cpuburn-a53.S
gcc -o cpuburn-a53 cpuburn-a53.S
./cpuburn-a53

You can run "./cpuburn-a53" at any time.

Note this program will cause the CPU to throttle, but it must not crash!
Leave it running for half an hour or so.
With my Kintaro heatsink it just starts to throttle after 9 minutes (at 70C) which shows how good the thermal and power management of the new 3B+ model is.

[mrdone]

RaTTuS, thanks for a suggestion! Although Kintaro heatsink looks more appealing, it seems that it is out of stock everywhere, so probably I will follow your recommendation

[mrdone]

jahboater, cpuburn got my RPi 3B+ to 80C in 70sec and caused it to go down beyond 1200 So, I think, the marvelous temps you get a mainly due to the Kintaro heatsink, although I should admit that 3B+ is a step from 3B in right direction. I wonder why foundation doesn't produce heatsinks themselves. It doesn't look like a titanic job, while right cooling may prevent a lot of problems. Meanwhile, I am really looking forward to see the results of testing of Flirc Gen2 by HawaiianPi to understand whether to buy an aluminium case from aliexpress suggested by RaTTuS or to wait till there will be spare Kintaros for sale.

[jahboater]

jahboater, cpuburn got my RPi 3B+ to 80C in 70sec and caused it to go down beyond 1200 So, I think, the marvelous temps you get a mainly due to the Kintaro heatsink, although I should admit that 3B+ is a step from 3B in right direction.

Cpu burn is a stress test and not representative of normal work loads. If your Pi can run it for a period without crashing, then you can be confident that it is totally stable when doing real work. It gives you a safety margin. It is also a good measure of how well a heat sink is performing. There is also the famous Linpack benchmark that tests the calculations as it goes. Here is a precompiled version: https://www.raspberrypi.org/forums/view ... 12#p926798

The 3B+ has considerable "thermal inertia" so it can run reasonable workloads for a few minutes without throttling. I suspect for most users, most of the time, the 3B+ will be OK with little or no additional cooling. If you are just twiddling LED's or learning to program, the 3B+ will stay cool.

[bensimmo]

Thanks for a quick reply! I use conky widget to monitor frequency, but during the test it always shows 1.4Ghz , although there certainly were drops in frequency. Given that sysbench sometimes produces very strange results and conky is not very reliable can you recommend a better tools to test RPi's performance?

Maybe and probably because the Pi does not report its true speed via the usual reporting path due to the way the Pi works.
Make sure it is reading (most things Pi) with
vcgencmd commands.
I.e. to read temp is vcgencmd measure_temp
CPU speed
vcgencmd measure_clock arm

[mrdone]

jahboater, the blessing and the curse of RPi 3B and 3B+ is that they are very close to a decent substitute of a desktop PC. That is why you always want to push it a bit harder to have smoother user experience.

For example, I didn't buy RPi to substitute my desktop. I bought it to run some Pyhon scripts to crawl the web 24/7, but since motherboard of my desktop PC had died I was forced to use it as a desktop replacement. And I suspect that a lot of those who bought RPi to learn programming or for whatever other educational or not so much purpose quite soon realize that they might not need another PC at home. And it is only later they realize that their lovely RPi is suitable for almost everything, but some very needed stuff. At this point they want to push RPi's performance a bit harder to be able to browse mail with new gmail interface without huge lags or to be able to use google maps (which, for whatever reason, are completely unusable with RPi, while are quite smooth if you use them with any browser at mobile phone ).

I guess my point is that it never hearts to allow people to squeeze every bit of performance of a SBC they can even if initially it was intended for a very basic stuff. There are so many interesting use case scenarios that RPi was not designed for, but which appeared just recently and they are really exciting. I think Foundation should allow people to decide for themselves what they need and what they don't. For example, I really do not understand why they can not produce model C with characteristics comparable to other top performing SMC. As an economist, I do not buy an argument that they want to produce SBC as affordable as possible and that is why they are so much limited in technical solutions that they can't use USB 3. From what I see, there is a strong demand for more powerful model for which they can charge price which is triple of the current price. At the same time, this will allow them to sell model B to kids from 3rd world at the price of its production cost, which I believe is much cheaper than the current price. Sorry for so many words

[jahboater]

The blessing and the curse of RPi 3B and 3B+ is that they are very close to a decent substitute of a desktop PC.

Very close I would say. I recently "house-sat" someones else house while they were away on holiday.
Rather than take a big Intel PC or a laptop, I took a 3B+, a 19in monitor, mouse and keyboard. Just as an experiment!
Could I use it for a week as my only computer?

Yes! Yes! Yes! it worked fine.
A £32 SBC I definitely could live with as my only computer!

[mrdone]

bensimmo, thanx for the suggestion! I started to use the script shared by jahboater, which happen to use the commands you have mentioned.

[LTolledo]

Could I use it for a week as my only computer?

Yes! Yes! Yes! it worked fine.
A £32 SBC I definitely could live with as my only computer!

Been using this RPi3B+ as desktop PC for over 5 months, after replacing my RPi3B which was used as desktop PC for over a year.
Am accessing and writing to this forum using my RPi3B+.
I've demoted my Intel/AMD PCs for just playing online games. Other than that they're just collecting dust...

During my travels, I always bring my (then) RPi3B, Wireless Keyboard and mouse, PSU, HDMI cable, and connect it to hotel's TV and WiFi.
Its sufficient for my requirements...

[HawaiianPi]

HawaiianPi,

If you have some spare time, please could you run this on a 3B+ in a FLIRC case, noting the ambient temp.
I am interested in how it compares to the Kintaro heatsink.

Raspberry Pi 3B+ in Flirc Gen2 case with updated Raspbian Stretch Desktop OS booted from a SanDisk Ultra A1 32GB uSD card. I am using the thermal pad supplied by Flirc with the case (no additional cooling in use). System is connected to a 1080p HDMI monitor, speakers, and a Logitech K400r wireless keyboard using their Unifying dongle. Internet connection is via built-in WiFi.

According to the weather guessers it's currently 60F/15.5C outside, but the thermometer on my AC (which is currently idle) reads 68F/20C.

Before I tried your logging script I ran a test with my own temperature script immediately before and after the test.

Code: Select all

~ $ pitemp;sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run;pitemp

LoloPi3 Mon Sep 10 22:46:11 PDT 2018
Raspberry Pi 3 Model B Plus Rev 1.3
GPU 42.9 C
CPU 42 C

throttled=0x0

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          79.6554s
    total number of events:              10000
    total time taken by event execution: 318.5791
    per-request statistics:
         min:                                 31.62ms
         avg:                                 31.86ms
         max:                                 51.03ms
         approx.  95 percentile:              32.17ms

Threads fairness:
    events (avg/stddev):           2500.0000/23.04
    execution time (avg/stddev):   79.6448/0.01


LoloPi3 Mon Sep 10 22:47:31 PDT 2018
Raspberry Pi 3 Model B Plus Rev 1.3
GPU 52.1 C
CPU 51 C

throttled=0x0

Then I created your script and here's the result I logged.

Code: Select all

~ $ ./templog.sh
Time      Temp    CPU        Throttle       Vcore
22:52:53 38.1'C  600MHz 0000000000000000000 1.2V
22:53:03 42.9'C 1400MHz 0000000000000000000 1.3750V
22:53:13 45.1'C 1400MHz 0000000000000000000 1.3750V
22:53:24 46.2'C 1400MHz 0000000000000000000 1.3750V
22:53:34 46.2'C 1400MHz 0000000000000000000 1.3750V
22:53:44 46.7'C 1400MHz 0000000000000000000 1.3750V
22:53:55 47.8'C 1400MHz 0000000000000000000 1.3750V
22:54:05 47.2'C 1400MHz 0000000000000000000 1.3750V
22:54:15 47.8'C 1400MHz 0000000000000000000 1.3750V
22:54:25 41.9'C  600MHz 0000000000000000000 1.2V

~ $ sysbench --test=cpu --cpu-max-prime=20000 --num-threads=4 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          79.8253s
    total number of events:              10000
    total time taken by event execution: 319.2578
    per-request statistics:
         min:                                 31.62ms
         avg:                                 31.93ms
         max:                                 76.77ms
         approx.  95 percentile:              32.22ms

Threads fairness:
    events (avg/stddev):           2500.0000/23.04
    execution time (avg/stddev):   79.8144/0.00

And you guys must have the patience of Job to use your Pi as a daily driver. I do have one 3B+ set up as a second/backup computer, and I find it frustratingly slow when I use it.

[mrdone]

HawaiianPi, thank you very much for sharing these results with us. They look cool in both senses of this word Unfortunately, Flirc Gen2 with shipping to my country that I found costed a bit too much, so I have ordered another aluminium case suggested in this thread by RaTTuS at aliexpress and if it will show similar performance I will be more than happy.

Concerning RPi as a daily driver. A boot time of 11sec is almost a killer feature by itself. Another astonishing thing for a Windows user is an update of 234Mb that will use additionally only 1024b of disk space