Heater
Posts: 15811
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspberry Pi Benchmarks

Mon Dec 16, 2019 8:12 pm

I'm not sure what I think. The statement makes me think that there are tons of benchmark results here but done in uncontrolled and/or unmonitored conditions. Thus rendering them all meaningless!
Memory in C++ is a leaky abstraction .

ejolson
Posts: 5143
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi Benchmarks

Mon Dec 16, 2019 8:40 pm

Heater wrote:
Mon Dec 16, 2019 8:12 pm
I'm not sure what I think. The statement makes me think that there are tons of benchmark results here but done in uncontrolled and/or unmonitored conditions. Thus rendering them all meaningless!
Another difficulty is that--in addition to ambient temperature--variations in manufacturing appear to make some SOCs more prone to overheating than others. To obtain meaningful statistics, one would need to bake an ensemble of 100 or more raspberry pies at different temperatures just to get a taste for the resulting performance characteristics.

If you have the time for such things, I would suggest first obtaining suitable grant funding. In addition to providing important performance and reliability data, such a baking project might offer enough publicity--if accompanied by a suitable blog with video clips--to make it priority fundable. On the other hand, I'm just another person with a seat in the peanut gallery, so maybe not, either.

emma1997
Posts: 764
Joined: Sun Nov 08, 2015 7:00 pm
Location: New England (not that old one)

Re: Raspberry Pi Benchmarks

Mon Dec 16, 2019 11:59 pm

ejolson wrote:
Mon Dec 16, 2019 8:40 pm
To obtain meaningful statistics, one would need to bake an ensemble of 100 or more raspberry pies at different temperatures just to get a taste for the resulting performance characteristics.
For me to get a 'taste' it's enough to count out boot time (one ringy-dingy, two ringy-dingy, three...), see how long it takes to load in a browser, or copy a big file. But then I'm also a peanut (or some kind of nut, lol) and not so much interested in comparing exotic compilers or fine tuning fake benchmarks.

Standard Temperature and Pressure are good enough for me too. Anything close to room temp. No need for thermostatic controlled autoclaves or liquid nitrogen chip coolers. No heatsinks to be fair, unless one comes with it installed from factory. Maybe set Pi on edge if that.

On the other hand I do think it's ridiculous to say benchmarks like these are of zero use. Not perfect but not totally pointless either. If nothing else floats the boat for some who are into that stuff.

ejolson
Posts: 5143
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi Benchmarks

Tue Dec 17, 2019 12:30 am

emma1997 wrote:
Mon Dec 16, 2019 11:59 pm
Standard Temperature and Pressure are good enough for me too. Anything close to room temp. No need for thermostatic controlled autoclaves or liquid nitrogen chip coolers.
While I generally agree with what you've said, the difficulty with standard room temperature is that it varies from about 60F to 90F (15C to 32C) depending on where a person lives and the season. For example, during winter the ambient room temperature here in the high desert is about 54F in the morning. During summer afternoon room temperatures can easily surpass 90F without air-conditioning, an evaporative cooler or conveniently placed trees. The present house is fortunately air-conditioned; the others were not and surely not everyone's house is.

jahboater
Posts: 5646
Joined: Wed Feb 04, 2015 6:38 pm
Location: West Dorset

Re: Raspberry Pi Benchmarks

Wed Dec 18, 2019 10:42 am

Could we say that at end of each benchmark run, the command "vcgencmd get_thottled" must return zero?
Then temperature, cooling, power supply issues are moot.

ejolson
Posts: 5143
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi Benchmarks

Wed Dec 18, 2019 3:58 pm

jahboater wrote:
Wed Dec 18, 2019 10:42 am
Could we say that at end of each benchmark run, the command "vcgencmd get_thottled" must return zero?
Then temperature, cooling, power supply issues are moot.
I recollect, possibly wrongly, that the 3B+ can throttle from 1400 to 1200 MHz without tripping the get_throttled flag. Is something similar possible with the 4B?

In a way, the most certain test whether the system goes into throttling is the measurement of known benchmark performance as a function of time. To be absolutely sure the work is getting done as fast as it should be, one may also need an external time source. Note that wrong assumptions about the way certain ARM processors report processor speeds have been discovered in recent times by such methods. In particular, if you are paying attention to performance, it becomes immediately obvious when a Linux driver says one thing but the processor is doing something else.

Back on topic, since source code is provided, it should be possible to reproduce the top performance results given here to determine whether your Pi is running correctly. Not only does such comparison check for throttling but it can also detect gradual degradation in performance (say with an SD card) correlated to subsequent failure.
Last edited by ejolson on Wed Dec 18, 2019 4:52 pm, edited 4 times in total.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5501
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Raspberry Pi Benchmarks

Wed Dec 18, 2019 4:07 pm

ejolson wrote:
Wed Dec 18, 2019 3:58 pm
I recollect, possibly wrongly, that the 3B+ can throttle from 1400 to 1200 MHz without tripping the get_throttled flag. Is something similar possible with the 4B?
The throttle from 1400->1200 on 3B+ trips a different throttle flag, bit 3 (THROTTLED_SOFT_TEMP).
All of the Pi4 throttle steps trigger bit 1 (THROTTLED_HIGH_TEMP).

RoyLongbottom
Posts: 334
Joined: Fri Apr 12, 2013 9:27 am
Location: Essex, UK
Contact: Website

Re: Raspberry Pi Benchmarks

Fri Dec 20, 2019 11:14 am

jahboater wrote:
Wed Dec 18, 2019 10:42 am
Could we say that at end of each benchmark run, the command "vcgencmd get_thottled" must return zero?
Then temperature, cooling, power supply issues are moot.
It would be of more interest to identify how much throttling. This can be seen to change rapidly and might only affect a small section of a program. Knowing what temperature the different levels of throttling occur, maximum temperature could be useful.

Working on the verge of throttling, the lower MHz might only have applied to a small part of the running time, making no real difference to overall performance, and “has been throttled” indications being meaningless. Also, as I demonstrated, with I/O speed limitations, throttling might not matter at all.

RoyLongbottom
Posts: 334
Joined: Fri Apr 12, 2013 9:27 am
Location: Essex, UK
Contact: Website

Re: Raspberry Pi Benchmarks

Fri Dec 20, 2019 11:57 am

Heater wrote:
Mon Dec 16, 2019 8:12 pm
I'm not sure what I think. The statement makes me think that there are tons of benchmark results here but done in uncontrolled and/or unmonitored conditions. Thus rendering them all meaningless!
This is an open forum for anyone to contribute. So, there can be no defined structure. My benchmark results appear in the same form structured report, available at ResearchGate. These reports go back to indicating performance of the some of the first commercially available computers, providing a place in computer performance history for the Raspberry Pi. The first one for the Raspberry Pi 4 was endorsed by the CEO of Raspberry Pi Foundation, on announcement of the Pi 4:

https://twitter.com/ebenupton/status/11 ... 90?lang=en

Return to “General programming discussion”