bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 9:57 am

I did some performance testing on CPU + Eth only.

https://github.com/tinspin/rupy/wiki/Comet-Stream

I hope the admins won't delete this post as it's only discussing how you can learn and experiment.

Stuff that kids with computers should probably do anyways!

From my point of view (a server cluster builder) the Raspberry Pi 3 is falling behind in perf./price.

But it's still a great achievement, don't get me wrong; I have 10 RPi's.

I'll update this chart with the XU-4 when I get it.
Last edited by bullen on Thu Mar 10, 2016 4:55 pm, edited 3 times in total.
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26094
Joined: Sat Jul 30, 2011 7:41 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 10:05 am

Why are you comparing with Pi1? Those are four years old....
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
jojopi
Posts: 3194
Joined: Tue Oct 11, 2011 8:38 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 10:19 am

Are you the author of this rupy / comet stream thing? The benchmark looks biased towards sending very short messages, which might indeed tend to penalise systems where Ethernet is connected via USB.

It would probably be more useful to test CPU and Ethernet individually. I like "openssl speed" as a single-threaded pure CPU test.

Alternatively, a benchmark that replicates real workloads could be interesting. Almost nothing that anyone runs on Linux is actually written in C#.

fruit-uk
Posts: 609
Joined: Wed Aug 06, 2014 4:19 pm
Location: Suffolk, UK

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 10:53 am

Why do these silly threads always appear after a new release?

I am very happy with the performance of all my Pis in their individual roles - and the prices too.

I also have a cubietruck, it has performed well but I suspect it will be replaced soon with a PI as it has neither the support nor userbase that Pis have built up over the last few years.

OP perhaps you can include a section for support in your next set of benchmarks.

User avatar
GTR2Fan
Posts: 1601
Joined: Sun Feb 23, 2014 9:20 pm
Location: South East UK

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 11:02 am

Basing a benchmark comparison between any two or more similar things based on the result of one test is totally nuts. A Pi is infinitely faster at running native ARM code than a top of the range Intel i7, therefore, the Pi is infinitely superior to any i7. Hoorah! :D
Pi2B Mini-PC/Media Centre: ARM=1GHz (+3), Core=500MHz, v3d=500MHz, h264=333MHz, RAM=DDR2-1200 (+6/+4/+4+schmoo). Sandisk Ultra HC-I 32GB microSD card on '50=100' OCed slot (42MB/s read) running Raspbian/KODI16, Seagate 3.5" 1.5TB HDD mass storage.

Heater
Posts: 15508
Joined: Tue Jul 17, 2012 3:02 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 4:22 pm

bullen
I did some performance testing on CPU + Eth only.
https://github.com/tinspin/rupy/wiki/Comet-Stream
...
From my point of view ... the Raspberry Pi 3 is falling behind in perf./price.
I don't see think your test results support your point of view. Let's see:

Code: Select all

        CPU usage in    out   total    watt   mess./joule  temp.*
OPi 1	200%	  4.000	9.000 13.000 / 2.5  = 5.200	       70C
RPi 3	130%	  4.000	9.000 13.000 / 2    = 6.500	       60C
1) Both Pi3 and OPi are handling the same volume of messages in and out - It's a tie.

2) The Pi3 is using a lot less CPU time to do this. Presumably leaving time
for application code to actually do something. - A win for the Pi.

3) The Pi3 is using 20% less power to do this exact same job at the same speed - A win for the Pi.

4) The Pi is running 10 degrees C cooler to do exactly the same job at the same speed - A win for the Pi.

5) The Pi is delivering more sages per joule - Well, it would be given the above.

Just looking at the CPU usage vs message rate here is enough to tell us the Pi is way ahead on performance.

Now. We could argue price. That is price/performance ratio. Well, The OPi is certainly cheaper.

All benchmarks should be taken with a jar salt of course so I'm not convinced any of this is meaningful on it's own,
perhaps as another data point.
Memory in C++ is a leaky abstraction .

bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 4:52 pm

Yep, the big win is the 10$ price.

I ordered 2x yesterday and together including shipping they cost less than the shipping only of two RPi 3 from the UK.

They actually cost less each than the 32GB SD cards I'm going to put in them!?

If you are building a cluster of hundreds of machines that adds up.

Another upside is the 3.0mm holes for distance stacking, the RPi have 2.9mm holes that I have to drill to 3.0mm.

Only two minor downsides: 512MB RAM and you need to buy 5.5mm to 4.0mm adaptors for the power supply!?

Edit: bonus picture for a somewhat constructive thread, the RPi in the picture is actually an RPi 2 and not the RPi3 used to test.

Image

Can't increase image size?! https://dl.dropboxusercontent.com/u/135 ... angepi.png
Last edited by bullen on Thu Mar 10, 2016 7:00 pm, edited 3 times in total.
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26094
Joined: Sat Jul 30, 2011 7:41 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 4:58 pm

bullen wrote:Yep, the big win is the 10$ price.

I ordered 2x yesterday and together including shipping they cost less than the shipping only of two RPi 3 from the UK.

They actually cost less each than the 32GB SD cards I'm going to put in them!?

If you are building a cluster of hundreds of machines that adds up.

Another upside is the 3.0mm holes for distance stacking, the RPi have 2.9mm holes that I have to drill to 3.0mm.

Only one minor downside: You need to buy 5.5mm to 4.0mm adaptors for the power supply!

Edit: bonus picture for a mildly constructive thread, the RPi in the picture is actually an RPi 2 and not the RPi3 used to test.

http://rupy.se/IMG_3089.JPG
Well, for $10, you get a board that has three less USB ports, half the memory, A7 rather than A8 cores, no Wifi or Bluetooth. You are pretty much getting what you pay for.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

mikerr
Posts: 2825
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 5:34 pm

jamesh wrote:Why are you comparing with Pi1? Those are four years old....
Pi1 = same speed as the B+ and Pi Zero in the current pi lineup.
Android app - Raspi Card Imager - download and image SD cards - No PC required !

bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 5:36 pm

@jamesh Absolutely, I was just so happy I got anything to work on the board at all.

Armbian has come a long way!

Also remains to be seen the stability long term both on the hardware side (physical failures) and software side (kernel panic, memory leaks etc).

But so far it seems to work fine.
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

User avatar
joan
Posts: 14849
Joined: Thu Jul 05, 2012 5:09 pm
Location: UK

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 5:43 pm

mikerr wrote:
jamesh wrote:Why are you comparing with Pi1? Those are four years old....
Pi1 = same speed as the B+ and Pi Zero in the current pi lineup.
It will depend on your tests.

My original Pi B using soft float outperforms my other Pi B rev2 and Pi B+. My Pi Zero is about twice as fast as the PiBs and B+..

User avatar
kusti8
Posts: 3439
Joined: Sat Dec 21, 2013 5:29 pm
Location: USA

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 5:59 pm

The Pi Zero although having the same original chip is clocked faster, meaning it should be a bit faster than the original. It definately feels faster.
There are 10 types of people: those who understand binary and those who don't.

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 6:13 pm

bullen wrote:I did some performance testing on CPU + Eth only.
I hope the admins won't delete this post as it's only discussing how you can learn and experiment.
While I would oppose deleting any posts barring something extreme, you should be chastised for not being even slightly clear what you are talking about. I don't think calling something an OPi is enough. I googled that and apparently it's a lacquer for fingernails or something like that.

After jumping around I see that you are talking about the "Orange Pi". Yet Another Dumb Name winner to go with Banana Pi and the all time winner, Lemon Pi. But no one makes pies out of oranges, so I'm not sure whether or not in some sense we don't have a new winner.

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 6:21 pm

bullen wrote:Yep, the big win is the 10$ price.
They actually cost less each than the 32GB SD cards I'm going to put in them!?
The real big point here isn't the Orange Pi boards being cheap but that they cost less than the sdcards you were going to put in them. Net boot is huge for anyone looking to build with these things in a low cost manner and as far as I know only the pi people are paying attention so far to this point.
If you are building a cluster of hundreds of machines that adds up.
The problem here is that it is senseless to build a cluster of hundreds of machines and not pick a decently done up board. Now I'd say that the pi3 isn't the choice for such a thing either but given that the Orange Pi has only a half a gig of RAM, it *really* isn't the right choice.
Only two minor downsides: 512MB RAM and you need to buy 5.5mm to 4.0mm adaptors for the power supply!?
Any buying of adapters is going to once again eat into your budget disproportionally. I was looking at using a normal computer case and power supply and getting molex splitters that fit to the end of the power leads. The ends could be cut off and the correct power adapters cheaply crimped on. Obviously one could simply clip the power supply ends but that means less flexibility if changes need to be made.

bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 6:35 pm

stderr wrote:
bullen wrote:Yep, the big win is the 10$ price.
They actually cost less each than the 32GB SD cards I'm going to put in them!?
The real big point here isn't the Orange Pi boards being cheap but that they cost less than the sdcards you were going to put in them. Net boot is huge for anyone looking to build with these things in a low cost manner and as far as I know only the pi people are paying attention so far to this point.
If you are building a cluster of hundreds of machines that adds up.
The problem here is that it is senseless to build a cluster of hundreds of machines and not pick a decently done up board. Now I'd say that the pi3 isn't the choice for such a thing either but given that the Orange Pi has only a half a gig of RAM, it *really* isn't the right choice.
Only two minor downsides: 512MB RAM and you need to buy 5.5mm to 4.0mm adaptors for the power supply!?
Any buying of adapters is going to once again eat into your budget disproportionally. I was looking at using a normal computer case and power supply and getting molex splitters that fit to the end of the power leads. The ends could be cut off and the correct power adapters cheaply crimped on. Obviously one could simply clip the power supply ends but that means less flexibility if changes need to be made.
Net boot sounds awesome, but I'm actually building a "truly" distributed system (almost decentralized, machines across different countries behaving like one cluster), and it's better that they are standalone in that scenario.

Memory is only a problem if you have bloated software, I've built this appserver/database from the ground up to be super optimized for memory, performance and developer ease of use.

The adapters are like 0.5GBP each and yes I use a camera 5.5mm 1-to-many splitter with metal powersupply, so you need adapters for RPi too (the 4.0mm choice is what puzzles me):

Image

Can't increase image size?! http://host.rupy.se/cluster.jpg
Last edited by bullen on Thu Mar 10, 2016 6:59 pm, edited 4 times in total.
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

W. H. Heydt
Posts: 12123
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 6:48 pm

bullen wrote: Memory is only a problem if you have bloated software, I've built this appserver/database from the ground up to be super optimized for memory, performance and developer ease of use.
Speaking as someone who has been programing for a bit over 50 years, I can tell you that the better your optimization, the harder it will be to maintain and add features. And if you are concerned about the cost of hardware, just wait until you find out the cost of people.

bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 6:52 pm

W. H. Heydt wrote:
bullen wrote: Memory is only a problem if you have bloated software, I've built this appserver/database from the ground up to be super optimized for memory, performance and developer ease of use.
Speaking as someone who has been programing for a bit over 50 years, I can tell you that the better your optimization, the harder it will be to maintain and add features. And if you are concerned about the cost of hardware, just wait until you find out the cost of people.
Have you tried µSOA?

https://github.com/tinspin/rupy/wiki/Process
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

LucasDany
Posts: 1
Joined: Thu Mar 10, 2016 6:54 pm
Contact: Website

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 6:59 pm

It's hard to compare with Pi1 ! I don't know OPi 1 enought to answer correctly but I prefer RPi 3

W. H. Heydt
Posts: 12123
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 9:06 pm

bullen wrote:
W. H. Heydt wrote:
bullen wrote: Memory is only a problem if you have bloated software, I've built this appserver/database from the ground up to be super optimized for memory, performance and developer ease of use.
Speaking as someone who has been programing for a bit over 50 years, I can tell you that the better your optimization, the harder it will be to maintain and add features. And if you are concerned about the cost of hardware, just wait until you find out the cost of people.
Have you tried µSOA?

https://github.com/tinspin/rupy/wiki/Process
Many scheme have come and gone for increasing programmer productivity. Some of them even work at some companies.

Heater
Posts: 15508
Joined: Tue Jul 17, 2012 3:02 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 9:24 pm

bullen,
If you are building a cluster of hundreds of machines that adds up.
Not only is that a really weird use case I'm wondering if it makes any sense at all.

If you want serious performance for a good price you will be using an Intel mega beast. Or many other solutions.
I'm actually building a "truly" distributed system (almost decentralized, machines across different countries behaving like one cluster), and it's better that they are standalone in that scenario.
Ah, so performance of a node is not actually a problem for you. All that cooperation over then net over huge distances is.
Yawn...
From the link:

Errors: Should be pushed from each live client to the service developers smart watch in realtime.

OK, not only "yawn" but, are you insane?
Memory in C++ is a leaky abstraction .

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 10:03 pm

Heater wrote: > Errors: Should be pushed from each live client to
> the service developers smart watch in realtime.
>
OK, not only "yawn" but, are you insane?
Not smart watch, William Gibson Neuromancer port on back of skull.

bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Thu Mar 10, 2016 11:17 pm

Smart watch irony, with a tiny logic of responsibility. I actually sleep better if I get punished by my mistakes, a bit insane, but still true.

My system uses async-to-async headerless HTTP with "less headers" to synchronize all the nodes concurrently.

This means zero IO-wait and scales great on small data and read intensive solutions.

About Intel: Most of the energy used by computers is used by idle CPU, Intel consumes at least 10x the energy while idle compared to ARM. While Intel still has a peak performance about twice any ARM, that race is going to become narrower when ARM reaches peak computing litho. at around 10nm; because Intel is already there and my gut feeling is RISC will also be able to "turbo".

At the end of our 200 year infinite growth paradigm (coal, oil and gas are just finite, very old, sunrays stored by trees and plants), it's going to be interesting to see how data centers supply their electricity. (fission, solar, wind and even hydro are all net loosing proposals, only photosynthesis is net positive on this planet as far as we know)

As for VR and Bitcoin those are together with ARM computers probably the most interesting things one can work on today, and electric bikes!
Last edited by bullen on Fri Mar 11, 2016 10:37 am, edited 1 time in total.
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

Heater
Posts: 15508
Joined: Tue Jul 17, 2012 3:02 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Fri Mar 11, 2016 4:13 am

Bullen,

I'd appreciate it if you could explain what you mean by "async-to-async headerless HTTP"

I think I get the "async" part but HTTP is pretty much defined by it's headers.
Memory in C++ is a leaky abstraction .

bullen
Posts: 316
Joined: Sun Apr 28, 2013 2:52 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Fri Mar 11, 2016 7:36 am

Heater wrote:Bullen,

I'd appreciate it if you could explain what you mean by "async-to-async headerless HTTP"

I think I get the "async" part but HTTP is pretty much defined by it's headers.
You can read about it here:

https://github.com/tinspin/rupy/wiki/Fuse

It's a way to hand over work between threads so they never linger on sockets. My system only waits it the network not in the CPU. Say you have a database (Oracle, Mysql or any other; elastic search just went back to sync because customers where having problems with async = it's kinda complicated, which is what I'm trying to solve) In all systems there is always a thread on the DB client waiting for the database to reply. In my system not only is the client async, so threads send data on the socket and go on other business until the socket tells them "DB request was done, give me a thread to do the work", but also the incoming request is async, the there is no IO-wait anywhere in the whole platform. This allows it to scale much better and be very resilient to overload = it just keep working no matter the load = no freezing or crashes (everyone that has built any highly used system knows that the bottleneck is always IO and not CPU, until peak computing at least). Rupy has a built in queue!

It's tricky to wrap your head around because all systems are sync on the network today because making the systems work async is nerve wrecking; when you add the observer (log, debugger etc.) to see what is wrong the problem disappears, just like in quantum physics! So you have to "guess" ant try... over and over again.

Hence elastic search giving up...

The headers are just removed to save bandwidth and CPU parsing time. That's nothing strange:

I only send:

GET /hello HTTP/1.1\r\n
Host: some.host.com\r\n
Head: less\r\n
\r\n

Instead of regular request which looks something like:

GET /hello HTTP/1.1\r\n
Host: some.host.com\r\n
Connection: keep-alive\r\n
Accept: */*\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36\r\n
Referer: some.referer.com\r\n
Cookie: key=iHXR\r\n
\r\n

But you are right it's not headerless it's "less headers"!

And I load balance with sticky sockets instead of cookies.
https://github.com/tinspin/rupy - A tiny Java async HTTP application server.

Heater
Posts: 15508
Joined: Tue Jul 17, 2012 3:02 pm

Re: RPi 1, RPi 2, RPi 3 vs. OPi 1

Fri Mar 11, 2016 11:08 am

Ah, so you are reinventing the async wheel for Java.

Async programming like that is what we have all been doing with node.js and Javascript for years now.

Of course async programming in JS is much easier as JS was designed to be async from the very beginning. Node.js is 100% async unless you go out of your way.

Performance might be an issue but I found that similar services using node and JSON were as performant as our old C++ and XML versions!

What do you mean by "sticky sockets"? Is that to do with the "keep-alive" header?
Memory in C++ is a leaky abstraction .

Return to “Off topic discussion”