User avatar
Zilla707
Posts: 55
Joined: Fri Aug 23, 2019 11:04 pm

Raspberry Pi Cluster Computer?

Wed Oct 16, 2019 10:44 pm

Hello, I was wondering if its possible to build a cluster computer out of Ras Pi's. I know that its possible but the trick is you need a program that knows how to use all of the resources from all the pi's. The Pi is a bit slow, (compared to modern laptops and desktops), which is fine, and to be expected, but playing games on it, or running, like, a big code editor makes it lag a bit. So my question is; is there a way to link several pi's so that they can run 1 normal program (like minecraft pi), with combined computing power, maybe using a special flavor of linux, (idea: "Raspbian Cluster"), that instead of needing a certain program to spread the load out, it was spread out by the os? If this is just a dumb idea, or absurdly hard, please tell me so, its just an idea.
Aim for perfect and you'll hit somewhere near pretty good. (maybe...)

trejan
Posts: 758
Joined: Tue Jul 02, 2019 2:28 pm

Re: Raspberry Pi Cluster Computer?

Wed Oct 16, 2019 11:10 pm

Zilla707 wrote:
Wed Oct 16, 2019 10:44 pm
So my question is; is there a way to link several pi's so that they can run 1 normal program (like minecraft pi), with combined computing power, maybe using a special flavor of linux, (idea: "Raspbian Cluster"), that instead of needing a certain program to spread the load out,
No. It is technically possible but not feasible in practice. Communication overheads would mean performance would be awful and worse than running it on a single node. You'll be spending most of your time keeping the memory contents in sync across all the nodes as every single change would need to be immediately replicated.

W. H. Heydt
Posts: 11019
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: Raspberry Pi Cluster Computer?

Wed Oct 16, 2019 11:28 pm

You need to pick things to run on a cluster that can be partitioned into separate tasks doing a part of of the overall process. Clusters--when they scale to modern supercomputers--are specifically for doing those sorts of things. A cluster of Pis is a perfectly good way to learn how to program for such problems. However, it would be very poor at any program that usually runs as a single thread.

Heater
Posts: 13701
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 12:09 am

You could build a fault tolerant messaging system on a Pi cluster with NATS: https://nats.io/

You could build a fault tolerant SQL database on a Pi cluster with CocroacjDB: https://www.cockroachlabs.com/

Perhaps others could suggest ready made applications for cluster of processors.

Things like the above would not get a performance boost from being a cluster but they do get the reliability of multiple redundant computers.

You could experiment with the techniques used in creating software for super computers with MPI: https://www.codingame.com/playgrounds/3 ... -computing

Again, not worth it for the performance but a cheap, compact way to learn about techniques.
Memory in C++ is a leaky abstraction .

User avatar
HawaiianPi
Posts: 4793
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 1:17 am

Zilla707 wrote:
Wed Oct 16, 2019 10:44 pm
So my question is; is there a way to link several pi's so that they can run 1 normal program (like minecraft pi)...
Simple answer:
In any practical sense, no.


Longer answer:
Machine clusters run specialised operating systems and software for very specific computational tasks. You don't run general purpose software like web browsers, office applications or games on clusters. In addition, the number of Pi computers it would require to significantly beat the performance of a modern multi-core Intel or AMD based computer would probably cost as much as the Intel/AMD computer (if not more).


Raspberry Pi supercomputer?
There are a lot of websites and youtube videos that talk about Raspberry Pi Supercomputers, and that is a misnomer. A cluster of Raspberry Pi computers will in no way be "super" in performance. So why do people build them? I'd say mainly for education (learning how to program distributed task software for clusters). The other reason would be economics. Genuine supercomputers not only cost buckets of money to build or buy, they also cost a lot to run (they can cost millions of dollars a year for the electricity to power them). So it makes sense to have a lower cost, less power hungry system for software development, which also allows for less downtime on the real supercomputers.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

fruitoftheloom
Posts: 20943
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 6:18 am

Zilla707 wrote:
Wed Oct 16, 2019 10:44 pm
Hello, I was wondering if its possible to build a cluster computer out of Ras Pi's. I know that its possible but the trick is you need a program that knows how to use all of the resources from all the pi's. The Pi is a bit slow, (compared to modern laptops and desktops), which is fine, and to be expected, but playing games on it, or running, like, a big code editor makes it lag a bit. So my question is; is there a way to link several pi's so that they can run 1 normal program (like minecraft pi), with combined computing power, maybe using a special flavor of linux, (idea: "Raspbian Cluster"), that instead of needing a certain program to spread the load out, it was spread out by the os? If this is just a dumb idea, or absurdly hard, please tell me so, its just an idea.

Yes this myth that a Raspberry Pi Single Board Computer can be joined together to make a super duper multi-core i7 killer is just that a myth.

You would not for one instant consider joining 2 PCs or Laptops together.

The Raspberry Pi SBC is a computer in its own right just as s PC or Laptop is, and yes we have this idea regularly mooted.

https://www.raspberrypi.org/forums/view ... 3&t=254400
Retired disgracefully.....

User avatar
B.Goode
Posts: 8917
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 7:35 am

[Irony... ] Almost identical question and consequent replies here, as long ago as Tuesday 15 October 2019, so probably no longer relevant? /[Irony... ]

https://www.raspberrypi.org/forums/view ... 0#p1552092

fruitoftheloom
Posts: 20943
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 7:42 am

B.Goode wrote:
Thu Oct 17, 2019 7:35 am
[Irony... ] Almost identical question and consequent replies here, as long ago as Tuesday 15 October 2019, so probably no longer relevant? /[Irony... ]

https://www.raspberrypi.org/forums/view ... 0#p1552092

https://www.raspberrypi.org/forums/viewtopic.php?t=1849

Seems not much has changed in 8 years :lol:
Retired disgracefully.....

User avatar
Zilla707
Posts: 55
Joined: Fri Aug 23, 2019 11:04 pm

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 1:03 pm

Yeah, i looked at some of the other threads, but it is kinda hard to find an answer sometimes. Well, that’s too bad, and thanks for the help.
Aim for perfect and you'll hit somewhere near pretty good. (maybe...)

fruitoftheloom
Posts: 20943
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 2:17 pm

Zilla707 wrote:
Thu Oct 17, 2019 1:03 pm
Yeah, i looked at some of the other threads, but it is kinda hard to find an answer sometimes. Well, that’s too bad, and thanks for the help.

If you had a 10 year old Core2Duo Laptop & PC would you join them together to make a Quad-Core i3 ?

Why is this impossibility "bad" ? it is how a computer works !!
Retired disgracefully.....

User avatar
DougieLawson
Posts: 36331
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 4:01 pm

fruitoftheloom wrote:
Thu Oct 17, 2019 2:17 pm
Zilla707 wrote:
Thu Oct 17, 2019 1:03 pm
Yeah, i looked at some of the other threads, but it is kinda hard to find an answer sometimes. Well, that’s too bad, and thanks for the help.

If you had a 10 year old Core2Duo Laptop & PC would you join them together to make a Quad-Core i3 ?

Why is this impossibility "bad" ? it is how a computer works !!
There's clearly something out there that causes folks to completely fail to grasp how a loosely or tightly coupled clustering computer works, how you have to create a specific workload (small packets of data that are spread around the computing elements). We get too many "Can I join two Raspberries to play Minecraft at twice the framerate?" posts on here, the folks that post this stuff must be getting that hair-brained idea from somewhere (or they've failed to do any research before posting). Or they're failing to pay any attention to their computer science lecturers.

Even with early 1980's mainframes with shared memory, a shared disk I/O subsystem and a move from one processor (IBM D/T3083) to two (IBM D/T3081) then to four (IBM D/T3048 (which was two back-to-back 3081s)) we didn't get four times the computing power on that 3084. The signalling between processors (to protect one engine from overwriting another engines's shared memory) cost a significant percentage of the engine. And like a four core Raspberry that did share all the resources between all of the engines.

The Raspberry Pi cluster isn't tightly coupled, all data transfer between cluster members is going via USB or ethernet, it's very much loosely coupled (it's only tightly coupled on the four cores on each Raspberry). As has been stated it's a good tool to learn how to break a workload into chunks, how to write the code that the cluster uses to schedule things running each chunk on an engine (and watching for an engine going idle) and how to glue the result from each engine back into a coherent results set. It's ideal for functionality but it won't be good for speed.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
clicky
Posts: 413
Joined: Thu Oct 25, 2012 7:34 am

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 4:11 pm

In theory there are plenty of possibilities but in practical terms nobody has implemented it, yet.

There's your chance! :D

fruitoftheloom
Posts: 20943
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 4:17 pm

DougieLawson wrote:
Thu Oct 17, 2019 4:01 pm
fruitoftheloom wrote:
Thu Oct 17, 2019 2:17 pm
Zilla707 wrote:
Thu Oct 17, 2019 1:03 pm
Yeah, i looked at some of the other threads, but it is kinda hard to find an answer sometimes. Well, that’s too bad, and thanks for the help.

If you had a 10 year old Core2Duo Laptop & PC would you join them together to make a Quad-Core i3 ?

Why is this impossibility "bad" ? it is how a computer works !!
There's clearly something out there that causes folks to completely fail to grasp how a loosely or tightly coupled clustering computer works, how you have to create a specific workload (small packets of data that are spread around the computing elements). We get too many "Can I join two Raspberries to play Minecraft at twice the framerate?" posts on here, the folks that post this stuff must be getting that hair-brained idea from somewhere (or they've failed to do any research before posting). Or they're failing to pay any attention to their computer science lecturers.

Even with early 1980's mainframes with shared memory, a shared disk I/O subsystem and a move from one processor (IBM D/T3083) to two (IBM D/T3081) then to four (IBM D/T3048 (which was two back-to-back 3081s)) we didn't get four times the computing power on that 3084. The signalling between processors (to protect one engine from overwriting another engines's shared memory) cost a significant percentage of the engine. And like a four core Raspberry that did share all the resources between all of the engines.

The Raspberry Pi cluster isn't tightly coupled, all data transfer between cluster members is going via USB or ethernet, it's very much loosely coupled (it's only tightly coupled on the four cores on each Raspberry). As has been stated it's a good tool to learn how to break a workload into chunks, how to write the code that the cluster uses to schedule things running each chunk on an engine (and watching for an engine going idle) and how to glue the result from each engine back into a coherent results set. It's ideal for functionality but it won't be good for speed.

I liked the Dual CPU 6 core Opteron Motherboards of circa 2009......
Retired disgracefully.....

Heater
Posts: 13701
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 5:24 pm

If you want to build a ARM powered cluster super computer this is how you do it:

Image

The SpiNNaker, one million ARM 9 cores and a network fabric at Manchester:
https://www.manchester.ac.uk/discover/n ... irst-time/

Build instructions here:
http://apt.cs.manchester.ac.uk/projects ... hitecture/

Brought to you by Steve Furber, original designer of the ARM we know and love.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3728
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 6:53 pm

Heater wrote:
Thu Oct 17, 2019 5:24 pm
If you want to build a ARM powered cluster super computer this is how you do it:

Image
I like how the air-conditioning ductwork is positioned at slightly different angles at the top of each cabinet and looks like giant springs. I wonder why they didn't make the SpiNNaker out of Raspberry Pi computers. Could that be because of some hidden rivalry?

On the liberal frontier we used the spare time between gunfights and horse rustling to interconnect 96 dual-socket quad-GPU nodes with a 100Gb/s non-blocking Intel Omni-Path fabric. This configuration was designed to support remote direct memory access, which in theory makes it easier to write parallel code. While the researchers here are generally used to dodging bullets, such nonuniform memory architectures still require that one be very careful to obtain anything resembling high performance.

Even making efficient use of the cache on a Raspberry Pi is surprisingly difficult. Moreover, with a cluster of Pi computers, the differences between local memory compared to network bandwidth and latency are much greater.

Moonmarch
Posts: 155
Joined: Thu Mar 07, 2019 1:34 am

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 10:34 pm

The ARM CPU is near identical in function in comparison to the Intel/AMD computers the software available on the Intel/AMD computers can be ported to ARM computers the ARM computers will have less software overall, Intel/AMD computers are much faster, the size of the computers will be larger, the power requirements will be higher, if the computer form factor or power requirements are not important then use a powerful single Intel/AMD computer.

The same can be said about multiple graphic cards on the same motherboard if the program does not use or is not compatible with multi GPU configurations then you will not see any benefits of using a multiple GPU computer, there used to be graphic cards with 2 graphic processors built into the same graphic card the power requirements needed to use the hardware was through the roof.

Here is a picture of a computer using several video cards:
https://www.rockpapershotgun.com/2015/0 ... od-or-bad/
Image

User avatar
HawaiianPi
Posts: 4793
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: Raspberry Pi Cluster Computer?

Thu Oct 17, 2019 11:44 pm

clicky wrote:
Thu Oct 17, 2019 4:11 pm
In theory there are plenty of possibilities but in practical terms nobody has implemented it, yet.

Los Alamos National Lab has a large 750 Pi cluster they use for software development and testing. They plan on extending it up to as many as 10,000 Pi computers (40,000 cores).
  • "According to Gary Grider, head of its LANL's HPC division, the new Raspberry Pi cluster can offer the same testing capabilities as a dedicated testbed, which could cost $250m and use 25MW of energy."

Oracle has a large 1,060 Pi cluster they use to promote their software (and for PR, because it's cool). The Raspberry Pi supercomputer runs Oracle Autonomous Linux as well as Java and is equipped with a nine display visualisation wall.
  • "There was very little point. Raspberry Pi clusters don't pull out much in the way of performance, but Oracle was stressing the coolness of what you can do with a $35 computer. Oracle said that it could have used a virtualised Arm server rather than creating a cluster of Raspberry Pis, but big clusters are cool."
Oracle-Supercomputer.jpg
Oracle-Supercomputer.jpg (74.49 KiB) Viewed 1011 times
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

Heater
Posts: 13701
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 12:55 am

ejolson,
I wonder why they didn't make the SpiNNaker out of Raspberry Pi computers. Could that be because of some hidden rivalry?
Could it be for performance reasons?

The SoC used on SpiNNaker boards have 18 ARM cores with tightly coupled RAM on chip and a custom mesh router circuits. There is 40 or so of those cores on each board in SpiNNaker.

http://apt.cs.manchester.ac.uk/projects ... SpiNNchip/

Besides I don't think there was a Pi when SpiNNaker was started.

As for "rivalry" Steve Furber is a Cambridge graduate and was of course in Cambridge with Acorn when they invented the ARM.

I'm sure they would let you have some time on SpiNNaker if you asked:
http://apt.cs.manchester.ac.uk/projects ... ct/Access/

Let's see how fast that million digit fibo can really go...
Memory in C++ is a leaky abstraction .

User avatar
Gavinmc42
Posts: 3935
Joined: Wed Aug 28, 2013 3:31 am

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 1:23 am

The SoC used on SpiNNaker boards have 18 ARM cores with tightly coupled RAM on chip and a custom mesh router circuits
Sounds a bit like those Transputer CPUs. Many core with routing networks.
Is there an Occam port for Pi's?
Would that be of any use on 4 core Pi's?
With the parallelism and communications built into the chip and the language interacting with it directly, writing code for things like device controllers became a triviality; even the most basic code could watch the serial ports for I/O, and would automatically sleep when there was no data.
Sounds like Parallax Prop chips?
Speaking of which, the Prop2 eval board might be here before Xmas?
Cluster of Prop2s?
Probably one per customer for 2019?
Anyway time to see if the software will run on Pi4.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

ejolson
Posts: 3728
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 2:57 am

Moonmarch wrote:
Thu Oct 17, 2019 10:34 pm
Here is a picture of a computer using several video cards:
https://www.rockpapershotgun.com/2015/0 ... od-or-bad/
Image
It's odd that the picture depicts a GPU cryptocurrency mining rig and then the article is about video games.

As you have indicated, how much GPU accelerators help with general-purpose computing tasks definitely depends on the task. While the types of algorithms that can be parallelized has increased dramatically with support for dynamic parallelism in CUDA, in general GPUs seem most useful as vector processors for linear algebra. I guess that is why the new supercomputers Summit and Sierra built by IBM have, respectively, six and four NVIDIA GPUs per node.

Although the Pi is a fantastic device for learning about programming, clusters and parallel symmetric multiprocessing, it is not so useful as a heterogeneous compute architecture for GPUs, FPGAs, vector processors and AI accelerators. This is not surprising considering the price ratio between a single IBM AC922 and the 4B is more than 2000 fold.

At the same time, I find it amusing that a single Raspberry Pi 4B is more than 12 times faster than the first Linux cluster--made out of sixteen 486DX4 computers--assembled by Donald Becker in 1994. Now the two fastest supercomputers in the world run Red Hat. When will there be a supercomputer that runs Raspbian?

Heater
Posts: 13701
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 6:24 am

Gavinmc42,
Sounds a bit like those Transputer CPUs. Many core with routing networks.
Is there an Occam port for Pi's?
Would that be of any use on 4 core Pi's?
Of course, my old university CS department has an open source development of an OCCAM compiler:
https://www.cs.kent.ac.uk/projects/ofa/kroc/

Looks like it should be buildable on Raspbian:
http://projects.cs.kent.ac.uk/projects/ ... stallation

Source code on github here:
https://github.com/concurrency/kroc

I thought this project was dead a decade ago but looks like the last commit was only 5 years ago. Wonder if they are still workig with it back in uni.
Memory in C++ is a leaky abstraction .

fanoush
Posts: 485
Joined: Mon Feb 27, 2012 2:37 pm

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 6:42 am

fruitoftheloom wrote:
Thu Oct 17, 2019 6:18 am
You would not for one instant consider joining 2 PCs or Laptops together.
I guess that's true but looks like it would be for wrong reason as we get this question again and again. Maybe two laptops are too big or too expensive or it would be confusing to have two screens or two keyboards. Pi is small and cheap so the idea comes naturally ;-)

Heater
Posts: 13701
Joined: Tue Jul 17, 2012 3:02 pm

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 6:58 am

The idea comes as naturally as the thought that welding two Mini vans together will make them faster. Which is equally naturally dismissed after one second. It's not clear to me how the idea ever makes it as far as this forum.
Memory in C++ is a leaky abstraction .

ejolson
Posts: 3728
Joined: Tue Mar 18, 2014 11:47 am

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 9:26 am

Heater wrote:
Fri Oct 18, 2019 6:58 am
The idea comes as naturally as the thought that welding two Mini vans together will make them faster. Which is equally naturally dismissed after one second. It's not clear to me how the idea ever makes it as far as this forum.
It didn't take long to find

Image

In my opinion, the reason people keep asking about Pi clusters is
  • The internet is full of pictures.
  • They are cheap.
Generally speaking clusters also work.
Otherwise governments, research universities and other organizations wouldn't spend millions of dollars building them. However, there does appear to be a misunderstanding about what sort of work a cluster might be used for.

Surprisingly, there also appears to be a similar misunderstanding about the advantages of welding two minivans together. As demonstrated by the similarly constructed jeepney

Image

the welded-together vehicle does not go faster, but instead carries more people.
Last edited by ejolson on Fri Oct 18, 2019 2:50 pm, edited 1 time in total.

User avatar
clicky
Posts: 413
Joined: Thu Oct 25, 2012 7:34 am

Re: Raspberry Pi Cluster Computer?

Fri Oct 18, 2019 9:48 am

Actually - I was considering having two Pis (due to memory constraints of one): one connected to monitor and keyboard running X windows and another on network using first as X terminal. Also, first (with SSD) would share disk over nfs (at least /home if not more!).

That way I would be able to decide which app to start where and have ability to run two memory hungry apps at the same time. That can be scaled slightly further as well.

Downside (think I haven't sorted out, logically speaking) is how you would log in both at the same time, how to use first Pi's windows manager for both and such... That would be cool to have windows (and session) manager that can manage windows from several X servers on one X terminal.

Return to “General discussion”