raymate
Posts: 53
Joined: Sat Sep 28, 2013 3:56 pm
Location: Canada / UK
Contact: Website

Is it posible to use a cluster in this way....

Mon Feb 19, 2018 8:45 pm

Not even sure of this is possible, could I use a cluster of say 3-4 Pi's to do this.

One of them runs the full blown desktop OS and you use the other PI's as just an extension of CPU power, effectively could the main Pi think it has 12 or 16 cores when combined with the other Pi's in the cluster to increase the main Pis power?

Or does Pi clusters not work like that.
__________________________________
Pi model B / Pi 2 / Pi 3 / Pi Zero W

scotty101
Posts: 2856
Joined: Fri Jun 08, 2012 6:03 pm

Re: Is it posible to use a cluster in this way....

Mon Feb 19, 2018 9:14 pm

Clusters only work if the software has been developed to work across multiple devices.
Standard applications haven't been built to support this.
Electronic and Computer Engineer
Pi Interests: Home Automation, IOT, Python and Tkinter

fruitoftheloom
Posts: 16618
Joined: Tue Mar 25, 2014 12:40 pm

Re: Is it posible to use a cluster in this way....

Mon Feb 19, 2018 10:05 pm

raymate wrote:
Mon Feb 19, 2018 8:45 pm
Not even sure of this is possible, could I use a cluster of say 3-4 Pi's to do this.

One of them runs the full blown desktop OS and you use the other PI's as just an extension of CPU power, effectively could the main Pi think it has 12 or 16 cores when combined with the other Pi's in the cluster to increase the main Pis power?

Or does Pi clusters not work like that.

Asked many times and no idea where people get this idea...

The Dramble is a good read how to use a Cluster:

https://www.pidramble.com
Adieu

droleary
Posts: 141
Joined: Fri Feb 09, 2018 3:45 am
Location: Minneapolis, MN USA
Contact: Website Skype

Re: Is it posible to use a cluster in this way....

Tue Feb 20, 2018 7:44 pm

fruitoftheloom wrote:
Mon Feb 19, 2018 10:05 pm
Asked many times and no idea where people get this idea...
The get the idea because it's not an unreasonable question to ask, at least for most people's level of understanding when it comes to technology. Instead of giving a hard "no" every time it comes up, perhaps there should be some documented "best practices" when it comes to building out from the installation of a single RPi. Not just clustering, but using it as a thin client and running apps remotely for example. It all depends on what problems the OP really wants to solve.

raymate
Posts: 53
Joined: Sat Sep 28, 2013 3:56 pm
Location: Canada / UK
Contact: Website

Re: Is it posible to use a cluster in this way....

Tue Feb 20, 2018 10:59 pm

fruitoftheloom wrote:
Mon Feb 19, 2018 10:05 pm
raymate wrote:
Mon Feb 19, 2018 8:45 pm
Not even sure of this is possible, could I use a cluster of say 3-4 Pi's to do this.

One of them runs the full blown desktop OS and you use the other PI's as just an extension of CPU power, effectively could the main Pi think it has 12 or 16 cores when combined with the other Pi's in the cluster to increase the main Pis power?

Or does Pi clusters not work like that.

Asked many times and no idea where people get this idea...

The Dramble is a good read how to use a Cluster:

https://www.pidramble.com
If its been asked many times, then perhaps someone should come up with a method of maybe making it do this.

This kind of configuration to enrol a cluster would seemingly benefit a few people.

I have seen the Dramble site before and is a good source on one way a cluster could be used.

Simple saying you have no idea where people get this idea from seems to me people are thinking outside the box on what should be possible, with the right coding I dont see what it could not be actually doable. If I have the knowledge to write such code I would for sure take it on.

Using an external CPU to offload some of the main load from the master CPU should be possible.

Of course I dont fully know how clusters can work that why I'm hear to learn what is possible with the tools we currently have to hand.
__________________________________
Pi model B / Pi 2 / Pi 3 / Pi Zero W

User avatar
davidcoton
Posts: 2958
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Is it posible to use a cluster in this way....

Tue Feb 20, 2018 11:27 pm

Since it is such an obvious idea, and since sites like pidramble show that people have tried to find ways to build clusters, but have only found limited answers, doesn't that give a clue that it is not as easy as it sounds?
There is a lot of information on this site as to exactly what the difficulties are, a few searches should find the relevant threads.
It boils down to the fact that clustering in the fullest sense needs very fast interconnections, much faster than anything that current Pis provide.
"Thanks for saving my life." See https://www.raspberrypi.org/forums/viewtopic.php?p=1327656#p1327656
“Raspberry Pi is a trademark of the Raspberry Pi Foundation”

droleary
Posts: 141
Joined: Fri Feb 09, 2018 3:45 am
Location: Minneapolis, MN USA
Contact: Website Skype

Re: Is it posible to use a cluster in this way....

Wed Feb 21, 2018 8:25 pm

davidcoton wrote:
Tue Feb 20, 2018 11:27 pm
Since it is such an obvious idea, and since sites like pidramble show that people have tried to find ways to build clusters, but have only found limited answers, doesn't that give a clue that it is not as easy as it sounds?
There is a lot of information on this site as to exactly what the difficulties are, a few searches should find the relevant threads.
It boils down to the fact that clustering in the fullest sense needs very fast interconnections, much faster than anything that current Pis provide.
I think that's the mistake being made by those who think of "clustering" in the most sophisticated sense possible rather than trying to figure out what the people asking really want. It's quite possible that what they'd like to do isn't that hard, but it just isn't something most people think about doing unless they have a bunch of spare RPis sitting around. For example, just being able to run a big app (e.g., browser with multiple tabs) on another RPi and set it to -display on the local machine might be enough to make significant use of a "cluster" of otherwise idle RPi machines.

Are there any stats on how many schools have RPi labs? Or classes where students each have an RPi that could be used together for a group project? There has to be some interest in forming ad-hoc networks of machines to do something interesting. Some solution(s) that people can play with as a way to begin thinking about clustering.

User avatar
davidcoton
Posts: 2958
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Is it posible to use a cluster in this way....

Wed Feb 21, 2018 8:43 pm

So, why don't you tell us what you mean by clustering? Then we can answer the question you think you are asking, instead of having to repeat long-established truths about what is normally meant by clustering.

Networking Pis is trivial, and that allows separation of the compute and display functions. It allows co-operative data processing, as long as each processes are largely independent of others running at the same time. However, the Pi's I/O limitations severely restrict the amount of data that can be flung around, which limits what is usually considered as "clustering" -- that is, using multiple Pis as if they were a single PC with more cores and more memory. As I said before, to do that in a useful manner requires much faster comms. Pidramble and similar projects show what can be done, beyond that is a hard fundamental barrier.

If you have an application that just requires junks of the process to be run on independent processors, yes you can do that. Try plugging four Pi0 into the USB ports of a Pi3B, possibly with a networked Pi0W to control via SSH or VNC. It's been done, it works. But building a super-Pi by "clustering" multiple Pis? Currently, no.
"Thanks for saving my life." See https://www.raspberrypi.org/forums/viewtopic.php?p=1327656#p1327656
“Raspberry Pi is a trademark of the Raspberry Pi Foundation”

fruitoftheloom
Posts: 16618
Joined: Tue Mar 25, 2014 12:40 pm

Re: Is it posible to use a cluster in this way....

Wed Feb 21, 2018 10:01 pm

raymate wrote:
Mon Feb 19, 2018 8:45 pm
Not even sure of this is possible, could I use a cluster of say 3-4 Pi's to do this.

One of them runs the full blown desktop OS and you use the other PI's as just an extension of CPU power, effectively could the main Pi think it has 12 or 16 cores when combined with the other Pi's in the cluster to increase the main Pis power?

Or does Pi clusters not work like that.

What was written by the OP is above and that is asking if a cluster of RPis can be seen as a Multi-Core CPU and the answer is an emphatic NO !!!


The only way to achieve this is to use a board which uses a Multi-Core CPU, such as:

https://www.96boards.org/product/mediatek-x20/

https://www.amazon.com/dp/B073QNPH21
Adieu

User avatar
RaTTuS
Posts: 9899
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Is it posible to use a cluster in this way....

Thu Feb 22, 2018 2:54 pm

putting distcc on all raspberry's and using that to build things [i.e. kernel] is about as much clustering as your going to get
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

droleary
Posts: 141
Joined: Fri Feb 09, 2018 3:45 am
Location: Minneapolis, MN USA
Contact: Website Skype

Re: Is it posible to use a cluster in this way....

Thu Feb 22, 2018 4:21 pm

davidcoton wrote:
Wed Feb 21, 2018 8:43 pm
Networking Pis is trivial, and that allows separation of the compute and display functions. It allows co-operative data processing, as long as each processes are largely independent of others running at the same time. However, the Pi's I/O limitations severely restrict the amount of data that can be flung around, which limits what is usually considered as "clustering" -- that is, using multiple Pis as if they were a single PC with more cores and more memory.
I'm simply saying that it is a mistake to think purely in the terms of the hardware when people start asking about "clustering". I don't think it helps most people when the only solutions offered are about high-performance computing. I think the RPi community would be well served by showing specific ways they can flexibly use multiple computers, even if it ultimately isn't particularly quick or efficient. It should be about educating users about all the possibilities, not just jumping right into how someone would build a supercomputer.

fruitoftheloom
Posts: 16618
Joined: Tue Mar 25, 2014 12:40 pm

Re: Is it posible to use a cluster in this way....

Thu Feb 22, 2018 4:33 pm

droleary wrote:
Thu Feb 22, 2018 4:21 pm
davidcoton wrote:
Wed Feb 21, 2018 8:43 pm
Networking Pis is trivial, and that allows separation of the compute and display functions. It allows co-operative data processing, as long as each processes are largely independent of others running at the same time. However, the Pi's I/O limitations severely restrict the amount of data that can be flung around, which limits what is usually considered as "clustering" -- that is, using multiple Pis as if they were a single PC with more cores and more memory.
I'm simply saying that it is a mistake to think purely in the terms of the hardware when people start asking about "clustering". I don't think it helps most people when the only solutions offered are about high-performance computing. I think the RPi community would be well served by showing specific ways they can flexibly use multiple computers, even if it ultimately isn't particularly quick or efficient. It should be about educating users about all the possibilities, not just jumping right into how someone would build a supercomputer.

droleary and your contribution is apart from just hyperbole ??


The Dramble setup is a good read............... :)

https://www.jeffgeerling.com/project/ra ... pi-dramble
Adieu

BOINGBOING CEO
Posts: 2
Joined: Tue Jan 23, 2018 10:44 am

Re: Is it posible to use a cluster in this way....

Fri Feb 23, 2018 7:01 pm

I am following this Topic closely; as I clearly am looking at the Clustering possibility all wrong.

If I am required to make it a singular Pi to task; then have one Master upload each tasked single purpose outcome... (This is where I attempt to explain "The idea" - forgive me...) Two rat's... a home made cage built on the cheap... video, IR camera(s), proximity sensors, a means of weighing the Rat's as they climb; of course the assembling and tracking of such data over time... monitoring fitness over time of the Raspberry Pi Routie Rat's known as Alpha and Bravo. The Video will be displayed eventually on the BoingBoingBingBing dot COM and on the Facebook page at https://www.facebook.com/WeNeedTimsandBobsandTinas/

droleary
Posts: 141
Joined: Fri Feb 09, 2018 3:45 am
Location: Minneapolis, MN USA
Contact: Website Skype

Re: Is it posible to use a cluster in this way....

Fri Feb 23, 2018 7:27 pm

fruitoftheloom wrote:
Thu Feb 22, 2018 4:33 pm
droleary and your contribution is apart from just hyperbole ??
What hyperbole? If there are featured projects that aren't about traditional "clustering", please link to them! And please bend the ear of whoever is in charge of www.raspberrypi.org so that they can think about curating projects/documents that show how a classroom (or anyone else) can best make use of multiple RPi computers without having to dedicate them to the cluster. As it stands, they really don't seem to be encouraging people to find interesting ways of linking their machines together, so I'm not sure where you expect me to be contributing.

Return to “Advanced users”

Who is online

Users browsing this forum: No registered users and 17 guests