Bramble RPi Cluster Up And Running With MPICH2!


9 posts
by philleonard » Tue Jun 19, 2012 1:39 am
Hey guys!

I'm the proud owner of two Pis (please don't hate me he says!), and I have spent the last day getting MPICH2 (Message Passing Interface), compiled and up and running on them. Things have gone well and the results are pretty good it seems so far. I have spent the last couple of hours writing an extensive tutorial on how to get setup :)

Here is the link: http://westcoastlabs.blogspot.co.uk/2012/06/parallel-processing-on-pi-bramble.html (It's a bit long to post here).
It also includes a .deb file with the arm compiled program, which makes things a lot quicker! :D

The whole thing seems to work well on old and new firmware :)
Credit goes to the team who built the super awesome & portable MPICH2 :)

Cheers!

Phil

P.S. I didn't come up with the ingenious name Bramble, another user did :)
Posts: 5
Joined: Sun Jun 17, 2012 7:59 pm
Location: Liverpool
by zardoz99 » Tue Jun 19, 2012 10:19 am
Nice!! Add that to my work on getting Raspis into an HA configuration for management nodes and you are starting to create a real LPC cluster. Well, you REALLY can't call it an HPC cluster, can you? :lol: :lol: :lol:

Z.
User avatar
Posts: 137
Joined: Fri Jan 13, 2012 2:25 pm
by philleonard » Tue Jun 19, 2012 5:19 pm
Hahaha :P This is true!
Posts: 5
Joined: Sun Jun 17, 2012 7:59 pm
Location: Liverpool
by dcuk » Tue Jun 26, 2012 2:51 pm
Niice! Just got to wait for my 2nd Pi to arrive now...
Posts: 1
Joined: Tue Jun 26, 2012 2:50 pm
Location: UK
by SN » Tue Jun 26, 2012 3:30 pm
zardoz99 wrote:Nice!! Add that to my work on getting Raspis into an HA configuration for management nodes and you are starting to create a real LPC cluster. Well, you REALLY can't call it an HPC cluster, can you? :lol: :lol: :lol:

Z.

This was my observation too.... I see a quite complicated setup just to allow the nodes to talk to each other :?

Now may be I'm missing something here but if you were using this method to spread computing load around, would a 'farm' of devices all chatting via http (in SOAP fashion) not achieve the same thing, perhaps with a shared NFS store picking up any very large input data (such as program executables) and maybe an additional shared MySQL DB for coordinating output
Steve N – binatone mk4->intellivision->zx81->spectrum->cbm64->cpc6128->520stfm->pc->raspi ?
User avatar
Posts: 1008
Joined: Mon Feb 13, 2012 8:06 pm
Location: Romiley, UK
by philleonard » Wed Jun 27, 2012 2:51 pm
SN wrote:I see a quite complicated setup just to allow the nodes to talk to each other :?

Yeah I think that is more my ambiguous and unnecessary explanations, as opposed to it being complicated :P I might make a Bash or Python script to make it a quicker process, where you just select whether this is the Master or the Slave node, and just enter all the IP addresses in.
Yeah I recommended the use of a NFS, and maybe the possibility of a Network Boot I wonder? I was advised from other sources that a MPI would be good for a small Beowulf cluster project. I'm afraid I couldn't add any useful insight into using a SOAP setup :P But it sounds like a good idea!
Cheers!
Phil
Posts: 5
Joined: Sun Jun 17, 2012 7:59 pm
Location: Liverpool
by philleonard » Fri Jul 13, 2012 5:21 pm
Hey guys, I have made a Python Script for installing and configuring MPICH2, and also another useful script which sets up RSA public/private key access between Raspberry Pis;

Installation Script
RSA Setup Script

Or for MPICH2 installation use the commands;
Code: Select all
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/install.py
sudo python install.py

and for RSA;
Code: Select all
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/rsa.py
python rsa.py

I apologise in advance if they are at all faulty (first Python programming I have done). It would be greatly appreciated if you could let me know of any faults!
Cheers!
Posts: 5
Joined: Sun Jun 17, 2012 7:59 pm
Location: Liverpool
by LakeyC » Mon Aug 05, 2013 9:10 pm
philleonard wrote:Hey guys, I have made a Python Script for installing and configuring MPICH2, and also another useful script which sets up RSA public/private key access between Raspberry Pis;

Installation Script
RSA Setup Script
malvern Dentist

Or for MPICH2 installation use the commands;
Code: Select all
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/install.py
sudo python install.py

and for RSA;
Code: Select all
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/rsa.py
python rsa.py

I apologise in advance if they are at all faulty (first Python programming I have done). It would be greatly appreciated if you could let me know of any faults!
Cheers!


You just saved me multiple hours setting up MPICH2, thank you SO MUCH! :)
Posts: 1
Joined: Mon Aug 05, 2013 9:08 pm
by More » Tue Nov 05, 2013 4:24 pm
As a beginner, I ran into a problem..

Whenever I try to execute
Code: Select all
mpiexec -f machinefile -n 100 ./cpi

it returns
Code: Select all
bash: mpiexec: command not found


I installed MPICH2 through install.py, so I don't think that's where it went wrong..

EDIT:

The install.py IS where I went wrong. Apparently it doesnt install MPICH properly. I installed by downloading the source and compiling/installing, and everything works smoothly.
User avatar
Posts: 1
Joined: Tue Nov 05, 2013 4:21 pm
Location: Netherlands