PallasMedia
Posts: 2
Joined: Fri Jul 17, 2020 8:19 pm
Location: Huntsville, AL
Contact: Twitter

Planning for my first Pi cluster

Fri Jul 17, 2020 8:35 pm

Hey everyone, I've been working with Raspberry Pi's a little here and there ever since my first Pi which I believe was the Raspberry Pi 2 Model B. Most of my experience has been with setting up the each generation with RetroPie to see how performance has improved and also setting up a couple of Raspberry Pi Zero W's in kiosk mode for a local non-profit. For a while now I've been wanting to build my own cluster but didn't have any ideas for what I would use it for until now. I've come up with a couple of uses case. First I want to build a cluster that is running a kubernetes to host my side projects (I'm a web developer). Second I want to have a cluster to run a personal Minecraft server. I stream on Twitch and my community is made up of both gamers and tech related people in the IT/Web Dev industry. So I was thinking it 'd be cool to be able to have a Minecraft server we could all play on. Now that you know what I want to do my question is this. Do I have to set up 2 indvidual clusters with one being used for each scenario or can I setup a larger single cluster and designate a number of Pi's for each thing? It'd be cool if I could just have a single cluster and tell Kubernetes to only run on these 5 Pi's so that the other 5 could be used however I see fit. Is this possible?

bigcrater
Posts: 12
Joined: Sat Apr 14, 2018 2:53 am

Re: Planning for my first Pi cluster

Sat Jul 18, 2020 5:42 am

I set up a cluster once upon a time and wound up not needing it, so my experience is pretty limited. I used the corosync and pacemaker stack. I'm sure you can set up one big cluster and designate this and that subset for a given task. But the way you asked the question suggests a misunderstanding. What's the point of having one cluster if the tasks are split into two non-interacting subsets? That's basically the same thing as two separate clusters, right? On the other hand, it is true that the corosync and pacemaker stack will let you designate which tasks go to which machines and you could, if you wanted, allow machines from one subset to take over for the other subset in a failover. So I'm not sure I'm addressing your question.

As a pure aside, I struggled, stupidly as it turns out, to get drbd to work on my Pi cluster. I was using an SD card mounted through the USB ports, but it turns out that that won't work -- or at least, wouldn't back in the day. IIRC there was an issue with SD cards not playing nice with the drbd software. Would that work with hard disks or ssds? dunno. and maybe that's changed now.

If you're going for blazing speed -- a Pi cluster isn't going to get you there. So is this a load balancing or high availability cluster? I'm pretty darn sure that if you have a massive process on one machine in the cluster that fails that that process is dead -- that is, other machines will take over and let the process start all over. (Ah, I'm not familiar with minecraft, it sounds like a game.) So I'm personally skeptical about using a cluster for a compute intensive game. If a machine crashes an hour into a game (depending on how the game is written) I would expect that that game is vaporized . Though I'm not a gamer; so maybe every game is already capable of saving the state as it runs. Still, I have trouble imagining a Pi cluster as anything other than high availability. To get speed on a big computational problem you need to be able to send numbers around pretty fast. Even a gigabit ethernet has a theoretical max of around 12.5M doubles/sec and that's just not fast enough fast enough (IMHO).

My cluster was running a dns server. And I only wanted to gain experience with running a cluster. I never intended to run the Pi's in production. If DNS crashes during a lookup and it fails over, chances are you only lose whatever was happening at the exact moment of the crash.

j.

epoch1970
Posts: 5131
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Planning for my first Pi cluster

Sat Jul 18, 2020 9:45 am

PallasMedia wrote:
Fri Jul 17, 2020 8:35 pm
Do I have to set up 2 indvidual clusters with one being used for each scenario or can I setup a larger single cluster and designate a number of Pi's for each thing? It'd be cool if I could just have a single cluster and tell Kubernetes to only run on these 5 Pi's so that the other 5 could be used however I see fit. Is this possible?
Yes possible. Cluster membership usually doesn’t spread like a virus. There are some prerequisites to be met, so surely you can have 5 clustered Pis and 5 non clustered, or 2 independent clusters of 5, or a cluster of 10 running 2 workloads spread differently among the nodes.

I have little experience with k8s and I’m not ready to swallow the Helm pill, but what I’d do next is try to run 2 hello world workloads in a test cluster, possibly using VMs to emulate the physical nodes.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

fruitoftheloom
Posts: 23337
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: Planning for my first Pi cluster

Sat Jul 18, 2020 10:51 am

PallasMedia wrote:
Fri Jul 17, 2020 8:35 pm
Hey everyone, I've been working with Raspberry Pi's a little here and there ever since my first Pi which I believe was the Raspberry Pi 2 Model B. Most of my experience has been with setting up the each generation with RetroPie to see how performance has improved and also setting up a couple of Raspberry Pi Zero W's in kiosk mode for a local non-profit. For a while now I've been wanting to build my own cluster but didn't have any ideas for what I would use it for until now. I've come up with a couple of uses case. First I want to build a cluster that is running a kubernetes to host my side projects (I'm a web developer). Second I want to have a cluster to run a personal Minecraft server. I stream on Twitch and my community is made up of both gamers and tech related people in the IT/Web Dev industry. So I was thinking it 'd be cool to be able to have a Minecraft server we could all play on. Now that you know what I want to do my question is this. Do I have to set up 2 indvidual clusters with one being used for each scenario or can I setup a larger single cluster and designate a number of Pi's for each thing? It'd be cool if I could just have a single cluster and tell Kubernetes to only run on these 5 Pi's so that the other 5 could be used however I see fit. Is this possible?


https://www.jeffgeerling.com/projects

https://www.raspberrypi.org/blog/five-y ... -clusters/
Rather than negativity think outside the box !
RPi 4B 4GB (SSD Boot)..
Asus ChromeBox 3 Celeron is my other computer...

PallasMedia
Posts: 2
Joined: Fri Jul 17, 2020 8:19 pm
Location: Huntsville, AL
Contact: Twitter

Re: Planning for my first Pi cluster

Sat Jul 25, 2020 8:15 pm

bigcrater wrote:
Sat Jul 18, 2020 5:42 am
But the way you asked the question suggests a misunderstanding. What's the point of having one cluster if the tasks are split into two non-interacting subsets? That's basically the same thing as two separate clusters, right?
Sorry for the delay in responding, been a crazy week. I don't think I"m confused but maybe I am. From what I've read Kubernetes (or even Docker Swarm) would allow me to run multiple Docker containers (tasks) on the cluster. I just wasn't sure if I could group multiple Pi's in the cluster to focus on a task so that the Minecraft server can get more resources for the game and so that the rest could be used for whatever else I want to use it for.

Return to “Networking and servers”