MuntyScruntfundle
Posts: 151
Joined: Fri Oct 27, 2017 11:14 pm

Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 3:16 am

Ok, so, after my messing about with led panels and clusters I've been approached to build a 128 node cluster. I'm sightly annoyed I don't have the money to build my own, I'll have to do with my modest 8 node system, but at least I'll get a go with this one I guess.

I've spent the day sourcing kit, pretty much done except for power. I have a couple of options:
1) 8 port 60/80w usb chargers from the like of Amazon for around £25, 16 needed, around £400 total.
2) industrial 16 port units, loads of power, £400-500 each, 8 needed around, up to £4,000 total.

Option 2 blows the budget right out of the door, down the high street and into the drain behind the grotty pub where the scrotes do things I don't want to know about. In fact that's the full cost of the pis. (it's possible I would have been classed as one of those scrotes a few decades ago!)

I know (in theory) the more expensive rack units will be more reliable, but we just can't do it sadly. I'd really like to present the monster in a decent cabinet with switches and power units all wired looking very professional, but I get the feeling after a few conversations the end user would prefer a more Heath Robinson finish with 3d printed mounting boards and fan

Option 1 would allow the use of usb splitter cables, this would keep the price down a bit and certainly add to the hairy look! Although I think we're going to need more fans.

Has anyone here attempted such a stupid ahem, interesting project, and if so what was used for power?

Many thanks.

rotwang
Posts: 239
Joined: Sat Dec 07, 2013 1:12 pm

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 9:46 am

You might consider looking for 5v switchmode PSU's. These should be available with ratings up to 50amps and beyond.
Roger

User avatar
RaTTuS
Posts: 10102
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 10:54 am

a quick look at ebay - many other are available - and are probably better
https://www.ebay.co.uk/itm/LED-Power-Su ... 1984443990
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

User avatar
Joel_Mckay
Posts: 141
Joined: Mon Nov 12, 2012 10:22 pm
Contact: Website

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 11:14 am

As rotwang has alluded, there are quite large power supplies available. However, you may want to look for an older pre-Rosewill unit that actually has more power available on the +5v rail.
These can be a little tricky to setup if you have never worked with load minimums before, but we use them at our club all the time:
http://www.micrometer.xyz/cdn/node/32
...we were running a rather large fiber coupled diode laser with a modded ATX at one point... ;-)

Additionally, a 4U rack mount case that can take older ATX supplies should only be under $80... and if you are using a low cost network switch it could likely fit in box too. https://www.newegg.com/Product/ProductL ... ageSize=36

@ RaTTuS
As a side note, stay away from the proprietary China LED ballasts as they have terrible regulation output, and may be impossible to repair/replace in the future. Most of our project was done because of the problems these would cause with our equipment... we have seen around 10% to 20% output drift under load from these units. Fine for what they were designed for... driving large LEDs.... but not safe for connecting to logic level stuff.

Best of luck,
~J~

mikerr
Posts: 2728
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 11:40 am

Do you want to completely DIY this ?

If you want more ready made solutions, check out the various Bitscope Blades,
http://bitscope.com/product/blade/

e.g. the quattro blade:
http://my.bitscope.com/store/?p=view&i=item+2
its powering each pi via GPIO header - much neater than lots of micro usb cables everywhere

they do a 40 node pi rack which is lots of quattro blades in a nice rack:
http://my.bitscope.com/store/?p=list&a=list&i=cat+blade

...and pricing is relatively low IMO

Alternatively you could give each Pi a PoE HAT, and power/connect via ethernet
- 96 port PoE ethernet switches do exist - that'll be really expensive though !
Android app - Raspi Card Imager - download and image SD cards - No PC required !

User avatar
Joel_Mckay
Posts: 141
Joined: Mon Nov 12, 2012 10:22 pm
Contact: Website

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 12:20 pm

mikerr wrote:
Tue Oct 23, 2018 11:40 am
they do a 40 node pi rack which is lots of quattro blades in a nice rack:
http://my.bitscope.com/store/?p=list&a=list&i=cat+blade
Those look awesome, but the onboard regulators look a bit small for four pi3b+, as we've measured about 1.8A peak loads with wifi running (although, the pi2 were nowhere near that high). Maybe bitscope can do a special order or something for this project, as these look like a serious time saver if they can meet the design requirements.

If you are in the US, there are also low-cost switch options if you are ok with used service hardware:
https://unixsurplus.com/products/dell-f ... no-modules

Cheers,
~J~

mikerr
Posts: 2728
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 12:50 pm

Surprisingly they do seem to use the same 3-4A regulator on both the Duo Pi and Quattro Pi
which gives the duo-based racks more headroom (2A per pi vs 1A per pi)
( the 20 node rack uses duo, the 40 node quattro pi)
Android app - Raspi Card Imager - download and image SD cards - No PC required !

MuntyScruntfundle
Posts: 151
Joined: Fri Oct 27, 2017 11:14 pm

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 3:02 pm

I’m actually kitted out quite well with variable power supplies, oscilloscopes, soldering kit and all sorts of magical electrical gubbins, but most of my electrical knowledge is invested in the world of high powered electric RC vehicles, so I know a bit, but not a lot.

If I were to take a bunch of the led drivers with a high enough ougtput to power a number of pis, what would be the easiest way to wire up a load of usb sockets? I was thinking hard wired would be a fun option, but if I have a plug at both ends of the cable I have two options for resetting a problem pi. Or I could soldier up an in-line switch on each cable, but I fear I would lose the will to live after number 47.

Cambrionix have just got back to me, the price for their units the U16 is right on the mark, until you add vat and the cost of the power supplies. I think I might need a more diy solution. But, it has to be solid and reliable.

drgeoff
Posts: 8598
Joined: Wed Jan 25, 2012 6:39 pm

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 4:11 pm

Powering, not charging!

User avatar
mikronauts
Posts: 2643
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 5:05 pm

I would use atx power supplies, 5v directly, and 12v regulated down to 5v.

MuntyScruntfundle wrote:
Tue Oct 23, 2018 3:02 pm
I’m actually kitted out quite well with variable power supplies, oscilloscopes, soldering kit and all sorts of magical electrical gubbins, but most of my electrical knowledge is invested in the world of high powered electric RC vehicles, so I know a bit, but not a lot.

If I were to take a bunch of the led drivers with a high enough ougtput to power a number of pis, what would be the easiest way to wire up a load of usb sockets? I was thinking hard wired would be a fun option, but if I have a plug at both ends of the cable I have two options for resetting a problem pi. Or I could soldier up an in-line switch on each cable, but I fear I would lose the will to live after number 47.

Cambrionix have just got back to me, the price for their units the U16 is right on the mark, until you add vat and the cost of the power supplies. I think I might need a more diy solution. But, it has to be solid and reliable.
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

ejolson
Posts: 2001
Joined: Tue Mar 18, 2014 11:47 am

Re: Oh damn, it's happened! Charging a lot of pis?

Tue Oct 23, 2018 5:28 pm

MuntyScruntfundle wrote:
Tue Oct 23, 2018 3:16 am
Ok, so, after my messing about with led panels and clusters I've been approached to build a 128 node cluster. I'm sightly annoyed I don't have the money to build my own, I'll have to do with my modest 8 node system, but at least I'll get a go with this one I guess.
The advantage of the Bitscope solution, as I understand it, is the ability to remotely power on and off each Pi as well as network boot them. Both of these features become important once the size of the cluster scales beyond about 20 nodes.

As Pi reliability is greatly dependent on quality of the power supply, here is an area where you don't want to cut too many corners. I've always wondered where to obtain a high-capacity computer-grade 5V power supply or whether a higher voltage supply with separate 5V switching regulators for each Pi would be better.

From what I understand, not all network switches allow a Pi 3B+ to network boot, so these have to be chosen carefully. I think a 3B+ will continue to boot from the network even if it has an SD card provided the FAT-formatted boot partition on the card is empty. Thus, you could design an initial RAM filesystem that loads over the network, checks if the root partition on the SD card needs to be imaged or resynched and after that pivots root to the SD card. Hadoop clusters boot in this fashion as well as some Linux distributions designed for clusters. Rolling your own for the Pi should not be very difficult and would provide the centralised maintenance advantages of network booting along with the performance advantages of having the root filesystem mounted from local storage.

In the other hand, it might be preferable to go without SD cards entirely and mount the root filesystems of all 128 machines from a single server. That server should not be another Pi. Note that the first suggested method of imaging and resynching SD cards could be done using another Pi without needing a faster server.

It would be great to hear how things progress.

Return to “General discussion”