hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

First "REAL" RPi Rack

Fri Jul 22, 2016 8:10 am

Hi guys,

we built what i believe is the first "REAL" RPi Rack. It has 3U size and we are able to fit 70 rpis inside @600mm case depth. Inside this custom case we integrate switches already so there are just a few LAN outputs outside + we managed to get remote reboots for each raspi on our pcb boards. Each rpi is easy to manage, no screws or tools required to exchange defective part.
This is our 3rd version. It supports Rpi 1/2/3/ and Banana PI.

Image

Our v4 will be able @42U rack @950mm depth fit 1350 RPIs inside.

Any suggestions for improvements are welcome.

PiAhoy
Posts: 24
Joined: Sat Aug 29, 2015 1:27 pm

Re: First "REAL" RPi Rack

Fri Jul 22, 2016 8:47 am

Uhh..what do you use it for?

ejolson
Posts: 2752
Joined: Tue Mar 18, 2014 11:47 am

Re: First "REAL" RPi Rack

Fri Jul 22, 2016 7:18 pm

hrs wrote:Any suggestions for improvements are welcome.
It is not clear from the picture, but are there heatsinks and/or heatpipes on the Pi 3s? Have you stress tested the unit with all Pi computers under load?

As for uses, maybe your company could sponsor a team in one of the supercomputer student cluster competitions. Such competitions have 3kw power limit and ARM is starting to look competitive from a flop/watt point of view.

hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

Re: First "REAL" RPi Rack

Sat Jul 23, 2016 3:12 am

@PiAhoy - our client is using them for clustering. Currently we got 5 of there which makes 350 RPIs total in cluster.

@yes each rpi got own heatsink, stress test was running for hours without problems.

Image

User avatar
dasmanul
Posts: 502
Joined: Wed Sep 30, 2015 10:20 am
Location: Frankfurt, Germany

Re: First "REAL" RPi Rack

Sat Jul 23, 2016 8:32 am

Very nice! :) Although I'll admit I don't personally see the point of a cluster of Pi's...

Naive question regarding the heatsinks: Wouldn't it have been better to attach them turned 90° to better align with the airflow?

hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

Re: First "REAL" RPi Rack

Sat Jul 23, 2016 12:25 pm

@dasmanul - the potential is large. Imagine 70 standalone computers ( or clustered ) @150Watt in 3U form. Some of our customers use it for dedicated server rental ( even its rpi its still better than virtual server ) and some using as hardware h264 transcoder. About heatsinks u right, but in version1 case we used different fixation, so now they look like turned wrong. But it does not matter they do not generate too much heat, even without heatsinks they working fine.

ejolson
Posts: 2752
Joined: Tue Mar 18, 2014 11:47 am

Re: First "REAL" RPi Rack

Sat Jul 23, 2016 7:54 pm

hrs wrote:@dasmanul - the potential is large. Imagine 70 standalone computers ( or clustered ) @150Watt in 3U form. Some of our customers use it for dedicated server rental ( even its rpi its still better than virtual server ) and some using as hardware h264 transcoder. About heatsinks u right, but in version1 case we used different fixation, so now they look like turned wrong. But it does not matter they do not generate too much heat, even without heatsinks they working fine.
Heat convection is naturally vertical and likely to be more significant near the surface of the heatsink than the air movement caused by the fans. The fans are needed to prevent heat from accumulating inside the box, but it is likely the current configuration for the heatsinks is the right one.

The only way to know for sure is to measure the internal CPU temperature with the heatsinks in one orientation and again in the other orientation. To negate the effects of silicon variations, the exact same Pi computers should be used for both tests.

For dedicated server rental, individual Pi computers may have certain security advantages over virtual servers. For example side-channel attacks based on shared caches could be avoided. At the same time 1GB RAM and 100mbit networking speed could be limiting. Does the h264 transcoder application leverage the GPUs?

hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

Re: First "REAL" RPi Rack

Sun Jul 24, 2016 2:33 am

@ejolson - that is correct. h264 transcoder uses just GPU and small amount of cpu if audio conversion is needed. Our tests show that single rpi2 can make up to 5 h264 conversions real time @ static bitrate 1.2Mbit.

asavah
Posts: 348
Joined: Thu Aug 14, 2014 12:49 am

Re: First "REAL" RPi Rack

Sun Jul 24, 2016 6:20 pm

@hrs
May I ask what software with hw transcode support is being used?

Awesome rack btw.

hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

Re: First "REAL" RPi Rack

Sun Jul 24, 2016 6:41 pm

@asavah - its our own developed software based on omx

ejolson
Posts: 2752
Joined: Tue Mar 18, 2014 11:47 am

Re: First "REAL" RPi Rack

Sun Jul 24, 2016 7:41 pm

hrs wrote:@ejolson - that is correct. h264 transcoder uses just GPU and small amount of cpu if audio conversion is needed. Our tests show that single rpi2 can make up to 5 h264 conversions real time @ static bitrate 1.2Mbit.
That is very impressive. I would have thought the GPU in the Pi could only transcode one stream at a time. What is the resolution and frame rate?

hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

Re: First "REAL" RPi Rack

Sun Jul 31, 2016 8:34 am

@ejolson SD at 25fps

kevinpr
Posts: 1
Joined: Fri Sep 09, 2016 12:44 pm

Re: First "REAL" RPi Rack

Fri Sep 09, 2016 12:48 pm

This is a very impressive solution! Can you share the name of your company along with contact information?

jrm
Posts: 1
Joined: Sat Jan 21, 2017 6:14 pm

Re: First "REAL" RPi Rack

Sat Jan 21, 2017 6:19 pm

@rps do you plan to sell the rack in a near future ?
Regards,

Jeremy

tutzeroone
Posts: 1
Joined: Tue Apr 04, 2017 1:12 pm

Re: First "REAL" RPi Rack

Tue Apr 04, 2017 2:20 pm

@hrs

Hello, i'm very interested by this project, andI've got a few questions :

- Are there units for sale, do you have a website or some pricing you could communicate ?
- How are the RPis connected to the main boards underneath (usb/ethernet/custom), is that same connection used to old the cards in place ?
- Would the project be compatible with other cards, (Odroid boards for example) ?

Thanks in advance for your answers !

Chorizo.Omelet
Posts: 1
Joined: Wed Sep 13, 2017 12:08 am

Re: First "REAL" RPi Rack

Wed Sep 13, 2017 12:16 am

Has anyone been successful in contacting HRS?

Thanks

hrs
Posts: 9
Joined: Fri Jul 22, 2016 7:40 am

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 7:04 am

pretty late but still

@tutzeroone
- we did that for our client, we did not think about the sale yet
- connected via eth, it holds rpi in place and is connected to switch
- yes its compatible with most credit card sized computers

User avatar
HawaiianPi
Posts: 3758
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 11:37 am

Whenever I see one of these all I can think about is all the wasted space and ports. Why are these always based on the consumer models with all the extra parts that aren't needed or used?

I mean, I understand that when it's done on a small scale by a hobbyist or school, but if you are going to all the trouble of making a custom rack for high density clusters, then why not build a rack based on the CM3. You could eliminate a lot of the interconnecting cables and fit a whole lot more in the same space.

Imagine a backplane with rows (and rows) of CM3 slots. Power distribution, networking and other I/O all routed through the backplane. Much neater and cleaner (no birds nest of cables), and you could fit a ton more CM3 boards in the same space.
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 21888
Joined: Sat Jul 30, 2011 7:41 pm

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 11:53 am

HawaiianPi wrote:
Wed Dec 06, 2017 11:37 am
Whenever I see one of these all I can think about is all the wasted space and ports. Why are these always based on the consumer models with all the extra parts that aren't needed or used?

I mean, I understand that when it's done on a small scale by a hobbyist or school, but if you are going to all the trouble of making a custom rack for high density clusters, then why not build a rack based on the CM3. You could eliminate a lot of the interconnecting cables and fit a whole lot more in the same space.

Imagine a backplane with rows (and rows) of CM3 slots. Power distribution, networking and other I/O all routed through the backplane. Much neater and cleaner (no birds nest of cables), and you could fit a ton more CM3 boards in the same space.
I'd guess its more expensive to build you own carrier boards with ethernet chips etc, than just use the Pi3.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
HawaiianPi
Posts: 3758
Joined: Mon Apr 08, 2013 4:53 am
Location: Aloha, Oregon USA

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 12:00 pm

jamesh wrote:
Wed Dec 06, 2017 11:53 am
I'd guess its more expensive to build you own carrier boards with ethernet chips etc, than just use the Pi3.
True, but it would be so awesome!
My mind is like a browser. 27 tabs are open, 9 aren't responding,
lots of pop-ups...and where is that annoying music coming from?

Heater
Posts: 11818
Joined: Tue Jul 17, 2012 3:02 pm

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 12:08 pm

HawaiianPi,
..why not build a rack based on the CM3...?
It's more expensive. Isn't it?

CM3 from Digikey are $53. The Pi 3 only $40.

mic_s
Posts: 85
Joined: Sun Oct 26, 2014 4:15 pm

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 12:41 pm

It's more expensive.
The other way around.

Element14:
Pi3B, 10+ 36.9$, 150+ 34.12$
CM3, 10+ 30.49$, 100+ 29.68$
CM3LT, 10+ 25.25$, 100+ 24,31$

Heater
Posts: 11818
Joined: Tue Jul 17, 2012 3:02 pm

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 12:43 pm

Hmm...Digkey seems to be way out of line there then.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 21888
Joined: Sat Jul 30, 2011 7:41 pm

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 1:18 pm

Heater wrote:
Wed Dec 06, 2017 12:43 pm
Hmm...Digkey seems to be way out of line there then.
Yup. Not sure what they are thinking.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

MarkTF
Posts: 292
Joined: Tue Mar 03, 2015 4:59 pm

Re: First "REAL" RPi Rack

Wed Dec 06, 2017 2:23 pm

HawaiianPi wrote:
Wed Dec 06, 2017 11:37 am
. . . if you are going to all the trouble of making a custom rack for high density clusters, then why not build a rack based on the CM3. You could eliminate a lot of the interconnecting cables and fit a whole lot more in the same space. . .
It's probably relevant that the original post is from July 2016 and the CM3 wasn't announced until January 2017.

Return to “General discussion”