Proposal: RPI CM Supercomputer/HA Architecture

[gridrun]

I wish I could implement this:



Maybe even a scaled down version limited to 4 CMs, with dual GigE instead of dual 10GigE.

[riklaunim]

It may be nice on drawings but also it must be practical. Do you have a use case that would make it worth the money, as well as time-money to develop software that would be able to use such architecture?

Nowadays there are GPU "supercomputers" or servers with high computing powers from Intel (like with their coprocessors). Very popular is OpenStack for open source cloud based services and computing. With that pretty drawings make practical clouds that can do given work. There was a lot on OpenStack on the PyCon PL conference I was lately. Red Hat, Mirantis, Intel and probably also Microsoft were involved

[gridrun]

Oh, I was just thinking a cluster full of highly available, super-low-power LINUX servers.

Existing software would be able to use this architecture. Each CM sees a GigE LAN interface on it's USB port, the FPGA provides standard Ethernet switching between CMs and the outside world. One such board being basically equal to a bunch of Raspberry Pis, joined by an Ethernet Switch, by each berry's Ethernet Port. Same approach as for example http://www.theregister.co.uk/2013/05/20 ... i_cluster/. But this time @ 480mps per CM, and without the cumbersome Ethernet cabling and external Switch. And without the n USB-LAN adapter chips for each berry.

Development would go into the FPGA. Presumably IP cores for GigE/10G MAC, Ethernet Switching and USB can be had on the market. Joining it all together would require some work, especially the necessary modifications to the USB Ethernet device.

[joan]

Oh, I was just thinking a cluster full of highly available, super-low-power LINUX servers.

Existing software would be able to use this architecture. Each CM sees a GigE LAN interface on it's USB port, the FPGA provides standard Ethernet switching between CMs and the outside world. One such board being basically equal to a bunch of Raspberry Pis, joined by an Ethernet Switch, by each berry's Ethernet Port. Same approach as for example http://www.theregister.co.uk/2013/05/20 ... i_cluster/. But this time @ 480mps per CM, and without the cumbersome Ethernet cabling and external Switch. And without the n USB-LAN adapter chips for each berry.

Development would go into the FPGA. Presumably IP cores for GigE/10G MAC, Ethernet Switching and USB can be had on the market. Joining it all together would require some work, especially the necessary modifications to the USB Ethernet device.

Wouldn't a multi-core modern chip be cheaper, faster, and more practical?

[gdt]

The Raspberry Pi does about 380 million floating point operations per second. A server processor purchased with supercomputing in mind will do about 400,000 MFLOPS. So to be competitive with a single server of a typical supercomputer cluster you'll need over a thousand Raspberry Pis. A single rack of supercomputer servers is equivalent to more than 150,000 Raspberry Pis.

If it makes you feel any better, the supercomputing installation I am associated with run Raspberry Pis to do DHCP, DNS and to provide access to serial consoles on the management network of the supercomputer.

The RPi is usefu for supercomputer architecture *research*. Because it's so low powered, inefficiencies are not hidden by the sheer throughput of the faster machines.

[gridrun]

If you want a cluster for sheer power, you're certainly NOT going to build it with Raspberry Pis. Agreed.
For research and education, it would be great though. And, again, I could see practical use with low volume, HA services. The BCM SoC has very modest power requirements...

However my focus was more on the "use an FPGA instead of an external Switch and tons of wiring" aspect. People do build Raspberry Pi clusters, after all.