Ben_is_best
Posts: 89
Joined: Fri Dec 25, 2015 11:01 pm
Location: USA

2 node raspberry pi supercomputer

Thu Dec 31, 2015 6:32 pm

I am trying to make a raspberry pi supercomputer with a raspberry pi zero and a raspberry pi 2. I have powered the raspberry pi zero from the raspberry pi 2's USB ports, and I am using Internet sharing with the GPIO. How would you make a supercomputer like that?

Thanks in advance.

Edit: I now have a Raspberry Pi 3. Does that make a difference?
Last edited by Ben_is_best on Sun May 08, 2016 12:59 pm, edited 3 times in total.

Heater
Posts: 12960
Joined: Tue Jul 17, 2012 3:02 pm

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 8:54 pm

I'm curious. How are you sharing the internet, or doing any IP communication for that mater, over the GPIO?

I know one way to do it but it would be terrible slow.

User avatar
Laurens-wuyts
Posts: 716
Joined: Wed Aug 21, 2013 7:35 pm
Location: Belgium
Contact: Website

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 9:32 pm

Maybe some program/driver of your own which can write at 20MHz to the GPIO (Maximum on zero) and then a 8 bit/16bit databus? :)
Just thinking out of the box :P

Laurens

User avatar
mikronauts
Posts: 2708
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 9:39 pm

1) simply plug the zero into one of the Pi 2's USB ports, and use USBNET as suggested in another thread

2) that would not be a super computer, but a small cluster, google definition of super computer
Ben_is_best wrote:I am trying to make a raspberry pi supercomputer with a raspberry pi zero and a raspberry pi 2. I have powered the raspberry pi zero from the raspberry pi 2's USB ports, and I am using Internet sharing with the GPIO. How would you do it?

Thanks in advance.
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

Heater
Posts: 12960
Joined: Tue Jul 17, 2012 3:02 pm

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 9:40 pm

Inside the box we find that there is a UART on the GPIO pins.

So connect the Pi together using their UARTs and run PPP (or SLIP) on each of them. PPP will create a network interface that connects the two Pi.

I have no idea what the top speed of the UART can be.

Faffing around creating drivers to use other features of the GPIO seems like a waste of life.

Whatever you do, you don't end up with any kind of super computer.

If I wanted to tinker and learn about clustering things I'd be using the same machine at each node. Just to make life simpler. Probably use Pi 2.

That's not to say that there might not be very good uses for a Pi 2 and a Zero together.

Heater
Posts: 12960
Joined: Tue Jul 17, 2012 3:02 pm

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 9:42 pm

I was wondering about the networking over USB idea. There is a slight chance that using PPP over the UARTs may be a bit quicker. One is not shoveling all data through the same interface.

fruitoftheloom
Posts: 20139
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 9:51 pm

REMOVED RESPONSE
Last edited by fruitoftheloom on Thu Dec 31, 2015 10:11 pm, edited 1 time in total.
adieu

Asus CS10 Chromebit / HP Envy 4500 Wireless Printer / Raspberry Pi Model 2B v1.1 / RealVNC Software...

gregeric
Posts: 1509
Joined: Mon Nov 28, 2011 10:08 am

Re: 2 node raspberry pi supercomputer

Thu Dec 31, 2015 10:00 pm

fruitoftheloom wrote:
Heater wrote:I was wondering about the networking over USB idea. There is a slight chance that using PPP over the UARTs may be a bit quicker. One is not shoveling all data through the same interface.
Did you mean my post regards USBNET which I undertook as proof of concept:

viewtopic.php?f=63&t=130810



http://www.linux-usb.org/usbnet
There are other USB 2.0 high speed (480 Mbit/sec) host-to-host link products available too, with data transfer speeds that comfortably break 100BaseT speed limits. A single USB 2.0 link should handle even full duplex 100BaseT switches (100 MBit/sec in each direction) with capacity to spare, and more capable EHCI controllers should handle several such links.
Most definitely not, cos Heater was speculating whether separate serial PPP may be quicker than shoving everything via that single crippled USB.

User avatar
Jim Manley
Posts: 1600
Joined: Thu Feb 23, 2012 8:41 pm
Location: SillyCon Valley, California, and Powell, Wyoming, USA, plus The Universe
Contact: Website

Re: 2 node raspberry pi supercomputer

Fri Jan 01, 2016 7:49 am

gregeric wrote:
fruitoftheloom wrote:
Heater wrote:I was wondering about the networking over USB idea. There is a slight chance that using PPP over the UARTs may be a bit quicker. One is not shoveling all data through the same interface.
Did you mean my post regards USBNET which I undertook as proof of concept:
viewtopic.php?f=63&t=130810
http://www.linux-usb.org/usbnet
There are other USB 2.0 high speed (480 Mbit/sec) host-to-host link products available too, with data transfer speeds that comfortably break 100BaseT speed limits. A single USB 2.0 link should handle even full duplex 100BaseT switches (100 MBit/sec in each direction) with capacity to spare, and more capable EHCI controllers should handle several such links.
Most definitely not, cos Heater was speculating whether separate serial PPP may be quicker than shoving everything via that single crippled USB.
Max serial speed over an older UART is typically 115,200 bps if you're talking to something like an Arduino or a PC serial port due to clock crystal stability (how closely does it remain to its rated center frequency over time vs. temperature, humidity, vibration, etc.). The latest UARTs can get up to around 1 Mbps, but all of this is highly dependent on the quality of the signal, which is determined by many factors including cable length/conductor gauge/shielding, signal voltage level, induced noise (RS-232 is single line and therefore not very noise-tolerant, as opposed to differential signals over two lines with simultaneous opposing polarities like RS-422, USB, Firewire, etc.), clock crystal stability, impedance matching, black helicopter and alien transmissions, etc.

There's an interesting thread at viewtopic.php?f=44&t=17559 where Gert (who designed that part of the SoC) notes that the theoretical maximum speed of the GPIO ports is upwards of 100 MHz each, but someone in that thread found about a 4 Mbps limit in OS level code and it wasn't clear how that limit could be changed without totally screwing up timing of all of the other protocols such as SPI, I2C, etc. Gert noted that there are only four phased-lock loops (PLLs) in the hardware that all clock signals have to divide down in order to get other frequencies that would correspond to canonical data rates. However, if two Pii are communicating, the specific data rates are unimportant after all other factors are optimized.

I2C is practically limited to about 3.2 Mbps due to its single-line nature, but SPI, with its control lines, has a much higher upper limit, with 100 Mbps theoretically possible, depending on the hardware interface (some devices are limited to as little as about 6 Mbps). There's that 4 MBps OS limit, but whatever the max speed is on a single Pi GPIO SPI connection, multiple SPI connections can be established via software setup, with the maximum number of simultaneous possible connections depending on whether 26-pin or 40-pin GPIO ports are available between the Pii.

I find that I have to keep reminding everyone that quoted data rates, including for USB, are all _theoretical_instantaneous_upper_limits_ that are _never_ achieved in the Real World, and far from it. So, although the theoretical instantaneous speed of USB 2.0 is 480 Mbps, due to cable lengths/conductor gauge/shielding quality/connector wear or corrosion/etc., whether cables are kept straight, bent, or coiled, the presence of potential interfering RF signals, etc., the actual "goodput" (a real telecom technical term for a number less than theoretical "throughput" that takes these kinds of real world factors into account through actual measurement) can easily be only 10% of the theoretical maximum. Couple that limit with the fact that the Pi's single shared bus for USB, Ethernet, etc., is a significant chokepoint, and your USB port isn't nearly as fast as you might think. If you have any other USB peripherals connected, the goodput drops even further.

As they say, Your Mileage May Vary, Batteries not included, Objects in Mirror Are Closer Than They Appear, Do Not Spindle, Fold, or Mutilate ... :lol:
The best things in life aren't things ... but, a Pi comes pretty darned close! :D
"Education is not the filling of a pail, but the lighting of a fire." -- W.B. Yeats
In theory, theory & practice are the same - in practice, they aren't!!!

danjperron
Posts: 3376
Joined: Thu Dec 27, 2012 4:05 am
Location: Québec, Canada

Re: 2 node raspberry pi supercomputer

Fri Jan 01, 2016 2:20 pm

If you are interested in PPPD

Check my post
viewtopic.php?p=826629#p826629

And then an another post I set the speed to 4M baud without any problem. I didn't try higher baud rate. Maybe it would work, maybe not.

viewtopic.php?p=849657#p849657

User avatar
DougieLawson
Posts: 35535
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: 2 node raspberry pi supercomputer

Fri Jan 01, 2016 3:42 pm

If you want to set up PPPD on a zero take a look at the instructions in The MagPi issue#41 pages50-51
Note: Having anything remotely humorous in your signature is completely banned on this forum. Wear a tinfoil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

mikerr
Posts: 2770
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: 2 node raspberry pi supercomputer

Sun Jan 03, 2016 12:10 pm

Enable gadget mode on the zero, and you get usb networking for free

single USB cable will both power and network the zero to pi2

Here's my setup:

Image

using ready made image for the pi zero to appear as virtual ethernet:
https://sourceforge.net/projects/pizero-usbhost/
Android app - Raspi Card Imager - download and image SD cards - No PC required !

fruitoftheloom
Posts: 20139
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: 2 node raspberry pi supercomputer

Sun Jan 03, 2016 12:31 pm

mikerr wrote:Enable gadget mode on the zero, and you get usb networking for free

single USB cable will both power and network the zero to pi2

Here's my setup:

Image

using ready made image for the pi zero to appear as virtual ethernet:
https://sourceforge.net/projects/pizero-usbhost/
Is the change to "Gadget" Mode purely a software configuration in config.txt ?

Is it feasible to use for internet access on the RPi0 ?

What would happen if a normal Jessie OS is booted with the USB plugged in, as per your pic,
would you have a situation where you have 2 hosts and smoke signals ?
adieu

Asus CS10 Chromebit / HP Envy 4500 Wireless Printer / Raspberry Pi Model 2B v1.1 / RealVNC Software...

mikerr
Posts: 2770
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: 2 node raspberry pi supercomputer

Sun Jan 03, 2016 1:38 pm

fruitoftheloom wrote:Is the change to "Gadget" Mode purely a software configuration in config.txt ?
It's all software, but a bit more than a line in config.txt
-different kernel and files, gadget/device mode:
viewtopic.php?f=29&t=129653
fruitoftheloom wrote:Is it feasible to use for internet access on the RPi0 ?
Yes, same procedure with iptables as internet sharing eth0 to wlan0
(pizeroes show up as network interfaces usb0, usb1)

I can't seem to get >100Mbps out though, despite the virtual ethernet reporting 425Mbps,
similar to your usbnet tests.
fruitoftheloom wrote:What would happen if a normal Jessie OS is booted with the USB plugged in, as per your pic,
would you have a situation where you have 2 hosts and smoke signals ?
No smoke, just no usb comms.
Android app - Raspi Card Imager - download and image SD cards - No PC required !

fruitoftheloom
Posts: 20139
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: 2 node raspberry pi supercomputer

Sun Jan 03, 2016 2:12 pm

mikerr wrote:
fruitoftheloom wrote:What would happen if a normal Jessie OS is booted with the USB plugged in, as per your pic,
would you have a situation where you have 2 hosts and smoke signals ?
No smoke, just no usb comms.
That is good to know in case one boots up with wrong card.


Thanks for other information and yes was surprised by lack of Ethernet Speed
over USB, considering a RPi 1B+ is supposedly not dissimilar to a RPi 2B...
:?
adieu

Asus CS10 Chromebit / HP Envy 4500 Wireless Printer / Raspberry Pi Model 2B v1.1 / RealVNC Software...

Ben_is_best
Posts: 89
Joined: Fri Dec 25, 2015 11:01 pm
Location: USA

Re: 2 node raspberry pi supercomputer

Sun Jan 03, 2016 11:39 pm

mikerr wrote:Enable gadget mode on the zero, and you get usb networking for free

single USB cable will both power and network the zero to pi2

Here's my setup:

Image

using ready made image for the pi zero to appear as virtual ethernet:
https://sourceforge.net/projects/pizero-usbhost/
Would you need to have additional software on the Pi 2 to get it to work?

plugwash
Forum Moderator
Forum Moderator
Posts: 3433
Joined: Wed Dec 28, 2011 11:45 pm

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 3:57 am

Unless things have changed recently the driver needed for the host end is already in the standard Pi kernel.

You will probablly want some custom scripting to do something sensible with all the interfaces that appear.

ejolson
Posts: 3228
Joined: Tue Mar 18, 2014 11:47 am

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 4:10 am

mikerr wrote:Here's my setup:

Image
It is great to see small clusters built with the Pi Zero using almost no additional hardware. As mentioned in other threads, such a cluster could serve many educational purposes.

User avatar
karrika
Posts: 1057
Joined: Mon Oct 19, 2015 6:21 am
Location: Finland

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 6:12 am

This is really amazing work. Four computing nodes and an Ethernet port. A year ago I was struggling to get something like this as a companion for Red Pitaya. Unfortunately I tried the SPI which was too slow so the project was dropped.

Naturally I would go for faster Raspberries and not PiZero's. The architecture of using USB ports in this way is exciting.

ejolson
Posts: 3228
Joined: Tue Mar 18, 2014 11:47 am

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 7:25 am

karrika wrote:Naturally I would go for faster Raspberries and not PiZero's. The architecture of using USB ports in this way is exciting.
The Linux Gadget driver only works on the Zero, A and A+ models. The Zero is particularly nice because it can be can be simultaneously networked and powered through a standard USB cable.

There are lots of threads recently that say, "wouldn't it be nice if there was a version of the Zero which also had feature X." For building educational clusters I would suggest feature X be a 4 core CPU. Given how similar the BCM2835 is to the BCM2836 this should be pretty easy for the foundation to do if they wanted.

User avatar
rpdom
Posts: 14715
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 7:38 am

ejolson wrote:There are lots of threads recently that say, "wouldn't it be nice if there was a version of the Zero which also had feature X." For building educational clusters I would suggest feature X be a 4 core CPU. Given how similar the BCM2835 is to the BCM2836 this should be pretty easy for the foundation to do if they wanted.
Apart from the extra complications of having the RAM chip mounted on the other side of the board instead of on top of the BCM2835. Having components on only one side of the board and using the BCM2835 with its PoP mounted RAM are two major factors in reducing the cost of the Zero.

User avatar
karrika
Posts: 1057
Joined: Mon Oct 19, 2015 6:21 am
Location: Finland

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 8:19 am

ejolson wrote:
karrika wrote:Naturally I would go for faster Raspberries and not PiZero's. The architecture of using USB ports in this way is exciting.
The Linux Gadget driver only works on the Zero, A and A+ models. The Zero is particularly nice because it can be can be simultaneously networked and powered through a standard USB cable.
That is ok. 3 PiZero's and a Pi 2 is more than enough for my needs. The Ethernet is needed for feeding high speed digitized data into the cluster. A separate SPI-based Ethernet can be added for exporting the data. The bandwidth of the exported stuff is minimal compared to the input.

Worker threads exported to PiZero's may change the game. The good thing is that it will not only add calculation power. It will also add RAM for data storage. At least I would like to try this setup.

Ben_is_best
Posts: 89
Joined: Fri Dec 25, 2015 11:01 pm
Location: USA

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 12:40 pm

plugwash wrote:Unless things have changed recently the driver needed for the host end is already in the standard Pi kernel.

You will probablly want some custom scripting to do something sensible with all the interfaces that appear.
Would you have to set up the software or does it just "start working" right when you plug in the Zeros?

ejolson
Posts: 3228
Joined: Tue Mar 18, 2014 11:47 am

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 5:31 pm

mikerr wrote: I can't seem to get >100Mbps out though, despite the virtual ethernet reporting 425Mbps,
similar to your usbnet tests.
That is interesting, I wonder if the bottleneck is with the built-in hub of the Pi 2B or the USB stack itself. Can you run netperf for the network bandwidth tests? It would be interesting to know the following:

Speed of 2B to Zero. Start "netserver" on the Zero and then on the 2B run "netperf -H x.y.z.w" where x.y.z.w is the IP number of the Zero.

Speed of the Zero to 2B. Start "netserver" on the 2B and then on the Zero run "netperf -H x.y.z.w" where x.y.z.w is the IP number of the 2B.

Speed of the Zero to a different Zero. Start "netserver" on one of the Zeros and then on the other Zero run "netperf -H x.y.z.w" where x.y.z.w is the IP number of the first Zero.

Make sure to kill all running copies of netserver between each test. For the last test you will need to enable IP forwarding on the Pi 2B if you have not already done this, so that the 2B can serve as a router to allow one Zero to reach the other Zero.

My guess is that the network speed between the Zero and the 2B will be about the same in either direction and that the speed between two Zeros will be about half that. However, since 100 mbps is well below the 480 mbps USB2 spec, it is possible there is enough bandwidth available that the Zero to Zero speed is also close to 100 mbps.
Last edited by ejolson on Mon Jan 04, 2016 5:47 pm, edited 2 times in total.

fruitoftheloom
Posts: 20139
Joined: Tue Mar 25, 2014 12:40 pm
Location: Delightful Dorset

Re: 2 node raspberry pi supercomputer

Mon Jan 04, 2016 5:44 pm

ejolson wrote:
mikerr wrote: I can't seem to get >100Mbps out though, despite the virtual ethernet reporting 425Mbps,
similar to your usbnet tests.
That is interesting, I wonder if the bottleneck is with the built-in hub of the Pi 2B.
I ran iPerf using a USBNET Cable and I had similar results to mikerr, unfortunately do not have a A+ for comparison, see:

viewtopic.php?f=36&t=131042
adieu

Asus CS10 Chromebit / HP Envy 4500 Wireless Printer / Raspberry Pi Model 2B v1.1 / RealVNC Software...

Return to “General discussion”