Turning the R-Pi into a parallel System

[VassBent]

Hello to everyone

As this is my first post on this site, my congratulations to the creators of the R-Pi for a great conception. Actually, the word "great" appears to be an understatement, but it will have to do for now, as I believe that there is some room for further improvement. And this brings me to the question I would like to ask the developers. However, since 42 pages of topics have already been posted, if someone else has brought up the same question before, my apologies & please refer me to the right topic. So, here goes

Would it be a good idea to look into the possibility of having several Pi-s working in parallel to one another, say by making it possible for them to dock on a platform (to be created) in a way similar to the way the PCI cards dock on a PC mainboard ?? And, to take the whole idea even further down that path, to make such platforms connect to one another, forming co-operating clusters of Pi-s ??? That would really boost the capabilities of the Pi-s by applying on them the principles of Parallel Programming (= OpenCL), if more computing power is required by some applications , say in engineering, at a remarkably low power consumption and price.

Even further down the road, the same idea could be materialized wirelessly, turning a bunch of PI-s into a distributed system, where adding new or removing existent PI-s would add or subtract "nodes to the ring", so to speak...but that is almost science fiction at this point, i suspect.

Thank you aforehand for your time
V

[DeeJay]

http://www.southampton.ac.uk/~sjc/raspberrypi/

[W. H. Heydt]

Aside from the link DeeJay posted, you're asking for some kind of system bus. I don't know if the BCM2835 even has one, and if it does, how far signals on it could be driven and at what speed.

[Danara]

One of the two unpopulated connectors on the Pi is intended for a camera, which might have the bandwidth required to connect to another Pi (at least in one direction anyway). One of the downsides of the SOC is that a lot of the high speed bus connectivity is locked inside the silicon sandwich. The upside is the nice small package and amazing price / performance, of course.

Apart from that, if you are interested in development of parallel computing apps, you might want to look at the parallella board http://www.parallella.org/. The board is brand new, coming out a Raspberry Pi inspired initiative via http://www.kickstarter.com/, (sound familiar?) and should be shipping the first units in a month or so. It is quite similar to the Pi, but with a faster dual core ARM, more memory, and large number of parallel processors (16 or 64). I think the kickstarter backers get a 16 core version for USD$99 and the 64 core will be around twice that price. This make the cost per cpu (or GFLOP) much less than it would be if connected a large number of Raspberry Pi board together to get the same computational ability at a similar power budget.

[VassBent]

Again, thank you all for the time you've all taken to reply. Pls keep in mind that i am a mechanical and not an electrical engineer, and thus I might ask questions that sound a bit intuitive and naive, so pls bear with me.

For the time being, agreeing with DeeJay, the idea of connecting all R-Pis on a switch by using the ethernet jack appears to be the only viable solution, especially if W.H, Heydt is right and the processor used is not designed to support a bus such as the one needed for what i asked about to be implemented. However, it is not exactly the fastest possible implementation of a R-Pi being turned into a parallel system, especially since the ethernet speed is limited to 100MBps.
On the other hand, not only would my idea make the whole system of Pi-s a whole lot faster if there were future versions of Pi-s with new processors that could dock on a platform (which would also have its own processor and RAM as well, in order to play host to the pi-s and assign threads) or platforms able to dock to one another, but it would also rid us of:

1. switches, ethernet cables and other needed auxiliary hardware
2. the knowledge needed to handle and administrate the subnetwork you have just created, using the switch and ethernet jacks.

Of course, since i mentioned it in my initial post, making the whole communication of the Pi-s wireless would be also limiting the communication bandwidth, but the ability to store each Pi seperately from one another might make that sacrifice in performance worthwhile...


All in all, what I was asking was, for future development of the Pi-s, whether the development team would deem that idea worthy of looking into for some of the next versions of Pi, redesigning what part of the hardware would be needed.

[VassBent]

One of the two unpopulated connectors on the Pi is intended for a camera, which might have the bandwidth required to connect to another Pi (at least in one direction anyway). One of the downsides of the SOC is that a lot of the high speed bus connectivity is locked inside the silicon sandwich. The upside is the nice small package and amazing price / performance, of course.

Apart from that, if you are interested in development of parallel computing apps, you might want to look at the parallella board http://www.parallella.org/. The board is brand new, coming out a Raspberry Pi inspired initiative via http://www.kickstarter.com/, (sound familiar?) and should be shipping the first units in a month or so. It is quite similar to the Pi, but with a faster dual core ARM, more memory, and large number of parallel processors (16 or 64). I think the kickstarter backers get a 16 core version for USD$99 and the 64 core will be around twice that price. This make the cost per cpu (or GFLOP) much less than it would be if connected a large number of Raspberry Pi board together to get the same computational ability at a similar power budget.

Thank you for bringing that to my attention, i had no idea about such a platform being launched anytime soon !!! The reason i asked about Pi-s "getting together", so to speak, is that the power consumption is quite low and it makes it probably more worthwhile than simply getting a GPU of 2,000 cores and 6GB of RAM for, say, 800$ and work parallel there, burning a very nice 300Watts in the process. It is not that working parallel can't be done with a GPU and leave the Pi-s alone, it's if it can be done with the greatest power efficiency and smallest amount of money (to run something on a GPU, u also need the rest of the computer such as the CPU, the memory, the HD, the tower etc and that costs a lot).

The only difference between that platform and what i propose is that my idea is completely scalable at will, whereas with the parallela board appears not to be. Regardless, thank you for your answer

[Danara]

I hear you on the power issue. I believe that even with the 64 core unit, the parallella still consumes only 5 watts total. The intention for the parallella core is to be able to target the smartphone market, so power consumption is very high on the criteria list. They hope to release a 1000 core unit next year (in 2014), but I don't think they have hinted at the power consumption yet for that version.

Cheers
Dan

[khulat]

Thank you for bringing that to my attention, i had no idea about such a platform being launched anytime soon !!! The reason i asked about Pi-s "getting together", so to speak, is that the power consumption is quite low and it makes it probably more worthwhile than simply getting a GPU of 2,000 cores and 6GB of RAM for, say, 800$ and work parallel there, burning a very nice 300Watts in the process. It is not that working parallel can't be done with a GPU and leave the Pi-s alone, it's if it can be done with the greatest power efficiency and smallest amount of money (to run something on a GPU, u also need the rest of the computer such as the CPU, the memory, the HD, the tower etc and that costs a lot).

The only difference between that platform and what i propose is that my idea is completely scalable at will, whereas with the parallela board appears not to be. Regardless, thank you for your answer

It is very probable that you will get better power efficiency and more bang for your buck if you just use a normal Computer with a modern GPU. The RasPi can be used to learn about parallel computing, but it's not cost or power effective.

[cleverca22]

in theory, you could MAYBE connect the lcd (output) ribbon connector to the camera (input) ribbon socket between 2 pi's (with proper pin adapting)
and then custom gpu code (something end-users cant do) could abuse that as a single direction high speed bus

you could then connect a whole set of pi's in a ring, and make a custom high speed bus between all of them

another option (also something end-users cant do), would be to abuse the lcd+camera interface, as a high speed bus to add gigabit ethernet (custom board and custom gpu firmware)

thats about the only ways i can see of getting high a high speed bus added to a pi, and both require broadcom wirting gpu firmware to help it

[p_krishnat]

I have made simulink (MATLAB) model to process image.
i have divided image into blocks and i want to process it in parallel.
Plz help in this regards.
any idea how to process them in parallel????

[ejolson]

You should probably start a new thread for this question. Does Matlab/Simulink even run on a Raspberry Pi?

[RaTTuS]

https://twitter.com/9_ties/status/702335927490781184 but start a new thread

[p_krishnat]

You should probably start a new thread for this question. Does Matlab/Simulink even run on a Raspberry Pi?

I thought my question is relavent to this thread.
I'll make new thread.
thanks for reply.

[Kurt in Space]

I set up 5 raspberry Pi's using the ethernet thru a switch to act in parallel. It is possible to use the GPIO lines to make a primative bus, but that would require writing your own operating system to take advantage of it. Good for a doctoral thesis, not good for a hobbiest.

http://landfallnet.blogspot.com/2016/05 ... puter.html

What I found is it's far easier to use the paralleled computers as a distributed computing platform that as a cluster. Writing the software to have all the cpu cores acting on one application is a lot of work. I do it, but you will put a lot of time into it if you are just looking for an increase in speed. Any you can't really share the RAM on each seperate PI.

If you have a lot of tasks that need to be performed at the same time, it works very well and is worth pursuing.

[HawaiianPi]

...The only difference between that platform and what i propose is that my idea is completely scalable at will, whereas with the parallela board appears not to be. Regardless, thank you for your answer

The Parallella has Gigabit Ethernet, so it's much more scale-able than the 10/100 Ethernet of the Pi. It also has more GPIO, so it may be possible for interconnections that way (haven't read the docs).

What you are proposing is really beyond the scope of what the Pi was created for. It's an educational tool for introducing school children to programming. While it can be used to teach the concepts of clustering and parallel computing, it's simply not fast enough for any practical application of such technologies. For that, the Parallella would be a much better choice.

[Heater]

If you want to cluster Pi to get more performance in some complex calculations forget it. The overheads of communicating Pi to Pi will kill any gains.

If you want to build a fail safe, fault tolerant system, then by all means build a cluster of PI and checkout various data bases and such that can do that.

If you want a cheap way to learn about clustering/parallel processing then go ahead. Just don't expect to get winning performance.

[TideMan]

Dividing the problem into blocks and solving each block separately is pseudo-parallelisation. You don't need to communicate between processors during the computation, but pull the results together at the end.
With a RPi3, that has 4 cores, you should be able to simply open Matlab in 4 different windows and run your computation for each block on a separate Matlab. The system will assign each problem to a different core. I've tried it on Windows, but not RPi.