Parallella: A Supercomputer For Everyone

[jamesh]

I think if it's their forum they are in their rights to say you're starting to get on my nerves.

"their forum". AIUI the person whose nerves are being rattled works for Broadcom and is quite insistent that Broadcom and the Foundation are not the same.

Would you care to explain what you mean by rattled? Surely you mean 'annoyed that someone comes on the Raspi forum (which you help moderate as a volunteer) slagging off the device and advertising a different one that hasn't yet been built as an alternative'? That's not being rattled. It also has nothing to do with working at Broadcom or not (and your implication that the Broadcom and the Foundation are the same is rather insulting - they clearly are not).

Now, the Beaglebone black - now that is a product that exists, and is much closer to home. Not this device (which is a great idea, but no Raspi replacement).

[drgeoff]

I think if it's their forum they are in their rights to say you're starting to get on my nerves.

"their forum". AIUI the person whose nerves are being rattled works for Broadcom and is quite insistent that Broadcom and the Foundation are not the same.

Would you care to explain what you mean by rattled? Surely you mean 'annoyed that someone comes on the Raspi forum (which you help moderate as a volunteer) slagging off the device and advertising a different one that hasn't yet been built as an alternative'? That's not being rattled. It also has nothing to do with working at Broadcom or not (and your implication that the Broadcom and the Foundation are the same is rather insulting - they clearly are not).

Mixed metaphor. I was thinking of "rattlling someone's cage". Though yours does seem to more easily rattled than most, it wasn't my intention to insult or upset you or anyone else, so I have no problem to apologise if I did.

I never implied that Broadcom and the Foundation are the same. Quite the opposite. I was pointing out in a whimsical manner (you did see the smiley at the end?) that the "their" in "their forum" wasn't really applicable because jamesh works for Broadcom (and Broadcom and the Foundation are not the same).

[Ravenous]

Now That's What I Call Backpedalling... oh well.

I'm sure you didn't mean it to sound that bad.

[pygmy_giant]

Am I right in thinking that parallel processors come into their own when performing operations on data that are not dependant on being done in any particular order?

I visualise them as a swarm of munchkins attacking a problem en-masse.

Can you instruct each munchkin to work on a seperate designated area of memory?

Can you nominate a head munchkin to coordinate the many so that each reports back to him on completing their little task to be allocated the next job off the coordinator's pile?

Is a workplace metaphor with a manager and body of staff helpful for understanding paralell processing?

[feverish]

Like ants swarming, aren't super-dooper 'cheap' all-singing computers on dongles now emerging from the woodwork, Rikomagic's MK802IV being but one example? They seem to be aimed more at the frenetic-excitement geeky mob than what strikes me as the powerful ambition to raise the technical literacy of children propounded by Raspi. Please, Raspi, don't be persuaded by all the hype to take your eyes off that laudable objective.

[Ravenous]

Is a workplace metaphor with a manager and body of staff helpful for understanding paralell processing?

Good example (and a bad one at the same time).

If you have enough workers, the manager would be fully occupied, and get hot under the collar.

If a year later you decide you can have twice as many workers (and you will) then the system will collapse, because the manager can't keep up. All of the workers will be queueing for more work. So that particular system is not scalable.

There are probably better arrangements, but I wouldn't know in detail. We need an expert to tell us, as one or two people have already said

[morphy_richards]

is it a case of you can make it up as you go along?

So, like a swarm of homogeneous intelligent ants but working in a socialist boss-less utopia?

[pygmy_giant]

Ravenous said:

If you have enough workers, the manager would be fully occupied, and get hot under the collar.

I thought the 2x big CPUS on the Paralella could be the bossess and the many comparatively feeble cores could be the workers. As they are bigger and faster the bosses could therefore keep up with the workers who due to their numbers are in combination more powerful than the bosses?

[Average Mailman]

Very cool project but, I'll stick with my RPi. It's cheaper and offers everything I need to have an excellent modding experience.

[Ravenous]

I thought the big CPUs were in fact the cores themselves... I doubt they would hard wire it to one topology.

There are a few examples on the kickstarter page, showing examples you would use it for. (Fast Fourier Transforms and Matrix calculations.) Clue: it's not easy, it's a step or two above Scratch.

[pygmy_giant]

no... you get a dual core ARM 9 processor and plus the 64 cores on the Epiphany chip (http://www.parallella.org/board/)

[jamesh]

I'd suggest that for those who are interested in parallel processing, its worth looking up the definitions of SIMD (single instruction multiple datastream - used by Raspi GPU/VPU ), MIMD (multiple instruction multiple datastream). These are a couple of ways of parallel processing. They might help explain some basics (I'm no expert on this stuff!). It's a very interesting topic.

If you really interested in learning and writing stuff then its possible to simulate 'pure' parallel processing on something like the Raspi (or any single core unit). Obviously instead of things happening in parallel the single CPU would have to share its load of virtual parallel nodes, so it runs much more slowly. But still possible to learn. Would make an interesting project to write a parallel language (like Occam), but I'm sure there are simulators out there already to play with.

[jamesh]

I'd suggest that for those who are interested in parallel processing, its worth looking up the definitions of SIMD (single instruction multiple datastream - used by Raspi GPU/VPU ), MIMD (multiple instruction multiple datastream). These are a couple of ways of parallel processing. They might help explain some basics (I'm no expert on this stuff!). It's a very interesting topic.

If you really interested in learning and writing stuff then its possible to simulate 'pure' parallel processing on something like the Raspi (or any single core unit). Obviously instead of things happening in parallel the single CPU would have to share its load of virtual parallel nodes, so it runs much more slowly. But still possible to learn. Would make an interesting project to write a parallel language (like Occam), but I'm sure there are simulators out there already to play with.

ref. Occam - http://www.cs.kent.ac.uk/projects/ofa/kroc/
http://www.raspberrypi.org/phpBB3/viewtopic.php?t=24795

[morphy_richards]

Can't you use that for (among other things) solving complex problems? As long as it's finite it travels down paths and forks until it reaches the solution?

[rurwin]

If you have spare processors, then yes. In all other cases it is more efficient to prune the decision tree so you do not do processing that you do not need to do.

The popular example would be a game of chess. It is trivially easy to always win a game of chess by finding all possible moves, and then all possible opponent moves and so on, then find the path through that decision tree where you cannot lose. Unfortunately there is not enough space in the universe for the processors you would need in order to do it in parallel, and not enough time left before the sun blows up to do it sequentially. You have to prune the tree to something more manageable, and that is where the clever tricks come in.

[morphy_richards]

I have an idea for a problem...
Designing school timetables is notoriously difficult. Matching individuals to groups to classes to lessons to teachers.
Lessons to numbers of periods (1 period, doubles, triples, one and a double, three singles etc...)
Available teachers to suitable classrooms.
Available periods to teachers....

Will think about defining the problem some more but afaik there is no current user friendly solution.
We have in the past paid quite a lot of money to external consultants to do it and their solution was pants.

[aTao]

I have an idea for a problem...
Designing school timetables is notoriously difficult. Matching individuals to groups to classes to lessons to teachers.
Lessons to numbers of periods (1 period, doubles, triples, one and a double, three singles etc...)
Available teachers to suitable classrooms.
Available periods to teachers....

Will think about defining the problem some more but afaik there is no current user friendly solution.
We have in the past paid quite a lot of money to external consultants to do it and their solution was pants.

There are many problems that do not easily fit into digital processing, real life being one of them. Parallel processing has been around quite long enough to show that its a dead end.It might seem to offer powerful solutions but any gain is lost in set up overheads.

The real problem of processing development, as I see it, is the CPU(s)/ storage distinction. The next breakthrough will not be faster or bigger but more distributed, processing spread around in and as part of storage.

For the timetable problem hen consider "The Timetable" as a central goal, the restraints are arranged around the goal and interact with each other striving to reach the goal.

[morphy_richards]

Parallel processing has been around quite long enough to show that its a dead end.It might seem to offer powerful solutions but any gain is lost in set up overheads..

With all due respect that sounds a bit like saying "Nobody will ever need more than 640k RAM"

[Bakul Shah]

Designing school timetables is notoriously difficult.

Timetable construction is an NP-complete problem. See Jeff Kingston's paper on timetable construction complexity. Available on his home page @ Uni of Sydney. It also points to some free software for High school timetabling. A good survey o timetabling is here (in PDF). Google scholar points to lots of interesting papers including some using parallel algorithms!

[jamesh]

In a related way, I always find it amusing that many people are always wanting more cores in their smartphones/tablets/desktops etc. When in fact, you would get more done by just having a single core going really fast. It would also use less power IIRC. More cores is a form of parallelism of course, MIMD.

[Ravenous]

I always find it amusing that many people are always wanting more cores in their smartphones/tablets/desktops etc.

How disappointed I was when I started getting two then four core PCs, and realised that they were no faster (for almost all appllications) than before.

(Before anyone gets angry, I'm not saying parallelism fails - I'm saying it works if your application can use it. There are applications everywhere of course, but the average person has no idea what they are.)

[mikerr]

Yes you lose alot of time in the initial setup (for GPGPU style parallelism), so it needs to be a large dataset/long running time to benefit,
and more focused since those GPU "processors" are only fast and able at a restricted use ( no branching, restricted data)

For multi CPU/multi thread style parallelism (2-4 full cores) the situation is different, and more a matter of programmers just not using multithreading much.

AIUI the individual epiphany/parallela CPUs are much more capable than your average GPU stream processor, that's what makes it more interesting.

[lavers]

How disappointed I was when I started getting two then four core PCs, and realised that they were no faster (for almost all appllications) than before.

This is true if you're hoping to get more performance out of a single application. For me, the greatest benefit of multicore has been that I can do something else while one application is monopolizing a core.

[pygmy_giant]

I would have thought that multiple cores would be perfct for multi-tasking OSes?

[Ravenous]

I would have thought that multiple cores would be perfct for multi-tasking OSes?

Correct, what I meant was, my first multi core PCs were no faster when running computer graphics applications like POV Ray, or photoshop. HOWEVER as lavers said, you can still use the PC for something else while one core is hammering flat out on that application.

The "problem" I was talking about is when you want to run just one program, fast. Of course if the program can be rewritten to use all the available cores it'll fly. Or if you can run two copies of the same program working on multiple images (POV ray can do this, if I remember correctly...)