GPU Processing API


120 posts   Page 1 of 5   1, 2, 3, 4, 5
by tech2077 » Sat May 19, 2012 5:17 pm
I've recently recieved my Raspberry Pi, and while it has an alright CPU, it has an amazing GPU. Will there be any APIs similar to OpenCL or CUDA, and if not, how can we use the GPU to our advantage in userspace applications.
Posts: 2
Joined: Sat May 19, 2012 5:14 pm
by Narishma » Sat May 19, 2012 6:42 pm
You can only use the GPU through OpenGL ES or OpenVG.
Posts: 150
Joined: Wed Nov 23, 2011 1:29 pm
by Morgaine » Sat May 19, 2012 10:26 pm
Until people start reverse-engineering the VideoCore. And with millions of Pi in the world, that will certainly happen.
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by jamesh » Sun May 20, 2012 10:52 am
Morgaine wrote:Until people start reverse-engineering the VideoCore. And with millions of Pi in the world, that will certainly happen.


They can try, but they won't succeed.

To the OP - there is no access to the GPU's processors except via the provided libraries. I did spend some time trying to think of a way of giving some very basic access to some vector processing, but it's not an easy task.
Moderator
Moderator
Posts: 10568
Joined: Sat Jul 30, 2011 7:41 pm
by Morgaine » Mon May 21, 2012 1:11 am
jamesh wrote:
Morgaine wrote:Until people start reverse-engineering the VideoCore. And with millions of Pi in the world, that will certainly happen.

They can try, but they won't succeed.

With millions of Pi in the world, precedent shows that the likelihood of it not being reverse engineered and hacked it as close to zero as makes no difference. Even boxes with just a few thousands of users get hacking interest. The Pi's millions (soon) will be irresistible. :D

Remember that the only thing that stopped the PS3 from being hacked (and it's one of the few high profile platforms that hasn't been, for long anyway) was that the hardware community wasn't working with the bare metal, but jailed by Sony's hypervisor which was implemented without any major holes.

That's not the case with Pi's SoC --- the bare metal is available, so the only thing that stops reverse engineering is obscurity. And obscurity is not security. :P

Morgaine.
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by Narishma » Mon May 21, 2012 4:50 am
Hacking a device to allow you to run unsigned code isn't as complex and tedious as reverse-engineering a GPU and writing a driver for it.
Posts: 150
Joined: Wed Nov 23, 2011 1:29 pm
by khulat » Mon May 21, 2012 5:14 am
Also the security of the PS3 is very overrated. Signing your applications with a "random" key is not the pinnacle of cryptography. As far as i remember the PS3 was hacked rather quickly after Sony chose to disallow using Linux on it.
Posts: 104
Joined: Sun Feb 12, 2012 9:43 pm
by rurwin » Mon May 21, 2012 7:30 am
And the fact that Linux could exist on it indicates that the machine code was known, as was the architecture and many facets of the boot process -- SDKs were available for games writers for example. Nothing is known about the GPU except the stream of binary that is the boot loader. It is not impossible for that to be reverse engineered, but it is orders of magnitude more difficult than the PS3 hack.
User avatar
Moderator
Moderator
Posts: 2890
Joined: Mon Jan 09, 2012 3:16 pm
by Morgaine » Mon May 21, 2012 6:58 pm
rurwin wrote:Nothing is known about the GPU except the stream of binary that is the boot loader.

A huge amount is known or knowable about the GPU because the OpenGL ES 2.0 libraries and drivers are all accessible to the root user even for those parts that are not open sourced. Even crypto can be cracked with differential cryptoanalysis when you have the ability to change inputs and monitor outputs, and in the case of the GPU there is nothing as hard as a crypto barrier nor hypervisor protection for the hardware hackers to contend with.

The only thing standing in the way of all the hidden parts of the SoC being reverse-engineered is their obscurity, and that is only a soft barrier. Whether the hardware hacking community will actually embark on this is not known but can probably be assumed just from the sheer number of units and low cost. If they do, there don't appear to be many blockers on the path, other than the obscurity which they so enjoy battling.

Morgaine.
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by tufty » Mon May 21, 2012 8:31 pm
Morgaine wrote:The only thing standing in the way of all the hidden parts of the SoC being reverse-engineered is their obscurity, and that is only a soft barrier. Whether the hardware hacking community will actually embark on this is not known but can probably be assumed just from the sheer number of units and low cost. If they do, there don't appear to be many blockers on the path, other than the obscurity which they so enjoy battling.

I'll bet you a beer and a half-price ski pass nobody manages to actually get the GPU to do anything useful outside of the provided interfaces.
Posts: 1330
Joined: Sun Sep 11, 2011 2:32 pm
by Morgaine » Mon May 21, 2012 9:42 pm
Time will tell. My money is on the hardware crackers. Precedent suggests that betting against them is extremely unwise. :D
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by shirro » Mon May 21, 2012 10:59 pm
Morgaine wrote:Time will tell. My money is on the hardware crackers. Precedent suggests that betting against them is extremely unwise. :D


If it was cracking I would be on your side. But this is a huge reverse engineering effort. We don't know the architecture let alone the instruction set. We would need decompilers and build chains. This is a lot bigger than recovering an encryption key. I am sure partial knowledge will come eventually but probably long after the Pi has ceased to be of interest to most people. Though hopefully in time for Pi retro full hardware emulation in 30 years when $5 multicore computers with terabytes of memory are commonplace.
Posts: 248
Joined: Tue Jan 24, 2012 4:54 am
by Morgaine » Tue May 22, 2012 2:07 am
shirro: With that I certainly agree. :)

And your excellent phrase "long after the Pi has ceased to be of interest" is worth pointing out to Pi fanbois who have a poor grasp of historical context.

The Pi is just a board, and it has earned its place in the history of computing because of its extremely insightful goals and its unprecedented drop of the price of modern computing to $25. But it has lots of problems, and it will be quite retro within a year or two. Being a fanboi is quaint but ultimately stupid. Yes, STUPID. My message to fanbois is "Don't be stupid by being a fanboi". Rejoice in the good things of the Pi, and move forward.

Eben and Liz have both stated that they look forward to Pi clones and similar projects in this area, because the goal is technical education, so the more players that assist education the better. I agree totally, as I have experienced the same issues in academia as Eben. It makes the fanboism that is rampant on the RPF forum particularly annoying. The goal isn't Pi. The goal is technical education. And if better boards than the Pi emerge for education, great, support them. The goal is technical education.

shirro, this has strayed off your point quite a bit, but your final phrase of "in 30 years when $5 multicore computers with terabytes of memory are commonplace" refocuses the issue really well. We're not here to promote a particular board, but to play our part in an extended process of technological empowerment of our youngsters. Current products are just markers along the road.

Morgaine.
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by jamesh » Tue May 22, 2012 2:08 pm
I'm sure the Raspi will cease to be on interest within a couple of years. However, it will be replaced by another cheap small device that will be of interest for a couple of years, then that will be replaced etc.

However, there's nothing to say those subsequent devices won't be using a similar/same GPU! Or a souped up version of the current GPU? Same interface, just faster.
Moderator
Moderator
Posts: 10568
Joined: Sat Jul 30, 2011 7:41 pm
by jamesh » Tue May 22, 2012 2:27 pm
shirro wrote:
Morgaine wrote:Time will tell. My money is on the hardware crackers. Precedent suggests that betting against them is extremely unwise. :D


If it was cracking I would be on your side. But this is a huge reverse engineering effort. We don't know the architecture let alone the instruction set. We would need decompilers and build chains. This is a lot bigger than recovering an encryption key. I am sure partial knowledge will come eventually but probably long after the Pi has ceased to be of interest to most people. Though hopefully in time for Pi retro full hardware emulation in 30 years when $5 multicore computers with terabytes of memory are commonplace.


Cracking the message passing interface would be pretty trivial to a cracker. Writing a new GPU blob would not be. As has been said before, first you have to write a compiler/assembler for the customer instruction set....and it get worse from there. 18k registers to figure out, millions of lines of code to replicate, including some fairly esoteric assembler for the more obscure HW block (did know that the GPU has a number of internal processors, which use different instructions sets?).
If people want to try, good luck to them, but I think there may be better targets to put your skills toward.
Moderator
Moderator
Posts: 10568
Joined: Sat Jul 30, 2011 7:41 pm
by Neil » Tue May 22, 2012 8:00 pm
*tish* *tish* James.... no need to make it any easier for them!
Posts: 88
Joined: Thu Sep 29, 2011 7:10 am
by akhilpo » Thu May 24, 2012 7:31 pm
Why didn't the makers of the pi choose a processor which is more open and well documented? >_<
Posts: 6
Joined: Thu May 24, 2012 7:17 pm
by Narishma » Thu May 24, 2012 9:07 pm
akhilpo wrote:Why didn't the makers of the pi choose a processor which is more open and well documented? >_<

Because I don't think there's such a thing. I can only think of 2 companies that provide documentation for their GPUs (Intel and AMD) and they are both way too expensive for something like a Raspberry Pi.
Posts: 150
Joined: Wed Nov 23, 2011 1:29 pm
by jamesh » Thu May 24, 2012 9:17 pm
akhilpo wrote:Why didn't the makers of the pi choose a processor which is more open and well documented? >_<


Eben Upton, the founder of the Raspberry Pi foundation, works for Broadcom and had a part in designing the chip. This means he was able to get a good price on the part, but also that he has managed to leverage a lot of development work done by Broadcom in order to design the Raspberry PI.

And as said above, you cannot buy a device of this power AND have the documentation for the price paid for the Broadcom part.

I'd go as far to say the Raspberry Pi would not have been possible without this link. Now of course, with manufacturers realising there is a place for devices like this, things may change.
Moderator
Moderator
Posts: 10568
Joined: Sat Jul 30, 2011 7:41 pm
by Morgaine » Thu May 24, 2012 9:27 pm
akhilpo wrote:Why didn't the makers of the pi choose a processor which is more open and well documented? >_<

I seem to recall that this exact question was asked on the blog and managed to get an official answer along the lines of "There are no SoCs of any brand that provide a good multimedia-capable GPU that is also open."

For ARM that does indeed appear to be the case, sadly. AMD does now supply full documentation even for modern GPUs, but it's not relevant to ARM. I'm not altogether sure of the situation with ARM's own MALI-400 GPU, because although ARM's own driver is closed source, there is an effort currently ongoing to write an open source driver for it, so some open documentation may be available.

I agree with the implication that this is less than ideal for education. It's especially bad in that any problems that are found in code that touches the closed hardware cannot be fixed by the open community, nor can improvements be made except under NDAs with a single company. That's bad on multiple grounds.


Morgaine.
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by Narishma » Thu May 24, 2012 9:42 pm
Morgaine wrote:I'm not altogether sure of the situation with ARM's own MALI-400 GPU, because although ARM's own driver is closed source, there is an effort currently ongoing to write an open source driver for it, so some open documentation may be available.

If you are talking about the lima effort, it's being done by reverse-engineering the ARM closed source driver. They only started a few months back, so there won't be anything useful for a while. There's also a similar effort for Qualcomm's Adreno GPU but that one just started a few weeks back.
Posts: 150
Joined: Wed Nov 23, 2011 1:29 pm
by AlArenal » Thu May 24, 2012 10:19 pm
Morgaine wrote:I agree with the implication that this is less than ideal for education. It's especially bad in that any problems that are found in code that touches the closed hardware cannot be fixed by the open community, nor can improvements be made except under NDAs with a single company. That's bad on multiple grounds.


Even more less ideal for education would have been not having to be able to build a pi and sell it for $25/$35 because your hardware costs a lot more and you don't have free of cost support by chip vendor employees.

There is nothing like an ideal platform. The reasons start with the fact, that there is not one single definition of "ideal" in this context.

Different hardware and we wouldn't have this discussion out of numerous reasons...
Posts: 141
Joined: Sun Feb 26, 2012 6:58 pm
Location: Germany
by Morgaine » Thu May 24, 2012 10:39 pm
AlArenal wrote:Even more less ideal for education would have been not having to be able to build a pi and sell it for $25/$35 because your hardware costs a lot more and you don't have free of cost support by chip vendor employees.

That certainly was a valid reason back in the days when the Foundation was self-funding the Pi. Without this help, perhaps the Pi might not have launched at all.

But notice that it is no longer a valid reason now. With the industrial muscle of RS and Premier Farnell behind production, and manufacturing volumes ramping up massively and heading into the dozens or hundreds of millions of units for education, the extra help that Broadcom gave one of its own developers is no longer needed.

The Pi is a regular commercial product now, and undoubtedly Broadcom is looking forward to the huge business benefits of sole dominion in a very high profile market niche and, if governments get on board for IT education, guaranteed volume sales.


Morgaine.
Intolerance is a failure of education. Education is predicated on tolerance of the uneducated.
User avatar
Posts: 141
Joined: Mon Mar 12, 2012 1:13 am
by mahjongg » Fri May 25, 2012 12:06 am
That factor has no real world implications for Broadcom.
As long as they gamble their company on such a choice you cannot reasonably expect them to go that road. But then miracles have happened before.

I think this is just a severe case of wishful thinking on the part of people who hope this will happen.

The only thing we can reasonable expect is for Broadcom to put some of their own engineers on the more reasonable requests for specific GPU software, but its not you can put some third party engineers on it, let them sign non disclosure forms, and expect something out of that soon. Experts who know something about programming this GPU are all already working for Broadcom now, or else for their competition.
User avatar
Moderator
Moderator
Posts: 4506
Joined: Sun Mar 11, 2012 12:19 am
by shirro » Fri May 25, 2012 12:22 am
I think it is great that teams are reverse engineering the other ARM SoC GPUs and I hope an effort starts for the Videocore. But there is a huge difference between the functionality of a driver produced from partially reverse engineering a product like this and one that has been designed by full time engineers with access to all the docs. Any reverse engineering project is almost certainly going to be totally irrelevant to education users of the Pi during its lifetime and probably not much use to anyone else either. Still people don't climb mountains for practical benefits.

Like running windows or flash or any of the other strange things that people request of the Pi the correct answer is possible - yes , practical - no. The bulk of users would be better off understanding the limitations of the Pi and adapting to the reality or finding an alternative product.

Now if someone at Broadcom snuck out a small skunkware vector processing lib for the videocore...
Posts: 248
Joined: Tue Jan 24, 2012 4:54 am