Bare Metal VideoCore

[hermanhermitage]

Hey all. On one of the threads someone was asking for bootcode.bin source - the thread appears to be locked. Try going to http://hermanhermitage.github.com/video ... s/dis.html for a little videocore dissembler toy. Use at your own risk

In terms of progress and accuracy, we have lots of the scalar done (including floating point), and on paper have lots of the 8/16bit integer SIMD decoded as well (called vector around these parts) - but I need some more verification before I commit to the arch definition. We havent tackled the dual core nature of the VPU yet, nor have any understanding of the chipset registers or the 3d pipeline such as the QPUs.

We have the reset bootrom dumped and will push a program to spit a hex dump of it out the Mini UART on the weekend - (we wont publish the said bootrom as its not ours - but we can point you in the right direction). We have some monitor tools that may or may not make it soon. We have no video, usb or ethernet, so all our work is via the Mini UART.

We are making excellent headway and looking to launch the Raspberry Pi internals / Project Neckbeard site soon. Please PM me if you are interested in getting involved. I cant seem to send PMs yet, so please include your email details.

We want to reach out to all the bare metallers in here to gauge the level of interest into going deeper on the VideoCore. I wanted to get in touch with all the usual suspects - DexOS, Cycl0ne, valtonia, tufty, romell, etc.

I'm particularly keen to hear from those interested in helping out with documenting, tooling and also working out a way of adding raw VideoCore access to the existing linux infrastructure. This would be some low hanging fruit to let us start exploiting more of the silicon on the die.

Assuming we open up the instruction set enough, I hope Broadcom will see the advantage of adding, documenting or exposing some hooks that let us hit the silicon via the mailbox.

We are just at the beginning of the journey. The scalar instruction set and vector instruction set are actually just the thin end of the wedge. The real work is in understanding the hardware registers.

Its all about the journey! We are driven to understand everything down to the nuts and bolts. Out intent is to expose as much functionality as we can for those that love to know how things work, and those that want to get the most from their RaspberryPi.

-- HH & everyone at #raspberrypi-internals on freenode.

[Cycl0ne]

Hi,

nice. But I wouldnt go so far by disassm the core. It would be enough if one would disasm the following file "libbcm_host.so" where all the commands are sitting for talking to the mailbox, this documented, would give us a real start.

Cheers.

[valtonia]

Also, there is talk of a new firmware release, the one that was used for the Android 4 port, which gives more access to the videocore - particularly the accelerated graphics.

There's a story on the main Rpi page about it, and this is where they say they're hoping to release the firmware for general use.

[gsh]

Let's just hope the IP company who's IP you are trying to reverse engineer don't send out the attack dogs...

Also lets hope they don't blame the foundation for this

[romell]

Hi there,

This seems like a fun and pretty ambitious project, but I'm still a bit sceptical for a couple of reasons:

* First of all, the last thing we want is for Broadcom to get cold feet and stop putting out firmware updates. The closed source nature of the firmware isn't really as bad as I thought when I started working with this device (as long as you don't want to do any kind of accelerated video ofc, in that case, it sucks). The developers has also been very helpful in adding requested features to the firmware. Of course I'm also very curious to see how the VideoCore works internally, but in this case it might be better to leave it alone...

* Is it really possible? I mean we're dealing with a fairly large number of unknowns here. As you said, you are only scratching the surface at this point. Digging deeper would take some quite substantial effort I think.

Now i might sound very pessimistic about the whole idea - I'm not! If you are having fun, go for it, but please think twice before publishing any internals. Both for your own sake and for the sake of the Raspberry Pi community.

Good luck (I think)

[DexOS]

* First of all, the last thing we want is for Broadcom to get cold feet and stop putting out firmware updates. The closed source nature of the firmware isn't really as bad as I thought when I started working with this device (as long as you don't want to do any kind of accelerated video ofc, in that case, it sucks). The developers has also been very helpful in adding requested features to the firmware. Of course I'm also very curious to see how the VideoCore works internally, but in this case it might be better to leave it alone...

I 100% agree with this bit that romell wrote.
Yes its possible to reverse engineered anything, give time and knowledge.
But whether you do it, is down to two things.
1. Do i need that info, which in this case i do not.
2. Does Broadcom given me enough info to do my job and the answer is yes.

So good luck, but count me out.

[Cycl0ne]

I 100% agree with this bit that romell wrote.
Yes its possible to reverse engineered anything, give time and knowledge.
But whether you do it, is down to two things.
1. Do i need that info, which in this case i do not.
2. Does Broadcom given me enough info to do my job and the answer is yes.

So good luck, but count me out.

Yeah, i see this too. Have you ever poked into a nvidia driver or an ati driver? its too complicated to be of efford. and this VC baby is even more complicated because it has alot of other features: USB, Sound, MMC. all integrated into it.

As i said in my first post it would be cool if they published the mailbox data. thats more than enough. because think of all those demo coders and baremetal ppl booting in their graphic/sound/whatever program without going into linux, like in the old days of computer, with bootloader games/demos/intros/....
with the mailbox data you can do everything you like from: Sound, x264 Video, 3d, graphics,.... and it should be fast enough for everyone.

[hermanhermitage]

Let's just hope the IP company who's IP you are trying to reverse engineer don't send out the attack dogs...

Also lets hope they don't blame the foundation for this

Ok, not the response I was hoping for - we are keen to support the RasPi community and the foundation.

The ability to disassemble the bootcode.bin, loader.bin has been disabled (and the bootrom dumper release is on hold).

Let's take this discussion out-of-band and find a solution.

[hermanhermitage]

...nor have any understanding of the chipset registers or the 3d pipeline such as the QPUs.

The QPU instruction format has now been decoded. This means we have the majority of instruction formats handled (the VPU vector unit is still only partially done). The process we used is pretty much covered here: http://www.youtube.com/watch?v=46jO96bz_Fo.

[jackokring]

I do not think it worthwhile at the moment to bare metal on such a VLSI. To further the educational goals a simpler interface to 3D, 2D and console 1D and audio all under a vertex media roof. In twenty years when the users have grown up more, and have to remould the driver layer to the most efficient command language in use, this may become a worthwhile pass time.

In some countries there is no crime in reverse engineering for research purposes. But then something inventively novel must be remoulded with it, to become more state of the art before any commercial interest lost can be considered not due to copying.

Cheers Jacko

[tufty]

It's exactly as 'worthwhile' a passtime as herman and the rest consider it to be. Realistically, the fact a team of amateurs can get this far points out the futility of trying to keep such stuff 'soupar sekrit eye pee' - broadcom's competitors, if they consider it worthwhile for their purposes, will already have done all this and more.

As for utility, I can personally think of more than a few uses for it, right here, right now (a long way from broadcom's goals for the soc, but close to the foundation's goals for the pi). Herman knows that, of course :)

Oh, and the obligatory:

QPU decoded? Bravo, even if I'm the only one clapping.

[Cycl0ne]

From me too. GZ Hermanhermitage

[dwelch67]

I am impressed with the work that has been done by HH and friends.

Independent of what you use it for I found the instruction set to be interesting, variable word length with many ways to encode the same operation depending on the size of the operands. I think it is definitely something educational, esp for folks interested in bare metal. Far more interesting than x86.

[jamesh]

I'm rather intrigued in how they managed to reverse engineer the QPU instruction set. As far as I can tell, that would be bordering on the impossible without help from somewhere...for example, how do you know where in the blob there is QPU code to analyse...

[hermanhermitage]

I'm rather intrigued in how they managed to reverse engineer the QPU instruction set. As far as I can tell, that would be bordering on the impossible without help from somewhere...for example, how do you know where in the blob there is QPU code to analyse...

The claim reads: "The QPU instruction format has now been decoded". So we know the structure but not necessarily the substance of the magic.

Only publicly available information/data/binaries are used - and typically on a RPi itself - its more fun this way (well there is another reason...)! In terms of knowing where to look there are many options ranging from active (elicit the platform to generate QPU code, or throw bit patterns at the hardware and see what happens) to passive blob analysis.

(Edit: forgot the obligatory link giving away one of our secrets: http://www.youtube.com/watch?v=y7DClAaCssM).


Still a long way from running custom VPU & QPU work loads from user space. But its an exciting possibility to think one day RPi users might be able to leverage all the Raspberry Magic in the SoC.

[jamesh]

I'm rather intrigued in how they managed to reverse engineer the QPU instruction set. As far as I can tell, that would be bordering on the impossible without help from somewhere...for example, how do you know where in the blob there is QPU code to analyse...

The claim reads: "The QPU instruction format has now been decoded". So we know the structure but not necessarily the substance of the magic.

Only publicly available information/data/binaries are used - and typically on a RPi itself - its more fun this way (well there is another reason...)! In terms of knowing where to look there are many options ranging from active (elicit the platform to generate QPU code, or throw bit patterns at the hardware and see what happens) to passive blob analysis.

(Edit: forgot the obligatory link giving away one of our secrets: http://www.youtube.com/watch?v=y7DClAaCssM).


Still a long way from running custom VPU & QPU work loads from user space. But its an exciting possibility to think one day RPi users might be able to leverage all the Raspberry Magic in the SoC.

Ah OK, that sounds more feasible - I think you can get some of that from the available patent documentation!

Still think its going to be a long process i.e. years.

[rew]

When sufficient interest is generated within a community, the community might be able to provide those man-years in months or even weeks....

[jamesh]

I meant years with a big team! After all, that what is takes to write software on it now, with a group of experienced people with all the documentation!

There are over 100 people working full time on Videocore HW and code at Broadcom.

[Burngate]

In mathematical terms, a hard problem has three separate parts - proving a solution exists, proving that a particular solution is correct, and finding the solution.
Elsewhere, Einstein took years finding relativity, but months to convince people he was right, and it's a week's job teaching it to someone.

So it takes many man-years for a team to write the software, but it'd take fewer man-years to work out what they did.

[rurwin]

The interesting stuff here is not controlling the screen, USB, sound, etc. etc., which is going to be very difficult. The interesting stuff is to be able to write GPU code for vector processing and so forth. That should be possible so long as enough of the interfacing is understood to get the data in and out.

IP-wise, I don't think Broadcom have any legal footing to complain. The GPU code is only protected by trade-secret, and that relies on contract. Nobody has signed any contracts or accepted any licenses other than the GPL, so they are not bound. Whether Broadcom like it or not is, of course, a different matter, and it is important to the foundation's goals to keep Broadcom sweet.

[jamesh]

And then what? You've worked out what it does and how it does it. What do you do next? Rewrite it better? Add features?

I think that's my main comment on this - why do people think they will be able to do any better than the experts on the Videocore? Adding features that the experts don't have time for would be one area, but you cannot add features to the current binary without having the original source and knowing how to integrate stuff because of the way it all bolts together.

So at the end of the day, you have the ability to write machine code (unless you also write an assembler), but no way to run it on the Videocore (unless you write your own bootloader and comms stack) and its not going to work with the Foundation supplied binary. I just think people are wasting their time - not in a bad way (it's a great intellectual exercise), but there are much more useful things they could be doing!

[jamesh]

Crossed posts with Rurwin, but points remain.

Also, Broadcom don't seem particularly worried at this stage, probably because they know how complicated it all is!

People do need to be aware of DMCA if they are in the State's though

[DexOS]

So at the end of the day, you have the ability to write machine code (unless you also write an assembler),

There's no need to write a assembler, they could just write a add on to fasm like they did for fasmarm, as its designed to be module.

[hermanhermitage]

The interesting stuff is to be able to write GPU code for vector processing and so forth. That should be possible so long as enough of the interfacing is understood to get the data in and out.

+1 You have perfect clarity on the plan!

[jamesh]

So at the end of the day, you have the ability to write machine code (unless you also write an assembler),

There's no need to write a assembler, they could just write a add on to fasm like they did for fasmarm, as its designed to be module.

Just out of interest can it cope with mixed vector/scaler code? I'm thinking it probably can.