Libraries, codecs, OSS

Here’s a post from JamesH, one of the team working on the Raspberry Pi’s software. He works for Broadcom as his day job, and knows the BCM2835 as well as anybody; you will also have seen him posting in our forums and spanking trolls in our blog comments. Thanks JamesH! 

There have been quite a few questions in the forums and on the comments about what libraries will be available, what codecs, what is open source etc. This short post will try and give people some idea of what will be available at or around launch time. It won’t be comprehensive – I am sure that for some it will generate more questions than answers, but I hope it will be of help.

Firstly, libraries. Any distribution will need to supply a set of closed source libraries that give access to the GPU acceleration features. The libraries that will be available are :

  • OpenGL ES 2.0
    OpenGL is a 3D library, very commonly used on desktops and embedded systems. It is defined by the Khronos Group.
  • OpenVG
    OpenVG is a 2D vector drawing library, also commonly used on desktops and embedded systems. Again, defined by the Khronos Group.
  • EGL
    EGL is an interface between Khronos rendering APIs such as OpenGL ES or OpenVG and the underlying native platform window system.
  • OpenMAX IL
    OpenMAX supplies a set of API’s that provides abstractions for routines used during audio, video, and still images processing. OpenMAX defines three layers, this is the IL layer, which provides an interface between media framework such as Gstreamer and a set of multimedia components (such as codecs).

The first three adhere to the standard Linux library API’s, so should be a straight forward swap in for applications that use them. OpenMAX IL does not have a standard API at this stage, so is a custom implementation. All these libraries are as supplied by Broadcom, the SoC (System On Chip)  provider.

There is loads of information on Wikipedia and the Khronos website on these API’s.

Codecs

Two licensed codecs will be provided at launch, MPEG4 and h.264.  Codec licences have quite an impact of the cost of the device which is why there are only two at this stage. There are non-licensed Codecs such at MPEG2, VC1 etc, but for the moment they will not be accelerated by the GPU.

Dom adds: As an aside, the GPU can hardware decode H264, MPEG1/2/4, VC1, AVS, MJPG at 1080p30. It can software (but still vector accelerated) decode VP6, VP7, VP8, RV, Theora, WMV9 at DVD resolutions. We are restricted due to licensing what we can support. We should be able to support VP8, MJPG and Theora, as I believe they are license free.

Open vs Closed Source

The Open/Closed source debate can become quite heated, as those perusing the comments and forums may have noticed. As stated above, the host side libraries for the graphics acceleration are closed source and are provided by the SoC supplier. The Foundation has no control over the closed nature of these libraries. Since the vast majority of people simply use libraries such as these, it was deeded a trade off worth making to get the high graphics performance.  It’s worth noting there are no other SoC devices with a similar graphics performance that are open source. There is no GPL issue here, these are user side libraries not linked in any way to the kernel.

There are a few drivers for the SoC which are linked in to the kernel, these are GPLed and hence OSS. One of these drivers is the interface from the user space libraries to the GPU. The user side libraries use this ‘driver’ to communicate with the GPU and tell it what to do.

Here’s a handy diagram that may help visualise what’s what.

RaspberryArch

What's OSS, and what's not?

Finally, for those of you wondering why we do not sell the device as a kit, one reason can be seen in the picture below. The picture shows, on the left, the PoP memory and it’s BGA array. This fits on top of the chip on the right, the BRCM2835 SoC (underside showing). As you might imagine, it’s very difficult to position and then manually solder these small SoC devices. Each of those dots needs its own individual dot of solder, and the two packages have to be lined up perfectly. Some people in our forums say they have tried to solder BGA components and succeeded. We know of many, many more who have failed; and there is no recovering once you have failed. Usually, it’s robots called Pick and Place machines that do all the positioning, after the PCB has had solder paste applied to all the pads which helps the parts stick to it. The board then goes off to a reflow oven where the solder is melted, completing the electrical connection.

brcm2835 plus memory

The BRCM2835 (right) and its accompanying memory, on JamesH's dainty fingertip. Click to enlarge.

So there you have it, a very quick introduction to the libraries, codecs, and even a quick explanation of how the PCB’s are populated!