DSP Access

[timothybean]

Sorry if this post is redundant. People on this forum get in very deep conversation and I am getting lost in. My basic question:

Will there be access to the DSP to do video encoding/decoding? I understand there may be some licencing issues with MPEG-2/MPEG4/ etc. Just wondering if I can use it for this or if that is a feature that we are not able to use? As well, would this require some sort of SDK/Licensing/NDA with Broadcom to do any of this?

I really think this is a great project and can't wait to see all the good things that will come from it.

[liz]

Not initially, no - although it looks like now we will definitely have both MPEG2 and MPEG4 along with h.264, so I hope that'll save you any problems you might have been anticipating.

[timothybean]

Liz,

That sounds great. I am wondering, would those be hardware accelerated or just decoded on the ARM side? I don't really care about writing DSP code, but would like for decoding to automatically be offloaded to the DSPs.

Thanks!

[obarthelemy]

I'm in a bit of a haze... the way I understand it, the Pi's chip has at least 3 distinct processing units: CPU (ARMv6), FP (VFPv2), GPU (Videocore 4). Does it have a DSP unit too, and why use that for encoding or decoding H.264, which the Videocore is supposed to do ? I understand non-H.264 is not supported by the GPU, but H.264 is, and can be both encoded and decoded on the GPU ? Does "having MPEG 2 & 4 codecs" means that can get offloaded to the GPU ? both encode and decode ?

[jamesh]

Well, I think people are using the word DSP and GPU interchangeably, although its not strictly correct.

There are quite a number of different cores on the chip, the Arm and its floating point unit, and all those in the GPU itself (and you need both hands and some feet to count all of those)

The GPU is best for encoding and decoding codec tasks, and has a number of cores dedicated to just that for the formats you mentioned.

[nufan]

jamesh/anyone else

If the PI will support mpeg2 decoding, and (in another thread) you mention that it has plenty of juice to encode as well, would it be possible to transcode mpeg2->h264 in near-realtime?

the IO would likely be small: a typical 1080 US broadcast signal is only about 2mbyte/sec, and h264 should be somewhat smaller.

Are there any docs or explanations about how the user interacts with the video decoding/encoding features of the system? eg what programs know how to play video? (xine/mplayer/vlc/etc).. what plugins are used?

How is video encoding accomplished?

thanks!

[jamesh]

You might be able to transcode, but not I think at 1080p - that requires too much oomph. 720 might be possible, SD more likely. The problem isn't the bitrate but the size of the image.

No docs as yet, we are still working to determine what we can get out in time for launch and what will have to come later. We will be supplying our own player at first, until some of the kinks are ironed out of the drivers and connection.

Video encoding currently works directly from an on board camera. Right now, I don't think we can encode any arbitrary video stream from the Arm, although that will come. At launch it's playback that will definitely be available.