CampGareth
Posts: 6
Joined: Sat Aug 06, 2011 12:39 am

Re: Hardware accelerated audio/video en/decoding

Mon Aug 29, 2011 6:52 pm

I'd like to know whether there's any hardware acceleration of audio encoding or decoding, as far as I'm aware there is AAC decoding to match the H.264 codec nicely but I don't know of anything else.

See, I run a Subsonic server which is a combination of audio transcoding and a web server to provide a music streaming service but with all your own music. It currently runs off an Atom dual core and I'd like to switch it to a RasPi but I'm concerned about the speed of decoding FLAC, OGG or MP3 (not to mention H.264 and xvid) and then re-encoding them in MP3 or... uh, H.263, VP6, and H.264. The decoder tends to be ffmpeg, encoder can be ffmpeg or lame.

So, what does the RasPi have hardware acceleration for?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Mon Aug 29, 2011 7:10 pm

I'll have to get back to you with the list on that one....although the GPU can do lots of stuff, I'm not sure how much is allowed out in to kernel land - there are often licence issues with certain codecs.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

CampGareth
Posts: 6
Joined: Sat Aug 06, 2011 12:39 am

Re: Hardware accelerated audio/video en/decoding

Mon Aug 29, 2011 7:17 pm

Well, I wouldn't mind having to use custom software for the job so long as it's not a real step backwards in quality or features (constant quality when encoding H.264 is fantastic) and can be controlled via the command line. If all of this encoding and decoding ends up dumped on the CPU then subsonic will be... interesting let's say :P

frafra
Posts: 19
Joined: Sun Aug 28, 2011 2:20 pm
Contact: Website

Re: Hardware accelerated audio/video en/decoding

Tue Aug 30, 2011 9:29 am

It would be usefull. What kind of license issues? Just support vp8, theora, vorbis and flac :D

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Tue Aug 30, 2011 9:19 pm

Some codecs have licencing issues, I think they are usually covered in the cost ofht eSoC though, but I really don;t know much about that.

Not able to find out the codec list today - I know what the SoC can support, but I don't know what is exposed to Linux, which is not necessarily the same.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

tan
Posts: 1
Joined: Sat Sep 03, 2011 12:09 pm

Re: Hardware accelerated audio/video en/decoding

Sat Sep 03, 2011 2:09 pm

I have a dream... : )
Let the Raspberry Pi have GPU codec support for VP8 and Ogg (Theora/Vorbis) with driver and codec framework support all the way to at least one browser like chromium or firefox.
In other words, let me able to start my browser and view 720p or 1080p content without worries to violate any patents.
Will this dream come true?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Sat Sep 03, 2011 3:50 pm

It supports VP8 on the GPU - not sure what gets exported to linux though. Not sure about Ogg - doesn't ring any bells.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

hsalonen
Posts: 19
Joined: Wed Aug 31, 2011 9:11 am
Contact: Website

Re: Hardware accelerated audio/video en/decoding

Sat Sep 03, 2011 4:06 pm

Ogg is what Spofity uses.

CampGareth
Posts: 6
Joined: Sat Aug 06, 2011 12:39 am

Re: Hardware accelerated audio/video en/decoding

Sat Sep 03, 2011 4:39 pm

Quote from hsalonen on September 3, 2011, 17:06
Ogg is what Spofity uses.

I'd hope Jamesh only meant that he hadn't heard it mentioned with regards the GPU (i.e. "I don't know any facts atm") as opposed to not having heard of ogg at all (though vorbis audio and theora video are the codec names, ogg is just a container).

Scribe
Posts: 98
Joined: Wed Sep 28, 2011 12:00 pm
Contact: Website

Re: Hardware accelerated audio/video en/decoding

Fri Oct 14, 2011 9:14 am

Quote from jamesh on September 3, 2011, 16:50
It supports VP8 on the GPU - not sure what gets exported to linux though. Not sure about Ogg - doesn't ring any bells.

Theroa is the video format and contained by OGG. It's licence free and was weighed up at one point as 'the' HTML5 codec but now the spec is much more open.

I think this is an important one to support as it can be used freely and has efficiencies similar to H.264 which are ideal if the Pi's going to have a chance at accelerated encoding.

The library is libtheroa http://theora.org/ which has ARM optimisations, I'm not sure about OpenMAX support but I know a lot of framework was put in place for the OMAP3's DSP.

Technically this codec is a superset of VP3 and you can repackage VP3 to Theora.

For me personally this is a big one as the integrated browser 3D engine I'm using can stream oggs containing theora.

Would love for the team to keep us updated on this one.

Many thanks

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Fri Oct 14, 2011 10:04 am

OGG Theora (yes, I had heard of it!) decode is pretty much experimental on the GPU - and I don't think it is exported to Linux at this stage. It's also a software implementation, so no HW support, so its unknown whether it will be faster than doing the decode on the Arm itself. I would think so, but it's still early development.

For the moment, I guess you can use the Arm side library, although how fast that would be I don't know.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Scribe
Posts: 98
Joined: Wed Sep 28, 2011 12:00 pm
Contact: Website

Re: Hardware accelerated audio/video en/decoding

Fri Oct 14, 2011 10:30 am

Quote from jamesh on October 14, 2011, 11:04
OGG Theora (yes, I had heard of it!) decode is pretty much experimental on the GPU - and I don't think it is exported to Linux at this stage. It's also a software implementation, so no HW support, so its unknown whether it will be faster than doing the decode on the Arm itself. I would think so, but it's still early development.

For the moment, I guess you can use the Arm side library, although how fast that would be I don't know.

Yeah this is my problem, I'm really looking to live encode a webcam stream (low fps is fine) but there's no way it'll happen smoothly without GPU acceleration.

How is OpenMAX support looking on the Pi at the moment?

Here's the little PDF I read about Theroa and linux: http://elinux.org/images/0/09/.....LF2010.pdf

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Fri Oct 14, 2011 11:02 am

We are actually thinking of providing a wrapper over OpenMAX to make things a bit easier. You get a subset of the full OpenMAX, but Max is such a total and utter PITA to use that is a worthwhile trade off!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Rasz
Posts: 24
Joined: Thu Oct 13, 2011 10:28 pm

Re: Hardware accelerated audio/video en/decoding

Fri Oct 14, 2011 11:21 am

Quote from Scribe on October 14, 2011, 10:14
Theroa is the video format and contained by OGG. It's licence free and was weighed up at one point as 'the' HTML5 codec but now the spec is much more open.


Lets face it, it would take many many dev hours to implement it so You, Stallman and two other people on the internet could watch 3 clips encoded with it.
I am more concerned with Xvid and Divx, afaik they arent patent encumbered and are very popular.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Sat Oct 15, 2011 7:31 am

Well, it's already implemented, but needs more optimisation. That would need to be done at Broadcom, so up to them whether it gets done. The customer is always right, so if a customer asks for it....(and pays for it)....

I'll need to check up on the implementation state of xvid/divx - although they both have possible patent issues (as they are MPEG4 part 2).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

kattle87
Posts: 34
Joined: Mon Oct 10, 2011 7:50 am

Re: Hardware accelerated audio/video en/decoding

Sat Oct 15, 2011 4:27 pm

I'm extremely interested in hardware-supported encoding, too... Mostly because I got now used to a nice transcoding for video-on-demand solution (currently using VLC) and it would be really nice to have access to that on the raspberry. Would mean I don't need my PC powered on all the time any more.
Even though I would like to say H264 is superior to VP8/Theora, and for that reason alone (and for my really poor bandwidth) I'd love to see H.264 hardware encoding support.
I would enjoy some updates from the team, too

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23708
Joined: Sat Jul 30, 2011 7:41 pm

Re: Hardware accelerated audio/video en/decoding

Sun Oct 16, 2011 8:55 am

H2654 and VP8 are pretty similar TBH, not much in it. H264 probably shades it.

The Raspberry will support H264 decode certainly, there are some issues with encode from Linux at the moment, but I am sure it will get there eventually.

Updates from the team? Not sure what you mean.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Return to “General discussion”