User avatar
dividuum
Posts: 181
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Dual HEVC decoding

Wed Sep 18, 2019 2:59 pm

I've modified dom's 2711 FFmpeg branch and I'm playing with it trying to decode multiple HEVC streams at the same time. From what I understand, the code should support multiple threads working with the argon (is there an official name for it?) hardware decoder. At least that's what I understand from the usage of thread_idx and the various "thread local variables".

This works decoding a single video when setting the av decoder options 'threads' value to (for example) "auto". I've now refactored the code, so only a single instance of the RPI_DEBUG struct is shared (and ref-counted) across multiple decoder instances, while each decoder has their own pool of zc images. This all works ok when decoding a single video, with or without multiple threads.

When decoding two videos, the following happens and both videos corrupt rather quickly:

Image

I suspect that there's still some shared state that I'm missing in rpi_hevc_start_frame, rpi_hevc_decode_slice or rpi_hevc_end_frame. My naive understanding so far is that those functions either have their state in the respective AVCodecContext, or in the "slotted" variables selected by thread_idx. So multiple invocations for two videos should only refer to either their state in the AVCodecContext or in the thread local state selected during each rpi_hevc_start_frame/rpi_hevc_decode_slice/rpi_hevc_end_frame cycle.

To further prevent state leaking, I've also tried to lock frame decoding to a single user, by acquiring a global lock at the start of rpi_hevc_start_frame and releasing it at the end of rpi_hevc_end_frame. This also doesn't help. Anyone have any suggestion what I might overlook?
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7415
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Dual HEVC decoding

Thu Sep 19, 2019 1:14 pm

I haven't played with it yet, but I know the person who has been integrating hardware acceleration into VLC and Chromium has.
I've asked him the question of whether he had multiple decodes sorted. I think he had, but had done a couple of fixes. I recall something along the lines of a block of memory being allocated which was sufficient for 3 clients, but something wrong in there.

(edit: correct typo of encodes for decodes to avoid confusion)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
dividuum
Posts: 181
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Dual HEVC decoding

Thu Sep 19, 2019 2:02 pm

6by9 wrote:
Thu Sep 19, 2019 1:14 pm
I haven't played with it yet, but I know the person who has been integrating hardware acceleration into VLC and Chromium has.
I've asked him the question of whether he had multiple encodes sorted. I think he had, but had done a couple of fixes. I recall something along the lines of a block of memory being allocated which was sufficient for 3 clients, but something wrong in there.
Chromium or VLC already have HEVC support? I took a brief look at the RPi chromium patches (correct me, if I'm looking in the wrong place), but I don't see anything that would hint at HEVC support in there. Multiple old-style MMAL vc.ril.video_decode components for H264 at the same time work fine. It's only the HEVC support in the 2711 branch of FFmpeg where I didn't yet get that to work properly, yet.
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23876
Joined: Sat Jul 30, 2011 7:41 pm

Re: Dual HEVC decoding

Thu Sep 19, 2019 2:15 pm

dividuum wrote:
Thu Sep 19, 2019 2:02 pm
6by9 wrote:
Thu Sep 19, 2019 1:14 pm
I haven't played with it yet, but I know the person who has been integrating hardware acceleration into VLC and Chromium has.
I've asked him the question of whether he had multiple encodes sorted. I think he had, but had done a couple of fixes. I recall something along the lines of a block of memory being allocated which was sufficient for 3 clients, but something wrong in there.
Chromium or VLC already have HEVC support? I took a brief look at the RPi chromium patches (correct me, if I'm looking in the wrong place), but I don't see anything that would hint at HEVC support in there. Multiple old-style MMAL vc.ril.video_decode components for H264 at the same time work fine. It's only the HEVC support in the 2711 branch of FFmpeg where I didn't yet get that to work properly, yet.
Both still in testing AIUI, but VLC is pretty close to release.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Return to “OpenMAX”