fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Clear last rendered frame

Sun Sep 01, 2019 8:28 pm

Hi,
I’m using OpenMAX in my program for rendering H264 video streams (I have a decoder, scheduler and renderer). The stream can be interrupted (paused), and when that happens, I’d like to clear the last rendered frame, so that what’s underneath is displayed again, like before the video rendering started.
I’ve tried flushing the tunnels, but that doesn’t change anything.

Is there any API function for that or do I have to destroy the whole rendering pipeline and create it anew every time playback is paused?

Any help is highly appreciated!

User avatar
dividuum
Posts: 188
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Clear last rendered frame

Mon Sep 02, 2019 11:29 pm

fd_ wrote:
Sun Sep 01, 2019 8:28 pm
Is there any API function for that or do I have to destroy the whole rendering pipeline and create it anew every time playback is paused?
I suggest using OMX_IndexConfigDisplayRegion on the video_render component to move the video out of the visible screen area or into a lower layer.
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 8:25 am

Thanks for your suggestion! I'm a bit reluctant to choose this solution for two reasons though:
1) According to https://www.raspberrypi.org/forums/view ... p?t=163620, setting the layer requires the component to be in a state other than executing. I'd rather not juggle around with component states too often.
2) I'd have to move rendering back again when playback continues, which I also find rather inelegant.

I was hoping there's a convenient function for clearing the video display until the next frame is rendered :?

User avatar
dividuum
Posts: 188
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Clear last rendered frame

Tue Sep 03, 2019 8:46 am

fd_ wrote:
Tue Sep 03, 2019 8:25 am
1) According to https://www.raspberrypi.org/forums/view ... p?t=163620, setting the layer requires the component to be in a state other than executing. I'd rather not juggle around with component states too often.
Oh. I didn't know that and have been modifying these parameters for years without any problems. Guess that happens when all the "documentation" is basically syntax only and no semantics :cry:
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 9:23 am

Oh, that's interesting :? ! I'm now using your suggested solution and indeed, no state changes are required. Thanks again! :D

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7513
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Clear last rendered frame

Tue Sep 03, 2019 9:29 am

Apologies if you've read something extra into one of my comments.

OMX IL 101
OMXConfigAnything should be settable in any state.
OMXParamAnything is only settable when the port/component is disabled.

Spec
3.2.2.9.3 Error Conditions
The following error conditions can occur:
• OMX_ErrorIncorrectStateOperation when the OMX_SetParameter
function is called and the component is not in the OMX_StateLoaded state, or the
port is not disabled.
OMX_IndexConfigDisplayRegion should be settable in any state, and will be applied at the next vsync.

If you have a tunnel into video_render, then I would have expected flushing the tunnel to have worked.
Have you got a simple test app that I can try (much as I hate trying to find my way through the IL framework)?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 9:46 am

Thanks for the clarification @6by9 !
My code is on GitHub, so if you have an iOS or macOS device at hand, you should be able to reproduce the issue with the code base before the last commit: https://github.com/FD-/RPiPlay/tree/344 ... e1b695d4a7. All OpenMAX code is in renderers/video_renderer_rpi.c. The video_renderer_flush function is called when the connection to a device is closed eg. when "Stop mirroring" is pressed on an iOS device. UPDATE: You'll have to remove the call to video_renderer_draw_background from the video_renderer_flush function (This was a failed attempt to get rid of the last rendered video frame).

I'll try to see if I can also reproduce the issue in the hello_video sample later today.

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 1:47 pm

Deleted since this was a duplicate of what I posted below.
Last edited by fd_ on Tue Sep 03, 2019 1:49 pm, edited 1 time in total.

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 1:49 pm

6by9 wrote:
Tue Sep 03, 2019 9:29 am
If you have a tunnel into video_render, then I would have expected flushing the tunnel to have worked.
Have you got a simple test app that I can try (much as I hate trying to find my way through the IL framework)?
I have now investigated the issue and created a simple demo program.

It seems like calling flush (after all data was sent into the pipeline) doesn't really clear frames that have been sent into the pipeline but were not yet displayed (eg due to scheduling). This might be intended behavior (since we're just flushing the tunnels, not the components linked by the tunnels), but it seems like it's not documented. And also, I don't think there's any way to also flush the components, is there?

For demonstrating the issue, I've modified the adafruit hello_video loop sample. My modifications are available here: https://github.com/FD-/pi_hello_video

Compiling the code as-is demonstrates that despite flushing the pipeline after all data is fed into it, the last frame is still stuck at the end. The only effect of the flush is a short flicker when all data has been sent into the pipeline (but a few seconds are yet to be displayed).

Adding another getchar() above the flush, you can see that suddenly, the flush seems to clear the last frame. This indicates the problem is linked to timing.

When uncommenting the workaround, you can see that sending an empty buffer (with EOS flag) through the pipeline and waiting until it exits the pipeline seems to also 'fix' the issue.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7513
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Clear last rendered frame

Tue Sep 03, 2019 2:05 pm

Ah, more details. video_scheduler is in play. I'd only been looking at video_render.
The brief flash of black sounds like video_scheduler to video_render is correctly flushed, but then the clock keeps running and video_scheduler delivers the queued buffers.

It does appear at first glance that calling flush on the video_scheduler input port does not release queued images. Stopping the clock should flush all buffers, as does disabling the port.

I do feel that calling flush on the scheduler input port should flush the queued buffers, so I may see if there is a quick fix there. I have a minor reluctance to go hacking around too much in there as others may be relying on the old behaviour.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7513
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Clear last rendered frame

Tue Sep 03, 2019 2:33 pm

Video_scheduler is a red herring I think.

You're flushing the tunnels, but not the input port of video_decode. The decoder therefore still has a chunk of encoded bitstream to decode, and keeps going with that after your flush.
The workaround is pretty much the expected way to do things, in that you are waiting for the tunnel to clear via the EOS rather than abandoning it mid-stream.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 2:38 pm

6by9 wrote: Video_scheduler is a red herring I think.

You're flushing the tunnels, but not the input port of video_decode. The decoder therefore still has a chunk of encoded bitstream to decode, and keeps going with that after your flush.
The workaround is pretty much the expected way to do things, in that you are waiting for the tunnel to clear via the EOS rather than abandoning it mid-stream.
Alright, thanks for the clarification! I had just found that (EOS) part in the Adafruit sample I modified and am now using it in my code. :D

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7513
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Clear last rendered frame

Tue Sep 03, 2019 2:43 pm

fd_ wrote:
Tue Sep 03, 2019 2:38 pm
Alright, thanks for the clarification! I had just found that (EOS) part in the Adafruit sample I modified and am now using it in my code. :D
Pointing out that it's also at the end of hello_video would probably be considered rude ;)
https://github.com/raspberrypi/userland ... deo.c#L178
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Clear last rendered frame

Tue Sep 03, 2019 2:54 pm

6by9 wrote:
Tue Sep 03, 2019 2:43 pm
Pointing out that it's also at the end of hello_video would probably be considered rude ;)
https://github.com/raspberrypi/userland ... deo.c#L178
Admittedly, I didn't consult the samples but searched through the IL components docs, skimmed through OMXPlayer code, consulted Google and eventually posted after not finding any suitable thread. However, IMHO, the problem here was that I searched for the wrong terms (See thread title) and even when I saw that code in the Adafruit sample, I initially figured it's only needed for tearing down the pipeline, not for flushing it.

Please let me stress that I am aware of how annoying it must be to often have to answer repetitive questions, so I appreciate your efforts even more! Thanks for your patience! You guys rock :D 8-)

Return to “OpenMAX”