OpenMax rendering onto OpenGL texture


72 posts   Page 2 of 3   1, 2, 3
by Twinkletoes » Sat Dec 08, 2012 5:19 pm
Ta muchly!!
Posts: 210
Joined: Fri May 25, 2012 9:44 pm
by luc4 » Sun Dec 09, 2012 3:04 pm
In case it can be useful to someone, I posted here: http://thebugfreeblog.blogspot.it/2012/ ... -h264.html a sample code which uses the egl_render component to render h264 and compressed images to an OpenGL texture. The code is a mess, but it builds and work if you have Qt 5 compiled. I used the hello_video code as the starting point of the code.
Hope you it helps!
Posts: 29
Joined: Mon Nov 12, 2012 12:28 am
by Comics » Fri Dec 14, 2012 3:30 pm
Hello,

I've been looking at the code but i still have the same issue on mine(without using QT).
The callback doesn't get fired for the fill buffer.

Has anyone solved this?
Posts: 4
Joined: Fri Dec 14, 2012 3:29 pm
by Comics » Mon Dec 17, 2012 4:55 pm
Well,

It seems like i found the initial issue problem(i wasn't properly settings the pNativeWindow to the display).

I'm now suck in another point.
While trying to get the read_media component to read a file.
Has anyone ever tried this component?
It always gives me this error:

Code: Select all
Debug from ILCLient: ÈT¡: insufficient resources 0 (1153223636)


When trying to change to state Executing the read_media.
I've seen that for the GPU to access local filesystem files, it needs vcfiled to be running, and i tried to run it using debug and i can't see any requests.
Code: Select all
vcfiled --debug --foreground
vcfiled started


Any opinions?
Posts: 4
Joined: Fri Dec 14, 2012 3:29 pm
by dom » Mon Dec 17, 2012 5:39 pm
Comics wrote:I'm now suck in another point.
While trying to get the read_media component to read a file.
Has anyone ever tried this component?


In my post pointing at this software I said
Note: the "playback_il" interface uses a read_media openmax component running on the GPU.
This isn't available (it is better done by arm) and you'll need video decode to be done more like hello_video or omxplayer.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by Comics » Mon Dec 17, 2012 6:43 pm
dom wrote:
Comics wrote:I'm now suck in another point.
While trying to get the read_media component to read a file.
Has anyone ever tried this component?


In my post pointing at this software I said
Note: the "playback_il" interface uses a read_media openmax component running on the GPU.
This isn't available (it is better done by arm) and you'll need video decode to be done more like hello_video or omxplayer.


First of all thanks for the reply.
Oh sorry i didn't understood it that way.
You mean to say that the component is disabled? ( I would prefer using it if it's enabled because that way the GPU can handle all the libavformat stuff).
How about these guys?
https://github.com/peterderivaz/pyopenmax/issues/1
They say that it works for them
Posts: 4
Joined: Fri Dec 14, 2012 3:29 pm
by dom » Tue Dec 18, 2012 12:01 am
Comics wrote:First of all thanks for the reply.
Oh sorry i didn't understood it that way.
You mean to say that the component is disabled? ( I would prefer using it if it's enabled because that way the GPU can handle all the libavformat stuff).
How about these guys?
https://github.com/peterderivaz/pyopenmax/issues/1
They say that it works for them

It worked once. But as that github issue says, it no longer works.
Reading files from the GPU is not the right way to do it. Suppose you want the file to come from a a http stream? Suppose you want to support an unsupported format (e.g. a .ts file)?
Have a look at hello_video for how to pass encoded video data into the video_decode component without using read_media.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by luc4 » Wed Dec 26, 2012 3:15 am
The egl_render component with the code I posted above seems to play nicely with the scene graph in Qt 5.0: http://thebugfreeblog.blogspot.it/2012/ ... ering.html. It would be great if Qt guys could integrate it into the Qt Multimedia module!
Posts: 29
Joined: Mon Nov 12, 2012 12:28 am
by jvcleave » Tue Jan 29, 2013 7:35 am
Has anyone successfully been able to use OMX_UseEGLImage with Raspbian? I think @luc4 is using the softfp Wheezy
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by Comics » Tue Jan 29, 2013 9:00 am
jvcleave wrote:Has anyone successfully been able to use OMX_UseEGLImage with Raspbian? I think @luc4 is using the softfp Wheezy


Hello,

Yes it works.
Posts: 4
Joined: Fri Dec 14, 2012 3:29 pm
by jvcleave » Wed Jan 30, 2013 3:23 am
Hey Comics, Would you have any working code available? I have built this one in hopes to track down the issue:

https://github.com/jvcleave/hello_video_fillbuffer

any insight would be appreciated!
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by OtherCrashOverride » Sat Feb 02, 2013 3:37 am
I have been having the exact same issue: egl_render never calls the OpenMax IL FillBufferDone callback and the OpenGL texture data is never written to by the component.

I noticed the bug report at https://github.com/raspberrypi/firmware/issues/143 which should probably be updated to state this is a egl_render component issue. Other components I have used seem to fire the callback properly.

I am using Rasbian "wheezy" hardfloat distro 2012-12-16 from the download page and have tried updating the firmware to a more recent version.
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by OtherCrashOverride » Mon Feb 04, 2013 1:23 am
Seeing the same results on softfloat debian from download page.

Perhaps its related to certain boards? I have a 512MB Raspberry Pi Model B.
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by OtherCrashOverride » Tue Feb 05, 2013 2:20 am
As this issue relates to the binary blob that is provided, can the person responsible for maintaining it provide a status on the bug report for this? Is it being investigated? Is it a confirmed bug? Is it user side error?

Unlike issues that occur in the kernel, we do not have access to the source code or tools to 'help ourselves' with this one. We can not set breakpoints or inspect anything that happens in the GPU code which is where the egl_render components resides.

Is there any additional information that needs to be provided to assist with this issue? What can be done to better help us to help the GPU firmware maintainer resolve this matter?
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by jvcleave » Tue Feb 05, 2013 2:42 am
OtherCrashOverride actually the source code is available at https://github.com/raspberrypi/userland. I have started digging in and to try and figure out where it drops off but it gets pretty gnarly in there.

I think the source maintainers are more likely to see the issue on github or other channels so it would probably help if you could chime in there.

I think this thread could be helpful with anyone that had this working previously to verify whether it works in newer firmware and with raspian.
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by OtherCrashOverride » Tue Feb 05, 2013 4:41 am
The userland source code provided mainly wraps calls to the GPU and therefor is not of much help.

This is where i suspect the problem lies: https://github.com/raspberrypi/userland/blob/master/interface/vmcs_host/vcilcs_out.c#L585

Notice the comment "If an output port is marked as an EGL port, we request EGL to notify the IL component when it's allowed to render into the buffer. "

The call is defined here https://github.com/raspberrypi/userland/blob/master/interface/khronos/ext/egl_openmaxil_client.c#L36 which wraps another call to the GPU binary blob.
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by jvcleave » Tue Feb 05, 2013 4:59 am
I found another instance of the second call which I suspected to be an issue here as well

https://github.com/raspberrypi/userland ... ost.c#L136

I thought that was sort of a forward declaration but I put a printf statement in there and it is called

I also put printf statement here:
https://github.com/raspberrypi/userland ... out.c#L873

and this was only called by the decoder, never the renderer
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by OtherCrashOverride » Tue Feb 05, 2013 5:11 am
That was it!

The issue is not the code, its the *LINKING* order. You must link the OpenMax layer *before* linking the other libraries to prevent the stub from being called. I now have a confirmed working sample.


Thanks for the help in solving this. I am using a different codebase rather than the sample provided, so let me know if you are still having any issues.
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by jvcleave » Tue Feb 05, 2013 5:26 am
no shit - I have been messing with this way to long so anything you can send my way would be appreciated :)
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by OtherCrashOverride » Tue Feb 05, 2013 6:05 am
Have not tested this, but changing the link flags in your makefile should probably resolve it.

https://github.com/jvcleave/hello_video_fillbuffer/blob/master/Makefile#L3

try something like "LDFLAGS=-lilclient + LDFLAGS".

The goal is to have the libraries link in the following order:
1) openmaxil.so (otherwise egl_render won't work)
2) bcm_host.so
3) GLESv2.so (otherwise egl wont work)
4) EGL.so
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by jvcleave » Tue Feb 05, 2013 6:31 am
still nothing :(

did you recompile ilclient?
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by OtherCrashOverride » Tue Feb 05, 2013 6:39 am
I'm not using ilclient at all. I am not even using C. I am using C# with handmade bindings to openmax, etc. which is why my code would be of no use to you.

I will download your code to my Pi and see if I can spot the issue.
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by jvcleave » Tue Feb 05, 2013 7:03 am
that would be awesome - can you ldd your binary and post that?
Posts: 30
Joined: Thu May 24, 2012 10:27 pm
by OtherCrashOverride » Tue Feb 05, 2013 8:22 am
I have tried modifying the code and make files without any success. I am not familiar enough with ilclient to know what its doing or not doing behind the scenes. The only 'magic' my code required to function was the previously mentioned change in linking order. As long as you are seeing the stub function called, egl_render will not work.
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am
by jvcleave » Tue Feb 05, 2013 9:02 am
thanks for trying - glad to know someone has it going :)

even though it is a different code base I am still curious to see anything you can send (I'm jvcleave on gmail). Beyond the linking order I wonder if I am missing some concept or order of operations. However in my example I would think the Callback would be called as the OMX_BufferFill call is successful
Posts: 30
Joined: Thu May 24, 2012 10:27 pm