User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Couldn't you open the firmware?

Sun Nov 18, 2012 10:34 pm

I've been working on raspberrypi flavoured webgl bindings for node.js as some of you might know. I'm almost done with them and I will release the first version of them soon.

I found out I'd like to do a replacement API for egl and dispmanx. The API of firmware is molded to follow the APIs they implement, so it's an apparent roadblock that I don't have access to that firmware.

Code: Select all

/* display_id would be retrievable from udev.
 * Any layer could be attached to any display device or detached from it.
 */
GetDisplay(display_id) -> display
DiscardDisplay(display)
AttachLayer(display, layer, x, y, width, height, z)
DetachLayer(display, layer)

/* Layers would be resizable handles.
 */
CreateLayer(width, height) -> layer
ResizeLayer(layer, width, height)
DiscardLayer(layer)

/* Layers could be shared between processes.
 */
GetLayer(layer_id) -> layer
LayerId(layer) -> layer_id
LayerAccess(pid, flags) // flags: READ_FLAG | DRAW_FLAG

/* Surfaces would use those handles and things drawn to them would go to the layer.
 */
GetSurface(layer, attributes) -> surface
DiscardSurface(surface)
SwapBuffers(surface)

/* For context it'd be enough to know which API is in use.
 */
CreateContext(api) -> context
MakeCurrent(draw, read, context)
GetCurrent() -> (draw, read, context)
DestroyContext(context)

/* Layers could be used as textures.
 */
GetImage(layer) -> image
DiscardImage(image) -> image

elatllat
Posts: 1337
Joined: Sat Dec 17, 2011 5:05 pm

Re: Couldn't you open the firmware?

Mon Nov 19, 2012 10:41 pm

could you not just use a wrapper?
2160p + USB3 + Wayland: https://hardkernel.com

FAQ : https://raspberrypi.stackexchange.com

Unanswered: https://www.raspberrypi.org/forums/search.php?search_id=unanswered

User avatar
mahjongg
Forum Moderator
Forum Moderator
Posts: 10704
Joined: Sun Mar 11, 2012 12:19 am
Location: South Holland, The Netherlands

Re: Couldn't you open the firmware?

Mon Nov 19, 2012 10:42 pm

also, they cannot give away something they do not own.

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 7:12 am

elatllat wrote:could you not just use a wrapper?
My concept of graphics context API is incompatible with EGL. It cannot be implemented on top of a wrapper. In fact I don't even know if it can be done even if firmware were open, but I'd get onto it.
mahjongg wrote:also, they cannot give away something they do not own.
I don't think that matters much. You see that the foundation is highest in the command chain. And I keep thinking they'd be capable of getting the platform completely open anytime soon. Millions of sold units mean quite something for broadcom and they know it. Even their competitors know it I'm sure about that.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20059
Joined: Sat Jul 30, 2011 7:41 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 7:54 am

Actually, you are limited to EGL or dispmanx. That what the underlying software AND hardware is designed around, so even thought the Arm side is a wrapper to some code on the GPU side, that GPU code cannot really change much either. And of course, it all optimised to infinity and beyond, so anything you try and do would be slower and less effective that what is already there.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20059
Joined: Sat Jul 30, 2011 7:41 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 7:55 am

cheery wrote: I don't think that matters much. You see that the foundation is highest in the command chain. And I keep thinking they'd be capable of getting the platform completely open anytime soon. Millions of sold units mean quite something for broadcom and they know it. Even their competitors know it I'm sure about that.
Millions of units sold = drop in the water compared to other chips sales I'm afraid. It's not completely insignificant, but its not a big earner either.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 10:55 am

Are you certain that the EGLconfig is required by hardware still?

If I could hook EGLImage to dispmanx element, that might help out quite a lot. I could then just always create PixmapSurfaces, and have the compositing and fullscreen behave similarly. Another thing that might help would be if I could draw to DISPMANX_RESOURCE_T and use them as textures.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20059
Joined: Sat Jul 30, 2011 7:41 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 11:47 am

cheery wrote:Are you certain that the EGLconfig is required by hardware still?

If I could hook EGLImage to dispmanx element, that might help out quite a lot. I could then just always create PixmapSurfaces, and have the compositing and fullscreen behave similarly. Another thing that might help would be if I could draw to DISPMANX_RESOURCE_T and use them as textures.
Not sure what your question is. The underlying hardware and software is pretty much 'dispmanx', with EGL implemented on top of it I think. So just use dispmanx.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 12:32 pm

jamesh wrote:Not sure what your question is. The underlying hardware and software is pretty much 'dispmanx', with EGL implemented on top of it I think. So just use dispmanx.
Well my question is: Do I have some sane alternate way to doing this stuff, instead of what I'm currently doing?

To get a surface that can appear in opengl texture, I need to create a pixbuffer and send it as global image handle elsewhere.

To get a surface that appears on the display, I need to get a dispmanx element and use it to create a window.

These two routes to get a surface are entirely different and that's what makes things lot more complex than I accept.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20059
Joined: Sat Jul 30, 2011 7:41 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 2:38 pm

I think there are some specific dispmanx functions to help with stuff like this - but I'm afraid I am not an expert on this stuff can cannot help with those. Dispmanx is the way to go though. Openning the GPU code won;t make any difference - you'll still need to use dispmanx. I think. I need to ask around to confirm!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 3:07 pm

If you ask around, I'd also like to know about how to get those surfaces resize with minimum effort.

Heater
Posts: 9463
Joined: Tue Jul 17, 2012 3:02 pm

Re: Couldn't you open the firmware?

Tue Nov 20, 2012 7:27 pm

Cheery,
You see that the foundation is highest in the command chain
Actually I doubt it. They are just another customer. I belive that GPU architecture and firmware is used in other devices and has huge sales outside rhe Raspi Foundation.

Still, given the huge success of the Raspi the might have some "pull" in getting features added to the firmware if you really need them and shout loud enough.

Sadly I know nothing about what you are trying to do. But I have to say the idea of OpenGL and Node.js is awesome.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5144
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Couldn't you open the firmware?

Wed Nov 21, 2012 11:03 am

cheery wrote:Are you certain that the EGLconfig is required by hardware still?

If I could hook EGLImage to dispmanx element, that might help out quite a lot. I could then just always create PixmapSurfaces, and have the compositing and fullscreen behave similarly. Another thing that might help would be if I could draw to DISPMANX_RESOURCE_T and use them as textures.
This isn't really possible. The texture format used by the 3D hardware is not the same as the linear formats used by dispman. You can't just cast pointers around.

The solution is to use glGetPixels to get pixels from openGL. Use vc_dispmanx_resource_write_data to put pixels into a dispmanx buffer.

If this is made to work, and profiling shows these calls are the bottleneck to smooth performance, we can investigate solutions.

You probably also want to read:
http://www.raspberrypi.org/phpBB3/viewt ... 53&t=15376

where X composition is done through dispmanx and GLES which might make what you are trying to do much simpler.

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Couldn't you open the firmware?

Wed Nov 21, 2012 1:27 pm

I made a diagram to make it less ambiguous what's the problem here.
graphics_context.png
graphics_context.png (52.52 KiB) Viewed 1399 times
The two first rows depicts the two examples I've found about setting up a graphics context on raspberry pi. Having to support both ways to create drawing surface makes things complicated though.

The third row is something that I'd prefer instead, because it'd be easier for compositor to function that way. If there's ways to get it behave like this then sure I'd gladly just implement things instead of whining. Especially if it was documented somewhere.

Wayland, btw. is going to have exactly same issues. As far as I heard on their channel, they only have software rasterizing supported at the moment. It might be worthwhile to invest your time into getting things solved out here..

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Couldn't you open the firmware?

Wed Nov 21, 2012 1:43 pm

dom wrote:The solution is to use glGetPixels to get pixels from openGL. Use vc_dispmanx_resource_write_data to put pixels into a dispmanx buffer.

If this is made to work, and profiling shows these calls are the bottleneck to smooth performance, we can investigate solutions.
Nice thing in the dispmanx and those globalimages is that they're sort of retained mode. glGetPixels/write_data would either require me to implement some threading hack to the compositor's side, or get the dispmanx resource handle transferred over to the client.

I'd expect it'd be simplest just if we could:
  • provide buffers that are attachable to both a texture and a display
  • allow them to be resized, at least by the process that draws on the surface.

Return to “General discussion”

Who is online

Users browsing this forum: Baidu [Spider], Douglas6, HawaiianPi, procount, RedSnowDonkey, steinerlein and 48 guests