blackshard83
Posts: 67
Joined: Fri Jan 10, 2014 8:31 am

EGL and OpenVG broadcom extensions, what they do?

Thu Sep 28, 2017 11:10 am

I see that in /opt/vc/include egl and openvg directories there are some import header files dealing with some broadcom specific extensions to EGL and OpenVG, but I can't find any documentation about them.

I'm curious about what they do and if can be generally useful. I guess that most probably they are undocumented because they are for internal use (like MMAL things, or so) but maybe someone already tinkered with.

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Thu Sep 28, 2017 1:04 pm

Still learning but now at a usable level. ie I can make Pi's do something useful 8-)
https://ultibo.org/forum/viewtopic.php? ... 9&start=30

Plenty of links to follow, Ajstarks for OpenVG in C.
EGL seems like a middle layer between Open?? and DispmanX.
EGL, DispmanX not so well documented but Andrew from Melbourne and Jan also from Melbourne has figured somethings out :D
https://github.com/AndrewFromMelbourne
http://www.apress.com/gp/book/9781484224717
I have yet to get Jan's book, but github has the code
https://github.com/apress/raspberry-pi- ... video-prog

My simple interpretation of this code, probably wrong, incomplete :oops:
DispmanX sets up the displays, lcd, hdmi etc and makes a rectangle.
EGL then turns this rect into a surface, OpenVG etc write to that surface.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

blackshard83
Posts: 67
Joined: Fri Jan 10, 2014 8:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Thu Sep 28, 2017 2:17 pm

Thanks for the answer, but it does not help. I was specifically asking some documentation about EGL and OpenVG broadcom extensions.
EGL and OpenVG libraries are well documented by khronos specifications available on their site for free

User avatar
Paeryn
Posts: 1608
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: EGL and OpenVG broadcom extensions, what they do?

Thu Sep 28, 2017 6:24 pm

What Broadcom OpenVG extensions? The only extensions I can see for OpenVG are the standard Khronos and NDS ones.

Some of those EGL extensions more than likely aren't available (e.g. the performance stats ones). The only function I know about is eglSaneChooseConfig() which is just a modified eglChooseConfig() where the configs are always sorted with colour depths lowest first (EGL says if you specify a colour's depth greater than 0 then the configs will be sorted with the highest available colour depth first). It makes choosing a config with RGB565 a lot easier as eglChooseConfig() will try and give you an RGB888 first whereas with eglSaneChooseConfig() the first returned config is lowest the hardware can do whilst meeting your requirements.
She who travels light — forgot something.

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Fri Sep 29, 2017 4:09 am

OpenVG to EGL to dispmanx to VC via VCOS.

Peter Lemon''s code gives you a clue into how the VC4 ThreedX RTOS in start_x.elf does the control list stuff
https://github.com/PeterLemon/Raspberry ... ontrolList
The Videocore manual section 9 is control list

The extensions are probably just the EGL to BCM interface, like there is a EGL to Nvidia one.
How much the Videocore VG hardware can do of the OpenVG standard we don't know.
There may be some software in the VC ThreedX RTOS that translates and/or emulates the OpenVG/EGL stuff.

Unless you get the source for start_x.elf (not likely) we can only guess and reverse engineer.
Some of this reverse engineering is in the baremetal posts.

Not sure what extensions you mean. eglext.h?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

LdB
Posts: 544
Joined: Wed Dec 07, 2016 2:29 pm

Re: EGL and OpenVG broadcom extensions, what they do?

Fri Sep 29, 2017 6:32 am

You aren't helping him Gavin. What he is asking about is there is some hidden extensions (function pointers) which aren't part of the normal specs that either Broadcom or Eric Anholt has put in.

As you know I have been playing around with these suckers right on the VC4 interface but my own documentation is based from Eric's code and notes. I am not sure how much is exposed in the standard headers and not inclined to waste time to track them down. There are lots of things that is on the VC4 code that lies outside any of those specs that may have been dragged up for there own testing or use. I would expect some of the ANGLE stuff for example may well be exposed because it looks like he did some work with MS on it.

To some degree Eric is publishing it as he does the new driver upgrades but I would think he would balk at wasting time to document what was an old driver interface from some point back in history. The driver does the specs it sets out to do and is documented to that level.

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Sat Sep 30, 2017 7:32 am

OP was not specific about which includes.

Off track , but these are pretty interesting, did not really notice them before.
https://github.com/raspberrypi/firmware ... include/IL
These would be OpenMAX IL, interface layer.
https://www.khronos.org/registry/OpenMA ... cation.pdf

EGL/VG all seems to have Khronos or KHR in them and there is data on them at Khronos
https://www.khronos.org/registry/OpenVG ... ending.txt

But hidden extensions are by definition hidden until they are not ;)

Is the OP is talking about this one?
https://github.com/raspberrypi/firmware ... ext_brcm.h
This looks interesting, a fast way to get mono images for CV stuff?
/
* Fastpath for creating greyscale textures from a single plane of a
* MMAL opaque buffers. */
Compare the patents against the register list and it is clear, not everything is clear
https://github.com/hermanhermitage/vide ... 5-Overview
Compare this old one
https://github.com/hermanhermitage/vide ... n#region_5
against the newer stuff in the kernel
https://github.com/anholt/linux/blob/rp ... vc4_regs.h
https://01.org/linuxgraphics/gfx-docs/drm/gpu/vc4.html

Check video
https://anholt.github.io/twivc4/2017/09/26/twiv/
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

LdB
Posts: 544
Joined: Wed Dec 07, 2016 2:29 pm

Re: EGL and OpenVG broadcom extensions, what they do?

Sat Sep 30, 2017 9:18 am

Yes and he discusses the special KMS interface he has given us :-)

That is why I abandoned trying to do the userland stuff Ultibo released and went after the VC4 itself. There is a lot more down in the VC4 than we currently have access to via the drivers.

In the new planes code he is describing blending which is contrast to the status of blending in the normal GL pipeline code .... more work to look at :-)

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Sat Sep 30, 2017 10:53 am

It will be interesting to compare Eric's open stuff to the closed blob.
As far as I am concerned it just gives another option to use in Ultibo.
But this is just the 3D graphics Eric is mostly working on.
Time to look at his HVS code too?

There are a few things still to be figured out, all those audio formats (secret audio DSP?), hardware jpeg blocks.
ISP - image sensor pipeline, totally separate hardware block? Could these be used for other stuff?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

blackshard83
Posts: 67
Joined: Fri Jan 10, 2014 8:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Mon Oct 02, 2017 7:35 am

I was exactly thinking about https://github.com/raspberrypi/firmware ... ext_brcm.h include header.

Listing the official EGL extensions I get these ones: EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_vg_parent_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_lock_surface, so there are no proprietary broadcom extensions compiled in the drivers, I guess.

Also I took a look to https://github.com/raspberrypi/firmware ... VG/vgext.h and found some interesting extensions (like the advanced blending modes and other VGU things) , but yet querying current OpenVG driver says that the only extension available is VG_KHR_EGL_image

I guessed someone in the foundation could explain better those headers: maybe those functions have just been deprecated long time ago or have never been implemented correctly...

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Thu Oct 05, 2017 11:21 am

Ultibo has a OpenVG example, it will display an image.
https://github.com/ultibohub/Examples/t ... sDemo/RPi2
The interesting thing is it uses just the normal VCOS interface, no extra VC4 library is needed.
As jpeg decoding is done in the hardware block in the VC4, it is fast.
The magic for this would be in start_x.elf?

I guess this would be an extension over the normal OpenVG and as VG calls EGL is should be in there somewhere.
And probably also in DispmanX?
So pretty much any baremetal method would be able to use jpeg decoding.

By extension h.264 is in hardware so baremetal video is possible too?
Hmm, maybe not , there is a OpenMax layer library.
https://github.com/ultibohub/Examples/t ... Video/RPi2

I do remember Eben saying the OpenMax was written because the VC4 video stuff was hard to decipher and use?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4430
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGL and OpenVG broadcom extensions, what they do?

Thu Oct 05, 2017 12:21 pm

Gavinmc42 wrote:
Thu Oct 05, 2017 11:21 am
Ultibo has a OpenVG example, it will display an image.
https://github.com/ultibohub/Examples/t ... sDemo/RPi2
The interesting thing is it uses just the normal VCOS interface, no extra VC4 library is needed.
As jpeg decoding is done in the hardware block in the VC4, it is fast.
The magic for this would be in start_x.elf?
Pardon? VCOS is an abstraction for OS functions such as mutexes, timers, threads, etc.
Did you mean normal VG interface? Yes, that would get remoted to VC4 (via VCHIQ as the RPC), and so could potentially make use of hardware blocks. I've never investigated the VG code to see what blocks of hardware it makes use of.

start.elf/start_x.elf/start_db.elf is the VPU firmware. Anyt VCHIQ or mailbox call that Linux or other OS makes will be to some matching code within that firmware.
Gavinmc42 wrote:I guess this would be an extension over the normal OpenVG and as VG calls EGL is should be in there somewhere.
And probably also in DispmanX?
So pretty much any baremetal method would be able to use jpeg decoding.

By extension h.264 is in hardware so baremetal video is possible too?
Hmm, maybe not , there is a OpenMax layer library.
https://github.com/ultibohub/Examples/t ... Video/RPi2

I do remember Eben saying the OpenMax was written because the VC4 video stuff was hard to decipher and use?
Er, no. OpenMAX IL was implemented as that was the emerging standard back in 2006/7. Reality turned out slightly differently with the "standard" being badly defined such that clients generally required some customisation for each platform.
MMAL was written because IL was hard to decipher and use. With it not requiring a standards body and committees to be involved, it meant it could be adapted to the use cases we (Broadcom) cared about.

There was never a need to open the internal codec API, and I doubt there ever will be.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Thu Oct 05, 2017 10:08 pm

MMAL was written because IL was hard to decipher and use.
My faulty memory stands corrected.

VCHIQ/VCOS? The OpenVG stuff bypasses the VCOS?
I just assumed the JPEG extension required the VCOS..

Baremetal access to jpeg is a question I asked before.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4430
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGL and OpenVG broadcom extensions, what they do?

Wed Oct 11, 2017 1:44 pm

Gavinmc42 wrote:
Thu Oct 05, 2017 10:08 pm
MMAL was written because IL was hard to decipher and use.
My faulty memory stands corrected.

VCHIQ/VCOS? The OpenVG stuff bypasses the VCOS?
I just assumed the JPEG extension required the VCOS..
You've got all the VG client code in https://github.com/raspberrypi/userland ... khronos/vg (it also appears to use some stuff in https://github.com/raspberrypi/userland ... nos/common too).

Indirection hell, but:
vg_client.c calls RPC_CALLx() macros
Defined in common/khrn_client_rpc.h line 157 as RPC_DO(...) calls
RPC_DO(...) defined in [url=https://github.com/raspberrypi/userland ... nos/common
and that gets brokered over VCHIQ to the VPU.

Both the VG and common code make use of VCOS for asserts, semaphores/mutexes, and logging.
Gavinmc42 wrote:Baremetal access to jpeg is a question I asked before.
If you want access to any of the codec blocks then IL or MMAL are the only routes I'm aware of. I have no idea how VG is handling the JPEG as it doesn't appear to be calling into the firmware JPEG codec.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Paeryn
Posts: 1608
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: EGL and OpenVG broadcom extensions, what they do?

Wed Oct 11, 2017 2:20 pm

6by9 wrote:
Wed Oct 11, 2017 1:44 pm
If you want access to any of the codec blocks then IL or MMAL are the only routes I'm aware of. I have no idea how VG is handling the JPEG as it doesn't appear to be calling into the firmware JPEG codec.
Looking at the code Gavinmc42 linked to that looks like it is using a version of ajstarks' shapes library which itself uses OpenVG. The jpeg reading in there uses libjpeg to parse jpegs. OpenVG itself doesn't deal with jpegs.
She who travels light — forgot something.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4430
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGL and OpenVG broadcom extensions, what they do?

Wed Oct 11, 2017 2:28 pm

Paeryn wrote:
Wed Oct 11, 2017 2:20 pm
6by9 wrote:
Wed Oct 11, 2017 1:44 pm
If you want access to any of the codec blocks then IL or MMAL are the only routes I'm aware of. I have no idea how VG is handling the JPEG as it doesn't appear to be calling into the firmware JPEG codec.
Looking at the code Gavinmc42 linked to that looks like it is using a version of ajstarks' shapes library which itself uses OpenVG. The jpeg reading in there uses libjpeg to parse jpegs. OpenVG itself doesn't deal with jpegs.
Thank you. I was a little surprised if it did, but Gavinmc42 seemed confident it was hardware accelerated, and I took that at face value.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Gavinmc42
Posts: 1431
Joined: Wed Aug 28, 2013 3:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Fri Oct 13, 2017 6:32 am

Yep, looks like the Ultibo OpenVG examples is based on AJStarkes code and that uses JPEGlib so it is probably not accelerated :(
But who knows, you have to dig deep to find the real code and with so many layers I get confused.

This in VGShapes.pas

Code: Select all

function VGShapesCreateImageFromJpeg(const filename:String):VGImage;
{CreateImageFromJpeg decompresses a JPEG image to the standard image format
 source: https://github.com/ileben/ShivaVG/blob/master/examples/test_image.c}     
There must be some hardware JPEG somewhere in the hello_jpeg code?
This?
https://github.com/raspberrypi/userland ... brcmjpeg.c

It's all too confusing
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

blackshard83
Posts: 67
Joined: Fri Jan 10, 2014 8:31 am

Re: EGL and OpenVG broadcom extensions, what they do?

Tue Oct 17, 2017 10:29 am

Gavinmc42 wrote:
Fri Oct 13, 2017 6:32 am
Yep, looks like the Ultibo OpenVG examples is based on AJStarkes code and that uses JPEGlib so it is probably not accelerated :(
But who knows, you have to dig deep to find the real code and with so many layers I get confused.

This in VGShapes.pas

Code: Select all

function VGShapesCreateImageFromJpeg(const filename:String):VGImage;
{CreateImageFromJpeg decompresses a JPEG image to the standard image format
 source: https://github.com/ileben/ShivaVG/blob/master/examples/test_image.c}     
There must be some hardware JPEG somewhere in the hello_jpeg code?
This?
https://github.com/raspberrypi/userland ... brcmjpeg.c

It's all too confusing
OpenVG doesn't deal with JPEG at all, and it should not. OpenVG are 2D API specifications as much as OpenGL/GLES are 3D API specs. They don't deal with anything else that is render graphical primitives and JPEG is definitely not a primitive.

OpenMAX deals with accelerated JPEG decoding but, as 6by9 said, it is not very easy to implement even a simple decoder because OpenMAX requires a vast amount of boilerplate code and it is not very friendly.

Return to “Graphics programming”

Who is online

Users browsing this forum: No registered users and 2 guests