longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

EGLImageKHR to OpenMax

Tue Sep 18, 2018 4:06 pm

Hi, i have an EGL image, how can i pass it to the OpenMax encoder? i see that it supports this format as input port of the encoder ( OMX_COLOR_FormatBRCMEGL). How can i populate the the pBuffer and nFilledLen of OMX_BUFFERHEADERTYPE (http://maemo.org/api_refs/5.0/beta/libo ... y_p_e.html) with the EGL image? Thanks.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Tue Sep 18, 2018 4:24 pm

There's little point in quoting headers from Maemo - whilst IL is supposedly standardised, there are different versions. All the versions used on Pi are at https://github.com/raspberrypi/userland ... khronos/IL

TBH I have no idea whether parsing EGL images will work or not. I have recollections of it being used within Android, but that imposed a set of restrictions on the buffers in itself.
When set to OMX_COLOR_FormatBRCMEGL it expects to find a OMX_BRCMVEGLIMAGETYPE structure in the buffer. Most of the fields are obvious. nUmemHandle is the memory handle (as reported by "sudo vcdbg reloc") for the final EGL image, with nUmemOffset being any offset into that buffer.
It looks like it has a chance of doing something sensible if you feed it that, but I'll give no guarantees.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 7:35 am

Instead if i use MMAL encoder with MMAL_ENCODING_EGL_IMAGE could be better?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 9:04 am

longo92 wrote:
Wed Sep 19, 2018 7:35 am
Instead if i use MMAL encoder with MMAL_ENCODING_EGL_IMAGE could be better?
Will make no difference. It's the same underlying code, and the two formats are identical - https://github.com/raspberrypi/userland ... _il.c#L732
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 9:10 am

Ok, when i pass the buffer to the encoder should i use OMX_UseEGLImage or OMX_EmptyThisBuffer? both pointing to the OMX_BRCMVEGLIMAGETYPE ?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 10:48 am

Set the port definition with eCompressionFormat = OMX_VIDEO_CodingUnused and eColorFormat = OMX_COLOR_FormatBRCMEGL. I would expect nBufferSize to be sizeof(OMX_BRCMVEGLIMAGETYPE), or 24 bytes.

Cast the buffer pointer to be a OMX_BRCMVEGLIMAGETYPE and populate the fields. As previously stated, they're all obvious apart from nUmemHandle. I don't know how you get to that value from EGL. You can use "sudo vcdbg reloc" to list the relocatable heap handles, and it needs to match the one for your output object. I can ask a colleague if he remembers how to get hold of that handle.

Use OMX_EmptyThisBuffer as you are passing a buffer which references an EGL buffer, not an EGL handle.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 10:52 am

Oh, and the buffer object will always be treated as RGBX32. No other formats are supported.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 11:41 am

Discussion had with said colleague.
Create your EGL target buffer with EGL_IMAGE_BRCM_VCSM and you can get the relevant handle back - see vcsm_square in raspicam for an example.
There are a number of provisos that have to be followed, but it does appear to have a good chance of working.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 1:11 pm

OK, suppose that i retrieve the buffer: vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type)[/url][/url][/url][/url] , and after i can call th EmptyThisBuffer.
Example based on https://github.com/raspberrypi/userland ... m_square.c:

Code: Select all

static int vcsm_square_redraw(RASPITEX_STATE *raspitex_state)
{
    unsigned char *vcsm_buffer = NULL;
    VCSM_CACHE_TYPE_T cache_type;

    vcos_log_trace("%s", VCOS_FUNCTION);

    glClearColor(255, 0, 0, 255);

    GLCHK(glBindFramebuffer(GL_FRAMEBUFFER, fb_name));
    GLCHK(glViewport(0, 0, fb_width, fb_height));
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    // Fill the viewport with the camFill the viewport with the camera image
    GLCHK(glUseProgram(vcsm_square_oes_shader.program));
    GLCHK(glActiveTexture(GL_TEXTURE0));
    GLCHK(glBindTexture(GL_TEXTURE_EXTERNAL_OES, raspitex_state->y_texture));
    GLCHK(glBindBuffer(GL_ARRAY_BUFFER, quad_vbo));
    GLCHK(glEnableVertexAttribArray(vcsm_square_oes_shader.attribute_locations[0]));
    GLCHK(glVertexAttribPointer(vcsm_square_oes_shader.attribute_locations[0], 2, GL_FLOAT, GL_FALSE, 0, 0));
    GLCHK(glDrawArrays(GL_TRIANGLES, 0, 6));

    GLCHK(glFinish());

    // Make the buffer CPU addressable with host cache enabled
    vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type);
    if (! vcsm_buffer) {
        vcos_log_error("Failed to lock VCSM buffer for handle %d\n", vcsm_info.vcsm_handle);
        return -1;
    }
    
    [color=#FF0000]/*HERE: HOW CAN I CALL EmptyThisBuffer on vcsm_buffer?*/[/color]
    
    
    vcos_log_trace("Locked vcsm handle %d at %p\n", vcsm_info.vcsm_handle, vcsm_buffer);

    vcsm_square_draw_pattern(vcsm_buffer);

    // Release the locked texture memory to flush the CPU cache and allow GPU
    // to read it
    vcsm_unlock_ptr(vcsm_buffer);

    // Enable default window surface
    GLCHK(glBindFramebuffer(GL_FRAMEBUFFER, 0));

    // Draw the modified texture buffer to the screen
    GLCHK(glViewport(raspitex_state->x, raspitex_state->y, raspitex_state->width, raspitex_state->height));
    GLCHK(glUseProgram(vcsm_square_shader.program));
    GLCHK(glActiveTexture(GL_TEXTURE0));
    GLCHK(glBindTexture(GL_TEXTURE_2D, fb_tex_name));
    GLCHK(glEnableVertexAttribArray(vcsm_square_shader.attribute_locations[0]));
    GLCHK(glVertexAttribPointer(vcsm_square_shader.attribute_locations[0], 2, GL_FLOAT, GL_FALSE, 0, 0));
    GLCHK(glDrawArrays(GL_TRIANGLES, 0, 6));

    GLCHK(glDisableVertexAttribArray(vcsm_square_shader.attribute_locations[0]));
    GLCHK(glUseProgram(0));

    return 0;
}
the vcsm_buffer is the handler of the buffer to link to nUmemHandle of OMX_BRCMVEGLIMAGETYPE ? Or vcsm_buffer contain the egl_image pixel data in RGBA32 ? or none of the two?
P.S. using eCompressionFormat= OMX_VIDEO_CodingUnused and eColorFormat= OMX_COLOR_FormatBRCMEGL gives 15360 as nBufferSize.

timg236
Posts: 5
Joined: Thu Jun 21, 2018 4:30 pm

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 2:06 pm

vcsm_info.vcsm_handle is the memory handle for the pixel data and vcsm_buffer is the CPU pointer to the pixels. The pointer is obtained by this line of code
vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type);

EGL images in the driver are containers that store the pixel format, size and other data plus generally a mem-handle to the pixel data. Using VCSM should make it easier for you do do the video-encode bit because since you allocate the VCSM buffer you already know the mem-handle and therefore don't need to find a way to extract the mem-handle to the pixel data from the GL driver.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 2:52 pm

Code: Select all

 static int vcsm_square_redraw(RASPITEX_STATE *raspitex_state)
{
	RASPITEX_STATE * state = raspitex_state;	

    unsigned char *vcsm_buffer = NULL;
	static int onlyFirstTime  = 0;
   const int bytes_per_pixel = 4;
   int buffer_size = state->width*state->height*4;
	OMX_ERRORTYPE omx_res;
    VCSM_CACHE_TYPE_T cache_type;

    vcos_log_trace("%s", VCOS_FUNCTION);

    glClearColor(255, 0, 0, 255);

    GLCHK(glBindFramebuffer(GL_FRAMEBUFFER, fb_name));
    GLCHK(glViewport(0, 0, fb_width, fb_height));
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    // Fill the viewport with the camFill the viewport with the camera image
    GLCHK(glUseProgram(vcsm_square_oes_shader.program));
    GLCHK(glActiveTexture(GL_TEXTURE0));
    GLCHK(glBindTexture(GL_TEXTURE_EXTERNAL_OES, raspitex_state->y_texture));
    GLCHK(glBindBuffer(GL_ARRAY_BUFFER, quad_vbo));
    GLCHK(glEnableVertexAttribArray(vcsm_square_oes_shader.attribute_locations[0]));
    GLCHK(glVertexAttribPointer(vcsm_square_oes_shader.attribute_locations[0], 2, GL_FLOAT, GL_FALSE, 0, 0));
    GLCHK(glDrawArrays(GL_TRIANGLES, 0, 6));

    GLCHK(glFinish());

    // Make the buffer CPU addressable with host cache enabled
    vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type);
    if (! vcsm_buffer) {
        vcos_log_error("Failed to lock VCSM buffer for handle %d\n", vcsm_info.vcsm_handle);
        return -1;
    }
    vcos_log_trace("Locked vcsm handle %d at %p\n", vcsm_info.vcsm_handle, vcsm_buffer);
    //printf("Locked vcsm handle %d at %p\n", vcsm_info.vcsm_handle, vcsm_buffer);
	fflush(stdout);
	
	state->in_buffer = ilclient_get_input_buffer(state->video_encode, 200, 0);
		
		if(state->in_buffer != NULL) {
			printf ("in_buffer bot null \n");
			OMX_BRCMVEGLIMAGETYPE image;
			image.nWidth =vcsm_info.width;
			image.nHeight = vcsm_info.height;
			image.nStride = vcsm_info.width<<2;
			image.nUmemHandle =vcsm_info.vcsm_handle; //vcsm_info.vcsm_handle;
			image.nUmemOffset = 0;
			image.nFlipped = 0;
			state->in_buffer->pBuffer = &image;
			state->in_buffer->nFilledLen = sizeof(OMX_BRCMVEGLIMAGETYPE) ;
			 if (OMX_EmptyThisBuffer(ILC_GET_HANDLE(state->video_encode),state->in_buffer) != //state->in_buffer
 					OMX_ErrorNone) {
				 printf("Error emptying buffer!\n");
				}

	}
		
		
		
			 state->out_buffer = ilclient_get_output_buffer(state->video_encode, 201, 0);
			if ( state->out_buffer==NULL) {
				return -1;
			}
			 omx_res = OMX_FillThisBuffer(ILC_GET_HANDLE(state->video_encode),state->out_buffer);
			 if (omx_res != OMX_ErrorNone) {
				printf("Error filling buffer: %s\n", err2str(omx_res));
				return -1;
			}

	
	
	
	
    vcsm_square_draw_pattern(vcsm_buffer);

    // Release the locked texture memory to flush the CPU cache and allow GPU
    // to read it
    vcsm_unlock_ptr(vcsm_buffer);

    // Enable default window surface
    GLCHK(glBindFramebuffer(GL_FRAMEBUFFER, 0));

    // Draw the modified texture buffer to the screen
    GLCHK(glViewport(raspitex_state->x, raspitex_state->y, raspitex_state->width, raspitex_state->height));
    GLCHK(glUseProgram(vcsm_square_shader.program));
    GLCHK(glActiveTexture(GL_TEXTURE0));
    GLCHK(glBindTexture(GL_TEXTURE_2D, fb_tex_name));
    GLCHK(glEnableVertexAttribArray(vcsm_square_shader.attribute_locations[0]));
    GLCHK(glVertexAttribPointer(vcsm_square_shader.attribute_locations[0], 2, GL_FLOAT, GL_FALSE, 0, 0));
    GLCHK(glDrawArrays(GL_TRIANGLES, 0, 6));

    GLCHK(glDisableVertexAttribArray(vcsm_square_shader.attribute_locations[0]));
    GLCHK(glUseProgram(0));

    return 0;
}

I added the in_buffer , out_buffer and an encoder to RASPITEX_STATE.
I obtain OMX_StreamCorrupted :(

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 3:07 pm

longo92 wrote:
Wed Sep 19, 2018 2:52 pm
I obtain OMX_StreamCorrupted :(
From which call?

Post a simple complete app to github or similar and I'll try to give it a quick run to see what is going on. I'm not going to try write my own app to debug this.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Wed Sep 19, 2018 3:56 pm

https://github.com/AlessandroLongobardi ... ree/master

It is a demo based on RaspiStill and RaspiTex (i simply modify them)..
I added the video encoder, the associated buffer and other encoder's stuff in RASPITEX_STATE.
The Encoder Initializatione is in raspitex_init through int Create_ilclient_encoder(ILCLIENT_T **client,COMPONENT_T **video_encode, int width_enc,int height_enc, int bitrate, int fps, int color_format,RASPITEX_STATE* state). In RaspiTex.c i added the EmptyBufferDone and FillBufferDone callback.
During the vcsm_square_redraw function in vcsm_square.c i send the buffer to the encoder.
The output video is saved in a file called test.h264.
The execution command is this one: /raspistill -n -w 1280 -h 960 -g -t 0 -v -gw 0,0,640,480 -gs vcsm_square

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Thu Sep 20, 2018 3:49 pm

So close. You need the VPU MEM_HANDLE, not the vcsm handle.

Code: Select all

diff --git a/gl_scenes/vcsm_square.c b/gl_scenes/vcsm_square.c
index e5117b2..b6cccbc 100644
--- a/gl_scenes/vcsm_square.c
+++ b/gl_scenes/vcsm_square.c
@@ -308,7 +308,7 @@ static int vcsm_square_redraw(RASPITEX_STATE *raspitex_state)
                        image.nWidth =state->width;
                        image.nHeight = state->height;
                        image.nStride = state->width<<2;
-                       image.nUmemHandle =vcsm_info.vcsm_handle; //vcsm_info.vcsm_handle;
+                       image.nUmemHandle = vcsm_vc_hdl_from_hdl(vcsm_info.vcsm_handle); //vcsm_info.vcsm_handle;
                        image.nUmemOffset = 0;
                        image.nFlipped = 0;
                        state->in_buffer->pBuffer = &image;
And the warning

Code: Select all

 #warning OMX_SKIP64BIT is not defined - this will be incompatible with the VC GPU code.
is not one to be ignored.

Code: Select all

diff --git a/RaspiTex.h b/RaspiTex.h
index d69bafc..b718655 100644
--- a/RaspiTex.h
+++ b/RaspiTex.h
@@ -37,6 +37,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 #include <mmal/mmal.h>
 
 //INCLUDES ADDED
+#define OMX_SKIP64BIT
 #include <ilclient.h>
 #include <fcntl.h>
 #include <sys/types.h>
That runs and displays an image. video_encode is being given frames and encoding them. You've butchered the system with where buffers point etc, and it appears to be saving data to test.h264.
The recorded image is corrupted though as you've put in the OMX_BRCMVEGLIMAGETYPE that the image is 640x480 instead of 1024x1024. "./raspistill -n -w 1280 -h 960 -g -t 0 -v -gw 0,0,1024,1024 -gs vcsm_square" resolves that. Do note the requirement on RSO buffers to be a power of two in width.

You have then hit a firmware bug - I recently changed the image conversion routines at the front of video_encode to use the ISP instead of the VPU. Those routines don't have config for RGBX32. Either get a firmware from before July 9th, or wait for me to get the fix out.

With that I am recording the GL image correctly, although there is obviously some timing issue as there is significant tearing across the image. It looks like you're passing the image to video_encode before you've called vcsm_square_draw_pattern which adds the vertical bar to the image. In which case you definitely have a race condition as to which processes the pixels first, and a caching issue. Make the call to vcsm_square_draw_pattern and call vcsm_unlock_ptr (which will flush the ARM cache) BEFORE you pass the buffer to video_encode.
You are also running at a pretty low bitrate (500kbit/s), so the image quality isn't great. 4Mbit/s looks significantly better. I am getting what appear to be buffer skips, probably because critical threads are getting blocked whilst you write the data out. You'll need to refactor your code to reduce those.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Fri Sep 21, 2018 8:27 am

Thanks!!!! you're my saviour!! It works (rollback the firmware to the 3 of july)!!
Of course i use this code only as base/educational-example on how can encode EGLimage. If i make my own application i design it with the apporpiate accuracy.
I'm only a raspberry GPU-beginner proggrammer, this forum and code study is the only way to undestrand the things.
Just for curiosity what does the define OMX_SKIP64BIT ?? I always use OpenMax without define it.
And what does vcsm_vc_hdl_from_hdl (suppose perform some type of handler conversion) ? i cannot find its definition userland/interface/mmal/vc/.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Fri Sep 21, 2018 10:10 am

longo92 wrote:
Fri Sep 21, 2018 8:27 am
Thanks!!!! you're my saviour!! It works (rollback the firmware to the 3 of july)!!
Of course i use this code only as base/educational-example on how can encode EGLimage. If i make my own application i design it with the apporpiate accuracy.
I must confess to getting lost in your various hacks as to where the output buffers were being returned. I recognise that you were just trying things out, but it did all get a little messy in there. Overriding the FillBufferDone callback totally threw me! I'd also recommend that you don't do too much work in that callback context - I think you are blocking all IL communications at that point if you do significant amounts of work there. MMAL is a tad cleaner there in that the buffer callbacks are in a worker thread context.
longo92 wrote:I'm only a raspberry GPU-beginner proggrammer, this forum and code study is the only way to undestrand the things.
Just for curiosity what does the define OMX_SKIP64BIT ?? I always use OpenMax without define it.
When they wrote the IL spec they blundered pretty badly.
Not all compilers support 64 bit variables, or they don't require 64 bit alignment on 64 bit variables. The latter is the case with the VideoCore firmware.

The main bug bear is OMX_TICKS - https://github.com/raspberrypi/userland ... pes.h#L294
When inserted into OMX_BUFFERHEADERTYPE they didn't ensure that it naturally had the correct alignment, and therefore the firmware and the ARM (which does need 64bit alignment) disagree and will read the wrong value for nTimeStamp and all other fields after that nTimeStamp.

If they had thought about it and made sure there was padding or other fields to make nTimeStamp naturally aligned to a 64 bit boundary then there would be no problem ever.
If we'd noticed the issue early on then we could have forced the firmware to adopt 64bit alignment on nTimeStamp to work around it. Unfortunately we haven't found a way to retrospectively work around it without breaking all existing applications that (correctly) set OMX_SKIP64BIT. TBH IL is such a pain in the neck to work with that I think the preference is to deprecate it instead! Everything you've done should also work equally with MMAL which is generally easier to work with.
longo92 wrote:And what does vcsm_vc_hdl_from_hdl (suppose perform some type of handler conversion) ? i cannot find its definition userland/interface/mmal/vc/.
https://github.com/raspberrypi/userland ... csm.c#L649
vcsm in the kernel uses its own internal handles (not file descriptors for reasons best known to itself) and does things like locking the handle to processes and the like. The firmware doesn't know or care about Linux processes, and has a simpler handler system.
If you cat /sys/kernel/debug/vc-smem/state (may need sudo) then you'll see the mappings, such as

Code: Select all

Resource                b2be8300
           PID          1046
           RES_GUID     0x1000
           LOCK_COUNT   0
           REF_COUNT    1
           res_handle   0x4
           res_base_mem fe502000
           SIZE         65536
           DMABUF         (null)
           ATTACH         (null)
           SGT            (null)
           DMA_ADDR     0x00000000
and "sudo vcdbg reloc" shows

Code: Select all

[   4] 0x3e501ae0: used  68K (refcount 1 lock count 0, size    65536, align 4096, data 0x3e502000, d1Rual) 'sm_host_alloc'
to confirm that firmware handle 4 is the vcsm allocation.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Fri Sep 21, 2018 10:45 am

I must confess to getting lost in your various hacks as to where the output buffers were being returned. I recognise that you were just trying things out, but it did all get a little messy in there. Overriding the FillBufferDone callback totally threw me! I'd also recommend that you don't do too much work in that callback context - I think you are blocking all IL communications at that point if you do significant amounts of work there. MMAL is a tad cleaner there in that the buffer callbacks are in a worker thread context.
Yes, future steps is to use the equivalent mmal encoder instead of openMax encoder.

I have another question there is a workaround on this requirement
Do note the requirement on RSO buffers to be a power of two in width.
? Because it limits my resolution only to the ones having power of two.

Thanks a lot for your very professional answer.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Fri Sep 21, 2018 2:00 pm

longo92 wrote:
Fri Sep 21, 2018 10:45 am
I have another question there is a workaround on this requirement
Do note the requirement on RSO buffers to be a power of two in width.
? Because it limits my resolution only to the ones having power of two.

Thanks a lot for your very professional answer.
Nope, AIUI it's a limitation of the hardware. It's referenced in the errata list for the 3D block (VideoCoreIV-AG100-R.pdf page 109)
HW-2898 Address generation error for NPOT raster textures
BCM2835,BCM21553
For raster textures the address offset calculation
assumes power-of-2 widths when issuing memory
lookups. This is not true for mipmap level 0.
Workaround is to pad NPOT raster textures
appropriately
Note that that is the stride that must be a power of two, not the width. You should be able to pad the image to the POT without filling it completely with image data.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Fri Sep 21, 2018 3:08 pm

All Clear.
Another cuorisity question: could i pass the vcsm_buffer retrieve with vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type) to the pBuffer and the proper size to the EmptyThisBuffer? Of course the encoder input port is set with OMX_COLOR_Format32bitABGR8888 (in one of my previouse experiment, when i encode using the glReadPixels, i notice that GL_RBA = OMX_COLOR_Format32bitABGR8888). Thanks

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Fri Sep 21, 2018 3:16 pm

longo92 wrote:
Fri Sep 21, 2018 3:08 pm
All Clear.
Another cuorisity question: could i pass the vcsm_buffer retrieve with vcsm_buffer = (unsigned char *) vcsm_lock_cache(vcsm_info.vcsm_handle, VCSM_CACHE_TYPE_HOST, &cache_type) to the pBuffer and the proper size to the EmptyThisBuffer? Of course the encoder input port is set with OMX_COLOR_Format32bitABGR8888 (in one of my previouse experiment, when i encode using the glReadPixels, i notice that GL_RBA = OMX_COLOR_Format32bitABGR8888). Thanks
VCHIQ (the main IPC mechanism) is likely to complain if you do as it gets confused based on a set of Linux kernel page table flags. It'll be comparable to https://github.com/raspberrypi/userland/issues/386
But also with IL there is an inherent copy involved in passing buffers from ARM to VideoCore, so performance will suffer. That's probably why the BRCM EGL format was created in the first place.

If you switch to MMAL, then setting MMAL_PARAMETER_ZERO_COPY on the buffer and populating buffer->data with the vcsm vc handle, and you should get the equivalent zero copy option as BRCMEGL.
It's almost the same technique I use in yavta where I take a dmabuf for a CMA allocation from V4L2, use vcsm_import to import it into vcsm and get a vc mem_handle for it, and then pass those buffers into MMAL. Likewise drm_mmal for video_decode into DRM (Linux kernel rendering manager).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Thu Sep 27, 2018 9:33 am

If you switch to MMAL, then setting MMAL_PARAMETER_ZERO_COPY on the buffer and populating buffer->data with the vcsm vc handle, and you should get the equivalent zero copy option as BRCMEGL.
Regarding this quote, i have a question:which is the equivalent OMX_BRCMVEGLIMAGETYPE in MMAL?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Thu Sep 27, 2018 9:44 am

longo92 wrote:
Thu Sep 27, 2018 9:33 am
Regarding this quote, i have a question:which is the equivalent OMX_BRCMVEGLIMAGETYPE in MMAL?
https://github.com/raspberrypi/userland ... _il.c#L732

Code: Select all

{MMAL_ENCODING_EGL_IMAGE, OMX_COLOR_FormatBRCMEGL},
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Thu Sep 27, 2018 9:58 am

Ok, this is the encoding format. But OMX_BRCMVEGLIMAGETYPE is the data struct that i pass to the encoder (through the pBuffer of OMX_BUFFERHEADERTYPE), there is an equivalent struct in MMAL? Or pass directly the vcsm handle to Data field of MMAL_BUFFER_HEADER_T ?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5842
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: EGLImageKHR to OpenMax

Thu Sep 27, 2018 10:10 am

longo92 wrote:
Thu Sep 27, 2018 9:58 am
Ok, this is the encoding format. But OMX_BRCMVEGLIMAGETYPE is the data struct that i pass to the encoder (through the pBuffer of OMX_BUFFERHEADERTYPE), there is an equivalent struct in MMAL? Or pass directly the vcsm handle to Data field of MMAL_BUFFER_HEADER_T ?
Sorry, misread what you were looking for.
I suspect this path has never been used under MMAL. MMAL and IL share the same components with just the frameworks above changing, so the component is still looking for a buffer with the same contents as OMX_BRCMVEGLIMAGETYPE. Either use OMX_BRCMVEGLIMAGETYPE, or recreate your own version for MMAL (pull requests welcome).

However, as you say, if you have already set up the MMAL port format to match what EGL is creating, then you can set MMAL_PARAMETER_ZERO_COPY on the port, allocate a mmal_pool with buffers having a size of 0, and then put the vc_handle into buf->data field and set buf->length and buf->alloc_size appropriately.
There is nothing magical about the buffers that EGL creates via VCSM, so they can be treated as standard pixel buffers.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

longo92
Posts: 35
Joined: Mon Sep 03, 2018 3:45 pm
Contact: Website Skype

Re: EGLImageKHR to OpenMax

Thu Oct 04, 2018 10:08 am

Is possible to pass to the encoder opnmax JPEG del EGL image in the same way of the openMax encoder? does the jpeg encoder supports EGL image?

Return to “OpenGLES”