mmal resizer (isp) cropping support offsets?

[aire39]

I'm trying to crop a 1280x720 to 1080x720 with an x offset of 100. I have notice the width scale I set is correct but that the x offset that I set is being ignored. Is the isp able to do a crop with a x offset?

The image is not provided by the camera but by a separate pipeline not connected to the camera component where I manually provide the image.

Code: Select all

   //crop incoming 1280x720 to 1080x720 with an x offset of 100pixels

    MMAL_ES_FORMAT_T* input_format = input_port->format;
    mmal_format_copy(ispComponent->input[0]>format, splitterComponent->output[1]->format);

    input_format->encoding         = MMAL_ENCODING_I420;
    input_format->encoding_variant = MMAL_ENCODING_I420;
    input_format->es->video.width  = VCOS_ALIGN_UP(1280, 32);  // width needs to be a multiple of 32
    input_format->es->video.height = VCOS_ALIGN_UP(720 16); // height needs to be a multiple of 16
    input_format->es->video.crop.x = 100; // <-- This parameter is not being being used when I crop the image
    input_format->es->video.crop.y = 0;
    input_format->es->video.crop.width  = 1080;
    input_format->es->video.crop.height = 720;

Code: Select all

   // scale down the 1080x720 image to 720x480

    MMAL_ES_FORMAT_T* out_format = output_port->format;

    mmal_format_copy(output_port->format, input_port->format);
    out_format->encoding         = MMAL_ENCODING_I420;
    out_format->encoding_variant = MMAL_ENCODING_I420;
    out_format->es->video.width  = VCOS_ALIGN_UP(720, 32);  // width needs to be a multiple of 32
    out_format->es->video.height = VCOS_ALIGN_UP(480, 16); // height needs to be a multiple of 16
    out_format->es->video.crop.x = 0;
    out_format->es->video.crop.y = 0;
    out_format->es->video.crop.width  = 720;
    out_format->es->video.crop.height = 480;

[aire39]

I solved my own question

Code: Select all

    MMAL_PARAMETER_CROP_T crop = {{MMAL_PARAMETER_CROP, sizeof(MMAL_PARAMETER_CROP_T)}, {0,0,0,0}};
	crop.rect.x = 100;
	crop.rect.y = 0;
	crop.rect.width  = 1080;
	crop.rect.height = 720;
	mmal_port_parameter_set(ispComponent->input[0], &crop.hdr);

by setting MMAL_PARAMETER_CROP_T on the input of the ispComponent I was able to crop the image near what I expected. I seem to be 64 pixels off but this seems to be what I'm looking for unless there's a way to use the MMAL_ES_FORMAT_T parameters and I'm just missing something.

Code: Select all

    input_format->es->video.crop.width  = 1080; // change back to full frame size width 1280 (in my case)
    input_format->es->video.crop.height = 720;

The input port format width and height needs to be the same as the actual frame. Basically don't rely on the crop field to crop the image and just rely on the mmal_port_parameter_set function. This fixed my weird 64 pixels being off issue.

[6by9]

Sorry, I don't tend to follow this forum so hadn't seen the question.

Correct that format->es->video.crop.[x|y] are ignored. Many of the GPU components are wrapped over IL components, and that has no concept of an x/y offset.
format->es->video.crop.[width|height] should define the active portion of the buffer.
format->es->video.[width|height] describe the overall size of the buffer, including any padding that may be required by components.
It wouldn't be abnormal to see:

Code: Select all

width = 1920;
height = 1088;
crop.width = 1920;
crop.height = 1080;

for 1080P as the majority of components require the height to be padded to a multiple of 16 lines.
Likewise

Code: Select all

width = 736;
height = 480;
crop.width = 720;
crop.height = 480;

for 480P as the stride generally has to be a multiple of 32 pixels.

Can you provide some more detail on your comment "I seem to be 64 pixels off"? I'm happy to investigate, but need to know what I'm investigating.

[aire39]

I believe the 64 pixels off happened because I was changing the parameters of the crop.[width&height] of the input port of my resizer (using the isp component). I changed the parameters so that the crop.[width&height] of the input port just matched my actual frame [width&height] instead and relied on the mmal_port_parameter_set function only instead which fixed everything.

Is OpenMax supported better than MMAL now and days or should that not matter? I wasn't sure if I could get to most of the features available through openmax so I pretty much did everything under MMAL which works out great and am able to do 720p and 480p streams. I found an okay solution going from camera -> gpu -> encoder.

There's no section for a MMAL based approach here in the forums, that I see, so I'm not really sure where to post in concerns to MMAL. Would it best if I post under OpenMAX forum?

Code: Select all

   frameWidth  = 1280;
   frameHeight = 720;
   
   ...

    MMAL_PORT_T* input_port = resizerComponent->input[0];

    MMAL_ES_FORMAT_T* input_format = input_port->format;
    mmal_format_copy(input_port->format, splitterComponent->output[1]->format);

    input_format->encoding         = MMAL_ENCODING_OPAQUE;
    input_format->encoding_variant = MMAL_ENCODING_I420;
    input_format->es->video.width  = VCOS_ALIGN_UP(frameWidth, 32);  // width needs to be a multiple of 32
    input_format->es->video.height = VCOS_ALIGN_UP(frameHeight, 16); // height needs to be a multiple of 16
    input_format->es->video.crop.x = 0;
    input_format->es->video.crop.y = 0;
    input_format->es->video.crop.width  = 1080; // <-- this should just be set to frameWidth and should rely on setting the 
                                                //MMAL_PARAMETER_CROP_T parameters to the actual crop instead. This seems to cause conflict.
    input_format->es->video.crop.height = frameHeight;
    input_format->es->video.frame_rate.num = 0;
    input_format->es->video.frame_rate.den = 1;

    MMAL_PARAMETER_CROP_T crop = {{MMAL_PARAMETER_CROP, sizeof(MMAL_PARAMETER_CROP_T)}, {0,0,0,0}};
    crop.rect.x = 100;
    crop.rect.y = 0;
    crop.rect.width  = 1080;
    crop.rect.height = frameHeight;
    mmal_port_parameter_set(input_port, &crop.hdr);

[6by9]

I believe the 64 pixels off happened because I was changing the parameters of the crop.[width&height] of the input port of my resizer (using the isp component). I changed the parameters so that the crop.[width&height] of the input port just matched my actual frame [width&height] instead and relied on the mmal_port_parameter_set function only instead which fixed everything.

Sorry, getting lost there between format.es.video.[width|height] and MMAL_PARAMETER_CROP_T. I'm assuming the former as the latter would be crop.rect.[width|height].
Yes, format.es.video.[width|height] should be left describing the input buffer. The cropping rectangle will always be clipped to ensure that it can't try cropping a region that is outside the source image.

Is OpenMax supported better than MMAL now and days or should that not matter? I wasn't sure if I could get to most of the features available through openmax so I pretty much did everything under MMAL which works out great and am able to do 720p and 480p streams. I found an okay solution going from camera -> gpu -> encoder.

No, MMAL is the preferred API. As noted in the docs for the isp component, I've never used it under IL and have little intention of fixing it up should it not work.
MMAL has a fair number of options which give fairly large performance gains over IL, and it generally reports errors as the return code from function calls, instead of IL with the myriad of callbacks that it makes, and vast state machine.

There's no section for a MMAL based approach here in the forums, that I see, so I'm not really sure where to post in concerns to MMAL. Would it best if I post under OpenMAX forum?

Best to avoid the IL forum as it really isn't.
From the forums I regularly follow, in decreasing order of applicability:
- Graphics, sound and multimedia
- Graphics programming
- Camera board
- Advanced users
I do check all the "Hardware and peripherals" sub-forums, and General Discussion too. Others (such as this) I dip into when I'm waiting for something to compile!
I you have an easy to reproduce test case, then you're welcome to post it to https://github.com/raspberrypi/firmware/issues, but we really do want a runnable test case there with a full description of what you're trying to do and the results you're seeing. Those get triaged fairly quickly if a decent report is made.