sanoj
Posts: 2
Joined: Sat Oct 13, 2018 9:36 am

MMAL RGB to JPEG encoder width issues

Sat Oct 13, 2018 10:03 am

Hi,

I'm using my raspberry pi 3 model B alongisde with the camera module for an image recognition project.
For debugging my analysis I want to publish the captured and processed images through a REST API in JPEG format.
As I internally use RGB24 as image format I wrote a MMAL based encoder to make use of the JPEG compression hardware. I attached the source of the encoder to this post.

To my knowledge the MMAL_COMPONENT_DEFAULT_IMAGE_ENCODER requires that the input image buffers width is a multiple of 32.
However it should be able to crop up to 31 pixels from the right of the image to allow any size of images to be processed.

Below is the relevant code snippet for the input port configuration from my ImageEncoder class:

Code: Select all

size_t upWidth = VCOS_ALIGN_UP(width, 32);
size_t upHeight = VCOS_ALIGN_UP(height, 16);

auto in = encoderComp->input[0];
auto out = encoderComp->output[0];

in->format->encoding = MMAL_ENCODING_RGB24;
in->format->es->video.width = upWidth;
in->format->es->video.height = upHeight;
in->format->es->video.crop.x = 0;
in->format->es->video.crop.y = 0;
in->format->es->video.crop.width = width;
in->format->es->video.crop.height = height;

auto status = mmal_port_format_commit(in);
if (status != MMAL_SUCCESS) {
	log->error() << "Error commiting input port format: " << status;
	throw std::runtime_error("Error in encoder");
}
Everything works as expected as long as the encoder has to crop less than 16 pixels. However, when configuring it to crop 16 to 31 pixels, the MMAL API rejects the port configuration. The console output is as follows:

Code: Select all

mmal: mmal_vc_port_info_set: failed to set port info (2:0): EINVAL
mmal: mmal_vc_port_set_format: mmal_vc_port_info_set failed 0x1249e0 (EINVAL)
ERROR: Error commiting input port format: 3
I discovered this issue as I am using the camera module v2 in 2x2 binned mode with the full sensor. Therefore the width of my images is 1640 pixels. When upping this to a multiple of 32 the buffer width becomes 1664, meaning that 24 pixels have to be cropped at the right.

My question is now: Is this a bug, a hardware limitation or am I doing something wrong here?
Attachments
ImageEncoder.zip
(2.7 KiB) Downloaded 7 times

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5801
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: MMAL RGB to JPEG encoder width issues

Sat Oct 13, 2018 9:19 pm

Not something I can test right now, but I will look when I'm in the office again.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5801
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: MMAL RGB to JPEG encoder width issues

Mon Oct 15, 2018 1:15 pm

Sorry, your code is near useless without a Makefile or some form of test harness. Fortunately a one line change to raspistill changing the encoding on the camera stills port can exercise the same path.

It looks like image_encode got patched back in 2012 to support unaligned strides, but in the process resulted in the stride having to be the internal aligned value, which is aligned to a multiple of 16 in the case of RGB888 (32 for YUV formats).
I'll look at getting the firmware patched.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

sanoj
Posts: 2
Joined: Sat Oct 13, 2018 9:36 am

Re: MMAL RGB to JPEG encoder width issues

Mon Oct 15, 2018 3:58 pm

Thanks for the quick answer and sorry for not providing a buildable test case as i did not want to upload my entire project here.
I'm glad you were able to reproduce it anyway.
It looks like image_encode got patched back in 2012 to support unaligned strides, but in the process resulted in the stride having to be the internal aligned value, which is aligned to a multiple of 16 in the case of RGB888 (32 for YUV formats).
So I should align the stride to a multiple of 16 pixels when you have patched the firmware, right?
Or does this mean that no stride alignment at all is then necessary anymore?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5801
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: MMAL RGB to JPEG encoder width issues

Mon Oct 15, 2018 8:13 pm

sanoj wrote:
Mon Oct 15, 2018 3:58 pm
It looks like image_encode got patched back in 2012 to support unaligned strides, but in the process resulted in the stride having to be the internal aligned value, which is aligned to a multiple of 16 in the case of RGB888 (32 for YUV formats).
So I should align the stride to a multiple of 16 pixels when you have patched the firmware, right?
Or does this mean that no stride alignment at all is then necessary anymore?
The fix I'm intending should allow "width = crop.width" , or "width >= crop.width && !(width & (align-1)" where align is 16 for RGB formats, and 32 for YUV formats.
If using "width = crop.width" then there is a performance penalty as it has to be adjusted.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”