madCoder24
Posts: 18
Joined: Wed Feb 03, 2016 11:38 pm

OpenMax video_render scaling output

Sun Nov 20, 2016 4:04 pm

I haven't been able to find an answer to this question, and my searching of the forums hasn't yielded much results. I am asking about scaling video using OpenMax. Will OpenMax scale the video accordingly to match the TV native resolution? I am currently sending video with dimensions of 1600x900. How can I get OpenMax to scale the video up to the native TV resolution (1920x1080)? I am trying to use a OMX_CONFIG_DISPLAYREGIONTYPE to try and scale the video and this is set to the video_render component. It doesn't appear to be working though, so is there a different solution?

Thanks for any advice.

Current solution:

Code: Select all

OMX_CONFIG_DISPLAYREGIONTYPE display_region;
memset(&display_region, 0, sizeof(OMX_CONFIG_DISPLAYREGIONTYPE));
display_region.nPortIndex = 90;
display_region.set = (OMX_DISPLAYSETTYPE)(OMX_DISPLAY_SET_NUM | OMX_DISPLAY_SET_FULLSCREEN | OMX_DISPLAY_SET_MODE | OMX_DISPLAY_SET_DEST_RECT);
display_region.num = 0;
display_region.fullscreen = OMX_TRUE;
display_region.src_rect.width = 1600;
display_region.src_rect.height = 900;
display_region.src_rect.x_offset = 0;
display_region.src_rect.y_offset = 0;
display_region.mode = OMX_DISPLAY_MODE_FILL;
display_region.dest_rect.width = 1920;
display_region.dest_rect.height = 1080;
display_region.dest_rect.x_offset = 0;
display_region.dest_rect.y_offset = 0;

if(OMX_SetConfig(ilclient_get_handle(decoder.video_render), OMX_IndexConfigDisplayRegion, &display_region))
{
     MadLink_Logger::debug("failed to set display region!");
}

ilclient_change_component_state(decoder.video_render, OMX_StateExecuting);

dickon
Posts: 216
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 4:35 pm

There's a resize component you can drop into the pipeline, between the decoder and renderer. Just configure its input port parameters to the same as the decoder's output, and the output port to the native screen size, and it should all work.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4447
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 5:24 pm

Using resize at these resolutions will consume a very large amount of the GPU processing capability, and is unnecessary in this case.

Yes, video_render is controlling the Hardware Video Scaler hardware block in the SoC, which is totally capable of resizing as required. It's exactly what OMXPlayer and hello_video will do, and also any of the MMAL apps that use video_render (the same code is shared on the GPU side, it just changes the API).
You haven't set OMX_DISPLAY_SET_SRC_RECT in display_region.set, so the src_rect will be ignored. Generally not a problem as you're wanting the full source image displayed anyway.

You don't say what bit doesn't work or what your results are. Just setting display_region.fullscreen = OMX_TRUE; and display_region.set = OMX_DISPLAY_SET_FULLSCREEN; should be sufficient to get it fullscreen.

FYI The default mode is OMX_DISPLAY_MODE_LETTERBOX, and it will retain the aspect ratio of the image unless you set noaspect = OMX_TRUE; and mode |= OMX_DISPLAY_SET_NOASPECT;
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

madCoder24
Posts: 18
Joined: Wed Feb 03, 2016 11:38 pm

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 6:15 pm

The resulting image is displaying fullscreen, but it isn't showing the entire picture. Parts of the picture are hidden from view, due to the image not being scaled properly on the decoding side. Should I continue to use OMX_CONFIG_DISPLAYREGIONTYPE? Am I doing something incorrectly on the encoding side? Maybe not scaling correctly, before sending?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4447
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 7:12 pm

Overscan settings in config.txt?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

madCoder24
Posts: 18
Joined: Wed Feb 03, 2016 11:38 pm

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 7:19 pm

The only settings active in my config.txt are the following: "disable_overscan=1", "dtparam=audio=on", "hdmi_force_hotplug=1".

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4447
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 9:39 pm

You can check what the applied display list is. On my Pi:

Code: Select all

pi@raspberrypi:~ $ vcgencmd dispmanx_list
display:0 format:XRGB8888 transform:0 layer:-127 src:0,0,800,480 dst:0,0,800,480 cost:534 lbm:0
display:0 format:YUV_UV transform:0 layer:2 src:0,0,3264,2448 dst:80,0,640,480 cost:4864 lbm:10240
That's with the frame buffer (XRGB8888 layer) and raspivid running at full resolution (it was a command I had kicking about in my history), and hooked up to the 7" LCD, hence 800x480 as the output resolution.

Possibly worth checking it isn't your display overscanning - some do if in a CEA mode. Is the framebuffer displayed correctly on your display without any cropping?
Any applied overscan settings will be applied to the "dst" values displayed. For example the frame buffer when I've set 160 for all the overscan settings in config.txt I see:

Code: Select all

pi@raspberrypi:~ $ vcgencmd dispmanx_list
display:0 format:XRGB8888 transform:0 layer:-127 src:0,0,480,160 dst:160,160,480,160 cost:356 lbm:0
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

madCoder24
Posts: 18
Joined: Wed Feb 03, 2016 11:38 pm

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 10:06 pm

My applied display list is:

Code: Select all

pi@raspberrypi:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,1920,1080 dst:                    0,0,1920,1080 cost:1156 lbm:0
Is the framebuffer displayed correctly on your display without any cropping?
I believe so... If it helps at all, I changed my laptop resolution back to 1920x1080 and streamed to the dongle and the entire picture showed up as it should. But, if I switch it back to 1600x900 then I have part of the picture hidden not seen again on the RaspberryPi display. My OpenMax setup is currently programmed to accept YUV. If that makes a difference at all for this discussion.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4447
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax video_render scaling output

Sun Nov 20, 2016 10:48 pm

madCoder24 wrote:My applied display list is:

Code: Select all

pi@raspberrypi:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,1920,1080 dst:                    0,0,1920,1080 cost:1156 lbm:0
That's obviously not with your video render running, as that listing would appear to only be the frame buffer. Knowing the list when video_render is running would be useful.
madCoder24 wrote:
Is the framebuffer displayed correctly on your display without any cropping?
I believe so... If it helps at all, I changed my laptop resolution back to 1920x1080 and streamed to the dongle and the entire picture showed up as it should. But, if I switch it back to 1600x900 then I have part of the picture hidden not seen again on the RaspberryPi display. My OpenMax setup is currently programmed to accept YUV. If that makes a difference at all for this discussion.
What dongle? What streaming? Sorry, I'm not getting your setup at the moment.

The Pi output resolution is configured predominantly via tvservice. "tvservice -s" would confirm the mode that the display output is currently set to. I'm assuming that isn't getting changed.
How much of the image are you losing? If you're changing settings on a laptop that is doing the streaming, have you ensured that you're sending what you think you're sending? You're happy your laptop isn't cropping before sending?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

madCoder24
Posts: 18
Joined: Wed Feb 03, 2016 11:38 pm

Re: OpenMax video_render scaling output

Mon Nov 21, 2016 12:13 am

Okay, my corrected display list with video_render running is:

Code: Select all

pi@raspberrypi:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,1920,1080 dst:0,0,1920,1080 cost:1156 lbm:0
display:2 format:YUV_UV transform:0 layer:0 src:0,0,1600,900 dst:0,0,1920,1080 cost:1216 lbm:25600
I also ran the command "tvservice -s" below while video_render was running:

Code: Select all

pi@raspberrypi:~ $ tvservice -s
state 0x12000a [HDMI CEA (16) RGB lim 16:9], 1920x1080 @ 60.00Hz, progressive
What dongle? What streaming? Sorry, I'm not getting your setup at the moment.
To clarify my setup, I am streaming from a laptop to a raspberry pi. The streaming laptop first captures the desktop, then encodes the capture image with x264 and then streams that to the raspberry pi. The raspberry pi then renders the received frame.

I also double checked my laptop to make sure it is capturing the desktop correctly. It is capturing the entire desktop, correctly. The resulting images are below:

Laptop capture:
https://imagebin.ca/32gVVqVvowuK/laptop_capture.jpg

RaspberryPi Output:

https://imagebin.ca/32gVpW2OrJuv/raspberrypi_output.jpg

Hopefully these images are helpful.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4447
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax video_render scaling output

Mon Nov 21, 2016 2:26 pm

madCoder24 wrote:Okay, my corrected display list with video_render running is:

Code: Select all

pi@raspberrypi:~ $ vcgencmd dispmanx_list
display:2 format:XRGB8888 transform:0 layer:-127 src:0,0,1920,1080 dst:0,0,1920,1080 cost:1156 lbm:0
display:2 format:YUV_UV transform:0 layer:0 src:0,0,1600,900 dst:0,0,1920,1080 cost:1216 lbm:25600
I also ran the command "tvservice -s" below while video_render was running:

Code: Select all

pi@raspberrypi:~ $ tvservice -s
state 0x12000a [HDMI CEA (16) RGB lim 16:9], 1920x1080 @ 60.00Hz, progressive
That all looks sane on the assumption that the laptop is producing 1600x900.
madCoder24 wrote:
What dongle? What streaming? Sorry, I'm not getting your setup at the moment.
To clarify my setup, I am streaming from a laptop to a raspberry pi. The streaming laptop first captures the desktop, then encodes the capture image with x264 and then streams that to the raspberry pi. The raspberry pi then renders the received frame.

I also double checked my laptop to make sure it is capturing the desktop correctly. It is capturing the entire desktop, correctly. The resulting images are below:

Laptop capture:
https://imagebin.ca/32gVVqVvowuK/laptop_capture.jpg

RaspberryPi Output:
https://imagebin.ca/32gVpW2OrJuv/raspberrypi_output.jpg

Hopefully these images are helpful.
Yes, knowing what bits are being cropped does help greatly. It looks like it is correctly aligned on the top left, but then falling off the bottom and right edges of the screen - that's certainly not just an overscan issue.
How have you generated the screen captures? The laptop output one is 1600x900, but the Pi one is 1,331x749.

All the information you've provided says to me that your laptop isn't encoding what you think. Can you capture the stream and confirm using avprobe or similar that it is 1600x900.

My other thought is that you are encoding/streaming 1920x1080, but with that source rectangle of 1600x900 specified on video_render then you are telling it to crop the top left 1600x900 pixels from that screen. Try NOT specifying the source or destination rectangles and just let the GPU make it full screen, and also dump the PortDefinition of the decoder output port for your confirmation of what it believes the image format to be.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “OpenMAX”

Who is online

Users browsing this forum: No registered users and 2 guests