paulslocum
Posts: 75
Joined: Sun Apr 09, 2017 10:54 pm

Help grabbing Omxplayer video frames using OMX.broadcom.video_splitter

Mon Aug 05, 2019 9:20 am

I was able to successfully tunnel a "video_splitter" component into Omxplayer's components between the video scheduler and renderer, and I set up a "FillBufferDone" callback on an unused video_splitter output port, but I don't understand the data I'm getting back.

The size of the buffer corresponds to the video's resolution, and the information in the buffer appears to correspond to color information in the video, but it's only giving me 1382400 bytes for a 1280x720 video -- that's 1.5 bytes per pixel. I was expecting 24 bit YUV, and I thought that only the "resizer" component could change formats, so I'd appreciate any additional info that anyone has about what's going on here.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23372
Joined: Sat Jul 30, 2011 7:41 pm

Re: Help grabbing Omxplayer video frames using OMX.broadcom.video_splitter

Mon Aug 05, 2019 9:31 am

paulslocum wrote:
Mon Aug 05, 2019 9:20 am
I was able to successfully tunnel a "video_splitter" component into Omxplayer's components between the video scheduler and renderer, and I set up a "FillBufferDone" callback on an unused video_splitter output port, but I don't understand the data I'm getting back.

The size of the buffer corresponds to the video's resolution, and the information in the buffer appears to correspond to color information in the video, but it's only giving me 1382400 bytes for a 1280x720 video -- that's 1.5 bytes per pixel. I was expecting 24 bit YUV, and I thought that only the "resizer" component could change formats, so I'd appreciate any additional info that anyone has about what's going on here.
You are getting YUV420, with the U and V channels 'compressed'. Google will find the spec for it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7143
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Help grabbing Omxplayer video frames using OMX.broadcom.video_splitter

Mon Aug 05, 2019 10:54 am

Chroma subsampling is the phrase that James was looking for.
https://en.wikipedia.org/wiki/Chroma_subsampling#4:2:0
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23372
Joined: Sat Jul 30, 2011 7:41 pm

Re: Help grabbing Omxplayer video frames using OMX.broadcom.video_splitter

Mon Aug 05, 2019 10:55 am

6by9 wrote:
Mon Aug 05, 2019 10:54 am
Chroma subsampling is the phrase that James was looking for.
https://en.wikipedia.org/wiki/Chroma_subsampling#4:2:0
Yup. It's a sort of compression.....!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

paulslocum
Posts: 75
Joined: Sun Apr 09, 2017 10:54 pm

Re: Help grabbing Omxplayer video frames using OMX.broadcom.video_splitter

Mon Aug 05, 2019 12:14 pm

Thanks, I appreciate the quick response! Is this the correct data format at this link:

https://en.wikipedia.org/wiki/YUV#Y%E2% ... conversion

Pasted from the article:

Code: Select all

size.total = size.width * size.height;
y = yuv[position.y * size.width + position.x];
u = yuv[(position.y / 2) * (size.width / 2) + (position.x / 2) + size.total];
v = yuv[(position.y / 2) * (size.width / 2) + (position.x / 2) + size.total + (size.total / 4)];
rgb = Y′UV420toRGB888(y, u, v);

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5318
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Help grabbing Omxplayer video frames using OMX.broadcom.video_splitter

Mon Aug 05, 2019 1:05 pm

That is the correct layout.

Note, it only works for nicely aligned size.width, size.height (multiples of 16 or 32). 1280x720 should be okay.

Otherwise you need to use aligned_width and aligned_height from component in the formula and treat width and height as crop values
(pixels between width and aligned_width are padding).

Return to “Graphics programming”