rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

jpeg decoder

Wed Feb 27, 2013 6:52 am

Hi,
I want to use either OpenGL, OpenCV, or OpenMAX for just reading a jpeg image, and convert it to RGB raw data.
I know that OpenCV is more of a computer vision library and the other ones are more of a rendering tool, but I believe that they all can be used if the purpose is just for decoding the image, is this right?
Please feel free to correct me.

Now, if the purpose is just pure decoding an nothing more, then how do I know which library will perform relatively faster than the other ones?
All I care about is to have a faster decoder. Can someone tell me which one should I use?

Also, now that we have both CPU and GPU, how can I make use of the availability of GPU, and make sure that decoding will be handed over to GPU for faster processing?
Is there a certain library that I can use to make sure that the decoding will be done in GPU?

Is GPU only helpful, if we are trying to display images? Or it could be equally helpful even if we are only doing decoding of jpeg images, without displaying them?

Basically, how to utilize GPU to its fullest when it comes to decoding images? And, which library will be the fastest? How about libjpeg library?

I would appreciate some help.

Thanks,
--Rudy

OtherCrashOverride
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am

Re: jpeg decoder

Wed Feb 27, 2013 8:14 am

OpenCV offers nothing to the PI because there is no SIMD available on it. OpenGL does not provide any support for JPEG at all. That leaves OpenMax IL which does provide access to a JPEG codec. Examples usage can be found here https://github.com/raspberrypi/firmware ... hello_jpeg

You will have to test and measure to find out what performance advantages it offers over libjpeg.

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: jpeg decoder

Wed Feb 27, 2013 3:20 pm

Okay thanks.
But how OpenMAX is able to do some acceleration if there is no Neon SIMD support in pi?
By the way, I am not sure what this VPU support is that is mentioned in this post:
http://www.raspberrypi.org/phpBB3/viewt ... 24&t=31004

If there is truly a VPU available, how can I make use of it with OpenMAX?
And, even though OpenMAX IL has codec libraries, but OpenMAX DL has accelerated Image Codecs (http://en.wikipedia.org/wiki/Openmax). Can PI make use of this library?
How would it accelerate it, if there is no SIMD support?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23888
Joined: Sat Jul 30, 2011 7:41 pm

Re: jpeg decoder

Wed Feb 27, 2013 3:23 pm

The supplied OpenMAX components run on the GPU (where appropriate) and use the VPU on that (or HW blocks for even more speed).

It is not possible to write your own OMX components and run them on the GPU though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: jpeg decoder

Wed Feb 27, 2013 3:47 pm

Thanks for the reply.
Okay that makes sense, but I guess my question is more about understanding how OpenMAX is able to off load tasks to GPU when appropriate?
The reason is because I am planning on writing my own DSP algorithm in C/C++ to do some eye detection, and I want to know how can I offload more tasks to GPU when appropriate?
And since there is no SIMD support, I am not really sure how to do this !!!
And, since OpenMAX was able to achieve some hardware acceleration, I want to see what guidelines need to be followed to achieve a certain level of hardware optimization in Pi?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23888
Joined: Sat Jul 30, 2011 7:41 pm

Re: jpeg decoder

Wed Feb 27, 2013 4:58 pm

rudy01 wrote:Thanks for the reply.
Okay that makes sense, but I guess my question is more about understanding how OpenMAX is able to off load tasks to GPU when appropriate?
The reason is because I am planning on writing my own DSP algorithm in C/C++ to do some eye detection, and I want to know how can I offload more tasks to GPU when appropriate?
And since there is no SIMD support, I am not really sure how to do this !!!
And, since OpenMAX was able to achieve some hardware acceleration, I want to see what guidelines need to be followed to achieve a certain level of hardware optimization in Pi?
As a third party you will not be able to write code that runs on the GPU. Its a closed binary blob. So you are limited to the Arm.

There may be components already present that can do work though, depending on what you want to do.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: jpeg decoder

Wed Feb 27, 2013 6:21 pm

Thanks again for the reply.
Lastly, want thing that I would like to know is that if and only if I am using OpenMAX to deocde images, and nothing else, then in that case will decoding the image be done on GPU? (Remember that I am not willing to display or use any other graphical tools for viewing or rendering, and instead i will be developing my own DSP algorithm.)

The reason I am asking is because if the answer is no, then I would just use OpenCV, since I have some minor experience with it in the past.
Basically, Will I gain any performance advantage (e.g. speed) for ONLY decoding an image using OpenCV vs OpenMAX? (in my scenario, I will be decoding images frequently almost once every second)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23888
Joined: Sat Jul 30, 2011 7:41 pm

Re: jpeg decoder

Wed Feb 27, 2013 8:27 pm

Most OpenMAX components run on the GPU - so JPEG encode/decode, video preview, Video encode/decode, camera etc. It not just running on a DSP in the GPU - there a many HW blocks dedicated to these tasks, plus a twin core scaler/vector processor to plug them altogether.

However, audio stuff I believe is done on the Arm because of the high monetary cost of having audio codecs running on the GPU.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

OtherCrashOverride
Posts: 582
Joined: Sat Feb 02, 2013 3:25 am

Re: jpeg decoder

Wed Feb 27, 2013 8:57 pm

As a point of clarification, yes, you can use openmax to encode/decode jpeg images without the requirement they be displayed. The data is read from/written to memory buffers.

rudy01
Posts: 38
Joined: Tue Feb 19, 2013 3:16 am

Re: jpeg decoder

Wed Feb 27, 2013 9:24 pm

Ok thanks a lot for the explanation.
--Rudy

alik55
Posts: 1
Joined: Mon Feb 16, 2015 2:28 am

Re: jpeg decoder

Wed Feb 18, 2015 2:33 am

Sorry to bump an old thread. It appears that the hw decoder is not capable of decoding jpegs without loss of quality - it appears the image_decode outputs only YUV420 format which uses 6 bytes for every 4 pixels instead of 3 bytes/pixel as in RGB888. I used jpeg-decoder.c from:
http://jan.newmarch.name/LinuxSound/Div ... PiOpenMAX/
It does not use resizer and prints the color format output by the decoder as 0x14 which is YUV420.

Documentation for OMX.broadcom.image_decode (http://home.nouwen.name/RaspberryPi/doc ... ecode.html) states that:
The input port supports several compressed formats, the output port supports several colour formats. However, the component doesn't support colour format changing, so the colour format will be set to match that emitted by the encountered image format.

So it would like the decoder must output whatever color format is on the input where I have jpeg sampled at 4:4:4 and not 4:2:0.

Can someone confirm that or am I missing something?

Return to “Graphics programming”