Is accelerated image proccessing posible?

[BillyBag2]

Can the RPi use the GPU to do accelerated image proccessing without displaying the results on screen?

[jamesh]

What specifically do you want to do?

[BillyBag2]

Things like gamma correction, saturation, HDR, merging two images, colour replacment. If there were multiple input images they would be the same size in pixels and be aligned over eachother. Simple things would be on a per pixel basis. It would be nice to do stuff that was influenced by neighbouring pixels, but being able to do the simple sums on a per-pixel basis would be useful too.

A case study could be merging N aligned images into one to reduce noise, just a simple vector mean.

[jamesh]

The GPU can already do most of those things (in real time at 30fps) using the onboard ISP, but its difficult to get access to it from the Arm core. I suppose it might be possible with some alterations to the OpenMAX interface to allow input to the ISP from a OMX component rather that the onboard camera.

You can use dispmanx to merge images.

Most are not possible from the Arm right now though.

[Janq]

Things like gamma correction, saturation, HDR, merging two images, colour replacment. If there were multiple input images they would be the same size in pixels and be aligned over eachother. Simple things would be on a per pixel basis. It would be nice to do stuff that was influenced by neighbouring pixels, but being able to do the simple sums on a per-pixel basis would be useful too.

A case study could be merging N aligned images into one to reduce noise, just a simple vector mean.

You can do all of these things with OpenGLES2.0 shaders.

Gamma correction:

vec4 t=texture2D(Texture,uv);
t=pow(t,g_Gamma);


Saturation:

vec4 t=texture2D(Texture,uv);
float luminance=dot(t.xyz,vec3(0.2125,0.7154,0.0721));
t.xyz+=((t.xyz-luminance)*g_Saturation);


HDR is a very vague term - you can't actually display high dynamic range images without a high dynamic range monitor and the PI doesn't support those. You probably mean tone mapping? Look that up, its more complex than a few lines of code


Merging two images:

vec4 t1=texture2D(Texture1,uv);
vec4 t2=texture2D(Texture2,uv);
vec4 t=(t1*g_Blend)+(t2*(1.0-g_Blend))


Colour replacement is also a very vague term - there are lots of ways you can do that (eg. just simple chroma keying by comparing with a colour in the shader and using it as a mask - right up to full on colour correction using a 3D colour cube).

Anyway - the GPU is perfectly capable of doing these tasks just with standard GLES.

[BillyBag2]

Anyway - the GPU is perfectly capable of doing these tasks just with standard GLES.

Cool, can you point me at an example? I'm fine with the maths and vertors and stuff. Writing a shader tooks quite simple based on your examples. Also I can lookup getting data out of a file. I guess the example code needs to populate the "textures" with data, "run" the shader then shovel out the result. Thats the bit I have absolutely no idea how to do. Blending is probably a good exmaple as it is multiple input.

[jackokring]

The hardest part appears to be using EGL to make an off screen pbuffer or pixmap to render to, and to get back the result. It's quite a lot of boilerplate code involved.

[Twinkletoes]

Look at the triangle2 demo code in opt/vc/hello_pi. That uses a shader to generate a mandelbrot texture, which you could then read back from the texture buffer using standard openGL.

[BillyBag2]

Look at the triangle2 demo code in opt/vc/hello_pi. That uses a shader to generate a mandelbrot texture, which you could then read back from the texture buffer using standard openGL.

I'll try this, it sounds like a good foot in the door for what I want.