ilker.cakir
Posts: 3
Joined: Mon Nov 07, 2016 9:12 am

using multiple samplers in fragment shader

Mon Nov 07, 2016 9:41 am

Hi all,
I have been struggling to get a YUV->RGB conversion on the Raspberry PI 3 GPU using 3 sampler2D uniforms in the fragment shader and outputting one colour value as described in many articles I have found so far. Unfortunately I could not succeed. The output shows in correct size and luminance but color values are missing, giving the resulting picture a greenish look all over. Converting to RGB from YUV by intensive calculations using the CPU and rendering the resulting RGB texture by a shader works well, only if I use a single sampler2D uniform as input to the shader. I'm now left to thinking multiple samplers are not supported on the PI 3. Can multiple textures be used on the PI with opengl ES? Anything I'm missing? Any help is appreciated. Thanks in advance...

ilker.cakir
Posts: 3
Joined: Mon Nov 07, 2016 9:12 am

Re: using multiple samplers in fragment shader

Wed Nov 09, 2016 2:31 pm

Hi again, I could find a workaround by concatenating Y,U and V data frames into a single data chunk by memcpy() and using the following fragment shader:

"precision mediump float; \n"
"varying vec2 v_texCoord; \n"
"uniform vec2 texSize; \n"
"uniform sampler2D texture; \n"
"void main() \n"
"{ \n"
" float y, u, v, r, g, b; \n"
" float yx, yy, ux, uy, vx, vy; \n"

" vec2 texCoord=vec2(v_texCoord.x*texSize.x, v_texCoord.y*texSize.y);\n"
" float oe=mod(floor(texCoord.y/2.0), 2.0); \n"

" yx=v_texCoord.x; \n"
" yy=v_texCoord.y*2.0/3.0; \n"
" y=texture2D(texture, vec2(yx, yy)).s; \n"

" ux=oe*0.5+v_texCoord.x/2.0; \n"
" uy=4.0/6.0+v_texCoord.y/6.0; \n"
" u=texture2D(texture, vec2(ux, uy)).s; \n"

" vx=ux; \n"
" vy=5.0/6.0+v_texCoord.y/6.0; \n"
" v=texture2D(texture, vec2(vx, vy)).s; \n"

" y=1.1643*(y-0.0625); \n"
" u=u-0.5; \n"
" v=v-0.5; \n"
" r=y+1.5958*v; \n"
" g=y-0.3917*u-0.8129*v; \n"
" b=y+2.017*u; \n"

" gl_FragColor=vec4(r, g, b, 1.0); \n"
"} \n";

I'm planning to improve this by using all s,t,p,q instead of just s, which will increase resolution.

Regards,

ilker.cakir
Posts: 3
Joined: Mon Nov 07, 2016 9:12 am

Re: using multiple samplers in fragment shader

Sat Jan 20, 2018 9:11 am

Finally figured out how to use multiple textures on the Pi using gl with X11, here is how:

[Cover][https://github.com/ilkercakir/Cover]

Here is an alternative solution that uses gl with cairo:

[Cover.cairo.pi][https://github.com/ilkercakir/Cover.cairo.pi]

You may find YUV to RGB conversion on the GPU using Y,U,V as multiple texture inputs.

Return to “Advanced users”

Who is online

Users browsing this forum: No registered users and 19 guests