YUV to RGB conversion by fragment shader
android, camera, fragment-shader, opengl-es, shader
Solution
Not sure if you have already fixed this problem.My answer
- By default Camera output is NV12, but in fragment shader YUV to RGB you are using YV12 -> RGB. You will have to do `setPreviewFormat(ImageFormat.YV12);`, or may be use some other shader
There are 3 textures , make sure you do
`GLES20.glActiveTexture(GLES20.GL_TEXTURE2);` `GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, muTextureName)`
before call to any glTexImage2D. and glTexSubImage2D
You can also use glTexSubImage2D with every frame and glTexImage2D once.
size of U and V is same , atleast for YV12,
`System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2);`
should be `System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4);` change the size accordingly in the code.
Problem
I've a problem with convertion of camera preview in Android from YUV format to RGB. The purpose of conversion is to apply some effects. I try to convert by fragment shader because convertion by native code is slow (about 14fps). The reference which I've used is http://jyrom.tistory.com/m/post/view/id/187. I try to port this code to Android platform, but the result is black-green rectangles. But, I can watch some form through the output which I get. Could you please try to help me to resolve this issue. I believe this is popular problem: apply effects to camera preview. I also give a link to my project for testing: https://dl.dropbox.com/u/12829395/application/FilterGL/FilterGL.zip. Thank you. UPDATED: This is my onPreviewFrame method: ``` public void onPreviewFrame(byte[] data, Camera camera) { yBuffer.put(data); yBuffer.position(0); System.arraycopy(data, U_INDEX, uData, 0, LENGTH_4 * 2); uBuffer.put(uData); uBuffer.position(0); System.arraycopy(data, V_INDEX, vData, 0, LENGTH_4); vBuffer.put(vData); vBuffer.position(0); } ``` This is how I bind byte arrays to OpenGL texture in onDrawFrame method: ``` GLES20.glUniform1i(yTexture, 1); GLES20.glTexImage2D( GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, 320, 240, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, yBuffer); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); GLES20.glUniform1i(uTexture, 2); GLES20.glTexImage2D( GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, 160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, uBuffer); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); GLES20.glUniform1i(vTexture, 3); GLES20.glTexImage2D( GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, 160, 120, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, vBuffer); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); ``` And this is my fragment shader code: ``` #ifdef GL_ES precision highp float; #endif varying vec2 v_texCoord; uniform sampler2D y_texture; uniform sampler2D u_texture; uniform sampler2D v_texture; void main() { float nx,ny,r,g,b,y,u,v; nx=v_texCoord.x; ny=v_texCoord.y; y=texture2D(y_texture,v_texCoord).r; u=texture2D(u_texture,v_texCoord).r; v=texture2D(v_texture,v_texCoord).r; y=1.1643*(y-0.0625); u=u-0.5; v=v-0.5; r=y+1.5958*v; g=y-0.39173*u-0.81290*v; b=y+2.017*u; gl_FragColor = vec4(r,g,b,1.0); } ```