8
$\begingroup$

I am attempting to convert YV12 to RGB using the following fragment shader:

precision mediump float;
uniform sampler2D tex0, tex1, tex2;
varying vec2 v_texCoord;

void main(void) {
    float r, g, b, y, u, v;
    y = texture2D(tex0, v_texCoord).x;
    u = texture2D(tex1, v_texCoord).x;
    v = texture2D(tex2, v_texCoord).x;

    y = 1.1643 * (y - 0.0625);
    u = u - 0.5;
    v = v - 0.5;

    r = y + 1.5958 * v;
    g = y - 0.39173 * u - 0.81290 * v;
    b = y + 2.017 * u;

    gl_FragColor = vec4(r, g, b, 1.0);
}

Below is the original image followed by the converted image. Based on the output, can you determine where I might look to determine what is causing the improper decoding?

Source Decoded image

$\endgroup$
  • $\begingroup$ As said by first responders, there is definitely a problem with the number of bytes per line. I wonder another thing. YV12 has subsampled U and V channels. How do you manage this dimension change with your shader? What are the sizes of your textures? Do you let OpenGL handle this? $\endgroup$ – Stéphane Péchard Oct 5 '11 at 9:40
  • $\begingroup$ In the draw loop, image data is given to me in a 3 element array where the order is always YUV (regardless of YV12 or YUV420p etc). I use the full width/height for the Y element and shift the width/height right 1 (divide by 2) for the UV elements. These textures are then bound to the shader which handles the rest. Texture sizes that I've been working with are typically 1280x720. $\endgroup$ – Error 454 Oct 5 '11 at 17:21
5
$\begingroup$

The output image has lines shifted one against another, which is typically due to incorrect [byte] distance between the lines (stride). That is, the color conversion math is perhaps correct, however you are either taking source pixel from wrong position and/or putting output pixel into wrong place.

For performance reasons YV12 images might be using extended strides in hardware to facilitate performance. For example, 320x240 YV12 image might have a stride of 512 and this means that the data buffer is a 512x240 data space (512x240 bytes for Y, followed by two 256x120 arrays for V and U) where the payload image on the left, and there is invisible padding on the right.

$\endgroup$
  • $\begingroup$ I do see that I am not accounting for stride. I will implement this when I can and update the question once I can verify if this is indeed the problem. $\endgroup$ – Error 454 Oct 5 '11 at 17:24
  • $\begingroup$ In the end, stride was the answer. It was the only thing I changed and now everything works. Great eye! $\endgroup$ – Error 454 Oct 8 '11 at 0:33
1
$\begingroup$

I don't know anything about fragment shader programming, but to me it looks like the word size of the input and output vary. And to me it looks like, you don't pay attention to unpacking and repacking the YV12 bytes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.