Correct YUV422 to RGB conversion

Question

I've been trying to tackle a YUV422 into a RGB conversion problem for about a week. I've visited many different websites and have gotten different formulas from each one. If anyone else has any suggestions I would be glad to hear about them. The formulas below give me an image with either and overall purple or a green hue in them. As of this moment I haven't been able to find a formula that allows me to get back a proper RGB image. I have include all my various chunks of code below.

    //for(int i = 0; i < 1280 * 720 * 3; i=i+3)
    //{
    //  /*m_RGB->imageData[i] = pData[i] + pData[i+2]*((1 - 0.299)/0.615);
    //  m_RGB->imageData[i+1] = pData[i] - pData[i+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[i+2]*((0.299*(1 - 0.299))/(0.615*0.587));
    //  m_RGB->imageData[i+2] = pData[i] + pData[i+1]*((1 - 0.114)/0.436);*/

    //  m_RGB->imageData[i] = pData[i] + 1.403 * (pData[i+1] - 128);
    //  m_RGB->imageData[i+1] = pData[i] + 0.344 * (pData[i+1] - 128) - 0.714 * (pData[i+2] - 128);
    //  m_RGB->imageData[i+2] = pData[i] + 1.773 * (pData[i+2] - 128);
    //}

    for(int i = 0, j=0; i < 1280 * 720 * 3; i+=6, j+=4)
    {
        /*m_RGB->imageData[i] = pData[j] + pData[j+3]*((1 - 0.299)/0.615);
        m_RGB->imageData[i+1] = pData[j] - pData[j+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[j+3]*((0.299*(1 - 0.299))/(0.615*0.587));
        m_RGB->imageData[i+2] = pData[j] + pData[j+1]*((1 - 0.114)/0.436);
        m_RGB->imageData[i+3] = pData[j+2] + pData[j+3]*((1 - 0.299)/0.615);
        m_RGB->imageData[i+4] = pData[j+2] - pData[j+1]*((0.114*(1-0.114))/(0.436*0.587)) - pData[j+3]*((0.299*(1 - 0.299))/(0.615*0.587));
        m_RGB->imageData[i+5] = pData[j+2] + pData[j+1]*((1 - 0.114)/0.436);*/

        /*m_RGB->imageData[i] = pData[j] + 1.403 * (pData[j+3] - 128);
        m_RGB->imageData[i+1] = pData[j] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128);
        m_RGB->imageData[i+2] = pData[j] + 1.773 * (pData[j+1] - 128);
        m_RGB->imageData[i+3] = pData[j+2] + 1.403 * (pData[j+3] - 128);
        m_RGB->imageData[i+4] = pData[j+2] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128);
        m_RGB->imageData[i+5] = pData[j+2] + 1.773 * (pData[j+1] - 128);*/

        BYTE Cr = pData[j+3] - 128;
        BYTE Cb = pData[j+1] - 128;
        /*m_RGB->imageData[i] = pData[j] + Cr + (Cr >> 2) + (Cr >> 3) + (Cr >> 5);
        m_RGB->imageData[i+1] = pData[j] - ((Cb >> 2) + (Cb >> 4) + (Cb >> 5)) - ((Cr >> 1) + (Cr >> 3) + (Cr >> 4) + (Cr >> 5));
        m_RGB->imageData[i+2] = pData[j] + Cb + (Cb >> 1) + (Cb >> 2) + (Cb >> 6);
        m_RGB->imageData[i+3] = pData[j+2] + Cr + (Cr >> 2) + (Cr >> 3) + (Cr >> 5);
        m_RGB->imageData[i+4] = pData[j+2] - ((Cb >> 2) + (Cb >> 4) + (Cb >> 5)) - ((Cr >> 1) + (Cr >> 3) + (Cr >> 4) + (Cr >> 5));
        m_RGB->imageData[i+5] = pData[j+2] + Cb + (Cb >> 1) + (Cb >> 2) + (Cb >> 6);*/

        /*int R1 = clamp(1 * pData[j] + 0 * Cb + 1.4 * Cr, 0, 255), R2 = clamp(1 * pData[j+2] + 0 * Cb + 1.4 * Cr, 0, 255);
        int G1 = clamp(1 * pData[j] - 0.343 * Cb - 0.711 * Cr, 0, 255), G2 = clamp(1 * pData[j+2] - 0.343 * Cb - 0.711 * Cr, 0, 255);
        int B1 = clamp(1 * pData[j] + 1.765 * Cb + 0 * Cr, 0, 255), B2 = clamp(1 * pData[j+2] + 1.765 * Cb + 0 * Cr, 0, 255);*/

        /*int R1 = clamp(pData[j] + 1.403 * (pData[j+3] - 128), 0, 255), R2 = clamp(pData[j+2] + 1.403 * (pData[j+3] - 128), 0, 255);
        int G1 = clamp(pData[j] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128), 0, 255), G2 = clamp(pData[j+2] + 0.344 * (pData[j+1] - 128) - 0.714 * (pData[j+3] - 128), 0, 255);
        int B1 = clamp(pData[j] + 1.773 * (pData[j+1] - 128), 0, 255), B2 = clamp(pData[j+2] + 1.773 * (pData[j+1] - 128), 0, 255);*/

        int R1 = clamp((298 * (pData[j] - 16) + 409 * (pData[j+3] - 128) + 128) >> 8, 0, 255), R2 = clamp((298 * (pData[j+2] - 16) + 409 * (pData[j+3] - 128) + 128) >> 8, 0, 255);
        int G1 = clamp((298 * (pData[j] - 16) - 100 * (pData[j+1] - 128) - 208 * (pData[j+3] - 128) + 128) >> 8, 0, 255), G2 = clamp((298 * (pData[j+2] - 16) - 100 * (pData[j+1] - 128) - 208 * (pData[j+3] - 128) + 128) >> 8, 0, 255);
        int B1 = clamp((298 * (pData[j] - 16) + 516 * (pData[j+1] - 128) + 128) >> 8, 0, 255), B2 = clamp((298 * (pData[j+2] - 16) + 516 * (pData[j+1] - 128) + 128) >> 8, 0, 255);

        //printf("R: %d, G: %d, B: %d, R': %d, G': %d, B': %d \n", R1, G1, B1, R2, G2, B2);

        m_RGB->imageData[i] = (char)R1;
        m_RGB->imageData[i+1] = (char)G1;
        m_RGB->imageData[i+2] = (char)B1;
        m_RGB->imageData[i+3] = (char)R2;
        m_RGB->imageData[i+4] = (char)G2;
        m_RGB->imageData[i+5] = (char)B2;

        /*m_RGB->imageData[i] = (char)(clamp(1.164 * (pData[j] - 16) + 1.793 * (Cr), 0, 255));
        m_RGB->imageData[i+1] = (char)(clamp(1.164 * (pData[j] - 16) - 0.534 * (Cr) - 0.213 * (Cb), 0, 255));
        m_RGB->imageData[i+2] = (char)(clamp(1.164 * (pData[j] - 16) + 2.115 * (Cb), 0, 255));
        m_RGB->imageData[i+3] = (char)(clamp(1.164 * (pData[j+2] - 16) + 1.793 * (Cr), 0, 255));
        m_RGB->imageData[i+4] = (char)(clamp(1.164 * (pData[j+2] - 16) - 0.534 * (Cr) - 0.213 * (Cb), 0, 255));
        m_RGB->imageData[i+5] = (char)(clamp(1.164 * (pData[j+2] - 16) + 2.115 * (Cb), 0, 255));*/
    }

Any help is greatly appreciated.

Answer 1

Assuming packed 422 I don't see any of your blocks sampling the input data correctly. In packed 422 the input data will go Y1U1Y2V1 Y3U2Y4V2 where the overall image is a Y (luma) image at full resolution and one each of U and V each at half horizontal resolution.

Here's where I would start: Unpack alternating values of the input and extract a grayscale image:

for (uint i = 0, j = 0; i < 1280 * 720 * 3; i += 3, j += 2) {
    m_RGB->imageData[i] = pData[j];
    m_RGB->imageData[i+1] = pData[j];
    m_RGB->imageData[i+2] = pData[j];
}

Once you have that tuned to produce a grayscale image then introduce U and V by looking at pData[j+1] and pData[j+3] (or, on even pixels, pData[j-1] and pData[j+1]). Simplifying that is why some algorithms do two YUV pixels at a time.

When that works consider extracting the U and V images and properly resampling them to full resolution to produce a 444 image. Simply duplicating U and V for adjacent pixels is like upscaling by duplicating pixels.

(Note that other arrangements like 420 have even more complicated co-siting)

Answer 2

Some clues to help you along:

You are confusing Cr with Cb.

Assuming UYVY/422

Y1 = data[j+0];
Cr = data[j+1];
Y2 = data[j+2];
Cb = data[j+3];

Your conversion calculation are wierd, and incorrect for HD.

For SD

R = max(0, min(255, 1.164(Y - 16) + 1.596(Cr - 128)));
G = max(0, min(255, 1.164(Y - 16) - 0.813(Cr - 128) - 0.391(Cb - 128)));
B = max(0, min(255, 1.164(Y - 16) + 2.018(Cr - 128)));

For HD

R = max(0, min(255, 1.164(Y - 16) + 1.793(Cr - 128)));
G = max(0, min(255, 1.164(Y - 16) - 0.534(Cr - 128) - 0.213(Cb - 128)));
B = max(0, min(255, 1.164(Y - 16) + 2.115(Cr - 128)));

You could simply use ConvertFrame which is a part of the Decklink SDK.

Answer 3

I also struggled with the conversion

// Get the bytes
var u = bytes[0]; 
var y1 = bytes[1];
var v = bytes[2];
var y2 = bytes[3];

// Convert, cast to signed byte is important!
var r = y + (1.403 * (sbyte)v);
var g = y - (0.344 * (sbyte)u) - (0.714 * (sbyte)v);
var b = y + (1.770 * (sbyte)u);

if (r < 0)
    r = 0;
else if (r > 255)
    r = 255;

if (g < 0)
    g = 0;
else if (g > 255)
    g = 255;

if (b < 0)
    b = 0;
else if (b > 255)
    b = 255;

return Color.FromArgb((byte)r, (byte)g, (byte)b);

u and v are sbyte, and y is just a byte.

Answer 4

Your problem is that there are lots of YUV422 formats out there. You must find the exact one (the FOURCC index for the specific video you're using), and then figure out the correct way to decode it.

What you can do is to save some video from your board, open it in VLC, and look at the Codec details to find the exact FOURCC used.

http://www.fourcc.org/yuv.php