Characterizing my RAW camera Output

I have a very curious problem with my Leopard Imaging M021 camera. The company doesn't really support linux, so the camera will only output raw data. I need it to run on a Beagleboard, so that's why I'm putting effort into getting it to work.

An employee of theirs has told me that it is in YUY2 format (usually 16 bits per pixel), but the high 4 bits are always 0, and the lower 12 bits contain information like so:

Using the command:

fswebcam --device /dev/video0 --resolution 1280x720 --dumpframe test.raw

I get a file 1,843,200 bytes long, meaning a 1280x720 image with 2 bytes per pixel (16 bits per pixel).

HOWEVER, the only way I've been able to actually get the image to display correctly is if I use IrfanView's RAW image display and set it to 12 bits per pixel, not normalized, with a bayer pattern of GR, and a vertical flip. I have no idea why the vertical flip is required, because the other settings will display a skewed, weird image but not flipped. Then I use 12 BPP and it is flipped.

I suppose since the high 4 bits are always 0, that causes it to in actuality be 12 bits per pixel rather than 16?

I need to know what is really going on with the bytes in the file in order to write the conversion algorithm myself (unless anyone knows of an open source program that does the same thing as IrfanView).

Using Python, I've made a really quick script to just pull the Y component out and view it (expecting a greyscale version of the image), but I get a very distorted version. Is there something wrong with my code? Am I pulling out the data in the wrong order? What the heck does "not normalized" mean in IrfanView? Why is the GR Bayer pattern setting required to see the image in RGB?

with open('test.raw', 'r+b') as f:
    Y0 = []
    U = []
    Y1 = []
    V = []
    vals = [Y0, U, Y1, V]
    val = f.read(1)
    pixel = int.from_bytes(val, byteorder='big')
    i = 0
    vals[i].append(pixel)
    while val:
        i += 1
        val = f.read(1)
        if val != "":
            pixel = int.from_bytes(val, byteorder='big')
            vals[i % 4].append(pixel)

k = 0
with open("1.test", "w") as f:
    for i in range(720):
        for j in range(640):
            f.write(str(Y0[k]))
            f.write(" ")
            f.write(str(Y1[k]))
            f.write(" ")
            k += 1
        f.write("\n")

Resulting junk image:

I would appreciate anyone's help or advice on this.

EDIT:

Further possibly helpful evidence. If I run this in matlab, just treating every 2 bytes as the exact pixel value:

fid = fopen('test.raw', 'r');
[I, count] = fread(fid , [1280, 720], 'uint16');
imagesc(I')
colormap(gray);

I get this image:

I'm still missing color information, because I just disregarded it. And it's still skewed a little bit. But it looks better. If you zoom in, the pattern of the image skew is good pixel, black pixel, good pixel, black pixel, etc. Does someone more knowledgeable of cameras and colors know what that is indicative of?

EDIT 2:

With the expert help of Mark Ransom, I wrote up a nice OpenCV script to read in the data, utilize CV_BayerGR2RGB to convert to RGB, and view the image. It works!

#include <vector>
#include <iostream>
#include <stdio.h>
#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>

int main() {
    // Each pixel is made up of 16 bits, with the high 4 bits always equal to 0
    unsigned char bytes[2];

    // Hold the data in a vector
    std::vector<unsigned short int> data;

    // Read the camera data
    FILE *fp = fopen("test.raw","rb");
    while(fread(bytes, 2, 1, fp) != 0) {
        // The data comes in little-endian, so shift the second byte right and concatenate the first byte
        data.push_back(bytes[0] | (bytes[1] << 8));
    }

    // Make a matrix 1280x720 with 16 bits of unsigned integers
    cv::Mat imBayer = cv::Mat(720, 1280, CV_16U);

    // Make a matrix to hold RGB data
    cv::Mat imRGB;

    // Copy the data in the vector into a nice matrix
    memmove(imBayer.data, data.data(), data.size()*2);

    // Convert the GR Bayer pattern into RGB, putting it into the RGB matrix!
    cv::cvtColor(imBayer, imRGB, CV_BayerGR2RGB);

    cv::namedWindow("Display window", cv::WINDOW_AUTOSIZE);
    // *15 because the image is dark
    cv::imshow("Display window", 15*imRGB);

    cv::waitKey(0);

    return 0;
}

The evidence that you have, first that each byte has the upper 4 bits as 0 and the fact that you can see a proper color picture from Irfanview, tells me that the employee was incorrect and the format isn't actually YUY2 - it's GRGB.

This code should pull out the green pixels and double them up so you can get an idea of what the image looks like. Note that the even and odd lines will alternate between starting with G and (R or B).

with open('test.raw', 'r+b') as f:
    raw_g = []
    for y in range(720):
        for x in range(1280//4):
            if y % 2:
                r1, r0, g1, g0, b1, b0, g3, g2 = f.read(8)
            else:
                g1, g0, r1, r0, g3, g2, b1, b0 = f.read(8)
            g0 = g0 << 8 | g1
            g2 = g2 << 8 | g3
            raw_g.extend([g0, g0, g2, g2])

You'll probably want to right-shift these values >> 4 before you write them out to a gray-scale file.

Edit: The code had one remaining bug (an extra read), and I didn't realize the values were little-endian instead of big-endian. Those changes are already applied to the code sample above.

Since the values seemed a little dark, I also decided to apply a gamma correction - instead of n >> 4 I used int(255.999 * (n / 4095.0)**(1/2.2))