Detect if image is color, grayscale or black and w

Question

I extract pages images from a PDF file in jpeg format and I need to determine if each image is much more grayscale, color ou black and white (with a tolerance factor).

I have found some ways to work with color detection with PIL ( here and here ) but I can't figure out how to answer this simple (visual) question : is it much more black and white, color or grayscale image ?

I prefer working with Python and PIL for this part but I could use too OpenCV if someone has a clue (or solution).

Answer 1

I personally prefer the answer of TomB. This is not a new answer, I just want to post the Java version:

private Mat calculateChannelDifference(Mat mat) {   

    // Create channel list:
    List<Mat> channels = new ArrayList<>();

    for (int i = 0; i < 3; i++) {
        channels.add(new Mat());
    }

    // Split the channels of the input matrix:
    Core.split(mat, channels);

    Mat temp = new Mat();

    Mat result = Mat.zeros(mat.size(), CvType.CV_8UC1);

    for (int i = 0; i < channels.size(); i++) {

        // Calculate difference between 2 successive channels:
        Core.absdiff(channels.get(i), channels.get((i + 1) % channels.size()), temp);

        // Add the difference to the result:
        Core.add(temp, result, result);
    }

    return result;
}

The result is the difference as an matrix, this way you could apply some threshold and even detect shapes. If you want the result as a single number, you will just have to calculate the average value. This can be done using Core.mean()

Answer 2

I tried Gepeto's solution and it has a lot of false positives since the color grand variances can be similar just by chance. The correct way to do this is to calculate the variance per pixel. Shrink down the image first so you don't have to process millions of pixels.

By default this function also uses a mean color bias adjustment, which I find improves the prediction. A side effect of this is that it will also detect monochrome but non-grayscale images (typically sepia-toned stuff, the model seems to break down a little in detecting larger deviations from grayscale). You can separate these out from true grayscale by thresholding on the color band means.

I ran this on a test set of 13,000 photographic images and got classification with 99.1% precision and 92.5% recall. Accuracy could probably be further improved by using a nonlinear bias adjustment (color values must be between 0 and 255 for example). Maybe looking at median squared error instead of MSE would better allow e.g. grayscale images with small color stamps.

from PIL import Image, ImageStat
def detect_color_image(file, thumb_size=40, MSE_cutoff=22, adjust_color_bias=True):
    pil_img = Image.open(file)
    bands = pil_img.getbands()
    if bands == ('R','G','B') or bands== ('R','G','B','A'):
        thumb = pil_img.resize((thumb_size,thumb_size))
        SSE, bias = 0, [0,0,0]
        if adjust_color_bias:
            bias = ImageStat.Stat(thumb).mean[:3]
            bias = [b - sum(bias)/3 for b in bias ]
        for pixel in thumb.getdata():
            mu = sum(pixel)/3
            SSE += sum((pixel[i] - mu - bias[i])*(pixel[i] - mu - bias[i]) for i in [0,1,2])
        MSE = float(SSE)/(thumb_size*thumb_size)
        if MSE <= MSE_cutoff:
            print "grayscale\t",
        else:
            print "Color\t\t\t",
        print "( MSE=",MSE,")"
    elif len(bands)==1:
        print "Black and white", bands
    else:
        print "Don't know...", bands

Answer 3

We use this simple function to determine the color-factor of an image.

# Iterate over all Pixels in the image (width * height times) and do this for every pixel:
{
    int rg = Math.abs(r - g);
    int rb = Math.abs(r - b);
    int gb = Math.abs(g - b);
    diff += rg + rb + gb;
}

return diff / (height * width) / (255f * 3f);

As gray values have r-g = 0 and r-b = 0 and g-b = 0 diff will be near 0 for grayscale images and > 0 for colored images.

Answer 4

You can use the cv::Mat::channels() operator and that can tell you whether it is a "grayscale" (i.e., 2 channel) or "color" (i.e., 3-channel) image. For black and white, you will need set deeper tests based on grayscale since the definition varies.

Answer 5

I have found a way to guess this with the PIL ImageStat module. Thanx to this post for the monochromatic determination of an image.

from PIL import Image, ImageStat

MONOCHROMATIC_MAX_VARIANCE = 0.005
COLOR = 1000
MAYBE_COLOR = 100

def detect_color_image(file):
    v = ImageStat.Stat(Image.open(file)).var
    is_monochromatic = reduce(lambda x, y: x and y < MONOCHROMATIC_MAX_VARIANCE, v, True)
    print file, '-->\t',
    if is_monochromatic:
        print "Monochromatic image",
    else:
        if len(v)==3:
            maxmin = abs(max(v) - min(v))
            if maxmin > COLOR:
                print "Color\t\t\t",
            elif maxmin > MAYBE_COLOR:
                print "Maybe color\t",
            else:
                print "grayscale\t\t",
            print "(",maxmin,")"
        elif len(v)==1:
            print "Black and white"
        else:
            print "Don't know..."

The COLOR and MAYBE_COLOR constant are quick switches to find the differences between color and grayscale images but it is not safe. As an exemple, I have several JPEG images that are view as color but in real are grayscale with some color artefacts due to a scan process. That's why I have another level to note really shure color image from the others.

If someone has a better approch, let me know.