How to find the error image?

Question

I've got a problem with image processing, but I don't know how to solve it.

I have 2 pictures: - Pic 1: http://goo.gl/BBxVl - Pic 2: http://goo.gl/X0VFW

The pic 2 actually express the pic 1 but it was covered by the object. I am writing a program using matlab code to define that: If the picture is a the fulfilled (perfect) one, I will "imshow" it. But if there are any errors with the picture, the program will show the notice board for the user.

Until this time, I still can not solve it because I don't know where to begin; I also don't have any definition about the shape and the color of the object that cover the picture.

Answer 1

If the images you have posted are good examples of the problem, I would suggest the following algorithm:

1. substract both images from each other
2. determine the area of difference, this will give you the shape of the object that covers the "perfect" image
3. look along the edges of the distortion which of the two images is a better fit.

Answer 2

My idea also starts with the absolute difference between both images. The issue here is that you might get a lot of regions that weren't modified at all, due to compression and the inner working of some file formats (like jpg). For instance, here are two sample images and the binary difference between them to highlight every modified point, although I only manually modified the visible rectangles. In the non-binary difference you hardly notice all these points, but they are there. A threshold solves the problem here, I played with the value 20:

Now, to determine which of the images is the "good" one, I used an inpainting algorithm (you can find one such implementation at http://www.cc.gatech.edu/~sooraj/inpainting/). The reasoning for this is that the final in-painted image is more likely to resemble the image before inpainting if it does not contain the artificial patches. So, the "bad" image is the one with greater difference after the inpainting process. For this, you calculate again the absolute difference, now between the inpainted image and the original one for both cases. Then use some measure in the regions with absolute difference > 0. A dummy sum of the intensities in grayscale, in this example, gives 424454 for the image without the rectangles and 758366 for the other one.

Here is the masked image for in-painting, the "good" image inpainted, the "bad" image inpainted, and the respective absolute differences in grayscale:

Answer 3

So you don't know which one of them is the correct one, and you want to choose the correct one? Or you have a ground truth image that you can compare the input with? Because if you know how the correct image looks like, you can just do a simple subtraction between the images and find the erroneous one. So I don't think that's the question.

So, in the case you have two images and you have to determine which one is correct, do you know the type of the error? It is always a "patch" added on the image? Is it always rectangle or it can also be blended with the background image?

One idea would be to cut the image in parts and do histogram analysis for every one of them, but it will only work if the original image is quite constant and the patched object differs a lot from the background.

Maybe you can do edge detection on the image and then try to detect straight lines (Hough line detection can do that) so that you can search for a rectangular shape. (If you have the lines as a result then the algorithm for finding a rectangle is independent of the shape and size of the rectangle. You only check the line direction)

If you can describe with more details what are the restrictions of your error objects, maybe we can think of a better idea.