detect paper from background almost same as paper

Question

I am trying to detect a piece of paper from an image. Well I have had help from posts like this . But one difference is that my background color is almost same as the paper color, and I am getting wrong result.

[edit] By wrong result I mean the paper outline contour is not detected at all. instead the largest contour covers the whole image.

image1

image2

My code so far (using emgu cv and c#)

MemStorage storage = new MemStorage();
        List<Contour<Point>> candidateList = new List<Contour<Point>>();
        List<double> areaList = new List<double>();

        Image<Bgr, Byte> inputImage = new Image<Bgr, Byte>(image);
        //Rectangle roi = new Rectangle(15, 15, image.Width - 15, image.Height - 15);
        //inputImage.ROI = roi;
        //inputImage = inputImage.Copy();

        double threshHeight = inputImage.Size.Height * 0.50;
        double threshWidth = inputImage.Size.Width * 0.50;


        //std::vector<std::vector<cv::Point> > squares;
        //cv::Mat pyr, timg, gray0(_image.size(), CV_8U), gray;
        int thresh = 50, N = 5;
        //cv::pyrDown(_image, pyr, cv::Size(_image.cols/2, _image.rows/2));
        //cv::pyrUp(pyr, timg, _image.size());
        //std::vector<std::vector<cv::Point> > contours;

        Image<Gray, Byte> [] gray0 = new Image<Gray,byte>[3];


        for( int c = 0; c < 3; c++ ) {
            try{
                int [] ch = {c, 0};
                gray0[c] = new Image<Gray,byte>(inputImage.Size);
                CvInvoke.cvMixChannels(new IntPtr[] { inputImage }, 1, new IntPtr[]{gray0[c]}, 1, ch, 1);
                Image<Gray, Byte> gray = new Image<Gray,byte>(inputImage.Size);
                for (int l = 0 ; l < N ; l++){
                    if (l == 0){
                        Image<Gray, Byte> cannyImage = gray0[c].Canny(0, thresh, 5);
                        CvInvoke.cvDilate(cannyImage, gray, IntPtr.Zero, 1);
                        //CvInvoke.cvShowImage("ch " + c + "-" + l, gray);
                    }else{
                        CvInvoke.cvThreshold(gray0[c], gray, (l + 1)*255/N, 255, Emgu.CV.CvEnum.THRESH.CV_THRESH_BINARY);
                        //CvInvoke.cvShowImage("ch " + c + "-" + l, gray0[c]);
                    }

                    //CvInvoke.cvShowImage("image", gray);    

                    for (Contour<Point> contours = gray.FindContours(Emgu.CV.CvEnum.CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, Emgu.CV.CvEnum.RETR_TYPE.CV_RETR_LIST, storage); contours != null; contours = contours.HNext){
                        Contour<Point> currentContour = contours.ApproxPoly(contours.Perimeter * 0.02, storage);
                        if (currentContour.Count() >= 4){
                            if (currentContour.Area > 3000){
                            //if (currentContour.BoundingRectangle.Width >= threshWidth && currentContour.BoundingRectangle.Height > threshHeight){
                                candidateList.Add(currentContour);
                                areaList.Add(currentContour.Area);
                                inputImage.Draw(currentContour, new Bgr(255, 0, 0), 1);
                            }
                        }
                    }
                }




            }catch(Exception ex){
                Debug.WriteLine(ex.Message);
            }


        }

        /* finding the biggest one */
        double area = -1.0;
        Contour<Point> paper = null;
        for (int i = 0 ; i < candidateList.Count ; i++){
            if (areaList[i] > area){
                area = areaList[i];
                paper = candidateList[i];
            }
        }

        if (paper != null){
            if (paper.BoundingRectangle.Width >= threshWidth && paper.BoundingRectangle.Height > threshHeight){
                inputImage.Draw(paper, new Bgr(0, 0, 255), 2);

            }
        }
        return inputImage.ToBitmap();

Please let me know how to process these images.

Answer 1

I did this in matlab (sorry I'm not really proficient at OpenCV) but you should be able to emulate the code. I tried to make it very simple. I noticed that the gradient of the original images really highlights where the paper is. So I used that to make a "rough" outline of the paper. Using the gradient is a good starting point, and maybe you can start from there. I just downsampled, then upsampled the image (emulating morphological operations to clean the image, since you lose all the small details).

You might get better results if you smooth the image out first (with a Gaussian filter maybe) I didn't try it, but maybe you can give it a try. Here is the result

And here is the code for reference

im1 = imread('http://www.imageno.com/image.php?id=ai7b91pm9fcs&kk=1089743759');
im2 = imread('http://www.imageno.com/image.php?id=k99c9xpd6phs&kk=3354581295');

%converts to grayscale
gim1 = rgb2gray(im1);
gim2 = rgb2gray(im2);

%gets size of images
[m1, n1] = size(gim1);
[m2, n2] = size(gim2);

%takes gradient of image
[Gx1, Gy1] = gradient(double(gim1));
[Gx2, Gy2] = gradient(double(gim2));

%takes magnitude of gradient in X and Y direction
Gxy1 = sqrt(Gx1.^2 + Gy1.^2);
Gxy2 = sqrt(Gx2.^2 + Gy2.^2);

%downsamples image (to reduce noise)
scale_factor = 100;
small1 = imresize(Gxy1, [m1/scale_factor n1/scale_factor]);
small2 = imresize(Gxy2, [m2/scale_factor n2/scale_factor]);

%upsamples image (to original size)
big1 = imresize(small1, [m1 n1]);
big2 = imresize(small2, [m2 n2]);

%converts to binary mask
bw1 = (big1 >= 1);
bw2 = (big2 >= 1);

%displays images
figure(1);
subplot(2,4,1);imshow(gim1);title('grayscale 1');
subplot(2,4,5);imshow(gim2);title('grayscale 2');

%these gradients are a little deceiving. In matlab when it sees an image 
%of type "double" its maps it so 0=black and 1=white. anything >=1 gets 
%clipped to 1
subplot(2,4,2);imshow(Gxy1);title('gradient 1');
subplot(2,4,6);imshow(Gxy2);title('gradient 2');

subplot(2,4,3);imshow(big1);title('down->up sampled 1');
subplot(2,4,7);imshow(big2);title('down->up sampled 2');

%this is just some matlab witchcraft so I can multiply the 2D mask with a
%3D image (r g b) in a very efficient manner
subplot(2,4,4);imshow(bsxfun(@times,im1,uint8(bw1)));title('masked image 1');
subplot(2,4,8);imshow(bsxfun(@times,im2,uint8(bw2)));title('masked image 2');