How to perform Auto crop for document Recognize im

Question

I want to make a an application like a cam scanner for cropping a document.

But I need same functionality like my two images..

First Images shown image captured by camera..

Second image recognize a captured image part like this..

I research more and more but not getting any out put so, I ask here if,any one done this tell me..

Thanks

Answer 1

Here is a exactly same demos that you want to achieve

https://github.com/Aniruddha-Tapas/Document-Scanner (Has auto detection feature)

https://github.com/jhansireddy/AndroidScannerDemo

Answer 2

Pass your image mat in this method:

       void findSquares(Mat image, List<MatOfPoint> squares) {
    int N = 10;

    squares.clear();

    Mat smallerImg = new Mat(new Size(image.width() / 2, image.height() / 2), image.type());

    Mat gray = new Mat(image.size(), image.type());

    Mat gray0 = new Mat(image.size(), CvType.CV_8U);

    // down-scale and upscale the image to filter out the noise
    Imgproc.pyrDown(image, smallerImg, smallerImg.size());
    Imgproc.pyrUp(smallerImg, image, image.size());
    // find squares in every color plane of the image
    Outer:
    for (int c = 0; c < 3; c++) {

        extractChannel(image, gray, c);

        // try several threshold levels
        Inner:
        for (int l = 1; l < N; l++) {

            Imgproc.threshold(gray, gray0, (l + 1) * 255 / N, 255, Imgproc.THRESH_BINARY);


            List<MatOfPoint> contours = new ArrayList<MatOfPoint>();

            // find contours and store them all as a list
            Imgproc.findContours(gray0, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);

            MatOfPoint approx = new MatOfPoint();

            // test each contour
            for (int i = 0; i < contours.size(); i++) {

                approx = approxPolyDP(contours.get(i), Imgproc.arcLength(new MatOfPoint2f(contours.get(i).toArray()), true) * 0.02, true);

                // square contours should have 4 vertices after approximation
                // relatively large area (to filter out noisy contours)
                // and be convex.
                // Note: absolute value of an area is used because
                // area may be positive or negative - in accordance with the
                // contour orientation
                double area = Imgproc.contourArea(approx);

                if (area > 5000) {

                    if (approx.toArray().length == 4 &&
                            Math.abs(Imgproc.contourArea(approx)) > 1000 &&
                            Imgproc.isContourConvex(approx)) {

                        double maxCosine = 0;
                        Rect bitmap_rect = null;
                        for (int j = 2; j < 5; j++) {
                            // find the maximum cosine of the angle between joint edges
                            double cosine = Math.abs(angle(approx.toArray()[j % 4], approx.toArray()[j - 2], approx.toArray()[j - 1]));
                            maxCosine = Math.max(maxCosine, cosine);
                            bitmap_rect = new Rect(approx.toArray()[j % 4], approx.toArray()[j - 2]);

                        }

                        // if cosines of all angles are small
                        // (all angles are ~90 degree) then write quandrange
                        // vertices to resultant sequence
                        if (maxCosine < 0.3)
                            squares.add(approx);

                    }
                }
            }
        }
    }
}

In this method you get four point of document then you can cut this image using below method:

      public Bitmap warpDisplayImage(Mat inputMat) {
    List<Point> newClockVisePoints = new ArrayList<>();

    int resultWidth = inputMat.width();
    int resultHeight = inputMat.height();

    Mat startM = Converters.vector_Point2f_to_Mat(orderRectCorners(Previes method four poit list(like : List<Point> points)));

    Point ocvPOut4 = new Point(0, 0);
    Point ocvPOut1 = new Point(0, resultHeight);
    Point ocvPOut2 = new Point(resultWidth, resultHeight);
    Point ocvPOut3 = new Point(resultWidth, 0);



        ocvPOut3 = new Point(0, 0);
        ocvPOut4 = new Point(0, resultHeight);
        ocvPOut1 = new Point(resultWidth, resultHeight);
        ocvPOut2 = new Point(resultWidth, 0);
    }

    Mat outputMat = new Mat(resultWidth, resultHeight, CvType.CV_8UC4);

    List<Point> dest = new ArrayList<Point>();
    dest.add(ocvPOut3);
    dest.add(ocvPOut2);
    dest.add(ocvPOut1);
    dest.add(ocvPOut4);


    Mat endM = Converters.vector_Point2f_to_Mat(dest);

    Mat perspectiveTransform = Imgproc.getPerspectiveTransform(startM, endM);

    Imgproc.warpPerspective(inputMat, outputMat, perspectiveTransform, new Size(resultWidth, resultHeight), Imgproc.INTER_CUBIC);


    Bitmap descBitmap = Bitmap.createBitmap(outputMat.cols(), outputMat.rows(), Bitmap.Config.ARGB_8888);
    Utils.matToBitmap(outputMat, descBitmap);



    return descBitmap;
}

Answer 3

I know I am too late to answer but it might be helpful to someone.

Try the following code.

@Override
protected void onDraw(Canvas canvas) {

  super.onDraw(canvas);
  path = new Path();

  path.moveTo(x1, y1);        // this should set the start point right

  //path.lineTo(x1, y1);    <-- this line should be drawn at the end of     course,sorry
  path.lineTo(x2, y2);
  path.lineTo(x3, y3);
  path.lineTo(x4, y4);
  path.lineTo(x1, y1); 
  canvas.drawPath(path, currentPaint);

}

Answer 4

I assume your problem is to detect the object to scan.

Object detection mechanisms like pattern matching or feature detection won't bring you the results you are looking for as you don't know what exactly is the object you are scanning.

Basically you search for a rectangular object in the picture.

A basic approach to this could be as following:

• Run a canny edge detector on the image. It could help to blur the image a bit before doing this. The edges of the object should be clearly visible.

• Now you want to do a Hough transform to find lines in the picture.

• Search for lines with an angle around 90deg to each other. The problem would be to find the right ones. Maybe it is enough to use the lines closest to the frame of the picture that are reasonably parallel to them.

• Find the intersecting points to define the edges of your object.

At least this should give you a hint where to research further.

As further steps in such an app you will have to calculate the projection of the points and do a affine transform of the object.

I hope this helps.

After writing all this i found this post. It should help you lot.

As my answer targets OpenCV you have to use the OpenCV library. In Order to do this, you need to install the Android Native Development Kit (NDK). There are some good tutorials on how to use OpenCV on Android on the OpenCV for Android page.

One thing to keep in mind is that almost each function of the Java wrapper calls a native method. That costs lots of time. So you want to do as much as possible in your native code before returning your results to the Java part.

Answer 5

I have created a git repo with code for native support, that is cropping the image the right way, please find it on: link.

Feel free to edit code if you come up with a better solution.