Normalize car plate for OCR in OpenCV C++

Question

I'm doing some simple OCR car plate recognition system. I'm using HaarCascades to find car plate, and next i need to normalize this plate, to put it into my OCR module. I'm using floodfill to find main contours of a car plate, and then i perform Hough transform, to find upper and lower boarders of a car plate:

Here's a part of code, where i perform Hough transform^

HoughLinesP(canny_img, lines, 1, CV_PI/180, 80, 80, 30 );

    for ( size_t i = 0; i < lines.size(); i++ ) {  
        line (output, Point(lines[i][0], lines[i][3]), Point(lines[i][4], lines[i][5]), Scalar(0,0,255), 1, 8 );
    }

Now i need to cut and rotate this picture along this two lines. How can i do this? i understand that i need to use point Point(lines[i][0])..Point(linesi), but what i should do with them?

So basically, i need to get something like that:

1. Image, that i got using HaarCascades

2. After some transformation i need to get something like this:

So at the first step i need to cut only upper and lower boarders.

Answer 1

You need to use affine transformations, here there is tutorial. In your situation you need to choose some size of car plate, for example 20x100. Your destination points will be 3 corners of non rotated rectangle of choosen size and source points will be 3 corners of founded car plate. I hope it is clear, if it is'not, let me know - i will make some example.

*\\EDIT:
Ok, i've made some example. Here is the code:

cv::Mat img = cv::imread("D:\\temp\\car_plate.jpg");
cv::Point2f a1(25, 18), b1(279, 27), c1(279, 79), a2(0, 0), b2(img.size().width, 0), c2(img.size().width, img.size().height);
//cv::Point2f a1(0, 16), b1(303, 28), c1(303, 81), a2(0, 0), b2(img.size().width, 0), c2(img.size().width, img.size().height);
cv::Point2f src[] = {a1, b1, c1};
cv::Point2f dst[] = {a2, b2, c2};
cv::Mat warpMat = cv::getAffineTransform(src, dst);
cv::warpAffine(img, img, warpMat, img.size());
cv::imshow("result", img);
cv::waitKey(-1);
return 0;

And results:


If you will use the code without any modification you will get the first result, if you comment second line and uncomment third line you will get second result (i think that's what you wanted). To get the second result you just need to find the points where upper and lower lines cross the image border. I've marked it here:

So basically you need to use red points. To calculate their positions you just need to find where blue lines (which if i understand correct you already have) cross the image border.

Answer 2

Here is a solution with EmguCv:

var src = new[] { new PointF(approxContour[0].X, approxContour[0].Y), new PointF(approxContour[1].X, approxContour[1].Y), new PointF(approxContour[2].X, approxContour[2].Y), new PointF(approxContour[3].X, approxContour[3].Y) };
var dst = new[] { new PointF(0, 0), new PointF(0, 400), new PointF(400, 400), new PointF(400, 0) };

var tmp = new UMat();
var matrix = CvInvoke.GetPerspectiveTransform(src, dst);
CvInvoke.WarpPerspective(imageRGB, tmp, matrix, new Size(400, 400));
viewer.Image = tmp;