De-skew characters in binary image

Question

I'm working on number plate recognition. The problem is that I have to de-skew the characters in a binary image to increase the accuracy of template matching.

I have done a lot of pre-processing to remove unnecessary pixels of the image and I could segment the characters out. But unfortunately they are skewed.

From... converting to greyscale to binary

Then.. pre-processing techniques..

After segmentation..

As can be observed in the last image, the characters are skewed and this will lead to inaccuracy for template matching to perform recognition purposes.

Most of the researchers are using Hough transform to perform the de-skew operation but is there an easier way to do this?

Answer 1

You can find the rotation angle of your skewed black and white data also by principal component analysis of a set of points consisting of all white pixels in your image.

Here is the code:

% load image
img = imread('skewed.png');
img = img(:, :, 1);
img = double(img);

% perform pca on cloud of white points
[r, c] = find(img);
coeff = pca([r,c]);
angle = atan2(coeff(1,1), coeff(1,2));

% rotate back
img = imrotate(img, angle / pi * 180);
imwrite(img > 0, 'deskewed.png');

Input:

Output (rotation angle ~10.3 deg):

Answer 2

There are ways to deal with this. Some on the matching part to avoid the unskew operation itself like this:

• OCR and character similarity

But you want to unskew so:

1. detect the rotation angle/skew slope

   Obtain bounding box, then cast vertical scan lines and remember first hit point and last regress line through all of them

   

2. rotate/skew back by it

   So either use atan2 to obtain the angle or directly construct 2D homogenous 3x3 transform matrix based on basis vectors (one is the line and second is its perpendicular vector). For more info see:

   • Understanding 4x4 homogenous transform matrices

3. Now the rotated/unskew image will be still skewed bud in much much lower rate

   so you can apply #1,#2 in the horizontal axis too but this time you need to unskew only (do not use rotation). Usually the remnant skew ratio is small so this step is not necessary.

[notes]

You can boost the precision by filtering out wrong points or by carefully selecting the start point of scan lines so they hit in the right place of characters (you obviously know the characters count).

[edit1] small example

Here small example of output for your image (Negative as my functions are expecting white paper and black font):

As you can see the rotation and skew is much much smaller.