Image comparison - rotation, alignment and scaling

Question

I have the following needs. There are some number of forms, i.e blanks - for example the ones used in surveys. The ones which aren't filled with information, I will call image templates from now on. Apart from the image templates, I have also many images, which are essentially the image templates filled with information. For example, there is a survey and there are two blanks for filling - these are the image templates. Many people have filled the blanks with their personal information and these are the images.

The image templates are scanned in a perfect shape. But many of the scanned images are tilted or not properly aligned, or maybe scaled. So I have the following requirement - every image must be recognized to which image template it belongs. After it is recognized, it must be properly skewed, aligned and scaled to the image template.

I know this is a complex task and that's why I need a library, preferably a C# one. I have found AForge, but till now I have only seen a suitable method for skewing. Essentially I need a library which takes as input an image template and an image, and sets a flag if the image does not match to the image template. But if it matchs it must return the appropriate skew angle, alignment and scaling.

If you have any ideas or used such a library, I will appreciate it greatly.

Wish you all the best,

Petar

Answer 1

The problem seems to be an image registration problem coupled with some template matching problem.

• image registration

Depending on how the scanned document may be distorted (scale factor, rotation, skew...) one can register images using simple rigid transform (i.e. translation + rotation, only two corresponding points are needed) to more complex one such as non rigid transform (more corresponding point are needed). The corresponding points can be manually given but ideally should be automatically detected.

ITK library includes several methods for image registration

• template matching problem

Once your images are aligned, the comparison between an image and possible templates database could be achieved by first extracting characteristic features in the image and comparing them to your template database. This is very general and should be refined with respect to the image used.

There is other way that combine both image registration and template matching

• the Bag Of Features approach which consists in extracting interesting points (robust to several types of image deformation) from the image, the points generate a signature that characterizes the image, the image comparison being in fact a signature comparison.

Answer 2

I used to work for a company, Accusoft Pegasus, which has some interesting forms recognition software. I've not seen their FormFix tool in action in a few years, but it should be able to do what you are needing.