image registration(non-rigid \ nonlinear)

Question

I'm looking for some algorithm (preferably if source code available) for image registration. Image deformation can't been described by homography matrix(because I think that distortion not symmetrical and not homogeneous),more specifically deformations are like barrel/distortion and trapezoid distortion, maybe some rotation of image. I want to obtain pairs of pixel of two images and so i can obtain representation of "deformation field".

I google a lot and find out that there are some algorithm base on some phisics ideas, but it seems that they can converge to local maxima, but not global. I can affort program to be semi-automatic, it means some simple user interation.

maybe some algorithms like SIFT will be appropriate? but I think it can't provide "deformation field" with regular sufficient density.

if it important there is no scale changes.

example of complicated field http://www.math.ucla.edu/~yanovsky/Research/ImageRegistration/2DMRI/2DMRI_lambda400_grid_only1.png

Answer 1

What you are looking for is "optical flow". Searching for these terms will yield you numerous results.

In OpenCV, there is a function called calcOpticalFlowFarneback() (in the video module) that does what you want. The C API does still have an implementation of the classic paper by Horn & Schunck (1981) called "Determining optical flow".

You can also have a look at this work I've done, along with some code (but be careful, there are still some mysterious bugs in the opencl memory code. I will release a corrected version later this year.): http://lts2www.epfl.ch/people/dangelo/opticalflow

Besides OpenCV's optical flow (and mine ;-), you can have a look at ITK on itk.org for complete image registration chains (mostly aimed at medical imaging).

There's also a lot of optical flow code (matlab, C/C++...) that can be found thanks to google, for example cs.brown.edu/~dqsun/research/software.html, gpu4vision, etc

-- EDIT : about optical flow --

Optical flow is divided in two families of algorithms : the dense ones, and the others. Dense algorithms give one motion vector per pixel, non-dense ones one vector per tracked feature.

Examples of the dense family include Horn-Schunck and Farneback (to stay with opencv), and more generally any algorithm that will minimize some cost function over the whole images (the various TV-L1 flows, etc).

An example for the non-dense family is the KLT, which is called Lucas-Kanade in opencv.

In the dense family, since the motion for each pixel is almost free, it can deal with scale changes. Keep in mind however that these algorithms can fail in the case of large motions / scales changes because they usually rely on linearizations (Taylor expansions of the motion and image changes). Furthermore, in the variational approach, each pixel contributes to the overall result. Hence, parts that are invisible in one image are likely to deviate the algorithm from the actual solution.

Anyway, techniques such as coarse-to-fine implementations are employed to bypass these limits, and these problems have usually only a small impact. Brutal illumination changes, or large occluded / unoccluded areas can also be explicitly dealt with by some algorithms, see for example this paper that computes a sparse image of "innovation" alongside the optical flow field.

Answer 2

i found some software medical specific, but it's complicate and it's not work with simple image formats, but seems that it do that I need. http://www.csd.uoc.gr/~komod/FastPD/index.html Drop - Deformable Registration using Discrete Optimization