I need to register some 3D facial landmarks given for each frame of a video. For this task, I am trying to find out a transformation matrix between a few landmark coordinates given for the consecutive frames. For example, 3D coordinates of 3 landmarks in frame 1 and frame 2 are given as:
frame1 = [2 4 15; 4 15 14; 20 11 7]
frame2 = [16 5 12; 5 7 9; 11 6 19]
I have tried using imregtform function provided by matlab and ABSOR tool for matlab.
tform = imregtform(frame1, frame2, 'affine','OnePlusOneEvolutionary','MeanSquares');
tform = absor(frame1, frame2)
The following error occurs when using imregtform:
Error using imregtform>parseInputs (line 261)
The value of 'MovingImage' is invalid. All dimensions of the moving image should be greater than 4.
Error in imregtform (line 124)
parsedInputs = parseInputs(varargin{:});
Note: ABSOR doesn't find affine transformation, it finds similarity transformation.
First of all, 3 points are too little to recover affine transformation -- you need 4 points. For N-dimensional space there is a simple rule: to unambiguously recover affine transformation you should know images of N+1 points that form a simplex --- triangle for 2D, pyramid for 3D, etc. With 3 points you could only retrieve 2D affine transformation. A good explanation of why this is the case you may find in "Beginner's guide to mapping simplexes affinely".
Regarding some retrieval algorithm. I'm afraid, I don't know Matlab to provide you with appropriate code, but I worked with Python a little bit, maybe this code can help (sorry for bad codestyle -- I'm mathematician, not programmer)
import numpy as np
# input data
ins = [[1, 1, 2], [2, 3, 0], [3, 2, -2], [-2, 2, 3]] # <- points
out = [[0, 2, 1], [1, 2, 2], [-2, -1, 6], [4, 1, -3]] # <- mapped to
# calculations
l = len(ins)
B = np.vstack([np.transpose(ins), np.ones(l)])
D = 1.0 / np.linalg.det(B)
entry = lambda r,d: np.linalg.det(np.delete(np.vstack([r, B]), (d+1), axis=0))
M = [[(-1)**i * D * entry(R, i) for i in range(l)] for R in np.transpose(out)]
A, t = np.hsplit(np.array(M), [l-1])
t = np.transpose(t)[0]
# output
print("Affine transformation matrix:\n", A)
print("Affine transformation translation vector:\n", t)
# unittests
print("TESTING:")
for p, P in zip(np.array(ins), np.array(out)):
image_p = np.dot(A, p) + t
result = "[OK]" if np.allclose(image_p, P) else "[ERROR]"
print(p, " mapped to: ", image_p, " ; expected: ", P, result)
This code demonstrates how to recover affine transformation as matrix and vector and tests that initial points are mapped to where they should. It is based on equation presented in "Beginner's guide to mapping simplexes affinely", matrix recovery is described in section "Recovery of canonical notation". The same authors published "Workbook on mapping simplexes affinely" that contains many practical examples of this kind.
