how do we extract scale matrix from model view matrix? Right now I am taking length of each coloumn, but it fails when the scale is negative.
here is my code:
float xs =
matrix[0][0] * matrix[0][1] * matrix[0][2] * matrix[0][3] < 0 ?
-1 : 1;
float ys =
matrix[1][0] * matrix[1][1] * matrix[1][2] * matrix[1][3] < 0 ?
-1 : 1;
float zs =
matrix[2][0] * matrix[2][1] * matrix[2][2] * matrix[2][3] < 0 ?
-1 : 1;
glm::vec3 new_scale;
new_scale.x = xs* glm::sqrt(
matrix[0][0] * matrix[0][0] + matrix[0][1] * matrix[0][1]
+ matrix[0][2] * matrix[0][2]);
new_scale.y = ys* glm::sqrt(
matrix[1][0] * matrix[1][0] + matrix[1][1] * matrix[1][1]
+ matrix[1][2] * matrix[1][2]);
new_scale.z = zs* glm::sqrt(
matrix[2][0] * matrix[2][0] + matrix[2][1] * matrix[2][1]
+ matrix[2][2] * matrix[2][2]);
For example:
float []mat={0.032254f, 0.000000f, 0.000000f, 0.000000f, 0.000000f, -0.0052254f, 0.000000f, 0.000000f, 0.000000f, 0.000000f, 0.4332254f, 0.000000f, 0.000000f, 0.000000f, 0.000000f, 1.000000f};
see homogenous transformation matrices. The way you are extracting scale is OK. So what about the sign? Your current approach will not work if your transformation contains rotations ...
Another problem is that you can not know which scale is negated and which not because if you negate single axis you can obtain the same result if you negate any other one and rotate to match the position. If you negate 2 axises you get the original matrix with different rotation.
The best you can do is detect if your matrix has inverted 1 or 3 axis:
Create sign table for original undistorted matrix
for example unit matrix but if you have different starting point use that
sz0=dot(cross(X0,Y0),Z0);
sy0=dot(cross(Z0,X0),Y0);
sx0=dot(cross(Y0,Z0),X0);
where X0,Y0,Z0
are extracted axises vectors from your start point matrix
compute the signs for your current matrix
sz1=dot(cross(X1,Y1),Z1);
sy1=dot(cross(Z1,X1),Y1);
sx1=dot(cross(Y1,Z1),X1);
where X1,Y1,Z1
are extracted axises vectors from your actual matrix
compare signs and deduce which axises scales are negative
if (sx0*sx1<0)||(sy0*sy1<0)||(sz0*sz1<0)
then one or all 3 axises are negated but you can not know which ... Also all 3 sign comparisons should have the same result.
[edit1] clarifications
X=(matrix[0][0],matrix[0][1],matrix[0][2])
dot(a,b)=a.x*b.x+a.y*b.y+a.z*b.z
is scalar multiplication of vectors (dot product)
c=cross(a,b) ... c.x=a.y*b.z+a.z*b.y c.y=a.z*b.x+a.x*b.z c.z=a.x*b.y+a.y*b.x
is vector multiplication (cross product)
So when you compute cross of two vectors you get vector perpendicular to both operands. As matrix axis vectors should be perpendicular by multipliyng 2 axises you get the third. The dot product just compare if the original and computed third axis are in the same direction ... This way is invariant on rotations