I can't figure out the formula to compute the bank (roll) angle from the up and lookat vectors, though I feel this angle must be measured in tha plan normal to the lookat vector. Any hint appreciated. FYI I use WPF.
I have posted another question here, which is the same problem, but expressed only using math.
This is the final code to determine Bank. Note that I needed to determine the sign of the angle:
// project Y on plan perpendicular to look
Vector3D Yproj = new Vector3D(
-(lookDirection.Y * lookDirection.X),
1 - (lookDirection.Y * lookDirection.Y),
-(lookDirection.Y * lookDirection.Z));
Yproj.Normalize();
// get absolute angle between Y projected and Up
double absAngle = Vector3D.AngleBetween(upDirection, Yproj);
// magic formula
Vector3D cross = Vector3D.CrossProduct(upDirection, Yproj);
double dot = Vector3D.DotProduct(lookDirection, cross);
// set actual signed angle
BDeg = (dot >= 0) ? absAngle : -absAngle;