Unstable values in ArUco pose estimation

Question

I'm trying to find the orientation of the camera using Aruco marker. Euler angles extracted from the rotation matrix are unstable beyond a certain point. As the distance of the camera increases from the marker, the yaw angle values of camera is just unstable. The "Z" axis on the marker flips. The euler angles are jittery, not the same in every frame and take time to stabilize. How do I obtain some reliable values of the yaw angle and distance between the camera and marker? I am trying to find the pose of moving camera w.r.t a static marker. I implemented solvePnP and solvePnPRansac both yielding in unstable results. The rotation matrix obtained after converting rotation vectors from estimatePoseSingleMarker seems alright up to a certain point but loses stability. How do I go about this? Thank you

Answer 1

In general, you won't get accurate camera pose estimation from a single marker. The solution is to add more markers. You could use either a marker board, or a more sparse pattern of markers.

As a single marker gets further from the camera, several factors work to reduce the accuracy of the marker pose estimate.

1. the projected size of the marker becomes smaller and more quantized by the pixel grid. Distance is estimated by inverse perspective division, so it becomes less accurate as distance increases.

2. perspective distortion reduces, approaching a parallel projection. In a parallel projection the marker has two equally viable orientations, which may be returned alternately (see https://en.wikipedia.org/wiki/Necker_cube). The orientation of the marker relative to the camera is also significant - in more perpendicular views of the marker (orthographic projection), pitch and yaw of the marker are ambiguous, compared to oblique views. Reduced perspective distortion with distance makes this effect worse, and will cause the calculated camera pose to yaw, pitch, and move laterally.

3. given the smaller number of pixels in the marker, small scale effects such as sensor noise and quantization become more significant, reducing stability from frame to frame and causing jitter.

As you have discovered, pose estimation works OK in close-up, oblique views of a single marker, because the projected points given to solvePnP() are far apart and have large perspective distortion. By adding more markers, you always have ideal projected points for solvePnP().