Finding distance of rectangle with known aspect ra

Question

I'm working on an OpenCV program to find the distance from the camera to a rectangle with a known aspect ratio. Finding the distance to a rectangle as seen from a forward-facing view works just fine:

The actual distance is very close to the distance calculated by this:

            wtarget · pimage
d = c ——————————————————————————
       2 · ptarget · tan(θfov / 2)

Where w_target is the actual width (in inches) of the target, p_image is the pixel width of the overall image, p_target is the length of the largest width (in pixels) of the detected quadrilateral, and θ_fov is the FOV of our webcam. This is then multiplied by some constant c.

The issue occurs when the target rectangle is viewed from a perspective that isn't forward-facing:

The difference in actual distance between these two orientations is small, but the detected distance differs by almost 2 feet.

What I'd like to know is how to calculate the distance consistently, accounting for different perspective angles. I've experimented with getPerspectiveTransform, but that requires me to know the resulting scale of the target - I only know the aspect ratio.

Answer 1

Here's what you know:

1. The distance between the top left and top right corners in inches (w_target)
2. The distance between those corners in pixels on a 2D plane (p_target)

So the trouble is that you're not accounting for the shrinking distance of p_target when the rectangle is at an angle. For example, when the rectangle is turned 45 degrees, you'll lose about half of your pixels in p_target, but your formula assumes w_target is constant, so you overestimate distance.

To account for this, you should estimate the angle the box is turned. I'm not sure of an easy way to extract that information out of getPerspectiveTransform, but it may be possible. You could set up a constrained optimization where the decision variables are distance and skew angle and enforce a metric distance between the left and right points on the box.

Finally, no matter what you are doing, you should make sure your camera is calibrated. Depending on your application, you might be able to use AprilTags to just solve your problem.