3D stereo, bad 3D coordinates

Question

I'm using stereo vision to obtain 3D reconstruction. I'm using opencv library. I've implemented my code this way:

1) Stereo Calibration

2) undistort and Rectification of image pair

3) disparity map - using SGBM

4) 3D coordinates calculating depht map - unsing reprojectImageTo3D();

Results:

-Good disparity map, and good 3D reconstruction

-Bad 3D coordinates values, the distances don't corresponde to the reality.

The 3D distances, the distante between camera and object, have 10 mm error and increse with distance. I,ve used various baselines and i get always error. When i compare the extrinsic parameter, vector T, output of "stereoRectify" the baseline match. So i dont know where the problem is.

Can someone help me please, thanks in advance

CAlibration:

http://textuploader.com/ocxl http://textuploader.com/ocxm

Answer 1

Ten mm error can be reasonable for stereo vision solutions, all depending of course on the sensor sensitivity, resolution, baseline and the distance to the object.

The increasing error with respect to the object's distance is also typical to the problem - the stereo correspondence essentially performs triangulation between the two video sensors to the object, and the larger the distance is the derivative of the angle between the video sensors to the object translates to larger distance on the depth axis, which means larger error. Good example is when the angle between the video sensors to the object is almost right, which means that any small positive error in estimating it will throw the estimated depth to infinity.

The architecture you selected looks good. You can try increasing the sensors resolution, or maybe dig in to the calibration process which has a lot of room for tuning in the openCV library - making sure only images taken with the chessboard being static are selected, choose higher variety of the different poses of the chessboard, adding images until the registration between the two images drops below the maximal error you can allow, etc.