Opencv-python: cv2.stereoCalibrate behaves differently to c++ implementation

[AndrewMartchenko]

When I call cv2.stereoCalibrate(...) with 14 element distortion coefficient vectors, the returned distortion coefficient vectors have only 5 elements.

Actual behaviour (Python 3.8.10)

When the following gets called

ret, mtxL, distL_out, mtxR, distR_out, R, T, E, F = cv2.stereoCalibrate(obj_pts, img_ptsL, img_ptsR,
                                                                        mtxL, distL_in,
                                                                        mtxR, distR_in,
                                                                        (w, l), criteria_stereo, flags)

Input left distortion parameters are:

distL_in =
[[-9.28027807e+00  5.37383609e+01 -3.06749966e-04  3.71928959e-03
  -2.39827139e+02 -9.28424990e+00  5.31178190e+01 -2.35537833e+02
   0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
   7.26131243e-03 -4.69211193e-03]]

Output left distortion parameters are:

distL_out =
[[-9.28027807e+00  5.37383609e+01 -3.06749966e-04  3.71928959e-03
  -2.39827139e+02]]

The flags have been set as follows:

flags = 0
flags |= cv2.CALIB_RATIONAL_MODEL
flags |= cv2.CALIB_TILTED_MODEL
flags |= cv2.CALIB_FIX_INTRINSIC
flags |= cv2.CALIB_USE_INTRINSIC_GUESS
flags |= cv2.CALIB_FIX_FOCAL_LENGTH
flags |= cv2.CALIB_FIX_PRINCIPAL_POINT
flags |= cv2.CALIB_FIX_ASPECT_RATIO
flags |= cv2.CALIB_FIX_TANGENT_DIST
flags |= cv2.CALIB_FIX_K1
flags |= cv2.CALIB_FIX_K2
flags |= cv2.CALIB_FIX_K3
flags |= cv2.CALIB_FIX_K4
flags |= cv2.CALIB_FIX_K5
flags |= cv2.CALIB_FIX_K6
flags |= cv2.CALIB_FIX_S1_S2_S3_S4
flags |= cv2.CALIB_FIX_TAUX_TAUY

Expected behaviour (C++)

When I run the same function in c++ with 14 element vectors for the distortion coefficients, I get the correct result - the vectors remain 14 elements long.

I noticed in the c++ version that if I only enable the cv::CALIB_FIX_INTRINSIC flag, then the output distortion coefficient vectors do get reduced to 5 elements each, just like in the Python version.

The rotation R and translation T matrices produced by the python and C++ implementation are very different

Python translation vector (with 5 element output distortion vectors):
T = [-13.38267045 1.59477086 22.51344999]

C++ translation vector (with 14 element output distortion vectors):
T = [-5.067262324969635 0.01664551456133418 0.04499006665577673]

Also for your reference here is the C++ translation vector when only cv::CALIB_FIX_INTRINSIC flag is enabled (hence with 5 element output distortion vectors):
T = [-12.44688911164896 2.005676550526267 22.86746701014153]

• operating system: Windows 10
• architecture: amd64
• opencv-python version: 4.5.2

[alalek]

Please ensure that arrays are passed to C++ code with the same layout. Use:

• cv.utils.dumpInputArray() or cv.utils.dumpInputArrayOfArrays()

---

See also:

• https://github.com/opencv/opencv/issues/18413

[AndrewMartchenko]

Hi alalek,

Thank you for your response.

As far as I can tell there are no problems with the way arrays are passed into the C++ code. The C++ code behaves as expected - 14 element distortion vectors are passed in and stereoCalibrate outputs the vector unchanged (as expected with the flags that I have enabled).
The problem is with the python implementation - I pass in a 14 element distortion array and a 5 element distortion array is returned.

Also, the array shapes does not seem to be the issue because:

1. Both the C++ Python implementation run and produce and output without any assertion errors.
2. The produce almost the same rotation and translation matrices when only the cv::CALIB_FIX_INTRINSIC flag is enabled.
3. I tried reshaping the 'obj_pts' and 'img_pts' arrays as suggested in opencv#18413, but this did not affect the python output in any way.

It looks like the python stereoCalibrate function is only using the first 5 elements in the distortion coefficient arrays regardless of the flags passed into the function, where as the C++ version is correctly using all 14 distortion coefficients.

[alalek]

Please note that there is no dedicated Python implementation. OpenCV Python binding is a wrapper over C++ code. We need to ensure that the right C++ functions are called and the right parameters are passed to C++ code.

[AndrewMartchenko]

Thanks alalek. Yes, I am aware of this. I use the lazy language "Python implementation" to discriminate between the C++ and python tests that I had performed.
I'm not familiar with the process used to generate the wrappers, but is there a way to look at the wrapper code for the stereoCalibrate function?

[asmorkalov]

@AndrewMartchenko Could you provide full code to reproduce the issue?

[AndrewMartchenko]

@asmorkalov Attached is a simple example showing different results in python and C++. Both the Python and C++ implementations load most of the stereoCalibrate function argument from the attached XML file.

Here is the output I get form Python and C++ code:

Python:

Camera distortion before
[[ 1.06660525e+01]
 [-3.28527568e+00]
 [-2.23977786e-04]
 [ 2.75405261e-04]
 [-4.55870275e+00]
 [ 1.08407489e+01]
 [-1.39339436e+00]
 [-6.19501003e+00]
 [ 0.00000000e+00]
 [ 0.00000000e+00]
 [ 0.00000000e+00]
 [ 0.00000000e+00]
 [ 0.00000000e+00]
 [ 0.00000000e+00]]
Camera distortion after
[[ 1.06660525e+01]
 [-3.28527568e+00]
 [-2.23977786e-04]
 [ 2.75405261e-04]
 [-4.55870275e+00]]
Rotation matrix
[[ 0.96736086 -0.04897439  0.24862517]
 [-0.01872689  0.96464671  0.26288025]
 [-0.25270985 -0.25895605  0.93224433]]
Translation matrix
[[-481.32733751]
 [-418.26756498]
 [  35.81753018]]

C++:

Camera distortion before
[10.66605247510551;
 -3.285275677907718;
 -0.0002239777860614761;
 0.0002754052605782622;
 -4.558702746668033;
 10.84074885514549;
 -1.393394357728932;
 -6.195010029831366;
 0;
 0;
 0;
 0;
 0;
 0]
Camera distortion after
[10.66605247510551;
 -3.285275677907718;
 -0.0002239777860614761;
 0.0002754052605782622;
 -4.558702746668033;
 10.84074885514549;
 -1.393394357728932;
 -6.195010029831366;
 0;
 0;
 0;
 0;
 0;
 0]
Rotation matrix
[0.9505968917080613, -0.01674142345003515, 0.3099762478252793;
 -0.08370759464721581, 0.9477324625852618, 0.3078899445588518;
 -0.298929068632259, -0.3186265903890352, 0.8995101488164035]
Translation matrix
[-355.88832121915;
 -256.9820616336252;
 569.5169203322572]

Notice that the camera_distortion parameters after calibration are different. The resulting rotation and translation matrices are also different

stereo_calibrate_example.cpp.txt
stereo_calibrate_example.py.txt
stereo_calibration_arguments.xml.txt

Thank you!

[AleksandrPanov]

@AndrewMartchenko
stereoCalibrate signature:

stereoCalibrate(objectPoints, imagePoints1, imagePoints2, cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2,
                imageSize[, R[, T[, E[, F[, flags[, criteria]]]]]])

stereo_calibrate_example.py.txt:

...
cv2.stereoCalibrate(object_points, camera_image_points, projector_image_points, new_camera_matrix,
                    camera_distortion, new_proj_matrix, proj_distortion, (2448, 2048),
                    criteria_stereo, flags)
...

Parameter flags is after criteria, and as flag use first element of criteria. Swap this parameters or consider using "names" (flags=flags, criteria=criteria_stereo)

This will fix your problem.

[AndrewMartchenko]

@AleksandrPanov Thank you, that fixed the problem. I can't believe I missed this.