Openmvg: How to get the homography matrix, fundamental matrix, or essential matrix between 2 images?

[Fu416]

Hi, I want to get the homography matrix, fundamental matrix, or essential matrix between 2 images. How can I get them？

Thank you!

[pmoulon]

Hi @Fu416
You have to run the examples:

1. Tell to cmake that you want to build the examples:
   cmake -DCMAKE_BUILD_TYPE=RELEASE -DOpenMVG_BUILD_EXAMPLES=ON . ../openMVG/src/

Then look to the too sample that explain how to compute the homography, fundamental or the essential matrix (last one require an intrinsic calibration)
https://github.com/openMVG/openMVG/tree/master/src/openMVG_Samples/multiview_robust_fundamental
https://github.com/openMVG/openMVG/tree/master/src/openMVG_Samples/multiview_robust_homography
https://github.com/openMVG/openMVG/tree/master/src/openMVG_Samples/multiview_robust_essential

You can either modify the sample to use your image name and export the computed matrix to the console or a file.

If you want to compute it on a large number of image pair, you can use the SfM pipeline and modify the matching_image_collection engine in order to export the computed matrices.

[Fu416]

Hi @pmoulon ,

Thank you for your reply.

If I want to output the Projection matrix of every image，what should I do?

Thank you.

[rperrot]

If you want the projection matrix, you need the intrinsic K, the rotation R and the translation t, then you have to use the function P_From_KRt to get the projection matrix.

[pmoulon]

PS; You can see also in this sample how to get the projection matrices for a camera in a sample that involve robust essential matrices estimation + camera and structure refinement:
https://github.com/openMVG/openMVG/blob/master/src/openMVG_Samples/multiview_robust_essential_ba/robust_essential_ba.cpp#L238

[pmoulon]

Or if you have many image I advise you to use the entire OpenMVG pipeline and use the existing exporter (exporter for PMVS can create a file for each image with the projection matrix)

[Fu416]

Hi @pmoulon ，
I use 100_7101.jpg and 100_7104.jpg in Sceaux Castle dataset for 3D reconstruction. I find the rotation matrix in sfm_data.bin is not the same as the data in bundle.rd.out. The sign of the data has changed. Why?

such as:
1. sfm_data.bin
"extrinsics": [
{
"key": 0,
"value": {
"rotation": [
[
0.99999998478836649,
0.0001062807029505072,
-0.0001383028523857756
],
[
-0.00010627222586567678,
0.99999999247429006,
6.1299542541261954e-005
],
[
0.00013830936630340756,
-6.1284843856857917e-005,
0.99999998855734351
]
],
"center": [
-4.8229853254524932e-005,
0.00019790228247651016,
-0.0012591135363402774
]
}
},

2. bundle.rd.out

Bundle file v0.3

2 1642
2881.25 0 0
1 0.000106281 -0.000138303
0.000106272 -1 -6.12995e-005
-0.000138309 6.12848e-005 -1
4.80347e-005 0.00019783 -0.00125913
2881.25 0 0
0.948885 0.0349406 0.313683
0.0279933 -0.999253 0.0266262
0.314379 -0.0164841 -0.949154
-0.943443 -0.0578759 -0.326474
-0.176377 -0.859484 3.77376

[pmoulon]

The sign is different since the camera convention is not the same in Bundler and in OpenMVG ;-)

See here: https://github.com/openMVG/openMVG/blob/master/src/software/SfM/main_openMVG2PMVS.cpp#L252

BTW, you can parse the JSON file and build the camera matrix by using P = K [R|t] = K [R | -RC]

[naveenjafer]

@pmoulon I have a query regarding the R and T estimation in case of incremental SFM, as far as I understand once we have two initial views with sufficient number of matches and baselines that are not too close, the R and T of one matrix is calculated with respect to one of them isnt it? Where is the global reference created/set? Most work I have read proceeds with R and T of one camera view with respect to some other camera view and this process is iteratively carried out and finally the true R and T of the global reference system from the reference of each camera view can be calculated. I am a little confused as to how the R and T for the first view is calculated.

[pmoulon]

The incremental SfM pipeline is explained here [1].

Basically the first pair (the 3D seed) is computed by having one camera as reference and the second one is computed by relative pose. Since there is some bundle adjustment later in the process and not camera pose are set to fix, the global coordinates of all the camera can change a bit from iteration to another that is why you don't see a camera with a perfect Identity pose in the final sfm_data file.

[1] Moulon Pierre, Monasse Pascal and Marlet Renaud. ACCV 2012. Adaptive Structure from Motion with a contrario model estimation.

[Fu416]

Hi @pmoulon ,
Thank you for the above reply.
When I use SIFT and AKAZE_MLDB to extract features separately, I find the time of AKAZE to extract features and match descriptors are much slower than SIFT. But MLDB is binary descriptor, why is much slower than SIFT?
1. AKAZE_MLDB
--describerMethod AKAZE_MLDB
--upright 0
--describerPreset NORMAL
--force 0
--numThreads 0
- EXTRACT FEATURES -
0% 10 20 30 40 50 60 70 80 90 100%
|----|----|----|----|----|----|----|----|----|----|

---

Task done in (s): 48.624

• PUTATIVE MATCHES -
  Use: exhaustive pairwise matching
  Using BRUTE_FORCE_HAMMING matcher
  Using the OPENMP thread interface
  0% 10 20 30 40 50 60 70 80 90 100%
  |----|----|----|----|----|----|----|----|----|----|

---

Task (Regions Matching) done in (s): 113.966

1. SIFT
   --describerMethod SIFT
   --upright 0
   --describerPreset HIGH
   --force 0
   --numThreads 0
2. EXTRACT FEATURES -
   0% 10 20 30 40 50 60 70 80 90 100%
   |----|----|----|----|----|----|----|----|----|----|

---

Task done in (s): 28.796

• PUTATIVE MATCHES -
  Use: exhaustive pairwise matching
  Using FAST_CASCADE_HASHING_L2 matcher
  Using the OPENMP thread interface
  0% 10 20 30 40 50 60 70 80 90 100%
  |----|----|----|----|----|----|----|----|----|----|

---

Task (Regions Matching) done in (s): 2.406

[pmoulon]

It's simply because the complexity of AKAZE is higher than SIFT. AKAZE is using a non linear scale space and involve more computation that the DoG pyramid used in SIFT.

Moreover note that you can use the -n X option to run the extraction on X thread. It will greatly speed up the extraction process.

Regarding the matching: FAST_CASCADE_HASHING_L2 vs BRUTE_FORCE_HAMMING. You compare a brute force approach vs. an approximate nearest neighbor approach. It's normal that the brute force is slower. If you want a very fast binary matching I would advise you to use a method like Norouzi, Mohammad, Ali Punjani, and David J. Fleet. Fast search in hamming space with multi-index hashing, Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012..