Openmvg: Global SfM bug

[LingyuMa]

I am recently using OpenMVG to do some reconstruction. I have found that the incremental method works fine. However, the Global method does not work and show me the following error when I am using the same parameters as in incremental method:
Invalid matches file.

However, when I change the matching part from "-g f" to "-g e", the global can work, but the result (robust.ply and colorized.ply) is empty. It seems it cannot generate any reasonable result. Could anyone help me to figure out what is going wrong? Is there any specific requirement for Global method? (for example, the calibration matrix '-k' should be given in advance or something)

[whuaegeanse]

the global sfm use the file named "matches.e.bin", while the sequential sfm use the file named "matches.f.bin". You should change the the matching part and the global reconstruction use the file "matches.e.bin" in source code. The way to fix this is to change the file name "matches.f.bin" to "matches.e.bin" after matching process but before the global reconstruction.

[pmoulon]

1. Check that all your image are linked to valid intrinsic data
   -> id_intrinsic in your sfm_data.json file must be set to a small value (0 for all your view, in the case of shared intrinsic).
   -> you can share here your sfm_data.json file and I will tell you if it's the case or not.
2. Does the global SfM produce a file called cloud_and_poses.ply => check it

FYI, it's possible that for your scene GlobalSfM fails due to image correspondence or because your image have few overlap.

@whuaegeanse
=> I think it's better to use matches.f.txt -g ffor sequential and matches.e.txt -g e for global calibration. BTW, you can rename the file in order to use whatever matches your want.

[LingyuMa]

The json files for both methos are here:
https://drive.google.com/file/d/0BwWAt5w3811Wa3VyZFNTMXd5UWM/view?usp=sharing
I checked it, It is very different.

In the clound_and_poses.ply, it has 60000+ lines, but the values in every line are exactly the same.

[pmoulon]

Seems you are trying to reconstruct a tank ;-)

When I look to the data, it seems that intrinsics data are ok.
But you are right, there is something that is making the global reconstruction unsuccessful, all the camera position seems to be at the position.

Hard to say without experimenting with your dataset, what is happening. The problem can come from rotation or translation computation/averaging.

You can try those two test in order to see if the distortion or the translation solver are the cause of the problem (on openMVG_main_GlobalSfM add those parameters, one at a time):
-f ADJUST_FOCAL_LENGTH|ADJUST_PRINCIPAL_POINT (distortion will not be refined)
And then on a second run try:
-t 1 (will use another method to fuse all the relative translations)

Perhaps the matches are too noisy for the global SfM on this scene...

I saw that you provided .json file for those reconstruction...
Are you using the last master branch? or an older version.
FYI, OpenMVG exports now binary files by default; that you can convert to json, ply files with openMVG_main_ConvertSfM_DataFormat

[LingyuMa]

I have changed the source file a little bit in order to export json file. I am using version 1.0 now. The interesting thing is when I am using the earlier version (maybe 0.9 or earlier) on the same image set. The global reconstruction works. So I am not sure why it does not work on the current version.

[LingyuMa]

I tried to run -f ADJUST_FOCAL_LENGTH|ADJUST_PRINCIPAL_POINT and -t 1
Could you help me to check the results I obtained?
https://drive.google.com/file/d/0BwWAt5w3811Wa0Q0dWg3NGFHSWc/view?usp=sharing

[pmoulon]

Results are better, sfm_data_noDist.json shows that the translation fusion or the BA have done something strange 'reducing the scene to a very small scene', maximum camera motion amplitude is about 1e-5.
If you want I try to look to the problem more deeply I would need the matches folder (with the *.feat files, the matches/sfm_data.json file and the matches/matches.e.bin). You do not need to share to me 'images & *.desc files'

[LingyuMa]

Hi,
Below is the match folder, I run both incremental and global in the folder, so there are matches.e.bin and matches.f.bin
https://drive.google.com/file/d/0BwWAt5w3811WWjFDaHZ5WnV6WTg/view?usp=sharing

[pmoulon]

@LingyuMa Ok the problem is the one I though about (the BA that make the scene too small, and so we have later one precision loss on camera position).

One wait to avoid this effect consists in set Ceres to do not use a LOSS_FUNCTION too early in the motion refinement process.

Here the fix you can try; replace this function

bool GlobalSfMReconstructionEngine_RelativeMotions::Adjust()

by the code provided here
https://gist.github.com/pmoulon/4874ce4af910a3392e80e8af54b2d967

Then you will have your point cloud & camera motion:

[LingyuMa]

Thank you for your help! I found that there is only one line change in the code. What will this change influence the performance of SFM(to remove the loss function part).

[LingyuMa]

Also, I get this bug: class openMVG::sfm::Bundle_Adjustment_Ceres’ has no member named ‘ceres_options’

[pmoulon]

I forgot to mention that you must use the develop branch.
It will not change performance, since the loss function removal is only for a tiny part of the process.

[LingyuMa]

OK, it works in Linux Thank you!

[LingyuMa]

Hi, I made some comparisons between the sfm results of v0.8.1 and the develop branch. I found that the result of the most current version is less accurate than the old version (Which is weird), I have some accurately measured data as ground truth? Could you tell me if there is any way to increase the accuracy?

[pmoulon]

The only major diff comes from the way that triplets of translation are chosen.
The changes was made at this commit https://github.com/openMVG/openMVG/commit/5f75e143818b430962e120ecf9c525b658160e7a#diff-f40f0f8b2bc4b35b985d1299da9ad673

Try to take the project at the commit just before this change to see if this commit was the one or not...
=> git checkout c089b71bf443b938da5a34e01c27e34a286269db

Else we must use git bisect in order to track which commit made the change you have observed.

[LingyuMa]

Thanks, I will check if this is the reason why the accuracy decreases.

Btw, Do you think I can apply some Multicore Bundle Adjustment (pba) to accelerate the openMVG a bit (just curious)?

[pmoulon]

I don't know if the time difference will be large, I use Ceres in multi-thread mode already.
Perhaps PBA can be faster, but on the other hand PBA licence is restrictive and it's API is less generic.

Perhaps you can have more gain, by compiling SuiteSparse in multithread mode and use it in Ceres.

[LingyuMa]

Hi, I have tried to checkout to the git checkout c089b71bf443b938da5a34e01c27e34a286269db,
but it seems that the result does not improve. Could you think of any changes that might influence the accuracy?

[pmoulon]

Since you used an old version, the only major change I think about is a change about the default feature matching method.
In order to check that, you can try to perform the matching using ANNL2 instead of using the default FASTCASCADEHASHINGL2.
openMVG_main_ComputeMatches -n ANNL2 -f 1 ...

info. => f 1 is used to recompute matches, even they already exist.

[pmoulon]

@LingyuMa Can we close this issue?
Or do we need to discuss it further.
It would be interesting to have your feedback regarding accuracy vs. matching method.

[pmoulon]

Feel free to reopen, if you have any other questions.