Openmvg: [SfM/pipelines] Incremental/Sequential SfM - Automatic pair selection to start the reconstruction

[pmoulon]

Automatic initial pair selection to seed the reconstruction:
Naive method:

• Will use the median angles between the bearing vectors of pairs that support valid intrinsic data & valid robust estimation of an essential matrix.

[pmoulon]

Thanks to @fabiencastan for testing.

[MEGA728]

Hi !
How to use this automatic pair selection to start the reconstruction? I didnt find this option in the
incremental pipeline.

[pmoulon]

Automatic pair selection is the default choice.

[MEGA728]

Really? But it always stops at "type INITIAL pair ids: X enter Y enter". What should I do if I want to let the program choose the initial pair automatically?
Thanks!

[pmoulon]

Did you check if you have some intrinsic data configured for your images?

• the field intrinsics in the sfm_data.json file must not be empty -> intrinsic data is required at least for two images that share some keypoint to bootstrap the 3d reconstruction.

[MEGA728]

I have intrinsic data in the sfm_data.json file. The images were taken by one camera, so the intrinsics are the same.

The log is as follows. I see a step of "Automatic selection of an initial pair:", but after that it still asks me to " type INITIAL pair ids: X enter Y enter".

Sequential/Incremental reconstruction
Perform incremental SfM (Initial Pair Essential + Resection).

• Features Loading -
  0% 10 20 30 40 50 60 70 80 90 100%
  |----|----|----|----|----|----|----|----|----|----|

---

Track building

Track filtering

Track export to internal struct

Track stats

-- Tracks Stats --
Tracks number: 63
Images Id:

0, 1, 2,

TrackLength, Occurrence
3 63

Automatic selection of an initial pair:
0% 10 20 30 40 50 60 70 80 90 100%
|----|----|----|----|----|----|----|----|----|----|

---

---

SequentialSfMReconstructionEngine::ChooseInitialPair

Pairs that have valid intrinsic and high support of points are displayed:

Choose one pair manually by typing the two integer indexes

(0,1) 63 matches
(0,2) 63 matches
(1,2) 63 matches

type INITIAL pair ids: X enter Y enter

[pmoulon]

Do you can share your dataset to let me play with?

[MEGA728]

Sure. Thanks!
simulation_data.zip

[MEGA728]

I may find where the problem is. In sequential_SfM.cpp I saw a function AutomaticInitialPairChoice(Pair & initial_pair), with three parameters:
const unsigned iMin_inliers_count = 100;
const float fRequired_min_angle = 3.0f;
const float fLimit_max_angle = 60.0f;

In my case, there are only 63 matches of points between each pair of images. So I changed iMin_inliers_count into 50. But the automatic initial pair choosing is still not implemented.

Is changing the code in sequential_SfM.cpp the right way to do it, or should I make the change somewhere else?

[pmoulon]

I will try to check this week and provide you some feedback.

[sebastienchappuis]

Hello, I am right now facing some similar issues as @MEGA728 regarding that subject since I am trying to have a completely automated pipeline.
Would it be reasonable to have a « fully automatic » mode where the user input is never asked for ? I guess in that case, the initial pair would be the one that has the most matches, and if the reconstruction fails then be it.

What do you think ?

[pmoulon]

The main problem is that the pair with most of the match can also be a pair without parallax or an two image taken exactly at the same spot.
You can easily suggest two initial image for the IncrementalSfM pipeline. Feel free to write a script that find the pair you want to use and send it to the IncrementalSfM binary.

You can also help to improve the initial pair choice strategy!

@sebastienchappuis Did you meet also a case when you very few common points between your images?

[sebastienchappuis]

@pmoulon I've encountered both the following cases :

• few common points (below the iMin_inliers_count threshold)
• a lot of common points, but a scoring_angle which ends up being smaller than the fRequired_min_angle threshold. I end up with an empty scoring_per_pair.

I guess it's alright if the reconstruction fails when those conditions are not met, but the way it is done now is that the user input is asked and the whole process is put on hold. Now, if I want to run muliple reconstructions one after the other with a script, I would like the user not to have to check his computer everynow and then to see if one of the reconstructions are hanging...

And I guess that regarding @MEGA728 problem, the question is do we keep those thresholds hardcoded, or would it make more sense to make them data members of the SequentialSfMReconstructionEngine that could be initialized at will via the constructor ?

ps : If you prefer I can also open a proper issue on that.

pps : There's also a different problem which is that sometimes a failing automatic pair detection will succeed if you run it a second time, because of the non-deterministic random variable initialization in the robustRelativePose function, but I guess that you are already tackling that problem on the develop_random_number_reproducibility branch.