Meshroom: FeatureExtraction - SURF and BRISK

[leviresende]

For a while, I am using the software Meshroom, and then I just decided to try using other algorithms for features extraction, such as SURF and BRISK, and by default, these 2 aren't available there. So, my question: is it possible to implement these two algorithms in some script and then attached it to the software? Whether yes, how can I do it?

[fabiencastan]

Hi Levi,

It would be interesting to add support for more feature types into Meshroom.

You can implement new feature detectors here:
https://github.com/alicevision/AliceVision/tree/develop/src/aliceVision/feature
And declare the new describers here:
https://github.com/alicevision/AliceVision/blob/develop/src/aliceVision/feature/imageDescriberCommon.hpp

You have some examples of feature types from OpenCV here:
https://github.com/alicevision/AliceVision/tree/develop/src/aliceVision/feature/openCV
So if you plan to add SURF and BRISK from opencv implementation, that would be a very good starting point.

And finally add the arguments on the Meshroom nodes (FeatureExtraction, FeatureMatching, StructureFromMotion) declaration:
https://github.com/alicevision/meshroom/tree/develop/meshroom/nodes/aliceVision

We can setup a confcall if you want to discuss about the implementation details.

[leviresende]

Thanks @fabiencastan !

I already have these algorithms in python, but I saw that the codes of the extractors are in c++ . First of all, I'm gonna convert them, and then I'll let you know. Or, if you would rather we can to have a confcall first.

[fabiencastan]

Sounds good to me!
You can create a Pull Request with "[WIP]" in the title, so we can follow your progression and put remarks while you're working on it to ensure that it will be merged at the end.

[leviresende]

@fabiencastan as feedback I'll summarize what I'm doing (please correct me if something is wrong.)
As I'll work with OpenCV, I need to create the files:
-> ImageDescriber_SURF_OCV.cpp (done)
-> ImageDescriber_SURF_OCV.hpp (done)
-> ImageDescriber_BRISK_OCV.cpp
-> ImageDescriber_BRISK_OCV.hpp

For SURF I'll use the SIFT algorithms as a base, and for BRISK I'll use AKAZE algorithms.

Once I have that files, I should edit the folowing files:
-> regionsFactory.hpp
-> imageDescriberCommom.cpp
-> imageDescriberCommom.hpp
-> ImageDescriber.cpp
-> Meshroom nodes

Thanks

[leviresende]

I've done guys! I hope that is fine. In case of issues let me know.

[fabiencastan]

Great!
Could you create the PR on AliceVision? Currently the PR is on your fork between 2 of your branches.

[leviresende]

@fabiencastan I don't how to do that. Can you help me?

[fabiencastan]

When you create a new PR you can choose the base repository:

Use "alicevision/AliceVision" as base repository.

[leviresende]

I merged the code here.

[fabiencastan]

I have create the PR on AliceVision:
https://github.com/alicevision/AliceVision/pull/702

[fabiencastan]

@leviresende if you are interested to continue on this topic after this first nice contribution, there are some other feature types from opencv that would be interesting to support:
https://docs.opencv.org/3.4/d0/d13/classcv_1_1Feature2D.html
In particular: "orb", "daisy", "latch" and "fast".

And it would be also interesting to add another marker detector from opencv:
https://docs.opencv.org/trunk/d5/dae/tutorial_aruco_detection.html

[leviresende]

For sure @fabiencastan! However, I can't do that for now (I need to finish some projects this year yet) ... As I said before, I'm using the Meshroom for a while, and I think that'll pleasure to help you guys with this project. As soon as possible, I'll come back!