Meshroom: [request] Complete json settings file

[nicofirst1]

Is your feature request related to a problem? Please describe.
I would like to run the meshroom pipeline from command line using a custom setting in json format. The problem is that I have no clue on how the _settings.json_ should be constructed.

Describe the solution you'd like
A default _setting.json_ would be ideal to understand the structure or just to have a general overview of the parameters.

[natowi]

This might be interesting for you: http://filmicworlds.com/blog/command-line-photogrammetry-with-alicevision/ (run Alicevision Meshroom from .bat files)

[nicofirst1]

This is more of an alternative solution to my problem.
I would rather run the the whole pipeline with _meshroom_photogrammetry_ modifying the _setting.json_ programmatically depending on the dataset.

[fabiencastan]

Yes, we should provide an exemple of setting.json in the manual.

[simogasp]

Hi @nicofirst1 , it is actually quite simple but, obviously, it is not documented at all.
If u look at the .mg project file, u'll see that it is a simple json with the description of the graph and the nodes with all the input/output entries and internal parameters. For example

    "graph": {
         # the node Feature extraction: by default nodes have the same name as their type with a _N prefix to numerate them (in case u have multiple nodes of the same type in ur pipeline
        "FeatureExtraction_1": {
            # its input parameters contain both the entries (the connections to other nodes) and the actual parameters for the algorithm
            "inputs": {
                # these are input parameters with the relevant values
                "verboseLevel": "info", 
                "describerPreset": "high", 
                "describerTypes": [
                    "sift"
                ], 
                "forceCpuExtraction": false, 
                 # this is a input connection, in this case it takes as input the output param{CameraInit_1.output}  of the CameraInit node
                "input": "{CameraInit_1.output}"
            }, 
            "nodeType": "FeatureExtraction", 
            "uids": {
                "0": "6f5afb5e15ae6c53668bb213677f0d1be9e07fa5"
            }, 
            "parallelization": {
                "blockSize": 40, 
                "split": 1, 
                "size": 15
            }, 
            # these are output connections, same principle as the input connections
            "outputs": {
                "output": "{cache}/{nodeType}/{uid0}/"
            }, 
            "position": [
                0, 
                0
            ], 
            "internalFolder": "{cache}/{nodeType}/{uid0}/"
        }, 
       ....
      # etc

To override the parameters you need to create a setting.json in which you declare the new values for each desired parameter. The structure is very similar, for each node that needs changes you just re-declare the parameter that you want to change. This is, for example, a setting.json that I use to override the parameters of 6 nodes:

{
   # for the node FeatureExtraction i changed 3 parameters. 
   # You just use a flat structure, i.e. no need to use input and output as declared in the mg file
    "FeatureExtraction_1": {
        "describerPreset": "high",
        "forceCpuExtraction": "0",
        "describerTypes": [
            "sift"
        ]
    },
    "FeatureMatching_1": {
        "guidedMatching": "0",
        "describerTypes": [
            "sift"
        ]
    },
    "StructureFromMotion_1": {
        "useLocalBA": "0",
        "lockAllIntrinsics": "0"
    },
    "PrepareDenseScene_1": {
        "outputFileType": "png"
    },
    "Meshing_1": {
        "maxInputPoints": "50000000",
        "pixSizeMarginFinalCoef": "50"
    },
    # here for example i changed some input connection to publish more data in the Publish node
    "Publish_1": {
        "output": "testSfM20180524env",
        "inputFiles": [
            "{Texturing_1.outputMesh}",
            "{Texturing_1.outputMaterial}",
            "{Texturing_1.outputTextures}",
            "{StructureFromMotion_1.outputViewsAndPoses}",
            "{PrepareDenseScene_1.outputUndistorted}"
        ]
    }
}

So as you see, as long as you have some code/lib to write a json file it is quite simple to do programmatically.
I hope this helps to get you started.
We will try to add the documentation of the command line soon...

[nicofirst1]

I wrote a semi-complete _settings.json_ with default values. Hope this helps

{
    "FeatureExtraction_1": {
        "describerPreset": "normal",
        "forceCpuExtraction": "0",
        "describerTypes": [
            "sift"
        ],
        "verboseLevel":"info"
    },
    "FeatureMatching_1": {
        "guidedMatching": "0",
        "describerTypes": [
            "sift"
        ],
        "verboseLevel":"info"

    },
    "StructureFromMotion_1": {
        "useLocalBA": "1",
        "lockAllIntrinsics": "0",
        "describerTypes": [
            "sift"
        ],
        "minInputTrackLength":"2",
        "lockScenePreviouslyReconstructed": "0",
        "maxNumberOfMatches": "0",
        "verboseLevel":"info"

    },
    "PrepareDenseScene_1": {
        "verboseLevel":"info"
    },
    "DepthMap_1": {
        "downscale": "2",
        "sgmMaxTCams": "10",
        "refineMaxTCams": "6",
        "verboseLevel":"info"
    },
    "DepthMapFilter_1": {
        "minNumOfConsistentCams": "3",
        "minNumOfConsistentCamsWithLowSimilarity": "4",
        "verboseLevel":"info"
    },

    "Meshing_1": {
        "maxInputPoints": "50000000",
        "maxPoints": "5000000",
        "estimateSpaceMinObservationAngle": "10",
        "colorizeOutput": "0",
        "verboseLevel":"info"

    },
    "MeshFiltering_1": {
        "removeLargeTrianglesFactor": "60",
        "keepLargestMeshOnly":"0",
        "iterations": "5",
        "verboseLevel":"info"

    },

    "Texturing_1": {
        "textureSide": "8192",
        "downscale":"1",
        "outputTextureFileType":"png",
        "unwrapMethod":"Basic",
        "useUDIM":"1",
        "fillHoles":"0",
        "verboseLevel":"info"

    }

}

[nicofirst1]

Are there any update on this?
Using the _setting.json_ file I provided overrides _every_ parameter.
So if you use a custom setting file without specifying all the possible parameters it will run the pipeline without them, thus getting stuck at featureExtraction.

[uberVelocity]

Are there any updates on this issue?

[KonstantinRr]

There does not to be a '.mg' file in this or the main AliceVision repository. Where is it located or are there any new updates on this issue?

[natowi]

@KonstantinRr You can save the default graph from the GUI

[nicofirst1]

Using the GUI default graph yields error for every key in the setting dictionary.
Seems like there is a big difference between the GUI setting and the mesh_compute one.

[natowi]

This might be useful:
https://github.com/alicevision/meshroom/pull/413 and #632

meshroom_photogrammetry: "--save" does not disable the computation so you can save your batch reconstruction project and open it later interactively in meshroom. Additional --compute option if you need only to create a meshroom project file from this command line.

new positional argument to set project file or input folder
new "--pipeline" option to override the default pipeline with your own project file. It also supports an environment variable MESHROOM_DEFAULT_PIPELINE.