Meshroom: CCTag not working at all [bug]

[SugoiShades]

I have Meshroom 2019.1.0

I recently printed out all of the CCTags from this repository and positioned them around my space, and if I check the box cctag3 (or cctag4 even though I'm not using it) in the FeatureExtraction node it will immediately "crash" meaning the node immediately turns red indicating an error. I set the verbose level to "Fatal" , "Error" , and "Trace" in an attempt to diagnose the issue, yet the logs showed nothing other than my device info and the node settings. This error occurs on any level of describer present and with any combination of Descriptor types as long as it includes one of the "cctag" descriptors. I'll include a screenshot of the Meshroom program and a few of the photos i used for the project (just to prove how easily visible the cctags are to the camera) I'm not sure if there's something funky with my install of meshroom or what, but I hope my report helps

Meshroom Screenshot:

Example Photo 1:

Example Photo 2:

Example Photo 3:

[natowi]

You need to use the latest 2019.2 version.
Also: you can not use different or additional describers in later nodes,
when using 3 describers in node A you can use the same (or less) in B

[SugoiShades]

Oh, I was unaware of an update. but what exactly do you mean by node A and node B? are you just referring to a node later down the line?

[natowi]

@SugoiShades " are you just referring to a node later down the line?"
yes

[lexvandersluijs]

Hi,

First of all, thanks for making a GREAT photogrammetry program. Love the graph-based UI and the fact that it works on 4K screens :-) )

Encouraged by the exchange above, I also tried it using CCTag 3 and the latest MeshRoom (2019.2.0)

I created a set of 38 photos with 5 markers in the scene, using a Galaxy Note 8 smartphone, resolution is 2880x2160.

After dragging and dropping the pictures, I activate CCTag 3 in the FeatureExtraction node, so now both SIFT and this one are active.

I save the project in the folder where the pictures are, then press Start.

The system crunches for a while (fan blowing, CPU at about 40%) the program halts, which is mainly indicated by the Start butting becoming green again. There is no obvious error at the bottom of the Output log (pastebin, copied from the Command window, not possible from the UI <-- this could be convenient), but there is an "ERROR" string in the Status tab of the FeatureExtraction node (pastebin).

Upon further inspection I see this message in the log "ERROR:root:Error during Graph execution Error on node "FeatureExtraction_1(0)"

Any idea what could be causing this? Perhaps the camera intrinsics are required for this feature?

Have you guys as authors done an end-to-end test of a reconstruction with CCTag markers in it?

Any help would be appreciated.

[lexvandersluijs]

To partially self-answer.. I found this wonderful list of sensor information used in Samsung phones, among others https://en.wikipedia.org/wiki/Samsung_CMOS and added this line to cameraSensors.db

samsung;SM-N950F;9.96;

That seems to help a lot, the process now completes to the textured mesh stage.

However, this is most likely not the right value for this entry: it is the "Optical Format" sensor size, not the width. Does anyone have an equation to do the conversion (given the aspect ratio of the sensor)?

This may also explain that the current result is a garbled mess of triangles. Or would the system compute the correct Focal Length calibration of the camera automatically, during the reconstruction?

Note that I later realized that CCTag 3 most likely also needs to be activated in the FeatureMatching and Structure From Motion nodes (does not change quality of result).

Idea: perhaps the system can give a warning in the matching- or SfM-stage when there are certain features in the input dataset but the node has not been configured to use them?

[natowi]

@lexvandersluijs see https://github.com/alicevision/meshroom/wiki/Add-Camera-to-database so use 5.75

[lexvandersluijs]

Thanks for the link!

When I tried 5.75 I did not get a result yet, but when I calculated the sensor width using pixel size and horizonal resolution (using this link on the page you mentioned: https://www.vision-doctor.com/en/camera-calculations/sensor-diagonal-sensor-ratio.html) I arrived at 5.64, and with that value I get a 3D mesh that looks very sensible, even at the right scale!

I don't really understand why the system is so sensitive to this value, as most MVG systems will estimate the FL as part of the process, since with simpler lenses this will depend on the focus distance, so from photo to photo. But at least the setup is working now :-)

All right, I'm off to do more experiments!
Thanks again,
Lex

[lexvandersluijs]

Hi natowi,

I have created a dataset where there is a lot of internal self-similarity in the physical world, so this would be a good case to apply the CCTag technology.
However, when I enable CCTag 3 in the three required phases, the program ends with an error (see the earlier pastebin, or the one linked below) in the FeatureExtraction node.

When I disable CCTag 3, the FeatureExtraction stage completes, but ultimately the mesher fails, which is very understandable given the extreme likelyhood of camera-pose errors due to feature mismatches.

In other words, the error appears to be related to the CCTag subsystem. The log is here but does not look very useful: https://pastebin.com/n42rex6f

Image resolution is 2880 x 2160

Edit: I now have three datasets which exhibit this problem:

• one set of 68 images at 16:9 aspect ratio
• (assuming the aspect ratio was the problem) a set of 68 images at 4:3 aspect ratio
• (assuming the reflections in the image were the problem, causing duplicate CCTag detections, a set of 62 images of a similar yet completely diffuse object

Would you be interested in having a look at the dataset(s)? My mail address is my github username at gmail.com, if you would send me your address I can transfer the files via wetransfer.com. Or I can put them online somewhere, there is no privacy-sensitive information in there.

[lexvandersluijs]

Ok, I had a hunch and did some additional tests:

• I tried the dataset which worked earlier but from another folder with a longer name ("C:\Projects\rnd\MeshroomTestWithDesk") , and got the same failure as with the other datasets.

• Then I moved the photos from the working project to a folder with a shorter name ("C:\Projects\rnd\mrt") and then it works again. Weird.

However why I put a dataset that hasn't worked yet into a folder with a shorter name, it still failed. Reducing the number of photos didn't make a difference either, neither did reducing the resolution. The only thing I could do to get a result from the troublesome dataset was to disable CCTag detection.

Maybe the above gives an additional clue.

By the way, having a look at the process using Microsoft Process Monitor (or alternatively, Process Explorer) might help as well. There are some error messages in there regarding use of the Registry, although these could be benign.

[canonex]

On Meshroom 2019.2.0, I have the same problem.
Here is an example image in which it has been recognized just one CCTag (expected 3/4).

I'm using Sift e Cctag3 for:

• FeatureExtraction
• FeatureMatching
• StructureFromMotion

Intrinsic: the lens was not that of the camera and so intrinsic data is unknown.
I have tried to insert the proper focal length in the CameraInit but nothing changes:

• in Initial Focal Length I left -1
• in Focal Length was 7242.640687119285: I set 24. Is this correct?

Thank you,
Riccardo

[fabiencastan]

Regarding large CCTag, there is a known issue that has been fixed after the 2019.2 release, see https://github.com/alicevision/CCTag/pull/116.
So it will be fixed in the next release of Meshroom.

You have to make a conversion to set the focal length to 24mm as the unit used in Meshroom is in pixels.

[fabiencastan]

If you don't have focal metadata on your images, you can adjust the "Default Field Of View" on the CameraInit before dropping your images in Meshroom. So it will use this angle in degree as a default value to initialize the focal length.

[canonex]

I thought it was the last official build, I'll wait for the next.

About converting mm to pixel... I don't understand, doesn't it depend on resolution?
Anyway, what's the multiplication factor?

Thank you

[miquelrosell99]

Hello,
Where can I find information about how to use CCTag to obtain scale accurate scans? I can't find it anywhere, so I ask here. Sorry if it's not the proper place. Thanks!

[TR7]

Hello,
Where can I find information about how to use CCTag to obtain scale accurate scans? I can't find it anywhere, so I ask here. Sorry if it's not the proper place. Thanks!

I am very interested in that too! Have you found a solution yet?
Or do you know how to identify which of the points in the 3d point cloud (after SfM) correspond to the CCTAGs?

[natowi]

@miquelrosell99 @TR7 this is not yet supported. See https://github.com/alicevision/meshroom/issues/855 for details and workarounds

[fabiencastan]

Retrieve scene scale using CCTag markers is done in the development branch but not yet in a release.
The problem with large CCTag has been solved.

[natowi]

@fabiencastan is this included in https://github.com/alicevision/AliceVision/pull/695

[canonex]

Retrieve scene scale using CCTag markers is done in the development branch but not yet in a release.
The problem with large CCTag has been solved.

Is there a plan for a new release?
I would like to use this feature during a modeling class.
Thank you.

[natowi]

@canonex You could try the AliceVision (CLI) snapshot build.

[miquelrosell99]

Retrieve scene scale using CCTag markers is done in the development branch but not yet in a release.
The problem with large CCTag has been solved.

I downloaded this repo this morning, and I'm using it rn, launching Meshroom from a .bat file. Will I be able to use this feature that way?
Edit: when trying to run Meshroom from latest repo update I get this error on Image Matching...

Edit 2: Using latest AliceVision snapshot solved the issue

[fabiencastan]

@natowi Yes, it's included in alicevision/AliceVision#695. (The other one is not related).

@canonex I don't have a date for the next release yet.

[miquelrosell99]

@fabiencastan @natowi i Know it's OT but... After solving the issue with ImageMatching i get a similar one on DepthMap

Also the StructureFromMotion point cloud is not showing idk why

[fabiencastan]

This is not related to the issue.
Use the mailing list for discussion or open a new issue for another problem. Be careful to provide enough information to enable someone to help you. Basically if you have troubles on a node, the first thing to do is to enable maximum verbosity and send the log file.

[miquelrosell99]

I see. Thanks.
I'm still a noob with how github works

[fabiencastan]

No pbl. The only rule is to keep one problem per issue. So feel free to open another issue.
For open questions and discussions, it is better to use the mailing list.
Best,

[canonex]

@canonex You could try the AliceVision (CLI) snapshot build.

Unfortunately, all my Cuda hardware are under Linux :(
I'm no expert in compiling, but maybe I could try... waiting for the next official release.
Thank you.

[canonex]

CCTag3 working on Meshroom 2020.1.1 🎉