Openmvg: [Question] Sfm Data Export, Feature 2D location in image.

[ghalmets]

Hello,
first of all thank you for the nice toolbox.

I was wondering if the Observation that are exported using sfm_data_io_baf are real observations based on the Feature extractor or "virtual" (reprojected) observations based on {the 3D feature, camera pose and camera model}.

If the latter, why? This information is redundant giving all the data. In opposite, the original feature location cannot be recovered. Is there a way to directly export the original 2D observation location?

Cheers,
Georg

[pmoulon]

Hi,
SfM_Data store the raw feature location.
As you can see here they are not modified: https://github.com/openMVG/openMVG/blob/master/src/openMVG/sfm/sfm_data_io_baf.hpp#L104

Perhaps we can update the doc here https://openmvg.readthedocs.io/en/latest/openMVG/sfm/sfm/ since it was not clear for you.

[pmoulon]

BTW, I would like recommend you to use the json export if you need to parse a file with the data.
Or use the OpenMVG API and read the sfm_data.bin.

[ghalmets]

Hi,

thank you for the quick reply.
I have actually checked the code already and I originally planned to export the data into the BAL format.

After I have exported the data I displayed the image pairs (view i, view n) and highlighted the feature matches according to the observation data. I figured out that they sometimes simply do not show the same feature at the indicated pixel coordinates although they should according to the SFM data. That happens for approx 5 out of 90 views.

Hence, I modified the cpp file to export the observation information using the simple peace of code at the end of this comment.

I attached my gathered Debug Info and the two images. According to the data, the feature with the ID 0, was observed in view 1 and 4 at the exactly same location, even the 3 digits match what is actually impossible (cf. first 2 lines at the top of the image I've attached). According to the reconstruction the views have a similar pose (although its obviously not the same pose). Because of the really similar poses, the same 3D feature location and the same intrinsic camera values it would explain a lot if the observations are virtual calculated before they are stored in the sfm_data.bin and later exported to any other format.

If this is not the case, where does this exactly same feature location in pixel coordinates for two different views come from?

Thank you and all the best,
Georg

Setup:
Data: every 10th rgb image of fr3/structure_texture_far
from https://vision.in.tum.de/data/datasets/rgbd-dataset/download#
Sfm reconstruction:

$path/openMVG_main_SfMInit_ImageListing -i $workdir/../../rgb/ -d ~/workspace/openMVG/src/openMVG/exif/sensor_width_database/sensor_width_camera_database.txt  -o $workdir/matches/ -k "536.009080;0;321.563744;0;535.772125;245.723102;0;0;1" -g 1 -c 3
$path/openMVG_main_ComputeFeatures -i $workdir/matches/sfm_data.json -o $workdir/matches/ -n 6 -f 0 -m SIFT -p HIGH
$path/openMVG_main_ComputeMatches -i $workdir/matches/sfm_data.json -o $workdir/matches/ -g e -f 1 -r .6
$path/openMVG_main_GlobalSfM -i $workdir/matches/sfm_data.json -m $workdir/matches/ -o $workdir/global_reconstruction/

const Landmarks & landmarks = sfm_data.GetLandmarks();
        //Export Features
        unsigned int i=0;
        for (const auto & iterLandmarks : landmarks )
        {
            // Export visibility information
            // X Y Z #observations id_cam id_pose x y ...
            const double * X = iterLandmarks.second.X.data();

            //Export Observation
            const Observations & obs = iterLandmarks.second.obs;
            for ( const auto & iterOb : obs )
            {
                const IndexT view_key = iterOb.first;
                const View * v = sfm_data.GetViews().at(view_key).get();

                double uv[2];
                uv[0]= iterOb.second.x(0);
                uv[1]= iterOb.second.x(1);
                std::cout<<"[feature_id,view_id, [u,v], pose_id, [pose], image_path]: "
                                <<i<<" , "
                                <<v->id_view<<" , "
                                <<"["<<uv[0]<<" ,  "<<uv[1]<<"]"<<", "
                                 << v->id_pose;

                const double * rotation = poses.at(v->id_pose).rotation().data();
                const double * center = poses.at(v->id_pose).center().data();
                std::cout<<", [ "<<center[0]<<" , "
                       <<center[1]<<" , "
                       <<center[2]<<" , "
                      <<rotation[0]<<" , "
                     <<rotation[1]<<" , "
                    <<rotation[2]<<" , "
                        <<rotation[3]<<" , "
                        <<rotation[4]<<" , "
                        <<rotation[5]<< " ], "
                        << v->s_Img_path<<std::endl;
            }
            i++;
        }

[pmoulon]

As you see here the point positions that are exported by the GlobalSfM triangulation are the raw feature positions:
https://github.com/openMVG/openMVG/blob/master/src/openMVG/sfm/pipelines/global/sfm_global_engine_relative_motions.cpp#L359

Please does not use feature_id, every track have already a unique ID store in iterLandmarks.first

[pmoulon]

You can also check the pairwise matches by using the binary exportMatches in order to see if the pairwise matches are ok.

[ghalmets]

Thank you, I will check the exportMatches to see if they are okay and rewrite my code to use the key. If that also doesn't help, I will try to use the json export to check if the wrong feature location is also in there.
I will leave a message as soon I have new results.

Cheers, Georg

[ghalmets]

Hi,

Problem solved!

Short update:
I found the same wrong data points in the Json file.

However, I pulled in the newest changes, recompiled everything, replaced feature_id as recommended the key and reran the global_reconstruction. The json file as well as my BAL exports look perfect now.

Cheers, Georg