Orb_slam2: Get live images?

[notenti]

Has anyone had any luck getting images from a camera in real time not using ROS?

[cipri-tom]

Hey. Did you have any success with this ?

[notenti]

@cipri-tom yeah I did. I just overwrote mono_tum.cc with this code:

#include <iostream>
#include <algorithm> 
#include <fstream>
#include <chrono>
#include <opencv2/core/core.hpp>
#include <opencv2/opencv.hpp>
#include <System.h>

using namespace std;
int main(int argc, char **argv) 
{
   if(argc != 3) 
   {
     cerr << endl << "Usage: ./path_to_PF_ORB path_to_vocabulary path_to_settings path_to_dev_video" << endl;
     return 1;
   }
   cv::VideoCapture cap(1);
   if (!cap.isOpened())
   {
     cerr << endl  <<"Could not open camera feed."  << endl;
     return -1;
   }
   // Create SLAM system. It initializes all system threads and gets ready to process frames.
   ORB_SLAM2::System SLAM(argv[1],argv[2],ORB_SLAM2::System::MONOCULAR,true);

   cout << endl << "-------" << endl;
   cout << "Start processing sequence ..." << endl;

   // Main loop
   int timeStamps=0;
   for(;;timeStamps++)
   {
     //Create a new Mat
     cv::Mat frame;

     //Send the captured frame to the new Mat
     cap>>frame;

     // Pass the image to the SLAM system
     SLAM.TrackMonocular(frame, timeStamps);
    }

   // Stop all threads
   SLAM.Shutdown();

    // Save camera trajectory
    SLAM.SaveKeyFrameTrajectoryTUM("KeyFrameTrajectory.txt");
    return 0;
}

Just note the VideoCapture cap(1) may change on your system depending on where your USB cam (or other cam) is publishing images. Mine just happened to be /dev/video1 (webcam was /dev/video0). Let me know if this helps.

[cipri-tom]

That's exactly what I was thinking of doing! Thank you!
However, I was worrying of the timeStamps parameter. Is it just so the SLAM can order the frames or they have to be more accurate ? I see you're just incrementing them so I suppose it's the former, i.e. used just for ordering without the need of accuracy.

[notenti]

@cipri-tom Yeah, honestly that was the only part that was confusing when I was reading through the initial code (mono_tum.cc), but I figured I would give it a shot with an increment every frame. Thankfully it worked, so I think your assessment is correct in that it is just for ordering. It's possible that this is off base, but I haven't seen what I would consider poor performance doing it this way, so you should be okay.

[carlosfarinhas]

@notenti Thank you so much! It is working on Windows 10 too. Build with Visual Studio 15.
One think I still cannot do is to use ORBvoc.bin, in order to increase speed...

[Toumi0812]

Hi @notenti,
It works, thank you very much

[carlosfarinhas]

Hi @notenti, I managed to speed up loading with ORBvoc.bin.
But pagonlin and webcam still slow..

[akashshar]

@notenti thanks for the code. What do you mean by path_to_PF_ORB? What argument should I pass here?
Thanks!

[cipri-tom]

@akashshar path_to_PF_ORB is the path to the binary resulted when you compile this project/file

[notenti]

@akashshar @cipri-tom is correct

[carlosfarinhas]

Do you guys have a unity 3d sample of this work?

[XuzzLLL]

Hi @notenti,Thank you so much!
It works, thank you very much

[Jasonzhang11111]

Hi, could anyone help me to clarify what are these inputs: ./path_to_PF_ORB path_to_vocabulary path_to_settings path_to_dev_video?

if you could give an example it will be great, thank you

[notenti]

@Jasonzhang11111 it's been a while since I've looked at this code, but I'm fairly certain path_to_PF_ORB is just the binary you get as a result of compiling the project. path_to_vocabulary and path_to_setting are most likely configuration files or configuration folders that ORB_SLAM uses...you will want to double check that, though. Really not sure what path_to_dev_video is. My first thought was that it was the /dev/videoX interface, but I really am not sure. Sorry I couldn't be more helpful!

[Jasonzhang11111]

Thank you @notenti for your help.

I also realise inconsistency in the original code. argv[3] and argv[4] are never being used. And although there are four inputs, the code checks if number of inputs are equal to 3! It is a little bit confusing.

[barloccia]

Hello everyone,
i would like to overwrite mono_tum.cc as well, when i do then i compile from terminal using g++ (i'm on ubuntu 16.04) but i receive 'fatal error: System.h: No such file or directory'. The library is in the same directory but i think something different has to be do, Can you explain me how to modify single files without re-build everything? Thanks for the help and sorry for the off-topic!

[varunns]

Hi @notenti, I managed to speed up loading with ORBvoc.bin.
But pagonlin and webcam still slow..

@carlosfarinhas were you able to solve this issue? I am actually not able to get live feed when I am using pangolin at all. Was wondering if you solved this.

Thank you very much!!