Airsim: Add Sensor to car's reporter

[DrifterFun]

• OS:Windows 10; Airsim Version :1.1.7 Unreal Version: 4.18 ;
• Could you please tell to how to add sensors in AirLib to car?
• I find that there are sensors on Multirotor's reporter and not on Car's reporter. I have read the code of CarPawn and FlyingPawn. However, I couldn't find the place define sensors on Multirotor.

[TheGreatGalaxy]

you can find answer though the following links:
https://github.com/Microsoft/AirSim/blob/master/docs/settings.md
https://github.com/Microsoft/AirSim/blob/master/docs/sensors.md

[DrifterFun]

Thanks for your reply. I followed the documents you provided above. My setting file like following:

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,

  "Vehicles": {

        "PhysXCar": {
            "VehicleType": "PhysXCar",
            "AutoCreate": true,
            "Sensors": {
                "IMU": { 
                    "SensorType": 2,
                    "Enabled" : true
                }
            }
        }
   }
}

However, I still can't find the IMU data in car's reporter when I press ";" in AirSim gameMode.

[TheGreatGalaxy]

sorry , i am not understand about "press ; to find data". I think you should use API in your code to collect sensor's data.

[DrifterFun]

dahewujiang, thinks for your reply. When you press ";" in AirSim, there will be a reporter showing on your screen. I found that the value of sensors will show in reporter of drone and not in car's.

I agree with the idea you said (we should use API to get data). However, I noticed that there are no API to read data of IMU or GPS.

[DrifterFun]

I studied the code, and I found that the reason for the phenomenon is that the difference of GameMode.

• Car's Gamemode is based on "SimModeBase";
• Drone's GameMode is based on "SimModeWorldBase";

However, I still have no idea about how to modify Car's Gamemode and make it based on "SimModeWorldBase".

[DrifterFun]

Dear community: I found a way to add sensor data to car's reporter (however I don't think it is an appropriate way to do such stuff). It just a quick way to get the same consequence. You just need to do steps following:

• Change the Setting.json as above.

• Change updateDebugReport Function in SimModeBase.cpp as following:

void ASimModeBase::updateDebugReport(msr::airlib::StateReporterWrapper& debug_reporter)
{
    debug_reporter.update();
    debug_reporter.setEnable(EnableReport);

    if (debug_reporter.canReport()) {
        debug_reporter.clearReport();

        for (auto& api : getApiProvider()->getVehicleSimApis()) {
            PawnSimApi* vehicle_sim_api = static_cast<PawnSimApi*>(api);
            msr::airlib::StateReporter& reporter = *debug_reporter.getReporter();

            std::string vehicle_name = vehicle_sim_api->getVehicleName();

            reporter.writeHeading(std::string("Vehicle: ").append(
                vehicle_name == "" ? "(default)" : vehicle_name));

            const msr::airlib::Kinematics::State* kinematics = vehicle_sim_api->getGroundTruthKinematics();

            reporter.writeValue("Position", kinematics->pose.position);
            reporter.writeValue("Orientation", kinematics->pose.orientation);
            reporter.writeValue("Lin-Vel", kinematics->twist.linear);
            reporter.writeValue("Lin-Accl", kinematics->accelerations.linear);
            reporter.writeValue("Ang-Vel", kinematics->twist.angular);
            reporter.writeValue("Ang-Accl", kinematics->accelerations.angular);
        }

        /*Add Sensordata to reporter*/
        if (getApiProvider() == nullptr)
            return;

        for (auto& sim_api : getApiProvider()->getVehicleSimApis()) {
            PawnSimApi* pawn_sim_api = static_cast<PawnSimApi*>(sim_api);
            std::string vehicle_name = pawn_sim_api->getVehicleName();
            msr::airlib::StateReporter& reporter = *debug_reporter.getReporter();

            msr::airlib::VehicleApiBase* api = getApiProvider()->getVehicleApi(vehicle_name);
            if (api != nullptr) {

                msr::airlib::uint count_imus = api->getSensors().size(msr::airlib::SensorBase::SensorType::Imu);

                for (msr::airlib::uint i = 0; i < count_imus; i++) {
                    // TODO: Is it incorrect to assume LidarSimple here?
                    const msr::airlib::ImuSimple* imu =
                        static_cast<const msr::airlib::ImuSimple*>(api->getSensors().getByType(msr::airlib::SensorBase::SensorType::Imu, i));
                    if (imu != nullptr) {
                        reporter.writeValue("IMU-Ang", imu->getOutput().angular_velocity);
                        reporter.writeValue("IMU-Lin", imu->getOutput().linear_acceleration);
                    }
                }

                msr::airlib::uint count_gpss = api->getSensors().size(msr::airlib::SensorBase::SensorType::Gps);

                for (msr::airlib::uint i = 0; i < count_gpss; i++) {
                    // TODO: Is it incorrect to assume LidarSimple here?
                    const msr::airlib::GpsSimple* gps =
                        static_cast<const msr::airlib::GpsSimple*>(api->getSensors().getByType(msr::airlib::SensorBase::SensorType::Gps, i));
                    if (gps != nullptr) {
                        reporter.writeValue("GPS-Loc", gps->getOutput().gnss.geo_point);
                        reporter.writeValue("GPS-Vel", gps->getOutput().gnss.velocity);
                        reporter.writeValue("GPS-Eph", gps->getOutput().gnss.eph);
                        reporter.writeValue("GPS-Epv", gps->getOutput().gnss.epv);
                    }
                }

                msr::airlib::uint count_barometers = api->getSensors().size(msr::airlib::SensorBase::SensorType::Barometer);

                for (msr::airlib::uint i = 0; i < count_barometers; i++) {
                    // TODO: Is it incorrect to assume LidarSimple here?
                    const msr::airlib::BarometerSimple* barometer =
                        static_cast<const msr::airlib::BarometerSimple*>(api->getSensors().getByType(msr::airlib::SensorBase::SensorType::Barometer, i));
                    if (barometer != nullptr) {
                        reporter.writeValue("Baro-Alt", barometer->getOutput().altitude);
                        reporter.writeValue("Baro-Prs", barometer->getOutput().pressure);                   }
                }

                msr::airlib::uint count_magnetometers = api->getSensors().size(msr::airlib::SensorBase::SensorType::Magnetometer);

                for (msr::airlib::uint i = 0; i < count_magnetometers; i++) {
                    // TODO: Is it incorrect to assume LidarSimple here?
                    const msr::airlib::MagnetometerSimple* magnetometer =
                        static_cast<const msr::airlib::MagnetometerSimple*>(api->getSensors().getByType(msr::airlib::SensorBase::SensorType::Magnetometer, i));
                    if (magnetometer != nullptr) {
                        reporter.writeValue("Mag-Vec", magnetometer->getOutput().magnetic_field_body);
                    }
                }
            }
        }

    }
}

[tiax615]

Well,,But how to use API to get the Imu data?

[DrifterFun]

I have found the API for IMU data before. However, I found there are not such APIs and Airsim's author said that they plan to add such API. If you only want to get the ground truth you can refer this issue

[pengjili]

@DrifterFun please forgive my very silly question. Now I add your code into my project, but everytime I try to build it I find it just doesn't work. The way I build it is running build.cmd. So can you tell me how to additional build Airsim?

[DrifterFun]

@pengjili Thanks for you try my code. However, it is just not a good way to get data. I did not try to build the project through build.cmd. I additional build Airsim in Unreal Project. In other words, I just change the codes in the "Plugin" folder and rebuild it in an Unreal project.
I hope this reply will give some help.

[pengjili]

@DrifterFun Thanks so much. This is really a smarter way. Now I think I want to write some code based upon your code to extract the IMU data to a txt file. I don't know if it's gonna work. I also really hope Airsim developers can add new sensor APIs.

[Misslindalian]

Hello,May I ask you a few questions? Does the Airsim platform know the sensor information of other cars? The research I did not only needed to use the information of the car, but also the information of the vehicles in other scenes.Thanks a lot

[madratman]

@pengjili @tiax615 @DrifterFun see https://github.com/Microsoft/AirSim/pull/1920 for IMU, baro, magneto, and gps API