Airsim: Correct way to get IMU data - Code snippet and Explanation [not bug]

[tejaswid]

Here I show how to get the IMU data correctly .
@sytelus and @madratman please comment on this and verify this. It would be nice if you can add this to the documentation after review. This post helps answer #1826 #1285.

Incoherent / Ambiguous documentation regarding getKinematicState()

The documentation suggests (not directly but most readers understand it this way) that the way to get IMU data is to do one of the following which is incorrect.

Incorrect ways to get IMU

For Car using Python API --- incorrect

client = airsim.CarClient()
lin_acc_1 = client.getCarState().kinematics_estimated.linear_acceleration  
lin_acc_2 = simGetGroundTruthKinematics().linear_acceleration  

For MultiRotor using Python API --- incorrect

client = airsim.MultirotorClient()
lin_acc_1 = client.getMultirotorState().kinematics_estimated.linear_acceleration
lin_acc_2 = simGetGroundTruthKinematics().linear_acceleration

For Car using C++ API --- incorrect

msr::airlib::CarRpcLibClient client;
lin_acc_1 = client.getCarState().kinematics_estimated.linear_acceleration  
lin_acc_2 = client.simGetGroundTruthKinematics().accelerations.linear

For MultiRotor using C++ API --- incorrect

msr::airlib::MultirotorRpcLibClient client;
lin_acc_1 = client.getCarState().kinematics_estimated.linear_acceleration
lin_acc_2 = client.simGetGroundTruthKinematics().accelerations.linear

The above snippets output the linear acceleration of the body in the world coordinate frame and NOT the simulated IMU reading. This is mentioned in the docs here. I have verified this too.

Correct IMU reading

A simulated IMU should output accelerations of the body in the body frame along with gravity. An example of how to do this is shown in this example file. If I am not mistaken, this file is what is called to output the IMU data in the Debug console (pressing ~ in Unreal Engine).

However it is not exposed in the API. So here is a snippet on how to get the IMU data.

Correct way to get IMU

C++

while(1)
    {
        msr::airlib::MultirotorRpcLibClient client;  // or msr::airlib::CarRpcLibClient
        msr::airlib::Kinematics::State k_state = client.simGetGroundTruthKinematics();
        msr::airlib::Environment::State e_state = client.simGetGroundTruthEnvironment();

        // timestamp
        auto t = Utils::getTimeSinceEpochSecs();

        // angular velocity of IMU - angular velocity in the body frame
        msr::airlib::Vector3r w_b = k_state.twist.angular;
        // linear acceleration of body in world frame
        msr::airlib::Vector3r la_b_w =  k_state.accelerations.linear;
        // subtract gravity
        la_b_w = la_b_w - e_state.gravity;
        // get acceleration in the body frame - IMU linear acceleration
        msr::airlib::Vector3r la_b = msr::airlib::VectorMath::transformToBodyFrame(la_b_w, k_state.pose.orientation, true);

       // add noise and bias as shown in the above example file

       // add sleep here to control the frame rate if necessary
       std::this_thread::sleep_for(std::chrono::duration<double>(period)
    }

Python
Unfortunately it is not possible to duplicate the above code directly as the function transformToBodyFrame is not exposed in the Python API. One can re-implement the Cpp version of this transformation if necessary.

For ROS
Use the above values t, la_b and 'w_b' to populate a sensor_msgs::imu object

The configuration on which I tested my code

• Ubuntu 16.04
• Unreal Engine 4.19.2
• AirSim at commit 1fd6a3

[madratman]

Hey @tejaswid , yes you're definitely doing the right thing. However, that AirLib/include/sensors/imu/ImuSimple.hpp you linked is not an "example" file. It's the internal implementation of the IMU sensor itself.

I've added a client side API msr::airlib::ImuBase::Output getImuData(const std::string& imu_name = "", const std::string& vehicle_name = "") in the first commit of PR https://github.com/Microsoft/AirSim/pull/1920, which has the return output has a timestamp, quaternion, angular velocity, and linear acceleration here

We'll add a corresponding ROS publisher in PR #1911.

[tejaswid]

Thanks @madratman for clarifying.
I shall close this issue here since you are adding this functionality in the upcoming PRs.

[Hezzzzzy]

@madratman
hi folks,
I'm sorry for my stupid request， but I'm not good at python call the c++ api, can u add this api to python version?

thx very much!

[Hezzzzzy]

@madratman
I have written a python api for ur C++ code, but I find the timestamp always be 0, the others are perfect,
Is there some problem in timestamp?

[v-prgmr]

@Hezzzzzy , Would be of great help if you could share the python API that you created.
Thank you once again Hezzzy

[Hezzzzzy]

@Hezzzzzy , Would be of great help if you could share the python API that you created.
Thank you once again Hezzzy

I also want to share the python api...but I find the timestamp in this api has something wrong, because it always be 0, when I solve this problem , I will share the api in the first time

[madratman]

update: I've added new sensor APIs to hello_drone.py as well for sample usage

@Hezzzzzy @vishal-prgmr , I have updated #1920's IMU commit with python API
note that the name is now getImuData and not simGetImuData

While that PR is not finished, here's how you can play with it.
First, let's check out that PR in a local branch :

Assuming "upstream" points to https://github.com/Microsoft/AirSim. (it would be "origin" by default)

$ cd AirSim;
git checkout upstream/pull/1920:PR/1920

Rebuild airsim, and update it in your unreal project
Assuming linux, that's a

$ ./clean_rebuild.sh
# Install python client from PR branch, not pip install airsim 
$ cd PythonClient
$ pip setup.py install # might need to sudo here
# update Blocks environmetn with new airsim plugin
$ cd Environments/Blocks
$ rm -rf Plugins/AirSim
$ ./update_from_git.sh

run the unreal project,
test IMU API . Here's a simple test:

$ python
Python 2.7.12 (default, Nov 12 2018, 14:36:49) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import airsim
>>> 
>>> client = airsim.MultirotorClient()
>>> client.confirmConnection()
Connected!
Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

>>> client.enableApiControl(True)
>>> client.armDisarm(True)
True
>>> 
>>> client.getImuData()
<instance> {   u'angular_velocity': <instance> {   u'x_val': -0.0012795625953003764,
    u'y_val': 0.001729224226437509,
    u'z_val': -0.0024662762880325317},
    u'linear_acceleration': <instance> {   u'x_val': 0.04842264950275421,
    u'y_val': -0.05737577751278877,
    u'z_val': -9.794503211975098},
    u'orientation': <instance> {   u'w_val': 1.0,
    u'x_val': 0.0,
    u'y_val': 0.0,
    u'z_val': 0.0},
    u'time_stamp': 1556579567317047296}
>>> client.getImuData()
<instance> {   u'angular_velocity': <instance> {   u'x_val': -0.000671305984724313,
    u'y_val': -0.0016184370033442974,
    u'z_val': 0.004206037148833275},
    u'linear_acceleration': <instance> {   u'x_val': -0.045689914375543594,
    u'y_val': -0.027373025193810463,
    u'z_val': -9.791869163513184},
    u'orientation': <instance> {   u'w_val': 1.0,
    u'x_val': 0.0,
    u'y_val': 0.0,
    u'z_val': 0.0},
    u'time_stamp': 1556579568067063296}
>>> client.getImuData()
<instance> {   u'angular_velocity': <instance> {   u'x_val': -0.0009696148335933685,
    u'y_val': -0.0013910006964579225,
    u'z_val': 0.003138612024486065},
    u'linear_acceleration': <instance> {   u'x_val': -0.05306378751993179,
    u'y_val': 0.07138057798147202,
    u'z_val': -9.87125301361084},
    u'orientation': <instance> {   u'w_val': 1.0,
    u'x_val': 0.0,
    u'y_val': 0.0,
    u'z_val': 0.0},
    u'time_stamp': 1556579568613074944}

[Hezzzzzy]

@madratman it's awesome!!! thx for your patience!!

[v-prgmr]

@madratman , I did switch to the new PR from my master and rebuilt from scratch again. Initially i got "CarClient" doesnt have the attribute getImuData. Then i regenerated the Blocks.uproject and now i guess it screwed my build.
I think i will wait until the PR is completely finished.
Thank you for this PR though !

[madratman]

@vishal-prgmr , the car only comes with a GPS, while the multirotor loads with gps, imu, magnetometer, and barometer as mentioned in our readme here.

You can enable IMU in PhysxCar by adding it to your settings.json under sensor clause.
Note that IMU is sensortype enum 2 as mentioned here

I took your settings from https://github.com/Microsoft/AirSim/issues/1919 and dumped an IMU in there.
Note that name of your IMU is "IMU1", so you'd need to pass then in the API with a getImuData(imu_name='IMU')`

Lemme know if that works. PR is finished minus documentation. It shoudn't screw your build.
Compile AirSim again with build.cmd, delete Blocks/Plugins/AirSim, call update_from_git.bat and that should be it.

{
  "SettingsVersion": 1.2,

  "SimMode": "Car",
  "RecordUIVisible": true,
  "LogMessagesVisible": true,
  "ViewMode": "",
  "Recording": {
  "RecordOnMove": false,
  "RecordInterval": 0.05,
  "Cameras": [
      { "CameraName": "front_center", "ImageType": 0, "PixelsAsFloat": false, "Compress": true },
      { "CameraName": "front_center", "ImageType": 3, "PixelsAsFloat": false, "Compress": true }
    ]

  },

  "SubWindows": [
          {"WindowID": 0, "ImageType": 0, "CameraName": "front_center", "Visible": true},
          {"WindowID": 1, "ImageType": 3, "CameraName": "front_center", "Visible": true},
          {"WindowID": 2, "ImageType": 5, "CameraName": "front_center", "Visible": true}
          ],
     "Vehicles": {
        "EgoCar": {
            "VehicleType": "PhysXCar",
            "AutoCreate": true,
      "Cameras": {
        "front_center": {
        "CaptureSettings": [
          {
          "ImageType": 0,
          "Width": 1280,
          "Height": 960,
          "FOV_Degrees": 90

          },
          {
          "ImageType": 3,
          "Width": 1280,
          "Height": 960,
          "FOV_Degrees": 90

          },
          {
          "ImageType": 5,
          "Width": 1280,
          "Height": 960,
          "FOV_Degrees": 90

          }
        ],


        "X": 0, "Y": 0, "Z": -1.9,
        "Pitch": 0, "Roll": 0, "Yaw": 0    
        }
        },

      "Sensors": {
              "IMU1" : {
                "SensorType" : 2,
                "Enabled" : true
              },
                "LidarSensor1": { 
                    "SensorType": 6,
                    "Enabled" : true,
                    "NumberOfChannels": 16,
                    "Range": 100,
                    "RotationsPerSecond": 10,
                    "PointsPerSecond": 100000,
                    "X": 0, "Y": 0, "Z": -1,
                    "Roll": 0, "Pitch": 0, "Yaw" : 0,
                    "VerticalFOVUpper": 10,
                    "VerticalFOVLower": -10,
                    "HorizontalFOVStart": -360,
                    "HorizontalFOVEnd": 360,
                    "DrawDebugPoints": true,
                    "DataFrame": "VehicleInertialFrame"
                }   
            }
        }
    }
}

[v-prgmr]

@madratman, thank you so much. I will try this asap and get back to you. 😊

[madratman]

It's merged now btw

[v-prgmr]

Hi again.
Updated my master and built it again.
Still for my call client.getImuData(imu_name="IMU1")

I still get

CarClient object has no attribute getImuData.