Airsim: Build LIDAR capabilities in AirSim

[shrimalmadhur]

I'll be adding LIDAR capabilities.
ETA: 12/4 TBD

[mitchellspryn]

@shrimalmadhur Did you mean 12/4? It's currently 11/19...

[shrimalmadhur]

Thanks. Updated!

[sytelus]

Thanks. It would be great if you can summarize/outline your design.

[shrimalmadhur]

I saw your comments in some other Issue that says to extend the SensorBase class the same way other Sensors have extended it. And then I can use the Raycasting function which is already there in UAirBluePrintLib. I am planning to do it in the same way.

[tejaskhot]

Would this not be available for use with the drone?

[sytelus]

@tejaskhot yes. All AirSim sensors are reusable across vehicle types.

[nnayeer]

Any updates? Eagerly awaiting this.

[shrimalmadhur]

Got caught up in other stuff. Will update the issue once I have some update.

[realdealneil]

I am particularly interested in adding a laser altimeter to the simulator (or at least having ground truth for height-above-terrain). I'm guessing that's a subset of what you're planning to do?

[tejaskhot]

Is there any update on this?

[benasena]

I have most of the pieces of this working. I did the ray trace in VehiclePawnWrapper, then passed it through MultiRotorConnector to a variable added to Environment to get it in a distance sensor class based on the barometer implementation. I added the necessary reporters and MAVLink handlers, and can see the measurement in QGroundControl in the analyze widget and if I run "listener distance_sensor" in nsh it all looks good. I set EKF2_HGT_MODE=2 and it flies well in all modes.

For some reason it still doesn't seem to follow terrain, though. Am I missing parameters or is there something about the HIL firmware that ignores distance sensors? I believe I correctly implemented the members of the DISTANCE_SENSOR MAVLink message (http://mavlink.org/messages/common). My units are all centimeters there and I get meters when I run the nsh command. I did use Unix epoch time instead of boot time. Could that be it? Is there a good way to get an appropriate value for that in AirSim?

[nnayeer]

I work mainly on ROS, but I've managed to get a pointmap stream using depthperspective images (passing it through the appropriate conversions and ros_perception/depth_image_proc). This feels good for ground truth data but maybe some filters and noise would make it more like an actual lidar? Would be especially handy if I could spawn 4 cameras at the same point (rotated 90 degrees) and get a 360 degree pointcloud.

[benasena]

Hm, so when I set the output of the distance sensor to constant 0 and had EKF2_HGT_MODE set to use it, it seemed to fall back on the barometer so I'm not convinced it was ever using the distance sensor. I'll have to check the PX4 firmware code to see how this needs to be integrated I guess.

Interestingly, if I set it to computer vision height mode, it just rejected altitude control instead of falling back.

[benasena]

Looks like it gets copied here:
https://github.com/PX4/Firmware/blob/master/src/modules/ekf2/ekf2_main.cpp#L386
_vdist_sensor_type(this, "HGT_MODE", true, _params->vdist_sensor_type)

And is used here to determine the mode with fallbacks.
https://github.com/PX4/ecl/blob/master/EKF/control.cpp

It does look like it only rejects it based on timeout, so I'll see about using the boot time somehow rather than epoch time.

[zergler]

Thanks, any updates on this? For what I'm doing, it's very important to get as close as possible to the real deal, so the simulated LIDAR should be configurable and be able to model different LIDARs. Some of the parameters I see this sensor having are:

• Min and max laser range
• Number of lasers per sensor array
• Rotation speed of LIDAR sensor array
• Number of laser sets (usually 2)
• Vertical and horizontal field of view for each set
• Angle between each set
• Spacing between each set

[curly]

Hm, what I did was just for a fixed single laser. A scanning LIDAR shouldn't be much harder to implement, though there are more parameters to accommodate for specific implementations. In general, the rotation can be accomplished with modular arithmetic on UGameplayStatics::GetTimeSeconds(GetWorld()).

https://docs.unrealengine.com/latest/INT/API/Runtime/Engine/Kismet/UGameplayStatics/GetTimeSeconds/index.html

[benasena]

I finally got around to testing the utilization of the rangefinder for height estimation in SITL. At first it still wasn't reacting to terrain but then I realized it was setup for inav instead of ekf2. Whoops. I tested in SITL with SYS_MC_EST_GROUP and other parameters set appropriately and it worked with very clear terrain reaction. I definitely have settings for ekf2 on HIL thought and am not getting it to use the rangefinder as easily, but I might compare the settings more closely and do more tests.

[jeremypjoseph]

any update on adding scanning lidar to airsim?

[DrifterFun]

I notice that a similar system called Carla has Lidar ability. Is it posiible to transplant the codes into AirSim？