Airsim: Stabilizing camera images/Adding a gimbal

[hngenc]

We're running tracking algorithms on images we get from AirSim's cameras. However, when the drone stops or changes direction, the camera "wobbles", sometimes causing our tracking algorithm to fail.

Is there some way to stabilize the FPV camera so that it's roll and pitch remain constant? I'd like to have the same effect that I would get from attaching a gimbal to the drone.

[lovettchris]

Implementing a proper camera gimbal would be awesome (something like this). You might be able to just set the fpv camera's rotation using SetActorRotation to cancel the orientation of the drone, perhaps setting some limits so camera doesn't look inside the drone in extreme cases... see CameraDirector and UAirBlueprintLib::FollowActor for example of how we do that with the external chase camera.

[hngenc]

@lovettchris Thanks! I managed to simulate a gimbal by doing as you said. For anyone else interested in doing the same, I added #include "VehiclePawnBase.h" to AirBlueprintLib.cpp and then added the following three lines to the end of UAirBlueprintLib::FollowActor:

AActor * fpv_cam = ((AVehiclePawnBase*)followee)->getFpvCamera();
FRotator drone_orientation = ((AVehiclePawnBase*)followee)->getOrientation();
fpv_cam->SetActorRotation(FRotator(0, drone_orientation.Yaw, 0));

[lovettchris]

Hey congrats, that is cool to know it can be done in only 3 lines of code. Thanks for posting the answer!

[M-Kasem]

@lovettchris After the modification in the repo, I changed #include "VehiclePawnBase.h" to #include "VehiclePawnWrapper.h"
and these three lines in the end of UAirBlueprintLib::FollowActor

AActor * fpv_cam = ((VehiclePawnWrapper*)followee)->getCamera();
FRotator drone_orientation = ((VehiclePawnWrapper*)followee)->getOrientation();
fpv_cam->SetActorRotation(FRotator(0, drone_orientation.Yaw, 0));

Changed VehiclePawnBase to VehiclePawnWrapper and getFpvCamera() to getCamera()

and successfully recompiled AirSim, when I open the blocks Environment with the car it runs fine, after pressing f unreal crashes, and if I choose to run with the multirotor unreal also crashes, do you know how could I solve this problem, or how to add the gimbal in the last commit in the repo?

Thanks a lot.

[sytelus]

I just added Gimbal APIs in AirSim. Here's doc.