Godot: Crash in Bullet when changing a CollisionShape

[pouleyKetchoupp]

Godot version:
Master 9678a41b19e142bbc4c00f761a3a5a0cf2d4458e

OS/device including version:
Windows 10

Issue description:
Issue reproduced on master while testing the use case from #40283 (see https://github.com/godotengine/godot/issues/40283#issuecomment-657099590)

Callstack:

 godot.windows.tools.64.exe!abort() Line 77 C++
    [External Code] 
 godot.windows.tools.64.exe!btCollisionWorld::updateSingleAabb(btCollisionObject * colObj) Line 158 C++
 godot.windows.tools.64.exe!btCollisionWorld::updateAabbs() Line 212    C++
 godot.windows.tools.64.exe!btCollisionWorld::performDiscreteCollisionDetection() Line 229  C++
 godot.windows.tools.64.exe!btDiscreteDynamicsWorld::internalSingleStepSimulation(float timeStep) Line 475  C++
 godot.windows.tools.64.exe!btSoftRigidDynamicsWorld::internalSingleStepSimulation(float timeStep) Line 91  C++
 godot.windows.tools.64.exe!btDiscreteDynamicsWorld::stepSimulation(float timeStep, int maxSubSteps, float fixedTimeStep) Line 435  C++
 godot.windows.tools.64.exe!SpaceBullet::step(float p_delta_time) Line 363  C++
 godot.windows.tools.64.exe!BulletPhysicsServer3D::step(float p_deltaTime) Line 1553    C++
 godot.windows.tools.64.exe!Main::iteration() Line 2178 C++
 godot.windows.tools.64.exe!OS_Windows::run() Line 627  C++
 godot.windows.tools.64.exe!widechar_main(int argc, wchar_t * * argv) Line 161  C++
 godot.windows.tools.64.exe!_main() Line 183    C++
 godot.windows.tools.64.exe!main(int _argc, char * * _argv) Line 195    C++

Steps to reproduce:
Start the reproduction project and wait for the collision to happen.

Repro steps from the original issue:

1. Get two rigid bodies to collide, and detect each other colliding with the _body_entered signal.
2. Upon collision, get one of them to change it's collision shape, then remove itself from the scene

Minimal reproduction project:
RigidBodyBugMinimal-4.0.zip

[pouleyKetchoupp]

This issue seems to happen only on master, so it could be due to #38253 (we've synced with Bullet master at the time, not on a stable release).

We need to check if it still happens with the current master from Bullet, and if it's the case investigate to make sure we're not causing it and open an issue upstream if needed.

[madmiraal]

Actually this is not caused by the collision, it's caused by changing the CollisionShape. I've created an even simpler minimal reproduction project:
40311.zip

Changing the CollisionShape during _ready() is fine. Changing it later, in this case triggered by moving the mouse over the CollsionShape causes the crash:

extends RigidBody3D

func _ready():
    $CollisionShape3D.shape.extents.y = 0.5

func _on_RigidBody3D_mouse_entered():
    $CollisionShape3D.shape.extents.y = 1.0

[madmiraal]

This is a regression from #39726. Works fine with 480cb25961, crashes with bd3a468fc2.
cc @AndreaCatania

[AndreaCatania]

Internally the flush was moved into the flush_queries function; Since the shape change happens after the flush (into the iteration function) the shape is nullptr when the physics step is called, so the crash.

PhysicsServer3D::get_singleton()->sync();
PhysicsServer3D::get_singleton()->flush_queries(); // <--- Shape is not changed at this point.

if (OS::get_singleton()->get_main_loop()->iteration(frame_slice * time_scale)) { // <--- Now the shape changes here.
    exit = true;
    break;
}

PhysicsServer3D::get_singleton()->step(frame_slice * time_scale); // <---- Process happens here, but the shape is nullptr.

I'll provide a fix shortly.

[AndreaCatania]

Fixed with: https://github.com/godotengine/godot/pull/40252

[madmiraal]

Reopening, because it still crashes on master 9adf6d3441d4927bd (i.e. after #40252) with MRP 40311.zip.

Console output:

Set to 1.0
Set to 0.5
pure virtual method called
terminate called without an active exception

Stack trace:

#0 ??   __GI_raise (sig=sig@entry=6) (../sysdeps/unix/sysv/linux/raise.c:50)
#1 0x7ffff7516859   __GI_abort() (abort.c:79)
#2 0x7ffff78ed951   ??() (/lib/x86_64-linux-gnu/libstdc++.so.6:??)
#3 0x7ffff78f947c   ??() (/lib/x86_64-linux-gnu/libstdc++.so.6:??)
#4 0x7ffff78f94e7   std::terminate() () (/lib/x86_64-linux-gnu/libstdc++.so.6:??)
#5 0x7ffff78fa245   __cxa_pure_virtual() (/lib/x86_64-linux-gnu/libstdc++.so.6:??)
#6 0x4bb8e4d    btPolyhedralConvexShape::localGetSupportingVertexWithoutMargin(this=0x7fff84000f10, vec0=...) (thirdparty/bullet/BulletCollision/CollisionShapes/btPolyhedralConvexShape.cpp:408)
#7 0x4ba913b    btConvexInternalShape::localGetSupportingVertex(this=0x7fff84000f10, vec=...) (thirdparty/bullet/BulletCollision/CollisionShapes/btConvexInternalShape.cpp:54)
#8 0x4bb6666    btSubsimplexConvexCast::calcTimeOfImpact(this=0x7fffffffb098, fromA=..., toA=..., fromB=..., toB=..., result=...) (thirdparty/bullet/BulletCollision/NarrowPhaseCollision/btSubSimplexConvexCast.cpp:55)
#9 0x4b8ff17    btCollisionWorld::rayTestSingleInternal(rayFromTrans=..., rayToTrans=..., collisionObjectWrap=0x7fffffffb3a0, resultCallback=...) (thirdparty/bullet/BulletCollision/CollisionDispatch/btCollisionWorld.cpp:320)
#10 0x4b8fcff   btCollisionWorld::rayTestSingle(rayFromTrans=..., rayToTrans=..., collisionObject=0x8126e60, collisionShape=0x7fff84000f10, colObjWorldTransform=..., resultCallback=...) (thirdparty/bullet/BulletCollision/CollisionDispatch/btCollisionWorld.cpp:284)
#11 0x4cbbdba   btSoftRigidDynamicsWorld::rayTestSingle(rayFromTrans=..., rayToTrans=..., collisionObject=0x8126e60, collisionShape=0x7fff84000f10, colObjWorldTransform=..., resultCallback=...) (thirdparty/bullet/BulletSoftBody/btSoftRigidDynamicsWorld.cpp:312)
#12 0x4cbc83d   btSoftSingleRayCallback::process(this=0x7fffffffb7e0, proxy=0x9aec3f0) (thirdparty/bullet/BulletSoftBody/btSoftRigidDynamicsWorld.cpp:238)
#13 0x4c9f087   BroadphaseRayTester::Process(this=0x7fffffffb6f8, leaf=0x920b9d0) (thirdparty/bullet/BulletCollision/BroadphaseCollision/btDbvtBroadphase.cpp:236)
#14 0x4c9e6ef   btDbvt::rayTestInternal(this=0x974fe18, root=0x920b9d0, rayFrom=..., rayTo=..., rayDirectionInverse=..., signs=0x7fffffffb7f8, lambda_max=9999.99902, aabbMin=..., aabbMax=..., stack=..., policy=...) (thirdparty/bullet/BulletCollision/BroadphaseCollision/btDbvt.h:1267)
#15 0x4c9cc10   btDbvtBroadphase::rayTest(this=0x974fe10, rayFrom=..., rayTo=..., rayCallback=..., aabbMin=..., aabbMax=...) (thirdparty/bullet/BulletCollision/BroadphaseCollision/btDbvtBroadphase.cpp:263)
#16 0x4cbbb5c   btSoftRigidDynamicsWorld::rayTest(this=0x8a86390, rayFromWorld=..., rayToWorld=..., resultCallback=...) (thirdparty/bullet/BulletSoftBody/btSoftRigidDynamicsWorld.cpp:257)
#17 0x4c8e2b3   BulletPhysicsDirectSpaceState::intersect_ray(this=0x99282b0, p_from=..., p_to=..., r_result=..., p_exclude=..., p_collision_mask=4294967295, p_collide_with_bodies=true, p_collide_with_areas=true, p_pick_ray=true) (modules/bullet/space_bullet.cpp:101)
#18 0x5f7b502   Viewport::_notification(this=0x9972b30, p_what=26) (scene/main/viewport.cpp:775)
#19 0x4173fab   Viewport::_notificationv(this=0x9972b30, p_notification=26, p_reversed=false) (./scene/main/viewport.h:87)
#20 0x4173e5f   Window::_notificationv(this=0x9972b30, p_notification=26, p_reversed=false) (./scene/main/window.h:40)
#21 0x73b08db   Object::notification(this=0x9972b30, p_notification=26, p_reversed=false) (core/object.cpp:806)
#22 0x5f491d9   SceneTree::_notify_group_pause(this=0x8370b70, p_group=..., p_notification=26) (scene/main/scene_tree.cpp:818)
#23 0x5f48c6f   SceneTree::iteration(this=0x8370b70, p_time=0.0166666675) (scene/main/scene_tree.cpp:411)
#24 0x3d89b7c   Main::iteration() (main/main.cpp:2307)
#25 0x3d487e6   OS_LinuxBSD::run(this=0x7fffffffe110) (platform/linuxbsd/os_linuxbsd.cpp:238)
#26 0x3d45d7c   main(argc=1, argv=0x7fffffffe5f8) (platform/linuxbsd/godot_linuxbsd.cpp:58)

[AndreaCatania]

Can you please check: https://github.com/godotengine/godot/pull/41096 ?