Godot: Collision solver issue with some ConvexPolygonShape2D?

[akien-mga]

Operating system or device - Godot version:
Mageia 6 x64, Godot 2.0.4.1 and master HEAD.

Issue description (what happened, and what was expected):
Characters seem to get stuck in ConvexPolygonShape2Ds with a shape like that:

or that:

Note that the same outline drawn as a _Concave_PolygonShape2D works just fine.

Steps to reproduce:

• Open attached project in the editor
• Toggle visible collision shapes
• Go to the bottom with the arrow keys, then run left or right in one of the tiles with the problematic collision shape
• See that you get stuck

Link to minimal example project (optional but very welcome):
convexpolygon_bug.zip

[Zylann]

The shapes are concave. Isn't ConvexPolygonShape2D not supposed to handle these?

[TheoXD]

You've discovered a black hole xD It is sucking in the object literally.

[akien-mga]

The shapes are concave. Isn't ConvexPolygonShape2D not supposed to handle these?

I also wondered about this, but I showed the screenshot to @reduz and he agreed it looks weird, so I preferred opening an issue to get to the bottom of this. I'm not sure how the convex/concave notions should be understood for the collision shapes. Here clearly this specific curve is concave, but does it make the whole shape a "concave shape"? What about a shape with a concave curve followed by a convex curve?

[TheoXD]

I just changed the indices order and it works as intended (for convex shape), only collision for that shape is not visible on debug (probably because of the order change and faces being one sided)

[bojidar-bg]

Well, the definition of "convex region" is to be unable to find any segment that has its both ends inside the region, but is not a subset of the region (i.e. doesn't cross to the other side of the boundary).

In this sense, @akien-mga's idea of having a concave curve and then a convex curve fails, since you are still able to draw such a segment.

[akien-mga]

So I guess this shape would have undefined behaviour:

[TheoXD]

@akien-mga, it's still the same invalid convex as in the test project, since angle between two edges is too extreme (in this case over 180 degrees). Flipping normal for only those edges should fix the problem.

[Zylann]

Just curious, is there a reason Concave and Convex are separated? If we had only one, there wouldn't be wrong cases when drawing it if the engine can detect it is concave or convex.

[eon-s]

This affects master too, OP example ported to 3:
convexpolygon_bug_v3.zip

[samdze]

The bug appears with move_and_slide too. It's really annoying, at this point concave shapes can't be used in tilemaps.

[Calinou]

@akien-mga Can you (or anyone else) still reproduce this bug in Godot 3.2.1 or 3.2.2beta4?

[eon-s]

I could not reproduce it on latest beta with move_and_collide nor move_and_slide, using the ported example.

[Calinou]

@eon-s Alright, closing then.