mapbox-gl-js version: 1.4.1 (I guess almost every version has this issue)
browser: Chrome 78
platform: Windows 10

move towards the right direction
move towards the right direction at the first moment
and then move towards reverse direction till stop moving
I've noticed the same bug while working on the LOD support #8975. I don't have a production ready fix for this but the cause is well known.
We're applying inertia to the mouse cursor and with fast enough mouse movement this can fling the cursor way above the screen. When this behavior is combined with bigger pitch angles the flinged position might end up being above the horizon (ie. ray cast towards this point will never hit the ground when travelling forward). This will result in intersections being reported somewhere "behind" they ray (t < 0) causing the inverse movement direction.
Most helpful comment
I've noticed the same bug while working on the LOD support #8975. I don't have a production ready fix for this but the cause is well known.
We're applying inertia to the mouse cursor and with fast enough mouse movement this can fling the cursor way above the screen. When this behavior is combined with bigger pitch angles the flinged position might end up being above the horizon (ie. ray cast towards this point will never hit the ground when travelling forward). This will result in intersections being reported somewhere "behind" they ray (t < 0) causing the inverse movement direction.