Sdk: Dart VM randomly crashes on application with FFI

[rootext]

Steps to reproduce:

1. Run an application which uses Dart FFI.
2. Application randomly crashes after few seconds.
   Some observations:
   If application crashes it crashes in few seconds after start.
   If application doesn't crash in few seconds it works perfectly even if the same code runs (in infinite loop for example).

Call stack:

    ntdll.dll!RtlpOptimizeConditionVariableWaitList()   Unknown
    ntdll.dll!RtlSleepConditionVariableSRW()    Unknown
    KERNELBASE.dll!00007ff944bceead()   Unknown
    dart.exe!00007ff741754433() Unknown
    dart.exe!00007ff7418fa02b() Unknown
    000001a36abc3cb9()  Unknown
    000001a36b6c3061()  Unknown
    0000008e6a7fec68()  Unknown

I can provide crash dump if it is needed.

Dart VM version: 2.3.0 (Fri May 3 10:32:31 2019 +0200) on "windows_x64"

[mraleph]

@rootext yes, please provide the crash dump.

[rootext]

dart.exe.18048.dmp.zip

[dcharkes]

I wonder why this doesn't show up on our build bots. We do run longer tests (for example ffi/function_stress_test), and tests are enabled on Windows (not excluded in status files, and not approved failures), and these tests do not flake on the Windows bots. I'll try to reproduce it on my Windows machine.

Version 2.3.0 was branched from the master branch on April 17, it might be possible that we have already fixed this issue. But I do not see any obvious CLs since April 17 fixing a flaky crash.

From the dump it looks like we are returning in from a call to SDL2 and try to exit a safepoint.

> dqs @rsp L100
0000008e`6a7fe930  00007ff9`485afb0c ntdll!RtlSleepConditionVariableSRW+0x18c
0000008e`6a7fe938  00000000`00000000
0000008e`6a7fe940  00000000`00000000
0000008e`6a7fe948  00000000`00000000
0000008e`6a7fe950  00000000`00000000
0000008e`6a7fe958  00000000`00000000
0000008e`6a7fe960  0000008e`6a7fe958
0000008e`6a7fe968  00000000`00000000
0000008e`6a7fe970  00000000`000043bc
0000008e`6a7fe978  00000003`00000000
0000008e`6a7fe980  000001a3`6b383ea0
0000008e`6a7fe988  000001a3`6b6eb2b1
0000008e`6a7fe990  000001a3`6ae50d60
0000008e`6a7fe998  00000000`fffffffb
0000008e`6a7fe9a0  00007ff9`44bceead KERNELBASE!SleepConditionVariableSRW+0x2d
0000008e`6a7fe9a8  00000000`00000000
0000008e`6a7fe9b0  000001a3`6b383ea0
0000008e`6a7fe9b8  0000008e`6a7fec38
0000008e`6a7fe9c0  00000000`00000000
0000008e`6a7fe9c8  00000000`00000000
0000008e`6a7fe9d0  00000000`00000000
0000008e`6a7fe9d8  00000000`00000000
0000008e`6a7fe9e0  00007ff7`41754433 dart!dart::Monitor::Wait+0x23 [c:\src\dart-sdk\sdk\runtime\vm\os_thread_win.cc @ 336]
0000008e`6a7fe9e8  00000000`00000000
0000008e`6a7fe9f0  00000000`00000000
0000008e`6a7fe9f8  00000000`00000000
0000008e`6a7fea00  00000000`00000000
0000008e`6a7fea08  000001a3`6ae50d60
0000008e`6a7fea10  00007ff7`418fa02b dart!dart::SafepointHandler::ExitSafepointUsingLock+0x5b [c:\src\dart-sdk\sdk\runtime\vm\heap\safepoint.cc @ 177]
0000008e`6a7fea18  00000000`00000000
0000008e`6a7fea20  00000000`00000000
0000008e`6a7fea28  00000000`00000000
0000008e`6a7fea30  00000000`00000000
0000008e`6a7fea38  000001a3`6baf6bf1
0000008e`6a7fea40  000001a3`6abc3cb9
0000008e`6a7fea48  000001a3`6b6c3061
0000008e`6a7fea50  0000008e`6a7fec68
0000008e`6a7fea58  00000000`00000000
0000008e`6a7fea60  00000000`00000000
0000008e`6a7fea68  00000000`00000000
0000008e`6a7fea70  000001a3`6c41aa84
0000008e`6a7fea78  00000000`00000000
0000008e`6a7fea80  0000008e`6a7fec68
0000008e`6a7fea88  000001a3`6b6c3061
0000008e`6a7fea90  000001a3`6baf6bf1
0000008e`6a7fea98  000001a3`6d1e28f8
0000008e`6a7feaa0  000001a3`6c6d3500
0000008e`6a7feaa8  000001a3`6c6d2cd0
0000008e`6a7feab0  000001a3`6abc3c20
0000008e`6a7feab8  000001a3`6ab40041
0000008e`6a7feac0  000001a3`6b6eb2b1
0000008e`6a7feac8  000001a3`6ae50d60
0000008e`6a7fead0  000001a3`6b6eb2b1
0000008e`6a7fead8  000001a3`6c71e848
0000008e`6a7feae0  000001a3`6c3e5000
0000008e`6a7feae8  00000000`00000000
0000008e`6a7feaf0  000001a3`78000700
0000008e`6a7feaf8  00000000`3f800000
0000008e`6a7feb00  00000000`00000000
0000008e`6a7feb08  00000000`00000000
0000008e`6a7feb10  00000000`00000000
0000008e`6a7feb18  00000000`00000000
0000008e`6a7feb20  00000000`00000000
0000008e`6a7feb28  00000000`00000000
0000008e`6a7feb30  00000000`00000000
0000008e`6a7feb38  00000000`00000000
0000008e`6a7feb40  00000000`00000000
0000008e`6a7feb48  00000000`00000000
0000008e`6a7feb50  00000000`00000000
0000008e`6a7feb58  00000000`00000000
0000008e`6a7feb60  00000000`00000000
0000008e`6a7feb68  00000000`00000000
0000008e`6a7feb70  00000000`00000000
0000008e`6a7feb78  00000000`00000000
0000008e`6a7feb80  00000000`00000000
0000008e`6a7feb88  00000000`00000000
0000008e`6a7feb90  00000000`00000000
0000008e`6a7feb98  00000000`00000000
0000008e`6a7feba0  00000000`00000000
0000008e`6a7feba8  00000000`00000000
0000008e`6a7febb0  00000000`00000000
0000008e`6a7febb8  00000000`00000000
0000008e`6a7febc0  00000000`00000000
0000008e`6a7febc8  00000000`00000000
0000008e`6a7febd0  00000000`00000000
0000008e`6a7febd8  000001a3`7844470c
0000008e`6a7febe0  000001a3`6bae5e81
0000008e`6a7febe8  0000008e`6a7fec68
0000008e`6a7febf0  000001a3`6b6c3061
0000008e`6a7febf8  00000000`00000000
0000008e`6a7fec00  000001a3`6ab40041
0000008e`6a7fec08  000001a3`6ab40041
0000008e`6a7fec10  000001a3`6ab40041
0000008e`6a7fec18  00000000`6c77b980 SDL2+0x3b980
0000008e`6a7fec20  000001a3`6bae5e81
0000008e`6a7fec28  000001a3`6b6eb2b1
0000008e`6a7fec30  000001a3`6ab40041
0000008e`6a7fec38  0000008e`6a7fec68
0000008e`6a7fec40  000001a3`7844467f
0000008e`6a7fec48  000001a3`68e52460
0000008e`6a7fec50  00000000`6c77b980 SDL2+0x3b980
0000008e`6a7fec58  000001a3`6c03dca1
0000008e`6a7fec60  000001a3`6b6eb2f1
0000008e`6a7fec68  0000008e`6a7fed00

SafepointHandler::ExitSafepointUsingLock related files:

• https://github.com/dart-lang/sdk/blob/master/runtime/vm/compiler/assembler/assembler_x64.cc#L208-L211
• https://github.com/dart-lang/sdk/blob/master/runtime/vm/compiler/stub_code_compiler_x64.cc#L213-L222
• https://github.com/dart-lang/sdk/blob/master/runtime/vm/runtime_entry.cc#L2760-L2766

[rootext]

reproduced on 2.3.1-dev
shortest example https://gist.github.com/rootext/39c7b84a8fc04343ee80a6aa7b9d17a8

[dcharkes]

Thanks for the minimal example!

I am able to reproduce this on 2.3.0 Windows x64 release and debug. It is also reproducible on master branch: Dart VM version: 2.3.1-edge.e7f79849953d8f85243cda71d8337f2054241c0a (Tue May 21 08:53:08 2019 +0000) on "windows_x64".

I have not been able to reproduce this on Linux. (The main while loop reaches 8 million iterations after 10 minutes every time.) So, this seems to be an OS specific flaky crash.

[dcharkes]

@rootext fix is underway: https://dart-review.googlesource.com/c/sdk/+/103136

[rootext]

Tested on commit 21b74fdf79e026164e7761710465c850d1054218.
Thank you for fix.