FSharp.Compiler.Service ParseAndCheckProject crash with stackoverflow exception during typecheck on Mac OS
The minimized the repro point to """ usage
NOTE FCS v28.0.0 was integrated from visualfsharp master to Integrate visualfsharp master up to 8dfc02feb , but may happen in previous versions too, need to check, there where issues from users also in v27
An example is https://github.com/sjanahan/ionide-crash who crash on latest FSAC (FCS v28.0.0) in vscode+ionide on mac.
The issue is the usage of a big triple quoted """ string like this one
A repro using latest FSharp.Compiler.Service package v28.0.0 as library is in https://github.com/enricosada/fcs-crash/ , who just invoke ParseAndCheckProject on that project
The crash is shown on travis build matrix ( https://travis-ci.org/enricosada/fcs-crash/builds/524417824 ) where succeed on unix and fails on osx
NOTES
dotnet build is okref https://github.com/fsharp/FsAutoComplete/issues/356
ref https://github.com/ionide/ionide-vscode-fsharp/issues/1039
No crash
Stackoverflow, with a stacktrace like backend-err.log
That stacktrace is from a .NET Version on Rider, who happen in a private codebase. The .NET Core version doesnt show the stacktrace for the SO
The minimal repro doesnt trigger in Rider but does in FSAC (FCS v28.0.0)
Stack overflow in unmanaged: IP: 0x10744b82b, fault addr: 0x7000076f5fb8
Stack overflow: IP: 0x10744b82b, fault addr: 0x7000076ecff8
Stacktrace:
at <unknown> <0xffffffff>
at string.CtorCharArrayStartLength (char[],int,int) [0x00061] in <53dca12e92aa4e3387f3b954838dd249>:0
<...>
at string.CreateString (char[],int,int) [0x00000] in <53dca12e92aa4e3387f3b954838dd249>:0
at (wrapper managed-to-managed) string..ctor (char[],int,int) [0x00000] in <53dca12e92aa4e3387f3b954838dd249>:0
at Internal.Utilities.Text.Lexing.LexBuffer`1<char>.LexemeString (Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer._fslex_tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,int,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00193] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
...... tons more of at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexer.tripleQuoteString (Microsoft.FSharp.Compiler.AbstractIL.Internal.ByteBuffer,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Compiler.Range/range, Microsoft.FSharp.Core.FSharpFunc`2<bool, Microsoft.FSharp.Core.FSharpFunc`2<byte[], Microsoft.FSharp.Compiler.Parser/token>>>,Microsoft.FSharp.Compiler.Range/range,Microsoft.FSharp.Compiler.Lexhelp/lexargs,bool,Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilterImpl.runWrappedLexerInConsistentLexbufState () [0x00031] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilterImpl.popNextTokenTup () [0x0001a] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilterImpl.peekNextTokenTup () [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilterImpl.pushCtxtSeqBlock (bool,Microsoft.FSharp.Compiler.LexFilter/AddBlockEnd) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilterImpl.hwTokenFetch (bool) [0x0466a] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilterImpl.Lexer<a_REF> (a_REF) [0x00021] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilter.popNextToken () [0x0001a] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter.loop@2264-37 (Microsoft.FSharp.Compiler.LexFilter/LexFilter,Microsoft.FSharp.Core.Unit) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.LexFilter/LexFilter.Lexer<a_REF> (a_REF) [0x00002] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.SourceCodeServices.Parser/createLexerFunction@1534<a_REF>.Invoke (a_REF) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Internal.Utilities.Text.Parsing.Implementation.interpret<tok_REF> (Internal.Utilities.Text.Parsing.Tables`1<tok_REF>,Microsoft.FSharp.Core.FSharpFunc`2<Internal.Utilities.Text.Lexing.LexBuffer`1<char>, tok_REF>,Internal.Utilities.Text.Lexing.LexBuffer`1<char>,int) [0x0013c] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Internal.Utilities.Text.Parsing.Tables`1<tok_REF>.Interpret (Microsoft.FSharp.Core.FSharpFunc`2<Internal.Utilities.Text.Lexing.LexBuffer`1<char>, tok_REF>,Internal.Utilities.Text.Lexing.LexBuffer`1<char>,int) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.CompileOps.ParseInput (Microsoft.FSharp.Core.FSharpFunc`2<Internal.Utilities.Text.Lexing.LexBuffer`1<char>, Microsoft.FSharp.Compiler.Parser/token>,Microsoft.FSharp.Compiler.ErrorLogger/ErrorLogger,Internal.Utilities.Text.Lexing.LexBuffer`1<char>,Microsoft.FSharp.Core.FSharpOption`1<string>,string,System.Tuple`2<bool, bool>) [0x00070] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.SourceCodeServices.Parser/[email protected] (Internal.Utilities.Text.Lexing.LexBuffer`1<char>) [0x0006b] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexhelp/usingLexbufForParsing@92<a_REF>.Invoke (Microsoft.FSharp.Core.Unit) [0x00000] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexhelp.reusingLexbufForParsing<a_REF> (Internal.Utilities.Text.Lexing.LexBuffer`1<char>,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, a_REF>) [0x00026] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.Lexhelp.usingLexbufForParsing<a_REF> (Internal.Utilities.Text.Lexing.LexBuffer`1<char>,string,Microsoft.FSharp.Core.FSharpFunc`2<Internal.Utilities.Text.Lexing.LexBuffer`1<char>, a_REF>) [0x0001a] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Compiler.SourceCodeServices.Parser.parseFile (string,string,Microsoft.FSharp.Compiler.SourceCodeServices.FSharpParsingOptions,string) [0x00066] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at <StartupCode$FSharp-Compiler-Service>.$Service/[email protected] (Microsoft.FSharp.Core.Unit) [0x0006a] in <5b5b17b04d124a0ca7450383b0175b5b>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl/callA@522<b_REF, a_REF>.Invoke (Microsoft.FSharp.Control.AsyncParams`1<b_REF>) [0x00051] in <5a7d678a904cf4daa74503838a677d5a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl/queueAsync@434<a_REF>.Invoke (Microsoft.FSharp.Core.Unit) [0x00033] in <5a7d678a904cf4daa74503838a677d5a>:0
at <StartupCode$FSharp-Core>.$Control.loop@124-50 (Microsoft.FSharp.Control.Trampoline,Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Control.FakeUnitValue>) [0x00000] in <5a7d678a904cf4daa74503838a677d5a>:0
at Microsoft.FSharp.Control.Trampoline.ExecuteAction (Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Control.FakeUnitValue>) [0x00017] in <5a7d678a904cf4daa74503838a677d5a>:0
at Microsoft.FSharp.Control.TrampolineHolder.Protect (Microsoft.FSharp.Core.FSharpFunc`2<Microsoft.FSharp.Core.Unit, Microsoft.FSharp.Control.FakeUnitValue>) [0x00031] in <5a7d678a904cf4daa74503838a677d5a>:0
at <StartupCode$FSharp-Core>.$Control/[email protected] (object) [0x00017] in <5a7d678a904cf4daa74503838a677d5a>:0
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (object) [0x00007] in <53dca12e92aa4e3387f3b954838dd249>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00071] in <53dca12e92aa4e3387f3b954838dd249>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) [0x00000] in <53dca12e92aa4e3387f3b954838dd249>:0
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in <53dca12e92aa4e3387f3b954838dd249>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <53dca12e92aa4e3387f3b954838dd249>:0
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <53dca12e92aa4e3387f3b954838dd249>:0
at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) [0x0001e] in <53dca12e92aa4e3387f3b954838dd249>:0
The method seems ok to tail call
.method assembly static class FSharp.Compiler.Parser/token
tripleQuoteString(
class FSharp.Compiler.AbstractIL.Internal.ByteBuffer sargs_0,
class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<valuetype FSharp.Compiler.Range/range, class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<bool, class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<unsigned int8[], class FSharp.Compiler.Parser/token>>> sargs_1,
valuetype FSharp.Compiler.Range/range sargs_2,
class FSharp.Compiler.Lexhelp/lexargs sargs_3,
bool skip,
class Internal.Utilities.Text.Lexing.LexBuffer`1<char> lexbuf
) cil managed
{
.custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationArgumentCountsAttribute::.ctor(int32[])
= (
01 00 03 00 00 00 04 00 00 00 01 00 00 00 01 00 // ................
00 00 00 00 // ....
)
// int32[3]
/*( int32(4) // 0x00000004
int32(1) // 0x00000001
int32(1) // 0x00000001
)*/
.maxstack 9
// [1846 7 - 1846 100]
IL_0000: ldarg.0 // sargs_0
IL_0001: ldarg.1 // sargs_1
IL_0002: ldarg.2 // sargs_2
IL_0003: ldarg.3 // sargs_3
IL_0004: ldarg.s skip
IL_0006: ldc.i4 178 // 0x000000b2
IL_000b: ldarg.s lexbuf
IL_000d: tail.
IL_000f: call class FSharp.Compiler.Parser/token FSharp.Compiler.Lexer::_fslex_tripleQuoteString(class FSharp.Compiler.AbstractIL.Internal.ByteBuffer, class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<valuetype FSharp.Compiler.Range/range, class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<bool, class [FSharp.Core]Microsoft.FSharp.Core.FSharpFunc`2<unsigned int8[], class FSharp.Compiler.Parser/token>>>, valuetype FSharp.Compiler.Range/range, class FSharp.Compiler.Lexhelp/lexargs, bool, int32, class Internal.Utilities.Text.Lexing.LexBuffer`1<char>)
IL_0014: ret
} // end of method Lexer::tripleQuoteString
None
Happen in mac osx only, in this repro does happen in .NET Core Runtime, but seen the same in .NET Framework Runtime too with private codebase
in the repro is using .NET Core v2.1.401 but newer version fails too.
/cc @sjanahan @Krzysztof-Cieslak @baronfel @alfonsogarciacaro
@auduchinok happen in latest Rider too, but not yet minimized private codebase, do you have seen something similar?
@dustinmoris maybe is similar to your issue, can you repro with that project?
SO exceptions are a party with FCS on mac since the first netcore versions. There's obviously a problem with tail calls in netcore runtime on macos. That's what should be fixed instead of pinpointing all the issues as they appear :/
@alfonsogarciacaro i got the same error on Rider who afaik use mono, so it not just on .NET Core, maybe there is more visibile (stack deep smaller?)
Maybe that code shouldnt use recursion+tail at all? to fix the root issue F# side.
Meanwhile i'll open an issue in https://github.com/dotnet/coreclr to ask for guidance about that, if it's expected or can be mitigated somehow or is a bug who need to be fixed.
@enricosada in Rider we currently use Mono on Mac and Linux and .NET Framework on Windows.
@auduchinok happen in latest Rider too, but not yet minimized private codebase, do you have seen something similar?
We've seen it on a particular bigger file with a similar stacktrace, issues are: https://github.com/mono/mono/issues/11933 and https://github.com/JetBrains/fsharp-support/issues/23, but this particular exception seems to be fixed.
added https://github.com/dotnet/coreclr/issues/24249 in coreclr to ask about tail calls.
Closed as duplicated of https://github.com/dotnet/coreclr/issues/2556 but https://github.com/dotnet/coreclr/issues/24249#issuecomment-486722733 contains useful info
Another thing worth considering is that the processing of very large literals has been a source of stack overflows and has recently been fixed: #6258
The actual fix isn't released in anything other than a VS preview right now, but eventually FCS can update and it's worth seeing if the problem still persists. Just want to rule out anything we know here before we dive into CoreCLR issues.
@cartermp the fslex and large-expression updates are in FCS 28, which is merged up to commit 8dfc02feb in this repository (ie 28 days ago).
@enricosada COMPlus_DefaultStackSize=180000 could be a reasonable quick-fix for ionide, if we added a set of settings for environment-variables to use when spawning FSAC. I tried it with my stack-overflowing work project and things work great.
my steps (macos):
export COMPlus_DefaultStackSize=180000 in a shell, so that child processes will inherit that valuecode . to spawn vscodeFSharp.fsacRuntime from net to netcore@baronfel Is this one as well? #6294
Yes. Everything from March 28th and back is in FCS 28.
@baronfel good workaround, but it depends on stacktrace deep anyway.
As a note, this https://github.com/sjanahan/rider-crash-reproducer/ is a repro with Rider on mono, we cannot apply same workaround of .net core, and will crash anyway on fsac .net
@enricosada I've opened a ticket with JetBrains about the crash in Rider
Please let me know how if there's anything else I can do to help get this resolved. Thanks!
I will try this repo a little bit later as I'm on a train right now, but I just want to add that the StackOverflowException which I am getting on macOS with .NET Core in Ionide is/must be a regression, because it only started happening after upgrading to Ionide 3.34.0 and above. When I revert Ionide back to 3.33.0 then everything works. So it might be worth checking what has changed in the dependencies which Ionide relies on between those two versions.
@dsyme @KevinRansom usually how these kind of issues are fixed?
rewriting the code without recursion?
Dunno if feasible, but adding a new warning if recursion doesnt satify @jkotas rules on the comment https://github.com/dotnet/coreclr/issues/2556#issuecomment-169755535 may make FCS/fsc safer
The simple rule that should hold across platforms over time is that the fast tailcall will be used if both of the following are true:
- Return value and call target arguments are all either primitive types, reference types, or valuetypes with a single primitive type or reference type fields
- The aligned size of call target arguments is less or equal to aligned size of caller arguments
This no longer reproduces with the 3.1.00 SDK, VS, or Ionide. This has likely been fixed by the numerous fixes in the compiler that we've made leading up to .NET Core 3.1.
Most helpful comment
This no longer reproduces with the 3.1.00 SDK, VS, or Ionide. This has likely been fixed by the numerous fixes in the compiler that we've made leading up to .NET Core 3.1.