Fsharp: fsc.exe exited with code -1073741571 (StackOverflowException with big literal list) VS2019 Preview 3

[marklam]

Creating a large list in a source literal breaks the compiler

Repro steps

Compile the following code:

```F#
module TestData

let expectedValues =
[
1
1
1
1
1

*continue this for another 400 lines*
```F#
        1 
    ]

TestData.zip

Expected behavior

A large (and boring) list is created

Actual behavior

The compiler terminates

Known workarounds

Create the list programmatically

Related information

Provide any related information

Microsoft Visual Studio Community 2019 Preview
Version 16.0.0 Preview 3.0
VisualStudio.16.Preview/16.0.0-pre.3.0+28608.199
Microsoft .NET Framework
Version 4.7.03190
Installed Version: Community
...
Visual F# Tools 10.4 for F# 4.6   16.0.0.0.  Commit Hash: b5c01b8cc65af6381d2dd558ee63a4aa9e0e98d5.
Microsoft Visual F# Tools 10.4 for F# 4.6

[TIHan]

Can confirm. This is a pretty bad one.

[nilekirk]

This also happens in preview 3 for huge if .. elif .. else, more than 196 it seems.

Repro

let isThisReallyOne n =
    if n = 1 then true
    elif n = 1 then true
    elif n = 1 then true
    ... more than 196 of these, including if and else
    else false

This is a regression, since it works in VS2017.

[cartermp]

@dsyme any ideas?

[dsyme]

I don't know of anything specific that would have caused this

[cartermp]

cc @KevinRansom @brettfo looks like we have some investigative work to do here

[dsyme]

The problem is similar to #6222 - fsc.exe is now a 64-bit process, and hence consuming more stack.

Adding

    <PlatformTarget Condition=" '$(TargetDotnetProfile)'!='coreclr'">x86</PlatformTarget>
    <PlatformTarget Condition=" '$(TargetDotnetProfile)'=='coreclr'">AnyCPU</PlatformTarget>

fixes the problem and restores compat. Would be interesting to know the coreclr compiler's behaviour here.

The general problem is that the input sizes accepted in various dimensions are determined partly by available process stack - they are not precisely specified nor do we test to precise numbers and fail above those numbers). Instead historically we've been able to remove these limits when we've encountered them by moving more stack to the heap for certain operations (e.g. collecting free variables down long chains of let) through standard continuation coding techniques in the compiler.

I suppose we should

1. move back to a 32-bit fsc.exe for compat
2. make the compiler handle arbitrary-sized input (subject to heap not stack) for the above.
3. work on at least having exact current size limits under test so we know when we blow them.

[cartermp]

Gotcha, so in terms of the immediate problem #6223 resolves it, but we likely need to do work to make it solved _better_?

[dsyme]

@cartermp Applying the same fix as #6223 (but to fsc.exe) resolves it. #6223 only dealt with fsi.exe

[abelbraaksma]

@dsyme, Alternatively, perhaps we can just increase the stack size of the fsc process? I know that's a bit of a sledge hammer approach, but it allows us to continue using the 64 bit version, which has other advantages like allowing for more heap space.

[KevinRansom]

@dsyme,

we should handle it without blowing the stack, and fix any others as we find them. I thought it was suspicious that the limit was 500, that is such an unusual computer number.

Coreclr stack overflows, 64 bit is the usual coreclr distribution.

[dsyme]

For Desktop compat it seems best to move back to 32bit. There are likely to be several such cases and I'm not sure it's worth our time to track thrm down one by one, with no workaround for the user (on netcore people have the workaround of using the desktop compiler). We didn't make this change deliberately.

Also some desktop type providers may also be 32bit dependent, so moving to 64bit fsc.exe is really a breaking change.

We should certainly also fix each case as we find them to support arbitrary sized inputs.

[dsyme]

In this particular case, the stack overflow is in PostTypeCheckSemanticChecks

    FSharp.Compiler.Private.dll!FSharp.Compiler.PostTypeCheckSemanticChecks.CheckExprNoByrefs(FSharp.Compiler.PostTypeCheckSemanticChecks.cenv cenv, FSharp.Compiler.PostTypeCheckSemanticChecks.env env, FSharp.Compiler.Tast.Expr expr) Line 684  F#
    FSharp.Compiler.Private.dll!F[e-mail](FSharp.Compiler.Tast.Expr expr) Line 1490 F#
    FSharp.Core.dll!Microsoft.FSharp.Primitives.Basics.List.iter<FSharp.Compiler.Tast.Expr>(Microsoft.FSharp.Core.FSharpFunc<FSharp.Compiler.Tast.Expr, Microsoft.FSharp.Core.Unit> f, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Tast.Expr> x) Line 91    F#
    FSharp.Core.dll!Microsoft.FSharp.Collections.ListModule.Iterate<FSharp.Compiler.Tast.Expr>(Microsoft.FSharp.Core.FSharpFunc<FSharp.Compiler.Tast.Expr, Microsoft.FSharp.Core.Unit> action, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Tast.Expr> list) Line 112    F#
    FSharp.Compiler.Private.dll!FSharp.Compiler.PostTypeCheckSemanticChecks.CheckExprsNoByRefLike(FSharp.Compiler.PostTypeCheckSemanticChecks.cenv cenv, FSharp.Compiler.PostTypeCheckSemanticChecks.env env, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Tast.Expr> exprs) Line 1490   F#
    FSharp.Compiler.Private.dll!FSharp.Compiler.PostTypeCheckSemanticChecks.CheckExprOp(FSharp.Compiler.PostTypeCheckSemanticChecks.cenv cenv, FSharp.Compiler.PostTypeCheckSemanticChecks.env env, FSharp.Compiler.Tast.TOp op, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Tast.TType> tyargs, Microsoft.FSharp.Collections.FSharpList<FSharp.Compiler.Tast.Expr> args, FSharp.Compiler.Range.range m, FSharp.Compiler.PostTypeCheckSemanticChecks.PermitByRefExpr context, FSharp.Compiler.Tast.Expr expr) Line 1373 F#
    FSharp.Compiler.Private.dll!FSharp.Compiler.PostTypeCheckSemanticChecks.CheckExprNoByrefs(FSharp.Compiler.PostTypeCheckSemanticChecks.cenv cenv, FSharp.Compiler.PostTypeCheckSemanticChecks.env env, FSharp.Compiler.Tast.Expr expr) Line 684  F#
    FSharp.Compiler.Private.dll!F[e-mail](FSharp.Compiler.Tast.Expr expr) Line 1490 F#

[cartermp]

Compat fix for this is in for desktop fsc.exe, so I'll close this out.

6294 is a more general fix for 32-bit and 64-bit, the latter of which will remain the assumed default for most people (since 64-bit .NET Core is what most people use).

[pauldorehill]

Having upgraded to VS2019 16.1.0 to get this fix, I'm getting an error of Exception of type 'System.OutOfMemoryException' was thrown. error FS0193 when building in debug mode (that didn't occur with 16.0.4), but I can now build in release mode.

[abelbraaksma]

@pauldorehill you should probably log that as a separate bug, it may not be related to this one

[cartermp]

@pauldorehill That sounds separate from this. Can you file an issue and supply a repro? Thanks!

[pauldorehill]

@cartermp I'm happy to file a repo, but I'll have to see if I can come up with some similar code as its for an internal app where I can't really share the code (this may take some time). I can compile the project using netcore (64bit) in both release & debug - its a set of generated files some of which are pretty large.