Runtime: gcAllowVeryLargeObjects: Add switch or allow by default in .NET Core

[nietras]

Besides setting an environment variable (COMPlus_gcAllowVeryLargeObjects=1) there is no way to set gcAllowVeryLargeObjects in .NET Core like in app.config for .NET Framework:

<configuration>
  <runtime>
    <gcAllowVeryLargeObjects enabled = "true" />
  </runtime>
</configuration>

Either add a switch for this, or enable this by default. Or other options that could be discussed here.

This originates in https://github.com/dotnet/corefx/pull/14396 where this switch is needed to be able to test clearing managed arrays with byte size larger than uint.MaxValue.

cc: @jkotas

[jkotas]

@KevinRansom mentioned to me that they need this for F# (F# is often used to process large data sets...).

[mikedn]

Are there any downsides? It seems to me that it should be enabled by default in .NET Core.

[HughPH]

I accept that this was merged over 2 years ago, but I'm getting OOM when creating a huge int array even on .Net Core 3.0 Preview 3. I'm compiling for x64 and running on Linux (so no COM+ here)

Any hints?

[jkotas]

What is the size of the array you are trying to create?

[HughPH]

About 14.5GB, 3,623,878,656 elements

[jkotas]

I assume this must be multi-dimensional array. Correct? The single-dimensional arrays cannot have more then ~2 billions elements.

I have tried running int[,] a = new int[1811939328,2]; and it works fine for me. Could you please share the exact code that you use to create the array?

[HughPH]

No, it's one-dimensional. I created it as two-dimensional and while it's created successfully, the process terminates when I try to write into it. Let me try again, and I'll also run it in the terminal and see if I get an exit code.

[HughPH]

Well now it's just making a liar of me. It's working as a 2-dimensional array this time.

Many thanks for your help, Jan!