Fsharp: Internal exception: Not enough storage is available to process this command.

[vsfeedback]

_This issue has been moved from a ticket on Developer Community._

---

I have separately reported this problem here: https://github.com/dotnet/fsharp/issues/9175#issue-616646049

Severity Code Description Project File Line Suppression State
Error FS3160 Problem reading assembly 'D:\Users\John.Zabroski\source\panda\trunk\Source\Company.Resources\bin\Debug\net48\WCompany.Resources.dll': The exception has been reported. This internal exception should now be caught at an error recovery point on the stack. Original message: Error opening binary file 'D:\Users\John.Zabroski\source\panda\trunk\Source\Company.Resources\bin\Debug\net48\Company.Resources.dll': Not enough storage is available to process this command.
) Company.FSharp.Legacy D:\Users\John.Zabroski\source\panda\trunk\Source\Company.FSharp.Legacy\Program.fs 1 Active

Severity Code Description Project File Line Suppression State
Error FS0229 Error opening binary file 'D:\Users\John.Zabroski\source\panda\trunk\Source\Company.Resources\bin\Debug\net48\Company.Resources.dll': Not enough storage is available to process this command. Company.FSharp.. Legacy D:\Users\John.Zabroski\source\panda\trunk\Source\Company.FSharp.LegacyTest.fs 1 Active

What's interesting to me is Visual Studio 2019 uses 470.6 MB with no solution loaded, and I can see in my Task Manager it is using 1530.1MB after the error occurs some time after the solution is loaded. Problem ALWAYS occurs after compiling code. The error never occurs when editing C# code, only when editing F# code. Philip Carter directed me to file the problem here instead of with F#.

---

Original Comments

Visual Studio Feedback System on 5/13/2020, 09:10 PM:

We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.

Visual Studio Feedback System on 6/18/2020, 07:44 PM:

I have detected that for the last 35 days, this issue didn't have much product team activity and a very small amount of new votes or comments. Based on this, its severity, and affected area, it’s my experience that this issue is very unlikely to be fixed.

---

Original Solutions

(no solutions)

[abelbraaksma]

I think this already has an answer here? https://github.com/dotnet/fsharp/issues/9175

[jzabroski]

@abelbraaksma No. It's a loop. @cartermp suggested I open that bug report with Visual Studio, and the Visual Studio team then opened this ticket here.

I commented this morning that I think the issue may in fact be an F# issue that occurs when (transitively?) referencing a Resources assembly.

[dsyme]

There is an underlying problem here, but it feels like there's not enough information here to make progress, or even to know the particular kind of memory issue that's at fault. The repro steps we have so far are at https://github.com/dotnet/fsharp/issues/9175#issue-616646049

Run large F# (FSharpo.Core 4.7.1) and C# mixed use sln via Build Solution, several times, updating code in F# slightly each time. A trivial code update would be to use Resharper Rename refactoring on a C# method, compile, get F# errors due to C# Rename refactoring not peering through to F#, and then manually update those references in F# as well, and recompile. On last recompile, nasal demons emerge.

[dsyme]

@cartermp What should we do with this?

[cartermp]

I don't know why this was duplicated over here by folks, since i did request to have things filed on devcommunity so that diagnostics can be shared. @dsyme there is a dump file and ETL trace here: https://dev.azure.com/devdiv/DevDiv/_search?type=workitem&lp=workitems-Project&text=DeveloperCommunityId%3A1029006&filters=Projects%7BDevDiv%7D&_a=search

Note this comment:

What’s interesting to me is Visual Studio 2019 uses 470.6 MB with no solution loaded, and I can see in my Task Manager it is using 1530.1MB after the error occurs some time after the solution is loaded.

So it's perhaps likely that older VS versions without the big changes we did to rename and find refs could have ballooned memory from 1.5GB to 2.7GB and ultimately caused something to go OOM. But that feels unlikely, unless the F# projects are are larger and contain more code than FSharp.Compiler.Private.dll.

[jzabroski]

@cartermp One of your developers told me that it was a constant string folding optimization implemented naively that causes this issue.

[jzabroski]

@cartermp The comment is here: https://github.com/dotnet/fsharp/issues/8044#issuecomment-649779548

[cartermp]

The only way to confirm this would be to look at the dump file in the report that was dupe'd to here. It's a reasonable guess though. If it's indeed due to processing resource files and those are read in as a giant string then that would likely do it.

[jzabroski]

Yeah, I admit, it's a conjecture, but I've made a 15+ year career out of being the dude who sees these type of bugs. It's a bizarre skill.

I'm not sure how reliable the dump file is. When I took it, I had a hard time trying to get a dump that "definitely" had the problem. What's strange is it really only happens when I edit a line of F# code. I never run into it when editing C# code in the same project. And developers on the team who never use F# (because they're afraid of it, etc) _never_ have this problem. We _also_ all have identical Visual Studios, configured via the vsconfig file on install.

If this is something you're really serious about solving, I can try to repro it again and submit a new dump, unless @TIHan has already reproduced it. Sadly, we've almost removed all F# from our system as a result of this bug. We just found the whole experience of restarting Visual Studio every 15 minutes to not be worth keeping F#.

[cartermp]

I'm going to drop any kind of categorization for this thing until someone looks at the dump file to see if there's any indication of what's going on in the stack trace.

[TIHan]

Looking at this carefully. It seems the issues are with the IDE.
The string constant folding optimization isn't the cause of this; the optimization only occurs at compile time and not in the IDE.

@jzabroski Could you try the latest VS? According to the report, you are using 16.5. In 16.6, we included a significant memory reduction change and wondering if it has improved things.

[abelbraaksma]

The message itself is very generic and has been reported for virtually every version of VS over the years, where the ultimate cause is usually not related to memory or disk, but to certain other (network or system) resources.

It may, for instance, be a resurgence of this issue caused by irpstacksize: https://stackoverflow.com/q/5592057/111575. Another cause can be not enough handles, or limitations on window resources: https://stackoverflow.com/a/508847/111575.

I know these possible causes are a long shot, but a typical OOM leads to different kinds of errors, usually, hence my guess to look elsewhere than just memory.

[TIHan]

@jzabroski Unfortunately, the current dump and ETL trace don't show where it happened nor show the exception itself. But, based on report "Error opening binary file", that's clearly indicating it is having trouble opening the binary file. We might need more information.

We have made changes to the metadata reader over the last 6 months in order to address some memory issues; my hunch your error may have something to do with those changes but I can't confirm it.

[jzabroski]

I am getting closer to a simpler repro of this issue, by luck.

In my case, I merely have to open a F# file (ImportedData.fs) and it instantly triggers the OOM issue. No batch compilation is required. 100% reproducible.

Some quick notes:

1. Open Visual Studio solution
2. Observe nothing in Errors list
3. Rebuild solution
4. Look at Errors List

Before I open the file, I have the following compiler warning in the Errors List:

Severity    Code    Description Project File    Line    Suppression State
Warning FS3189  Redundant arguments are being ignored in function 'nullArg'. Expected 1 but got 2 arguments.    Weiss.Panda.TradeAllocation.Legacy  D:\Users\John.Zabroski\source\panda\trunk3\Source\Company.Legacy\ImportedData.fs    460 Active

After I open the file, I get the following Visual Studio error in the Errors List:

Severity    Code    Description Project File    Line    Suppression State
Error   FS0229  Error opening binary file 'D:\Users\John.Zabroski\source\panda\trunk3\Source\Company.Resources\bin\Debug\net48\Company.Resources.dll': Not enough storage is available to process this command. Company.Legacy  D:\Users\John.Zabroski\source\panda\trunk3\Source\Company.Legacy\ImportedData.fs    1   Active
Error   FS3160  Problem reading assembly 'D:\Users\John.Zabroski\source\panda\trunk3\Source\Company.Resources\bin\Debug\net48\Company.Resources.dll': The exception has been reported. This internal exception should now be caught at an error recovery point on the stack. Original message: Error opening binary file 'D:\Users\John.Zabroski\source\panda\trunk3\Source\Company.Resources\bin\Debug\net48\Company.Resources.dll': Not enough storage is available to process this command.
)   Company.Legacy  D:\Users\John.Zabroski\source\panda\trunk3\Source\Company.Legacy\ImportedData.fs    1   Active

[abelbraaksma]

@jzabroski, is that file available somewhere si we dag try to repro it?

[jzabroski]

I hope to edge closer to getting a full repro, was just relaying my thoughts - obviously with a phase of the moon bug, you could change one thing and all of a sudden it disappears. But the one constant is F# (not C#).

[TIHan]

I think I may know what is happening. I think this is throwing because it's unable to create a view accessor for a memory mapped file due to not enough memory in the address space.

We create a memory mapped file and a view accessor per resource in an assembly. We do this to avoid GC heap but having the view accessor still uses memory from the address space. I've been working on a PR where we will only create the view accessor when we need it: https://github.com/dotnet/fsharp/pull/10173 - I'm wondering if this helps.

Nonetheless, if it does not help; I will revert the change that used a MMF for assembly resources.

[TIHan]

@jzabroski Here is a VSIX based on the changes from https://github.com/dotnet/fsharp/pull/10173 you can try out. Let's see if it resolves your issue.
VisualFSharpFull.zip