Nerdbank.gitversioning: Intermittent out-of-memory errors with the 3.4.153-alpha release in Visual Studio

[Numpsy]

Hi,

I tried installing the 3.4.142-alpha release into a small project a while back and it all seemed ok, but then more recently tried in with a much larger solution and started seeing #540.

I've now tried updating that larger solution to the 3.4.153-alpha package, and that seems to work ok for a few builds, but then starts failing with

1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018: The "Nerdbank.GitVersioning.Tasks.GetBuildVersion" task failed unexpectedly.
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018: System.IO.IOException: Not enough memory resources are available to process this command.
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at System.IO.MemoryMappedFiles.MemoryMappedView.CreateView(SafeMemoryMappedFileHandle memMappedFileHandle, MemoryMappedFileAccess access, Int64 offset, Int64 size)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateViewAccessor(Int64 offset, Int64 size, MemoryMappedFileAccess access)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitPack..ctor(GetObjectFromRepositoryDelegate getObjectFromRepositoryDelegate, Lazy`1 indexStream, Func`1 packStream, GitPackCache cache)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitPack..ctor(GetObjectFromRepositoryDelegate getObjectFromRepositoryDelegate, String indexPath, String packPath, GitPackCache cache)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitPack..ctor(GitRepository repository, String name)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitRepository.LoadPacks()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at System.Lazy`1.CreateValue()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at System.Lazy`1.LazyInitValue()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at System.Lazy`1.get_Value()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitRepository.TryGetObjectBySha(GitObjectId sha, String objectType, Stream& value)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitRepository.GetObjectBySha(GitObjectId sha, String objectType)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitRepository.GetCommit(GitObjectId sha, Boolean readAuthor)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.ManagedGit.GitRepository.GetHeadCommit(Boolean readAuthor)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.Managed.ManagedGitContext..ctor(String workingDirectory, String dotGitPath, String committish)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.GitContext.Create(String path, String committish, Boolean writable)
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Nerdbank.GitVersioning.Tasks.GetBuildVersion.ExecuteInner()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at MSBuildExtensionTask.ContextAwareTask.Execute()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
1>C:\Users\Richard Webb\.nuget\packages\nerdbank.gitversioning\3.4.153-alpha\build\Nerdbank.GitVersioning.Inner.targets(17,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()

And once it gets into that state it keeps failing until I close and reopen Visual Studio (just closing and reopening the solution doesn't seem to fix it).
I've also only seen the error when building inside Visual Studio (2019) - dotnet build from the command line seems ok.

The git repo for the project that gets the error is quite large (over 400 megabytes), whereas the one where it's ok is trivial. I'm not sure if there is an issue with large data, or if that case just happens to fail over some other issue more quickly?

[AArnott]

Thanks for the report, @Numpsy.
@qmfrederik Can you please investigate? I wonder if we're running out of memory, or if it could be a handles issue from not closing files after opening them?
I think you're caching data. Do you have a policy for releasing that memory? For a build maybe it's not an issue but in a long-lived process like VS we should probably release memory after the top-level request is satisfied.

[qmfrederik]

@Numpsy version 3.4.161-alpha, which is available on NuGet, should contain a fix for this. Could you give that version a try and let me know if it resolved the issue for you?

[AArnott]

I'm closing optimistically, but we can re-open if the problem still repros.

[Numpsy]

I've just given it a go, and the new version does seem to have fixed it :-)

[bddckr]

Same here, I've been running it for 2 hours now (yes I did lurk and grab it as soon as the pipeline pushed it 😆). Before the fix I had to restart VS after a bunch of builds, multiple times per hour. Since I installed the latest alpha I've not been restarting VS.

Thanks for the great work!

[qmfrederik]

Thanks for the feedback, @Numpsy, @bddckr!