Msbuild: MSBuild crashes if environment variable too long

[danmoseley]

.NET rejects env var values over 32KB with ArgumentException. MSBuild sets environment varibles in several places and in this place at least does not handle that nicely. In this particular callstack it should perhaps just not set that environment variable. Possibly it should fail at initialization though if there's an environment variable that is so long it wouldn't be able to reset it when the build is done.

Note there's a (shorter) limit on env var name also.

https://github.com/dotnet/coreclr/issues/9793

06:57:53 Starting Build at 6:57:53.56
06:57:53 BUILD: Commencing CoreCLR Repo build
06:57:53 BUILD: Checking prerequisites
06:57:56 BUILD: Using environment: "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\VsDevCmd.bat"
06:58:02 Installing .NET Version Manager to C:\Users\dotnet-bot.dnx\bin
06:58:02 Creating destination folder 'C:\Users\dotnet-bot.dnx\bin' ...
06:58:02 Installing 'dnvm.ps1' to 'C:\Users\dotnet-bot.dnx\bin' ...
06:58:02 Installing 'dnvm.cmd' to 'C:\Users\dotnet-bot.dnx\bin' ...
06:58:02 Adding C:\Users\dotnet-bot.dnx\bin to Process PATH
06:58:02 Adding C:\Users\dotnet-bot.dnx\bin to User PATH
06:58:06 Installing dotnet cli...
06:58:13 Restoring BuildTools version 1.0.27-prerelease-01209-01...
06:58:13 Initializing BuildTools ...
06:58:52 Updating CLI NuGet Frameworks map...
06:58:52 Done initializing tools.
06:58:52 Running: D:\j\workspace\x86_checked_w---b3a226f6\Tools\dotnetcli\dotnet.exe D:\j\workspace\x86_checked_w---b3a226f6\Tools\run.exe D:\j\workspace\x86_checked_w---b3a226f6\config.json build -Project=D:\j\workspace\x86_checked_w---b3a226f6\build.proj -generateHeaderWindows -NativeVersionHeaderFile="D:\j\workspace\x86_checked_w---b3a226f6\bin\obj_version.h" -BuildOS=Windows_NT -BuildType=Checked -BuildArch=x86
06:58:52 Running: D:\j\workspace\x86_checked_w---b3a226f6\Tools\msbuild.cmd /nologo /verbosity:minimal /clp:Summary /maxcpucount /nodeReuse:false /l:BinClashLogger,Tools/net45/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log D:\j\workspace\x86_checked_w---b3a226f6\build.proj /p:__BuildType=Checked /p:__BuildArch=x86 /p:__BuildOS=Windows_NT /t:GenerateVersionHeader /p:GenerateVersionHeader=true /p:NativeVersionHeaderFile=D:\j\workspace\x86_checked_w---b3a226f6\bin\obj_version.h
06:59:00
06:59:00 Unhandled Exception: Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Environment variable name or value is too long.
06:59:00 =============
06:59:00 System.ArgumentException: Environment variable name or value is too long.
06:59:00 at System.Environment.SetEnvironmentVariable(String variable, String value)
06:59:00 at Microsoft.Build.BackEnd.InProcNode.HandleShutdown(Exception& exception)
06:59:00 at Microsoft.Build.BackEnd.InProcNode.Run(Exception& shutdownException)
06:59:00
06:59:00 ---> System.ArgumentException: Environment variable name or value is too long.
06:59:00 at System.Environment.SetEnvironmentVariable(String variable, String value)
06:59:00 at Microsoft.Build.BackEnd.InProcNode.HandleShutdown(Exception& exception)
06:59:00 at Microsoft.Build.BackEnd.InProcNode.Run(Exception& shutdownException)
06:59:00 --- End of inner exception stack trace ---
06:59:00 at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
06:59:00 at Microsoft.Build.BackEnd.InProcNode.Run(Exception& shutdownException)
06:59:00 at Microsoft.Build.BackEnd.NodeProviderInProc.InProcNodeThreadProc()
06:59:00 at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
06:59:00 at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
06:59:00 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
06:59:00 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
06:59:00 at System.Threading.ThreadHelper.ThreadStart()
06:59:01 Command execution failed with exit code -532462766.

[danmoseley]

I opened dotnet/corefx#16766 to accept longer variables but there will always be a limit and MSBuild should not crash.

[joseotavioq]

@rainersigwald As we know that we have a limitation for environment variable values of 32,767 characters, should we first check and ignore the variable set if it has more than 32,767 characters?

[danmoseley]

The .NET Core change was to not attempt to validate the length ourselves (which we can get wrong depending on the platform, etc), but instead handle any OS error from trying to set it. MSBuild should do the same.

On recent .NET Core, you should still get ArgumentException
https://github.com/dotnet/runtime/blob/6072e4d3a7a2a1493f514cdf4be75a3d56580e84/src/libraries/System.Private.CoreLib/src/System/Environment.Variables.Windows.cs#L58-L61
(Not sure about Unix)

[danmoseley]

Handling ArgumentException will work just as well on .NET Framework as well, it's just that .NET will be throwing it based on its own (stricter) validation, vs .NET Core will be throwing it based on the OS rejecting it.

[danmoseley]

What you do if you get the exception is a decision to make, I guess either fail the build, silently continue, or log it could not be set and continue -- I'm guessing the latter.

[joseotavioq]

@danmosemsft I was taking a look at the .NET Framework source code and I saw that it will throw the ArgumentException error if the value length is greater or _EQUAL_ to 32767.
https://referencesource.microsoft.com/#mscorlib/System/environment.cs,902

Looking at the function SetEnvironmentVariableW documentation, which is used by .NET Core, we can see that the maximum size of a user-defined environment variable is 32,767 characters.

.NET Core implementation: https://github.com/dotnet/runtime/blob/master/src/libraries/Common/src/Interop/Windows/Kernel32/Interop.SetEnvironmentVariable.cs

SetEnvironmentVariableW documentation: https://docs.microsoft.com/en-us/windows/win32/api/processenv/nf-processenv-setenvironmentvariablew

I tried to set a value with more than 32767 characters with .NET Core and I did not get an error.

Example:

string environmentVariableName = "ABC";
System.Environment.SetEnvironmentVariable(environmentVariableName, new string('a', 999999));
var value = System.Environment.GetEnvironmentVariable(environmentVariableName);

Am I doing something wrong?

[danmoseley]

My guess is that Windows raised the limit again. Feel free to offer an issue PR for their docs.

Environment.SetEnvironmentVariable("ABC", new string('a', Int32.MaxValue/4))
Expression has been evaluated and has no value
Environment.GetEnvironmentVariable("ABC").Length
536870911

This demonstrates why it was a good change to make the OS validate the length, rather than .NET. We'd have gotten it wrong again if we did it.