Roslyn: Microsoft.Net.Compilers is bundled with old version of System.Collections.Immutable

[Tasteful]

Version Used:
VS 2015 Update 2

Steps to Reproduce:

1. Create an empty web application project TFM 4.6.1. The template contains reference to the package Microsoft.Net.Compilers version 1.0.0.
2. Add package reference to System.Collections.Immutable version 1.2.0

Expected Behavior:
Get a reference to C:\Users\user\Documents\Visual Studio 2015\Projects\WebApplication4\packages\System.Collections.Immutable.1.2.0\lib\portable-net45+win8+wp8+wpa81\System.Collections.Immutable.dll

Actual Behavior:
Got a reference to C:\Users\user\Documents\Visual Studio 2015\Projects\WebApplication4\packages\Microsoft.Net.Compilers.1.0.0\build\..\tools\System.Collections.Immutable.dll

[smithkl42]

This also happens on VS2015 Update 3. The net effect of this is that you can't upgrade to both the latest version of Microsoft.Net.Compilers and System.Collections.Immutable. My project is now full of errors like:

Api\Helpers\SwyfftResponseResults.cs(65,31): error CS1705: Assembly 'Swyfft.Services' with identity 'Swyfft.Services, Version=1.0.6023.20428, Culture=neutral, PublicKeyToken=null' uses 'Syste m.Collections.Immutable, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' which has a higher version than referenced assembly 'System.Collections.Immutable' with identity 'S ystem.Collections.Immutable, Version=1.1.37.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' [C:\source\swyfft\swyf-website\Swyfft.Web\Swyfft.Web.csproj]

[chriswelles]

When updating to the latest Microsoft.Net.Compiler, I see it forcing in it's own version of System.Collections.Immutable, as well as a number of others including Analyzer. Even if you try to reference them directly, the Compiler seems to override. I end up with similar errors. Happens with both Update 2 and 3, although update 3 seems to recognize the reference is coming out of the Compilers nuget package, whereas Update 2 seems to believe it's coming out of where you point it at (what you're trying to make happen).

[smithkl42]

BTW, I get the same error when using the previous version of the compiler - it still wants to force the old version of System.Collections.Immutable on me. My only solution so far has been to go back to the old (1.1.3) version of System.Collections.Immutable. At least at the moment - for me - running the latest System.Collections.Immutable seems to be incompatible with running any version of the compiler NuGet package.

[ljw1004]

I ran into the "old version of System.Collections.Immutable" problem in a completely different scenario, when authoring a VSIX: #12247

[chriswelles]

I will point out that when using Microsoft.Net.Compilers in an ASP.NET Core project (running on 4.6.1), it doesn't force the downgrade. It only seems to happen with the older style csproj projects as far as I've noticed.

[agocke]

@chriswelles Thanks for the pointer to this thread.

This is super weird, it looks like some kind of bug in the NuGet build task, or maybe the project system.

/cc @jasonmalinowski @Pilchie I think I'm running up against my understanding of how these references get pulled in, can you take a look?

[jasonmalinowski]

This is a bug in MSBuild, the Microsoft.CodeDom.Providers.DotNetCompilerPlatform package, or maybe even both!

The ResolveAssemblyReferences task in MSBuild is handed a list of "candidate assembly files", the idea being that assemblies being copied in as content should possible be used as the resolution, as opposed to picking up some other version:

    <ResolveAssemblyReference
        Assemblies="@(Reference)"
        AssemblyFiles="@(_ResolvedProjectReferencePaths);@(_ExplicitReference)"
        TargetFrameworkDirectories="@(_ReferenceInstalledAssemblyDirectory)"
        InstalledAssemblyTables="@(InstalledAssemblyTables);@(RedistList)"
        IgnoreDefaultInstalledAssemblyTables="$(IgnoreDefaultInstalledAssemblyTables)"
        IgnoreDefaultInstalledAssemblySubsetTables="$(IgnoreInstalledAssemblySubsetTables)"
        CandidateAssemblyFiles="@(Content);@(None)"

The Microsoft.CodeDom.Providers.DotNetCompilerPlatform package puts the compiler in as None items:

<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <None Include="$(CscToolPath)\*">
      <Link>roslyn\%(RecursiveDir)%(Filename)%(Extension)</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>
</Project>

As a result, the binary being copied in as a dependency of the project is being referenced, otherwise against your wishes.

MSBuild could fix this by changing ResolveAssemblyReferences to be smarter here and realize these really shouldn't be considered candidates. The Microsoft.CodeDom.Providers.DotNetCompilerPlatform could work around this by either having a task to do the copy explicitly (bypassing the item group), or only creating these items after ResolveAssemblyReferences runs. ResolveAssemblyReferences isn't giving any option to opt these particular items out, it appears; it just sees the items as strings without metadata.

I'm quite frankly unsure where to route this bug next. One workaround might be for affected projects would be to update the AssemblySearchPaths property in your project:

    <AssemblySearchPaths Condition=" '$(AssemblySearchPaths)' == ''">
      {CandidateAssemblyFiles};
      $(ReferencePath);
      {HintPathFromItem};
      {TargetFrameworkDirectory};
      {Registry:$(FrameworkRegistryBase),$(TargetFrameworkVersion),$(AssemblyFoldersSuffix)$(AssemblyFoldersExConditions)};
      {AssemblyFolders};
      {GAC};
      {RawFileName};
      $(OutDir)
    </AssemblySearchPaths>

To remove the {CandidateAssemblyFiles} option, but I don't know what else that would break.

[chriswelles]

@jasonmalinowski, would it perhaps make sense to address it in the same manner that .xproj projects take? They don't seem to have the same problems when tackling these same sets of packages. If the same approach would work, the additional consistency would be a plus.

[jasonmalinowski]

@chriswelles The systems are fundamentally different here.

[Tasteful]

@jasonmalinowski any progress to solve this issue?

[jasonmalinowski]

@Tasteful Nothing much yet, sorry.

[bnayae]

You can use the following work around until the issue will be fixed:

add new folder to the ASP.NET project (I called this folder BinFIx)
copy the System.Collections.Immutable.dll into the folder (and make sure it sets as content)

at the web config add the following probing

      <dependentAssembly>
        <assemblyIdentity name="System.Collections.Immutable" publicKeyToken="b03f5f7f11d50a3a" />
        <bindingRedirect oldVersion="1.1.36.0-1.2.0.0" newVersion="1.2.0.0" />
        <codeBase version="1.2.0.0" href="BinFix/System.Collections.Immutable.dll"/>
      </dependentAssembly>

(it should go under <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">)

now you're good to go

[paul1956]

Is there a similar workaround for a VSIX project? I tried adding the following to app.config

      <dependentAssembly>
        <assemblyIdentity name="System.Collections.Immutable" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-1.2.0.0" newVersion="1.2.0.0" />
        <codeBase version="1.2.0.0" href="BinFix/System.Collections.Immutable.dll"/>
      </dependentAssembly>

and get further but then crash with the following

System.MissingMethodException occurred
Message: Exception thrown: 'System.MissingMethodException' in mscorlib.dll
Additional information: Method not found: 'System.Collections.Immutable.ImmutableArray`1<Microsoft.CodeAnalysis.Diagnostic> Microsoft.CodeAnalysis.SemanticModel.GetDiagnostics(System.Nullable`1<Microsoft.CodeAnalysis.Text.TextSpan>, System.Threading.CancellationToken)'.

[jasonmalinowski]

@paul1956 app.configs aren't consumed in VSIX projects like that; you have to use something like ProvideBindingRedirectionAttribute. Be careful, since there's already existing redirects in place for the version we ship; in theory they should merge but there might be bugs. Shipping into Visual Studio is often a place where it helps to target what ships with the product than upgraded versions unless you really need something.

[paul1956]

All I did was update Visual Studio to Update 3, upgrade my NuGet packages and everything broke. At runtime system.collections.immutable.dll is not found even though it's NuGet package is there and the application builds without error.

[jasonmalinowski]

@paul1956 In general, if you're a Visual Studio extension, upgrading to newer NuGet packages is a tricky affair, because the moment you move to something newer than is installed (Update 3 still has 1.1.37 inside of it), things are going to break unless you specify your own redirects and codebases via the attributes. It's a very tricky affair, unfortunately. In any case, what you're experiencing is not related to this bug at all.

[paul1956]

@jasonmalinowski When I upgraded to Visual Studio 2015 Update 3, 42 NuGet Updates showed up. How would anyone know not to upgrade them or which are safe and which ones will break something. Rolling back the project to before update 3 and it works even under Update 3 as long as I don't upgrade the NuGet packages.

[jasonmalinowski]

@paul1956: the upgrading to Update 3 has nothing to do with this; the simple fact is upgrading NuGet packages is a dangerous operation in VSIX projects. That would be a conversation though at the Microsoft/extendvs side of things, not here, if you want to open a different issue.

[natemcmaster]

We ran into this issue on Entity Framework Core as well. It seems it is possible to workaround by altering the System.Collections.Immutable reference in csproj by setting "SpecificVersion" = true. NuGet defaults to "SpecificVersion" = false, which allowed the 1.1.36.0 version to end up in the build output.

<Reference Include="System.Collections.Immutable, Version=1.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
      <SpecificVersion>True</SpecificVersion> <!--  nuget installation defaults to "false" -->
      <HintPath>..\packages\System.Collections.Immutable.1.2.0\lib\portable-net45+win8+wp8+wpa81\System.Collections.Immutable.dll</HintPath>
      <Private>True</Private>
</Reference>

[natemcmaster]

@jasonmalinowski your suggestion above https://github.com/dotnet/roslyn/issues/12255#issuecomment-230045041 is good, and I think this fix can be added to the .props file in Microsoft.CodeDom.Providers.DotNetCompilerPlatform. This would prevent MSBuild from thinking anything in bin/debug/roslyn/*.dll is potential solution for <Reference> items.

[Tasteful]

With the new release of DotNetCompilerPlatform the reference is correctly linked. Update the Microsoft.CodeDom.Providers.DotNetCompilerPlatform package to version >=1.0.2

[smithkl42]

That worked for me. Thanks.

[ewahner]

This seems to still be a problem for sites built with TFS online. How would we go about updating that package there?

[Tasteful]

@ewahner If you update your local version of the source to use the new NuGet package and pushing the source into TFS, build should then use the new version when building.

[ewahner]

Yeah that is exactly what I did and I still get the error. Even worse...I have to add the redirect to my config in azure, because if I add it in my local config I get the same error having to do with the fact that I no longer have 1.1.36.

[ewahner]

It's definitely something with TFS build. If I publish to azure from my workstation, it works fine. If I run the build from TFS I get the Immutable error.

[Zapnologica]

I am on version DotNetCompilerPlatform 1.0.2 VS 2015 update 3 and I cant build my project.

Error CS0012 The type 'WebPageBase' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Web.WebPages, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'

[rifano84]

Another solution is to reference directly the System.Collections.Immutable.dll using the project references