SpecFlow doesn't work with .NET Core 3.1.200 SDK

[SabotageAndi]

SpecFlow Version:

• [x] 3.1
• [ ] 3.0
• [ ] 2.4
• [ ] 2.3
• [ ] 2.2
• [ ] 2.1
• [ ] 2.0
• [ ] 1.9

Used Test Runner

• [x] SpecFlow+Runner
• [x] MSTest
• [x] NUnit
• [x] Xunit

Version number:

Project Format of the SpecFlow project

• [ ] Classic project format using packages.config
• [ ] Classic project format using <PackageReference> tags
• [x] Sdk-style project format

.feature.cs files are generated using

• [x] SpecFlow.Tools.MsBuild.Generation NuGet package
• [ ] SpecFlowSingleFileGenerator custom tool

Visual Studio Version

• [x] VS 2019
• [ ] VS 2017
• [ ] VS 2015

Enable SpecFlowSingleFileGenerator Custom Tool option in Visual Studio extension settings

• [ ] Enabled
• [x] Disabled

Are the latest Visual Studio updates installed?

• [x] Yes
• [ ] No, I use Visual Studio version <Major>.<Minor>.<Patch>

.NET Framework:

• [ ] >= .NET 4.5
• [ ] before .NET 4.5
• [ ] .NET Core 2.0
• [ ] .NET Core 2.1
• [ ] .NET Core 2.2
• [x] .NET Core 3.0

Test Execution Method:

• [ ] Visual Studio Test Explorer
• [ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
• [x] Command line – dotnet test

Issue Description

When using the .NET Core 3.1.200 SDK, the MSBuild Generation fails with following error:

C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error : [SpecFlow] System.IO.FileNotFoundException: Could not load file or assembly 'TechTalk.SpecFlow, Version=3.1.0.0, Culture=neutral, PublicKeyToken=0778194805d6db41'. The system cannot find the file specified. [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error : File name: 'TechTalk.SpecFlow, Version=3.1.0.0, Culture=neutral, PublicKeyToken=0778194805d6db41' [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
t32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Reflection.RuntimeAssembly.GetCustomAttributes(Type attributeType, Boolean inherit) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Attribute.GetCustomAttributes(Assembly element, Type attributeTC:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType, Boolean inherit) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at TechTalk.SpecFlow.Generator.Plugins.GeneratorPluginLoader.LoadPlugin(PluginDescriptor pluginDescriptor) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.LoadPlugin(PluginDescriptor pluginDescriptor, IGeneratorPluginLoader pluginLoader, GeneratorPluginEvents generatorPluginEvents, UnitTestProviderConfiguration unitTestProviderConfigration) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.LoadPlugins(ObjectContainer container, GeneratorPluginEvents generatorPluginEvents, UnitTestProviderConfiguration unitTestProviderConfigration, IEnumerable`1 generatorPlugins) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.CreateContainer(SpecFlowConfigurationHolder configurationHolder, ProjectSettings projectSettings, IEnumerable`1 generatorPlugins) [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :    at SpecFlow.Tools.MsBuild.Generation.GenerateFeatureFileCodeBehindTask.Execute() [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :  [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]
C:\Users\awi\.nuget\packages\specflow.tools.msbuild.generation\3.1.62\build\SpecFlow.Tools.MsBuild.Generation.targets(93,5): error :  [C:\work\SpecFlow-Examples\NETCore Examples\BowlingKata-SpecFlowJson-xUnit\Bowling.SpecFlowXUnit\Bowling.SpecFlowXUnit.csproj]

Steps to Reproduce

Workaround

Set Environment variable to get all MSBuild behaviour:

MSBUILDSINGLELOADCONTEXT=1

[puco]

Having the same issue in Docker build. Was using mcr.microsoft.com/dotnet/core/sdk:3.1-alpine and switched to mcr.microsoft.com/dotnet/core/sdk:3.1.102-alpine. Noticed that 3.1.200 uses MSBuild 16.5, whereas 3.1.102 uses 16.4

[trondgod]

The SDK version 3.1.102 seems to be gone...
https://github.com/microsoft/azure-pipelines-tasks/issues/12540

[SabotageAndi]

@trondgod They have it on their download page: https://dotnet.microsoft.com/download/dotnet-core/3.1
Looks like 3.1.102 was released with 3.1.200 and the difference is the supported VS version (which is also MSBuild version).

[dasMulli]

MSBuild 16.5 has a new assembly loading approach for tasks that may have side effects. (which has been an ask for years, but it behaves a bit differently when loading dependencies)
You can try to set MSBUILDSINGLELOADCONTEXT=1 (env var) to get the old behavior.

[rainersigwald]

@SabotageAndi I suspect @dasMulli has correctly identified the reason this is happening. But I'm not sure exactly what the problem is. Is there an easy way to get a small repro project so that I can debug the assembly loading behavior?

[SabotageAndi]

@rainersigwald I am working on a repro right now.

[SabotageAndi]

@rainersigwald I have it.

You can find it here: https://github.com/SabotageAndi/GH1912

Simply build it with the included build.ps1 script.

If building with .NET Core SDK 3.1.200 I get following output:

PS C:\work\GH1912Repro> .\build.ps1
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 241,96 ms for C:\work\GH1912Repro\Common\Common.csproj.
  Restore completed in 298,88 ms for C:\work\GH1912Repro\MSBuildTask\MSBuildTask.csproj.
  Common -> C:\work\GH1912Repro\Common\bin\Debug\netstandard2.0\Common.dll
  MSBuildTask -> C:\work\GH1912Repro\MSBuildTask\bin\Debug\netcoreapp2.0\MSBuildTask.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.53
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 35,71 ms for C:\work\GH1912Repro\MSBuildTask\MSBuildTask.csproj.
  Restore completed in 35,71 ms for C:\work\GH1912Repro\Common\Common.csproj.
  Restore completed in 174,62 ms for C:\work\GH1912Repro\App\App.csproj.
  Restore completed in 174,59 ms for C:\work\GH1912Repro\Plugin\Plugin.csproj.
  Common -> C:\work\GH1912Repro\Common\bin\Debug\netstandard2.0\Common.dll
  MSBuildTask -> C:\work\GH1912Repro\MSBuildTask\bin\Debug\netcoreapp2.0\MSBuildTask.dll
  Plugin -> C:\work\GH1912Repro\Plugin\bin\Debug\netstandard2.0\Plugin.dll
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018: The "AssemblyLoadTask" task failed unexpectedly.
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018: File name: 'Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, BooleaC:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntC:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.RuntimeAssembly.GetCustomAttributes(Type attributeType, Boolean inherit)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Attribute.GetCustomAttributes(Assembly element, Boolean inherit)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttributeExtensions.GetCustomAttributes(Assembly element)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at MSBuildTask.AssemblyLoadTask.Execute() in C:\work\GH1912Repro\MSBuildTask\AssemblyLoadTask.cs:line 19
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:

Build FAILED.

C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018: The "AssemblyLoadTask" task failed unexpectedly.
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018: File name: 'Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.RuntimeAssembly.GetCustomAttributes(Type attributeType, Boolean inherit)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Attribute.GetCustomAttributes(Assembly element, Boolean inherit)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at System.Reflection.CustomAttributeExtensions.GetCustomAttributes(Assembly element)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at MSBuildTask.AssemblyLoadTask.Execute() in C:\work\GH1912Repro\MSBuildTask\AssemblyLoadTask.cs:line 19
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:
C:\work\GH1912Repro\App\App.csproj(15,5): error MSB4018:
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:02.23

With .NET Core SDK 3.1.102 I get this output:

Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 182,73 ms for C:\work\GH1912Repro\MSBuildTask\MSBuildTask.csproj.
  Restore completed in 182,73 ms for C:\work\GH1912Repro\Common\Common.csproj.
  Common -> C:\work\GH1912Repro\Common\bin\Debug\netstandard2.0\Common.dll
  MSBuildTask -> C:\work\GH1912Repro\MSBuildTask\bin\Debug\netcoreapp2.0\MSBuildTask.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.37
Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 29,24 ms for C:\work\GH1912Repro\Common\Common.csproj.
  Restore completed in 29,44 ms for C:\work\GH1912Repro\MSBuildTask\MSBuildTask.csproj.
  Restore completed in 141,4 ms for C:\work\GH1912Repro\App\App.csproj.
  Restore completed in 141,39 ms for C:\work\GH1912Repro\Plugin\Plugin.csproj.
  Common -> C:\work\GH1912Repro\Common\bin\Debug\netstandard2.0\Common.dll
  MSBuildTask -> C:\work\GH1912Repro\MSBuildTask\bin\Debug\netcoreapp2.0\MSBuildTask.dll
  Plugin -> C:\work\GH1912Repro\Plugin\bin\Debug\netstandard2.0\Plugin.dll
  App -> C:\work\GH1912Repro\App\bin\Debug\netcoreapp3.1\App.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:02.42

I hope this helps you.

[rainersigwald]

@SabotageAndi that is an amazing repro! Thank you! I reproduce the problem with it and will take a look.

[rainersigwald]

One quick note: with @SabotageAndi's repro, MSBUILDSINGLELOADCONTEXT=1 does appear to work around the problem. I would recommend that over downgrading VS or the SDK.

[SabotageAndi]

@rainersigwald any news on this issue?

[rainersigwald]

No news yet, sorry. Filed https://github.com/microsoft/msbuild/issues/5202 to keep it top of mind for my team.

[SabotageAndi]

Do you think we could do something on our side to get around the issue? More and more people are hitting this issue.

[rainersigwald]

Your repro case uses Assembly.LoadFile(path) to load a dependent assembly. On .NET Core 3.1 (at least, probably since 2.0?), this creates a new AssemblyLoadContext--on my machine it's named Assembly.LoadFile(S:\repro\SpecFlowOSS\SpecFlow\issues\1912\Plugin\bin\debug\netstandard2.0\Plugin.dll).

The docs for Assembly.LoadFile say

LoadFile does not load files into the load-from context, and does not resolve dependencies using the load path, as the LoadFrom method does.

That's exactly what's happening here: it's failing to resolve dependencies using the path of the loaded file.

This worked before 3.1.200/MSBuild 16.5, because MSBuild installed an assembly load hook into the _default_ ALC that would load dependencies from "any directory next to a loaded task assembly".

If I use LoadFrom instead,

diff --git a/MSBuildTask/AssemblyLoadTask.cs b/MSBuildTask/AssemblyLoadTask.cs
index d0c5010..fa497ba 100644
--- a/MSBuildTask/AssemblyLoadTask.cs
+++ b/MSBuildTask/AssemblyLoadTask.cs
@@ -14,7 +14,7 @@ namespace MSBuildTask
         public override bool Execute()
         {
             var path = Path.Combine(Environment.CurrentDirectory, "..", "Plugin", "bin", "debug", "netstandard2.0", "Plugin.dll");
-            var assembly = Assembly.LoadFile(path);
+            var assembly = Assembly.LoadFrom(path);

             var customAttributes = assembly.GetCustomAttributes();
             var pluginAttribute = (PluginMarkerAttribute)customAttributes.SingleOrDefault(a => a is PluginMarkerAttribute);

your repro case works fine.

I think that's the right way to go. I first thought that you might be using LoadFile to work around the original problem that caused us to introduce ALCs for tasks--being able to load the same task in multiple versions. But LoadFile doesn't do that, because the transitive references get loaded into the default ALC anyway.

[SabotageAndi]

Thanks for the input. I will try to use Assembly.LoadFrom and see if this changes something.

[magicben]

Thanks for the input. I will try to use Assembly.LoadFrom and see if this changes something.

Hi @SabotageAndi , hope everything is fine on your side.
Did you have time to work on this fix?
Take care.

[SabotageAndi]

@magicben I am still looking into it. No new news yet.

[magicben]

@magicben I am still looking into it. No new news yet.

thank you @SabotageAndi

[SabotageAndi]

@rainersigwald I made a mistake in my repro. I used the wrong method. In SpecFlow we already using Assembly.LoadFrom (https://github.com/SpecFlowOSS/SpecFlow/blob/master/TechTalk.SpecFlow.Generator/Plugins/GeneratorPluginLoader.cs#L16).

[rainersigwald]

@SabotageAndi it looks like Assembly.LoadFrom loads into AssemblyLoadContext.Default.

It's fairly easy to load in the current ALC, but it requires having access to the ALC API which is available only on core. This patch appears to work in limited testing on https://github.com/SpecFlowOSS/SpecFlow-Examples/tree/master/SpecFlow.json/BowlingKata-SpecFlowJson-xUnit

diff --git a/TechTalk.SpecFlow.Generator/Plugins/GeneratorPluginLoader.cs b/TechTalk.SpecFlow.Generator/Plugins/GeneratorPluginLoader.cs
index 59b21d8f..446f936b 100644
--- a/TechTalk.SpecFlow.Generator/Plugins/GeneratorPluginLoader.cs
+++ b/TechTalk.SpecFlow.Generator/Plugins/GeneratorPluginLoader.cs
@@ -13,7 +13,11 @@ namespace TechTalk.SpecFlow.Generator.Plugins
             Assembly pluginAssembly;
             try
             {
+#if NETCOREAPP
+                pluginAssembly = System.Runtime.Loader.AssemblyLoadContext.GetLoadContext(typeof(GeneratorPluginLoader).Assembly).LoadFromAssemblyPath(pluginDescriptor.Path);
+#else
                 pluginAssembly = Assembly.LoadFrom(pluginDescriptor.Path);
+#endif

             }
             catch(Exception ex)
diff --git a/TechTalk.SpecFlow.Generator/TechTalk.SpecFlow.Generator.csproj b/TechTalk.SpecFlow.Generator/TechTalk.SpecFlow.Generator.csproj
index 55b7a955..66d41d57 100644
--- a/TechTalk.SpecFlow.Generator/TechTalk.SpecFlow.Generator.csproj
+++ b/TechTalk.SpecFlow.Generator/TechTalk.SpecFlow.Generator.csproj
@@ -1,6 +1,6 @@
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>$(SpecFlow_Generator_TFM)</TargetFrameworks>
+    <TargetFrameworks>$(SpecFlow_Generator_TFM);$(SpecFlow_Tools_TFM)</TargetFrameworks>
     <AssemblyName>TechTalk.SpecFlow.Generator</AssemblyName>
     <AssemblyOriginatorKeyFile>$(SpecFlow_KeyFile)</AssemblyOriginatorKeyFile>
     <SignAssembly>$(SpecFlow_EnableStrongNameSigning)</SignAssembly>

[sdg002]

We faced this problem on Azure Devops.
We solved it for now by using a global.json in the root and specifying the framework

[kthayer424]

@sdg002 I have this issue in Azure Devops too and having trouble fixing, is your global.json just

{
    "sdk": {
        "version": "3.1.102"
    }
}

?

[bgoggan]

I just tried using version 3.1.102 in ADO and our restore step failed in the build process. From what I could tell in the logs that version didn't exist anymore:
2020-04-07T18:35:14.2876281Z 3.1.100 [C:\Program Files\dotnet\sdk]
2020-04-07T18:35:14.2876612Z 3.1.101 [C:\Program Files\dotnet\sdk]
2020-04-07T18:35:14.2876939Z 3.1.103 [C:\Program Files\dotnet\sdk]
2020-04-07T18:35:14.2877943Z 3.1.200 [C:\Program Files\dotnet\sdk]
2020-04-07T18:35:14.2878271Z 3.1.201 [C:\Program Files\dotnet\sdk]

I tried using 3.1.103 but encountered the same error prior to trying the global.json

[SabotageAndi]

@bgoggan, @kthayer424 & @sdg002
I would set the environment variable MSBUILDSINGLELOADCONTEXT to 1 as described in the workaround and not use an older SDK version.

[bgoggan]

@SabotageAndi Ok, adding that env variable to our build step resolved the issue without the need of the global.json. thanks.

[sdg002]

@bgoggan Please, did you explicitly add the variable MSBUILDSINGLELOADCONTEXT under the variables section of the YAML ?

[bgoggan]

@sdg002 I actually just used the ADO Gui and added it as a Variable within the pipeline. I didn't specifically update anything within the YAML

[arkiaconsulting]

@sdg002 if you’re using Yaml, add the environment variable to the env property of the DotNetCore build task

[bjuraga]

Hi @arkiaconsulting, looking at the docs for the DotNetCore task, there is no env: property on it
Docs: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops
Can you please share a code snippet, that would help a lot, thanks!

[arkiaconsulting]

@bjuraga

- task: DotNetCoreCLI@2
  displayName: Build whole solution
  env:
    MSBUILDSINGLELOADCONTEXT: 1
  inputs:
    command: build
    projects: $(SolutionRoot)/My.sln
    arguments: --configuration $(BuildConfiguration) --no-restore

[bjuraga]

@bjuraga

- task: DotNetCoreCLI@2
  displayName: Build whole solution
  env:
    MSBUILDSINGLELOADCONTEXT: 1
  inputs:
    command: build
    projects: $(SolutionRoot)/My.sln
    arguments: --configuration $(BuildConfiguration) --no-restore

Thanks, worked like a charm.
Now i gotta spend my weekend looking into more of these "hidden" options :)

[Kronos11]

It's great there is a workaround, however is there a plan to actually fix this in an upcoming release?

[Socolin]

If a project references Microsoft.XmlSerializer.Generator, this workaround cause another error.

For now I have to downgrade microsoft SDK version in azure pipelines.

[js8080]

For anyone who is running into this while running VS Code in remote containers, you can set the work-around environment variable in your devcontainer.json as follows:

    "remoteEnv": {
        "MSBUILDSINGLELOADCONTEXT": "1"
        }

Then rebuild your container and poof... error gone.

Though I will also clarify that I was getting a slightly different error as described in #1939

System.IO.FileNotFoundException: Could not load file or assembly 'TechTalk.SpecFlow, Version=3.1.0.0, Culture=neutral, PublicKeyToken=0778194805d6db41'. The system cannot find the file specified.

[rainersigwald]

If a project references Microsoft.XmlSerializer.Generator, this workaround cause another error.

@Socolin can you link to such a project? Or share the error?

[Socolin]

If a project references Microsoft.XmlSerializer.Generator, this workaround cause another error.

@Socolin can you link to such a project? Or share the error?

I checked out more in details, and it seems I got a transient error while I was testing this... I just tried this again and it works now.

The logs were (last error were on Microsoft.XmlSerializer.Generator):

```

[error]Path\Project1.csproj(0,0): Error NU1102: Unable to find package Newtonsoft.Json with version (>= 12.0.3)

• Found 1 version(s) in Microsoft Visual Studio Offline Packages [ Nearest version: 9.0.1 ]
  D:\a1\s\Path\Project1.csproj : error NU1102: Unable to find package Newtonsoft.Json with version (>= 12.0.3) [D:\a1\s\Path\Solution.sln]
  D:\a1\s\Path\Project1.csproj : error NU1102: - Found 1 version(s) in Microsoft Visual Studio Offline Packages [ Nearest version: 9.0.1 ] [D:\a1\s\Path\Solution.sln]
  

  [error]Path\Project1.csproj(0,0): Error NU1101: Unable to find package System.Configuration.ConfigurationManager. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages

D:\a1\s\Path\Project1.csproj : error NU1101: Unable to find package System.Configuration.ConfigurationManager. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages [D:\a1\s\Path\Solution.sln]

[error]Path\Project1.csproj(0,0): Error NU1101: Unable to find package System.Drawing.Common. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages

D:\a1\s\Path\Project1.csproj : error NU1101: Unable to find package System.Drawing.Common. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages [D:\a1\s\Path\Solution.sln]

[error]Path\Project1.csproj(0,0): Error NU1101: Unable to find package System.Runtime.Caching. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages

D:\a1\s\Path\Project1.csproj : error NU1101: Unable to find package System.Runtime.Caching. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages [D:\a1\s\Path\Solution.sln]

[error]Path\Project1.csproj(0,0): Error NU1101: Unable to find package System.Diagnostics.PerformanceCounter. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages

D:\a1\s\Path\Project1.csproj : error NU1101: Unable to find package System.Diagnostics.PerformanceCounter. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages [D:\a1\s\Path\Solution.sln]

[error]Path\Project1.csproj(0,0): Error NU1101: Unable to find package Polly. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages

D:\a1\s\Path\Project1.csproj : error NU1101: Unable to find package Polly. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages [D:\a1\s\Path\Solution.sln]
Restore failed in 41.08 ms for D:\a1\s\Path\Project1.csproj.

[error]Path2\Project2.csproj(0,0): Error NU1101: Unable to find package Microsoft.XmlSerializer.Generator. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages

D:\a1\s\Path2\Project2.csproj : error NU1101: Unable to find package Microsoft.XmlSerializer.Generator. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages [D:\a1\s\Path\Solution.sln]

[error]C:\Program Files\dotnet\sdk\3.1.201\NuGet.targets(124,5): Error : Value cannot be null. (Parameter 'path')

C:\Program Files\dotnet\sdk\3.1.201\NuGet.targets(124,5): error : Value cannot be null. (Parameter 'path') [D:\a1\s\Path\Solution.sln]
````

Sorry for the noise.

[zplan]

Is SpecFlow working on a real fix for this issue?
Thanks for an update.

[SabotageAndi]

@rainersigwald Which .NET Core version should we target for the MSBuild Task? Currently, it is targeting .NET Framework 4.7.1 and .NET Standard 2.0. I would change the targeting of .NET Standard to a .NET Core version.

[rainersigwald]

@SabotageAndi MSBuild itself targets netcoreapp2.1, so I'd probably go with that. Then you'll keep working on 2.1 LTS SDKs.

[shohnn]

Hi guys, same problem here but was wondering how/where to set the MSBUILDSINGLELOADCONTEXT = 1 environment variable since I'm using Mac...

edit: ~% MSBUILDSINGLELOADCONTEXT=1 dotnet build did the trick

[SabotageAndi]

Some status updates:
We (@tzongithub and me) started with the first step to fix this issue. That is to change our MSBuild task from .NET Standard 2.0 to .NET Core 2.1. PR: https://github.com/SpecFlowOSS/SpecFlow/pull/1968

The next step will be to change the Assembly loading code.

[SabotageAndi]

We merged the last PR for this to fix. Build is now running to create new packages.
When the release pipeline is finished, you find SpecFlow 3.3-beta packages on NuGet.org.

[CurlyFire]

@SabotageAndi Great! How long does a release stay in beta usually ?

[SabotageAndi]

@CurlyFire Depends from version to version. This will be at least 3 weeks in beta. We have some additional changes planned in our next iteration (starts next Tuesday).

[SabotageAndi]

We published a first 3.3-beta package to NuGet.org (https://www.nuget.org/packages/SpecFlow/3.3.15-beta) which a fix for this.

Please try it out and give us feedback if it is now working for you.

[js8080]

@SabotageAndi seems to be working for me.

[smushtaq]

We published a first 3.3-beta package to NuGet.org (https://www.nuget.org/packages/SpecFlow/3.3.15-beta) which a fix for this.

Please try it out and give us feedback if it is now working for you.

worked for me. I used 3.3.16-beta

[MarcWils]

Works like a charm.

[alanta]

3.3.16-beta works for me as well

[arkiaconsulting]

Confirmed 3.3.16-beta

[SabotageAndi]

Thanks for the feedback. I am closing now this issue.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.