Specflow: System of plugins is broken for Specflow 2.2.0

[DamirAinullin]

I support free plugin https://github.com/DamirAinullin/specflow-retry for Specflow.

After release of 2.2.0 I found the problem that the plugin is initialized two times (Method Initialize from GeneratorPlugin class is called two times). And in debug mode I see two same plugins in collection Plugins in SpecFlowConfiguration class. It does not reproduced for 2.1.0 or next pre-release versions.

From client view I see the following error:

Custom tool error: Generation error: Exception has been thrown by the target of an invocation. -> Collection was modified; enumeration operation may not execute.

[SabotageAndi]

Interesting. In the SpecFlow+Runner I can not see the described behavior.
Could you send us a PR with failing tests, that we can have a look at it?

[DamirAinullin]

Yes, sure, I'll do it a bit later.

Also you can try to open solution from https://github.com/DamirAinullin/specflow-retry. The problem is reproduced when I try to regenerate feature file.

[victorsantoss]

I get something similar to this.
On my case, any exceptions are blown up, but I'm also getting twice the plugin initialization.

Is there a way to avoid that and get the plugin initalized only once?

[pratishek]

I am trying to use @DamirAinullin plugin and it's throws the same error mentioned in the title:

error Generation error: Exception has been thrown by the target of an invocation. -> Collection was modified; enumeration operation may not execute.

@SabotageAndi : Any update on this ?

[mgladchenko]

This issue still exist in SpecFlow 2.2.0 and latest 2.3.0 releases.
And it did not occur in 2.1.0

Any suggestions?

[yesiamtom]

I am seeing the same issue in 2.3.0 release as @mgladchenko mentions. This plugin is extremely useful and being stuck on Specflow 2.1 is rather annoying.

I am using Specflow (and associated plugins) 2.3, Retry 2.3, NUnit 3.9, Selenium 3.1 and latest chromedriver. I can confirm the error appears the minute I add the section to app.config, according to the readme of Retry. I've not had any luck in changing the settings here so far.

[mgladchenko]

With SpecFlow 2.3.1 I get better error message, as follows:
`#error TechTalk.SpecFlow.Generator.Interfaces.TestGenerationError

error TechTalk.SpecFlow.Generator

error Server stack trace:

error at TechTalk.SpecFlow.Generator.Plugins.GeneratorPluginLoader.LoadPlugin(PluginDescriptor pluginDescriptor)

error at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.LoadPlugin(PluginDescriptor pluginDescriptor, IGeneratorPluginLoader pluginLoader, GeneratorPluginEvents generatorPluginEvents)

error at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.LoadPlugins(ObjectContainer container, IGeneratorConfigurationProvider configurationProvider, SpecFlowConfigurationHolder configurationHolder, GeneratorPluginEvents generatorPluginEvents, SpecFlowProjectConfiguration specFlowConfiguration)

error at TechTalk.SpecFlow.Generator.GeneratorContainerBuilder.CreateContainer(SpecFlowConfigurationHolder configurationHolder, ProjectSettings projectSettings)

error at TechTalk.SpecFlow.Generator.TestGeneratorFactory.CreateGenerator(ProjectSettings projectSettings)

error at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)

error at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

error Exception rethrown at [0]:

error at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

error at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

error at TechTalk.SpecFlow.Generator.Interfaces.ITestGeneratorFactory.CreateGenerator(ProjectSettings projectSettings)

error at TechTalk.SpecFlow.IdeIntegration.Generator.RemoteAppDomainTestGeneratorFactory.CreateGenerator(ProjectSettings projectSettings)

error at TechTalk.SpecFlow.IdeIntegration.Generator.GeneratorServices.CreateTestGenerator()

error at TechTalk.SpecFlow.IdeIntegration.Generator.IdeSingleFileGenerator.GenerateCode(String inputFilePath, String inputFileContent, GeneratorServices generatorServices, ProjectSettings projectSettings)

error at TechTalk.SpecFlow.IdeIntegration.Generator.IdeSingleFileGenerator.Generate(String inputFilePath, String inputFileContent, GeneratorServices generatorServices, CodeDomHelper codeDomHelper, ProjectSettings projectSettings)`

But. It does not tell me anything.
Anyone?

[bas010203]

I'm experiencing the same error when I try to add the retry plugin. Rather disappointing to read that this issue is open since July 2017 and the TechTalk / SpecFlow team doing nothing about it.

I'm now looking for an alternative to SpecFlow with a retry mechanism. Does anyone has a good alternative?

[SabotageAndi]

@bas010203 Sorry to hear that you will switch away from SpecFlow.

Please have in mind, that this is an open source project. It depends on contributions from us all.

About this issue: I have 2 plugins of my own and they don't have this problem. So for me looks like the issue is within the plugin. We have changed the plugin infrastructure in every minor release since 1.9 (except 2.3). So there is a probability that the plugin has to be changed.
I asked @DamirAinullin for a PR with failing tests, but I didn't get any yet. If I had one, this would be a really time saver for me to look at this issue. For debugging his whole plugin I simply have no time. Sorry, I would love to fix everything, but I have to prioritise to use my time effectively.

[victorsantoss]

But, is it correct that a plugin is initialized twice?
I did a small plugin similar to use the Microsoft Dependency Injection and I got exactly the same behavior.

@SabotageAndi , If you need any help to check this out, I can submit a plugin to check, I think that is not a problem with 3rd parties plugins, but with the way the plugins are initialized.

[SabotageAndi]

@victorsantoss I don't know. That's what @DamirAinullin wrote.
For debugging a red tests added to the SpecFlow test suite would be the easiest way for me to look at it.
Standalone projects are really hard to debug and that it's what is time consuming.

[victorsantoss]

That's what he wrote and what I see over my plugin.

[assembly: RuntimePlugin(typeof(MyPlugin))]

namespace Blablabla.MyPlugin
{
    public class MyPlugin : IRuntimePlugin, IDisposable
    {
        public void Dispose()
        {
            throw new NotImplementedException();
        }

        public void Initialize(RuntimePluginEvents runtimePluginEvents, RuntimePluginParameters runtimePluginParameters)
        {
          // Passing here twice!
            runtimePluginEvents.CustomizeGlobalDependencies += (sender, args) =>
            {
                args.ObjectContainer.RegisterTypeAs<ServiceBindingInstanceResolver, ITestObjectResolver>();
                args.ObjectContainer.RegisterTypeAs<ContainerFinder, IContainerFinder>();
            };

            runtimePluginEvents.CustomizeScenarioDependencies += (sender, args) =>
            {
                args.ObjectContainer.RegisterFactoryAs(() =>
                {
                    var containerBuilderFinder = args.ObjectContainer.Resolve<IContainerFinder>();
                    var containerBuilder = containerBuilderFinder.GetCreateScenarioContainer();
                    var container = containerBuilder.Invoke();
                    return container;
                });
            };       
        }        
    }    
}

[mgladchenko]

@bas010203 Regarding alternatives. If you are using VSTS you may use their build in retry mechanism. They released it about 1 month ago. That is the case for me. I'll probably switch to that.

[SabotageAndi]

should be resolved with #1126 thanks to @vitezp

[sonudavidson]

Hello there,

Is it possible to release the fixed #1126 version.?

Thanks!

[sonudavidson]

Hi @SabotageAndi ,

Please, Could you help me?
@DamirAinullin 's code to rerun a failed test helps our team reduce rework effort. Is it possible to release an intermediate version with #1126 fix?

Thanks!

[SabotageAndi]

@sonudavidson Sorry, I have no time currently to do a release to NuGet.org. But I am working on it to make releases faster.
In the meantime you can use the packages from our CI- feed: https://ci.appveyor.com/nuget/specflow-ci

[sonudavidson]

@SabotageAndi Thank you! For the prompt reply.!

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.