Specflow: BeforeScenario(tag) is called twice in certain scenarios

[jamescrowley]

Given the following feature file:

@someOtherTag @tagA
Feature: Some feature

Scenario: Some scenario
  Given abc
  When efg
  Then xyz

and the following step definitions class:

[Scope(Tag = "someOtherTag")]
[Binding]
public class Class1
{
    [BeforeScenario("tagA")]
    public void BeforeTagAScenario()
    {
        Console.WriteLine("BeforeTagAScenario");
    }

    [BeforeScenario]
    public void BeforeScenario()
    {
        Console.WriteLine("BeforeScenario");
    }
    [Given(@"abc")]
    public void GivenAbc()
    {
        ScenarioContext.Current.Pending();
    }

    [When(@"efg")]
    public void WhenEfg()
    {
        ScenarioContext.Current.Pending();
    }

    [Then(@"xyz")]
    public void ThenXyz()
    {
        ScenarioContext.Current.Pending();
    }

}

the output is

BeforeTagAScenario
BeforeScenario

I'm running SpecFlow 1.9.0, NUnit 2.6.2 and SpecFlow 1.9.1 of the VS 2012 add-in.

[0liver]

I can confirm the behavior using the same setup and versions. I even tried a slightly different setup: setting the tag to "tagC" in [BeforeScenario("tagC")] still executes that hook even though the scenario's tags are only mentioned in the [Scope] of the binding class.

Looks like some filtering is missing for the tags on the [BeforeScenario] hook.

[jamescrowley]

Actually, even weirder. Place the following in the class:

    [BeforeScenario("tagB")]
    public void BeforeTagBScenario()
    {
        Console.WriteLine("BeforeTagBScenario");
    }

and this gets called too. So the output is

 BeforeTagAScenario
 BeforeTagBScenario
 BeforeScenario

Experimenting with the placement of the [Scope] attribute, it seems it overrides any Tag filtering on the BeforeScenario attribute (whether placed on a class or a method). So a step definition like this:

    [BeforeScenario("tagB")]
    [Scope(Tag = "someOtherTag")]

will run for every "someOtherTag", and ignore the presence of tagB entirely.

[prakashmj59]

Yes even i have observed the issue with specflow 1.9.2.. However working with alternative to overcome the issue..

[prakashmj59]

It seems not only twice but also called as many times as there are super classes..
I tried following scenarios

Scenario 1:
"StepDefinition" inherits "TestBase" which inherits Hooks. Hooks has [BeforeScenario] and [AfterScenario]

So here [BeforeScenario] is called 3 times

Scenario 2:
"StepDefinition" inherits "TestBase" which inherits PageBase which intern inherits Hooks. Hooks has [BeforeScenario] and [AfterScenario]

So here [BeforeScenario] is called 4 times.

So based on no. of hierarchy [BeforeScenario] is called...

[thardy]

I've encountered this bug as well, and it's very bothersome. Does anyone know of a workaround, or is a fix in the works?

[QAVW]

When tried to debug one of many scenarios, below [AfterScenario] method is getting executed as many times as number of all existed scenarios+number of parent classes. I am using .net webdriver and Specflow. Any idea what's wrong?
[AfterScenario]
public void AfterScenario()
{
SignOut(driver);
driver.Manage().Cookies.DeleteAllCookies();
}

[Festivejelly]

I was getting similar behavior after every scenario. I noticed double entries in my results database, so I investigated. I checked the App.config and found I'd accidentally put in:

specFlow>
stepAssemblies>
stepAssembly assembly="MySharedBindings" />
/stepAssemblies>
/specFlow>

Normally this references an external library but I found it was referencing its own library. When I removed it I no longer got multiple entries.

So it seems by referencing the assembly you're working in, it tends to do a double cleanup.

Maybe its a completely separate issue but maybe give it a try :)

edit: I had to remove the "<" from each line as it wouldnt let me post it.

[darrencauthon]

@Festivejelly I think you could use the "xml" and "" wrapper around your XML to get it to post correctly, like this:

<specFlow>
<stepAssemblies>
<stepAssembly assembly="MySharedBindings" />
</stepAssemblies>
</specFlow>

[darrencauthon]

Sorry, what I mean to say is that you can use the backtick, like this:

``````

``````

You can also indent your line by 4 characters, which I used to show you how to do the backtick. :smile: The benefit to using the "backtick-backtick-backtick XML" is that you can define it as a XML block and get all of the proper XML color coding.

[darrencauthon]

But good find! @jamescrowley , are you doing what @Festivejelly noted above?

[bulinckx]

I can confirm it happens with AfterScenario too.

This happens because SpeckFlow ill instantiate all Features classes needed to run all steps for the scenario.

But ill keep the same context for each instance:

FeatureContext.Current.FeatureInfo.Tags
ScenarioContext.Current.ScenarioInfo.Tags

Example:

You got:

Feature_A.feature (with defines Scenario_A)
Feature_A.cs

Feature_B.feature (defines Scenario_B)
Feature_B.cs

In feature A you put Tag_A and in B Tag_B

At feature_A you are reusing a step from feature_B

When you run the Scenario_A SpeckFlows instantiate both Features classes, run Scenario_A and uses a method from B.

When A is done it ill run AfterScenario_A twice because both Instances share the same ScenarioInfo
My guess is the hook method are called at the instance desctructor/constructor and only the context is verified.

I work around by mimic a singleton using the context at the constructor and checking it at my CleanUp methods

Below a example where both my classes inherits from a SuperClass and uses a constructor (with one parameter).

// child Constructor
        internal MyFeatureClass_A(PropertyClass param) : base(param) { }

// superclass Constructor
 internal MyFeatureSuperClass(PropertyClass param)
        {
            MyFeatureSuperClass _instance;

            if (ScenarioContext.Current.TryGetValue<MyFeatureSuperClass>("Singleton", out _instance))
           {
               property = _instance.property;
               return;
           }

            // My superclass got a property and it need to be instatiated using the first tag and some parameter, ignore it if your life seems easier than mine =)
        property = new PropertyClass(FeatureContext.Current.FeatureInfo.Tags[0], param);

        ScenarioContext.Current.Set<MyFeatureSuperClass>(this, "Singleton");
    }


    [AfterScenario("@Tag_A")]
    internal void CleanUp_A()
    {
        MyFeatureSuperClass _instance = ScenarioContext.Current.Get<MyFeatureSuperClass>("Singleton");

        if (_instance.GetType() == this.GetType())
            do stuff
    }

[darrencauthon]

Thank you @bulinckx for the detailed description! Your information makes this an ticket that someone can execute.

_Side note: I believe you can use ScenarioContext.Current.Get<MyFeatureSuperClass> and ScenarioContext.Current.Set(this) if you want to get around using the "Singleton" string._

Next question... who wants to take a stab at it? I'm not qualified, as I don't think I've ever used a tag in the hundreds or thousands of specs I've written. It should be someone familiar with tags, and ideally with someone with a catalog of tests to verify the changes against.

In the interests of all involved, I'll be blunt: This is not only an edge case, but an edge case with an easy workaround. We need someone to raise their hands and take a stab at this problem with a Pull Request or it's not going to be done.

[Elli143]

I'm seeing this issue as well, and it makes it impossible to use tags in BeforeScenario / AfterScenario hooks.

In my case, I have three scenarios, each with its own tag. When I run my scenarios, all three BeforeScenario instances are called regardless of whether or not the tag is applied to that scenario.

[samholder]

Can you put a small example project which show exhibits this behavior onto
GitHub. I'll investigate if there is a reliable repro.
On 30 Jun 2015 21:15, "Elli143" [email protected] wrote:

I'm seeing this issue as well, and it makes it impossible to use tags in
BeforeScenario / AfterScenario hooks.

In my case, I have three scenarios, each with its own tag. When I run my
scenarios, all three BeforeScenario instances are called regardless of
whether or not the tag is applied to that scenario.

—
Reply to this email directly or view it on GitHub
https://github.com/techtalk/SpecFlow/issues/252#issuecomment-117328007.

[Elli143]

Here is my feature file:

@ExportsTests @GridExportTests
Feature: GridExportTests
    Tests export to excel within grid pages of the application

@InitiativeExport
Scenario: Initiative Export - Select all Physicians 
    Given I am on the Initiatives page
    When I select the first initiative
    etc.

@TargetExport
Scenario: Targets Export
    Given I am on the Targets page
    When I filter Employment Status by 'Employed by us'
    etc.

@TargetExportConnections
Scenario: Targets Export - Connections
    Given I am on the Targets page
    And no filters are applied
    etc.

And my BeforeScenario hooks:

        [BeforeScenario("InitiativeExport")]
        public void InitializeInitiativeExport()
        {
            Initialize(ExcelExportConstants.OpportunitiesExportNames.InitiativePhysiciansExport);
        }

        [BeforeScenario("TargetExport")]
        public void InitializeTargetExport()
        {
            Initialize(ExcelExportConstants.OpportunitiesExportNames.NetworkOpportunity);
        }

        [BeforeScenario("TargetExportConnections")]
        public void InitializeConnectionsExport()
        {
            Initialize(ExcelExportConstants.OpportunitiesExportNames.ExploreNetworkOpportunityExport);
        }

When I execute the test, all three BeforeScenario hooks are executed (regardless of tag) for each scenario. This means that the only test that actually passes is the @TargetExportConnections test, because it is the only test that is set up correctly (simply because it is the last BeforeScenario hook to run each time).

[samholder]

OK thanks. I'll create the test project with these files, but it might work
OK. A sample project which has the issue is much better add this will have
all of the necessary nuget packages etc. Can you confirm which version you
are using, both the extension and the nuget package?
On 30 Jun 2015 21:40, "Elli143" [email protected] wrote:

Here is my feature file:

@ExportsTests @GridExportTests
Feature: GridExportTests
Tests export to excel within grid pages of the application

@InitiativeExport
Scenario: Initiative Export - Select all Physicians
Given I am on the Initiatives page
When I select the first initiative
etc.

@TargetExport
Scenario: Targets Export
Given I am on the Targets page
When I filter Employment Status by 'Employed by us'
etc.

@TargetExportConnections
Scenario: Targets Export - Connections
Given I am on the Targets page
And no filters are applied
etc.

And my BeforeScenario hooks:

    [BeforeScenario("InitiativeExport")]
    public void InitializeInitiativeExport()
    {
        Initialize(ExcelExportConstants.OpportunitiesExportNames.InitiativePhysiciansExport);
    }

    [BeforeScenario("TargetExport")]
    public void InitializeTargetExport()
    {
        Initialize(ExcelExportConstants.OpportunitiesExportNames.NetworkOpportunity);
    }

    [BeforeScenario("TargetExportConnections")]
    public void InitializeConnectionsExport()
    {
        Initialize(ExcelExportConstants.OpportunitiesExportNames.ExploreNetworkOpportunityExport);
    }

When I execute the test, all three BeforeScenario hooks are executed
(regardless of tag) for each scenario. This means that the only test that
actually passes is the @TargetExportConnections test, because it is the
only test that is set up correctly (simply because it is the last
BeforeScenario hook to run each time).

—
Reply to this email directly or view it on GitHub
https://github.com/techtalk/SpecFlow/issues/252#issuecomment-117335050.

[Elli143]

I'm using SpecFlow 1.9.0
SpecRun Runner 1.2.0
NUnit 2.6.4
NUnit.Runners 2.6.4
Visual Studio 2013

Now that I look at the comments, the bug I'm describing might belong in a separate ticket. My issue is not that BeforeScenario is executed twice, it's that BeforeScenario is ignoring the tags.

[samholder]

I have tried to recreate the issue using the steps and bindings provided,
but as anticipated everything works ok for me, using Resharpers runner and
NCrunch's runner and SpecRun runner. i do have a later version of the VS
extension, but i don't believe that should matter. I get this output for
all runners:

InitiativeExport
Given I am on the Initiatives page
I am on the Initiatives page
-> done: Steps.GivenIAmOnTheInitiativesPage() (0.0s)
When I select the first initiative
I select the first initiative
-> done: Steps.WhenISelectTheFirstInitiative() (0.0s)

TargetExport
Given I am on the Targets page
I am on the Targets page
-> done: Steps.GivenIAmOnTheTargetsPage() (0.0s)
When I filter Employment Status by 'Employed by us'
I filter Employment Status by '(.*)'
-> done: Steps.WhenIFilterEmploymentStatusBy("Employed by us") (0.0s)

TargetExportConnections
Given I am on the Targets page
I am on the Targets page
-> done: Steps.GivenIAmOnTheTargetsPage() (0.0s)
And no filters are applied
no filters are applied
-> done: Steps.GivenNoFiltersAreApplied() (0.0s)

You can see the project I used to test here. Perhaps you could clone the
repo and test locally to see if you can repro locally? If this project runs
for you then you will need to create a minimal sample the shows the issue
for it to be investigated further.

Cheers!

On Tue, Jun 30, 2015 at 9:51 PM, Elli143 [email protected] wrote:

I'm using SpecFlow 1.9.0
SpecRun Runner 1.2.0
NUnit 2.6.4
NUnit.Runners 2.6.4

Now that I look at the comments, the bug I'm describing might belong in a
separate ticket. My issue is not that BeforeScenario is executed twice,
it's that BeforeScenario is ignoring the tags.

—
Reply to this email directly or view it on GitHub
https://github.com/techtalk/SpecFlow/issues/252#issuecomment-117338550.

[samholder]

@QAVW the cause of your issue is having steps defined in base classes. Put bluntly there is no reason to do this. All steps in specflow are global and so are accessible from anywhere, so putting them in inherited classes makes no sense. if you are doing it to share state in class variables, then I recommend moving that state into a separate class and using specflows context injection support to inject an instance of the state class into each step class. The DI framework specflow uses will insert the same object instance so the state will be shared.

@bulinckx its not as clear cut in your case, but given the language you have used i suspect that having steps or hooks defined in base classes is the cause of your issue as well.

If either of you can put a sample solution which shows this issue which does not use base classes then I will take a look at it when I get some time.

[samholder]

@prakashmj59 your issue is also due to inheritance. See the above post.

[samholder]

@jamescrowley @darrencauthon I've verified that this happens on the v2 branch as well, so I'll try and investigate when I get a bit of time. Thanks for the repro!

[ghost]

Any updates on this? I can certify that this is called with the same instance of the class, so a workaround could be to have some lock mecanishm to ensure is called once.
I'm using:
Specflow 2.0.0.0
Nunit 2.6.4
Specflow for VS 2015.1.2

This is my workaround

[Binding]
public class MyBinding
{
    public static long Number;
    private readonly long _number;

    public MyBinding(...)
    {            

    }

    [BeforeScenario(Order = 1)]
    public void Do()
    {
        if(_number>0) return;
        _number = Interlocked.Increment(ref Number);
        //do some stuff
    }

}

Using Interlocked just in case at some point 2 different instances are created to call the method.

[ghost]

Thanks!

[tadesushilgithub]

I have method under [AfterScenario, Scope(Tag = "ENABLE_NETWORK")] but It's calling method under this [AfterScenario] first. How to call method under this [AfterScenario, Scope(Tag = "ENABLE_NETWORK")] rather than exectuing method under [AfterScenario]? Please help someone

[SabotageAndi]

@tadesushilgithub Please create a new issue with more information about your problem, like used SpecFlow version, used test runner, code from your steps.

[samholder]

@tadesushilgithub if you tag a method with [AfterScenario] it will be executed after EVERY scenario. you can check inside the method to see if a tag exists and skip the contents of the method if so, but you have to do this manually.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.