Specflow: Be able to access MSTest TestContext in BeforeTestRun/AfterTestRun hook

[SabotageAndi]

It is possible to access the MSTest TestContext in the AssemblyInitialize hook of MSTest.

[AssemblyInitialize]
public static void AssemblyInit( TestContext context )
{
    ....
}

Since SpecFlow 3.1 it is not possible anymore to define your own AssemblyInitialize hook, as it is already used by SpecFlow.

Suggestion to fix this:
We have to pass the TestContext through to our BeforeTestRun`AfterTestRun` hooks.

[harvinders]

I am able to write following code. I saw in the generated code behind that TestContext is getting registered, so we can resolve it using constructor injection wherever we want.

    [Binding]
    class Hooks
    {
        private readonly ScenarioContext _scenarioContext;
        private readonly TestContext _testContext;
        private readonly Application _application;

        public Hooks(ScenarioContext scenarioContext, TestContext testContext, Application application)
        {
            _scenarioContext = scenarioContext;
            _testContext = testContext;
            _application = application;
        }
        [AfterStep]
        public void AfterStep()
        {
            if (null == _scenarioContext.TestError) 
                return;

            var screenShot = _application?.Session?.GetScreenshot();
            if (null == screenShot) 
                return;

            var path = Directory.GetCurrentDirectory() + $"{_scenarioContext.ScenarioInfo.Title}.png";

            screenShot.SaveAsFile(path);
            _testContext.AddResultFile(path);
        }
    }

[SabotageAndi]

@harvinders Yes, on a scenario level it is possible to access the TestContext.
Currently it is not possible to access it in the BeforeTestRun/AfterTestRun hooks

[mraming]

I'm in the same situation: We need the MsTest TestContext in the [BeforeTestRun] method now that we cannot use [AssemblyInitialize] anymore. Unfortunately, this means I cannot upgrade to SpecFlow 3.1 until this is somehow resolved.

[mraming]

@SabotageAndi I'd be willing to help implement this, but I need some guidance on what the desired approach would be. One way that's low risk and isolates this is could be to:

• Add a new attribute (e.g. MsTestAssemblyInitialize)
• In the generated MsTest.AssemblyHooks.cs, enhance the AssemblyInitialize method to first look in the current assembly for a static method with the new MsTestAssemblyInitialize attribute. When one exists, call it (passing in the TestContext).

That way, we have a hook that's guaranteed to execute before any of the SpecFlow code runs and the impact of this MsTest specific change to the rest of SpecFlow is zero.

Any thoughts/recommendations?

[lucamnn]

Hi, any plan or news on this issue?
We too need a way to start up our test framework (we use the AssemblyInitialize attribute)
We cannot upgrade to SpecFlow 3.1 or higher until this is somehow addressed.

[SabotageAndi]

@mraming sorry, I didn't saw your post. Are you still interested in helping to implement this?

We already get the TestContext as a parameter in our AssemblyInitialize method (https://github.com/SpecFlowOSS/SpecFlow/blob/master/Plugins/TechTalk.SpecFlow.MSTest.Generator.SpecFlowPlugin/build/MSTest.AssemblyHooks.template.cs#L13).

From there we need to pass the instance down the callstack until we can register it into the global container (https://github.com/SpecFlowOSS/SpecFlow/wiki/Available-Containers-&-Registrations#global-container). The challenge is, that we can't use the real type, because we don't want a dependency on MSTest in the Core Runtime.

And when it is in this container, you can simply get to this container with:

[BeforeTestRun]
public static void TestWithContainer(ObjectContainer testThreadContainer)
{
    var testContext = testThreadContainer.Resolve<TestContext>();
    //...
}

TBH I wasn't aware that you can get so to the TestThreadContainer in the BeforeTestRun hook.

[mraming]

@SabotageAndi Willing to help! Thus far, I've only been using SpecFlow and am new to developing SpecFlow itself so I have a steep learning curve ahead. But I'll give it a shot and let you know.

Thanks for the pointers thus far.

[godrose]

Shouldn’t injecting the container into the hook “just work”? Instead of
service location?

On Thu, 14 May 2020 at 14:37 Andreas Willich [e-mail]
wrote:

@mraming https://github.com/mraming sorry, I didn't saw your post. Are
you still interested in helping to implement this?

We already get the TestContext as a parameter in our AssemblyInitialize
method (
https://github.com/SpecFlowOSS/SpecFlow/blob/master/Plugins/TechTalk.SpecFlow.MSTest.Generator.SpecFlowPlugin/build/MSTest.AssemblyHooks.template.cs#L13
).

From there we need to pass the instance down the callstack until we can
register it into the global container (
https://github.com/SpecFlowOSS/SpecFlow/wiki/Available-Containers-&-Registrations#global-container).
The challenge is, that we can't use the real type, because we don't want a
dependency on MSTest in the Core Runtime.

And when it is in this container, you can simply get to this container
with:

[BeforeTestRun]public static void TestWithContainer(ObjectContainer testThreadContainer)
{
var testContext = testThreadContainer.Resolve();
//...
}
TBH I wasn't aware that you can get so to the TestThreadContainer in the BeforeTestRun hook.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SpecFlowOSS/SpecFlow/issues/1859#issuecomment-628575629,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB3OII6RQXW6NRCI3UQNPJDRRPJZBANCNFSM4KKBDN5A
.

>

Best Regards,
Gennady Verdel

[SabotageAndi]

Shouldn’t injecting the container into the hook “just work”? Instead of service location?
Which container do you mean @godrose? Sorry, I don't understand your comment.

[godrose]

Shouldn’t injecting the container into the hook “just work”? Instead of service location?
Which container do you mean @godrose? Sorry, I don't understand your comment.

@SabotageAndi

Suppose you have a hook like this:

internal sealed class SomeHook
{
     //...

[BeforeTestRun]
public static void TestWithContainer(ObjectContainer testThreadContainer)
{
    var testContext = testThreadContainer.Resolve<TestContext>();
    //...
}

     //...
}

You can simply write this instead:

internal sealed class SomeHook
{
     //...
     private readonly ObjectContainer _objectContainer;
     public SomeHook(ObjectContainer objectContainer) 
{
    _objectContainer = objectContainer;
}

[BeforeTestRun]
public static void TestWithContainer(ObjectContainer testThreadContainer)
{
    var testContext = _objectContainer.Resolve<TestContext>();
    //...
}

If I understand correctly this is merely question of constructor injection vs. method injection
https://softwareengineering.stackexchange.com/questions/142769/should-injecting-dependencies-be-done-in-the-ctor-or-per-method

Moreover, why exactly the TestContext itself is not available for constructor injection into SomeHook? I might be missing something.

[SabotageAndi]

We are talking about the BeforeTestRun/AfterTestRun hooks which are static methods and so no constructor is called. You can only get them via parameter.

[godrose]

How exactly is this hook used then? I might be missing the flow.
I inject the ObjectContainer into the hook and the hook gets resolved during the test run along with its dependencies. An example can be found here:
https://github.com/LogoFX/cli-dotnet/blob/master/specs/LogoFX.Cli.Dotnet.Specs.Steps/LifecycleHook.cs

So why is there no constructor call in this case?

[SabotageAndi]

@godrose for the AfterTestRun you are correct. There you can do something like the linked code. You set a static field in the BeforeScenario hook and then you can access it in the AfterTestRun.

But this approach is not possible in the BeforeTestRun hook.

[lucamnn]

@SabotageAndi i just was thinking if could be possible avoid to use the AssemblyInitializeAttribute at all.
This is a library and it could be used along other test libraries and frameworks (like in our case).

In my opinion you should provide a static initialization method and leave to test implementers the responsibility to call it in a local-to-project method decorated with AssemblyInitializeAttribute. It's just a matter of a clear documentation about it.

In a very simple manner, this could allow the possibility to combine the use of any test library.

[SabotageAndi]

@lucamnn Interesting suggestions.

I could think about having a switch, that you can define the [AssemblyInitializeAttribute] on your own. But this would always be opt-in, as this behavior change would break a lot of our users. Calling the code is not optional. We could probably make some checks that this is called and give some feedback to the user.

The flag should then also apply to the [AssemblyCleanupAttribute]. And here I have no idea how to make the same check and give the user feedback that she/he has to add some code that the code is called. Again this code is not optional.

[marcoamendola]

@SabotageAndi I also encountered the very issue @lucamnn had.
I still couldn't figure out any clean solution to launch some assembly-level initialization and cleanup code (required by my tests) along SpecFlow's one.
I understand your concerns about ensuring that startup and cleanup code are properly called, yet I think that having an explicit opt-in switch would be enough in order to leave that responsibility to the user.

[SabotageAndi]

@marcoamendola I am happy to review a PR that is adding this opt-in flag.

[marcoamendola]

@SabotageAndi, I've been digging in the source code, and it appears that such a flag (named GenerateSpecFlowAssemblyHooksFile) is already available.
It seems to exactly address the issue, since setting it to false in the project file turns off the generation of assembly hooks; obviously you have to call TestRunnerManager.OnTestRunStart and TestRunnerManager.OnTestRunEnd by yourself.
Am I missing something? Is this the intended use of this flag?

[mraming]

@SabotageAndi, @marcoamendola Indeed that looks to be a way to work around the issue. Turns out, #1787 mentions this as a workaround. I've just tested this in my project and it works lovely. This is a workable solution for me and I'm now running with the latest SpecFlow version :-)

The issue with this approach is that if the generated AssemblyInitialize methods would change, then projects omitting this would start to fail in subtle ways without noticing this. One solution could be to have a documented, public static method that would get called from the AssemblyInitialize that does the work.

[SabotageAndi]

Oh, I forgot about this property.
You could use this as a workaround. The code we have there is already the public static method you are requesting. We don't have any plans at the moment that will make a change in this API needed.

[marcoamendola]

Then I think that the issue regarding generated assembly hooks is solved.
Thanks @SabotageAndi for your work and your support and thanks @mraming for pointing out the original issue.

[SabotageAndi]

I still would like to see the TestContext is accessible in the BeforeTestRun/AfterTestRun, so I am not closing this issue.
The property is still only a workaround for me.

[mraming]

I’ll give it a bash to see what I can come up with 😊

Met vriendelijke groeten,
Best regards,

Mark Raming

Mark.[email protected][e-mail]
[phone]
[phone]

[cid:[email protected]]

Eraze Computer Software BV
Rietschotten 1 – 4751 XN Oud Gastel, The Netherlands
www.eraze.comhttp://www.eraze.com/ – KvK 27164810

From: Andreas Willich [e-mail]
Sent: Tuesday, May 26, 2020 15:16
To: SpecFlowOSS/SpecFlow [e-mail]
Cc: Mark Raming [e-mail]; Mention [e-mail]
Subject: Re: [SpecFlowOSS/SpecFlow] Be able to access MSTest TestContext in BeforeTestRun/AfterTestRun hook (#1859)

I still would like to see the TestContext is accessible in the BeforeTestRun/AfterTestRun, so I am not closing this issue.
The property is still only a workaround for me.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/SpecFlowOSS/SpecFlow/issues/1859#issuecomment-634017459, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADX2H5GC4ZHDWIAEJXCUAM3RTO6HLANCNFSM4KKBDN5A.

[SabotageAndi]

To make it easier for you all to vote on feature requests and influence our prioritization of them in our backlog, we created in our new support tool a special forum to have a better overview of all of them. We moved this feature request there and you can find it here. Sadly we could not take over the existing votes. So if you are still interested in this feature request, please vote again on the new list.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.