Specflow: "Run SpecFlow Scenarios" does not work with ReSharper 9

[gasparnagy]

"Run SpecFlow Scenarios" does not work with ReSharper 9.

See https://groups.google.com/forum/#!topic/specflow/fmxx8nZPxx8 for details

[gasparnagy]

I've updated the commands (see commit https://github.com/techtalk/SpecFlow.VisualStudio/commit/e2508905eb1096d534d222488e8725f45f70f7f5), but it did not help...

It shows an error like

[10:26:08.1571693] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.Reharper_ReSharperUnitTestRunContext" is not valid.
   at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
   at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

I have no further idea. I fear that this will not work (it was a hack anyway)...

If anyone has a good idea, go ahead and try to fix it.

[davidmcgregor]

I believe your commit has a typo -
"ReSharper.Reharper_ReSharperUnitTestRunContext"
should be
"ReSharper.ReSharper_ReSharperUnitTestRunContext"

Hope it helps.

[lindavz28]

Not sure if this is related or not, but this command doesn't work with Resharper 8.2 either. It's not Resharper 9 specific. I had a working system, upgraded to latest "Specflow for Visual Studio 2013" v2015.1.1 and started getting the "No tests found in file" error.

Specflow trace:
[11:37:29.1871281] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.Reharper_ReSharperUnitTestDebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)
[11:37:29.3071401] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.Reharper_ReSharperUnitTestDebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

[lindavz28]

OK, starting to understand the problem here.
Resharper6+ supports this command:

Resharper9 has changed to ReSharper.Resharper_ReSharperUnitTestDebugContext

From what I can see, you can't just change the Resharper6 class - that will break all current builds using Resharper 8.2 - which is what I've seen today. It's trying to run ReSharper.Reharper_ReSharperUnitTestDebugContext

(Edited) Better idea - you can look up the version of Resharper being used, and adjust the command accordingly.

I'm thinking something like this...
I have made the change and will do a PR if I can test it properly (First time working with VS extensions)

public int GetResharperVersion()
{
            var reSharperAssembly = AppDomain.CurrentDomain.GetAssemblies().FirstOrDefault(a => a.GetName().Name == "JetBrains.ReSharper.UnitTestFramework");
            return reSharperAssembly != null ? reSharperAssembly.GetName().Version.Major : -1;
}

protected override string GetRunInCurrentContextCommand(bool debug)
{
            // This should be stored in constructor so we don't run it all the time
            int currentVersion = GetResharperVersion();
            string commandFormat;
            if (currentVersion < 6)
            {
                commandFormat = "ReSharper.ReSharper_UnitTest_Context{0}";
            }
            else if (currentVersion < 9)
            {
                commandFormat = "ReSharper.ReSharper_ReSharper_UnitTest_{0}Context";
            }
            else
            {
                commandFormat = "ReSharper.ReSharper_ReSharperUnitTest{0}Context";
            }

            if (debug)
            {
                return string.Format(commandFormat, "Debug");
            }
            return string.Format(commandFormat, "Run");

}

[gasparnagy]

@lindavz28 yes. please try to do it.

[lindavz28]

@gasparnagy https://github.com/techtalk/SpecFlow.VisualStudio/pull/4
Only tested VS2013 + Resharper 8.2 to check that's working again.

(Edit) - Figured it out, need to set experimental hive option in RS9 installer. Will test tomorrow morning

[gasparnagy]

@lindavz28 super, thx

[lindavz28]

@gasparnagy Made my life a bit harder by trying to get XUnit working on RS9, but successfully ran specflow tests with RS9! :+1: :smile:

[gasparnagy]

@lindavz28 super! thx. I merged it.
Let's wait for a week, if there is any issue pops up with 2015.1 and if not, I'll update the VS gallery.
I close this issue now.

[lindavz28]

@gasparnagy When will you be updating the VS gallery?

[gasparnagy]

I have updated the VS gallery with the new version now.

[stickleprojects]

HI
I've uninstalled and updated to the v2015.1.2 and experiencing the same issue in vs2013/r#9.
Is there further uninstall steps I need to follow?

[stickleprojects]

Hi
Same problem, vs2010/r#9 (platform v6.0.2):

[11:23:47.6127725] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.ReSharper_ReSharper_UnitTest_DebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.Vs2010Integration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

resharper debug information, in case it is helpful:

JetBrains ReSharper Platform 6 Build 6.0.20141219.120158 on 2014-12-19 12:56:02Z
JetBrains ReSharper 9.0 Update 1 Build 9.0.20141219.130142 on 2014-12-19 13:12:31Z
JetBrains TeamCity Add-in 9.0 Update 1 Build 9.0.20141219.130134 on 2014-12-19 13:13:21Z

Application Packages:
JetBrains Fake VisualStudio VsPackageAssembly build 10.0.0.0 on 2015-03-19 13:38:37Z.
JetBrains Platform Core Ide build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform Core Shell build 6.0.20141219.120162 on 2014-12-19 12:53:09Z.
JetBrains Platform Core Text build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform Installer build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform Symbols build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform VisualStudio build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Psi.Features ExternalAnnotations build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features SolutionBuilder build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features src build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features Tasks build 6.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features TeamCity src build 9.0.20141219.130134 on 2014-12-19 13:13:21Z.
JetBrains Psi.Features VisualStudio build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains ReSharper src build 9.0.20141219.130142 on 2014-12-19 13:12:31Z.
Visual Studio 10.0.30319.1.

Copyright © 2015 JetBrains. All rights reserved.

[samholder]

hit reply on the wrong email @shaundodimead? :laughing:

[shaundodimead]

Yup :) fml - there's a github feature I didn't know existed...

[lindavz28]

The plugin being used is:TechTalk.SpecFlow.Vs2010Integration

My fixes were in the newer SpecFlow.VisualStudio plugin.

I'm guessing this is not backwards compatible with 2010/2012... I'll need
to do the same fix on this as well then....

On Wed, May 27, 2015 at 10:26 PM, stickleprojects [email protected]
wrote:

Hi
Same problem, vs2010/r#9 (platform v6.0.2):

[11:23:47.6127725] ReSharper6TestRunnerGateway: test tool error:
System.Runtime.InteropServices.COMException (0x80004005): Command
"ReSharper.ReSharper_ReSharper_UnitTest_DebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at
TechTalk.SpecFlow.Vs2010Integration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean
debug)

resharper debug information, in case it is helpful:

JetBrains ReSharper Platform 6 Build 6.0.20141219.120158 on 2014-12-19
12:56:02Z
JetBrains ReSharper 9.0 Update 1 Build 9.0.20141219.130142 on 2014-12-19
13:12:31Z
JetBrains TeamCity Add-in 9.0 Update 1 Build 9.0.20141219.130134 on
2014-12-19 13:13:21Z

Application Packages:
JetBrains Fake VisualStudio VsPackageAssembly build 10.0.0.0 on 2015-03-19
13:38:37Z.
JetBrains Platform Core Ide build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform Core Shell build 6.0.20141219.120162 on 2014-12-19
12:53:09Z.
JetBrains Platform Core Text build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform Installer build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform Symbols build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform VisualStudio build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Psi.Features ExternalAnnotations build 9.0.20141219.130140 on
2014-12-19 13:00:40Z.
JetBrains Psi.Features SolutionBuilder build 9.0.20141219.130140 on
2014-12-19 13:00:40Z.
JetBrains Psi.Features src build 9.0.20141219.130140 on 2014-12-19
13:00:40Z.
JetBrains Psi.Features Tasks build 6.0.20141219.130140 on 2014-12-19
13:00:40Z.
JetBrains Psi.Features TeamCity src build 9.0.20141219.130134 on
2014-12-19 13:13:21Z.
JetBrains Psi.Features VisualStudio build 9.0.20141219.130140 on
2014-12-19 13:00:40Z.
JetBrains ReSharper src build 9.0.20141219.130142 on 2014-12-19 13:12:31Z.
Visual Studio 10.0.30319.1.

Copyright © 2015 JetBrains. All rights reserved.

—
Reply to this email directly or view it on GitHub
https://github.com/techtalk/SpecFlow/issues/435#issuecomment-105855112.

[gasparnagy]

@lindavz28 yes, the update is made for VS2013 and VS2015. but @stickleprojects mentioned also VS2013. Now I'm confused.
Is it working now in VS2013 latest released version on VS gallery or not? And if not: was the fix not sufficient or I did something wrong with the release?

(I can make an updated release until Wed, but then I'm off for two weeks.)

[stickleprojects]

Further investigation, I noticed that Resharper did not have Keyboard commands similar to "unit", "test", "debug" mentioned above. Reinstall of latest version of R#9 fixed the issue in VS2013. Tested using latest Github source and Gallery install.

[lpanger]

FYI, it works for me.

• specflow plugin 2015.1.2
• resharper version 9.1 (not 9.1.1)
• visual studio 12.0.31101.00 update 4

[MattiLehtinen]

I get slightly different error:

ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.ReSharper_UnitTestRunContext" is not available.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

• SpecFlow for Visual Studio 2015 2015.1.2
• VS 2015 14.0.23107.0
• ReSharper 9.2

Tried to reinstall specflow and resharper with no success. Any tips?

[lindavz28]

@MattiLehtinen Go to Tools > Options > Keyboard bindings (Might be under Editor section).
Search for Resharper and then UnitTestRunContext (or something like that)
Have a look at what the command is showing - is it ReSharper.ReSharper_UnitTestRunContext or something else? What are you seeing instead?

Unless Resharper has gone and changed the command again for Resharper 9.2, it should work....

Final thing you can try is uninstall Resharper and install the specific version 9.1.3:
http://resharper-support.jetbrains.com/hc/en-us/articles/206104048-Where-can-I-download-an-old-previous-ReSharper-version-

[MattiLehtinen]

@lindavz28 The command seems to be there:

I get the same error with VS2013 too.

[MattiLehtinen]

@lindavz28 I installed Resharper 9.1.3 to VS2013 and it works. So I guess there's some breaking change on ReSharper 9.2.

[lindavz28]

@MattiLehtinen Glad you managed to get it working! I suspect a breaking change in 9.2 as well, though it's interesting that the command looks identical to what's in the options list. It might also be a bug on the Specflow side... I'll have a look at 9.2 when I get a chance.

[dbgrvorgau]

Hi any news on getting resharper 9.2 running with visual studio 2010 and specflow?

[Gonnagle]

Had the same issue as @MattiLehtinen with VS 2013, ReSharper 9.2 and SpecFlow 2015.1.2. Fixed for now by reverting back to ReSharper 9.1.3.

[sergiorykov]

The same issue in VS 2015, ReSharper (9.2 and 10) and SpecFlow 2015.1.2

Ctrl-T,R/D works from *.feature-file as a pretty useful workaround, but I occasionally find myself right-clicking and searching menu item :)

[rubig]

Hi guys,

Visual Studio version does not matter. What matters is the Resharper version.

Resharper 9.2+ not working with Specflow 2015. The reason is:

Specflow uses Resharper 9.1.2's key for running unit test:
ReSharper.ReSharper_ReSharper_UnitTest_RunContext

While on ReSharper 9.2+ the key has changed into
ReSharper.ReSharper_UnitTest_RunContext

I'll submit a pull request to get Specflow updated with Resharper 9.2+.

The code change will be something like:

Note 2 difference my code vs lindavz28's code

• ReSharper version that divides how unit test run command works is 9.2, not 9.
• ReSharper 9.2+ command for running unit test is ReSharper.ReSharper_UnitTest{0}Context, not ReSharper.ReSharper_ReSharperUnitTest{0}Context

```
protected override string GetRunInCurrentContextCommand(bool debug)
{
// This should be stored in constructor so we don't run it all the time
int currentVersion = GetResharperVersion();
string commandFormat;
if (currentVersion < 6)
{
commandFormat = "ReSharper.ReSharper_UnitTest_Context{0}";
}
else if (currentVersion < 9.2)
{
commandFormat = "ReSharper.ReSharper_ReSharper_UnitTest_{0}Context";
}
else
{
commandFormat = "ReSharper.ReSharper_UnitTest{0}Context";
}

  if (debug)
  {
      return string.Format(commandFormat, "Debug");
  }
  return string.Format(commandFormat, "Run");

}
```

Cheers,
Ming

[lindavz28]

How did that update go @rubig, did you test your update using Resharper 9.1 and 9.2?

I don't follow your logic though - my code as follows - any version that is 9 or more should use the command you've mentioned. Doesn't matter if it's 9.1, 9.2 or 9.3

protected override string GetRunInCurrentContextCommand(bool debug)
        {
            string commandFormat;
            if (_currentVersion < 6)
            {
                commandFormat = "ReSharper.ReSharper_UnitTest_Context{0}";
            }
            else if (_currentVersion < 9)
            {
                commandFormat = "ReSharper.ReSharper_ReSharper_UnitTest_{0}Context";
            }
            else
            {
                commandFormat = "ReSharper.ReSharper_UnitTest{0}Context";
            }

            if (debug)
            {
                return string.Format(commandFormat, "Debug");
            }
            return string.Format(commandFormat, "Run");

        }

Resharper 9 changed to use the following:
ReSharper.ReSharper_UnitTestDebugContext
ReSharper.ReSharper_UnitTestRunContext

That should be the case for 9.1, 9.2 and 9.3. There's no difference between 9.1 and 9.2 - if there was I would've implemented a fix immediately. That's the problem :)

Comparing 9.2:

With 9.1.3:

So I have 9.1.3 installed in my main VS 2015 (not the hive) and the command is identical.
Have you been able to debug the extension and actually follow through the logic to see what command is being used?

[lindavz28]

@rubig I would love to get this fixed btw, so if you have got something working please send that PR through :+1:

Latest discussion here - please continue in this thread rather than the closed issue
https://github.com/techtalk/SpecFlow/issues/477

I have the stack trace and the command being run is "ReSharper.ReSharper_UnitTestRunContext" It works if you run it in a command window, but not from the Specflow extension. :sob:

[gasparnagy]

@rubig @lindavz28 yes please. don't forget that the PR should be sent to the https://github.com/techtalk/specflow.visualstudio repo.

[dorothyvaliga]

Upvoting this please :+1: -- it degrades the user experience for SpecFlow! :sob:

[gasparnagy]

We release SpecFlow v2 tomorrow and the next step will be to make an update on the VS integration.

[lazytesting]

I agree with @dorothyvaliga. I recently upgraded tot VS2015 en ReSharper 10 and although there are workarounds available it's very frustrating that it's not possible to run the features from the context menu.

[dorothyvaliga]

Working now with VS 15 Specflow 2 Resharper 9.1.3

[darrencauthon]

Thank you @dorothyvaliga ! :)

How about you @lazytesting ?

[lazytesting]

It's now working partially for me with V2 (in combination with NUnit 3):

When I open the feature file and right click in the editor window and select run/debug still nothing happens (no errors either).

When I run them from the solution explorer (with right click on the feature file) the tests are working.
But when I don't have the feature file opened in VS I also get an error popup "Object reference not set to an instance of an object." but the tests are still running.

[lindavz28]

@darrencauthon @dorothyvaliga @lazytesting
Has been working for a while with 9.1.3 in V1. Something changed in version 9.2 of Resharper to make it break, but I thought it was only xunit affected. Looks like it's NUnit as well which means a deeper issue than just the command being run by Specflow.

See this issue which is the most current thread:
https://github.com/techtalk/SpecFlow/issues/477

@gavinlees said:
_@lindavz28 The issue showed up in when Jetbrains fixed this bug in R# 9.3 EAP build #3:
https://youtrack.jetbrains.com/issue/RSRP-446021_

[rubig]

Nice! I’ll give it a try on Monday.

[rubig]

@lindavz28
I am sorry for my late reply. I dived into Specflow source code, and I found Specflow 1.9.3 already contains the same fix I was going to add.

However, I can confirm this version of Specflow still has issue in running tests with Resharper 9.2.

This leads me to think my solution above is not the correct fix for this issue. I'll give Specflow 2 a try and see if it can run nicely with Resharper 9.2.

[crmckenzie]

I'm using Specflow 2 with Resharper 10 in VS 2015. Any word on how to make this work?

[IcodeNet]

Hi all - I am using
JetBrains ReSharper Ultimate 10.0.2 Build 104.0.20151218.120627
ReSharper 10.0.20151218.130009

and SpecFLow 2015.1.2 - 2015/05/26

Any ideas about how to make this work for my pair? Thanks in advance

[SabotageAndi]

@IcodeNet Keyboard shortcuts could work.
Have a look also at #477

[IcodeNet]

Hi guys I have managed to resolve it in my case by selecting in Specflow options menu Resharper as a test runner tool. It was Auto to start off .

hope it works for you as well

[lazytesting]

@IcodeNet for me that didn't work. But it helped me finding a better workaround. I set the Test Runner Tool to VisualStudio2012 and installed the Nunit VS plugin. So I'm now able to run the test from the context menus. But as they don't run in ReSharper I don't have the ReSharper test result view.

[folke123]

Does not work for Reshaper 2016.2.2

Pressing "run unit tests" in the context menu gives the error message "The operation could not be completed. Not implemented"

The same error if you do it by keyboard shortcut.

I am running VS2015 and the "Specflow for Visual Studio 2015 plugin"
and we are using Nunit 2.6.3 as a nuget for the solution

[SamJongenelen]

Same here, does not work. Set Test Runner Tool to Resharper doesn't work.
Setting to VS 2012 works.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.