Salesforcedx-vscode: Run All Tests fails in namespaced scratch org

[ajw725]

Summary

I'm developing on a managed package in a namespaced scratch org. When I try to run all of the Apex tests in a particular test class from the IDE with the "Run All Tests" link that appears at the top, it fails, saying "Unable to invoke any tests...This class name's value is invalid." Running a single test with the "Run Test" link above that specific method works fine. Running all the tests by doing "Invoke Apex Tests" from the command menu also works fine.

The problem appears to be that the namespace is prepended unnecessarily. If my namespace is "Test," and my class is called "TestClass," running all of the tests with the "Run All Tests" link produces a command like:
sfdx force:apex:test:run --tests Test.TestClass
When it fails, the error message says "This class name's value is invalid: Test.Test," which indicates that the namespace is being duplicated. If I modify the command to remove the namespace, as below, it works correctly:
sfdx force:apex:test:run --tests TestClass
The "Invoke Apex Tests" menu item appears to use a different flag, which produces this (also correct) command:
sfdx force:apex:test:run --classnames TestClass

Steps To Reproduce:

1. Create a scratch org with a namespace
2. Create an Apex test class with some tests
3. Click the "Run All Tests" link that should appear in the editor above the class definition

Expected result

All tests in the class should run

Actual result

No tests run. An error message is returned about a missing class / invalid class name.

VS Code Version: 1.34.0

SFDX CLI Version: sfdx-cli/7.6.0-28071a499b darwin-x64 node-v10.15.3

OS and version: macOS Mojave 10.14.5

[NightBijou]

@ajw725 Thanks Andrew for reporting this so succinctly. I am having the exact same problem: I am working in a namespaced scratch org and Run Test or Run Single Test works from VS Code whereas Run Tests or Run All Tests doesn't, failing with the error message you described: "This class name's value is invalid". I can confirm that it works from the command line when you remove the namespace from a class name. But this workaround does not work in Visual Studio Code, you can't change the way a command is invoked as far as I can tell.

VS Code Version: 1.39.2

SFDX CLI Version: sfdx-cli/7.28.7-7bceba6f24 win32-x64 node-v10.15.3

OS and version: Windows 10 Enterprise version 1809

This is a major nuisance because it effectively reduces the Visual Studio Code integration for test automation to running individual tests only.

[lcampos]

@NightBijou or @ajw725 are you adding the namespace to the configuration in the sfdx-project.json file ?

I'm currently running tests against a scratch org with a namespace and can't replicate this issue. I created the scratch org using the second generation packaging flow (DevHub linked to the Developer org that has a namespace). Did you create your scratch org that way ?

[NightBijou]

@lcampos Yes, my project contains the namespace. My sfdx-project.json file looks like this:

{
  "packageDirectories": [
    {
      "path": "force-app",
      "default": true
    }
  ],
  "namespace": "OurCoolNamespace",
  "sfdcLoginUrl": "https://login.salesforce.com",
  "sourceApiVersion": "45.0"
}

And here's my project-scratch-def.json file, in case it matters:

{
    "orgName": "MyCoolScratchOrgName",
    "edition": "Developer",
    "hasSampleData": true,
    "settings": {
        "orgPreferenceSettings": {
            "s1DesktopEnabled": true
        }
    }
}

Our DevHub is indeed linked to another org that has that namespace. Otherwise it would not be possible to push the project artifacts to a scratch org created by the DevHub. The product is already released and available on AppExchange, so our overall development process works.

As @ajw725 pointed out, running individual tests works. It's running multiple tests that doesn't. Something is wrong, but it beats me what it is.

[ajw725]

@lcampos yes to both. I have the namespace in the definition file, and I created the scratch org from a DevHub linked to the namespace. As @NightBijou said, I don't think we'd even be able to develop within the namespace if that weren't the case.

My config files look like the ones @NightBijou posted (plus more features/settings). It's creating a developer-edition org with the package namespace, and the source API version is 45.0.

[NightBijou]

I am still trying to figure out this problem, but no dice. I completely uninstalled and reinstalled VS Code, I tested with a different DevHub, also with a linked namespace org, I even reinstalled Salesforce CLI, same problem. Here's the full error log I get in VS Code when executing Run All Tests (except for changed names):

14:52:28.174 sfdx force:apex:test:run --tests MyNamespace.MyClass01Test --resultformat human --outputdir c:\Temp\.sfdx\tools\testresults\apex --loglevel error
ERROR running force:apex:test:run:  Unable to invoke any tests. Ensure the tests are loaded into the org or valid inputs are supplied. This class name's value is invalid: MyNamespace.MyNamespace. Provide the name of an Apex class that has test methods.
14:52:33.762 sfdx force:apex:test:run --tests MyNamespace.MyClass01Test --resultformat human --outputdir c:\Temp\.sfdx\tools\testresults\apex --loglevel error ended with exit code 1

The command is invoked as expected, adding the namespace before the name of the test class. But in the error message, you can see that internally it is trying to invoke MyNamespace.MyNamespace instead of MyNamespace.MyClass01Test.

This does not seem like a problem of the Visual Studio Code extension, because the error also occurs on the command line. Something fishy is going on in the CLI.

[ajw725]

@NightBijou that's a good point. It seems to me like the test run command is expecting a format like ClassName.testMethodName rather than NS.ClassName. To extend your example:

sfdx force:apex:test:run --tests MyClass01Test.myTestMethod --resultformat human --outputdir c:\Temp\.sfdx\tools\testresults\apex --loglevel error

correctly executes the single test called "myTestMethod." And running it like this:

sfdx force:apex:test:run --tests MyClass01Test --resultformat human --outputdir c:\Temp\.sfdx\tools\testresults\apex --loglevel error

correctly executes all tests in the class. So it seems like when you prepend the namespace to the class name, it gets incorrectly interpreted as a class name + test method name instead of namespace + class name.

[ajw725]

And FWIW, the same issue occurs whether you try to run the tests from an org developing within the namespace or from an org in which the package containing the test class has been installed.

I can open an issue over in the CLI repo to check with them about it.

[lcampos]

@ajw725 @NightBijou thanks for all the feedback and details. We were able to replicate this issue last night and we're currently working on a fix. The issue is in the CLI command (force:apex:test:run) and caused by a bug in the way we evaluate the namespace. This is why it sometimes removes or duplicates the namespace as part of the fullname of a class. We'll let you know once the fix is checked in and ready to be published.
cc @brpowell @ntotten

[NightBijou]

@lcampos @ajw725 Hi guys, that sounds great, I can't wait to see this fixed. Thanks for being so active on this issue.

[lcampos]

The fix for this was released as part of the Salesforce CLI version 7.31.0. Thanks for reporting it and helping us narrow it down.

[ajw725]

sweet - looks good to me now. thanks guys!

[NightBijou]

Very nice! I can confirm that the problem is resolved as far as I am concerned. Big thanks to everyone who contributed to fixing this issue!

[andrey-tsvor-vrp]

Hello @lcampos
Seems I faced the same problem in the following version of cli: sfdx-cli 7.84.2 (core)
Is it possible to fix it again?

[NightBijou]

Hello @lcampos, same over here. I found a workaround though, maybe you can try this too: Go to the Salesforce settings in VS Code and set the option salesforcedx-vscode-core.experimental.useApexLibrary to false.

Maybe this can tide you over until this is fixed.

[andrey-tsvor-vrp]

Hi @NightBijou
It helped me. Thanks!

[rajarak12]

Faced the same issue, workaround helped for now

[edralph]

Me three. Workaround fixed it for me.