Vscode-java-test: Java Test Runner fails to discover JUnit5 tests when invoked from method level

[sceee]

I have multiple JUnit4 tests already in a project and now added a new class with a JUnit5 test.

Unfortunately, since then I can no longer execute or debug tests from the method level. It just does not execute or jump in the debugger.

The log in the Java Test Runner log is

WARNUNG: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.util.PreconditionViolationException: Could not load class with name: my.package.StorageServiceTest
    at org.junit.platform.engine.discovery.MethodSelector.lambda$lazyLoadJavaClass$0(MethodSelector.java:157)
    at org.junit.platform.commons.function.Try$Failure.getOrThrow(Try.java:335)
    at org.junit.platform.engine.discovery.MethodSelector.lazyLoadJavaClass(MethodSelector.java:156)
    at org.junit.platform.engine.discovery.MethodSelector.getJavaClass(MethodSelector.java:135)
    at org.junit.vintage.engine.discovery.MethodSelectorResolver.resolve(MethodSelectorResolver.java:34)
    at org.junit.vintage.engine.discovery.MethodSelectorResolver.lambda$resolve$0(MethodSelectorResolver.java:30)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1378)
    at org.junit.vintage.engine.discovery.MethodSelectorResolver.resolve(MethodSelectorResolver.java:30)
    at org.junit.vintage.engine.discovery.VintageDiscoverer.collectTestClasses(VintageDiscoverer.java:64)
    at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:51)
    at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:62)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:168)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:155)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
    at com.microsoft.java.test.runner.junit5.CustomizedConsoleTestExecutor.lambda$executeTests$0(CustomizedConsoleTestExecutor.java:42)
    at com.microsoft.java.test.runner.junit5.CustomContextClassLoaderExecutor.invoke(CustomContextClassLoaderExecutor.java:29)
    at com.microsoft.java.test.runner.junit5.CustomizedConsoleTestExecutor.executeTests(CustomizedConsoleTestExecutor.java:38)
    at com.microsoft.java.test.runner.junit5.CustomizedConsoleLauncher.execute(CustomizedConsoleLauncher.java:29)
    at com.microsoft.java.test.runner.Launcher.main(Launcher.java:53)
Caused by: java.lang.ClassNotFoundException: my.package.StorageServiceTest
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
    at org.junit.platform.commons.util.ReflectionUtils.lambda$tryToLoadClass$9(ReflectionUtils.java:732)
    at org.junit.platform.commons.function.Try.lambda$call$0(Try.java:57)
    at org.junit.platform.commons.function.Try.of(Try.java:93)
    at org.junit.platform.commons.function.Try.call(Try.java:57)
    at org.junit.platform.commons.util.ReflectionUtils.tryToLoadClass(ReflectionUtils.java:695)
    at org.junit.platform.commons.util.ReflectionUtils.tryToLoadClass(ReflectionUtils.java:651)
    ... 17 more

I'm using the latest vscode with the latest Java Test Runner 0.14.1.

I have the following gradle dependencies in my build.gradle:

    testCompile group: 'org.junit.jupiter', name: 'junit-jupiter-api', version: '5.4.0'
    testCompile group: 'org.junit.jupiter', name: 'junit-jupiter-engine', version: '5.4.0'
    testCompile group: 'org.junit.vintage', name: 'junit-vintage-engine', version: '5.4.0'

Also, I have the test section in this file configured like the following:

test {
    // Enable JUnit 5 (Gradle 4.6+).
    useJUnitPlatform()

    // Always run tests, even when nothing changed.
    dependsOn 'cleanTest'

    // Show test results.
    testLogging {
        events "passed", "skipped", "failed"
    }
}

[jdneo]

@sceee Is it possible for you to share a sample project which can repro this issue?

[sceee]

@jdneo I created a sample project here: https://github.com/sceee/demo-vscode-junit5-debugger

When trying to debug it using Java Test Runner, I get the following output:

M�rz 10, 2019 11:49:46 VORM. org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNUNG: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.util.PreconditionViolationException: Could not load class with name: demo.vscode.junit5.debugger.AppTest
    at org.junit.platform.engine.discovery.ClassSelector.lambda$getJavaClass$0(ClassSelector.java:75)
    at org.junit.platform.commons.function.Try$Failure.getOrThrow(Try.java:335)
    at org.junit.platform.engine.discovery.ClassSelector.getJavaClass(ClassSelector.java:74)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1492)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
    at org.junit.vintage.engine.discovery.ClassSelectorResolver.resolve(ClassSelectorResolver.java:29)
    at org.junit.vintage.engine.discovery.VintageDiscoverer.collectTestClasses(VintageDiscoverer.java:64)
    at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:51)
    at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:168)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:155)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
    at com.microsoft.java.test.runner.junit5.CustomizedConsoleTestExecutor.lambda$executeTests$0(CustomizedConsoleTestExecutor.java:42)
    at com.microsoft.java.test.runner.junit5.CustomContextClassLoaderExecutor.invoke(CustomContextClassLoaderExecutor.java:29)
    at com.microsoft.java.test.runner.junit5.CustomizedConsoleTestExecutor.executeTests(CustomizedConsoleTestExecutor.java:38)
    at com.microsoft.java.test.runner.junit5.CustomizedConsoleLauncher.execute(CustomizedConsoleLauncher.java:29)
    at com.microsoft.java.test.runner.Launcher.main(Launcher.java:53)
Caused by: java.lang.ClassNotFoundException: demo.vscode.junit5.debugger.AppTest
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
    at org.junit.platform.commons.util.ReflectionUtils.lambda$tryToLoadClass$9(ReflectionUtils.java:732)
    at org.junit.platform.commons.function.Try.lambda$call$0(Try.java:57)
    at org.junit.platform.commons.function.Try.of(Try.java:93)
    at org.junit.platform.commons.function.Try.call(Try.java:57)
    at org.junit.platform.commons.util.ReflectionUtils.tryToLoadClass(ReflectionUtils.java:695)
    at org.junit.platform.commons.util.ReflectionUtils.tryToLoadClass(ReflectionUtils.java:651)
    ... 21 more

If I execute the tests with gradle, it works:

PS > .\gradlew clean test
Configuration on demand is an incubating feature.

> Task :test

demo.vscode.junit5.debugger.AppTest > testAppHasAGreeting() PASSED

BUILD SUCCESSFUL in 3s
4 actionable tasks: 4 executed

[jdneo]

@sceee Thank you for the sample project, It works well on my laptop:

Could you see any warning in the VS Code Problems panel?

[sceee]

@jdneo hm, strange.

No, I don't see any errors in the Problems panel:

[jdneo]

Try:

Execute palette command: Java: Clean the Java language server workspace

• Choose Restart and delete

And recompile your project by: Java: Force Java Compilation?

[sceee]

@jdneo now it worked. Do you have any hint how I can avoid cleaning up the workspace in the future to get this working?

[yaohaizh]

@sceee, something, the workspace will contain stale data when VSCode Java extension update, JDK changed. In this situation, it will need clean the workspace to regenereate the binaries. Not sure if this is the situation of your case here.

[jdneo]

Since the issue has been solved, I'll close it for now.

@sceee If you have more questions about the Java Language Support, feel free to raise them in the Gitter channel

Thanks.