Junit5: Error with 5.0.0-M3 in IntelliJ

[tetrodoxin]

When I changed the build from 5.0.0-M2 to 5.0.0-M3 within a project in IntelliJ 2016.3 I got following error, when trying to run tests;

Dez 05, 2016 10:30:44 PM org.junit.platform.launcher.core.ServiceLoaderTestEngineRegistry loadTestEngines
INFORMATION: Discovered TestEngines with IDs: [junit-jupiter, junit-vintage]
Exception in thread "main" java.lang.NoSuchMethodError: org.junit.platform.commons.util.Preconditions.notNull([Ljava/lang/Object;Ljava/lang/String;)[Ljava/lang/Object;
    at org.junit.platform.launcher.core.DefaultLauncher.registerTestExecutionListeners(DefaultLauncher.java:71)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:44)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

This error even comes up within the junit5-gradle-consumer sample project. M2 is fine, M3 produces this error, which tells me, that allegedly a method notNull(Object, String, Object) was called, which doesn't exists, but I don't find no clue for this strange behavior.

[sbrannen]

The problem is that IntelliJ IDEA is using org.junit.platform:junit-platform-launcher:1.0.0-M2 (the JAR that contains the DefaultLauncher) in conjunction with org.junit.platform:junit-platform-commons:1.0.0-M3 (the JAR that contains the Preconditions class) which is transitively pulled in via your org.junit.jupiter:junit-jupiter-api:5.0.0-M3 dependency.

To solve this problem, IntelliJ will need to ensure that it _picks_ the correct version of the Launcher that is compatible with test engines that transitively pull in the commons JAR.

In this particular case, you can work around this issue by adding the following explicit dependency to your Gradle build.

testCompile("org.junit.platform:junit-platform-launcher:1.0.0-M3")

I verified that this works for the junit5-gradle-consumer sample project, so please let us know if that works for you.

cheers!

[sbrannen]

This is somewhat related to #576.

[sbrannen]

@junit-team/junit-lambda and @akozlova,

I'm inclined to say this is a bug in or an issue for IDEA.

What are your thoughts on the matter?

[tetrodoxin]

Thanks to you.
Indeed that line helped, although I had to find out first, that the order of these lines seem to be important. Got still a lot to learn about gradle...

[akozlova]

See https://youtrack.jetbrains.com/issue/IDEA-164865 and https://github.com/junit-team/junit5/issues/565

IDEA bundles junit launcher jar, for 2016.3 it was the one from M2 as when IDEA was released it was the last one available. 2016.3.1 will be released this week with bundled M3. This means though that people who use M2 would see similar problems. We would publish the instruction how to deal with it together with the release notes.

What is the desired behavior for IDE then? Unfortunately we can't bundle all versions of launcher and there is no workaround without launcher jar on the user's classpath, at least I couldn't find one after I spend around a day.

[estekhin]

IntelliJ IDEA with M3 support has been released (https://blog.jetbrains.com/idea/2016/12/intellij-idea-2016-3-1-rc-updates-junit-5-support-to-m3/).
This issue should be closed as there is nothing to do on JUnit side.

[RobLewis]

I am using the 163.9166.20 build (Mac) and am getting the following when I try to run a simple method test:

Dec 07, 2016 8:25:12 AM org.junit.platform.launcher.core.ServiceLoaderTestEngineRegistry loadTestEngines
INFO: Discovered TestEngines with IDs: [junit-jupiter]
Internal Error occurred.
org.junit.platform.commons.util.PreconditionViolationException: Could not load class with name: DirectoryTest
    at org.junit.platform.engine.discovery.MethodSelector.lambda$lazyLoadJavaClass$0(MethodSelector.java:154)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at org.junit.platform.engine.discovery.MethodSelector.lazyLoadJavaClass(MethodSelector.java:153)
    at org.junit.platform.engine.discovery.MethodSelector.getJavaClass(MethodSelector.java:122)
    at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.lambda$resolveSelectors$3(DiscoverySelectorResolver.java:63)
    at java.util.ArrayList.forEach(ArrayList.java:1249)
    at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:62)
    at org.junit.jupiter.engine.JupiterTestEngine.resolveDiscoveryRequest(JupiterTestEngine.java:50)
    at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:43)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:109)
    at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:79)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

Process finished with exit code 254
Empty test suite.

[sbrannen]

@RobLewis, could you please open an issue directly with IntelliJ IDEA for the problem you're encountering?

[sbrannen]

Hi @akozlova,

Thanks for the link.

IDEA bundles junit launcher jar, for 2016.3 it was the one from M2 as when IDEA was released it was the last one available. 2016.3.1 will be released this week with bundled M3. This means though that people who use M2 would see similar problems. We would publish the instruction how to deal with it together with the release notes.

Glad to hear you're releasing 2016.3.1 with JUnit 5 M3 support so quickly. Awesome!

Publishing the instructions is, IMHO, the best solution to the problem for the time being.

The real issue is that we occasionally make breaking changes from one milestone to the next. However, this will not be the case once the GA releases are out the door. At that point, there should no longer be breaking changes in modules like junit-platform-commons.

What is the desired behavior for IDE then? Unfortunately we can't bundle all versions of launcher and there is no workaround without launcher jar on the user's classpath, at least I couldn't find one after I spend around a day.

I think the best the IDE can do is log a warning to the user if the user has included a version of a test engine in the classpath that requires a newer version of a JAR used by IDEA itself, which is exactly the case described in this GitHub issue.

And... again... once the GA releases are out, this sort of thing should (hopefully) no longer be a problem.

[sbrannen]

@estekhin

IntelliJ IDEA with M3 support has been released (https://blog.jetbrains.com/idea/2016/12/intellij-idea-2016-3-1-rc-updates-junit-5-support-to-m3/). This issue should be closed as there is nothing to do on JUnit side.

Well, that's only a release candidate, but yes... there will soon be a version of IDEA that supports JUnit 5.0 M3. However, that does not address the underlying issue. I am therefore leaving this issue open a bit longer in order for the @junit-team/junit-lambda team to discuss the topic internally (e.g., to decide if we want to include documentation for the work-around in the User Guide, etc.).

[avandeursen]

I had this problem too, for the junit5-maven-consumer example project (importing the dependencies via the pom into IntelliJ).

I can confirm that IntelliJ 2016.3.1 as mentioned by @estekhin https://github.com/junit-team/junit5/issues/586#issuecomment-265431975 fixes this problem (thanks everyone!)

What did _not_ work for this maven example was adding the launcher to the pom, inspired by the suggestion https://github.com/junit-team/junit5/issues/586#issuecomment-265012600 from @sbrannen. With that extended pom I could execute the new JUnit5 test case (in IntelliJ), but _not_ the old JUnit 4 test case.

[sbrannen]

@avandeursen, does the JUnit 4 test work for you if you add an explicit project dependency on the junit-vintage-engine (i.e., copied from the configuration of the maven-surefire-plugin)?

[RobLewis]

General comment: _apparently_ (I say because I can't be sure) my issue was due to compiler errors. I thought that selecting "Build, no error check" would prevent that from happening, but evidently not.
As a relative newcomer, it's a bad feeling being squeezed at the intersection of two complex systems (IDEA and JUnit), with so many obscure options and settings to deal with. And is it conceivable that JUnit might give a more helpful error message than PreconditionViolationException: Could not load class with name: DirectoryTest? WHY couldn't you load the class??

[sbrannen]

@RobLewis,

And is it conceivable that JUnit might give a more helpful error message than PreconditionViolationException: Could not load class with name: DirectoryTest? WHY couldn't you load the class??

Yes, that is conceivable. 😉

Please create a new issue to address that, and we'll see what we can do.

Thanks

[sbrannen]

@avandeursen, ping!

[marcphilipp]

Will be addressed in #602 and #609.

[anamikam1]

I am using IntelliJ 2017.2 and trying to run my first JUnit5 test and running into the following exception.

I have attached my POM, would it be possible for someone to please take a look?

Exception in thread "main" java.lang.NoSuchMethodError: org.junit.platform.commons.util.ReflectionUtils.getDefaultClassLoader()Ljava/lang/ClassLoader;
    at org.junit.platform.launcher.core.ServiceLoaderTestEngineRegistry.loadTestEngines(ServiceLoaderTestEngineRegistry.java:31)
    at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:42)
    at com.intellij.junit5.JUnit5IdeaTestRunner.createListeners(JUnit5IdeaTestRunner.java:39)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:45)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

The relevant dependencies in the POM are as follows:

<dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.0.2</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.0.2</version>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-commons</artifactId>
            <version>1.0.1</version>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-launcher</artifactId>
            <version>1.0.0-M2</version>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-runner</artifactId>
            <version>1.0.0-M4</version>
            <scope>test</scope>
        </dependency>

[sbrannen]

@anamikam1, you simply cannot mix different versions like that (especially not _Milestones_): that will never work.

Instead, use compatible versions -- for example, the latest release versions as documented here in the User Guide.

[anamikam1]

hi Sam,

Thanks for your response. I still can't get past the following error. Would it be possible to please take another look at my POM?

Exception in thread "main" java.lang.NoSuchMethodError: org.junit.platform.launcher.Launcher.execute(Lorg/junit/platform/launcher/LauncherDiscoveryRequest;)V
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:61)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

     <junit.version>4.12</junit.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-runner</artifactId>
            <version>1.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.vintage</groupId>
            <artifactId>junit-vintage-engine</artifactId>
            <version>4.12.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-launcher</artifactId>
            <version>1.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-commons</artifactId>
            <version>1.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apiguardian</groupId>
            <artifactId>apiguardian-api</artifactId>
            <version>1.0.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.opentest4j</groupId>
            <artifactId>opentest4j</artifactId>
            <version>1.0.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-params</artifactId>
            <version>5.0.3</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-engine</artifactId>
            <version>1.0.3</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-jdbc</artifactId>
            <version>5.0.2.RELEASE</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.platform</groupId>
                        <artifactId>junit-platform-surefire-provider</artifactId>
                        <version>1.0.3</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>
</project>

[sbrannen]

The version for junit-vintage-engine should be 4.12.3 (not 4.12.0).

Also, you don't need to declare the junit-jupiter-params dependency twice.

[sbrannen]

Also, it is highly recommended that you use IntelliJ 2017.3 or newer.

[sbrannen]

If you have any further problems, please ask a question on Stack Overflow, since this issue tracker is not intended for Q&A style discussions.

Thanks

[jdaviestx]

On the off chance it helps anybody else who runs into this issue - I had the exact same problem trying to run my unit tests in IntelliJ. It turned out that my unit test was importing org.junit.Test, rather than org.junit.jupiter.api.Test, but I didn't declare a dependency on junit-vintage in my pom.xml. The solution was to either explicitly declare the correct version of junit-vintage in my pom (4.12.0-M3 in my case), or switch my unit tests over to import org.junit.jupiter.api.Test.

[metabrain]

Exactly the same problem I had @jdaviestx , thanks! It fixed it for me as well. But I wonder, when both dependencies are loaded (vintage + jupiter) shouldn't both of them work? Both @Test annotations I mean. Or at least, return a more sane error.

Reproducing:
Any unit test with @Test from org.junit.Test instead of org.junit.jupiter.api.Test

Result:

Exception in thread "main" java.lang.NoSuchMethodError: org.junit.platform.launcher.Launcher.execute(Lorg/junit/platform/launcher/LauncherDiscoveryRequest;)V
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:61)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

Even thought this is more targeted towards JUnit5 integration in Intelij, I'll leave it here for future reference in case anyone else stumbles upon this error like I did in the future.

EDIT: I was wrong, see below.

[sbrannen]

But I wonder, when both dependencies are loaded (vintage + jupiter) shouldn't both of them work?

Yes, if both engines are present in the classpath, that will work. That's a core feature of the JUnit Platform.

[sbrannen]

The NoSuchMethodError you mention is the result of having incompatible versions of JUnit JARs in the classpath. It is not the result of using a different annotation.

[metabrain]

@sbrannen you are right, I must've done something else since it started working through maven but in Intelij it still doesn't work. I have a pretty old Intelij version (2017.1.3) so that might be the issue as others have pointed. I will try upgrading the version.

But as long as it is working through Maven this is for the JetBrains team and not JUnit5 team, I suppose.

Thanks for the reply again @sbrannen once more
Cheers

[sbrannen]

You're welcome, @metabrain!

Thanks for providing the feedback.

[sbrannen]

@metabrain, with regard to proper dependency management and use in IntelliJ, I highly recommend you read the IntelliJ IDEA section of the User Guide.