Junit5: Excluded tags don't work on Maven with JDK 9

Created on 27 Nov 2017  ·  24Comments  ·  Source: junit-team/junit5

From the junit5-vanilla-maven project, add a new test:

@Tag("filtered")
public class FilteredTest {
    @Test
    public void dummy() {  }
}

Exclude the tag in the pom.xml:

<configuration>
  <properties>
    <excludeTags>filtered</excludeTags>
  </properties>
</configuration>

Modified project: https://github.com/acogoluegnes/junit5-vanilla-maven

Exclusion works on JDK 8, but not on JDK 9:

$ ./mvnw --version
/home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T21:39:06+02:00)
Maven home: /home/acogoluegnes/.m2/wrapper/dists/apache-maven-3.5.0-bin/6ps54u5pnnbbpr6ds9rppcc7iv/apache-maven-3.5.0
Java version: 1.8.0_144, vendor: Oracle Corporation
Java home: /usr/local/java-install/jdk1.8.0_144/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-101-generic", arch: "amd64", family: "unix"
$ ./mvnw clean test
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building junit5-vanilla-maven 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ junit5-vanilla-maven ---
[INFO] Deleting /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ junit5-vanilla-maven ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ junit5-vanilla-maven ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ junit5-vanilla-maven ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ junit5-vanilla-maven ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ junit5-vanilla-maven ---

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.example.project.CalculatorTests
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.059 sec - in com.example.project.CalculatorTests

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.993 s
[INFO] Finished at: 2017-11-27T11:30:40+01:00
[INFO] Final Memory: 16M/199M
[INFO] ------------------------------------------------------------------------

$ ./mvnw --version
/home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T21:39:06+02:00)
Maven home: /home/acogoluegnes/.m2/wrapper/dists/apache-maven-3.5.0-bin/6ps54u5pnnbbpr6ds9rppcc7iv/apache-maven-3.5.0
Java version: 9, vendor: Oracle Corporation
Java home: /usr/local/java-install/jdk-9
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-101-generic", arch: "amd64", family: "unix"
$ ./mvnw clean test
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building junit5-vanilla-maven 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ junit5-vanilla-maven ---
[INFO] Deleting /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ junit5-vanilla-maven ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ junit5-vanilla-maven ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ junit5-vanilla-maven ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ junit5-vanilla-maven ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /home/acogoluegnes/temp/junit5-samples/junit5-vanilla-maven/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ junit5-vanilla-maven ---

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.example.project.CalculatorTests
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.106 sec - in com.example.project.CalculatorTests
Running com.example.project.FilteredTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - in com.example.project.FilteredTest

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.348 s
[INFO] Finished at: 2017-11-27T11:29:34+01:00
[INFO] Final Memory: 14M/47M
[INFO] ------------------------------------------------------------------------
Maven Surefire Platform execution
Source
picture acogoluegnes
👍1

Most helpful comment

We fixed Java 10 issue/NPE as well.

picture Tibor17 on 23 Jan 2018
👍4 🎉1

All 24 comments

Fascinating.

Can you please try a newer version of maven-surefire-plugin:2.19.1? There are issues with it, too, but maybe the exclude filter works... ;)

sormuras picture sormuras on 27 Nov 2017

Just reproduced it with our maven-consumer sample application:

Java 8 - Tests run: 5, Failures: 0, Errors: 0, Skipped: 1
Java 9 - Tests run: 6, Failures: 0, Errors: 0, Skipped: 1

sormuras picture sormuras on 27 Nov 2017

This is caused by a bug in Surefire that I reported a while back:
https://issues.apache.org/jira/browse/SUREFIRE-1445

marcphilipp picture marcphilipp on 27 Nov 2017

@marcphilipp Thanks for the pointer, I submitted a PR: https://github.com/apache/maven-surefire/pull/171

acogoluegnes picture acogoluegnes on 27 Nov 2017

@marcphilipp The mentioned SUREFIRE bug 1445 and the mentioned pull request were both closed 2 Dec, so I think this should no longer be status: blocked.

Foorack picture Foorack on 27 Dec 2017

@Foorack Thanks for the reminder. However, I'd like to keep this issue open until Surefire 2.21 is released and we can verify that it works.

marcphilipp picture marcphilipp on 28 Dec 2017

I can confirm we've observed the same behavior with surefire 2.19.1 and the platform support provider 1.0.2.

jponge picture jponge on 22 Jan 2018

@tibor17 Is there a scheduled date for the release that will include the fix?

marcphilipp picture marcphilipp on 22 Jan 2018

Surefire 2.21.0 compliant with Java 9 is about to release quite soon. We
have only one issue to fix and change build process.

I think the users will be able to download artifacts from Maven Central
next week.

Tibor17 picture Tibor17 on 23 Jan 2018
👍1

We fixed Java 10 issue/NPE as well.

Tibor17 picture Tibor17 on 23 Jan 2018
👍4 🎉1

Thanks, @Tibor17!

Glad to hear there will be an official fix soon. 👍

sbrannen picture sbrannen on 23 Jan 2018

@Tibor17 Any update on the 2.21.0 release?

jponge picture jponge on 16 Feb 2018

We will postpone one issue in another release 2.21.1 and we can wait to
Jenkins build to complete. If everything goes right I will start the
release vote on Apache these days.

On Fri, Feb 16, 2018 at 3:57 PM, Julien Ponge notifications@github.com
wrote:

@Tibor17 https://github.com/tibor17 Any update on the 2.21.0 release?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/junit-team/junit5/issues/1188#issuecomment-366257950,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA_yR5eHHgSaMp-09u-AY9Hnzz6MYbhXks5tVZd2gaJpZM4QrcOj
.

--
Cheers
Tibor

Tibor17 picture Tibor17 on 16 Feb 2018
👍2

Re-tested this with Java 9 and Java 10 using Surefire 2.21.0 and JUnit 5.2-SNAPSHOT. Excludes like a charm.

sormuras picture sormuras on 7 Mar 2018
👍2 🎉1

But doesn't work at all with https://github.com/junit-team/junit5-samples/tree/r5.1.0 (at least on Java 8 and 9).

$ git clone https://github.com/junit-team/junit5-samples.git
$ cd junit5-samples/junit5-maven-consumer
$ git co origin/r5.1.0
$ sed -i 's/2.19.1/2.21.0/g' pom.xml
$ ./mvnw clean test                                                                     
/home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building junit5-maven-consumer 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ junit5-maven-consumer ---
[INFO] Deleting /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ junit5-maven-consumer ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ junit5-maven-consumer ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ junit5-maven-consumer ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ junit5-maven-consumer ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ junit5-maven-consumer ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.724 s
[INFO] Finished at: 2018-03-12T14:41:36+01:00
[INFO] Final Memory: 18M/265M
[INFO] ------------------------------------------------------------------------
acogoluegnes picture acogoluegnes on 12 Mar 2018

I've also observed that no test are run indeed.

On Mon, Mar 12, 2018 at 2:46 PM, Arnaud Cogoluègnes <
[email protected]> wrote:

But doesn't work at all with https://github.com/junit-team/
junit5-samples/tree/r5.1.0 (at least on Java 8 and 9).

$ git clone https://github.com/junit-team/junit5-samples.git
$ cd junit5-samples/junit5-maven-consumer
$ git co origin/r5.1.0
$ sed -i 's/2.19.1/2.21.0/g' pom.xml
$ ./mvnw clean test
/home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building junit5-maven-consumer 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ junit5-maven-consumer ---
[INFO] Deleting /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ junit5-maven-consumer ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ junit5-maven-consumer ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ junit5-maven-consumer ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ junit5-maven-consumer ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 4 source files to /home/acogoluegnes/temp/junit5-samples/junit5-maven-consumer/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.21.0:test (default-test) @ junit5-maven-consumer ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.724 s
[INFO] Finished at: 2018-03-12T14:41:36+01:00
[INFO] Final Memory: 18M/265M
[INFO] ------------------------------------------------------------------------

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/junit-team/junit5/issues/1188#issuecomment-372314459,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABlae-t6zJzoL9YfjT6FFSU-DL-FHF1ks5tdnwrgaJpZM4QrcOj
.

jponge picture jponge on 12 Mar 2018

@acogoluegnes and @jponge,

As @sormuras mentioned:

Re-tested this with Java 9 and Java 10 using Surefire 2.21.0 and JUnit 5.2-SNAPSHOT. Excludes like a charm.

You have to use JUnit 5.2 snapshots (e.g., Platform 1.2.0-SNAPSHOT).

sbrannen picture sbrannen on 12 Mar 2018

For example, the following POM works locally for me on JDK 8, 9, and 10.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>junit5-vanilla-maven-snapshot</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
        <junit.jupiter.version>5.2.0-SNAPSHOT</junit.jupiter.version>
        <junit.platform.version>1.2.0-SNAPSHOT</junit.platform.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>${junit.jupiter.version}</version>
            <scope>test</scope>
        </dependency>
        <!-- To avoid compiler warnings about @API annotations in JUnit code -->
        <dependency>
            <groupId>org.apiguardian</groupId>
            <artifactId>apiguardian-api</artifactId>
            <version>1.1.0-SNAPSHOT</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <repositories>
        <repository>
            <id>oss-sonatype</id>
            <name>oss-sonatype</name>
            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

    <pluginRepositories>
        <pluginRepository>
            <id>oss-sonatype</id>
            <name>oss-sonatype</name>
            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.7.0</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.21.0</version>
                <configuration>
                    <includes>
                        <include>**/Test*.java</include>
                        <include>**/*Test.java</include>
                        <include>**/*Tests.java</include>
                        <include>**/*TestCase.java</include>
                    </includes>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.platform</groupId>
                        <artifactId>junit-platform-surefire-provider</artifactId>
                        <version>${junit.platform.version}</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

</project>

sbrannen picture sbrannen on 12 Mar 2018

Actually... it works on JDK 11, too. 😉

sbrannen picture sbrannen on 12 Mar 2018

I prefer not to use the snapshots. I had to revert back to Surefire 2.19.1, as the respective latest stable versions of all those libraries don't work together for the simplest use case. I'll try the latest JUnit 5 and platform versions. Happy to see it's fixed in the next release though.

acogoluegnes picture acogoluegnes on 12 Mar 2018

I prefer not to use the snapshots.

I totally understand that!

I wasn't actually suggesting that people use the snapshots for _production_. Rather, I was just pointing out that Maven Surefire 2.21.0 is currently only supported by the JUnit Platform in snapshots. 😉

I'm also just sticking with Surefire 2.19.1 for the time being (if I have to use Maven).

sbrannen picture sbrannen on 12 Mar 2018

The "only" issue with 2.19.1 is that tags get skipped on Java 9 :-)

On Mon, Mar 12, 2018 at 3:59 PM, Sam Brannen notifications@github.com
wrote:

I prefer not to use the snapshots.

I totally understand that!

I wasn't actually suggesting that people use the snapshots for
production. Rather, I was just pointing out that Maven Surefire 2.21.0
is currently only supported by the JUnit Platform in snapshots. 😉

I'm also just sticking with Surefire 2.19.1 for the time being (if I have
to use Maven).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/junit-team/junit5/issues/1188#issuecomment-372339613,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABlaUhLkbYjS_BaSHcY30Z8iq1XdaUqks5tdo04gaJpZM4QrcOj
.

jponge picture jponge on 12 Mar 2018

fwiw https://github.com/junit-team/junit5-samples/blob/master/junit5-vanilla-maven/pom-SNAPSHOT.xml

sormuras picture sormuras on 12 Mar 2018
👍1

The "only" issue with 2.19.1 is that tags get skipped on Java 9 :-)

Yeah... well... we unfortunately can't do anything about that. :(

sbrannen picture sbrannen on 12 Mar 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

PeterWippermann picture PeterWippermann  ·  5Comments
ondrejlerch picture ondrejlerch  ·  4Comments
blindpirate picture blindpirate  ·  5Comments
netzwerg picture netzwerg  ·  6Comments
aepfli picture aepfli  ·  4Comments