Junit5: Tag Expressions with spaces do not work with Surefire provider

[thecatwhisperer]

Overview

• Tag Expression used in Maven POM cannot include spaces.
• Tag Expression cannot use | (or) symbol, must use , (comma) for or instead

Versions

JDK:

java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

Junit Version: 5.1.1
Surefire version: 2.19.1 (2.21.0 fails with ClassNotFoundException: org.apache.maven.plugin.surefire.log.api.ConsoleLogger)

Examples

Given the following code:

public class TagExpressionTest {
    @Test
    @Tag("TagA")
    @DisplayName("Test with Tag A")
    public void testWithTagA() {
        out.println("Test with Tag A");
    }

    @Test
    @Tag("TagB")
    @DisplayName("Test with Tag B")
    public void testWithTagB() {
        out.println("Test with Tag B");
    }

    @Test
    @Tags({@Tag("TagA"), @Tag("TagB")})
    @DisplayName("Test with Tag B")
    public void testWithTagsAandB() {
        out.println("Test with Tags A & B");
    }
}

TagA & TagB

The following fails with PreconditionViolationException: Unable to parse tag expression "&": missing lhs and rhs operand for '&' at index <0>:

<includeTags>TagA &amp; TagB</includeTags>

however:

<includeTags>TagA&amp;TagB</includeTags>

is ok, runs only testWithTagsAandB

TagA or TagB

The following fails with PreconditionViolationException: Unable to parse tag expression "|": missing lhs and rhs operand for '|' at index <0>:

<includeTags>TagA | TagB</includeTags>

however:

<includeTags>TagA|TagB</includeTags>

is ok, runs testWithTagA, testWithTagB & testWithTagsAandB

[sbrannen]

What happens if you enclose the tag expression within a CDATA block?

[thecatwhisperer]

Same issue. Sorry, I had also tested that, should have included in comment as well.

I have a test-project setup, just not sure where to upload it? I can great a public github repo if it helps? Is pretty straight forward though.

[sbrannen]

Did you test it exactly like the following?

<includeTags><![CDATA[TagA & TagB]]></includeTags>

[thecatwhisperer]

Yes, via a profile:

 <profile>
    <id>tags-a-and-b-with-cdata</id>
    <properties>
        <includeTags><![CDATA[TagA & TagB]]></includeTags>
    </properties>
</profile>

[sbrannen]

Also, what version of the JDK are you using to run Maven?

[thecatwhisperer]

Updated report to include JDK info

[sbrannen]

I ask, because it may be the same as this: https://github.com/junit-team/junit5/issues/1188#issuecomment-372342701

[sbrannen]

Updated report to include JDK info

Ahhhh, you're on Java 8.

Then perhaps it is a different bug.

[sbrannen]

I was just able to reproduce this locally.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project junit5-maven-consumer: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: There was an error in the forked process
[ERROR] org.apache.maven.surefire.util.SurefireReflectionException: org.junit.platform.commons.util.PreconditionViolationException: Unable to parse tag expression "&": missing lhs and rhs operand for '&' at index <0>
[ERROR]     at org.apache.maven.surefire.util.ReflectionUtils.instantiateOneArg(ReflectionUtils.java:135)
[ERROR]     at org.apache.maven.surefire.booter.ForkedBooter.createProviderInCurrentClassloader(ForkedBooter.java:321)
[ERROR]     at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
[ERROR]     at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
[ERROR]     at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
[ERROR] Caused by: org.junit.platform.commons.util.PreconditionViolationException: Unable to parse tag expression "&": missing lhs and rhs operand for '&' at index <0>
[ERROR]     at org.junit.platform.launcher.TagFilter.lambda$parse$2(TagFilter.java:149)
[ERROR]     at org.junit.platform.launcher.tagexpression.ParseResult.tagExpressionOrThrow(ParseResult.java:42)
[ERROR]     at org.junit.platform.launcher.TagFilter.parse(TagFilter.java:148)
[ERROR]     at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
[ERROR]     at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
[ERROR]     at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
[ERROR]     at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
[ERROR]     at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
[ERROR]     at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[ERROR]     at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
[ERROR]     at org.junit.platform.launcher.TagFilter.parseAll(TagFilter.java:144)
[ERROR]     at org.junit.platform.launcher.TagFilter.includeMatching(TagFilter.java:135)
[ERROR]     at org.junit.platform.launcher.TagFilter.includeTags(TagFilter.java:91)
[ERROR]     at java.util.Optional.map(Optional.java:215)
[ERROR]     at org.junit.platform.surefire.provider.JUnitPlatformProvider.getFilters(JUnitPlatformProvider.java:157)
[ERROR]     at org.junit.platform.surefire.provider.JUnitPlatformProvider.<init>(JUnitPlatformProvider.java:91)
[ERROR]     at org.junit.platform.surefire.provider.JUnitPlatformProvider.<init>(JUnitPlatformProvider.java:85)
[ERROR]     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[ERROR]     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[ERROR]     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[ERROR]     at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
[ERROR]     at org.apache.maven.surefire.util.ReflectionUtils.instantiateOneArg(ReflectionUtils.java:131)
[ERROR]     ... 4 more

We'll look into it.

[sbrannen]

OK, this is definitely a bug.

And that bug resides in org.junit.platform.surefire.provider.JUnitPlatformProvider.getPropertiesList(String).

Namely, that method splits the input by space or comma.

So that obviously won't work for a tag expression.

Thanks for bringing this to our attention.

We'll find a solution. 😉

[sormuras]

Try it with 2.21.0 and the manual API fix from #801

[sbrannen]

@sormuras... see above. 😇

[sbrannen]

_in progress_

[sbrannen]

This has been fixed in maser in commit f4cd4bc8d294ef580e192034f5de833485e4c7fa.

[sbrannen]

Reopening to discuss backporting to 5.1.2 with the @junit-team/junit-lambda.

[sbrannen]

Team Decision: hold off on backporting this bug fix to 5.1.x for the time being due to the earlier planned release of 5.2.