Junit5: Introduce support for declarative test suites

[smoyer64]

Overview

As requested in issue #681 (Introduce junit-platform-suite-api module), the annotations used for test discovery and selection in the JUnitPlatform runner have been refactored into the junit-platform-suite-api. This allows a purely JUnit 5 test suite to be defined declaratively, though it currently requires an alternate engine (as opposed to an extension).

This issue describes a generalized JUnit platform suite that's suitable for the Jupiter test engine as well as any others that might be chosen. It's my opinion that unit tests rarely require test suites, but it becomes far more common when integration, system and acceptance tests are being executed due to the very common integration with external systems that should only be initialized before the entire test run. Examples of resources that can be safely shared among tests are database connection pools (or really any connection pool), test frameworks that require significant set-up time (e.g. GWT test suites eliminate 20s of HtmlUnit set-up time for each test - it's only incurred once at the beginning of each suite).

The examples shown below indicate how the @Suite annotation can be used in conjunction with those already in the junit-platform-suite-api module. Originally I'd also defined the @BeforeSuite and @AfterSuite annotations but after further consideration decided that since suites are really just an arbitrary collection of indeterminately nested test containers, it would be better if the @BeforeAll and @AfterAll annotations were contextual (nested in the same way the hierarchy would be presented in the test tree). I've also assumed that these annotations no longer require the static modifier as discussed elsewhere.

Imagine if you will a set of acceptance tests that have a custom reporting mechanism that should be configured for the entire test run. Included within this outer suite is a set of tests that require a database connection pool and a different set of tests that require an LDAP connection pool. The top-level suite definition for the acceptance tests might look something like:

@Suite
@IncludeEngines("jupiter-engine") // not required if there's only one engine on the classpath
@IncludeClassNamePatterns(".*AcceptanceTestSuite$")
public class AcceptanceTestSuite {

    @BeforeAll
    void setUp() {
        ... // code to set-up custom test reporting
    }

    @AfterAll() {
    void tearDown() {
        ... // code to tear-down custom test reporting
    }

}

There are two notable characteristics of this test suite:

1. It specifies which engine will be used for the entire container/test hierarchy (assuming there is more than one).
2. It includes potentially numerous other test suites as sub-containers which can each have their own contextual set-up and tear-down.

A child suite for tests that require a database connection pool might look something like this:

@Suite
@IncludeTags({"integration", "database"})
@IncludeClassNamePatterns(".*IT$")
public class DatabaseAcceptanceTestSuite {

    @BeforeAll
    void setUp() {
        ... // code to set-up the database connection pool
    }

    @AfterAll() {
    void tearDown() {
        ... // code to tear-down the database connection pool
    }

}

If run under Maven, test classes that end in "IT" would be run using the maven-failsafe-plugin. If there are other classes of integration tests not run during acceptance testing it would also be necessary to make sure the acceptance test classes were not run (as they'd fail without the suite's set-up methods).

Feature Request

Provide the means for arbitrarily nested, declarative tests suites in JUnit 5 including the execution of set-up and tear-down methods

Related Issues

• #330
• #456
• #746
• #687 (perhaps superseded by this issue)
• #964
• Eclipse IDE issue
• #1243

Deliverables

• [ ] TBD

[mlevison]

Please consider a new name other than suites - so that people with JUnit 3
history don't accidentally mis-use these for Unit Testing.

Cheers
Mark

[sbrannen]

Tentatively slated for M5.

[sbrannen]

@smoyer64, thanks for creating the ticket!

I think there are two main topics we need to sort out here:

1. suites within the JUnit Jupiter engine
2. declarative suite configuration at the Platform level

If I understand correctly, you are currently focusing on # 1; whereas, I would first like to focus on # 2.

Status Quo

• We have generic suite API annotations.
• We have a JUnit 4 based runner that supports them.

The Future

• Ideally we should have Platform-level support for them as well.
• If it makes sense to have additional support in JUnit Jupiter for "suites", we should consider that as well.

My Vision

I think it would be most beneficial to the entire community if the JUnit Platform provides support for declarative test suites, akin to what the JUnitPlatform runner already does, but without any ties to _any_ testing framework.

In other words, the _declarative test suite support_ should be analogous to using the ConsoleLauncher or JUnit Platform Gradle plugin.

This can be achieved by moving the internals for processing such suite annotations and building the corresponding LauncherDiscoveryRequest from the JUnitPlatform runner to a new home in the launcher module. This new "prediscovery" code could then be used to transparently add selectors and filters to the request _before_ any engines are asked to _discover_. For example, this could potentially be performed in the DefaultLauncher.

In this manner, all test engines could benefit from being able to be configured by a generic suite (which is already supported in build scripts). From the perspective of a TestEngine, the presence of selectors and filters added from the suite declaration would be no different than those added via a build script.

[sbrannen]

This allows a purely JUnit 5 test suite to be defined declaratively, though it currently requires an alternate engine (as opposed to an extension).

My above proposal avoids exactly such a need.

If we support suites at the platform level, there is no need for any additional test engines.

[sbrannen]

By the way, my proposal does not in any way preclude the introduction of suite support specific to a particular test engine.

The idea is that the Platform (at the launcher level) would only honor the suite API annotations if a special, platform-specific annotation is present (e.g., @JUnitPlatformSuite); otherwise, the Platform would simply ignore such annotations and leave the LauncherDiscoveryRequest unchanged.

[sbrannen]

@sbrannen - Hopefully this is enough to start the conversation.

@smoyer64 - yes, indeed it is! 👍

[sbrannen]

Another topic to sort out (which I didn't explain so well above) is whether _suites_ should be handled specially within the test plan.

With the generic approach (at the Launcher level), this would not be the case since test engines would not _see_ the presence of a suite as a first-class citizen in the test plan. As I mentioned above, suites would be invisible to test engines.

So, if we want to include first-class support for _suites_ as nodes within the test plan, that opens up a whole new can of worms! ... And I'm not so sure we want to go that route, at least not before GA.

[smoyer64]

@sbrannen I actually think we're exactly on the same page - #687 along with #681 were the two issues that I put in to allow an extension to implement suites. The engine I've already created works fine but with a couple more extension points it would be trivial to simply make it an extension.

Let's ignore the word "suite" for a moment - I think there's a great argument that these are simply hierarchical test containers that participate in the engine's life-cycle at their appropriate level of the hierarchy. And the junit-platform-suite-api annotations are simply a declarative specification describing what tests are contained within the container.

I wrote this issue as asked in #681 because I thought you were promoting a built-in suite (which may be worthwhile) but the original two issues were simply to make it easier to write extensions that implement arbitrary containers. Breaking the tie between junit-platform-suite-api and JUnit >=4.12 was definitely the right thing to do - perhaps this issue will be another meta-issue (like #14) that spawns a series of small steps towards the final goal.

EDIT - It appears we're trying to have one of those conversations where we're both talking at the same time ;) I also should have noted that the code blocks I put in the original description are indicative of what I'd like to write when creating suites. The only really important feature included that's lacking in JUnit 4 is that @BeforeSuite/@BeforeAll and @AfterSuite/@AfterAll can be nested based on the discovery hierarchy.

[mlevison]

I will make another attempt to explain why I think there are risks/issues with the name. I like the concept, but the linkage to the old names causes risk.

All of my work is consulting work in organizations. Frequently when I sit down with teams I get a chance to see their code. In too many cases I see people who have JUnit 3 style tests and suites in JUnit 4. When prompted they're not aware that this is no longer the preferred way.

What I have loved about JUnit 5 is that by and large the right choices will be made by default. Re-using the suites word by default will make easier for people to accidentally mis-use them in world where people use their "intelli-sense" type technology, instead of reading the documentation and reflecting.

Cheers
Mark aka @mlevison

[sormuras]

Do you have a better name in mind, Mark?

[sbrannen]

@mlevison, I have to admit that I don't really understand your concerns.

I personally cannot think of a better name than "suite" for something that allows you to group together a collection of tests that should be run together.

In fact, the Suite support in JUnit 4 achieves exactly what we are describing here:

Using Suite as a runner allows you to manually build a suite containing tests from many classes.

We also use the term _suite_ in the documentation for the JUnitPlatform runner:

When used on a class that serves as a test suite...

So I would argue that the familiarity with _suites_ from JUnit 3, JUnit 4, and other testing frameworks will make the intended usage on the JUnit Platform quite clear.

[sbrannen]

@sbrannen I actually think we're exactly on the same page...

@smoyer64, glad to hear it!

I'll have to look into the details at a later date though.

[smoyer64]

@sbrannen When I said to "ignore the word suite", I wasn't implying that I didn't think that name was appropriate ... it definitely feels familiar and I think it would actually be confusing to migrating users to suddenly use a different term (my intent in that paragraph was to point out that suites are a "kind of" test container).

I'll see about cleaning up my suite-engine and getting it moved into junit-pioneer - perhaps we can use it as a safe way to beta-test before committing to a course of action in platform/jupiter.

[mlevison]

On Fri, Mar 24, 2017 at 10:37 AM, Christian Stein [e-mail]
wrote:

Do you have a better name in mind, Mark?

​No you expect intelligent thought. I didn't yet. Ideas (that I don't
like): collection, series, set, map​ or if we're attached to Suite
something more expressive like IntegrationSuite that makes it clear that
these are not intended for Unit tests.

Cheers
Mark

>

On Fri, Mar 24, 2017 at 10:37 AM, Christian Stein [e-mail]
wrote:

Do you have a better name in mind, Mark?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/junit-team/junit5/issues/744#issuecomment-289040253,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAImZsXAUEHKkIlABffel3vXKWH562d9ks5ro9UcgaJpZM4Ml3IL
.

--

[image: headshot-square-300x300]
http://www.flickr.com/photos/36331075@N00/9674877046/ Mark Levison | 1
[phone] | Twitter https://twitter.com/mlevison | LinkedIn
http://ca.linkedin.com/in/marklevison | Facebook
https://www.facebook.com/agilepainrelief
Certified ScrumMaster Training: Vancouver
http://agilepainrelief.com/courses/vancouver | Edmonton
http://agilepainrelief.com/courses/edmonton | Ottawa
http://agilepainrelief.com/courses/ottawa | Montreal
http://agilepainrelief.com/courses/montreal | Toronto
http://agilepainrelief.com/courses/toronto
Certified Product Owner & Private Training also available ~ Our Training
Schedule http://agilepainrelief.com/courses/certified-scrum-agile-training
Agile Pain Relief Consulting http://agilepainrelief.com/ | Notes from a
Tool User http://agilepainrelief.com/notesfromatooluser
Proud Sponsor of Agile Tour Gatineau Ottawa http://goagiletour.ca/ and Agile
Coach Camp Canada http://agilecoachcampcanada.wordpress.com/

[mlevison]

On Fri, Mar 24, 2017 at 10:57 AM, Sam Brannen [e-mail]
wrote:

@mlevison https://github.com/mlevison, I have to admit that I don't
really understand your concerns.

I personally cannot think of a better name than "suite" for something that
allows you to group together a collection of tests that should be run
together.

In fact, the Suite support in JUnit 4 achieves exactly what we are
describing here:

Using Suite as a runner allows you to manually build a suite containing
tests from many classes.

​My point is that some years after the appearance I JUnit 4 - I still ​find
people writing their tests as if they were using JUnit 3. Not because this
was desirable, but because they didn't know that they didn't need suites
anymore. Often when I point out that they're not longer needed they tidy up.

My original concern, was that by reusing the name you risk the same cargo
cult habits with JUnit 5 as I've seen before.

Its cool if you disagree, am I at least making sense this time?

Cheers
Mark

[RupprechJo]

I represent a group of companies with about 100 Java Developer, written tenthousands of tests with JUnit. And for integration tests, suites (or whatever word you prefer for that) are essentially to have migration path to JUnit5. If you have long running preperation of integration tests, you cannot do this for every test class. To get predictable results, you sometimes need ordering test classes in suites. And I think JUnit Jupiter needs a native Suite definition without using the JUnit4 Runner!

[marcphilipp]

This issue won't make it into 5.0, but we're certainly considering it for 5.1.

[sbrannen]

FYI: I added a link to the related Eclipse IDE issue to this issue's description.

[sormuras]

Working on the "Jupiter-only" solution #456 with proposal #1143 I tend to agree that this suite described here is the better approach.

My vision extends the one presented by Sam at https://github.com/junit-team/junit5/issues/744#issuecomment-289029181 as it lifts the state of LauncherDiscoveryRequestBuilder to a first-class citizen. Let the end-user configure a "test run" programmatically. I envision a use-case like - all names are provisional:

class CustomTestSuites {
  @TestSuiteFactory
  LauncherDiscoveryRequest smokeTests() { ... }

  @TestSuiteFactory
  LauncherDiscoveryRequest regressionTests() { ... }

  @TestSuiteFactory
  LauncherDiscoveryRequest unitTests() { ... }

  @TestSuiteFactory
  LauncherDiscoveryRequest integrationTests() { ... }

  @TestSuiteFactory
  LauncherDiscoveryRequest systemTests() { ... }
}

Implementing the example described in the initial issue could look like:

@TestSuiteFactory
LauncherDiscoveryRequest acceptanceTests() {
  CustomTestReporter reporter = new ...
  return LauncherDiscoveryRequestBuilder.request()
      .displayName("Acceptance Test Suite")
      .beforeSuite(reporter::setUp)
      .afterSuite(report::tearDown)
      .scanClasspath()
      .selectors(
         selectSuite(this::databaseAcceptanceTests),
         selectSuite(this::loggingAcceptanceTests),
         selectSuite(this::userAcceptanceTests)
      )
      .filters(
         includeEngines("junit-jupiter")
      )
      .configurationParameter("all-test-are-running", "true")
      .build();
}

@TestSuiteFactory
LauncherDiscoveryRequest databaseAcceptanceTests() {
  Database database = new ...
  return LauncherDiscoveryRequestBuilder.request()
      .displayName("Database Acceptance Test Suite")
      .beforeSuite(database::setUp)
      .afterSuite(database::tearDown)
      .scanClasspath()
      .filters(
         includeTags("integration", "database"),
         includeClassNamePatterns(".*IT$")
      )
      .build();
} 

The annotation-based "configuration" would be an alternate way to configure custom test suites.

[sormuras]

@sebersole -- would the programmatic/declarative solution proposed here fit your needs as well?

[RupprechJo]

It is more detailled than we need, but it can work for migrating the Junit4 Suites we have

[sebersole]

@sormuras Give me a bit to wrap my head around.

[sebersole]

@sormuras To make sure I understand you... you are asking about this in relation to my desire to have before/after suite callbacks?

Personally I think this is a bit heavy handed for what I want/need. The primary use case I have is something @smoyer64 explicitly mentioned as well, namely setting up connection pools. In Hibernate we have found that creating/closing these takes a surprisingly significant portion of the overall test runs. Currently for each test class we create a pool and destroy it after. Obviously creating it once for the entire test run would be ideal - with the ability to integrate that with TestExecutionExceptionHandler (it would be safest to recreate the pool on any exceptions).

In fact I'd ideally really just like to be able to do this pool creation lazily when a test is recognized as needing it (however that happens) and then clean it up afterwards (or on exception, as noted above). That approach just needs an after-suite callback.

[michaelajr]

Hi. Just an end-user here. Figured I'd give my 2 cents. Hope that is ok.

First, my set up. I use Maven/Surefire to run tests. I configure SureFire to look for *Test.java files, and I configure it to use the junit-platform-surefire-provider. I then depend on junit-jupiter-engine in my tests dependencies. (I am NOT using Junit4 and the @RunWith(JUnitPlatform.class) annotation.)

So. All I want is the suite-api annotations to just work. I can then use the standard naming conventions for my top-level test class - and include specialized test classes (named without *Test.java so they are not included in SureFire test run). Something link this:

@ExtendWith(TestTimer.class)
@SelectClasses({LinkedListTest_add.class, LinkedListTest_remove.class})
public class LinkedListTest {

    @BeforeAll
    public void beforeAll() {
        ...
    }

    @AfterAll
    public void afterAll() {
        ....
    } 
}

This requires no new annotations or naming conventions. It just implements the suite-api annotations. Hope that makes sense. Thanks.

[RupprechJo]

The idea of michaelajr is completely enough

[sbrannen]

The idea of michaelajr is completely enough

Please keep in mind that frameworks are not developed or designed to suit the needs of an individual but rather for the community at large.

Thus, although the proposal by @michaelajr is certainly sufficient for some use cases, it would not suffice for users who need full control of the suite.

Furthermore, our current goal is to support suites on the Platform in general, not just within the Jupiter TestEngine.

[sbrannen]

Hi. Just an end-user here. Figured I'd give my 2 cents. Hope that is ok.

@michaelajr, yes, of course. That's totally _ok_. We welcome input from the community. 😉

This requires no new annotations or naming conventions.

That's not entirely true.

1. You in fact created your own naming conventions: *Test and *Test*. 😛
2. Your proposal does not introduce new annotations, but it does introduce a dependency on the platform suite annotations within the JUnit Jupiter engine, which is currently not our goal.

In any case, fear not: we'll do our best to come up with a convenient programming and runtime model for everyone.

[michaelajr]

@sbrannen Just to clarify, naming test classes *Test.java (or Test*.java) is convention for Maven/SureFire. Nothing new there. SureFire will pick them up automatically. By NOT using that convention for my method-level test classes (not naming them as Test*.java or *Test.java), they will not be picked up by Maven/SureFire. Then, using the suite-api annotations, I can include the method-level tests.

I agree that other use-cases should be thought through, but I encourage you to code the capability, not a test convention, and allow the users to compose their own suite solutions.

And finally, the ability to compose suites (using the suite-api annotations), AND still have the ExtendWith annotation, is super important. That's why I added that in my example. Platform vs Juniper, I don't know, but that is the functionality I will be looking for as user. Hope this helps.

M

[ps4os]

As mentioned at #1195 we would need an approach to mark Suite-Class/es with Meta-Annotations based on Tags (white- and black-listing). In our case we have an approach for JUnit 4 which is based on Meta-Annotations and therefore not just Text-Tags. Due to the current API of JUnit 5 we can't upgrade from JUnit 4 and I hope that it will be possible at some point.

[paul-nelson-baker]

Hi, I was asked to chime in by @marcphilipp a while back on stackoverflow. I'm not 100% sure how my use case fits exactly with this particular issue. As I glance through it, I see that this is more specifically looking at annotations to programmatically drive suite building, while I'm looking for a modern JUnit 5 equivalent for TestNG's xml suite. Here is the feature request with more specific details: https://github.com/junit-team/junit5/issues/1243

[sbrannen]

In our case we have an approach for JUnit 4 which is based on Meta-Annotations and therefore not just Text-Tags. Due to the current API of JUnit 5 we can't upgrade from JUnit 4 and I hope that it will be possible at some point.

@ps4os, why don't you just meta-annotate your meta-annotations with @Tag from JUnit Jupiter?

That should allow for seamless interoperability between JUnit 4 and JUnit Jupiter. 😉

[arturdryomov]

Just to clarify — is this gonna help with test suites discovery by IDE and other launchers? Especially using custom engines. Let me elaborate on that.

Let’s take Spectrum as an example. Spectrum tests barely look like classic JUnit tests — they do not have any methods annotated using the @Test annotation, everything is completely custom. At this point the single thing helping IDE to recognize Spectrum suites as actually suites is the @RunWith annotation. Removing it means that IDE sees the suite as a simple class and does not show any UI indication that it can be run (no Run buttons neither in context menu or in the editor itself). JUnit 5 eliminates runners concept, replacing them with test engines. I. e. there will be no @RunWith annotation if Spectrum was migrated to JUnit 5. In other words, IDE will not see a suite as a test suite.

The proposal seems like it can solve this by using @Suite and @IncludeEngines annotations, but I’m a bit confused if it is gonna help launchers with custom engines, so I’ve decided to ask just in case.

[sbrannen]

The proposal seems like it can solve this by using @Suite and @IncludeEngines annotations, but I’m a bit confused if it is gonna help launchers with custom engines, so I’ve decided to ask just in case.

Yes, a declarative @Suite class could select test classes specific to your custom engine via @SelectClasses.

That would allow the tests to be run within the IDE or build as a _suite_; however, that would not help the IDE to display a "run as test" icon or the like for an individual test class.

To make that possible, you could create a custom _marker annotation_ (e.g., @SpectrumTest) that is meta-annotated with @org.junit.platform.commons.annotation.Testable. @Testable exists exactly for that purpose.

Note that @IncludeEngines would only be necessary if you _only_ wanted to include a particular engine. Otherwise, all engines in the classpath would be used automatically.

[Syneil]

I'm looking for a way to execute the same test class with runtime-generated inputs. Is this ticket appropriate?

I.e., for some TestClass with the usual @BeforeEach and @Test etc methods, I want to define a TestClass(SomeType param, ...) constructor and then run the test with runtime-generated instances of SomeType. Somewhat similar to the JBehave Examples for scenarios.

[marcphilipp]

@Syneil That sounds more like #871.

[marcphilipp]

Experimental branch:
https://github.com/junit-team/junit5/tree/experiments/platform-suites

[sormuras]

The experimental branch is available via @jitpack and you can use it with Gradle (4.6+) like:

repositories {
    mavenCentral()
    maven { url 'https://jitpack.io' }
}

dependencies {
    testCompile 'com.github.junit-team.junit5:junit-jupiter-api:experiments~platform-suites-SNAPSHOT'
    testCompile 'com.github.junit-team.junit5:junit-platform-suite-api:experiments~platform-suites-SNAPSHOT'
    testRuntime 'com.github.junit-team.junit5:junit-jupiter-engine:experiments~platform-suites-SNAPSHOT'
    testRuntime 'com.github.junit-team.junit5:junit-platform-engine:experiments~platform-suites-SNAPSHOT'
    testRuntime 'com.github.junit-team.junit5:junit-platform-launcher:experiments~platform-suites-SNAPSHOT'
}

test {
    useJUnitPlatform()
}

[aaronhonra]

hi guys, from the overview mentioned above, @Suite annotation exists. What package does this reside exactly? Can't seem to find it junit-platform-suite-api from 1.2 Junit platform version. Is this still in experimental stage?

[sormuras]

You need to browse and use the experimental branch: https://github.com/junit-team/junit5/tree/experiments/platform-suites ... which might be merged into 1.3-SNAPSHOT, soon.

Artifact: junit-platform-suite-api
Package: org.junit.platform.suite.api
Compilation Unit: Suite.java

[phoenix384]

From the referenced issue #1520: It would be great, if nested suites would be supported:

@RunWith(JUnitPlatform.class)
@SelectClasses(SimpleTest.class)
public class Suite1 {
}

@RunWith(JUnitPlatform.class)
@SelectClasses(Suite1.class)
public class Suite2 {
}

So that executing Suite2 would execute Suite1 and its contents.

[fzakaria]

Feature request:

I would like to write a single test in lets say module "blackbox-tests" and normally have a parameter client resolve with an endpoint to my real running service.

It would be great if I could programmatically then bind the ParameterResolver to another implementation where it the extension will startup my service in the same JVM and perform the same suite of tests locally.

[sbrannen]

@fzakaria, if you want to run the same "test class" in JUnit Jupiter twice, with different parameterization each time, it sounds to me like #878 would be a better fit than "suites".

[fzakaria]

@sbrannen I actually want to give it different parametrization depending on the context -- ideally i would be nice to register maybe ExtendWith at the Suite level to apply to all test classes (something like that)

[sbrannen]

@fzakaria, you could always use @RegisterExtension instead of @ExtendWith and then programmatically _decide_ which extension to register or how to configure your extension. A static factory method or builder API for your extension would work nicely.

[kotse]

I get a timeouts from all artifacts from jitpack when I try to set up a very simple project to use this.

Error here shows the first only, but others fail as well.

Could not resolve com.github.junit-team.junit5:junit-jupiter-api:experiments\~platform-suites-SNAPSHOT.
Required by:
project :
Could not resolve com.github.junit-team.junit5:junit-jupiter-api:experiments~platform-suites-SNAPSHOT.
Unable to load Maven meta-data from https://jitpack.io/com/github/junit-team/junit5/junit-jupiter-api/experiments~platform-suites-SNAPSHOT/maven-metadata.xml.
Could not get resource 'https://jitpack.io/com/github/junit-team/junit5/junit-jupiter-api/experiments~platform-suites-SNAPSHOT/maven-metadata.xml'.
Could not GET 'https://jitpack.io/com/github/junit-team/junit5/junit-jupiter-api/experiments~platform-suites-SNAPSHOT/maven-metadata.xml'.
Read timed out

The experimental branch is available via @jitpack and you can use it with Gradle (4.6+) like:

repositories {
    mavenCentral()
    maven { url 'https://jitpack.io' }
}

dependencies {
    testCompile 'com.github.junit-team.junit5:junit-jupiter-api:experiments~platform-suites-SNAPSHOT'
    testCompile 'com.github.junit-team.junit5:junit-platform-suite-api:experiments~platform-suites-SNAPSHOT'
    testRuntime 'com.github.junit-team.junit5:junit-jupiter-engine:experiments~platform-suites-SNAPSHOT'
    testRuntime 'com.github.junit-team.junit5:junit-platform-engine:experiments~platform-suites-SNAPSHOT'
    testRuntime 'com.github.junit-team.junit5:junit-platform-launcher:experiments~platform-suites-SNAPSHOT'
}

test {
    useJUnitPlatform()
}

[sormuras]

Works for me. For example https://jitpack.io/com/github/junit-team/junit5/junit-jupiter-api/experiments~platform-suites-SNAPSHOT is resolved to https://jitpack.io/com/github/junit-team/junit5/junit-jupiter-api/experiments~platform-suites-r5.2.0-g2aa446a-83/index.html containing:

junit-jupiter-api-experiments~platform-suites-r5.2.0-g2aa446a-83-javadoc.jar
junit-jupiter-api-experiments~platform-suites-r5.2.0-g2aa446a-83-sources.jar
junit-jupiter-api-experiments~platform-suites-r5.2.0-g2aa446a-83.jar
junit-jupiter-api-experiments~platform-suites-r5.2.0-g2aa446a-83.pom
junit-jupiter-api-experiments~platform-suites-r5.2.0-g2aa446a-83.pom.md5
junit-jupiter-api-experiments~platform-suites-r5.2.0-g2aa446a-83.pom.sha1
maven-metadata-local.xml

[sormuras]

Which version of the JDK do you use to execute your Gradle build?

[kotse]

Which version of the JDK do you use to execute your Gradle build?

This was the issue, I was on Java 10.0.2, it is working with 9 :)

Thank you!

[sormuras]

Does it work with OpenJDK 11 as well?

[kotse]

Trying again with Java 9, 10 and 11, same code/gradle/machine/network/etc and it all works... Got a build working with maven as well. Something weird with jitpack I guess. Anyway - thank you sormuras for taking the time to look at this!

Now time to play with Suites, as this is blocking our full JUnit5 migration.

[jkimba3]

Will support for this issue be coming soon? Trying to decide if we just walk away from our TestSuites or wait with our fingers crossed that it will work sometime in the near future.

[sbrannen]

If by "soon" you mean sometime in 2019, then yes, that is the plan.

It may come in 5.5 but I think more realistically in 5.6.

[ratoaq2]

Just to add another use case for my need:
With declarative suites I could configure maven failsafe plugin to run suites in parallel using forks and each suite could run its tests sequentially (because we reuse the same spring context for each test suite).
That's a big use case for our current project which still uses JUnit 4

[phoenix384]

Now 5.6 is released and the release notes still say nothing about declarative suites. Any news on this?

[sbrannen]

No, there is unfortunately no update on the status of this other than that it is slated for the 5.7 backlog, which designates an interest in getting it implemented in time for 5.7 GA but without any guarantees for inclusion in 5.7.

[nicerloop]

Hi, I would like to use declarative suite, especially to launch different engines with different configuration parameters. I resurrected the experiments/platform-suites branch and ported it over 5.6.0 for my use, and would like to contribute to this feature for inclusion - see #2267. Can anybody give me guidance on all that is currently missing for this to be included?

[mpkorstanje]

Following up on the comments in #2295 I've implemented a prototype that seems to work for a very simple test case.

See: https://github.com/junit-team/junit5/pull/2416

Rather then making the execution of @Suite annotated classes an integral part of the JUnit Platform Launcher I've instead chosen to create a separate test engine that uses the JUnit Platform Launcher. The engine works by mapping the TestIdentifier used by the launcher to TestDescriptor used by the engine during discovery and in reverse during execution.

[mpkorstanje]

Made some improvements. Now with a working demo.

• https://github.com/mpkorstanje/junit-platform-suite-demo

[carlspring]

Hi,

Are there any updates on this?

[sbrannen]

@carlspring, you can follow the current PR (which is a work in progress) in #2416.