Junit5: Introduce predefined @Disabled*/@Enabled* variants

[ttddyy]

Overview

I just happened to have a test only failing on MS Windows machines and thought about JUnit Jupiter.

I could write a custom extension implementing ContainerExecutionCondition and TestExecutionCondition, but having a predefined @Disabled annotation such as @DisabledIfWindows or @Disabled(condition = IsWindowsEvaluator.class) to extend disabling feature might be handy.

Related Issues

• #1247

Deliverables

• [x] system property matches a pattern
• [x] environment variable matches a pattern
• ❌ current day of week matches one of the supplied enums
  
  
  
  • The team decided not to publish @EnabledOnDayOfWeek and @DisabledOnDayOfWeek.
  
  
• [x] current OS matches one of the supplied enums
• [x] current JRE version matches one of the supplied enums
• [x] Check for user configuration errors. For example, if a user declares @EnabledOnOs({}), that should result in an exception instead of a disabled result.
• [x] Introduce integration tests for each condition.
• [x] Write JavaDoc for each annotation.
• [x] Document in User Guide.
• [x] Document in Release Notes.

[ttddyy]

I just realized I should write like this to skip the test on windows machines for my use case.
assumeFalse(System.getProperty("os.name").toLowerCase().contains("win"))

However, I still think _annotating_ such condition would be easier to detect than writing assumptions in body of the test methods.

[sbrannen]

@ttddyy, that's an interesting idea, and I in fact had a similar idea (hence the new subject).

I'm not sure that we will introduce such mechanisms in core JUnit 5, but I do imagine they will end up somewhere (perhaps in an add-on module or in an example project).

p.s. this would be similar to @IfProfileValue in Spring, only much more powerful. :wink:

[sbrannen]

We have recently received a recommendation for introducing an annotation like @DisabledUntil( /* date */).

So that is also something to keep in mind.

I'm currently thinking we should support the following use cases.

• system property matches a pattern
• environment variable matches a pattern
• current date/time matches some criteria

[smoyer64]

I wanted to add a use-case to this issue that was described during the "Metrics-Driven Continuous Delivery in Practice [TUT5941]" session at JavaOne.

The presenter's company (Dynatrace) has developed a custom @Ignore annotation which allows tests to be ignored only if the issue (in this case in Jira) it's testing against is still open. This seems like a pretty clever TDD trick too.

[sbrannen]

Yep, that's also a clever use case.

But something like that is very specific to a particular technology. So I don't foresee that being implemented as a generic solution within JUnit itself.

[smoyer64]

No ... but the "condition =" idea above would satisfy both this use case and the @DisableUntil use case.

[sbrannen]

Looks like this was accidentally auto-closed via a commit...

Since we have not yet introduced predefined @DisabledIf/@EnabledIf variants, I am reopening this issue.

[sbrannen]

Tentatively slated for 5.1 RC1 and "team discussion" to decide if we want to implement a few of the most basic ones before 5.1 GA.

[sbrannen]

Update: added _Candidates for predefined @DisabledIf/@EnabledIf Variants_ section to issue description.

[sbrannen]

experimentation/POC _in progress_

[sbrannen]

Proposal

The _proof of concept_ proposal, as can be seen in the feature branch, currently includes the following pairs of annotations.

JVM System Properties

@EnabledIfSystemProperty(named = "key", matches = "regex")

```java
@DisabledIfSystemProperty(named = "key", matches = "regex")

#### Operating System Environment Variables

```java
@EnabledIfEnvironmentVariable(named = "key", matches = "regex")

```java
@DisabledIfEnvironmentVariable(named = "key", matches = "regex")

#### Days of Week

```java
@EnabledIfToday(is = MONDAY)

```java
@DisabledIfToday(is = { SATURDAY, SUNDAY })

#### Composed Annotations

Each of these annotations can naturally be used as a meta-annotation to create custom composed annotations such as `@EnabledOnWeekends`:

```java
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@EnabledIfToday(is = { SATURDAY, SUNDAY })
public @interface EnabledOnWeekends {
}

Feedback Requested

Feedback on the naming and style is welcome!

_Please do not comment on the implementation, tests, or lack of documentation at the moment._

[sbrannen]

At the moment I have aimed for consistency with regard to naming and style; however, some of the proposed names are rather long.

Thus, I've been considering alternatives like:

• @EnabledIfSysProp(named = "key", matches = "regex")
• @EnabledIfEnvVar(named = "key", matches = "regex")
• @EnabledOnDay(MONDAY)

[sormuras]

I like the long and descriptive names better. The can be read as a complete sentence.

The short ones could also be "senten(ce)-ized":

• @EnabledIfSystem(property = "key", matches = "regex")
• @EnabledIfEnviroment(variable = "key", matches = "regex")

Mh... 🤔

[sbrannen]

p.s. keep in mind that the current proposal attempts to create a _DSL_ using Java annotations, but users can still declare attributes in the unintended order. For example, it's perfectly valid to declare @EnabledIfSystemProperty(matches = "regex", named = "key"), but it does look quite odd.

[sbrannen]

I like the long and descriptive names better. The can be read as a complete sentence.

OK

The short ones could also be "senten(ce)-ized":

• @EnabledIfSystem(property = "key", matches = "regex")
• @EnabledIfEnviroment(variable = "key", matches = "regex")

That reads fine, but I'm hesitant to consume the "System" and "Environment" _namespaces_ for this purpose, since we may later want to support other things related to a system or environment.

[jbduncan]

@sbrannen I admit that I struggle to see how using the words "System" and "Environment" for the purpose of these annotations would prevent their use for other things. Would you kindly clarify things for me? :thinking:

[sbrannen]

Sure...

If we were to introduce annotations named @EnabledIfSystem(...) and @EnabledIfEnvironment(...), currently dedicated to system _properties_ and environment _variables_, respectively, those annotations would conflict with any other annotations we may decide to introduce later that also pertain to "system something" and "environment something". So in that sense, based on the names of the annotations, they effectively _consume_ those namespaces and prohibit easy support for other "somethings".

I'm not saying that we intend to introduce other "somethings", but I cannot foresee the future.

In terms of API design it's generally not a good idea to consume an entire namespace prematurely.

Of course, I may be overthinking this, but I prefer to err on the side of caution when it comes to OSS APIs. 😉

p.s. if that was too abstract for you, let me put it like this: we cannot have two pairs of annotations with different use cases that are both called @EnabledIfSystem(...) and @EnabledIfEnvironment(...) (at least not without introducing multiple attributes in those annotations and then having to build up tons of if-else blocks in the implementations of the conditions that support those use cases).

[jbduncan]

@sbrannen Thanks for the detailed explanation! I better understand your concerns now.

I can see now that we may indeed want to use @EnabledIfSystem and @EnabledIfEnvironment for different interpretations of the words "system" and "environment" for unknown reasons in the future. So I have no problems with rejecting those names and going for ones which are more specific like @EnabledIfSysProp or @EnabledIfSystemProperty instead.

[sbrannen]

Proposal for OSes

The _proof of concept_ proposal, as can be seen in the feature branch, now includes the following pair of annotations.

Operating Systems

@EnabledIfOs(is = WINDOWS)

@DisabledIfOs(is = { LINUX, MAC })

Feedback Requested

Feedback on the naming and style is welcome!

_Please do not comment on the implementation, tests, or lack of documentation at the moment._

[marcphilipp]

How about @EnabledOnOs(WINDOWS) /@DisabledOnOs({LINUX, MAC})?

[sbrannen]

I'm actually fine with those, @marcphilipp.

In fact, that's how I was going to do it originally, but then we started along the path of being totally consistent across all @EnabledIf* and @DisabledIf* annotations.

So I've just been sticking with consistency for the sake of consistency.

But... we can certainly decide against consistency in favor of UX if we so desire. 😉

[sbrannen]

Update

I've renamed two of the annotation pairs and renamed their is attributes to value.

Operating Systems

@EnabledOnOs(WINDOWS)

```java
@DisabledOnOs({ LINUX, MAC })

#### Days of Week

```java
@EnabledOnDayOfWeek(MONDAY)

```java
@DisabledOnDayOfWeek({ SATURDAY, SUNDAY })

#### Composed Annotations

Each of these annotations can naturally be used as a meta-annotation to create custom composed annotations such as `@EnabledOnWeekends`:

```java
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@EnabledOnDayOfWeek({ SATURDAY, SUNDAY })
public @interface EnabledOnWeekends {
}

[sbrannen]

FYI: I have moved the "candidates" to the _Deliverables_ section of this issue.

[marcphilipp]

Team Decision: Only introduce @Enabled*/@Disabled* for system properties, environment variables, OSes, and JVM versions.

[sbrannen]

Proposal for Java Runtime Environment versions

The _proof of concept_ proposal, as can be seen in the feature branch, now includes the following pair of annotations.

JRE Versions

@EnabledOnJre(JAVA_8)

@DisabledOnJre({ JAVA_9, JAVA_10 })

Feedback Requested

Feedback on the naming and style is welcome!

_Please do not comment on the implementation, tests, or lack of documentation at the moment._

[sbrannen]

This issue has been resolved in latest commits pushed to master.