Junit5: Convenient way to use static Java methods in @Enabled*/@Disabled*

[szpak]

Idea

I would like to be able to easily use the logic placed in my static Java methods to determine if a given test(s) should be executed or not.

Rationale

In some cases it is convenient to delegate the logic in conditional test execution to custom Java code. A real life usage - determine if strong cryptography is available in JDK to disable all related tests and fail just one to inform external contributor in meaningful way that his/her configuration is not optimal (I know it is somehow obsolete as of Java 11 and using OpenJDK all around, but at the time it was a real issue for us with Oracle JDK).

In Spock it can be easily done with:

@IgnoreIf({ SomeUtil.isStrongEncryptionSupported() })
class BrokenJceInstallationWarningSpec extends Specification {
    ....

In JUnit 5 it should be achievable with a few lines of Java Script and @EnableIf (looking at the JUnit 5 documentation) which is somehow inconvenient. In addition @EnableIf has been deprecated recently.

API proposal

As Java is more limited that Groovy it doesn't seem to be possible to get that level of API:

@EnabledX(() -> MyUtil.callMyStaticMethodReturningBoolean())
// or
@EnabledX(MyUtil::callMyStaticMethodReturningBoolean)

Therefore, it would probably be needed to use String literals:

@EnabledX(method = "com.example.MyUtil.callMyStaticMethodReturningBoolean")
// or
@EnabledX(package = "com.example", method = "MyUtil.callMyStaticMethodReturningBoolean")

If package is omitted JUnit could try to assume the same package as enclosing class to try to resolve the method.

I would exclude an ability to pass method arguments or JUnit context to a called method.

Please note that I don't know the internal of JUnit 5, so some of the idea could be hard/problematic to implement. Feel free to choose some other (convenient) way to implement that feature.

BTW, @EnabledX is also a place holder which should be changed to something more meaningful.

Deliverables

• [x] Convenient API to use static Java methods in @Enabled*/@Disabled*
• [x] Documentation update

[iherasymenko]

@szpak

You may implement your own _ExecutionCondition_ placing all the necessary logic in there and registering it using _@ExtendWith_

You may also create a meta annotation like this to make it even cleaner.

There is no need to have a special static method, but I believe you can achieve this utilizing the aforementioned extension points.

[sormuras]

Although I favor the idea of having explicit ExecutionCondition implementations doing the job -- I also like the idea of supporting convenient ways to achieve the same job. That was the underlying motivation to implement script-based execution conditions in the first place.

There's prior-art: MethodSource in Jupiter Params already supports specifying methods by name.

So, let's discuss this idea further.

[marcphilipp]

Related to https://github.com/junit-pioneer/junit-pioneer/issues/1

[marcphilipp]

If we decide to do this, we should support the same syntax and semantics as we do for @MethodSource. If @TestInstance(PER_CLASS) is used, we should allow non-static local methods. As for the signatures we should support, the following come to mind:

• boolean method()
• boolean method(ExtensionContext)
• ConditionEvaluationResult method()
• ConditionEvaluationResult method(ExtensionContext)

Let's decide in our next team call next week.

[marcphilipp]

Team Decision: Let's support the first and last of the above signatures and resurrect @EnabledIf/@DisabledIf.

[marcphilipp]

@szpak Are you interested in submitting a PR for this one?

[juliette-derancourt]

Should we allow to pass several conditions to the @EnabledIf/@DisabledIf annotations? (either by taking a list of methods instead of a single one, or by allowing several occurences of the annotations)
Or do we consider that in this case one can create a new method that's going to aggregate the output of said conditions?

If we consider the first option (allowing several methods), we'll have to determine what rules to apply in case of contradictory outputs... and maybe even make it parametrable? Could be something like:

@EnabledIf(conditions = { ... }, operator = Operator.AND)

Another thought: should we allow the use of both @EnabledIf and @DisabledIf on the same test/container? If so, same question as above, how do we deal with contradictory results?

[sormuras]

Let's keep it as simple as possible in both cases:

• imho, a single method suffices. Let the user care about delegating to other methods.
• let's apply the same rules we have in place for the other @Enabled.../@Disabled... variants. IIRC, the first one that disables the execution of the container/test "wins". Subsequent conditions are not evaluated.

[sbrannen]

I agree with @sormuras on both accounts.

[sormuras]

@juliette-derancourt I collected the commits that removed the script-based conditional execution logic here: https://github.com/junit-team/junit5/issues/1902 -- perhaps you may resurrect some of those ideas and types.

[juliette-derancourt]

imho, a single method suffices. Let the user care about delegating to other methods.

Completely agree with that 👍

let's apply the same rules we have in place for the other @Enabled.../@Disabled... variants. IIRC, the first one that disables the execution of the container/test "wins". Subsequent conditions are not evaluated.

Indeed, seems coherent to me.

I collected the commits that removed the script-based conditional execution logic here: #1902 -- perhaps you may resurrect some of those ideas and types.

Thanks! Will take a look ;)

[juliette-derancourt]

One last thought: for the @DisabledIf annotation, I'm not quite sure what would be the expected behavior in the case of a method returning a ConditionEvaluationResult...

    @Test
    @DisabledIf("enabled")
    void test() {
        // Should this be run?
    }

    @Test
    @DisabledIf("disabled")
    void test2() {
        // And this?
    }

    static ConditionEvaluationResult enabled(ExtensionContext extensionContext) {
        return ConditionEvaluationResult.enabled("This test is enabled");
    }

    static ConditionEvaluationResult disabled(ExtensionContext extensionContext) {
        return ConditionEvaluationResult.disabled("This test is disabled");
    }

I'm not even sure this makes sense...
A ConditionEvaluationResult already decides if a test is "enabled" or "disabled", so the information provided by the presence of either @EnabledIf or @EnabledIf would be redundant (and even incoherent).

[marcphilipp]

Team decision: Let's limit the scope of this issue to supporting boolean return values only.

[szpak]

Thanks @juliette-derancourt!