Bazel: JUnit 5 support

Created on 15 Nov 2018 · 15Comments · Source: bazelbuild/bazel

ATTENTION! Please read and follow:

if this is a _question_ about how to build / test / query / deploy using Bazel, ask it on StackOverflow instead: https://stackoverflow.com/questions/tagged/bazel

if this is a _discussion starter_, send it to [email protected]

if this is a _bug_ or _feature request_, fill the form below as best as you can.

Description of the problem / feature request:

JUnit v5 (Jupiter) is a significant new version of JUnit, bringing many new features and changes.

Bazel should natively support JUnit v5 (Jupiter) tests. JUnit v5 annotations and assertions cannot be run in v4.

Feature requests: what underlying problem are you trying to solve with this feature?

"bazel test" should run for JUnit5 tests

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Replace this line with your answer.

What operating system are you running Bazel on?

Windows.

What's the output of `bazel info release`?

0.19.1

Have you found anything relevant by searching the web?

https://github.com/JeffreyFalgout/bazel-junit5.git
It fails with all kind of errors.

Any other information, logs, or outputs that you want to share?

Will upload sample on request.

P3 team-Rules-Java feature request

Source

BlackIsTheNewBlack

👍32 👀1

Most helpful comment

Any update on a native solution for this?

raz-canva on 26 Mar 2019

👍17

All 15 comments

P1 because I consider this important feature.

irengrig on 16 Nov 2018

No can do, at least not with the urgency that P1 bugs need although I do agree that we should do this eventually.

lberki on 16 Nov 2018

Good news: The JUnit 5 team have a sample for using JUnit 5 in Bazel! :)

https://github.com/junit-team/junit5-samples/tree/master/junit5-jupiter-starter-bazel

jbduncan on 18 Nov 2018

👍5

Sure, it's not native, but it at least (should) fix the problem that the other existing example, https://github.com/JeffreyFalgout/bazel-junit5.git, doesn't work.

jbduncan on 18 Nov 2018

Thank you @irengrig and @lberki
Thanks @jbduncan for the tip

BlackIsTheNewBlack on 19 Nov 2018

❤1

Any update on a native solution for this?

raz-canva on 26 Mar 2019

👍17

The JUnit team's Bazel starter project (https://github.com/junit-team/junit5-samples/tree/master/junit5-jupiter-starter-bazel) only appears to work with the Java 8 Bazel toolchain.

This works fine (as does not specifying java_toolchain):
bazel test --java_toolchain=@bazel_tools//tools/jdk:toolchain_java8 //...

Specifying any other toolchain causes the ConsoleLauncher to run, but the test log shows that it doesn't find any tests (so it reports the tests as passing, even when there are failures):
bazel test --java_toolchain=@bazel_tools//tools/jdk:toolchain_java11 //...
(It's the same problem with toolchain_java9 and toolchain_java10).

The log output shows that it ran, but found no tests:
```>> cat bazel-testlogs/src/test/java/com/example/project/junit5-jupiter-starter-
exec ${PAGER:-/usr/bin/less} "$0" || exit 1

Executing tests from //src/test/java/com/example/project:junit5-jupiter-starter-bazel-test

Thanks for using JUnit! Support its development at https://junit.org/sponsoring
╷
└─ JUnit Jupiter ✔
Test run finished after 27 ms
[ 1 containers found ]
[ 0 containers skipped ]
[ 1 containers started ]
[ 0 containers aborted ]
[ 1 containers successful ]
[ 0 containers failed ]
[ 0 tests found ]
[ 0 tests skipped ]
[ 0 tests started ]
[ 0 tests aborted ]
[ 0 tests successful ]
[ 0 tests failed ]
```

Any clues as to what could be causing this? Maybe org.junit.platform.console.ConsoleLauncher's classpath doesn't have the test classes on it (they're declared in deps)?

martindow on 18 Nov 2019

Update on my issue above:
This appears to be a problem with the homebrew installation of bazel, version 0.29.0-homebrew. I've since used bazelisk to install version 0.29.0 directly and it's running the tests as expected.
I have no idea what the difference is between the two...

martindow on 18 Nov 2019

Might be useful to clarify for any other Bazel newbies like myself what the java_junit5_test rule in junit5.bzl (https://github.com/junit-team/junit5-samples/tree/master/junit5-jupiter-starter-bazel) is doing under the hood. It:

Declares the required JUnit5 Maven artifacts in WORKSPACE and
Creates a java_test rule with the main class set to org.junit.platform.console.ConsoleLauncher.

If you're using the new maven_install rule (https://github.com/bazelbuild/rules_jvm_external) and want to inline these steps into your project, then the following changes will cause the sample project to execute:

Add the maven_install rule to WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
RULES_JVM_EXTERNAL_TAG = "2.10"
RULES_JVM_EXTERNAL_SHA = "1bbf2e48d07686707dd85357e9a94da775e1dbd7c464272b3664283c9c716d26"
http_archive(
    name = "rules_jvm_external",
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    sha256 = RULES_JVM_EXTERNAL_SHA,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)
load("@rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
    artifacts = [
        "org.junit.jupiter:junit-jupiter-api:5.5.0",
        "org.junit.jupiter:junit-jupiter-engine:5.5.0",
        "org.junit.jupiter:junit-jupiter-params:5.5.0",
        "org.apiguardian:apiguardian-api:1.0.0",
        "org.opentest4j:opentest4j:1.1.1",
        "org.junit.platform:junit-platform-commons:1.5.0",
        "org.junit.platform:junit-platform-console:1.5.0",
        "org.junit.platform:junit-platform-engine:1.5.0",
        "org.junit.platform:junit-platform-launcher:1.5.0",
        "org.junit.platform:junit-platform-suite-api:1.5.0",
    ],
    repositories = [
        "http://central.maven.org/maven2",
    ],
)

Then change the test BUILD file to directly declare the java_test rule like this:

package(
    default_visibility = ["//src/test:__subpackages__"],
)

java_test(
    name = "junit5-jupiter-starter-bazel-test",
    srcs = glob([
        "*.java",
    ]),
    use_testrunner = False,
    main_class = "org.junit.platform.console.ConsoleLauncher",
    args = ["--select-package", "com.example.project"],
    deps = [
        "//src/main/java/com/example/project:junit5-jupiter-starter-bazel",
        "@maven//:org_junit_jupiter_junit_jupiter_api",
        "@maven//:org_junit_jupiter_junit_jupiter_engine",
        "@maven//:org_junit_jupiter_junit_jupiter_params",
        "@maven//:org_junit_platform_junit_platform_suite_api",
        "@maven//:org_apiguardian_apiguardian_api",
        "@maven//:org_opentest4j_opentest4j",
    ],
    runtime_deps = [
        "@maven//:org_junit_platform_junit_platform_commons",
        "@maven//:org_junit_platform_junit_platform_console",
        "@maven//:org_junit_platform_junit_platform_engine",
        "@maven//:org_junit_platform_junit_platform_launcher",
        "@maven//:org_junit_platform_junit_platform_suite_api",
    ],
)

Might be useful to someone...

martindow on 18 Nov 2019

👍13

@martindow Any luck experimenting with --reports-dir argument for getting a nice report of tests for Jenkins (and other CI systems)?