Junit5: Introduce global timeouts for tests

[arcuri82]

Feature Request

Currently, in JUnit 4 it is not possible to have global timeouts for the tests. The Timeout rule does not guarantee that any @After method will be called. See

https://github.com/junit-team/junit/issues/1231#issuecomment-167437481

and the recently updated

https://github.com/junit-team/junit/wiki/Timeout-for-tests

It would be good to have a global timeout in JUnit Jupiter. Even better, it would be great if it would be possible to set different global timeouts for @BeforeEach @Test, and @AfterEach, which could be overridden (if needed) directly in the test files via annotations.

Related Issues

• #20
• #1246

[deepakab03]

In JUnit 4, if a Test super class specifies a Timeout Rule, then any Test subclass automatically inherits that and it becomes a kind of global timeout - as long as the Test super class is extended of course. This is very much needed in JUnit 5 as a replacement for this JUnit 4 feature to catch rogue tests that keep running and tie up the local build / build server unnecessarily as well as catch actual bugs.
This global timeout could also be tuned to different values from the subclass for all the tests in that class or for specific tests.

[caduandrade]

I can't imagine using "assertTimeoutPreemptively" all over the code. Global timeout is very needed.

[sbrannen]

FYI: I posted an answer on Stack Overflow outlining how to implement a non-preemptive global timeout extension here: https://stackoverflow.com/a/50233807/388980

[FlorianSchaetz]

While this is a good step, a real timeout would be great, since it would, for example, prevent hanging tests from hanging on the build servers, etc. - which is why we used a timeout rule in Junit4 for all tests, just to make sure they would never hang and block a resource there.

Surrounding all test code with assertTimeoutPreemptively is, as everyone hopefully agrees, not really a good solution. Honestly, it's not even a good solution for one method, since it's clunky. An extension and annotation would be cleaner. Currently, this is the thing that worries me most for switching from JUnit4 to 5 (since we had multiple situations - for various reasons - in the past where tests did not finish properly... We could prevent that with the Timeout rule, but now...?).

[sbrannen]

Tentatively slated for the 5.3 backlog for _team discussion_

[sbrannen]

FYI: #20 is potentially a _blocker_ for this issue, but I have not yet labeled this issue as blocked since we have not yet decided how we wish to implement global timeouts.

[danielFesenmeyer]

I agree with @FlorianSchaetz and @deepakab03. We like to cancel our integration tests if they take too much time (e.g. waiting for an event from a queue). We don't want to surround all of our tests with assertTimeoutPreemptively().
Currently we use JUnit4 with the Timeout rule for this purpose, When trying to migrate that to JUnit5, I found this issue here. We deferred the migration now, hoping that this issue will be fixed soon.

[ehiggs]

This is a potential blocker for Hadoop's migration to JUnit5.

https://issues.apache.org/jira/browse/HADOOP-14693

[yb172]

Any updates on this one? Having assertTimeoutPreemptively(timeout, () -> { ... }); doesn't look like a good solution...

[marcphilipp]

It's on the agenda for 5.5 together with #20.

[sambsnyd]

@yb172 in the meantime, if you're using Gradle 5.0+ you can set timeouts on your Test tasks to prevent command line/CI builds from hanging indefinitely.

[sbrannen]

@yb172 in the meantime, if you're using Gradle 5.0+ you can set timeouts on your Test tasks to prevent command line/CI builds from hanging indefinitely.

Good to know. Thanks for sharing!

[marcphilipp]

_in progress_

[sormuras]

_Almost finished..._ Go, Marc, go! 🚴

[steveloughran]

Thank you! We in the Hadoop team owe you a beer or ice cream of your choice

[sormuras]

Beer-flavored ice cream?! 🍺🍨

[steveloughran]

if you can find some

[praenubilus]

Is there anyway we can have preemptive global timeout. The current global time out will hang if the program runs infinitely. The old timeout rule works in Junit4 is really helpful to abort the specific unit test in some scenarios.

[sbrannen]

Is there anyway we can have preemptive global timeout.

That's already the case.

The current global time out will hang if the program runs infinitely. The old timeout rule works in Junit4 is really helpful to abort the specific unit test in some scenarios.

If you have a use case that JUnit Jupiter does not work with, please open a new issue with further details.