Junit5: Document compatibility with JUnit 4 TestRules

[smoyer64]

The following is an analysis of which JUnit 4 TestRule functionality is provided by JUnit 5. This should be documented in an addendum to the User Guide.

• [ ] DisableOnDebug - partially accomplished using DisableIf/EnableIf, partially allowed via the use of @Tags as described in sections 3.7 Tagging and Filtering of the user guide. But is it completely covered? I was completely off base with this assessment. This @TestRule allows you to disable a test rule if the JVM is in debug mode and has absolutely nothing to do with deciding what tests should be run.
• [x] ExpectedException - use Assertions.expectThrows() as described in section 3.4 Assertions of the User Guide.
• [ ] ExternalResource
  
  
  
  • [ ] TemporaryFolder - Provide TemporaryDirectory which is used for internal JUnit 5 tests as part of https://github.com/junit-team/junit5-samples. See https://github.com/junit-team/junit5-samples/issues/4.
  
  
• [x] RuleChain - JUnit 5 extensions enable via @ExtendWith are executed in declaration order.
• [ ] Stopwatch - partially provided via the TimingExtension in section 5.6.1 Before and After Test Execution Callbacks of the user guide. See this comment: https://github.com/junit-team/junit5-samples/issues/4#issuecomment-190330403. Reporting of the result state could be accomplished by injecting TestReporter via parameter resolution (using @BeforeAll?) but there are potentially use cases that can't be handled.
• [x] TestWatcher
  
  
  
  • [x] TestName - inject TestInfo into the test using parameter resolution.
  
  
• [x] Timeout - JUnit 4 allows the timeout to be specified via @Test. Is there an equivalent in JUnit 5? For performance testing, it would also be nice to have the test fail if it exceeded a threshold.
• [x] Verifier
  
  
  
  • [x] ErrorCollector - use Assertions.assertAll() as described in section 3.4 Assertions of the User Guide.
  
  

Related Issues

• 169

  Related Blog Posts

• http://www.codeaffine.com/2016/04/06/replace-rules-in-junit5/

• http://blog.codefx.org/libraries/junit-5-conditions/

[sbrannen]

Great list, and... thanks for doing all of the research!

[sbrannen]

This is related to #169.

[sbrannen]

Related blog posts:

• http://www.codeaffine.com/2016/04/06/replace-rules-in-junit5/
• http://blog.codefx.org/libraries/junit-5-conditions/

[sbrannen]

FYI: timeouts are supported in JUnit Jupiter via the assertTimeout() and assertTimeoutPreemptively() methods in Assertions. See #438 and related commits for details.

[smoyer64]

Nice! I'll update the description and mark it complete when M3 becomes current.

[marcphilipp]

@smoyer64 Would you be interested to add a section that documents the migration strategy for rules to the User Guide?

There are already sections for "Migration Tips" and "Limited JUnit 4 Rule Support".

[smoyer64]

@marcphilipp I've started this task and, after thinking through how I'd like it to be organized, wanted to present my thoughts before getting too far ahead.

It's my opinion that those using TestRule are intimately familiar with the javadocs at http://junit.org/junit4/javadoc/latest/org/junit/rules/TestRule.html (and for the javadocs and examples provided for each of the implementations). My intention is to leverage that knowledge to provide side-by-side (well ... top and bottom) examples of JUnit4 and JUnit5 for each of the rules in the description of this issue. Each rule would also get a sub-heading to make them easy to find using the Java class name of the rule.

I've created the jupiter and vintage equivalents for ExpectedException as an example. Note that these classes are not yet annotated for inclusion in the documentation but they do pass spotless checks and execute successfully during the platform tests. The examples can be found at https://github.com/selesy/junit5/tree/issues/343-document-testrule-compatibility/documentation/src/test/java/example/testrule.

Please comment/critique both my intended approach and the way I've named/structure the example classes.

[marcphilipp]

I think that's a very good approach. Please proceed! 🙂

[smoyer64]

Full steam ahead ... I think I'll open a PR that's linked to this issue and my fork so that it's easier for the team to review my work. I'll mark it WIP so that it's clear that it shouldn't be merged yet.

[smoyer64]

I've created PR #660 to track the work I'm committing towards this issue and added a more detailed task list to the Deliverables section. If you see anything missing, please edit the description or leave a comment. I've also added strike-thru to the Definition of Done section to indicate which sections I believe apply to that PR.

[smoyer64]

Please review the Stopwatch section of the user's guide provided in PR #660 - if this format is acceptable to everyone, I'll keep writing.

I was also wondering about referring to the JUnit-Pioneer project from the users guide as it's my intention to maintain production quality versions of the StopwatchExtension and Stopwatch classes in that project. Would it be acceptable to make the documentation project dependent upon Pioneer?

@nicolaiparlog FYI

[marcphilipp]

@smoyer64 I finally got around to having a look at #660. Overall it looks good but it's missing a listing of StopwatchExtension and Stopwatch, isn't it?

[smoyer64]

@marcphilipp - If you say they're missing then they're missing! I actually had a rationale but I'll bow to the wishes of the JUnit 5 team.

1) This is only one of seven sub-sections on replacing JUnit 4 rules with JUnit 5 constructs. Not all of them require extensions as some (like error collector) are really documenting a change in programming technique. It felt like the section was getting a little bit long already but I'm willing to add these two listings.

2) Since there are a few of the JUnit 4 rules that require extensions, I think it would be better to provide production-ready code for those extensions and, since that seems outside the scope of the core JUnit 5 project I would propose that we maintain them at junit-pioneer. This obviously would cause a problem including the listing in the documentation as we really don't want to link from the documentation module to junit-pioneer. We also don't want to maintain them in two places. If it's acceptable, I was going to add a reference to the junit-pioneer project (for those who would rather add a small dependency than copy the code).

In any case, if the writing style is acceptable, I'll start working on the other sections later this week and we can fix up issues like this upon further reviews.

@nicolaiparlog

[sbrannen]

Nice! I'll update the description and mark it complete when M3 becomes current.

Regarding timeouts, it appears you didn't update the description. 😉

[sbrannen]

Would it be acceptable to make the documentation project dependent upon Pioneer?

I don't think that would be a good idea: it would introduce a chicken-and-egg problem regarding dependencies, and that's something we should avoid.

However, _referring to the JUnit-Pioneer project_ from the User Guide should not be a problem. We already do that for the SpringExtension.

[marcphilipp]

This is only one of seven sub-sections on replacing JUnit 4 rules with JUnit 5 constructs. Not all of them require extensions as some (like error collector) are really documenting a change in programming technique. It felt like the section was getting a little bit long already but I'm willing to add these two listings.

Actually, while reading it I wondered if we should rather show the Jupiter implementation instead of Vintage. But that might be confusing, too. 🤔

Would it be acceptable to make the documentation project dependent upon Pioneer?

I'm with Sam: Referring is fine, depending isn't.

[sbrannen]

@smoyer64, is this still _in progress_?

[marcphilipp]

@smoyer64 I've unassigned you and put this issue "up-for-grabs" for someone else to take over.