Junit5: Method `Assertions.assertThrows(expectedType, executable, message)` doesn't check the contents of the exception message.

[borisilic]

JUnit Version: 5.4.0

For this unit test

@Test
public void verifyErrorMessageWhenBranchCannotBeFound() {
        given(restTemplate.getForObject(any(), any(), (Object) any())).willThrow(NotFoundException.class);

        assertThrows(NotFoundException.class, () -> testInstance.retrieveBranchDetails("3001"),
                "I don't think this error messaging thing checker works");
}

Testing this code

public BranchDetailsResponse retrieveBranchDetails(String branchNumber) {
        try {
            return restTemplate.getForObject(url, BranchDetailsResponse.class, branchNumber);
        } catch (HttpStatusCodeException e) {
            throw new NotFoundException("");
        }
   }

I expect that the test will fail as the error message passed to the assertThrows method does not match the error message that is returned from the thrown exception.

I believe the offending piece of code is here in the class AssertThrows.java

@SuppressWarnings("unchecked")
private static <T extends Throwable> T assertThrows(Class<T> expectedType, Executable executable,
    Object messageOrSupplier) {
    try {
        executable.execute();
    }
    catch (Throwable actualException) {
                // This part here just returns so long as the two exceptions match without checking the messages
        if (expectedType.isInstance(actualException)) {
            return (T) actualException;
        }
        else {
            BlacklistedExceptions.rethrowIfBlacklisted(actualException);
            String message = buildPrefix(nullSafeGet(messageOrSupplier))
                    + format(expectedType, actualException.getClass(), "Unexpected exception type thrown");
            throw new AssertionFailedError(message, actualException);
        }
    }

    String message = buildPrefix(nullSafeGet(messageOrSupplier))
            + String.format("Expected %s to be thrown, but nothing was thrown.", getCanonicalName(expectedType));
    throw new AssertionFailedError(message);
}

The if statement which is bolded will return the actualException if it matches the expectedException but nowhere does it do any message error checking.

[sormuras]

The (optional) custom failure message you pass to any Assertions.assertXyz(..., message) method is never checked. For example, the assertThrows API documentation reads:

Fails with the supplied failure message.

If you want to check the message attached to the instance of the throwable, you need to evaluate the returned throwable instance on your own.

    @Test
    void exceptionTesting() {
        Exception exception = assertThrows(ArithmeticException.class, () -> calculator.divide(1, 0), "custom failure message");
        assertEquals("/ by zero", exception.getMessage());
    }

Adopted from https://junit.org/junit5/docs/current/user-guide/#writing-tests-assertions

[borisilic]

Got it. We used to have a convenience method for JUnit4 which did what I was describing and I just assumed that this would behave the same way. I think it would be useful to have this functionality as we often check error message responses.

[marcphilipp]

It was discussed and rejected recently: https://github.com/junit-team/junit5/issues/2128#issuecomment-567822073

[marcusvoltolim]

The (optional) custom failure message you pass to any Assertions.assertXyz(..., message) method is never checked. For example, the assertThrows API documentation reads:

Fails with the supplied failure message.

If you want to check the message attached to the instance of the throwable, you need to evaluate the returned throwable instance on your own.

    @Test
    void exceptionTesting() {
        Exception exception = assertThrows(ArithmeticException.class, () -> calculator.divide(1, 0), "custom failure message");
        assertEquals("/ by zero", exception.getMessage());
    }

Adopted from https://junit.org/junit5/docs/current/user-guide/#writing-tests-assertions

More clean example:

@Test
void test() {
    var throwable = assertThrows(RuntimeException.class, () -> {});
    assertThat(throwable).hasMessage("...");
}