Sdk: Cannot test for assertion errors

[filiph]

Not sure if this is a regression or if I'm just holding it wrong, but the following test fails:

import 'package:test/test.dart';

void main() {
  test('assert throws assert', () {
    expect(() {
      assert(false);
    }, throwsA(AssertionError));
  });
}

This test fails with:

Expected: throws Type:<AssertionError>
  Actual: <Closure: () => Null>
   Which: threw ?:<'file:///....dart': Failed assertion: line 754 pos 16: 'false': is not true.>
          stack dart:core                        _AssertionError._throwNew
                test/storyline_test.dart 754:16  main.<fn>.<fn>.<fn>
                package:test_api                 expect
                test/storyline_test.dart 753:7   main.<fn>.<fn>

Note that this is not the same as https://github.com/dart-lang/sdk/issues/34530, which lacked the () {} (as @mraleph pointed out).

Why I think this is important to fix: it's valuable to have potentially expensive assertions that only get executed during tests. In my concrete example, I have an O(n^2) assertion that I want to make sure works correctly, and that I also don't want anywhere near the shipped app.

• Dart SDK Version (dart --version)

Dart VM version: 2.6.0 (Thu Oct 24 17:52:22 2019 +0200) on "macos_x64"

• Whether you are using Windows, MacOSX, or Linux (if applicable)

MacOSX

• Whether you are using Chrome, Safari, Firefox, Edge (if applicable)

N/A

[filiph]

Ah! I figured it out. The correct test should look like this:

import 'package:test/test.dart';

void main() {
  test('assert throws assert', () {
    expect(() {
      assert(false);
    }, throwsA(isA<AssertionError>());
  });
}

We should document the matchers and how they work, or write a codelab. But that's outside the scope of this bug. Hopefully, others will find this bug when faced with a similar issue.

[vargavince91]

Having unreadable tests really bothered me: you can't read "throws a is a" without having to first stop to think about what is going on.

Here are two alternatives that might make your code more readable.

Generic solution

/// Returns a matcher for functions that throw object with type [T]
Matcher throwsTypeOf<T>() => throwsA(isA<T>());

Then you can:

expect(() { assert(false); }, throwsTypeOf<AssertionError>());

A solution just for assertion errors

// Inspired by the throws_matchers.dart file, that file includes
// mathcers like throwsArgumentError
/// A matcher for functions that throw AssertionError
final Matcher throwsAssertionError = throwsA(isA<AssertionError>());

Then in your expectations:

expect(() { assert(false); }, throwsAssertionError);

[jasperamorgan]

@vargavince91 It looks like TypeMatcher is recently deprecated.

'TypeMatcher has been deprecated because it is no longer used in framework(only in deprecated methods). '
'This feature was deprecated after v1.12.1.'

Combining your approach with the one from @filiph could achieve the same goal:

final Matcher throwsAssertionError = throwsA(isA<AssertionError>());

[artob]

It would be helpful to have the throwsAssertionError helper generally available as baseline. I know flutter_test has it, but is there any reason it's not available without Flutter as well?