Ember.js: Asserting error thrown no longer works as of 2.11

[ykaragol]

The following test, no longer run after we start to use 2.11.1.

test('it throws error my-custom-input is called', function(assert) {
    assert.throws(() => {
        this.render(hbs`{{my-custom-input}}`);
    }, /my-custom-input component error/, 'Error must have been thrown');
});

I saw mocha users had faced with nearly the same problems: https://github.com/emberjs/ember-mocha/issues/141

SO Question: http://stackoverflow.com/questions/42781085/ember-render-hbs-swallowing-thrown-error
Slack Archive: https://embercommunity.slack.com/archives/-testing/p1489479464001332

Repro
https://ember-twiddle.com/47c4cfbb571ac3fa83ab912f605ebc6a?fileTreeShown=false&openFiles=tests.integration.components.i-throw-test.js%2C

[Turbo87]

/cc @rwjblue @krisselden @workmanw

[efx]

I am seeing identical errors with should.js and expect style assertions, also using mocha. Though, I only see this when bumping from 2.11.2 to 2.11.3.
similar pattern failing using expect and should.js:

(function() {
    this.render(hbs`{{async-image}}`);
}).should.throw('need a source');

[pixelhandler]

Is this something that can be replicated using ember-twiddle?

[efx]

@pixelhandler i'm struggling getting an ember-mocha twiddle set up, any tips / examples?
started here: https://ember-twiddle.com/49a310682db5d8433dff5ef98ab75211?

[mupkoo]

I tried to reproduce it without any success. Maybe something related to the Ember versions in the Twiddle?

https://ember-twiddle.com/8932ace19517f83ef12693068381b6a3?openFiles=twiddle.json%2C

[ykaragol]

As one of the Stackoverflow comment says they could not reproduced it in twiddle.
But checkout this repo: https://github.com/feanor07/ember-component-init-error-swallowed

[nickiaconis]

I've reproduced this in a Twiddle. The key to reproduction is to run any acceptance test before running the test that asserts the thrown error. Running the acceptance test populates Ember.Test.adapter, and it is that adapter that handles the error before assert.throws can receive it.

https://ember-twiddle.com/47c4cfbb571ac3fa83ab912f605ebc6a?fileTreeShown=false&openFiles=tests.integration.components.i-throw-test.js%2C

cc @pixelhandler

[nickiaconis]

Here's a work-around for QUnit:

import Ember from 'ember';
import QUnit from 'qunit';
import { QUnitAdapter } from 'ember-qunit';

// Work-around for https://github.com/emberjs/ember.js/issues/15013
const originalAssertThrows = QUnit.assert.throws;
const rethrowAdapter = QUnitAdapter.extend({
  exception(error) {
    throw error;
  },
}).create();
QUnit.assert.throws = () => {
  const originalAdapter = Ember.Test.adapter;
  Ember.Test.adapter = rethrowAdapter;

  originalAssertThrows.apply(this, arguments);

  Ember.Test.adapter = originalAdapter;
};

[efx]

I've narrowed down the problem to the following change: https://github.com/emberjs/ember.js/commit/a90fa0641d16f12566cbfef16ea2604e68ed6dd9#diff-33ff1529a427727ffe2425f145375540L24. Reverting the change back to getOnerror() produces the expected assertions in my project. I'm going to keep tinkering if there is a way to exempt test assertions from dispatching the entire events.
that said workmanw might have the better/quicker solution here.

theory as why this happens

Chai's underlying assertThrows (https://github.com/chaijs/chai/blob/526f88585d32220d3777053530ab3182af7bf5d0/lib/chai/core/assertions.js#L2430-L2435) check expects the invoked function to execute synchronously, but the refactor to dispatchError executes asynchronously so chai's catch doesn't capture the error.
/cc @workmanw

[tzellman]

Just came across this same issue while attempting to update an addon from ember/cli 2.9. Has anyone found a decent workaround for ember-cli-chai usage?

[alvincrespo]

I'm currently working on our ember-link-with-follower update and running across this issue in our tests: https://github.com/echobind/ember-links-with-follower/pull/191. We're using ember-cli-mocha.

[pixelhandler]

@ykaragol I'm curious if this is still an issue in 2.16.x ?

[ykaragol]

@pixelhandler I didn't test it. I'd just use the workaround shown in the StackOverflow Question.

[rwjblue]

The ember-qunit-assert-helpers addon was specifically created to handle this scenario...

[rwjblue]

This was fixed by https://github.com/emberjs/ember.js/pull/15871, which essentially reverts https://github.com/emberjs/ember.js/pull/14898 and adds a bunch of tests.