Testcafe: ExternalAssertionLibraryError when assertion gets failed in testcafe

[sachin-funde]

getting assertion on failed assertion
await t.expect(false).ok("response of student school is not in arrayformat");
testcafe 1.7.1

[Farfurix]

@sachin-funde

Hello,

Please send us a small runnable project so that we can research it on our side, as I wasn't able to reproduce this issue under Ubuntu.

My setup:
test.ts:

import { Selector } from 'testcafe';

fixture `https://google.com`;

test('Test', async t => {
    await t
        .expect(false).ok('message');
});

Command:

npx testcafe firefox test.ts -r allure

I look forward to your response.

[devmondo]

if this helps, it happens also if we miss async keyword and it also causes the process to exit with code 1!

in PageModel.ts

  public async login(userName: string, password: string) {
    await t.expect(1).eql(2)

    }

then in test-example.ts

pageModel.login()

[miherlosev]

Hi, @devmondo

I cannot reproduce the problem again. Could you please provide a complete example that I can run locally without additional set up?

[devmondo]

hi @miherlosev,

sure thing :)

as I mentioned before, if we add the missing await the assertion will fail but it won't crash the process, but as soon as you miss the await the process crashes.

you don't necessarily need a page model, just use t.expect(1).eql(2) without await in the test body and it will lead to the same crashing behavior.

page-model.ts

import {Selector, t} from 'testcafe';

export class PageModel {
    public async login() {
    await t.expect(1).eql(2)
    }
}

export const pageModel = new PageModel();

test.ts

import { pageModel } from "./page-model";

fixture(`test`);
test(`test`, async () => {
    pageModel.login();
})

testcafe.js

const createTestCafe = require("testcafe");

let testcafe = null;

createTestCafe('localhost', 1337, 1338,null,true )
    .then(tc => {
        testcafe = tc;
        const runner = testcafe.createLiveModeRunner();
        return runner
            .src(['./test.ts'])
            .browsers(['chrome'])
            .run();
    })
    .then(failedCount => {
        console.log('Tests failed: ' + failedCount);
        testcafe.close();
    }).catch((e)=>{
    console.log(e.stack);});

result error

Make changes to the source files or press Ctrl+R to restart the
 test run.
ExternalAssertionLibraryError { code: 'E53', isTestCafeError: t
rue, callsite: CallsiteRecord { filename: 'C:\\projects\\test\\page-model.ts', lineNum: 4, callsiteFrameId
x: 6, stackFrames: [
      [CallSite],  [CallSite],  [CallSite],
      [CallSite],  [CallSite],  [CallSite],
      [CallSite],  CallSite {}, [CallSite],
      CallSite {}, [CallSite],  [CallSite],
      [CallSite],  CallSite {}, [CallSite],
      CallSite {}, [CallSite],  [CallSite],
      [CallSite],  [CallSite],  [CallSite],
      [CallSite],  [CallSite]
    ], isV8Frames: true }, errMsg: 'AssertionError: expected 1
to deeply equal 2' }
(node:25800) [DEP0018] DeprecationWarning: Unhandled promise re
jections are deprecated. In the future, promise rejections that
 are not handled will terminate the Node.js process with a non-
zero exit code.

using windows 10 64x, node 12.6.0 and latest testcafe

[devmondo]

i also tested @sachin-funde code above, but it does not crash, unless he is missing the await ... weird!
await t.expect(false).ok("response of student school is not in arrayformat");

[miherlosev]

@devmondo

Thank you for the provided information. I've reproduced the problem.

[sachin-funde]

@devmondo

Thank you for the provided information. I've reproduced the problem.

@devmondo

Thank you for the provided information. I've reproduced the problem.
How to fix this issue . please do the needful .. our tests are failing

[AlexSkorkin]

The fix is not available yet. We will update this thread once it is ready. You are welcome to submit your PR if you want to fix this issue on your own.

[faheemfactor]

As we upgrade to Node 13 test cafe fails.

[alexey-lin]

Thank you, we will take this into account. Also, please be aware that non-LTS versions of Node.js might have some bugs or regressions. If you face any (not related to this issue), please feel free to open a separate issue.

[jasongornall]

Is there any update on this? I just went on a wild goose chase and ended up here...

try {
fetch... // network call
await t.expect() // fails
} catch(e) {
e is a ExternalAssertionLibraryError when I spect it to be from the failed fetch (if it fails)

}|

[wentwrong]

@jasongornall, We have no updates regarding this issue yet.

If the fetch function originated from the node-fetch package, I suggest you place await before calling the function. Also, I recommend you move any TestCafe assertions outside of the try {} block to catch errors from the failed fetch (not TestCafe assertions). This example works fine for me:

import fetch from 'node-fetch';

fixture `GET requests via 'node-fetch' package`;

test(`Request to Github Users API`, async t => {
    const jsonResponse = await (await fetch('https://api.github.com/users/github')).json();

    await t.expect(jsonResponse).contains({ location: 'San Francisco, CA' });
});

test(`Request to invalid domain`, async t => {
    let jsonResponse = '';

    try {
        jsonResponse = await (await fetch('https://domain.invalid/')).json();
    }
    catch (e) {
        // do something with e
    }

    await t.expect(jsonResponse).contains({ location: 'San Francisco, CA' });
});