Cypress: Expect is failing but test is passing in cy.on confirm & alert message

[nimit199]

I wanted to test my content of the confirmation message but it seems that even if my test fails my case passes

it('Should Get Confirm Message', () => {
        cy.get('.button')
            .click();
        cy.on('window:confirm', str => {
            expect(2 + 2).to.equal(5);
        })
    })

The issue seems to be similar to
https://github.com/cypress-io/cypress/issues/3497
and
https://github.com/cypress-io/cypress/issues/2413

But their solution does not work for me.
The issue persists for window:alert

[dwelle]

As I explained in my SO answer, Cypress events are not guaranteed to be triggered within the timeframe of a test, thus without a bit more work you can't make test assertions within them.

With that being said, I believe Cypress should:

1. Display an error message when an assertion is made outside the running test.
2. The assertion itself shouldn't be displayed in Command Log in such cases.

[jennifer-shehane]

@nimit199 @dwelle You should just need to set up the listening above the actual action right?

it('Should Get Confirm Message', () => {
  cy.on('window:confirm', str => {
    expect(2 + 2).to.equal(5);
  })
  cy.get('.button').click();
})

[bahmutov]

I believe the test does not know that it needs to assert the window confirm, it finishes after the click then the window confirm triggers - with a failing assertion. By then the test has completed and does not change its status. The command log still shows a failing assertion though. Try adding another assertion there to force retrying until the confirm has executed

Sent from my iPhone

On Jan 1, 2020, at 23:30, Jennifer Shehane [e-mail] wrote:


@nimit199 @dwelle You should just need to set up the listening above the actual action right?

it('Should Get Confirm Message', () => {
cy.on('window:confirm', str => {
expect(2 + 2).to.equal(5);
})
cy.get('.button').click();
})
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[bahmutov]

Here is an example waiting for the confirm to have been called before finishing the test (the code calls window.confirm 1 second after the click), see https://github.com/cypress-io/cypress-test-tiny/tree/confirm-after-test

Without waiting

/// <reference types="cypress" />
it('confirms', () => {
  cy.visit('index.html')

  cy.on('window:confirm', (text) => {
    // make the assertion fail on purpose
    // but the test does NOT wait for this assertion to pass
    // By the time the assertion fails, the test has passed
    expect(false).to.be.true
  })

  cy.get('#click').click()
})

With waiting

/// <reference types="cypress" />
it('confirms', () => {
  cy.visit('index.html')

  let called
  cy.on('window:confirm', (text) => {
    // make the assertion fail on purpose
    // but the test does NOT wait for this assertion to pass
    // By the time the assertion fails, the test has passed
    expect(false).to.be.true
    called = true
  })

  cy.get('#click').click()
  cy.should(() => {
    expect(called).to.be.true
  })
})

When the test passes (comment out the false assertion, or change it to pass), then all is good

When the assertion is false, the test correctly fails (even if the error message claims the assertion was thrown from the application code)

I think all assertions that pass or fail AFTER the test has finished should have a warning icon ⚠️ next to them in the Command Log. On click, the DevTools should print a message in the console, explaining

This assertion failed / passed AFTER the test has completed.
Please ensure the test waits for every assertion before completion.
See `https://on.cypress.io/assertion-after-test-finished`

[dwelle]

Here's my take without using state:

cy.wrap(new Promise((resolve, reject) => {
  cy.on('window:confirm', msg => {
    try {
      // expect(msg).to.eq(confirmMessage); // optional
    } catch ( err ) {
      return reject(err);
    }
    resolve();
  });
  // set a timeout to ensure we don't wait forever
  setTimeout(() => {
    reject(new Error('window.confirm wasn\'t called within 4s'));
  }, 4000);
}), { log: false });

[jennifer-shehane]

Closing as resolved.

If you're experiencing a bug similar to this in Cypress, please open a new issue with a fully reproducible example that we can run. There may be a specific edge case with the issue that we need more detail to fix.