Cypress: Click and type events in after or afterEach hook don't work if one test fails

[StanFisher]

Current behavior:

Any click event in the after block fails if one of the tests in an it block fails.

Desired behavior:

Click events in the after block should succeed even if there are tests that fail.

Steps to reproduce:

I forked the cypress-tiny-test repository and set up a reproducible example in the latest commit.
https://github.com/StanFisher/cypress-test-tiny

Here's the commit containing the failing after block.
https://github.com/StanFisher/cypress-test-tiny/commit/c185242f3fdd30715d16fc44e393e092c30c4d58

Notice that if the failing test is changed to succeed, then the two click events in the after block succeed.

Versions

Cypress 3.1.2
macOS Mojave 10.14.1
Chrome 70.0.3538.102

┆Issue is synchronized with this Jira Bug by Unito

[StanFisher]

@jennifer-shehane Thanks for looking at this! Why was this labeled as a feature instead of a bug?

[bahmutov]

I marked it as a feature because once the test finishes, catching any possible errors is NOT something Mocha would do - because by then the test has finished. In Cypress the exception is there because the browser window still stays open after the test. If we closed the browser window immediately, the exception would never happen. So that's why I marked it as a feature

[jennifer-shehane]

I believed it was Mocha's implementation to not run after hooks on error of the test suite, but after further investigation, I see that Mocha does run after hooks on test failure. To quote the Mocha team on their hook execution:

• If before hook fails, tests in a suite (where the hook is) and sub-suites are not executed. If after hook exists, it is called.
• If beforeEach hook fails, remaining tests in that suite (where the hook is) or sub-suites are not executed. The remaining before hooks are also not executed. If afterEach hooks exist in that suite, or parent suites -- they get executed. The suite's after hooks are also executed.
• If after hook fails, no additional hooks are called.
• If afterEach hook fails, remaining tests in that suite (where the hook is) and sub-suites are not executed. The remaining after hooks (from parent suites) are executed. after hooks are executed 'up' starting from the suite of the last executed test, finishing with a suite where the hook is (inclusively).

After any hook error, the next sibling suite (relative to the suite with hook error) is executed.

This allows writing test hooks in a symmetrical manner:

• before (each): allocate all resources for this and child test suites or tests
• after (each): free all resources, allocated by before (each). Make sure, this will work correctly, even if some resource allocation failed -- it will still be called.

Based on this, I would qualify this as a bug in Cypress. We are upgrading Mocha to its latest version here https://github.com/cypress-io/cypress/issues/2528 I will see if the upgrade fixes this issue, if not, then it will require more investigation.

[jennifer-shehane]

I am unable to check this out in our v4 branch at the moment, but here is the reproducible code for the future. It's strange that the code in the after hook does run, but the click is not performed.

describe('page', () => {
  beforeEach(() => {
    cy.visit('https://www.cypress.io')
  })

  // ***** NOTE *****
  // The presence of this failing tests means that the clicks in the 'after' block don't work.
  // If this test is either removed or changed to be passing, then the 'after' block will work.
  it('fails', () => {
    cy.wrap(false).should('be.true')
  })

  after(() => {
    cy.get('#main-nav>li')
      .contains('Support').click()
    cy.url().should('contain', '/support/')
  })
})

[immohsin]

Any workaround for this one?
After i run spec file, i want to cleanup few thing in case if anything fails.

[toannguyen83]

I'm having the same issue. In the afterEach i need to perform cleanup after each test, undoing certain UI actions, but if the test failed, any action in afterEach doesn't work properly.

I have to do a weird workaround for now, having my clean up code in beforeEach, but, it won't perform clean up on the last test, so I need to also add a dummy test to the very end.

[NareshGuduri]

Is this fixed ?

[jennifer-shehane]

No this is not fixed. Issues will be closed when they are fixed.

[cvasile001]

Hi Everyone. I am also experiencing this issue. Is there any update on this issue? Maybe a workaround? @jennifer-shehane

[jennifer-shehane]

My previous example for this bug no longer exhibits the error properly since the website under test has changed.

I tested this against the upcoming Cypress 4.0 release and verified that this is not fixed in that release.

To reproduce

spec.js

describe('page', () => {
  context('after click clicks', () => {
    it('passes', () => {
      cy.visit('https://example.cypress.io').then(() => {
        expect(false).to.be.false
      })
    })

    after(() => {
      cy.get('.home-list')
        .contains('Querying').click()
      cy.url().should('include', '/querying')
    })
  })

  context('after click does not click', () => {
    it('fails', () => {
      cy.visit('https://example.cypress.io').then(() => {
        expect(false).to.be.true
      })
    })

    after(() => {
      cy.get('.home-list')
        .contains('Querying').click()
      cy.url().should('include', '/querying')
    })
  })
})

[Brestachan]

I hope this will be solved soon, or at least for someone to offer a workaround. I wrapped a lot of my tests with this, it seems to be super useful, except it doesn't work yet. :)

[cvasile001]

@Brestachan you can add a dummy test that always passes before the next set of tests run. This will ensure that your after block click action is executed properly. Not the best workaround, but I found this helpful for tests that I know are failing with open bugs that might be a while before they work again.

[stevenvachon]

This is a duplicate of #1477, by the way.

[jennifer-shehane]

Type also causes this error:

describe('after hook no click', () => {
  beforeEach(() => {
    cy.visit('https://example.cypress.io')
  })

  // The presence of this failing tests means that the clicks in the 'after' block don't work.
  it('fails', () => {
    cy.wrap(false).should('be.true')
  })

  after(() => {
    cy.get('.nav')
      .contains('li', 'Utilities').click()
    cy.url().should('include', '/utilities')
  })
})

describe('afterEach hook no type', () => {
  beforeEach(() => {
    cy.visit('https://example.cypress.io')
  })

  // The presence of this failing tests means that the clicks in the 'after' block don't work.
  it('fails', () => {
    cy.wrap(false).should('be.true')
  })

  afterEach(() => {
    cy.get('.nav')
      .contains('Utilities').click()
    cy.url().should('include', '/utilities')
  })
})

describe('after hook no type', () => {
  beforeEach(() => {
    cy.visit('https://example.cypress.io/commands/actions')
  })

  // The presence of this failing tests means that the clicks in the 'after' block don't work.
  it('fails', () => {
    cy.wrap(false).should('be.true')
  })

  after(() => {
    cy.get('#email1').type('foo').should('have.value', 'foo')
  })
})

describe('afterEach hook no click', () => {
  beforeEach(() => {
    cy.visit('https://example.cypress.io/commands/actions')
  })

  // The presence of this failing tests means that the clicks in the 'after' block don't work.
  it('fails', () => {
    cy.wrap(false).should('be.true')
  })

  afterEach(() => {
    cy.get('#email1').type('foo').should('have.value', 'foo')
  })
})

[stevenvachon]

Is it possible to get this into a minor or patch release? I cannot use the dummy test idea because then cookies will have already been cleared.

[TestLeafInc]

This issue is blocking several tests of us for straight solution. Though we have overridden the mocha on cypress, Is there a deadline for this please?

[wesam-mtour]

use force type and force click in after or afterEach Hooks