Cypress: Code in hooks and test are rerun when changing to a different superdomain

[davidzambrana]

Current behavior:

When running my spec file, I noticed that some code in the before hook (not beforeEach) is being run before each of my three test cases, causing my checks to fail. Want to remark that I am using different superdomains for every test, I don't see this when I run all the tests with the same superdomain.

Desired behavior:

Code in the before hook only runs once, before all the tests, no matter if using different superdomains.

Steps to reproduce:

I explain my setup. My spec file consists of:

- before()
- after()
- it() //superdomain 1
- it() //superdomain 2 using an iframe
- it() //superdomain 1

As I am using different superdomains, the only way that I've read to do it is visiting the different domains in different tests (https://docs.cypress.io/guides/guides/web-security.html#One-Superdomain-per-Test)

I've also set "chromeWebSecurity": false

Within the before hook I run several requests to some endpoints as a way to get some data that I will use through the tests; Get it once, and use it across the three it().

I've also set a console.log('This only happens in the beforeHook: ' + var) to see what's the value of my variable, and this is what I see in the run:

• Running the first it()
  

• Second it()
  

• Third it()
  

After every it() the console gets cleared, which is fine, but I don't expect to see the log again during the second and the third tests, as this is supposed to run only once. Notice how the variable gets an increment for each loop, as I would expect had I done it in a beforeEach hook.

In the left side of the window every seems fine as there is no command logging for the before hook when running the second and third tests, but the console.log reveals that.

Versions

• Cypress version: 3.0.1
• Browser: Chrome 66
• OS: Ubuntu Linux 16.04 LTS

[jennifer-shehane]

Thank you so much for the thorough bug report! Indeed the behavior does appear to be different when visiting different superdomains.

Cypress version 3.0.1, no configuration, support files, or plugins.

My recreation of the issue:

context('SubDomain Checks', () => {
  let beforeNum = 1
  let beforeEachNum = 1

  before(() => {
    console.warn(`BEFORE ${beforeNum}`)
    beforeNum++
  })

  beforeEach(() => {
    console.warn(`BEFORE EACH ${beforeEachNum}`)
    beforeEachNum++
  })

  it('visit 1 subdomain', () => {
    cy.visit('https://www.cypress.io')
  })

  it('visit 2 subdomain', () => {
    cy.visit('https://docs.cypress.io')
  })

  it('visit 3 subdomain', () => {
    cy.visit('https://example.cypress.io')
  })
})

context('SuperDomain Checks', () => {
  let beforeNum = 1
  let beforeEachNum = 1

  before(() => {
    console.warn(`BEFORE ${beforeNum}`)
    beforeNum++
  })

  beforeEach(() => {
    console.warn(`BEFORE EACH ${beforeEachNum}`)
    beforeEachNum++
  })

  it('visit 1 superdomain', () => {
    cy.visit('https://www.cypress.io')
  })

  it('visit 2 superdomain', () => {
    cy.visit('https://www.wikipedia.org')
  })

  it('visit 3 superdomain', () => {
    cy.visit('https://www.dictionary.org')
  })
})

Preserved console warnings for subdomain tests

Preserved console warnings for superdomain tests

[brian-mann]

It's because Cypress must swap to the new superdomain, which causes the specs to rerun.

We internally handle this by rehydrating the state of all the previous tests and ensuring that we skip over them if they previous ran.

However, the test that cause the change to the new superdomain is rerun, and because we internally skip the previous tests it then mocha believes this is the first test to be run, and then will rerun its before hook.

To account for this we need to be sure to skip before hooks when rehydrating the state.

[davidzambrana]

Thank you very much for your quick answer. Amazing job what you guys are doing 👏

[hackel]

Is there currently any work-around that might instruct Cypress to switch to the second superdomain as soon as the test starts, thereby avoiding running all of my setup code (seeding DB, etc.) twice?

[WesleySSmith]

I've found a similar issue, and I'm not sure if it's fully explained by @brian-mann 's comment above, so adding it here in case this is a different variation.

My spec file:

describe('Part 1', function () {

    before(function() {
        cy.task('log', "In 1-before");
    });

    beforeEach(function() {
        cy.task('log', "In 1-beforeEach");
    });

    specify('A', function() {
        cy.task('log', "In 1-A before visit");
       cy.visit("https://cypress.io");
       cy.task('log', "In 1-A after visit");
    });
});


describe('part 2', function () {

    before(function() {
        cy.task('log', "In 2-before");
    });

    beforeEach(function() {
        cy.task('log', "In 2-beforeEach");
    });

    specify('B', function() {
        cy.task('log', "In 2-B before visit");
        cy.visit("https://example.com");
        cy.task('log', "In 2-B after visit");
    });
});

The output:

In 1-before
In 1-beforeEach
In 1-A before visit
    ** visit causes restart, as expected
In 1-before
In 1-beforeEach
In 1-A before visit
In 1-A after visit
    ** visit works this time, as expected
    ** Moving on to second test
In 2-before
In 2-beforeEach
In 2-B before visit
    ** visit causes restart, as expected
In 1-before
    ** why are we re-running 1-before???
In 2-before
In 2-beforeEach
In 2-B before visit
In 2-B after visit

Note that after we move on the the "part 2" block and call cy.visit() in the 'B' test, the test runner restarts, (as explained above) but then, for some reason, the beforeEach from the _"Part 1"_ test is rerun, even though it's not associated with the test that visited the new superdomain.

[c32hedge]

I think it's worth mentioning here that my duplicate of this issue (#5319) does not involve visiting multiple superdomains--we encountered the issue when there are tests that don't visit a webpage, followed by a test that does; the first test that does a cy.visit() triggers this behavior.

[Bkucera]

Is there currently any work-around that might instruct Cypress to switch to the second superdomain as soon as the test starts, thereby avoiding running all of my setup code (seeding DB, etc.) twice?

@hackel good point. This could be solved by declarative inline test configuration in PR #5346, since we would know ahead of time to switch domains

[cypress-bot[bot]]

The code for this is done in cypress-io/cypress#7035, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[jennifer-shehane]

Sorry, we accidentally merged the PR above prematurely. This is still in the review stage. We will close this issue and comment when this issue is merged for the next upcoming release.

[cypress-bot[bot]]

The code for this is done in cypress-io/cypress#7154, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[cypress-bot[bot]]

Released in 4.8.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v4.8.0, please open a new issue.