Cypress: cy.wait() timeout even though XHR requests were made (graphql requests)

[daironmichel]

Current behavior:

We have a project using Relay which uses a single GraphQL API endpoint. Due to route nesting, most pages in this project call the GraphQL endpoint several times (anywhere from 2 to 6 calls) causing a waterfall of API calls.

To test if a page has loaded correctly we define a beforeEach block that starts cy.server() and defines the route cy.route('POST', '/graphql').as('graphql'). Then in the it(...) block we cy.visit('/') the page and then call cy.wait('@graphql') as many times as the endpoint gets hit.

Now requests to the endpoint always happen in the same order but responses do not (which is to be expected). When responses come back cy.wait() will sometimes get confused and timeout.

For example. Let's say the page does 3 requests and so we wait 3 times. The first request comes back and the first wait will pick it up. Now the test is waiting for the second request to resolve but the third request comes back instead before the second one. The second wait will then pick up the third response and the test continues to the third wait. Then the second response comes back but the third wait will not pick it up and times out saying that a response never occurred.

It might be that the second request resolved last but before the third cy.wait() call.

I hope this is not too confusing :)

Here's a screenshot of what I just explained. Note the response order in the console and that they all resolved correctly but cy.wait() timed out anyways.

Here is the same test passing.

The more cy.wait()s the higher the failure rate of the test. if you're only waiting for two requests it rarely fails.

We get the same behavior if we call cy.wait(['@graphql', '@graphql', '@graphql'])

Here's the example test code. I'll be linking the repo below

/// <reference types="Cypress" />

const getQueryName = function(xhr) {
  const regexp = /\w+Query/gm;
  const match = xhr.requestBody.query.match(regexp);
  return match[0];
};

context("Wait issue", () => {
  beforeEach(() => {
    cy.server();
    cy.route({
      method: "POST",
      url: "**/next-js-with-relay-modern-example",
      // optional route config added only for debugging
      onRequest: xhr => console.log("onRequest:", getQueryName(xhr)),
      onResponse: xhr => console.log("onResponse:", getQueryName(xhr)),
      onAbort: args => console.log("onAbort:", args)
    }).as("query");
  });

  it("does not timeout 01", () => {
    // first we vist url which triggers
    // waterfall calls to graphql endpoint
    cy.visit("/");

    // now we wait for requests to resolve
    // NOTE: .then(...) added only for debugging
    cy.wait("@query").then(xhr => cy.log("query1:", getQueryName(xhr)));
    cy.wait("@query").then(xhr => cy.log("query2:", getQueryName(xhr)));
    cy.wait("@query").then(xhr => cy.log("query3:", getQueryName(xhr)));
    cy.wait("@query").then(xhr => cy.log("query4:", getQueryName(xhr)));
    cy.wait("@query").then(xhr => cy.log("query5:", getQueryName(xhr)));
  });
});

Desired behavior:

cy.wait() picks up all responses correctly and does not timeout

Steps to reproduce: (app code and test code)

I have created an example project repo that you can clone and run locally

git clone [email protected]:daironmichel/cypress-wait-timeout-example.git
cd cypress-wait-timeout-example
yarn install
yarn dev

run test on cypress

yarn run cypress open

Run wait.spec.js

Note: You might need to run it 2 or 3 times to see it fail.

Versions

cypress 3.8.0

[daironmichel]

I tried this undocumented cypress example recipe but has the same problem.
https://github.com/cypress-io/cypress-example-recipes/blob/master/examples/server-communication__xhr-assertions/cypress/integration/multiple-requests.js

[daironmichel]

@jennifer-shehane sorry to bother you but is there any chance you can take a quick look at this? I'm really hoping you can help me on this one. thanks and happy holidays!

[jennifer-shehane]

@daironmichel Running the insturctions to start the repo ends up with this error during the build so that the server cannot run. Please fix this and comment when this is fixed so we can run.

> Using external babel configuration
> Location: "/Users/jennifer/Dev/cypress-wait-timeout-example/.babelrc"
[ error ] ./components/BlogPostPreview.js
Module not found: Can't resolve '../__generated__/BlogPostPreview_post.graphql' in '/Users/jennifer/Dev/cypress-wait-timeout-example/components'
[ event ] build page: /next/dist/pages/_error
[ wait ]  compiling ...
[ error ] ./components/BlogPostPreview.js
Module not found: Can't resolve '../__generated__/BlogPostPreview_post.graphql' in '/Users/jennifer/Dev/cypress-wait-timeout-example/components'

[daironmichel]

Hi @jennifer-shehane

Sorry about that! I forgot to commit a folder. I've fixed, cloned and checked that the example repo runs as instructed. Hopefully you won't get any errors now.

Thanks for taking the time to look at this.

[jennifer-shehane]

I can confirm this is happening from this repo's code with the test code below:

https://github.com/daironmichel/cypress-wait-timeout-example

git clone [email protected]:daironmichel/cypress-wait-timeout-example.git
cd cypress-wait-timeout-example
yarn install
yarn dev

run test on cypress

yarn run cypress open

context("Wait issue", () => {
  beforeEach(() => {
    cy.server()
    cy.route("POST", "**/next-js-with-relay-modern-example").as("query")
  });

  it("does not timeout 01", () => {
    cy.visit("/");

    cy.wait("@query")
    cy.wait("@query")
    cy.wait("@query")
    cy.wait("@query")
    cy.wait("@query")
  })
})

[oleksandrsubota]

Hi @daironmichel

I'm facing exactly the same behavior. Did you have the luck to find a solution?

I've tried

cy.wait("@query.all")

 cy.wait("@query.1")
 cy.wait("@query.2")
 cy.wait("@query.3")

but all of them had failed

[daironmichel]

@oleksandrsubota nope. I'm working on other stuff while I wait for the cypress team to fix this bug.

[EricPalmer22]

I have also run into this issue with regular XHR requests. I created a test project that reproduces the issue by forcing a delay on the first request.

I was about to open up a new issue when I found this one, so I'll just put my information and example here

Current behavior:

When you use a cy.wait() on an aliased route that matches two simultaneous requests, and the second request completes before the first one, cy.wait() yields the second request, and a second cy.wait() does not yield anything. Two requests were made that match the route, but only one is ever yielded.

Order of operations:

1. Set up cypress server a route
2. Click button which makes 2 API calls that both match the route
3. cy.wait() twice to attempt to capture both calls. If the second API call resolves before the first call, cy.wait() will never yield the first.

Desired behavior:

cy.wait() is able to yield both API calls that match the route, no matter the order in which they resolve.

Test code to reproduce

https://github.com/EricPalmer22/CypressExample

Versions

Cypress 4.1.0
Browser: Chrome

[dmytro-lymarenko]

I also have the same problem but I can't share the information about private project

[pravnyk]

I have the same problem. When this bug will be fixed?

[dmytro-lymarenko]

Any updates on this? We have a page which contains 4 similar tables for which we load the data from the same route but with different query. And this bug happens there almost always. In addition, all requests on our page is slow and today this page took almost 4 hour to finish!

[jennifer-shehane]

This repo also reproduces the problem: https://github.com/dmytro-lymarenko/cypress-bug from https://github.com/cypress-io/cypress/issues/6894

[jennifer-shehane]

This issue is still in the 'needs investigating' stage, which means no work has been done on this issue as of today, so we do not have an estimate on when this will be delivered.

[egdraper]

I was having the same issue. I am not sure if this helps you, but it fixed our problem. I ended up chaining the waits() together and WhaBAM everything started working.

    cy.visit("/")
      .wait("@query")
      .wait("@query")
      .wait("@query")
      .wait("@query")
      .wait("@query")

It also worked for us in a chained then()

    cy.visit("/").then(()=> {
      cy.wait("@query")
      cy.wait("@query")
      cy.wait("@query")
      cy.wait("@query")
      cy.wait("@query")
   }

Not sure what the underlying issue is here, but these methods seem to work in our case.

[daironmichel]

@egdraper are you sure that works?

I tried your code using @EricPalmer22 example but it still failed.

describe("Testing cypress", () => {
  beforeEach(function () {
    cy.server();
    cy.route("GET", "/test?**").as("getTest");
  });

  it("Captures and yeilds both calls", () => {
    cy.visit("http://localhost:3011");
    cy.get('[id="first"]').click();
    cy.wait("@getTest");
    cy.wait("@getTest");
  });

  it("Fails, the first wait yields the second call (because it completed first?)", () => {
    cy.visit("http://localhost:3011").then(() => {
      cy.get('[id="second"]').click();
      cy.wait("@getTest");
      cy.wait("@getTest");
    });
  });
});

[egdraper]

@daironmichel Is the getTest waits tied to the visit() or the click()? If it is tied to the click() you have to chain it to that. So you could try chaining it this way:

  cy.visit("http://localhost:3011").then(() => {
      cy.get('[id="second"]')
        .click()
        .wait("@getTest");
        .wait("@getTest");
    });

or

   cy.visit("http://localhost:3011")
   cy.get('[id="second"]')
     .click()
     .wait("@getTest");
     .wait("@getTest");

I just know I struggled with this same issue until I started doing it this way. I hope you can get it figured out.

[daironmichel]

@egdraper I tried chaining like you suggested

    cy.visit("http://localhost:3011").then(() => {
      cy.get('[id="second"]').click().wait("@getTest").wait("@getTest");
    });

Same problem with race condition. When the second request finishes before the first one the second cy.wait fails to resolve.

Thanks for help with trying all the different alternatives.

[egdraper]

@daironmichel I have been stewing over your issue and had a follow-up question if you haven't resolved it yet. In your codebase--not the test--is your second api call, that @getTest2 is waiting for, happening after you have fully waited for the first one to return. For example, are you set up this way in your codebase (roughly)? If so, I am curious if this is causing a timing issue with cypress.

async onClick() {
  await getTestApiCall()
  await getTestApiCall()
}

[daironmichel]

@egdraper my codebase runs as follows:

1. Page load fires several (lets say 3) requests to same api endpoint (graphql api). These calls not always happen in the same order. These requests are not sequential (nor do we want them to be). They fire somewhat in parallel.
2. The response time for each request varies depending on network traffic, db load, etc. So first request could take the most time to resolve sometimes.

[egdraper]

@daironmichel, We had the same issue that you are facing again in one of our tests. After looking at your situation compared to mine, I am wondering if it has to do with having 2+ cy.waits waiting for the same alias at the same time. This double-wait is the only case in any of our tests that is causing this problem (thus far).

Our new test, for example:

   cy.server().route("GET", "**/groups").as("apiGroups")
   ...
   cy.visit("/group")   //which calls the groups API twice
     .wait("@apiGroups").its("status").should("be", "200")
     .wait("@apiGroups").its("status").should("be", "200")

@jennifer-shehane Is it possible that a single .wait() is occasionally resolving both/multiple api calls captured by the same alias? I may go look through the source code, but I am wondering if you know?

[ambvdijk]

Is there an update on this issue? I'm experiencing the same problem...

[peterfortuin]

It looks like I also have this problem.

[ambvdijk]

Side note @jennifer-shehane, this behavior is not specifically related to GraphQL.
We are not using a GraphQL server. What we do have in common is that we have multiple XHR requests firing at the same time to the same URL but with different parameters.

Would really like to see this fixed to get rid of our cy.wait() calls now in our tests...

Love Cypress by the way!
Let me know if there is anything I could do to help or get more insight into the issue.

[evba1]

I have the same problem:

cy.route("GET", `${Cypress.env("apiUrl")}/**`).as("apiGetRequests");

click some link
cy.wait("@apiGetRequests");
click some other link
cy.wait("@apiGetRequests");

fails on second alias.
happens randomly, usually after making many requests

[evba1]

I have the same problem:

cy.route("GET", `${Cypress.env("apiUrl")}/**`).as("apiGetRequests");

click some link
cy.wait("@apiGetRequests");
click some other link
cy.wait("@apiGetRequests");

fails on second alias.
happens randomly, usually after making many requests

@jennifer-shehane any thoughts about this? This is getting very frustrating!!

[samtsai]

I have the same problem:

cy.route("GET", `${Cypress.env("apiUrl")}/**`).as("apiGetRequests");

click some link
cy.wait("@apiGetRequests");
click some other link
cy.wait("@apiGetRequests");

fails on second alias.
happens randomly, usually after making many requests

@jennifer-shehane any thoughts about this? This is getting very frustrating!!

Can you share a screenshot of the failure, specifically what the runner looks like? Or a reproducible repo?

[ambvdijk]

Any progress on this issue @jennifer-shehane?

[IgorPahota]

I am facing the same problem, I have 11 calls xhr of the same alias at the one time, I've trying to do it like this

const xhrAliasesArray = Array(11).fill('@odata')

cy.wait(xhrAliasesArray).then(xhrs => {
  xhrs.forEach(xhr => {
    expect(xhr.status).to.equal(200)
  })
})

but it fails, also i've tryed put some wait() between each xhr waiting

      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)
      cy.wait(500)
      cy.wait('@odata').its('status').should('eq', 200)

it worked for me, but it works very unstable and time to time, still looking for the correct answer for this question

[bahmutov]

Can someone try the new cy.route2 (which soon will be renamed to cy.http)?

[haggen]

For anyone having similar problems; In my case it was related to another issue; cy.wait will fail if the response had an empty body which will be forced if you responded with status 204 (no content).

It started working after I added a response. e.g.

cy.route2("POST", "/api/users/activation", { body: "Cypress is buggy" });` 

[EricPalmer22]

@bahmutov I tried cy.route2 on Cypress 5.6.0 with my test project as follows:

describe("Testing route2", () => {
  beforeEach(function () {
    cy.visit("http://localhost:3000");
    cy.route2({
      pathname: "/test",
    }).as("getTest");
  });

  it("Captures and yeilds both calls", () => {
    cy.get('[id="first"]').click();
    cy.wait("@getTest");
    cy.wait("@getTest");
  });

  it("Fails, the first wait yields the second call (because it completed first?)", () => {
    cy.get('[id="second"]').click();
    cy.wait('@getTest');
    cy.wait('@getTest');
  });
});

In the cypress log it looks like the requests are getting matched to the route as you can see the "getTest" alias tag, however cy.wait isn't returning them in either test.

Am I missing something?

[Saibamen]

@bahmutov

Can someone try the new cy.route2 (which soon will be renamed to cy.http)?

xD

[EricPalmer22]

Tested using intercept with Cypress 6.0.0 and it looks to be working, at least on the first try.

There might be a bug with the Cypress UI test runner. I've noticed that the first time I run tests in the Cypress UI they pass, however when I re-run tests but clicking the "Run All Tests" arrow button in the browser, cy.wait does not yield the calls. If I "Stop" the test runner in the Cypress UI, then click the spec file to run tests again, they pass the first time.

First run:

After clicking the arrow to rerun all tests:

[bahmutov]

Trying to reproduce the issue:

@daironmichel after cloning your example https://github.com/daironmichel/cypress-wait-timeout-example and starting the app I am getting 502 from the server

Then I forked https://github.com/dmytro-lymarenko/cypress-bug example and tried to see the failure with Cypress v4 and v6 and could not. My repo https://github.com/bahmutov/cypress-bug also includes https://github.com/bahmutov/cypress-repeat tool so that I could run the tests N times in a row. I ran the project 10 times with "test": "cypress-repeat run -n 10" but saw no failures. Then I changed the spec to run the test 100 times and I finally saw the failure!

/// <reference types="cypress" />
describe("cypress bug", () => {
    Cypress._.times(100, (k) => {
        it(`test ${k} of 100`, () => {
            cy.server();
            cy.route("get", "/api/posts").as("getPosts");

            cy.visit("http://localhost:3000");

            cy.wait("@getPosts");
            cy.wait("@getPosts");
            cy.wait("@getPosts");
            cy.wait("@getPosts");
        });
    })
});

I have replaced cy.server + cy.route with cy.intercept and ran the cypress open test again (100 times) and it works.

/// <reference types="cypress" />
describe("cypress bug", () => {
    Cypress._.times(100, (k) => {
        it(`test ${k} of 100`, () => {
            // cy.server();
            // cy.route("get", "/api/posts").as("getPosts");
            cy.intercept("GET", "/api/posts").as("getPosts");

            cy.visit("http://localhost:3000");

            cy.wait("@getPosts");
            cy.wait("@getPosts");
            cy.wait("@getPosts");
            cy.wait("@getPosts");
        });
    })
});

Then I ran it using cypress-repeat run -n 10 so it executed the test 1000 times total - no errors.

You can find my fork in https://github.com/bahmutov/cypress-bug

Suggestion

Try using Cypress v6.0.0 with cy.intercept and use cypress-repeat to run the test runner again and again to figure out flake. If all else fails, turn on test retries to get through this issue.

[smartgammadev]

Tested using intercept with Cypress 6.0.0 and it looks to be working, at least on the first try.

There might be a bug with the Cypress UI test runner. I've noticed that the first time I run tests in the Cypress UI they pass, however when I re-run tests but clicking the "Run All Tests" arrow button in the browser, cy.wait does not yield the calls. If I "Stop" the test runner in the Cypress UI, then click the spec file to run tests again, they pass the first time.

First run:

After clicking the arrow to rerun all tests:

Exactly the same issue for me too

[evba1]

This is how I have managed to get this working:

// Create a new command in the commands.js file to intrecept all your api requests.
// I have done this for my project which has 100+ api routes.
// This is a basic example just to give the idea.

Cypress.Commands.add("interceptApiRequests", () => {
  // enable stubbing
  cy.server();

  cy.route("GET", `${Cypress.env("apiUrl")}/users`).as("getUsers");
  cy.route("POST", `${Cypress.env("apiUrl")}/users`).as("postUser");
  cy.route("GET", `${Cypress.env("apiUrl")}/projectss*`).as("getProjects");
  ...
});

// After this I have added some global hooks in the support/index.js file to call the api interceptors
// I also have some logic for the login here but you can ignore that.

before(function () {
  cy.stubTrackingRequests();
  cy.interceptApiRequests();
  if (!this.test.file.includes("auth.js")) {
    cy.login(Cypress.env("user_email"), Cypress.env("user_password"));
  }
});

beforeEach(function () {
  cy.stubTrackingRequests();
  cy.interceptApiRequests();
});

// The last step is waiting for the api calls in your test file.
// some basic example to give the idea:

it('should save the user and go to the users list after clicking the save button', () => {
    ...
    cy.get(".save-button").click();
    // wait for api requests to finish
    cy.wait(["@postUser", "@getUsers"]);
    ...
})