Cypress: Flaky tests due to Cypress latencies in sending XHR requests

[dirtyhenry]

Current behavior:

Occasionally, a test will fail because an XHR request triggered by the app under test is actually sent on the network later than requested by the user action (ie past the 4s timeout — or my customized 20s for that matter, cf. screenshot below — of the next get query, which would have worked fine the rest of the time).

After investigating, here's what I can see:

1. According to our networking logs, our backend responded in 57ms.

2. According to cypress open logs, the request took 139ms:

   POST /api/OBFUSCATED 200 139.049 ms - -
   

3. According to Cypress's Test Runner, the request took 121375ms, which triggered a timeout:

By plugging Mitmproxy, I was able to determine the request goes through the network about 2 minutes later than expected.

Desired behavior:

The request should not be retained by Cypress and sent right away.

Steps to reproduce: (app code and test code)

Unfortunately, this behavior happens randomly and can not be reproduced at demand, especially since it requires a backend to respond to requests. It happens every couple of tests though, and seems to always take exactly 120 seconds before the request is actually sent.

Also, so far, it only concerned POST or PUT requests.

Versions

• Cypress 3.1.5
• macOS 10.14.3
• Google Chrome 72.0.3626.119 (Official Build) (64-bit)

Note: a first version of this issue was assuming the request was sent right away but the delay was created while processing the response. It turned out this first analysis was wrong and was the other way around: requests were sent too late.

[dirtyhenry]

💁‍♂️ I updated the issue for the following reason: at first I was assuming the request was sent right away but the delay was created while processing the response. It turned out this first analysis was wrong and the behavior was the other way around: requests were sent too late. Hence, I updated the issue accordingly.

[Bkucera]

@dirtyhenry not to say this isn't a bug, but it might help you to try out cypress-plugin-retries if you're having flaky tests. Official retry support is coming soon as well.

we're working on the proxy layer as part of #687, so this may be fixed alongside that. However I do not have a timeline

[jennifer-shehane]

@dirtyhenry We have some network speed improvements in 3.3.0. Can you update and let us know if you are still experiencing this issue?

[jennifer-shehane]

Unfortunately we have to close this issue due to inactivity. Please comment if there is new information after updating to the current Cypress version to provide concerning the original issue and we can reopen.