Cypress: __/#/tests/... page errors 404 - page not found on Chrome 72+

[pklejnowski]

Is this a Feature or Bug?

BUG

Current behavior:

Sometimes when I run all tests I have such error:

Important thing that I noticed If i remove tests "group_page.js" (check gif) it seems that the problem disappeared. "group_page.js" using different page: cy.visit(https://localhost:5002) for testing when the other tests that crashing using cy.visit(https://localhost:5000)

Check this gif, I had to run 3 times RUN ALL TESTS to make it work. Before always failed at login_page.js

I was trying hardcode url in cy.visit() (before I had url from cypress.json). The same result.

Steps to reproduce:

Run a few tests that include cy.visit(https://localhost:5002) for 1st test and cy.visit(https://localhost:5000) for 2nd test.

Versions

Cypress: 3.0.1
Windows 10
Chrome

[bgp1]

I want to add in that I have this same issue. Thought upgrading to 3.x branch would fix this but persists as it did in the 2.x branch.

I have two spec files, each testing a different user experience. One is on one domain, one is on another. The first spec file to run (alpha order first) completes without issue. When it tries to go to the second spec file it refreshes, runs the first spec file again, then when it hits the second spec file for a second time, results in a 404 not found for the /#/tests/__all path.

If I run each file individually the tests complete successfully. This is problematic as these run after each release and therefore both files need to be able to run for each request.

This occurs via interactive mode. If I run it via command line the first spec file completes successfully and the second file hangs. Cypress never terminates, which is also problematic.

Cypress 2.1.0 or 3.0.1
Windows 10
Chrome

[bgp1]

Wanted to add to my previous comment with additional information. I have been testing this and I can't seem to get this to work when it's on the same domain, either. The second feature I wanted to test is a self-hosted feature, meaning it's a JS file that other sites would add to their site. I was testing it on a different domain that we own. However, I have since created a page to host this for testing on the same domain as the other tests and it still does this same behavior as before. So I don't believe this to just be an issue because of two different domains.

I believe this is an issue with the tests being broken out into multiple spec files. Even though neither test file inherits from the other I am able to replicate that the first file completes, runs a second time (for some unknown reason) and then the second test file stalls.

I have been trying to see if I can get all tests to run in a single spec file, but so far that has proved unsuccessful for some unknown reason I am still trying to determine.

I tested this on both 2.x and 3.x versions and the problem persists when tests are split between files.

[pklejnowski]

@bgp1 Did you find any workaround so far?

[bgp1]

My workaround is to host the second application on the same domain as the first (I just created a page on there to trigger it) and then included all of the tests for both applications in a single spec file. Working for me, although not ideal.

[pklejnowski]

:/ I can't do the same thing. So I have to wait for fix...

[pklejnowski]

How long it can be in stage: "needs investigating"?

[pklejnowski]

@jennifer-shehane is it possible that you are going to fix it soon?

[jennifer-shehane]

I was not able to reproduce this issue.

I visited 2 websites on localhost with different ports - visiting the first url in spec1.js and the second url in spec2.js then running "All Tests" and ran this several times.

spec1.js

context('Site 1', () => {
  beforeEach(() => {
    cy.visit('http://localhost:8080/commands/actions')
  })

  it('get an el', () => {
    cy.get('.action-email')
  })
})

spec2.js

context('Site 2', () => {
  beforeEach(() => {
    cy.visit('http://localhost:3030/')
  })

  it('gets el', () => {
    cy.contains('Features')
  })
})

@pklejnowski There is some other factor that is influencing the crash and will have to have a reproducible issue in order to begin investigating what the issue is.

[jennifer-shehane]

@bgp1 In reference to your comment:

When it tries to go to the second spec file it refreshes, runs the first spec file again

The first test file is not running again, but since we did go to a different url in the main window browser, the first test in the Command Line does have to re - render, so it * looks * like it is running again, when the Command is just rendering to the fresh page.

[Gachapen]

I'm having the same issue, and am struggling to debug it. It works fine when running against localhost, but when I try running it against a deployed version of the application from my computer, I get 404 instead of the cypress runner application. The 404 page shown is the 404 page generated by our application (using Vue). This makes sense, because the URL path is /__/ which should result into a 404 in our application. Something seems to prevent the cypress runner from starting up, instead it loads the application directly.

This happens both when I run a single spec file and when I run all.

[SimoneDahmen]

I have the same Issue as @Gachapen. A "/_" is added to the app's URL and it's loaded in the chrome address bar not in the cypress-created view within.
happens with cy.visit():

describe('create content', function () {
        it('opening app', function () {
            cy.wait(1000),
                cy.visit('https://dev.site.com');
        });
});

and the result is our App's 404 page on baseurl + "/underscore"

For me it might be an issue that I'm using an external URL.

[acailly]

If it helps, I ran into this problem by following the tutorial.

Steps I followed:

mkdir cypress-test
cd cypress-test
yarn init -y
yarn add cypress --dev
yarn cypress open

Then create a sample_specs.js file with:

describe('My First Test', function() {
  it('Visits the Kitchen Sink', function() {
    cy.visit('https://example.cypress.io')
  })
})

Then click on this file in cypress GUI:

• the browser open at http://localhost:65327/__/#/tests/integration\sample_specs.js
• the browser navigate to https://example.cypress.io/__/#/tests/integration\sample_specs.js which gives a 404 error

Node version: 8.11.1
Npm version: 5.6.0
Yarn version: 1.10.1
OS: Windows 7

[SimoneDahmen]

thank you @acailly . That really did help because it made me check the examples again.

Did you check wether your sample_specs.js file was really in the directory you created? My cypress was not installed through yarn but npm, I see no other difference though

The browser also navigated to example.cypress.io but successfully:

[vadymkukhtinOD]

@jennifer-shehane @acailly @SimoneDahmen
Same issue here.
The reason for me is that I have service-worker which is precaching assets.
If I unregister service-worker everything works fine.
If I remove manually "/___/" on debug mode and reload page, tests are working fine.
The problem is that first test runs well, then cypress goes to 2nd one and 2nd test case starts on 404 "..../___/#/tests/integration" already even not reaching beforeEach func.

[acailly]

@vadymkukhtinOD I don't see any registered service-worker in the application tab of dev tools

[pklejnowski]

I don't have any service-worker and this problem still occurs in my application.
What is interesting. I have 3 test files. When I run each test separatly, then this problem is not occurs, but when I run all at once, then this problem occurs.

[blakelee-pendo]

+1
We're having the same issue

[vadymkukhtinOD]

@acailly Probably service-worker just covering the real issue. Still investigating

[matt212]

+1
We're having the same issue

[matt212]

@jennifer-shehane any updates or workaround since it's bit of a hinderance when writing test cases.

[jennifer-shehane]

Unfortunately we cannot investigate an issue if no reproducible example is provided. Can anyone provide a way to reproduce this exactly?

[SimoneDahmen]

@jennifer-shehane I am working on a Mac, have a parent folder with one cypress child and one yarn child. In the yarn child I have the app files, in the cypress/integration I have my tests files. Both child folders have their own git repository.
I installed cypress via console, using npm.

That very integration folder I open in cypress (cypress I have in my applications so I'm not starting it via terminal).
Then I click the file in the cypress window.
Then the chrome window opens and starts running the test but is redirected to the url mentioned above after a second.
This is the code I try to visit the page with:
it('opening /moments', function () { cy.visit('localhost:3000/moments'); });

When reproducing this again, I noticed that the first script I run after opening cypress works, but the ones afterwards don't. So when I close/stop the remote chrome window and try to run another test file, I will have this issue.

('this site does not exist – back to index')

[jennifer-shehane]

@SimoneDahmen Ok, you are running cypress in what we call 'global mode'.

I'm still a bit confused as to what your project directory looks like and which part you are opening in global mode from the screenshots and information given.

• Could you share your cypress.json configuration? As I'm sure you have edited some of the folder's locations like these: https://on.cypress.io/configuration#Folders-Files
• Could you draw an example of your folder config like below?

📁my-web-app -- // this folder I drag into cypress
  📁app
  📁cypress
    📁fixtures
    📁integration
    📁plugins
    📁support
  📁node_modules
  📄cypress.json
  📄package.json

[jarrodldavis]

I'm running into this same issue and it appears to be caused by Chrome/Chromium not respecting the --proxy-server command line flag (provided by Cypress when launching the browser) for localhost and 127.0.0.1 URLs. I confirmed this behavior using debugging output both by Cypress (using DEBUG=cypress:* ./node_modules/.bin/cypress open --config baseUrl=http://127.0.0.1:8080/) and Chromium's own network logging (chrome://net-export/).

Here's a snippet of those net-export logs:

{"params":{"priority":"HIGHEST","url":"http://127.0.0.1:8080/__/#/tests/integration/test.js"},"phase":1,"source":{"id":180,"type":1},"time":"8498558","type":2},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":100},
{"phase":2,"source":{"id":180,"type":1},"time":"8498559","type":100},
{"params":{"load_flags":9217,"method":"GET","privacy_mode":0,"url":"http://127.0.0.1:8080/__/#/tests/integration/test.js"},"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":96},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":99},
{"phase":2,"source":{"id":180,"type":1},"time":"8498559","type":99},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":113},
{"phase":2,"source":{"id":180,"type":1},"time":"8498559","type":113},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":114},
{"params":{"entry_hash":"0x52f2db36893f29dc"},"phase":1,"source":{"id":181,"type":13},"time":"8498559","type":355},
{"params":{"key":"http://127.0.0.1:8080/__/"},"phase":0,"source":{"id":181,"type":13},"time":"8498559","type":356},
{"phase":0,"source":{"id":181,"type":13},"time":"8498559","type":379},
{"phase":0,"source":{"id":181,"type":13},"time":"8498559","type":380},
{"params":{"net_error":-411},"phase":2,"source":{"id":180,"type":1},"time":"8498559","type":114},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":115},
{"params":{"entry_hash":"0x52f2db36893f29dc"},"phase":1,"source":{"id":182,"type":13},"time":"8498559","type":355},
{"params":{"key":"http://127.0.0.1:8080/__/"},"phase":0,"source":{"id":182,"type":13},"time":"8498559","type":356},
{"phase":0,"source":{"id":182,"type":13},"time":"8498559","type":360},
{"phase":0,"source":{"id":182,"type":13},"time":"8498559","type":361},
{"phase":2,"source":{"id":180,"type":1},"time":"8498559","type":115},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":116},
{"params":{"address_family":0,"allow_cached_response":true,"host":"127.0.0.1:8080","is_speculative":false},"phase":1,"source":{"id":183,"type":0},"time":"8498559","type":3},
{"phase":2,"source":{"id":183,"type":0},"time":"8498559","type":3},
{"phase":2,"source":{"id":180,"type":1},"time":"8498559","type":116},
{"params":{"is_preconnect":false,"url":"http://127.0.0.1:8080/__/#/tests/integration/test.js"},"phase":1,"source":{"id":184,"type":23},"time":"8498559","type":151},
{"phase":1,"source":{"id":180,"type":1},"time":"8498559","type":137},
{"params":{"source_dependency":{"id":184,"type":23}},"phase":0,"source":{"id":180,"type":1},"time":"8498559","type":152},
{"params":{"source_dependency":{"id":180,"type":1}},"phase":0,"source":{"id":184,"type":23},"time":"8498559","type":152},
{"phase":1,"source":{"id":184,"type":23},"time":"8498559","type":23},
{"params":{"pac_string":"DIRECT"},"phase":0,"source":{"id":184,"type":23},"time":"8498560","type":25},
{"phase":2,"source":{"id":184,"type":23},"time":"8498560","type":23},
{"params":{"proxy_server":"DIRECT"},"phase":0,"source":{"id":184,"type":23},"time":"8498560","type":153},

Notice the last line: {"proxy_server":"DIRECT"},.

Here's another snippet of those same logs, this time connecting to the same local server using my computer's network IP instead of 127.0.0.1:

{"params":{"priority":"HIGHEST","url":"http://10.0.0.249:8080/__/#/tests/integration/test.js"},"phase":1,"source":{"id":195,"type":1},"time":"8515014","type":2},
{"phase":1,"source":{"id":195,"type":1},"time":"8515015","type":100},
{"phase":2,"source":{"id":195,"type":1},"time":"8515015","type":100},
{"params":{"load_flags":9216,"method":"GET","privacy_mode":0,"url":"http://10.0.0.249:8080/__/#/tests/integration/test.js"},"phase":1,"source":{"id":195,"type":1},"time":"8515015","type":96},
{"phase":1,"source":{"id":195,"type":1},"time":"8515015","type":99},
{"phase":2,"source":{"id":195,"type":1},"time":"8515015","type":99},
{"phase":1,"source":{"id":195,"type":1},"time":"8515015","type":113},
{"phase":2,"source":{"id":195,"type":1},"time":"8515015","type":113},
{"phase":1,"source":{"id":195,"type":1},"time":"8515015","type":114},
{"params":{"entry_hash":"0xfcb31c53c472bc92"},"phase":1,"source":{"id":196,"type":13},"time":"8515015","type":355},
{"params":{"key":"http://10.0.0.249:8080/__/"},"phase":0,"source":{"id":196,"type":13},"time":"8515015","type":356},
{"phase":0,"source":{"id":196,"type":13},"time":"8515015","type":357},
{"params":{"net_error":-2},"phase":0,"source":{"id":196,"type":13},"time":"8515015","type":359},
{"phase":2,"source":{"id":196,"type":13},"time":"8515015","type":355},
{"params":{"net_error":-2},"phase":2,"source":{"id":195,"type":1},"time":"8515015","type":114},
{"phase":1,"source":{"id":195,"type":1},"time":"8515015","type":115},
{"params":{"entry_hash":"0xfcb31c53c472bc92"},"phase":1,"source":{"id":197,"type":13},"time":"8515015","type":355},
{"params":{"key":"http://10.0.0.249:8080/__/"},"phase":0,"source":{"id":197,"type":13},"time":"8515015","type":356},
{"phase":0,"source":{"id":197,"type":13},"time":"8515015","type":360},
{"phase":0,"source":{"id":197,"type":13},"time":"8515015","type":361},
{"phase":0,"source":{"id":197,"type":13},"time":"8515015","type":362},
{"phase":2,"source":{"id":195,"type":1},"time":"8515016","type":115},
{"phase":1,"source":{"id":195,"type":1},"time":"8515016","type":116},
{"phase":2,"source":{"id":195,"type":1},"time":"8515016","type":116},
{"params":{"is_preconnect":false,"url":"http://10.0.0.249:8080/__/#/tests/integration/test.js"},"phase":1,"source":{"id":198,"type":23},"time":"8515016","type":151},
{"phase":1,"source":{"id":195,"type":1},"time":"8515016","type":137},
{"params":{"source_dependency":{"id":198,"type":23}},"phase":0,"source":{"id":195,"type":1},"time":"8515016","type":152},
{"params":{"source_dependency":{"id":195,"type":1}},"phase":0,"source":{"id":198,"type":23},"time":"8515016","type":152},
{"phase":1,"source":{"id":198,"type":23},"time":"8515016","type":23},
{"params":{"pac_string":"PROXY localhost:58283"},"phase":0,"source":{"id":198,"type":23},"time":"8515016","type":25},
{"phase":2,"source":{"id":198,"type":23},"time":"8515016","type":23},
{"params":{"proxy_server":"PROXY localhost:58283"},"phase":0,"source":{"id":198,"type":23},"time":"8515016","type":153},

This time, Cypress' proxy server is being used: {"proxy_server":"PROXY localhost:58283"}

Additional notes:

• I'm using Cypress as configured by [Vue CLI], which provides a nice wrapper around cypress to launch webpack dev server and feed the URL to cypress using --config baseUrl=<url>
• I'm on a Mac running macOS Mojave Version 10.14.1 (18B75)
• I'm (currently) using Chromium Version 73.0.3634.0
• I installed the latest Stable Chrome (Version 71.0.3578.80) and this issue doesn't appear to occur, so perhaps upcoming versions of Chrome may have this issue (or Google is A/B testing something).
• I've tested with other versions (Beta and Dev), and the change appears to have occurred somewhere between the current Beta build (71.0.3578.80) and the current Dev build (72.0.3626.7).
  
  
  
  • The Beta build works – and has the same major version (71) as Stable – but the Dev version does not.
  
  

[jennifer-shehane]

Can confirm - this is happening on Chrome Version 73.0.3658.0 (Official Build) canary (64-bit) for me also.

cypress:server:server resolving visit { url: 'http://localhost:8080/commands/actions', headers: { host: 'localhost:52635', connection: 'Upgrade', pragma: 'no-cache', 'cache-control': 'no-cache', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3658.0 Safari/537.36', upgrade: 'websocket', origin: 'http://localhost:52635', 'sec-websocket-version': '13', 'accept-encoding': 'gzip, deflate, br', 'accept-language': 'en-US,en;q=0.9', 'sec-websocket-key': '1AwgO7Qj7Il2zAkm6MXl1A==', 'sec-websocket-extensions': 'permessage-deflate; client_max_window_bits' }, options: { failOnStatusCode: true, auth: null } } +1ms
cypress:server:server Getting remote state: { auth: undefined, props: null, origin: 'http://localhost:52635', strategy: 'file', visiting: undefined, domainName: 'localhost', fileServer: 'http://localhost:52636' } +0ms

[jennifer-shehane]

Checking through changelogs between Beta build (71.0.3578.80) and the current Dev build (72.0.3626.7)

I'm initially suspecting that this implementation change is the culprit - introduced in Version 72.0.3605.0 - which I can confirm is broken (showing 404 page). Commit: https://chromium.googlesource.com/chromium/src/+/da790f920bbc169a6805a4fb83b4c2ab09532d91

Commit Text

Implicitly bypass localhost when proxying requests.

This aligns Chrome's behavior with the Windows and macOS proxy resolvers (but not Firefox).

Concretely:

• localhost names (as determined by net::IsLocalhost) now implicitly bypass the proxy
• link-local IP addresses implicitly bypass the proxy

The implicit rules are handled by ProxyBypassRules, and it is possible to override them when manually configuring proxy settings (but not when using PAC or auto-detect).

This change also adds support for the "<-loopback>" proxy bypass rule, with similar semantics as it has on Windows (removes the implicit bypass rules for localhost and link-local).

The compatibility risk of this change should be low as proxying through localhost was not universally supported. It is however an idiom used in testing (a number of our own tests had such a dependency). Impacted users can use the "<-loopback>" bypass rule as a workaround.

Download Chromium Version 72.0.3598.0 (WORKING)
Download Chromium Version 72.0.3605.0 (BROKEN)

Further narrowing down of Changelog between broken versions

Other potentially relevant commits:

• https://chromium.googlesource.com/chromium/src/+/d31efeca7c0554367d4ba3a293a0a81b6fe872e0

Commits made after culprit commit that may help with solution (oldest -> newest commits)

• https://chromium.googlesource.com/chromium/src/+/5b1b6fbfb1953a06eb78ec1ea1bf70e124417edb
• https://chromium.googlesource.com/chromium/src/+/1deb8f2c7684650358ca739e7c34f2e17aaa6a0d
• https://chromium.googlesource.com/chromium/src/+/2e38d313a49bc8c28b8ce38f004776dbf4357b99
• https://chromium.googlesource.com/chromium/src/+/efdf315bc867468592bfe7a58da2225935281e7d
• https://chromium.googlesource.com/chromium/src/+/15242c0c52f055089ca410881dc049f43a2c4e62
• https://chromium.googlesource.com/chromium/src/+/7aeea3a2da27e2bf566cf11e75f960124061c79c

[jennifer-shehane]

Current Workaround

Add the code below to your cypress/plugins/index.js file to fix the 404 issue:

module.exports = (on) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') { 
      // ^ make sure this is your browser name, you may 
      // be using 'canary' or 'chromium' for example, so change it to match!
      args.push('--proxy-bypass-list=<-loopback>')
      return args
    }
  })
}

I've opened a PR to fix this is Cypress here: https://github.com/cypress-io/cypress/pull/3049

[cypress-bot[bot]]

Released in 3.1.5.

[smoothdvd]

It's not working in headless mode

[jennifer-shehane]

Hey @smoothdvd, could you open a new issue with a reproducible example of your problem? This issue has been fixed and likely you are experiencing a similar issue due to another cause. Thanks!

[yeya]

Same issue with cypress 3.1.5 and chrome 72.
I'm running npx cypress open on new project with no files at all. Just the auto generated specs.

Same issue with localhost site.

I tried to add the temp code to plugins but I see that the argument already there:
'--disable-default-apps', '--use-fake-ui-for-media-stream', '--use-fake-device-for-media-stream', '--proxy-server=http://localhost:30943', '--proxy-bypass-list=<-loopback>', '--proxy-bypass-list=<-loopback>' ]

[RicardoVaranda]

This is still happening to me in 3.2.0

[gwpetersenag]

I also am still seeing this exact issue on 3.2.0

[jennifer-shehane]

Hey if you're experiencing a similar issue, could you open a new issue with a complete reproducible example of your problem? Thanks!