Cypress: Document that cy.visit() requires baseUrl to avoid test restart (which loses all state)

[martinpitt]

I have a trivial test, which exercises beforeEach(): (Copied here for convenience):

describe('examples.cypress.io tests', function () {
    beforeEach('set up', function() {
        cy.log("XXX setup");
        cy.wait(2000);
    });

    it('click on "type"', function () {
        cy.log("XXX test");
        cy.wait(2000);
        cy.visit('https://example.cypress.io');
        cy.contains('type').click();
        cy.location('pathname').should('eq', '/commands/actions');
    });
})

It adds logs and waits so that it's easier to see what happens.

Current behavior:

• The beforeEach() runs, writes its message.
• The test starts, writes the message, gets up to the cy.visit(), then the page immediately shuts down without a visible error message.
• The beforeEach() runs again.
• The test runs again, this time cy.visit() and the subsequent steps succeed.

Generated video: http://piware.de/tmp/cypress-example.js.mp4

Desired behavior:

The test should succeed right away, and beforeEach and the test should run just once.

Steps to reproduce:

git clone https://github.com/martinpitt/cypress-bug-before.git
cd cypress-bug-before
npm install
$(npm bin)/cypress run --headed

This also happens in headless mode, or interactively with cypress open

Versions

Cypress: 3.1.0
Browser: Electron 59 (headless)
OS: Fedora 28 x86_64

[martinpitt]

This also happens with before(). I also tried to move the before() handler out of the describe into the global scope. Nothing makes a difference.

[egucciar]

@martinpitt IMO it looks like everything just ran fine... Cypresss looks very odd when it does a .visit and flashes/reloads window when new baseUrl is being hit first time..bet it has something to do with their origin policy. Anyway, it looks like

• Cypress loads without a visit and runs a few initial commands (expected)
• Cypress reaches the point where it visits. Detects a new origin is being hit, and therefore reloads the entire runner. This causes a flash (Expected)
• Cypress replays the events on the logger to bring the runner on the reloaded browser up to date (expected)
• Cypress continues to run remainder of commands (expected)
• Cypress marks tests as "passed" with 1 passed test (expected)

[martinpitt]

Thanks @egucciar for this explanation! This indeed matches the symptoms.

In my "real" test, beforeEach() initializes some state (like start a test VM and set it up - this does need to happen for each individual test, so I can't do it from outside). There I don't just get one repeat, but an infinite loop. I suppose whenever cypress restarts to load the page, it also starts a new VM, and thus a new web server, and thus a new origin.

There I store state in this.currentTest, as in this change to the reproducer here. It shows that in between these reloads, that state is also reset, so that cannot be used. However, state in the describe() scope or in the global scope is also reset -- i. e. it seems the entire source code file gets re-ran from scratch.

This seems to make it very difficult to maintain state across visit() reloads - one could use a task and write it to disk, but that's cumbersome. Is there a trick to it?

[martinpitt]

I tried to put the state into localStorage, but that gets reset as well.

[egucciar]

Perhaps because you need a domain before you attempt to store localStorage. I have great success setting localStorages after setting hbaseUrl and before hitting visit.

Unsure about the vm stuff. I set baseurl always as a first step in each test, so the runner doesn't reload when I visit. It reloads when I set it but I set it before issuing a single command

[bahmutov]

could you write state into JSON file and then load it back in?

On Tue, Oct 23, 2018 at 12:38 PM Erica Gucciardo [e-mail]
wrote:

Perhaps because you need a domain before you attempt to store
localStorage. I have treat success setting localStorages after setting
haseUrl and before hitting visit.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cypress-io/cypress/issues/2636#issuecomment-432194300,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACHApgGjVcOPoCRnf1RNodsRZbngFz6iks5unvGlgaJpZM4X0Ul2
.

--
Dr. Gleb Bahmutov, PhD

Schedule video chat / phone call / meeting with me via
https://calendly.com/bahmutov
gleb.[email protected] @bahmutov https://twitter.com/@bahmutov
https://glebbahmutov.com/ https://glebbahmutov.com/blog
https://github.com/bahmutov

[martinpitt]

@egucciar : Thanks for the baseUrl hint, I'll investigate that. There are also other approaches like temporarily changing localStorage cleaning, I'll play around with that as well. Setting baseURL sounds really promising, and a colleague of mine was successful with the localStorage approach.

could you write state into JSON file and then load it back in?

This is really awkward: It's certainly possible to find out the test name in before()/beforeEach(), so that you can create a testname-state.json, but where do you write it? You shouldn't use a predictable directory like just /tmp/, as that breaks running several tests in parallel and is prone to not cleaning up properly on failures. And if you use a mkdtemp style directory, you still need to remember the name of that -- and that's again this very problem.

Also, this is way too much effort IMHO, and too brittle. The objective of cypress.io is to do away with all the madness and workarounds that you have to do with other systems. And in general it does an amazing job of that (kudos!), it's just that this behavior of restarting the test from scratch on the first .visit() is surprising, not documented, and not obvious how to deal with.

I'll follow up here with a recipe once I found it. Issue #461 shows that I'm not alone here :)

[martinpitt]

@egucciar : Thanks so much for the baseUrl hint! :heart:

This works like a charm: https://github.com/martinpitt/cypress-bug-before/commit/372f56ea8955c6a

So this is now "just" a documentation issue: https://docs.cypress.io/api/commands/visit.html mentions setting baseUrl as a convenience feature, but not that it will cause a test restart if it is not start. Could that be explained there, please?

[jennifer-shehane]

I created a new issue in our docs to document that cy.visit() will reload the entire test when used here: https://github.com/cypress-io/cypress-documentation/issues/1154. Our documentation is open source and contributions are welcome. 😄

[JimLynchCodes]

I just posted about this issue here: https://github.com/cypress-io/cypress/issues/3454

This "test restarts after 3 seconds when you don't specify a baseUrl" is totally counter-intuitive, makes absolutely no sense, and should be patched in the Cypress source code ASAP!

[ItsWasteD]

This still isn't fixed for me. I have my baseUrl set and in some tests after some steps (it) it looks like the spec-file is reloaded. For me it is a problem because in one step a variable with a timestamp is set and after the spec-file is reloaded that timestamp isn't the same anymore. Therefore cy.contains(timestamp) isn't working because the file created with that timestamp in the name isn't found so the test is failing... What can I do?

[jennifer-shehane]

The spec file will reload if you visit a url different from the baseUrl set. If you don't want the spec file to reload, then define a baseUrl.

Fixing this is not simple, as Cypress has to navigate to the url under test and rehydrate the entire test suite whenever it encounters a new cy.visit() with a url different from the url it is on.

[ItsWasteD]

Im actually using a baseUrl but I have some test-suites, which need to visit a different host. For example, in my normal test I just visit the loadbalancer of an application, which directs it to one of two frontends. But I want to test that each of these frontend's are working so I just use the full URL for cy.visit() so my baseUrl=https://loadbalancer.tld.ch/ and in those two test-suites I just use cy.visit("https://frontend1.tld.ch/") and for the second frontend the same. But the funny thing is, that these aren't even the test where the spec file gets reloaded. It happens at a different test after already executing like three steps of the test-suite...

But there should be no problem with that right?? @jennifer-shehane

[jennifer-shehane]

It should reload whenever it hits a cy.visit() command visiting a domain different than the domain that is currently within the main window. It's possible there could be something else happening in your situation. I'd have to see an example to know for sure.

[ItsWasteD]

The weird thing is that it doesn't always happen. Sometimes it works but most of the time it isn't so I'm not sure if will be able to provide a reproducible test. There is just one visit command before it happens and its a normal visit command, with using the baseUrl, something like cy.visit("/testfolder"). I will try to get more information. Should I open a new Issue if I found a reproducible example or should I post it here? Because I'm not sure if it actually is the same problem because I have a baseUrl set plus I'm not visiting a different domain in the test where it happens. @jennifer-shehane