Testcafe: Disable page caching in browsers for login pages

[dbschwartz]

What is your Test Scenario?

When using a role, when localStorage is set programmatically (via a button click) and is then accessed on a new page, it becomes completely lost. This happens in Test Cafe versions 1.0.0 onward. NOTE: the role is not finished yet when this is happening.

What is the Current behavior?

A localStorage key becomes null after going to a new page in the middle of using a role

What is the Expected behavior?

The localStorage key should not become null in the middle of using a role

What is your web application and your TestCafe test code?

This is my demo repo https://github.com/dbschwartz/test-cafe-local-storage

It runs against a website I built to demonstrate it.

Your website URL (or attach your complete example):

The test code can be viewed in my demo: https://github.com/dbschwartz/test-cafe-local-storage/blob/master/test.js

Your complete test code (or attach your test files):

 import { Selector } from 'testcafe';
import { testRole } from './roles/role'


fixture `This is an example showing the test working without using a role`
    .page `https://dbschwartz.github.io/local-storage-demo`

// Tests
test('Text typing basics', async t => {
    await t
    .click(Selector('#first'))
    .click(Selector('#second'))
    .click(Selector('#third'))

});



fixture `This is an example showing the test failing while without using a role`
    .page `https://dbschwartz.github.io/local-storage-demo`
    .beforeEach( async t => {
        await t.useRole(testRole)
    });

// Tests
test('Placeholder', async t => {
});

Your complete test report:

✖ Placeholder

   1) - Error in Role initializer -
      A JavaScript error occurred on
      "https://dbschwartz.github.io/local-storage-demo/localstorage.html".
      Repeat test actions in the browser and check the console for errors.
      If you see this error, it means that the tested website caused it. You
      can fix it or disable tracking JavaScript errors in TestCafe. To do
      the latter, enable the "--skip-js-errors" option.
      If this error does not occur, please write a new issue at:

   "https://github.com/DevExpress/testcafe/issues/new?template=bug-report.md".

      JavaScript error details:
      TypeError: Cannot read property '0' of null
          at
      https://dbschwartz.github.io/local-storage-demo/localstorage.html:15:26

Screenshots:

Steps to Reproduce:

1. Go to my demo repo: https://github.com/dbschwartz/test-cafe-local-storage
2. Clone the repo
3. Do an npm install
4. Run the tests via npm test
5. See the error...

Your Environment details:

• testcafe version: 1.1.4
• node.js version: 8.10.0
• command-line arguments: testcafe chrome test.js
• browser name and version: Chrome
• platform and version: Window 10
• other: This has already been referenced in another issue

[AlexKamaev]

I see the same error and think that it does not relate to the role mechanism (at least in this example).
I found that the localstorage.html page is invalid, since it has an incorrect script tag:

<script='./script.js'></script>

Moreover, the script section is defined after the html closing tag.
I fixed these issues in my local example and your tests work fine.
If you still have other problems, please feel free to update your example with new tests.

[dbschwartz]

@AlexKamaev , I don't believe that's the issue.

I fixed the script tag <script='./script.js'></script> to <script='script.js'></script> so that now includes a valid javascript file and I added a </html> tag to the end localstorage.html and the problem is still happening.

The problem is during the role the data in hammerhead|storages-sandbox-temp in localStorage is not being transferred to hammerhead|storage-wrapper||dbschwartz.github.io correctly as it does without a role.

Without a role the data in hammerhead|storage-wrapper||dbschwartz.github.io is

[["storageBucket"],["[{\"extra\":\"adfasdfasdf\",\"testObj\":{\"abcdef\":\"12345\",\"ghijkl\":\"123123\"},\"link\":\"https://devexpress.github.io/testcafe/example/\"}]"]]

but with a role it's [[],[]].

That is why when a role is used, the test fails for the JavaScript error of accessing an undefined localStorage key. The link also no longer exists in localStorage.html because it's depending on localStorage to create it.

I will post screenshots as soon as I am able.

[AlexKamaev]

I was able to reproduce the issue. I need some additional time to research it in detail.

[dbschwartz]

Thanks @AlexKamaev, I really appreciate it.

[AlexKamaev]

I researched your issue and found that it's related to the Network Cache. You can see the detailed information in this question on StackOverflow.
Could you please try to run your tests with --disk-cache-size=1 flag, e.g testcafe "chrome --disk-cache-size=1" test.js?
Meanwhile, I recommend that you use this workaround. However, I hope that we will fix this behavior in the near future.

[dbschwartz]

Thank you so much for your help @AlexKamaev! This works!

The only caveat I might add is that the problem is also happening in Firefox, so cacheing would have to be disabled there too and perhaps IE, Edge and Safari.

I find it odd that it's cacheing issue because localStorage is supposed to happen on the browser itself, rather than through Network Requests. Maybe since TestCafe runs all its commands through the hammerhead server is the reason behind the cacheing issue?

Thanks so much for finding a workaround, I hope there is a patch soon so that we can run our tests in all browsers again.

[AlexKamaev]

I find it odd that it's cacheing issue because localStorage is supposed to happen on the browser itself, rather than through Network Requests. Maybe since TestCafe runs all its commands through the hammerhead server is the reason behind the cacheing issue?

You are right, the cause of the issue is in the core mechanisms.

The only caveat I might add is that the problem is also happening in Firefox, so cacheing would have to be disabled there too and perhaps IE, Edge and Safari.

We will check this behavior in other browsers as well.

[AlexKamaev]

@dbschwartz
I researched the issue and found the workaround that seems to be suitable for your case. Take special note that it uses our internal API, and you need to use it carefully.
You can create a custom http request hook in the following manner:

import { RequestHook } from 'testcafe';

class CustomRequestHook extends RequestHook {
    _onConfigureResponse (event) {
        super._onConfigureResponse(event);

        event.setHeader('cache-control', 'no-store');
    }

    async onRequest (event) {
    }
    async onResponse (event) {
    }
}

const hook = new CustomRequestHook(/.*/);

fixture `...`
    .requestHooks(hook)

Here I override the internal _onConfigureResponse method, which allows you to modify response headers and prevent a web page from caching.
I recommend you use this workaround until the issue is fixed.

[Evilweed]

@dbschwartz sadly Your workaround with CustomRequestHook does not compile for version 1.2.0 :( Can You provide one for this version?

Adding --disk-cache-size=1 does not help in my case.

[Farfurix]

@Evilweed

Hello,

This workaround works in the latest v1.2.0 version.
If you have any workaround questions to implement your scenario, it will be best to ask them on StackOverflow. We try to keep the GitHub issues tracker for bugs and feature requests only.
And, of course, feel free to open a new report if you encounter any bugs.

[dbschwartz]

Thank you so much for all your help @AlexKamaev.

@Evilweed,
I tried the workaround and it works in v1.2.0, but there are two caveats:

1. If you do all browsers at the same time (testcafe all test.js) you will get a JavaScript error with No stack trace that will fail all the tests.
2. After you do the failing testcafe all test.js, the browsers do not automatically close.

Other than that this seems to be a good workaround! 👍

Thank you so much TestCafe team for all your hard work

[AllanNobre]

@Evilweed and @Farfurix

Hello,

I was having the same problem, the two workarounds didn't work for my tests that used Role.

However, when I removed the .page (...) from my fixture ... - using {preserveUrl: true} on my Role -, the login token stored in Local Storage was not deleted between the execution of the tests.

I suspect that the behaviour - Local Storage is being lost while using Role - might be related to the use of .page (...) in fixture along with {preserveUrl: true} in Role.

When I used both together the Token behavior in Local Storage varies between:

• 'null' value just after login, within Role code execution;
• valid value within the execution of the Login on Role code and value 'null' shortly after running the Role and running the tests;

Then, as a workaround, what worked for me was:

Role:

export const adminRole = Role(
  loginPageUrl,
  async t => {
    // Perform login
    await t
      .typeText(userNameInput, USER)
      .typeText(userPasswordInput, PASSWORD)
      .click(loginPageButton);

    // Just for test: Verify if Login with userRole was performed
    const loginToken = await getLocalStorage('Token');
    console.log('IN ROLE');
    console.log(loginToken); // <-- Unpredictable behavior when used with `.page (...)` in fixture: null or valid value only at this point in the rest of the tests is always null
  },
  { preserveUrl: true },
);

// Workaround to keep Local Storage between useRole tests;
class CustomRequestHook extends RequestHook {
  _onConfigureResponse(event) {
    super._onConfigureResponse(event);
    event.setHeader('cache-control', 'no-store');
  }
  async onRequest(event) {
    event;
  }
  async onResponse(event) {
    event;
  }
}

export const roleWorkaroundHook = new CustomRequestHook(/.*/);

Fixture and test:

fixture`Visit x page`
  .requestHooks(roleWorkaroundHook)
  // .page(xPageUrl) <-- Uncomment to verify abnormal behaviour
  .beforeEach(async t => {
    await t.useRole(adminRole);
    // Just for test
    console.log('Fixture');
    console.log(await getLocalStorage('Token'));
  });

test('X render with logged user', async t => {
  // Just for test
  console.log('test1 - before.js');
  console.log(await getLocalStorage('Token'));
  await t
    .navigateTo(xPageUrl)
    .hover(...)
    .rightClick(...)
    .click(...)
    .expect(getCurrentPageUrl())
    .contains(xPageUrl, { timeout: 5000 });

  // Just for test
  console.log('test1 - after.js');
  console.log(await getLocalStorage('Token'));
});

test('X does not render with unlogged user', async t => {
  // Just for test
  console.log('test2 - before.js');
  console.log(await getLocalStorage('Token'));

  await useAnonymousRole(); // <-- Need reload the page to real logout
  await t
    .navigateTo(xPageUrl)
    .expect(getCurrentPageUrl())
    .contains(loginPageUrl, { timeout: 5000 });

  // Just for test
  console.log('test2 - after.js');
  console.log(await getLocalStorage('Token'));
});

Auxiliary Functions:

export const getLocalStorage = ClientFunction(key => localStorage.getItem(key));

export const getCurrentPageUrl = ClientFunction(() =>
  window.location.href.toString(),
);

[AlexKamaev]

@AllanNobre
We are happy to hear that you found a workaround for your issue. The roles mechanism is quite complex, so there can be a lot of unexpected issues which can differ from one project to another. The @dbschwartz's issue is definitely related to the Network cache. This issue cannot be related to it. Despite you have a workaround we will appreciate it if you share your project with us. It will allow us to research the problem in detail.

[AllanNobre]

@AlexKamaev
The app is in a private corporate environment so I'm not able to share :( , sorry!

[Farfurix]

@AllanNobre

Feel free to open a new issue if you will have some project (or public URL) to reproduce your scenario issues.

[lock[bot]]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.