Testcafe: Third Party Cookies Being Returned in Browsers that Have Them Disabled

[noahpc]

What is your Test Scenario?

I am making an http call that has the following response:

response headers: {
    "access-control-allow-credentials": "true",
    "access-control-allow-origin": "http://testclient.com",
    "cache-control": "no-cache",
    "content-type": "application/json;charset=UTF-8",
    "date": "Wed, 02 Oct 2019 21:11:58 GMT",
    "expires": "0",
    "p3p": "CP=NOI DSP COR NID PSAa PSDa OUR UNI COM NAV",
    "pragma": "no-cache",
    "set-cookie": [
        "_cookie1=41bbe664c37c7417c305fc1458547d60;Path=/;Domain=testserver.com;Expires=Sun, 28-Jun-2020 21:06:00 GMT;SameSite=None",
        "_cookie2=\"ACZ4nGNQMDFMSko1MzNJN\";Version=1;Path=/;Domain=testserver.com;Expires=Sun, 28-Jun-2020 21:06:00 GMT;Max-Age=23328000;SameSite=None",
        "_cookie3=\"ABR4nGNgYGCInSo0jwEGGBlYN6kDaQAkDAKC\";Version=1;Path=/;Domain=testserver.com;Expires=Sun, 28-Jun-2020 21:06:00 GMT;Max-Age=23328000;SameSite=None"
    ],
    "x-server": "10.8.0.233",
    "content-length": "105",
    "connection": "keep-alive"
}

What is the Current behavior?

hammerhead is rewriting the response as

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Content-Type: application/json;charset=UTF-8
Set-Cookie: s|58fWWs3h4|_cookie1|testserver.com|%2F|kbzkc340|k19rpp2b=41bbe664c37c7417c305fc1458547d60;path=/
Set-Cookie: s|58fWWs3h4|_cookie2|testserver.com|%2F|kbzkc340|k19rpp2b="ACZ4nGNQMDFMSko1MzNJN";path=/
Set-Cookie: s|58fWWs3h4|_cookie3|testserver.com|%2F|kbzkc340|k19rpp2b="ABR4nGNgYGCInSr0gwEGGBlYN6kDaQAqYALc";path=/
Pragma: no-cache
Cache-Control: no-cache
Date: Wed, 02 Oct 2019 21:13:28 GMT
Content-Length: 80
Expires: 0
Connection: keep-alive
p3p: CP=NOI DSP COR NID PSAa PSDa OUR UNI COM NAV
x-server: 10.8.0.233 

What is the Expected behavior?

I would expect the domain to be the same as the original response. TestCafe appears to be removing the domain of the cookies. This results in all cookies looking like first party cookies even when they are 3rd party.

Test Code

fixture`Example Test`
    .page('twitter.com');

test('Cookies get rewritten strangely', async t => {
        await new Promise(resolve => setTimeout(() => resolve(), milliseconds));
});

Screenshots:

Steps to Reproduce:

1. Run the example test code
2. Open up the debugger in Safari
3. Look for any google analytics calls that are trying to set third party cookies
4. Observe that the cookies are rewritten to be 1st party instead of third party because the domain parameter is removed. This also happens I would guess because the call is now technically coming from localhost through proxy instead of from the actual third party.

Your Environment details:

• testcafe version: 1.5.0
• node.js version: v12.8.1
• browser name and version: Safari 13.0.1
• platform and version: macOS 10.14.6

[LavrovArtem]

TestCafe uses its own cookie processing mechanism. Cookies are stored on the proxy server side. So, cookies which you've seen in the Network tab of DevTools are our service cookies for the synchronization between server and client.

[noahpc]

@LavrovArtem - The result I am seeing from this is that browser treats these cookies differently when under test and prevents us from being able to fully test scenarios that involve the browser blocking what should be 3rd party cookies.

Is this expected?

In fact, the more I look, I think that is actually the problem. The client-side testcafe code is properly respecting the domain value; however, it is not realizing that cookies should not be permitted to be set on that domain.

[Farfurix]

@noahpc

Hello,

It would be great if you could provide us with a project (or a public URL) to show how the TestCafe cookie processing mechanism negatively affects your page application logic. Our team will research it and check for a suitable solution.

[noahpc]

@Farfurix - Thank you for the response. Much of my work has been in a private environment, but I have gotten approval to put up a test site that will demonstrate the problem more fully.

I will use this site to show that on Safari with third-party cookies disabled, the third-party cookies are still being set by the test-cafe framework. I'm out of office today, but will update this ticket with the site link in the next couple days. Thanks again for your time!

[noahpc]

@Farfurix - I have set up a test page along and hopefully explained the issues I am seeing more clearly. While I setting it up, I realized that the behavior is different (but still unexpected) when a request logger is present versus when there is no request logger. I've added notes about both cases below. Let me know if anything else would be helpful for you. Thanks again!

Explanation of what the test page does:
1) Makes a call to bcp.crwdcntrl.net that will use set-cookie headers to set a 3rd party cookie
2) Pulls in an iframe that is hosted on tags.crwdcntrl.net that is able to determine whether the 3rd party cookie was set
3) Makes a postMessage to the iframe asking whether the 3rd party cookie was able to be set
4) Prints out 1st party if cookie was not set and 3rd party if cookie was set

Expected Behavior:
1) On Chrome, prints out 3rd Party because 3rd party cookies can be set
2) On Safari, prints out 1st Party because 3rd party cookies CANNOT be set

To See Expected Behavior:
1) Simply go directly to http://lotame-public-demo-dev.s3-website-us-east-1.amazonaws.com/ in chrome, observe it says 3rd Party
2) Simply go directly to http://lotame-public-demo-dev.s3-website-us-east-1.amazonaws.com/ in safari, observe it says 1st Party

Run the following tests cases:

Test Code #1: Test Cookies With Logging

import { Selector, RequestLogger, ClientFunction } from 'testcafe';

let requestLogger = RequestLogger(new RegExp('.*'), {
   logRequestHeaders: true,
   logResponseHeaders: true,
   logRequestBody: true,
   logResponseBody: true,
   stringifyRequestBody: true
});

fixture`Test Cookies With Logging`
   .page('http://lotame-public-demo-dev.s3-website-us-east-1.amazonaws.com/')
   .requestHooks(requestLogger);

const getUA = ClientFunction(() => navigator.userAgent);

test('The cookies should be set properly', async t => {
       const ua = await getUA();

       if (ua.includes('Chrome')) {
          await t.expect(Selector('#pid').textContent).contains('3rd Party');
       } else {
         await t.expect(Selector('#pid').textContent).contains('1st Party');
       }
});

Test Code #2: Test Cookies Without Logging

import { Selector, ClientFunction } from 'testcafe';

fixture`Test Cookies Without Logging`
   .page('http://lotame-public-demo-dev.s3-website-us-east-1.amazonaws.com/');

const getUA = ClientFunction(() => navigator.userAgent);

test('The cookies should be set properly', async t => {
       const ua = await getUA();

       if (ua.includes('Chrome')) {
          await t.expect(Selector('#pid').textContent).contains('3rd Party');
       } else {
         await t.expect(Selector('#pid').textContent).contains('1st Party');
       }
});

Fixture: Test Cookies With Logging
Browser: chrome
Result: PASS

Fixture: Test Cookies With Logging
Browser: safari
Result: FAIL

Fixture: Test Cookies Without Logging
Browser: chrome
Result: FAIL

Fixture: Test Cookies Without Logging
Browser: safari
Result: PASS

Observations of why I believe the tests are failing:

With Logging Enabled:

1. When the request logger is enabled, it intercepts the calls to bcp.crwdcntrl.net and rewrites the cookies in a special test-cafe format
2. The iframe call is also intercepted and served from the proxy.
3. It "appears" that testcafe realizes that the cookies were requested to be set on the same domain as the iframe
4. However, it does not seem to understand that safari has 3rd party cookies disabled and since the domain of the cookie does not match the page's domain, it should not actually set the Cookies
5. Result is that the browser acts like it can set third party cookies even in safari

Without Logging Enabled:

1. When the request logger is not enabled, the call to bcp.crwdcntrl.net is not being intercepted
2. Because the iframe call is still being intercepted and served from the testcafe proxy, the domain of the iframe now does not match crwdcntrl.net
3. Result is that Chrome does not set the 3rd party cookies due to the domain match even though it should

[alexey-lin]

Hi @noahpc,

Thank you for the project and your explanation. We'll research this behavior and update this thread once we have any results.

[noahpc]

Hi, just checking in to see whether there has been any progress on this or whether there is a workaround I can use in the meantime. Thanks!

[Farfurix]

@noahpc

Hello,

We are still researching this issue. We will update this thread if we have any new information.

[noahpc]

@Farfurix - Just checking in here - is there anything we can do to help or any expectation on when this ticket will change state? We are currently having to do a fair bit of our testing manually because of this issue.

[miherlosev]

Hi @noahpc

The information you provided is enough research the problem. Please wait until we find the cause of the problem.

[noahpc]

Just checking in again to see whether anyone was able to research this at all. If not, what would be the timeline for that?

If it is unlikely that you all would have the time to look into it, would it be possible for my team to suggest a fix through PR?

Up to this point, my team hasn't had the time to dig into the codebase , but if we were able to dedicate the time to it and find a potential solution, would you all be open to helping us merge that into the main branch?

[Dmitry-Ostashev]

Currently, we cannot provide any estimates on this issue. However, your PR would be greatly appreciated.

[markaconrad]

I am going start looking into submitting a fix PR for this. First step is to find out why the tests don't all pass on a fresh clone and initial npm install.

[markaconrad]

To close the loop here, I am no longer attempting a fix