Superagent: Cookies apparently not preserved across agent requests

[JeffML]

I have a typical login scenario which returns a session id. The second login test fails because another 'set-cookie' is in the header.

I can get this to work in supertest, which uses an older version of superagent, by calling res.jar.setCookie(res.headers['set-cookie'][0])in the first request.

var login = require('./login2.js');
var chai = require('chai');
var should = chai.should();

var server = process.env.SERVER_UNDER_TEST;

process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0;

var prefix = require('superagent-prefix')(server);
var request = require('superagent').agent();

describe("Cookie Test", () => {
    it("login", done => {
        login.login(request, prefix, (res) => {
            should.exist(res.headers['set-cookie']);
            done();
        });
    }));

    it("login", done => {
        login.login(request, prefix, (res) => {
            should.not.exist(res.headers['set-cookie']);
            done();
        });
    });
});

[kornelski]

We have tests for it and they pass:

https://github.com/visionmedia/superagent/blob/36ce8782842c2fee402013ff0650d7f8b310e3a7/test/node/agency.js#L82,L109

Do you do something differently from these tests? (e.g. check contents of the header to see if domains/paths/secure flags make sense)

[JeffML]

Here is a full test:

cookieTest.js

var login = require('./login2.js');
var chai = require('chai');
var should = chai.should();

var server = process.env.SERVER_UNDER_TEST;

process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0;
var prefix = require('superagent-prefix')(server);
var request = require('superagent').agent();

describe("Cookie Test", () => {
    it("test1", done => {
        login.testSite(request, prefix, (res) => {
            should.exist(res.headers['set-cookie']);
            done();
        });
    });

    it("test2", done => {
        login.testSite(request, prefix, (res) => {
            should.not.exist(res.headers['set-cookie']);
            done();
        });
    });
});

login2.js

var chai = require('chai');
var should = chai.should();

process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0;

exports.testSite = (agent, prefix, done) => {
    agent.get('/')
        .use(prefix)
        .type('json')
        .end(function(err, res) {
            if (err) {
                throw err;
            }

            res.ok.should.be.ok;

            done(res);
        });
};

dependencies:

  "dependencies": {
    "chai": "^3.5.0",
    "lodash": "^4.17.4",
    "supertest": "^2.0.1"
  },
  "devDependencies": {
    "eslint": "^3.14.1",
    "mochawesome": "^2.0.2"
  }

mocha installed globally

Run: SERVER_UNDER_TEST=https://www.yahoo.com mocha cookieTest

[kornelski]

I'm unable to reproduce it with the latest version of superagent

Cookie Test
✓ test1 (1775ms)
✓ test2 (1465ms)

2 passing (3s)

[JeffML]

Hmmm. Rechecking my dependencies, I have:

jeff.l@Jeffs-MacBook-Pro:~/Documents/workspace/unit-tests/mocha-chai_tests (master *=)$ npm ls -depth 0
[email protected] /Users/jeff.l/Documents/workspace/unit-tests/mocha-chai_tests
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] extraneous
├── [email protected] extraneous
├── [email protected] extraneous

└── [email protected]

$ SERVER_UNDER_TEST=https://www.yahoo.com mocha cookieTest

Cookie Test
✓ test1 (701ms)
1) test2

1 passing (1s)
1 failing

[JeffML]

I can provide you with the network log. That might help.

[kornelski]

Yes, request and response HTTP headers would clear things up

[JeffML]

First

request:

GET / HTTP/1.1
Host: www.yahoo.com
Accept-Encoding: gzip, deflate
User-Agent: node-superagent/3.4.0
Content-Type: application/json
Connection: close

response:

HTTP/1.1 200 OK
Expires: -1
Cache-Control: no-store, no-cache, private, max-age=0
Server: ATS
Via: http/1.1 ir5.fp.gq1.yahoo.com (ApacheTrafficServer)
Connection: close
Transfer-Encoding: chunked
Age: 0
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Vary: Accept-Encoding
Set-Cookie: autorf=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=www.yahoo.com
X-Frame-Options: DENY
Strict-Transport-Security: max-age=2592000
P3P: policyref="https://policies.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Date: Fri, 27 Jan 2017 21:30:42 GMT

second:

request:

GET / HTTP/1.1
Host: www.yahoo.com
Accept-Encoding: gzip, deflate
User-Agent: node-superagent/3.4.0
Content-Type: application/json
Connection: close

response:

HTTP/1.1 200 OK
Expires: -1
Cache-Control: no-store, no-cache, private, max-age=0
Server: ATS
Via: http/1.1 ir12.fp.gq1.yahoo.com (ApacheTrafficServer)
Connection: close
Transfer-Encoding: chunked
Age: 0
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Vary: Accept-Encoding
Set-Cookie: autorf=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=www.yahoo.com
X-Frame-Options: DENY
Strict-Transport-Security: max-age=2592000
P3P: policyref="https://policies.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Date: Fri, 27 Jan 2017 21:36:35 GMT

[kornelski]

This looks correct. Yahoo does not set any cookies, it removes one: expires=Thu, 01-Jan-1970

[JeffML]

Alright, trying again; this time SERVER_UNDER_TEST=https://www.mavericklabel.com/reskin/xml/headers.php mocha cookieTest

I'm wondering if the expires data (Nov. 19, 1981) is being interpreted as applying to the cookie as well as the rest of the response? I believe that should only apply to the response body (although the past date suggests a config issue). There's no expires date on the cookie, so it should be maintained for the session....browsers are treating the cookie this way.

first request:

GET /reskin/xml/headers.php/ HTTP/1.1
Host: www.mavericklabel.com
Accept-Encoding: gzip, deflate
User-Agent: node-superagent/3.4.0
Content-Type: application/json
Connection: close

first response:

HTTP/1.1 200 OK
Connection: close
Content-Length: 145
Content-Encoding: gzip
Vary: Accept-Encoding,User-Agent
set-cookie: PHPSESSID=5a6857299af944ebc047e6d38efc4b25ee5f66f6; path=/; domain=.mavericklabel.com
content-type: application/json
pragma: no-cache
cache-control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 19 Nov 1981 08:52:00 GMT
Access-Control-Allow-Methods: GET,POST,PUT,DELETE,OPTIONS
access-control-allow-origin: *
X-Powered-By: Express
Server: Apache/2.2.15
Date: Fri, 27 Jan 2017 22:31:54 GMT

second request:

Accept-Encoding:gzip, deflate
Content-Type:application/json
Host:www.mavericklabel.com
User-Agent:node-superagent/3.4.0

second response:

HTTP/1.1 200 OK
Connection: close
Content-Length: 144
Content-Encoding: gzip
Vary: Accept-Encoding,User-Agent
set-cookie: PHPSESSID=08e8314d1e5eef0e67d002beb9230b136536f6eb; path=/; domain=.mavericklabel.com
content-type: application/json
pragma: no-cache
cache-control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 19 Nov 1981 08:52:00 GMT
Access-Control-Allow-Methods: GET,POST,PUT,DELETE,OPTIONS
access-control-allow-origin: *
X-Powered-By: Express
Server: Apache/2.2.15
Date: Fri, 27 Jan 2017 22:34:46 GMT

[kornelski]

I've checked. This is because superagent doesn't know what the domain name is. You're requesting .get('/'), so the domain is undefined, but the cookie requires match on a specific domain.

agent.get('https://www.mavericklabel.com/reskin/xml/headers.php') works.

I guess superagent-prefix is a leaky abstraction. You could report the problem there.

[JeffML]

I'm using superagent-prefix, which I assumed prefixed the server name to the get path. However, if I don't use superagent-prefix and specify the full path (https://www.mavericklabel.com/reskin/xml/headers.php), then the test passes. Alright, good enough. Thanks.