Koa: SSL load balancer + secure cookies means lots of requests 500 error

[kevinburke]

I'm running a Koa app behind a load balancer that terminates SSL. I set app.proxy to true, and have these lines to enable secure session cookies:

const session = require('koa-session');
app.proxy = true;
app.use(session({secure: true}))

When the load balancer terminates SSL and sends through X-Forwarded-Proto: https requests, everything works fine and a secure cookie gets set.

However, any request without a X-Forwarded-Proto: https header generates the following stack trace:

Error: Cannot send secure cookie over unencrypted connection
    at Cookies.set (/Users/kevin/src/github.com/bigco/website/node_modules/koa/node_modules/cookies/lib/cookies.js:86:11)
    at Session.save (/Users/kevin/src/github.com/bigco/website/node_modules/koa-session/index.js:296:15)
    at commit (/Users/kevin/src/github.com/bigco/website/node_modules/koa-session/index.js:163:10)
    at Object.session (/Users/kevin/src/github.com/bigco/website/node_modules/koa-session/index.js:120:7)
    at session.next (<anonymous>)
    at Object.bodyParser (/Users/kevin/src/github.com/bigco/website/node_modules/koa-bodyparser/index.js:68:12)
    at bodyParser.next (<anonymous>)
    at onFulfilled (/Users/kevin/src/github.com/bigco/website/node_modules/koa/node_modules/co/index.js:65:19)
    at process._tickDomainCallback (internal/process/next_tick.js:135:7)

The offending code is here, in the cookies library:

    , secure = this.secure !== undefined ? !!this.secure : req.protocol === 'https' || req.connection.encrypted
    , cookie = new Cookie(name, value, opts)
    , signed = opts && opts.signed !== undefined ? opts.signed : !!this.keys

  if (typeof headers == "string") headers = [headers]

  if (!secure && opts && opts.secure) {
    throw new Error('Cannot send secure cookie over unencrypted connection')
  }

A response is that I should be performing http redirects at the load balancer layer and my Node app shouldn't handle http requests. Of course.

But there are a variety of reasons I might send requests without that header:

• my Node app needs to do custom logic to decide how to redirect HTTP requests to HTTPS
• the load balancer sends health checks and I can't configure the headers it sends, also it is not forwarding them for a health check
• automated tooling makes requests directly to an app host (say, a stats or expvar server)

Basically it doesn't seem like setting a secure cookie in an insecure environment should be an error, and it's confusing that every request has to set X-Forwarded-Proto: https even when it's not forwarding the request from anywhere else.

Alternatively, koa could just decide to not set a session cookie if there's no X-Forwarded-Proto: https header, or catch this error from the cookies library and handle it appropriately. But it doesn't seem very easy to handle in my own codebase.

Let me know if this is better to be reported against the cookies library. I think this is the right place for it, though.

[kevinburke]

Previously, previously, previously, previously, previously, previously.

[ericmdantas]

Fourth link is broken, by the way. I think you meant https://github.com/pillarjs/cookies/issues/71

[kevinburke]

Fourth link is broken, by the way. I think you meant pillarjs/cookies#71

Oh, thanks. Fixed!

[jonathanong]

not sure what you expect. you are sending a secure cookie over a non-secure connection. i would just not set the cookie on a non-secure connection but not allowing sessions on those requests. in other words, put your health check middleware above the session middleware.

this is also why i don't like session middleware that always create/check the session. i use https://github.com/koajs/redis-session-sets for a more functional version of sessions – sessions are only created and updated when i specifically ask for it.

[kevinburke]

this is also why i don't like session middleware that always create/check the session.

I guess I am confused, then, because the default behavior of koa-session is to create/check the session.

in other words, put your health check middleware above the session middleware.

We can do this, but this still means if I SSH to the host and curl at it to check that the server's up, it's going to 500 server error, I need to explain to everyone on my team why that's happening, etc.

Other features that might be useless when set in some conditions usually don't cause the server to error. An HSTS header has no effect when sent on a plain HTTP response, but frameworks don't usually throw a server error if you try to set it.

[kevinburke]

I'd expect for the session to be sent as a secure cookie (a misconfiguration that browsers would ignore) or I'd expect for the session cookie to not be set over HTTP. I wouldn't expect the server to error.

I imagine there are a non-zero number of people who have dropped the secure flag from their cookies to "solve" this problem.

[dead-horse]

@kevinburke I think you should open an issue in cookies.

in koa level, as @jonathanong said, you should put your health check middleware and redirect middleware above the session middleware. also koa-session won't touch the cookie if you don't manually set session.

[dead-horse]

close for now, feel free to add some more discussions.

[kevinburke]

I'd like to reopen this, it's still valid and still a problem. I opened an issue in the cookies library and they said "you should set secure: true when you create the Cookies object", but I can't do that, because it's created based on the properties on the request:

  context.cookies = new Cookies(req, res, {
    keys: this.keys,
    secure: request.secure
  });

here's the code for secure:

  get secure() {
    return 'https' == this.protocol;
  },

and protocol:

  get protocol() {
    var proxy = this.app.proxy;
    if (this.socket.encrypted) return 'https';
    if (!proxy) return 'http';
    var proto = this.get('X-Forwarded-Proto') || 'http';
    return proto.split(/\s*,\s*/)[0];
  },

[kevinburke]

For the moment, working around this with this middleware:

  app.use(function*(next) {
    this.cookies.secure = true;
    yield* next
  });

[kevinburke]

See linked ticket ^^ another place I wanted to set secure cookies from a non-secure environment

[twelve17]

I'm running into the same issue as @kevinburke . In short, I have an app that is in the following infra:

browser (https/443) -> AWS ALB (load balancer) -> Web App (http/80) with Proxy Middleware -> ALS ALB 2 (http/80) -> Web App 2.

I'm not in control of the infrastructure, but I know that the incoming request will be secure, so I want to force koa to send secure cookies.

To be fair, it's possible I'm running into this because ALBs may not chain X-Forwarded-* headers correctly, but nevertheless, it would be helpful to have an option to override the cookie setting in koa.

I adapted @kevinburke 's middleware for koa 2:

   app.use((ctx, next) => {
      ctx.cookies.secure = true;
      return next();
    });

[danieldahan]

@kevinburke not sure if this is still relevant to you. But the app.proxy (koajs) value needs to be set to true, so that the "X-Forwarded-Proto" value will be trusted, which should be set to "proxy_set_header X-Forwarded-Proto https;", in Nginx for example.

[kevinburke]

Sorry - the second sentence of the issue description says I set app.proxy to true. Am I missing something?

[danieldahan]

@kevinburke That is strange. I literally set this up now and it works as expected. I set the cookie separately from the session though. Here is some code:

/**
 * @module createApp
 */

import dotenv from 'dotenv'
dotenv.config()

import Koa from 'koa'
import KeyGrip from 'keygrip'
import koaViews from 'koa-views'
import koaStatic from 'koa-static'

import keygrip from '^/keygrip.json'

import { cookies } from '$/aurora'

export function createApp(env: any): any {
  const app: any = new Koa()
  app.proxy = process.env.DE_PROXY
  app.keys = new KeyGrip(keygrip.keys, keygrip.hash)

  app.use(cookies)

  app.use(koaStatic(env.assets, true ? {} : {
    maxage: 1209600,
  }))

  app.use(koaStatic(env.public, true ? {} : {
    maxage: 1209600,
  }))

  app.use(koaViews(env.views, { extension: 'pug' }))

  return app
}

import crypto from 'crypto'

import domains from '@/lib/domains'

export const cookies = async (ctx: any, next: any) => {
  const _s = ctx.cookies.get('_s', { signed: true })

  if ('undefined' === typeof _s) {
    const expires = new Date()
    expires.setDate(expires.getDate() + 30)

    ctx.cookies.set('_s', crypto.createHash('md5').update(String(Date.now() + Math.random()**2)).digest('hex'), { 
      expires, 
      path: '/', 
      domain: domains.cookies, 
      signed: true,
      httpOnly: true,
      secure: true,
      sameSite: 'Strict',
    })
  }

  await next()
}

And then for the NGINX proxy configuration:

proxy_set_header X-Forwarded-Proto https;

Aside from that, it should work.

[kevinburke]

Right, if you continue reading the original issue description, the issue is that it doesn't work _unless_ the load balancer sends an X-Forwarded-Proto header. See the section that starts with:

But there are a variety of reasons I might send requests without that header:

for example.

[danieldahan]

@kevinburke I should have read it through, I only skimmed the comments. That said, the protocol of proxy to upstream wouldn't really be followed if it didn't follow the steps of setting the header. You could create configurations for each environment and change the forwarded value, or you can intercept the header in a middleware step, and then update the secure value like you did. Aside from that, I follow the protocol completely, so I can't really suggest anything outside of that scope.