Serverless-express: 431 Request Header Fields Too Large

[larrybolt]

I tend to get a 431 Request Header Fields Too Large error with express when running serverless offline start (serverless-offline plugin). Deleting event.multiValueHeaders fixes it, but it might break other things, I need to do more research:

exports.handler = (event, context) => {
    delete event.multiValueHeaders;
    return awsServerlessExpress.proxy(server, event, context)
}

The project where I'm having this issue is also using serverless-typescript though.
This issue was very annoying to find so therefor sorry for lack of other information, I'll try to update this issue if I find out more.

[nadunindunil]

Hi @larrybolt, I am facing the same issue with a typescript-express project could you please share any information if you have done or found anything?

[nadunindunil]

I changed library to serverless-http and it worked

[georgeevans1995]

Has anyone found a solution to this without using serverless-http?

It only seems to be happening when you are using later versions of node. If I run version 8.10 everything works, but as AWS is deprecating this version I would like to run things locally on the correct versions.

[larrybolt]

@georgeevans1995 the solution initially suggested, delete event.multiValueHeaders; before passing the event to awsServerlessExpress.proxy seems to work for us.

[georgeevans1995]

@larrybolt Sorry I should have said in my previous comment, I tried this but I still got the same error.

[georgeevans1995]

Just to add some more detail to this from some testing, the error only seems to occur when I have an authorizer attached to the handler. I'm not sure how or why this would make a difference though?

[jenyayel]

"Newer" versions of Nodejs limited the size of request headers to 8K.

What aws-serverless-express proxy does, is takes event and context data, serialize it to JSON and stores it as encoded strings in x-apigateway-event and x-apigateway-context request headers. This makes the entire payload of headers before proxying much larger than original. I found that it was even storing request body once, which some nice person fixed here. But it was when a limit in nodejs was 80K.

When running with serverless-offline, the plugin mocks a lot of event and context data (to look real), which cause it to hit the limit of 8K.

I guess there is no easy fix for this, besides downgrading nodejs to the version that has limit of 80K (e.g. v10.13.0).

[jchadwick]

Why was this closed (without a comment)? This is still a major problem and from what I can see the only workarounds are "use an old Node version" or "use another library", neither of which is a practical solution.

[larrybolt]

@jchadwick Sorry about that, didn't know it was still relevant. Based on the comments it seemed like it was an issue with NodeJS and does work once deployed, thus more an issue with serverless-offline.

Hmm now thinking about it.. it might be possible to fix this with a serverless plugin for those using serverless-offline. But that would be more of a hack.

[AshUK]

This is very limiting, passing a large id_token for example, across the query path also invalidates the request. It is not just serverless-offline that is padding the payload.

[kennyhyun]

http.maxHeaderSize can be overridden by max-http-header-size (nodejs >12.6, >10.15)
https://nodejs.org/api/cli.html#cli_max_http_header_size_size (16KB on nodejs 14 by default)

in package.json

  "start": "node --max-http-header-size=16384 server.js",

[galesky]

http.maxHeaderSize can be overridden by max-http-header-size (nodejs >12.6, >10.15)
https://nodejs.org/api/cli.html#cli_max_http_header_size_size (16KB on nodejs 14 by default)

in package.json

  "start": "node --max-http-header-size=16384 server.js",

Building on top of that I'm using this as a workaround:
"start": "npx --node-arg=--max-http-header-size=64000 serverless offline"

[brettstack]

Fixed in v4