Node-fetch: Sockets staying open if body not read?

[wheresrhys]

I'm debugging an issue in AWS lambda, and exploring a hunch that it may be caused by sockets opened by node-fetch staying open.

My lambda is a 'healthcheck' endpoint that pings a number of dependencies, returning a 200 if it gets a 200 from all of them. These dependencies are called with GET requests, not HEAD (in order to imitate prod traffic more closely), but only the status is read from the response - the body is not processed at all.

I am seeing a lot of Error: getaddrinfo EMFILE ${dependencyUrl} at Object._errnoException (util.js:1022:11)` errors. Debugging so far strongly suggests that this is because the lambda is hitting its socket limit at the C level.

The lambda docs state that the lambda invocation will not release the container for reuse until the event loop is completely cleared. My working theory is that because I do not try to read the body, the socket remains open until all the content arrives in the C-layer, but the node-fetch library does not add anything onto the event loop queue, so the container is released for reuse while the socket is still open. This means that eventually the number of open sockets can build up.

Does all that make sense? And if it does, can you suggest any mitigations e.g. is there some way I can manually close a socket (what I'm asking for is something like cancellable fetches, which is still not fully specced, but I wonder if there is a hacky way it could be achieved in node-fetch by interacting with the socket directly)

[wheresrhys]

Just found this response to another issue. Would it be workable? https://github.com/bitinn/node-fetch/issues/95#issuecomment-260251236. Or something similar. res.body._dump() doesn't seem to exist in the src any more (maybe it was a v1 feature?)

[TimothyGu]

Yeah, you would need to abort the request (#95) to make it work with your use case. _dump is a private Node.js API, it's possible that they no longer expose it. I'll close this out and consider this a duplicate of #95, if that's okay with you.

[wheresrhys]

Yep, fine to close it. I solved it by passing in a httpAgent, then accessing the socket's .destroy() method via that.