Http-proxy-middleware: hpm 1.0.6 Erroring for sockets when target server is restarted

[ubreddy]

Is this a bug report?

Yes

Is this a feature request?

No

Steps to reproduce

Set up a proxy server with
app.use(
'/target1',
createProxyMiddleware('/target1', {
target: 'http://127.0.0.1:3000',
ws: true,
})
);

Expected behavior

On restarting target server.. and refreshing the client (establishing proxy req again) , the proxy should close the connection and start a new connection to target.

Actual behavior

(Write what happened.)

On restarting the target service the proxy service is crashing when client tries to connect... with the following error.
Error occurred while trying to proxy request /target1 from proxyserver to http://target1:8080 (ECONNRESET) (https://nodejs.org/api/errors.html#errors_common_system_errors)

throw er; // Unhandled 'error' event

Error [ERR_STREAM_WRITE_AFTER_END]: write after end

at Socket.Writable.write (_stream_writable.js:296:5)
at writeAfterEnd (_stream_writable.js:248:12)
at Socket.end (net.js:548:31)
at ProxyServer.defaultErrorHandler (/home/node/app/node_modules/http-proxy-middleware/dist/handlers.js:59:9)
at Socket.Writable.end (_stream_writable.js:584:10)
at ClientRequest.onOutgoingError (/home/node/app/node_modules/http-proxy/lib/http-proxy/passes/ws-incoming.js:157:16)
at ClientRequest.emit (events.js:189:13)
at ProxyServer.emit (/home/node/app/node_modules/eventemitter3/index.js:204:33)
at Socket.socketOnEnd (_http_client.js:433:9)
at ClientRequest.EventEmitter.emit (domain.js:441:20)
at Socket.emit (events.js:194:15)
at Socket.EventEmitter.emit (domain.js:441:20)
at endReadableNT (_stream_readable.js:1129:12)
at /home/node/app/node_modules/async-listener/glue.js:188:31
at process.internalTickCallback (internal/process/next_tick.js:72:19)
Emitted 'error' event at:
at errorOrDestroy (internal/streams/destroy.js:98:12)
[... lines matching original stack trace ...]
at writeAfterEnd (_stream_writable.js:250:3)
at Socket.Writable.write (_stream_writable.js:296:5)
at process.internalTickCallback (internal/process/next_tick.js:72:19)

Setup

• http-proxy-middleware: 1.0.6
• http-proxy-middleware configuration: see above
• server: normal express server with socket.io
• other relevant modules

client info

Chrome browser.

target server info

Simple express server with socket.io

Reproducible Demo

[steinybot]

This is quite frustrating. I am trying to use this with webpack-dev-server and when my server hot reloads changes the proxy crashes.

[rafaelsales]

Also experiencing this on 0.19.0

These look related

• https://github.com/chimurai/http-proxy-middleware/issues/44
• https://github.com/webpack/webpack-dev-server/issues/1642

[rafaelsales]

@ubreddy

It would be nice to have this handled by the http-proxy-middleware as it doesn't make sense to crash the server in this scenario. But here is a workaround:

app.use(
  '/target1',
  createProxyMiddleware('/target1', {
    target: 'http://127.0.0.1:3000',
    ws: true,
    onProxyReqWs: (proxyReq, req, socket) => {
      socket.on('error', function (error) {
        console.warn('Websockets error.', error);
      });
    }
  })
);

When you restart the target server, this will log the following instead of crashing the server:

Websockets error. Error: write EPIPE
    at afterWriteDispatched (internal/stream_base_commons.js:154:25)
    at writeGeneric (internal/stream_base_commons.js:145:3)
    at Socket._writeGeneric (net.js:783:11)
    at Socket._write (net.js:795:8)
    at writeOrBuffer (_stream_writable.js:353:12)
    at Socket.Writable.write (_stream_writable.js:303:12)
    at IncomingMessage.ondata (_stream_readable.js:713:22)
    at IncomingMessage.emit (events.js:315:20)
    at IncomingMessage.EventEmitter.emit (domain.js:485:12)
    at addChunk (_stream_readable.js:302:12)
    at readableAddChunk (_stream_readable.js:278:9)
    at IncomingMessage.Readable.push (_stream_readable.js:217:10)
    at HTTPParser.parserOnBody (_http_common.js:130:24)
    at Socket.socketOnData (_http_client.js:503:22)
    at Socket.emit (events.js:315:20)
    at Socket.EventEmitter.emit (domain.js:485:12)
    at addChunk (_stream_readable.js:302:12)
    at readableAddChunk (_stream_readable.js:278:9)
    at Socket.Readable.push (_stream_readable.js:217:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:186:23) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
}

or

Websockets error. Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:205:27) {
  errno: -54,
  code: 'ECONNRESET',
  syscall: 'read'
}

[saltire]

I've also seen ECONNRESET cause crashes with 1.0.6. Most of the time, errors seem to be caught properly, and log [HPM] Error occurred while trying to proxy request..., but on rare occasions they aren't and the server goes down. I've never had it happen in my dev environment, only a few times in production. 😰

It was pretty difficult to track this down. There's a similar workaround in https://github.com/chimurai/http-proxy-middleware/issues/327, but that didn't solve it for me. I've implemented the one above and caught a few errors with it, and haven't had another crash so far. 🤞

Seems like these errors are emitted directly from the socket and not from the proxy server instance. Does it make sense to add a listener like this by default?

[rafaelsales]

All proxy implementations that I've seen ignore connection errors by default, so my expectation with this lib is that it would have a default socket handler to catch and just log those to avoid bubbling up to the app server.

It was pretty hard to find the cause of this error in our stack as well.