Pm2: PM2-docker fails to intercept docker stop signal

[Sam-Jeston]

Expected behaviour

When stopping a container running pm2, with the docker stop command, I expect PM2 to intercept the signal and pass this to the node process.

Actual behaviour

The signal is not intercepted. The docker stop command hangs for 10 seconds (which is the default time), before sending a kill signal.

Steps to reproduce

index.js

require('net').createServer().listen()
console.log('Server listening')

process.on('SIGTERM', function () {
  console.log('SIGTERM received')
  clean().then(() => process.exit())
})

process.on('SIGINT', function () {
  console.log('SIGINT received')
  clean().then(() => process.exit())
})

function clean () {
  return new Promise((res) => {
    console.log('Starting some cleanup action...')
    setTimeout(res, 1000)
  })
}

process.on('exit', function () {
  console.log('Processed cleaned. Exiting.')
})

Dockerfile

FROM node:6.9
COPY . .
RUN npm i -g pm2
CMD ["pm2-docker", "index.js"]

1) docker build -t sigterm-image .
2) docker run --name sigterm sigterm-image
3) docker stop sigterm
4) Observe that no intercepts occur and the container hangs for 10s until docker sends the kill signal

Interestingly if the running container is interrupted from the terminal with Ctrl-C, PM2-Docker intercepts the signal as expected, like so:

[STREAMING] Now streaming realtime logs for [all] processes
2016-11-09-00:18:28 0|index    | Server listening
Exiting PM2
2016-11-09-00:18:35 PM2        | Stopping app:index id:0
2016-11-09-00:18:35 0|index    | SIGINT received
2016-11-09-00:18:35 0|index    | Starting some cleanup action...
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:35 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:36 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:36 PM2        | pid=23 msg=failed to kill - retrying in 100ms
2016-11-09-00:18:36 0|index    | Processed cleaned. Exiting.
2016-11-09-00:18:36 PM2        | App [index] with id [0] and pid [23], exited with code [0] via signal [SIGINT]
2016-11-09-00:18:36 PM2        | [HandleExit] PM2 is being killed, stopping restart procedure...
2016-11-09-00:18:36 PM2        | pid=23 msg=process killed
2016-11-09-00:18:36 PM2        | ===============================================================================
2016-11-09-00:18:36 PM2        | --- Stopping PM2 --------------------------------------------------------------
2016-11-09-00:18:36 PM2        | Time                 : Wed Nov 09 2016 00:18:36 GMT+0000 (UTC)
2016-11-09-00:18:36 PM2        | ===============================================================================

I have also uploaded the image publicly on Docker Hub as abxit/pm2-docker-interrupt-test for ease of testing.

Software versions used

OS         : Ubuntu 14.04 & 16.04
node.js    : 6.9
PM2        : 2.1.4

[Unitech]

Thanks for this great bug report.

After testing your example, I found that pm2-docker was only listening to the SIGINT signal (explaining why the graceful stop was working with CTRL-C). Whereas docker stop sends a SIGTERM.

The event has been added to the pm2-docker on the dev branch.

To try it:

FROM node:6.9
COPY . .
RUN npm install Unitech/pm2#development -g
CMD ["pm2-docker", "index.js"]

[Sam-Jeston]

That's excellent. Thanks for the quick response.

I can also confirm the development branch is intercepting the SIGTERM as expected. The dev install will suit my use case (which is currently pre-prod) until this commit makes it into a release.

[Unitech]

Thanks for testing. Any feedback about pm2-docker is warmly welcome.
The release will be made tomorrow
Cheers!

[Unitech]

published on [email protected]