Pm2: 3.2.x cluster processes unable to bind to tcp listener port

Created on 5 Oct 2018 · 7Comments · Source: Unitech/pm2

What's going wrong?

When starting pm2 in cluster mode on a multi-core machine, only one process is able to bind to a TCP listening port (express listen), the other processes report an Error: listen EADDRINUSE
Started using:

pm2 start app.js --name myname --no-daemon -i max

This appears to start 4 processes in fork mode with pm2 3.2.x, but it starts for processes in cluster mode in pm2 3.1.3

How could we reproduce this issue?

mkdir pm2test
cd pm2test
npm init
npm install express

npm install [email protected]

app.js:

const express = require('express');
let app = express();
app.listen(8080);

./node_modules/.bin/pm2 start app.js --name myname --no-daemon -i max
./node_modules/.bin/pm2 kill

npm install [email protected]

./node_modules/.bin/pm2 start app.js --name myname --no-daemon -i max
./node_modules/.bin/pm2 kill

Supporting information

===============================================================================
--- PM2 REPORT (Fri Oct 05 2018 10:55:38 GMT+0100 (BST)) ----------------------
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.2.1
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,start,app.js,--name,myname,--no-daemon,-i,max
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
uptime               : 0min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.2.1
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,report
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
cpus nb              : 4
freemem              : 1005617152
totalmem             : 16704774144
home                 : /home/pwinkler
===============================================================================
--- PM2 list -----------------------------------------------
┌──────────┬────┬─────────┬──────┬───────┬────────┬─────────┬────────┬───────┬───────────┬──────────┬──────────┐
│ App name │ id │ version │ mode │ pid   │ status │ restart │ uptime │ cpu   │ mem       │ user     │ watching │
├──────────┼────┼─────────┼──────┼───────┼────────┼─────────┼────────┼───────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ 1.0.0   │ fork │ 28038 │ online │ 0       │ 0s     │ 28.8% │ 40.3 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ 1.0.0   │ fork │ 28091 │ online │ 2       │ 0s     │ 0%    │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ 1.0.0   │ fork │ 28097 │ online │ 2       │ 0s     │ 0%    │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ 1.0.0   │ fork │ 28103 │ online │ 2       │ 0s     │ 0%    │ 31.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴──────┴───────┴────────┴─────────┴────────┴───────┴───────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
┌──────────┬────┬─────────┬──────┬───────┬────────┬─────────┬────────┬─────┬───────────┬──────────┬──────────┐
│ App name │ id │ version │ mode │ pid   │ status │ restart │ uptime │ cpu │ mem       │ user     │ watching │
├──────────┼────┼─────────┼──────┼───────┼────────┼─────────┼────────┼─────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ 1.0.0   │ fork │ 28038 │ online │ 0       │ 0s     │ 0%  │ 40.3 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ 1.0.0   │ fork │ 28091 │ online │ 2       │ 0s     │ 0%  │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ 1.0.0   │ fork │ 28097 │ online │ 2       │ 0s     │ 0%  │ 33.1 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ 1.0.0   │ fork │ 28103 │ online │ 2       │ 0s     │ 0%  │ 31.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴──────┴───────┴────────┴─────────┴────────┴─────┴───────────┴──────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app

===============================================================================
--- PM2 REPORT (Fri Oct 05 2018 10:58:21 GMT+0100 (BST)) ----------------------
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 3.1.3
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,start,app.js,--name,myname,--no-daemon,-i,max
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
uptime               : 0min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 3.1.3
node version         : 8.9.4
node path            : ./node_modules/.bin/pm2
argv                 : /home/pwinkler/.nvm/versions/node/v8.9.4/bin/node,/home/pwinkler/projects/test/pm2test/node_modules/.bin/pm2,report
argv0                : node
user                 : pwinkler
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
cpus nb              : 4
freemem              : 869748736
totalmem             : 16704774144
home                 : /home/pwinkler
===============================================================================
--- PM2 list -----------------------------------------------
┌──────────┬────┬─────────┬───────┬────────┬─────────┬────────┬──────┬───────────┬──────────┬──────────┐
│ App name │ id │ mode    │ pid   │ status │ restart │ uptime │ cpu  │ mem       │ user     │ watching │
├──────────┼────┼─────────┼───────┼────────┼─────────┼────────┼──────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ cluster │ 28797 │ online │ 0       │ 2s     │ 0.6% │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ cluster │ 28803 │ online │ 0       │ 2s     │ 0.6% │ 41.7 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ cluster │ 28813 │ online │ 0       │ 2s     │ 5.5% │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ cluster │ 28823 │ online │ 0       │ 1s     │ 8.3% │ 41.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴───────┴────────┴─────────┴────────┴──────┴───────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
┌──────────┬────┬─────────┬───────┬────────┬─────────┬────────┬─────┬───────────┬──────────┬──────────┐
│ App name │ id │ mode    │ pid   │ status │ restart │ uptime │ cpu │ mem       │ user     │ watching │
├──────────┼────┼─────────┼───────┼────────┼─────────┼────────┼─────┼───────────┼──────────┼──────────┤
│ myname   │ 0  │ cluster │ 28797 │ online │ 0       │ 2s     │ 0%  │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 1  │ cluster │ 28803 │ online │ 0       │ 2s     │ 0%  │ 41.7 MB   │ pwinkler │ disabled │
│ myname   │ 2  │ cluster │ 28813 │ online │ 0       │ 2s     │ 0%  │ 41.8 MB   │ pwinkler │ disabled │
│ myname   │ 3  │ cluster │ 28823 │ online │ 0       │ 1s     │ 0%  │ 41.5 MB   │ pwinkler │ disabled │
└──────────┴────┴─────────┴───────┴────────┴─────────┴────────┴─────┴───────────┴──────────┴──────────┘
 Use `pm2 show <id|name>` to get more details about an app

Pending Release

Source

HughePaul

👍4

Most helpful comment

I faced the similar issue when updated from v3.1.3 to v3.2.0: the app starts fine but it doesn't release a port on reload.
The reason is, somewhere app.exec_mode is compared with string 'cluster_mode', not just 'cluster'.
Earlier there was a function checkExecMode() which sets exec_mode to 'cluster_mode' if it was just 'cluster'. But it was removed.

So I defined exec_mod in my config file as 'cluster_mode' and everything works fine. You can try the same workaround.

vinogradov-m on 5 Oct 2018

👍3 ❤1

All 7 comments

same issue :(

ufukd on 5 Oct 2018

So I defined exec_mod in my config file as 'cluster_mode' and everything works fine. You can try the same workaround.

vinogradov-m on 5 Oct 2018

👍3 ❤1

wip

Unitech on 5 Oct 2018

I've missed that point.... I've disabled 'max' || 0 instances test because travis sometime has a ton of CPU.
Just fixed and added test: https://github.com/Unitech/pm2/commit/d5acb855328dfcd56647cdf1e416498df0a89b39

Once travis goes green I will hot publish this fix, expect it in one hour

Unitech on 5 Oct 2018

deployed:

$ npm install pm2@latest -g
$ pm2 update

Unitech on 5 Oct 2018

❤2

@Unitech Hello, this issue seems to be about Error: listen EADDRINUSE when using exec_mode: 'cluster'. @vinogradov-m pointed out a commit that referenced why this may be happening. There are also duplicate issues which reference this issue (#3996 and #3987).

But you seemed to have closed the issue after fixing an unrelated bug (setting instance to 'max' || 0).

Am I misunderstanding?

Note I am on 3.2.2 and have experienced the Error: listen EADDRINUSE bug -- and setting exec_mode: 'cluster_mode' fixed it. But I fear there is no issue tracking this problem, as they are all closed?

Thank you