Pm2: 2.0.0 introduced undocumented breaking changes to "pm2 startup"

[dmitry-mukhin]

Expected behaviour

startup script is created after running pm2 startup ubuntu

Actual behaviour

startup script is using wrong service control command

Steps to reproduce

$ sudo pm2 startup ubuntu
Platform ubuntu
Template
[Unit]
Description=PM2 process manager
Documentation=https://pm2.keymetrics.io/
After=network.target

[Service]
User=root
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=8
Environment=PATH=/usr/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
Environment=PM2_HOME=/home/ubuntu/.pm2
Restart=always
RestartSec=3

ExecStart=/usr/local/lib/node_modules/pm2/bin/pm2 resurrect --no-daemon
ExecReload=/usr/local/lib/node_modules/pm2/bin/pm2 reload all
ExecStopPre=/usr/local/lib/node_modules/pm2/bin/pm2 kill

[Install]
WantedBy=multi-user.target

Target path
/etc/systemd/system/pm2.service
Command list
[ 'chmod +x /etc/systemd/system/pm2.service',
  'systemctl enable pm2',
  'systemctl start pm2',
  'systemctl daemon-reload',
  'systemctl status pm2' ]
[PM2] Writing init configuration in /etc/systemd/system/pm2.service
[PM2] Making script booting at startup...
>>> Executing chmod +x /etc/systemd/system/pm2.service
[DONE] 
>>> Executing systemctl enable pm2
/bin/sh: 1: systemctl: not found
[DONE] 
>>> Executing systemctl start pm2
/bin/sh: 1: systemctl: not found
[DONE] 
>>> Executing systemctl daemon-reload
/bin/sh: 1: systemctl: not found
[DONE] 
>>> Executing systemctl status pm2
/bin/sh: 1: systemctl: not found
[DONE] 
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
$ pm2 save

[PM2] Remove init script via:
$ pm2 unstartup ubuntu

Software versions used

OS         : Ubuntu 14.04.5 LTS
node.js    : 6.9.2
PM2        : 2.2.2

[dmitry-mukhin]

I have to note that ignoring exit codes seems like a systemic problem for pm2 codebase.

[dmitry-mukhin]

found this: https://github.com/Unitech/pm2/blob/master/lib/API/Startup.js#L184
so there are two problems:

1. undocumented changes that
2. break existing deployment scripts

p.s.: pm2 startup ubuntu14 doesn't work too though :)

[vmarchaud]

Could you just try to run pm2 startup ?

Edit : how the change is undocumented : https://github.com/Unitech/pm2/blob/master/CHANGELOG.md#221-rc-220next

[dmitry-mukhin]

omitting platform gives the same result as specifying ubuntu14

$ sudo pm2 startup
[PM2] Init System found: upstart
Platform upstart
Template
#!/bin/bash
### BEGIN INIT INFO
# Provides:        pm2
# Required-Start:  $local_fs $remote_fs $network
# Required-Stop:   $local_fs $remote_fs $network
# Default-Start:   2 3 4 5
# Default-Stop:    0 1 6
# Short-Description: PM2 Init script
# Description: PM2 process manager
### END INIT INFO

NAME=pm2
PM2=/usr/local/lib/node_modules/pm2/bin/pm2
USER=root
DEFAULT=/etc/default/$NAME

export PATH=/usr/bin:$PATH
export PM2_HOME="/home/ubuntu/.pm2"

# The following variables can be overwritten in $DEFAULT

# maximum number of open files
MAX_OPEN_FILES=

# overwrite settings from default file
if [ -f "$DEFAULT" ]; then
      . "$DEFAULT"
fi

# set maximum open files if set
if [ -n "$MAX_OPEN_FILES" ]; then
    ulimit -n $MAX_OPEN_FILES
fi

get_user_shell() {
    local shell=$(getent passwd ${1:-`whoami`} | cut -d: -f7 | sed -e 's/[[:space:]]*$//')

    if [[ $shell == *"/sbin/nologin" ]] || [[ $shell == "/bin/false" ]] || [[ -z "$shell" ]];
    then
      shell="/bin/bash"
    fi

    echo "$shell"
}

super() {
    local shell=$(get_user_shell $USER)
    su - $USER -s $shell -c "PATH=$PATH; PM2_HOME=$PM2_HOME $*"
}

start() {
    echo "Starting $NAME"
    super $PM2 resurrect
}

stop() {
    super $PM2 kill
}

restart() {
    echo "Restarting $NAME"
    stop
    start
}

reload() {
    echo "Reloading $NAME"
    super $PM2 reload all
}

status() {
    echo "Status for $NAME:"
    super $PM2 list
    RETVAL=$?
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status
        ;;
    restart)
        restart
        ;;
    reload)
        reload
        ;;
    force-reload)
        reload
        ;;
    *)
        echo "Usage: {start|stop|status|restart|reload|force-reload}"
        exit 1
        ;;
esac
exit $RETVAL

Target path
/etc/init.d/pm2
Command list
[ 'chmod +x /etc/init.d/pm2',
  'touch /var/lock/subsys/pm2',
  'update-rc.d pm2 defaults' ]
[PM2] Writing init configuration in /etc/init.d/pm2
[PM2] Making script booting at startup...
>>> Executing chmod +x /etc/init.d/pm2
[DONE] 
>>> Executing touch /var/lock/subsys/pm2
touch: cannot touch ‘/var/lock/subsys/pm2’: No such file or directory
[DONE] 
>>> Executing update-rc.d pm2 defaults
 System start/stop links for /etc/init.d/pm2 already exist.
[DONE] 
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
$ pm2 save

[PM2] Remove init script via:
$ pm2 unstartup upstart

[vmarchaud]

Just read what PM2 give you : System start/stop links for /etc/init.d/pm2 already exist, pm2 is already registered as an upstart service, you should be able to sudo service pm2 status

[dmitry-mukhin]

Well, according to the code mentioned above, and the output of the command the touch /var/lock/subsys/pm2 is executed in any case.

[vmarchaud]

Yeah i know, that's the problem you reporting ?

[dmitry-mukhin]

While I don't think that the having code that does not work (touch /var/lock/subsys/pm2) and can falsely alert a user is a good thing, this is now what I'm trying to report here.

Currently this issue boils down to the fact that pm2 breaks backward compatibility without documenting this in the changelog or somewhere else where user can easily find without reading code.

In this state pm2 is not suitable for using production, IMO.

[vmarchaud]

We maintain the changelog in the root directory in our github repo, i can't see what we can do more to document the changement we make, and we know thats a breaking change too.
IMO saying that PM2 isn't suitable for using production just because we pushed a changement in a little module of PM2 can do is abusive.

PS : we will rethink the handling of error in the startup module

[dmitry-mukhin]

Changelog is there to log changes, including documenting changes that can break things.
Breaking things is totally OK when changing major versions, as long as user can either prepare for the upgrade or quickly find the root of problems (see example).

It's up to package maintainer to foresee and document such breaking changes. Failure to do this (and failure to admit problem later) breaks users' trust.

I can come a bit salty, and I'm sorry for that. I'm trying to be constructive though :)

[vmarchaud]

Totally get the constructive part, i was just highlighting that the 'salty' part isn't necessary here.
We can't keep these changes for a major version because our major iterations are really really slow, to waiting for 3.X to make a breaking changes isn't really usefull to our users.

Keep in minde that we are only two working on PM2 right now, Unitech and me. Im not complaining but we can't maintain all the documentations/changelogs/etc because we don't have time for this (we both work on keymetrics as our full time work).
We will try to resolve your issue and try our best to clearly explain breaking changes in the future

[vmarchaud]

We published modification to stop the execution flow under pm2 2.2.3.

[xoolooloo]

I having the same problem, is there any solution yet ? "pm2 startup ubuntu" does not work. It gives me the same error: "systemctl enable pm2 failed, see error above"

I was wondering if anybody can help me.

[vmarchaud]

@xoolooloo Just run pm2 startup

[swatiSJSU]

@vmarchaud even pm2 startup is not working for me ... it adds up my app.js to boot but when i check
"sudo pm2 list" it shows my app in errored state

[shapethink]

For me, pm2 startup yields:

[PM2] Init System found: systemd                  
[PM2] To setup the Startup Script, copy/paste the following command:
sudo env PATH=$PATH:/usr/bin /home/threadmetal/.npm-prefix/lib/node_modules/pm2/bin/pm2 startup systemd -u threadmetal --hp /home/threadmetal

which does not work -- but this does:
sudo env /home/threadmetal/.npm-prefix/bin/pm2 startup -u threadmetal --hp threadmetal