Pm2: Watch no longer working

Created on 2 Mar 2015  ·  37Comments  ·  Source: Unitech/pm2

PM2 0.12.4

Node v0.12.0

[{
"name": "[scrubbed]",
"script": "./scripts/start.js",
"watch": true,
"exec_mode": "fork_mode",
"instances": 1,
"node_args": "--debug=5858",
"cwd": "/[scrubbed]/current"
}]

2015-03-02 09:58:11: [PM2][WORKER] Started with refreshing interval: 30000
2015-03-02 09:58:11: [[[[ PM2/God daemon launched ]]]]
2015-03-02 09:58:11: BUS system [READY] on port /[scrubbed]/.pm2/pub.sock
2015-03-02 09:58:11: RPC interface [READY] on port /[scrubbed]/.pm2/rpc.sock
2015-03-02 09:58:11: Starting execution sequence in -fork mode- for app name:node-inspector id:0
2015-03-02 09:58:11: App name:node-inspector id:0 online
2015-03-02 09:59:23: Starting execution sequence in -fork mode- for app name:[scrubbed] id:1
2015-03-02 09:59:23: App name:[scrubbed] id:1 online
2015-03-02 09:59:26: Starting execution sequence in -fork mode- for app name[scrubbed] id:2
2015-03-02 09:59:26: App name:[scrubbed] id:2 online
2015-03-02 11:07:07: Process with pid 7048 still not killed, retrying...
2015-03-02 11:07:07: App name:[scrubbed] id:1 exited
2015-03-02 11:07:07: Process with pid 7048 killed
2015-03-02 11:07:07: Starting execution sequence in -fork mode- for app name:[scrubbed] id:1
2015-03-02 11:07:07: App name:[scrubbed] id:1 online

Bug
Source
picture gshively11

All 37 comments

Verified it doesn't work on pm2 0.12.7 also

gshively11 picture gshively11 on 2 Mar 2015

Are you on osx?

Just tried and indeed there is an issue with fsevents, maybe related to node 0.12.

Temporary fix:

  "watch_options" : {
      "useFsEvents": false
  }

Or don't install the chokidar fsevents module in the pm2 directory :

rm -r node_modules/chokidar && npm i chokidar --no-optional

I'll try to dig deeper and submit an issue to chokidar or fsevents about this.

soyuka picture soyuka on 2 Mar 2015
👍1

Yes, on OSX. The temporary fix you mentioned does not seem to work.

gshively11 picture gshively11 on 2 Mar 2015

The same problem on ubuntu+vagrant, pm2 doesn't see file changes

pm2 0.12.7
node 0.12.0

kryz81 picture kryz81 on 3 Mar 2015

We tested this yesterday with ubuntu/debian and there was no issues. Did you test the workarounds from below?

soyuka picture soyuka on 3 Mar 2015

I should mention that I'm using vagrant as well. And I'm watching a synched folder, so it's a symlink.

gshively11 picture gshively11 on 4 Mar 2015

So I mispoke when I said I was on OSX. My host is OSX, but PM2 is running on a CentOS 6.5 VM. My bad.

gshively11 picture gshively11 on 4 Mar 2015

@gshively11 @ChrisPiechowicz Could you please run the watcher test and give me the output in a gist?

Tests are located under:

bash test/bash/watch.sh

in the pm2 module directory.

Thanks!

soyuka picture soyuka on 4 Mar 2015

Ran it as the user which runs our node apps:

$pm2 kill
[PM2] Stopping PM2...
[PM2] deleteProcessId process id 0
[PM2] deleteProcessId process id 1
[PM2] deleteProcessId process id 2
[PM2] All processes has been stopped and deleted
[PM2] PM2 stopped

# Determine wget / curl
which wget > /dev/null
if [ $? -eq 0 ]
then
    http_get="wget"
else
    echo -e "\033[31mYou need wget to run this test \033[0m";
    exit 1;
fi

function fail {
  echo -e "######## \033[31m  ✘ $1\033[0m"
  exit 1
}

function success {
  echo -e "\033[32m------------> ✔ $1\033[0m"
}

function spec {
  RET=$?
  sleep 0.3
  [ $RET -eq 0 ] || fail "$1"
  success "$1"
}

function ispec {
  RET=$?
  sleep 0.3
  [ $RET -ne 0 ] || fail "$1"
  success "$1"
}

function should {
    sleep 0.5
    OUT=`$pm2 prettylist | grep -o "$2" | wc -l`
    [ $OUT -eq $3 ] || fail "$1"
    success "$1"
}

cd $file_path

function waituntil {
  for (( i = 0; i <= $1; i++ )); do
      sleep 0.2
      echo -n "."
  done
  echo ""
}

echo -e "\033[1mRunning tests:\033[0m"
Running tests:

#####################
# Watch for changes #
#####################

>server-watch.js
test/bash/watch.sh: line 22: server-watch.js: Permission denied

$pm2 kill
[PM2] Spawning PM2 daemon
[PM2] Success
[PM2] Stopping PM2...
[PM2] [WARN] No process found
[PM2] All processes has been stopped and deleted
[PM2] PM2 stopped

cp server-watch.bak.js server-watch.js
cp: cannot create regular file `server-watch.js': Permission denied

$pm2 start server-watch.js --watch
[PM2] Spawning PM2 daemon
[PM2] Success
[PM2] Process server-watch.js launched
┌──────────────┬────┬──────┬──────┬────────┬───────────┬────────┬─────────────┬──────────┐
│ App name     │ id │ mode │ PID  │ status │ restarted │ uptime │      memory │ watching │
├──────────────┼────┼──────┼──────┼────────┼───────────┼────────┼─────────────┼──────────┤
│ server-watch │ 0  │ fork │ 7691 │ online │         0 │ 0s     │ 14.469 MB   │  enabled │
└──────────────┴────┴──────┴──────┴────────┴───────────┴────────┴─────────────┴──────────┘
 Use `pm2 info <id|name>` to get more details about an app

should 'process should be watched' 'watch: true' 1
$pm2 prettylist | grep -o "$2" | wc -l
------------> ✔ process should be watched

echo "console.log('test');" >> server-watch.js
test/bash/watch.sh: line 32: server-watch.js: Permission denied

sleep 1

cat server-watch.js
var http = require('http');

http.createServer(function(req, res) {
  res.writeHead(200);
  res.end('hey');
}).listen(8000);
$pm2 list
┌──────────────┬────┬──────┬──────┬────────┬───────────┬────────┬─────────────┬──────────┐
│ App name     │ id │ mode │ PID  │ status │ restarted │ uptime │      memory │ watching │
├──────────────┼────┼──────┼──────┼────────┼───────────┼────────┼─────────────┼──────────┤
│ server-watch │ 0  │ fork │ 7691 │ online │         0 │ 2s     │ 14.469 MB   │  enabled │
└──────────────┴────┴──────┴──────┴────────┴───────────┴────────┴─────────────┴──────────┘
 Use `pm2 info <id|name>` to get more details about an app

should 'process should have been restarted' 'restart_time: 1' 1
$pm2 prettylist | grep -o "$2" | wc -l
########   ✘ process should have been restarted
gshively11 picture gshively11 on 4 Mar 2015

@soyuka I can't find watch.sh in pm2 dir, I don't have "test" directory..

kryz81 picture kryz81 on 5 Mar 2015

Same here on Ubuntu 14.04.1 LTS (inside a Vagrant Box as well as on DigitalOcean droplet) running PM2 v0.12.5 on node v0.10.25.
In this case the watched folder is a symlink too.
Output of the test script, see https://gist.github.com/akloeber/d18147f903353a11178b.

Setting useFsEvents to false does not help.

akloeber picture akloeber on 7 Mar 2015

:+1:

sheymann picture sheymann on 18 Mar 2015

Can also confirm that watch isn't working for me.

Ubuntu 14.04.2
Node v0.12.0

karan picture karan on 29 Mar 2015

Any idea when this issue might be fixed?

gshively11 picture gshively11 on 9 Apr 2015

@gshively11 I can't reproduce anymore, and I haven't find the source of this issue.

It seems to be very random and I've got absolutely no clue of what's going on here... I had the same error on my computer but I cleaned up my nvm node versions, updated iojs and it worked as before :| with node 0.12/0.10...

When I had the issue I took pm2 back to the first version that had the watch feature but still had this issue! The only thing that had change between then and now is that I had installed iojs.

soyuka picture soyuka on 9 Apr 2015

@soyuka It still does not work in [email protected] but I found out the following behavior while investigating:

The folder structure below my PM2's cwd is like that:

<PM2_CWD>
  bin
    app.js
    ...
  node_modules
    ...
  lib
    ...
  package.json

If I touch bin/app.js which is PM2's script entry point (or even another script inside the bin/ folder) PM2's watch mechanism works and the node is restarted. But if I touch a script in any other folder like lib/main.js there is no reaction in PM2.

Furthermore I was not able to bash test/bash/watch.sh as this script is no more present.

Environment:
pm2 0.12.11
node v0.10.25
Ubuntu 14.04.1 LTS

akloeber picture akloeber on 19 Apr 2015

Seems like it's working to me, please update your pm2/node versions if you still have issues.

soyuka picture soyuka on 19 Jun 2015

I'm having the same problems.

My versions are:

pm2 v0.14.3
node v0.10.39
Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-55-generic x86_64)

touch triggers the change event, but I need to be in the vagrant ssh session. Touching from my computer's terminal session does not work, nor does updating the file from a text editor. The test/bash/watch.sh isn't present for me to test.

FWIW, I'm running on the latest OSX, with vagrant v1.7.2.

joemckie picture joemckie on 7 Jul 2015

Is it possible that there are some symlinks and that followSymlinks option is not set properly?

soyuka picture soyuka on 7 Jul 2015

Possibly, I'm not 100% sure how vagrant sets up its synchronised folders. I've just tried setting the watch_options -> followSymlinks option to both true and false, but neither makes a difference.

joemckie picture joemckie on 7 Jul 2015

I just downloaded the development version of PM2 and ran the watch.sh test. It gave the same result as @gshively11 above.

joemckie picture joemckie on 7 Jul 2015

I'm having the exact same issue, cannot get the watch functionality to work reliably with Vagrant (1.7.2) and NFS shared folders on Ubuntu 14.04 (guest) and OSX 10.10.4 (host).

PM2: v0.14.3
NodeJS: v0.12.4

Bazze picture Bazze on 17 Jul 2015

@Bazze maybe you should open an issue on the chokidar repository about this.

soyuka picture soyuka on 17 Jul 2015

@soyuka: Thank you! Good idea, but I searched some amongst their issues instead and found this ticket (#242), seems to be pretty much related. I solve my issue for now by using the "usePolling": true watch option. Not optimal, but it works.

Bazze picture Bazze on 17 Jul 2015
👍1

@Bazze Oh interesting, I searched but never found this one.
Why are you thinking that usePolling is not optimal? If NFS doesn't manage file events it's the only solution.

1313 related

soyuka picture soyuka on 17 Jul 2015

@soyuka: It's not optimal as in it's checking for changes at a fixed time interval, instead of "real-time". But for now I can live with waiting a few seconds for the changes to be discovered. :)

Bazze picture Bazze on 17 Jul 2015

100 ms (default polling option) is very low, consider increasing to save CPU usage.

@Bazze hope it'll do the job, no better solution :/

soyuka picture soyuka on 17 Jul 2015

It's 100ms by default? Takes 1-2 seconds at least for me before it notices any changes, weird. I'm gonna experiment a bit with the interval!

@soyuka: So far I can't see why it wouldn't (except that it could be a bit faster) ;)

Bazze picture Bazze on 17 Jul 2015

@soyuka: Regarding your commit 77bb8d0, shouldn't the option be usePolling and not polling? At least usePolling is what I use and what seems to be in the chokidar documentation.

Bazze picture Bazze on 17 Jul 2015

@Bazze thanks, friday evening.....

soyuka picture soyuka on 17 Jul 2015

@soyuka I know the feeling, let's call it a day! :beers:

Bazze picture Bazze on 17 Jul 2015

Similar, but somewhat different error: https://gist.github.com/DenisGorbachev/523f3a937faec86b31dd

starfall@nx:~/workspace/PM2$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04 LTS"

starfall@nx:~/workspace/PM2$ git show
commit 733d631301394630637b3cdbfa449ab1ac158f10
Author: Unitech <[email protected]>
Date:   Wed Jul 22 17:09:19 2015 +0200

    0.14.5
DenisGorbachev picture DenisGorbachev on 2 Aug 2015

Watch & reload doesn't work on Debian 9.1x64 on DigitalOcean droplet. I have a simple helloWorld project and trying to change a project file in a text editor or by pulling changes from git repo and nothing happens :(

MihaelIsaev picture MihaelIsaev on 27 Jul 2017

--watch is not working on Debian 9

yass0016 picture yass0016 on 30 Jul 2017

I'm on Linux, and 

"watch_options" : {
      "useFsEvents": false
}

Worked for me ; Thanks!

yPhil-gh picture yPhil-gh on 23 Mar 2018

pm2 v2.9.3
macosx 10.11.6

"watch_options" : { "useFsEvents": false }

didn't work

catalsdevelop picture catalsdevelop on 11 Apr 2018

In my case, watch option not working when you start the pm2 which is once status: stopped.

I mean, if it is the first process, watch option works.
However, if you run pm2 stop ecosystem.config.js, and then pm2 start ecosystem.config.js, watch option won't work.

Mizumaki picture Mizumaki on 5 Apr 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

lefam picture lefam  ·  3Comments
psparago picture psparago  ·  3Comments
cchan picture cchan  ·  3Comments
mario-mui picture mario-mui  ·  3Comments
ldarren picture ldarren  ·  3Comments