Pm2: No error logs for a process with errored status on reboot.

Created on 21 Apr 2017  ยท  9Comments  ยท  Source: Unitech/pm2

No error logs for a process with errored status on reboot or ressurect.

This is making it very difficult to figure out why there is an error.

https://github.com/Unitech/pm2/issues/2540 is the same issue but is closed after the issue was abandoned.

These seem similar

2275 closed and combined into #2629 which is about refactoring watch and seems unrelated.

546 ...15 restarts, no logs

this stack overflow issue
http://stackoverflow.com/questions/38884063/pm2-not-logging-output-on-node-app-that-crashes-immediately

There is no error when running pm2 interactively.
A simple node server has no problem starting on boot

app001@ubuntu-xenial:~/.pm2/logs$ pm2 resurrect
[PM2] Resurrecting
[PM2] Restoring processes located in /home/app001/.pm2/dump.pm2
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Name โ”‚ mode โ”‚ status  โ”‚ โ†บ โ”‚ cpu โ”‚ memory    โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ main โ”‚ fork โ”‚ online  โ”‚ 0 โ”‚ 0%  โ”‚ 26.0 MB   โ”‚
โ”‚ www  โ”‚ fork โ”‚ errored โ”‚ 0 โ”‚ 0%  โ”‚ 0 B       โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
 Use `pm2 show <id|name>` to get more details about an app
app001@ubuntu-xenial:~/.pm2/logs$ ls -l
total 0
app001@ubuntu-xenial:~/.pm2/logs$ ps -ef | grep pm2
app001    1351     1  0 14:35 ?        00:00:00 PM2 v2.4.5: God Daemon (/home/app001/.pm2)

How could we reproduce this issue?

Supporting information

PM2 version: 2.4.5
Node version: v6.10.1
Linux ubuntu-xenial 4.4.0-72-generic  vagrant on Win 7 host


Need More Data

Most helpful comment

I had the same problem everything was resolved after running

pm2 kill
pm2 start <app_name>

All 9 comments

Can you check ~/.pm2/pm2.log and paste it here to see if there are errors in it ?

Thank you that's the log I was looking for! Strictly interpreted this closes my issue, and thank you for the prompt reply.

Maybe you can still help though..
It appears I'm suffering from this error on bootup.
http://stackoverflow.com/questions/27688804/how-do-i-debug-error-spawn-enoent-on-node-js

Do I need to set a boot environment variable?

Here is the log

`
app001@ubuntu-xenial:~$ cat ~/.pm2/pm2.log
2017-04-21 16:45:32: ===============================================================================
2017-04-21 16:45:32: --- New PM2 Daemon started ----------------------------------------------------
2017-04-21 16:45:32: Time : Fri Apr 21 2017 16:45:32 GMT+0000 (UTC)
2017-04-21 16:45:32: PM2 version : 2.4.5
2017-04-21 16:45:32: Node.js version : 6.10.1
2017-04-21 16:45:32: Current arch : x64
2017-04-21 16:45:32: PM2 home : /home/app001/.pm2
2017-04-21 16:45:32: PM2 PID file : /home/app001/.pm2/pm2.pid
2017-04-21 16:45:32: RPC socket file : /home/app001/.pm2/rpc.sock
2017-04-21 16:45:32: BUS socket file : /home/app001/.pm2/pub.sock
2017-04-21 16:45:32: Application log path : /home/app001/.pm2/logs
2017-04-21 16:45:32: Process dump file : /home/app001/.pm2/dump.pm2
2017-04-21 16:45:32: Concurrent actions : 2
2017-04-21 16:45:32: SIGTERM timeout : 1600
2017-04-21 16:45:32: ===============================================================================
2017-04-21 16:45:32: Starting execution sequence in -fork mode- for app name:main id:0
2017-04-21 16:45:32: [Watch] Start watching 1
2017-04-21 16:45:32: Starting execution sequence in -fork mode- for app name:www id:1
2017-04-21 16:45:32: App name:main id:0 online
2017-04-21 16:45:32: App name:www id:1 online
2017-04-21 16:45:32: Error: spawn node ENOENT
at exports._errnoException (util.js:1018:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
at onErrorNT (internal/child_process.js:367:16)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickDomainCallback (internal/process/next_tick.js:128:9)
```

No need for environment variable, that means the script that isn't at the same place after the reboot, don't forget to run pm2 save to save the process state before shutting down.

Thanks but pm2 save is not a solution, the location of the script does not change.

As you see, one application that was saved can't be respawned because of a EONENT so there must be something that disappear between when the dump has been made and when you resurrect PM2.

I think you are right, and now I know you are. The Express app is in a shared vagrant folder. There is nothing in it at the time PM2 tries to start the process. Vagrant copies it across later in the boot process.

Thank you for your help and patience.

Name โ”‚ mode โ”‚ status โ”‚ โ†บ โ”‚ cpu โ”‚ memory โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€
โ”ผโ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ demeter โ”‚ fork โ”‚ errored โ”‚ 0 โ”‚ 0% โ”‚ 0 B โ”‚
โ”‚ express โ”‚ fork โ”‚ online โ”‚ 588 โ”‚ 0% โ”‚ 16.1 MB โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€
โ”ดโ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
Use pm2 show <id|name> to get more details about an app

Supporting information
PM2 version: 2.7.2
Node version: v6.10.1
System : Win 7

PM.log:
2017-11-21 13:39:06: Error: spawn babel-node ENOENT
at exports._errnoException (util.js:1018:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:193:32)
at onErrorNT (internal/child_process.js:367:16)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickDomainCallback (internal/process/next_tick.js:128:9)

@vmarchaud @Johnpc123
cannot run

I had the same problem everything was resolved after running

pm2 kill
pm2 start <app_name>
Was this page helpful?
0 / 5 - 0 ratings