Pm2: PM2 cluster communication is confusing

[askhogan]

There are an assortment of examples to communicate from pm2 programmatic to child workers, but they are not documented well and contradictory.

1) there are references to amx, which is deprecated
2) there are references to pm2-interface, which is deprecated
3) there are references to doing a pmx.emit, but if you do that, then you are supposed to hook up an "action" in the child worker, which has nothing passed to it except a reply? How do you pass the thing data then, other than the string command name?
4) there are references to a launchBus, but this just receives messages, in the master, instead of being able to send them, which is great if you are tooling a complex PM2 script, but doesn't help with two way communication.

For most stuff PM2 makes clustering easier. For communication specifically, node native cluster as in process.send('data') and process.on('message') is far easier than the assortment of poorly documented functions in PM2 for cluster communication.

I am going to fall back to the deprecated PM2-Interface

[jshkurti]

I agree.
What kind of communication would you like to be implemented ?
Inter-workers communication ? How would that work ? Broadcast ? When a worker emits something all the other workers receive it ? Targeted worker ? Then how to identify each worker ?

For the PM2 communicating with workers part, I thought about something like
pm2 ipc <id/name> <msg> which would send message to the specified id or name of the cluster and can be read via process.on('message').

[askhogan]

That would be cool. I think the msgProcess that is integrated here is perfect.
https://github.com/Unitech/pm2-interface

ipm2.rpc.msgProcess(opts, fn)

I used this last night to solve an issue. It was funny, because I like the new Keymetrics.io panel so much!! I was like really working hard to not go back to native cluster methods to achieve.

[askhogan]

The main thing is just to be able to send an object across, because a single command string like I see in pmx action is not enough. There needs to be meta data around it. For example organizationId or userId.

[askhogan]

https://github.com/Unitech/PM2/blob/master/doc/PROGRAMMATIC.md

Like I have the message bus - just need the send. pm2.msgProcess

[crispkiwi]

@jshkurti, I think what you suggested would be amazing.

I'm looking for an easy way to set winston logging levels at runtime without having to expose an http endpoint to do it.

If I could use pm2 ipc myCluster/myAppPid "setLog:debug" or something similar it would be an awesome solution.

[jshkurti]

In addition to pm2 ipc I'm working on this https://github.com/jshkurti/ipc-shared .
It will be integrated into PM2 at some point with a flag such as pm2 start app.js -i 4 --shared in which case all the instances of a cluster would share an object as well as sending/receiving events between them.
For now, it is just a pretty README, there is no code yet.
I would love some feedback :)

[astanciu]

+1 on this issue. The really needs to be a way to send information between the processes. I have looked through all the issues that touch on this subject and they seem to point to https://github.com/Unitech/PM2/blob/master/doc/PROGRAMMATIC.md in some way or another, but that is a terrible piece of documentation (How does a child process emit to the bus?)

[chrizz1001]

I have commented on pm2 documentation page, but in case it gets deleted again, here it is:

var pm2 = require('pm2');

// get existing workers:
var pm2_worker_ids = [];

pm2.connect(function (err) {
  pm2.list(function (err, data) {

    for (var i in data) {
      if (data.hasOwnProperty(i)) {

        pm2_worker_ids.push(data[i].pm2_env.pm_id);

      }
    }

    pm2.disconnect(function () {});
  });
});


// Then to send a message to other workers:

pm2.connect(function () {
  for (var i in pm2_worker_ids) {

    if (pm2_worker_ids.hasOwnProperty(i)) {

      pm2.sendDataToProcessId(pm2_worker_ids[i], { // target pm_id here...
        type: 'some random text',
        data: data, // your actual data object you want to pass to the worker
        id: pm2_worker_ids[i], // ... and target pm_id here
        topic: 'some random text'
      }, function (err, res) {});
    }
  }
  pm2.disconnect(function () {});
});

to listen to the event you do the usual:

process.on('message', function (data) {
   console.log('your actual data object', data.data);
});

[thelonglqd]

Exactly ! The documentation of sendDataToProcessId function lacks of the pm2_id as the first argument.

We do not need something like launchBus to listen to the message also.

[thelonglqd]

In my case, the message object argument of sendDataToProcessId() function does not need to declare the target pm2_worker_id inside.

{
        type: 'some random text',
        data: data, // your actual data object you want to pass to the worker
        id: pm2_worker_ids[i], // whatever number you want.
        topic: 'some random text'
}

Cause of inside the node_modules/pm2/lib/CLI.js

CLI.sendDataToProcessId = function(proc_id, packet, cb) {
  packet.id = proc_id;

  Satan.executeRemote('sendDataToProcessId', packet, function(err, res) {
    if (err) {
      Common.printError(err);
      return cb ? cb({msg:err}) : Common.exitCli(cst.ERROR_EXIT);
    }
    Common.printOut('successfully sent data to process');
    return cb ? cb(null, res) : speedList();
  });
}

Whatever number you declared inside the packet object, It will be re-assigned to proc_id (the id you passed as the first argument of sendDatatoProcessId() function)

[Oceanswave]

+1

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.