Loopback: How to run scheduled tasks (delay or periodic) in a cluster of loopback app?

Created on 6 Jan 2015 · 5Comments · Source: strongloop/loopback

I want to run periodic tasks and delay tasks in loopback app. But slc run in production env starts a cluster of loopback app, and the master process doesn't run the app, only the worker process does.

So I can't simply use setTimeout() or modules like _node-cron_, _agenda_ in app, since they will be trigger in every worker process at same time. And I don't know how to make the workers to talk each other and vote one to run the task alone.

I think a message queue (like rabbitMQ) or task scheduler (like celery) could solve the problem, but I really like to keep app as simple as possible.

Does loopback have a solution for this or any suggestion?

Source

clarkorz

Most helpful comment

Perhaps your loopback (server) app isn't the right place for this. You can use you model's outside the context of a server (eg. a simple node script) like this:

#!/usr/bin/env node
var models = require('path/to/my/app/server/server.js').models;
var async = require('async');

//
// keep in mind that boot scripts will be executed!
//

// do your job...
async.each(Object.keys(models), function(modelName) {
  models[modelName].destroyAll(cb);
}, function(err) {
  console.log(err || "job's finished!");
});

// save this script as `my-cron-job`
// chmod +x my-cron-job
// and run with cron

ritch on 8 Jan 2015

👍6 😕2 🎉1

All 5 comments

Perhaps your loopback (server) app isn't the right place for this. You can use you model's outside the context of a server (eg. a simple node script) like this:

#!/usr/bin/env node
var models = require('path/to/my/app/server/server.js').models;
var async = require('async');

//
// keep in mind that boot scripts will be executed!
//

// do your job...
async.each(Object.keys(models), function(modelName) {
  models[modelName].destroyAll(cb);
}, function(err) {
  console.log(err || "job's finished!");
});

// save this script as `my-cron-job`
// chmod +x my-cron-job
// and run with cron

ritch on 8 Jan 2015

👍6 😕2 🎉1

Having something like https://github.com/LearnBoost/kue running a similar script is probably the way to go.

seriousben on 12 Jan 2015

Closing this for now... Please re-open if you have other related questions.

ritch on 16 Jan 2015

So, I have this very strange problem only when working with https://github.com/LearnBoost/kue which only occurs when I'm running the server with slc run --cluster=1 (or without the cluster)
So, I set up kue and as soon as I run something like kue.create(type, data).save() the worker crashes:

2016-04-15T13:36:04.965Z pid:39799 worker:0 ERROR supervisor worker id 1 (pid 39800) accidental exit with 1

No further information. No nothing.
However, if I run the app with node . everything is fine and works as expected.

Any ideas?

Thanks,
Chris

// edit opened a new ticket #2234

cspeer on 15 Apr 2016

👍1

@ritch I'm following your advice. And to make a simple test I set the Cron to run every 5min, and from the explorer I keep getting an endpoint (with just one record), and when the cron run (at min 5), the response takes a little more time than normal. Is it related to the boot script loading again? (Note, I also add the Async library). Is there a way to access the model, without loading the boot script again?