Agenda: Running multiple instances with one agenda db

[weeco]

Hello,
can I safely run multiple instances and assume that defined jobs won't get processed n times where n is the quantity of started instances?

I am using pm2 and I would like to start 6 instances, but obviously I don't want my jobs to execute 6 times at the same time as well. Would that work?

[lushc]

Yep it'll work since Agenda's locking mechanism relies on MongoDB's findOneAndUpdate behaviour for exclusively processing a job. However, you'll want to set a lock limit because otherwise, you'll probably find one instance will lock all available jobs for processing and starve the others.

[weeco]

@lushc Follow up question,
the default lock limit is set to 0 (unlimited). Why would I need to adapt this? Assuming I want one job to be processed only by one instance and another job which shall be processed by all instances every 30 minutes. How would the options object look like?

In the documentation you often say it's "per instance", but what exactly is considered an agenda instance when I am running a cluster with 8 instances and one mongodb for example?

[weeco]

Hm can no one help me?

[dmitrfokin]

Same question. We have multiple machines connected with load balancer. Each machine runs the application instanse, so I wonder if we can create a single queue for all the machines and app instances?