Bull: Multiple repeating jobs with the same cron pattern fails

[SyedWasiHaider]

Hello there,

I wanted to test if two jobs scheduled for the same time could be handled in one queue given there are at least two worker threads but for some reason only the first job shows up.

let testQueue = new Queue('testQueue', 'redis://127.0.0.1:6379');
testQueue.empty();
testQueue.process(3, function (job, done) {
    console.log('Hello this is my job:' + job.data.MyJobId);
    done();
});
testQueue.add({"MyJobId":123}, { repeat: { cron: '*/2 * * * * *' } })
testQueue.add({"MyJobId":456 }, { repeat: { cron: '*/2 * * * * *' } })

Here is the output after about 4-5 seconds:

Hello this is my job:123
Hello this is my job:123
Hello this is my job:123
Hello this is my job:123
Hello this is my job:123

I'd expect the other job to show up at some point but it never does.

On other hand if I do this:

let testQueue = new Queue('testQueue', 'redis://127.0.0.1:6379');
testQueue.empty();
testQueue.process(3, function (job, done) {
    console.log('Hello this is my job:' + job.data.MyJobId);
    done();
});
testQueue.add({"MyJobId":123}, { repeat: { cron: '*/2 * * * * *' } })
testQueue.add({"MyJobId":456 }, { repeat: { cron: '*/3 * * * * *' } })

I get something like this:

Hello this is my job:123
Hello this is my job:456
Hello this is my job:123
Hello this is my job:456
Hello this is my job:123

which seems to make sense. However, I still think the first behavior should be similar to this. I'm not sure if this is a limitation of redis ttl somehow or just something the library isn't setup to handle.

[SyedWasiHaider]

Seems it may have been addressed here: https://github.com/OptimalBits/bull/issues/601. However, the issue still stands. I'm using "bull": "^3.3.4"

[SyedWasiHaider]

Looks like this is just a silly case of me not RTFM: https://github.com/OptimalBits/bull/blob/master/REFERENCE.md#queueadd

For anyone that comes across this (and has bad googling skills like I do), the solution to the above is provide a unique job id in the opts object so for the above it would be:

let testQueue = new Queue('testQueue', 'redis://127.0.0.1:6379');
testQueue.empty();
testQueue.process(2, function (job, done) {
    console.log('Hello this is my job:' + job.data.MyJobId);
    done();
});
testQueue.add({"MyJobId":123}, { jobId: '123', repeat: { cron: '*/2 * * * * *' } })
testQueue.add({"MyJobId":456 }, { jobId:'456', repeat: { cron: '*/2 * * * * *' } })

Works like a charm.

[manast]

or just specify a different name when calling testQueue.add

[SyedWasiHaider]

One thing that doesn't seem to work well right now is queue.getJob('123') where (123 is an id for a repeating job). I may investigate this further and it may be related to: https://github.com/OptimalBits/bull/issues/766

[manast]

that will not work because in a repeatable job you get unique job ids per repetition, although they will be based on the given jobId.

[xoryouyou]

@manast is the method of adding a unique id to a repeated Job the preferred method?

It would be great if an add call with a repeat object would return a unique Id for the repeatable job.

Since I face the same problem as @SyedWasiHaider that I try to remove a repeatable job with the same cron schedule but different payload.

and Queue.getRepeatableJobs() only returns objects like this

 { key: '__default__::::*/1 * * * * *',
    name: '__default__',
    id: null,
    endDate: null,
    tz: null,
    cron: '*/1 * * * * *',
    next: 1536238965000 }

So I can't cancel a repeating job even when passing

{ cron: '*/1 * * * * *',  jobId: 123 }

to Queue.removeRepeatable