Agenda: Jobs not running on time when scheduled sooner than the processEvery() cadence

[edwin-jones]

Hi there,

I'm trying to upgrade to Agenda 2.0.0 and I noticed a bug that doesn't appear to happen in 1.0.3.

If you have an agenda instance set to process every minute or so, but schedule a job to run in 5 seconds it doesn't actually run until the next time agenda checks the DB _(i.e. the next minute or so)_. This means you either have to up how frequently you make requests to the DB or lose the ability to run jobs at a higher frequency than the timer you have set.

In 1.0.3 if a job was scheduled to run before the next job retrieval it would persist in memory and run, then update the job in the db before the next _findAndLockNextJob call.

[bonesoul]

same here

[sd1998]

@simison I would like to work on this

[simison]

@sd1998 PR's welcome!

[sd1998]

@simison I am new to this project could you help me to get started in the right direction? Thanks.

[simison]

README.md and especially debugging section should be useful: https://github.com/agenda/agenda#debugging-issues

At the top of that page, there's also Slack-link, where you can ask questions. :-)

[jaschaio]

Any update on this? I am encountering the same issue.

A bit of background: I am listening to events and while only one event is emitted due to how nodes custer mode works various workers can react to the same event. So I use agenda .unique() to make sure that only a single event gets processed:

    async onChange( change ) {

        // Create a new job
        var job = agenda.create( 'processDatabaseEvent', { change } );

        // Get idempotencyKey from the change event
        var { _id: { _data: idempotencyKey } } = change;

        // Make sure the job only runs once for this event
        job.unique( { idempotencyKey }, { insertOnly: true } );

        // Schedule to run now
        job.schedule( new Date() );

        // Save Job
        await job.save();

    }

But if the processEvery option is set to lets say the default 5 seconds in the worst case there is 5 seconds between scheduling the job and actually running it.

agenda.now() seems to execute a job immediately bypassing processEvery but unfortunately I can't use unique() in that case which means that I would run multiple jobs for a single event in node cluster mode.