Bull: All jobs are sent to "Waiting" queue

[aarondtaveras]

I'm new to Bull, and was doing some testing just fine with my jobs on a Redis queue before all of my jobs began immediately being put into waiting. There are no jobs currently active or failed, but everything is just sitting in the waiting queue. I am using a named queue, and am adding jobs as follows:

router.post('/job',async(req,res) => {
  let job = await workQueue.add('runCOJSearch', {bizname: req.body.bizname});
  res.json({id: job.id});
})

I am processing the job within a start function in a worker.js file. I am using Bull with throng for clustered processes. Here is my start function:

async function start() {
  // Connect to the named work queue
  let workQueue = new Queue('work', REDIS_URL);

  workQueue.process('runCOJSearch', async(job,data)=> {
    job.progress(100);
    return {message: "Success!"};
  })
  workQueue.on('global:completed', job => {
    console.log(`Job with id ${job.id} has been completed```);
  })

 }

Bull version: 3.7.0
Running on Windows 10. I am overseeing the status of my jobs by using Bull Arena v.2.6.2.

[Igor-Lopes]

According to the docs :

When a job is added to a queue it can be in one of two states, it can either be in the “wait” status, which is, in fact, a waiting list, where all jobs must enter before they can be processed, or it can be in a “delayed” status: a delayed status ...

It seems your consumer has the default concurrency limit of 1. So you should expect to have one job being processed at a time.

[aarondtaveras]

@Igor-Lopes

Currently, it seems like none of my jobs are being processed. The jobs don't leave from the waiting queue, and there is also no job in the active queue. They do not move from the waiting to active queue, and even before when they were in the active queue, they never moved to a completed status. I'm also using throng for clustering/concurrency, and I'm setting the concurrency to 2. Not sure if that changes anything...

Am I misunderstanding something? I feel like my example is pretty minimal and should at the very least create and connect to a queue that processes a very simple job -- returning a success message.

[Igor-Lopes]

@aarondtaveras , according to the docs

process(processor: ((job, done?) => Promise<any>) | string)
process(concurrency: number, processor: ((job, done?) => Promise<any>) | string)
process(name: string, processor: ((job, done?) => Promise<any>) | string)
process(name: string, concurrency: number, processor: ((job, done?) => Promise<any>) | string)

Your code:

workQueue.process('runCOJSearch', async(job,data)=> {
    job.progress(100);
    return {message: "Success!"};
  })

There is no 'data' param for an async processor, so it should be:

workQueue.process('runCOJSearch', async(job)=> {
    job.progress(100);
    return {message: "Success!"};
  })

[manast]

@Igor-Lopes answer should resolve this issue. Re-open if necessary.

[aarondtaveras]

@Igor-Lopes
@manast

I amended the code - the issue was actually a syntax error in the global on completed handler that was throwing an uncaught error which caused my worker file to crash and thus never process anything.

How then would I pass necessary data for my job without the data parameter? I was going to pass the data parameter to a function call within my process function. What would be the preferred way to do this?

[manast]

@aarondtaveras you just get it from job.data