Hi, I was using celery (with node-celery), and I'd like to switch to bull.
I have the following pattern in my app:
Service1 - Ask for data
|
| (HTTP long polling)
| --> Service2 - Start a job
|
| (celery/bull)
| --> workers
so I'm looking to replicate this pattern I have in node-celery :
router.get('/xxx', async function(req, res, next) {
const jobResult = await startTask(req.body);
res.json(jobResult);
});
As far as I understood reading the documentation I could do it this way with bull :
router.get('/xxx', async function(req, res, next) {
const jobId = await startTask(req.body);
queue.on('completed', function(result) {
if (result.jobId === jobId) {
res.json(result));
}
})
});
Is there a way to specifically wait for a job's results with bull or do I have to listen to all completed event and test the jobId ?
No, there is no simple way to do what you are trying to accomplish. Normally, the job producer is decoupled from the job consumer, but in this case you want the producer and consumer to be the same, which goes a bit against its design. But I wonder, lets say that there existed such a way to do what you want in bull, how would you handle the case where the long polling request gets unexpectedly aborted, or the connection to the queue gets aborted during the processing?
Thank you for your answer :)
The job take few seconds, most of the time, and is used to preview a 3D rendering but need to be done on a separated computer (with lots of ram).
In the case where the job doesn't complete within 30seconds, the user sees a "default" page with a "No preview" image.
The solution might be to rework our full flow, and use websocket to notify job completion, but the long polling solution made it easy for us to have a "simple" design: start job, display snipper, wait 30sec, if no answer serve default image, else serve preview.
I will close this issue now, re-open if you feel there is something else we can do.
Sorry for necroposting, but is Job#finished() in current version does exactly what requested?
Most helpful comment
Sorry for necroposting, but is
Job#finished()in current version does exactly what requested?