Kue: Why did you implement Kue?

Created on 23 Sep 2016  路  9Comments  路  Source: Automattic/kue

I know it's an odd question to ask but why did you implement Kue instead of using one of the established MQ servers?

Thanks

Discussion
Source
picture adriano-di-giovanni
👍1

Most helpful comment

We use Kue for all our backend processing, including payments and credit card processing. Previously we've used celery. Main reasons are:

  • It just works.
  • It's based nodejs so I can keep my code clean, re-use code, re-use unit tests
  • Backed by redis, proven fault proof insanely fast server
  • AWS friendly (via redis server)
  • It's easy to run Multiple workers in parallel using PM2
  • All the messaging is JSON based, after jobs are done I dump them to MongoDB
  • Survived blackfriday
picture rturk on 27 Sep 2016
👍7

All 9 comments

Kue is a task/job opinioned queue, not a generic message queue.
Distribution of workers, Job retries, scheduled jobs and simplicity was main purpose of kue when I joined this project.
Can you elaborate on some MQ names that can be simply replaced by Kue?

behrad picture behrad on 23 Sep 2016
👍1

I'm new to MQs so, I can't elaborate any name at the moment.
Can you answer your own question, please?

adriano-di-giovanni picture adriano-di-giovanni on 23 Sep 2016

Interested in this question too.

Also compare and contrast agenda vs kue (other than the obvious) would be appreciated.

aysark picture aysark on 24 Sep 2016

http://queues.io also may help guys

behrad picture behrad on 24 Sep 2016
👍1

And as @antirez says implementing a job queue on top of redis is almost deprecated, and Disque is born for such cases.
I'm thinking to re-write the next Node.js job queue on top of Disque...

behrad picture behrad on 24 Sep 2016

Distribution of workers, Job retries, scheduled jobs and simplicity was main purpose of kue when I joined this project.

@behrad Thanks for the info. This simple explanation did a lot for me, makes sense now why so many libs have been abstracting away certain queue complexities. I've been using jackrabbit so far for my worker queues but will have to test with Kue as well.

Also thanks for sharing http://queues.io I hadn't seen this before.

mitchellporter picture mitchellporter on 24 Sep 2016

We use Kue for all our backend processing, including payments and credit card processing. Previously we've used celery. Main reasons are:

  • It just works.
  • It's based nodejs so I can keep my code clean, re-use code, re-use unit tests
  • Backed by redis, proven fault proof insanely fast server
  • AWS friendly (via redis server)
  • It's easy to run Multiple workers in parallel using PM2
  • All the messaging is JSON based, after jobs are done I dump them to MongoDB
  • Survived blackfriday
rturk picture rturk on 27 Sep 2016
👍7

All the messaging is JSON based, after jobs are done I dump them to MongoDB

@rturk Would you mind elaborating on this.. what are you dumping into mongo and why?

mitchellporter picture mitchellporter on 21 Oct 2016

@mitchellporter I dump old jobs to MongoDB in order to keep redisDB storage footprint small.

rturk picture rturk on 15 Jan 2017
Was this page helpful?
0 / 5 - 0 ratings

Related issues

bahatron picture bahatron  路  5Comments
syadykin picture syadykin  路  8Comments
kithokit picture kithokit  路  5Comments
danpalmer picture danpalmer  路  8Comments
alexbudin picture alexbudin  路  5Comments