Sidekiq: Can I run multiple instances of Sidekiq with the same queues inside?

[zulhfreelancer]

I'm currently exploring Rancher and I'm planning to use it for production. I have 2 stacks right now; blue and green. Each stack will contains web and worker process.

My Rails app is using Sidekiq to process all the background jobs and it's doing well so far for us. FYI, now we are on one host with 2 processes; web and worker. Managing single host is straight forward.

As for my moving-to-Rancher plan, I plan to have multi-host kind of setup where each host will run one or more web and worker containers.

So, my question here is, can I run multiple Sidekiq instances with the same queues inside each of it?

Will it lead to any side effects ie race conditions, duplicate job execution between Sidekiq processes & etc?

Do I need to have special configuration for Redis to make this happen? For now, I'm using RedisCloud Standard plan and I'm just using the default configuration.

Really appreciate your feedback and thoughts. Thank you in advance!

Example of my config/sidekiq.yml file:

:concurrency: 5
:pidfile: tmp/pids/sidekiq.pid
staging:
  :concurrency: 10
production:
  :concurrency: 20
:queues:
  - default
  - mailers
  - reports

---

Ruby version: 2.4.1
Sidekiq / Pro / Enterprise version(s): Sidekiq open source 5.0.4

[mperham]

No problem, everything will work fine. Sidekiq is designed to run many processes.

[zulhfreelancer]

Thank you!

[mperham]

@adrianmann Please don't hijack closed issues with your own questions.

[jgomo3]

Could someone elaborate a bit on how the coordination between multiple sidekiq processes using the same queue works? Or is there any documentation on the feature?

Thank you.

[mperham]

@jgomo3 Please read my comment literally right above yours.

[jgomo3]

@mperham This is what I see:

No problem, everything will work fine. Sidekiq is designed to run many processes

and

@adrianmann Please don't hijack closed issues with your own questions.

I should be missing something.

[mperham]

You are hijacking a closed issue with your own questions. Open a new issue with your own context and specific question, even if it seems related to this issue.

[jgomo3]

@mperham Oh, I see now. Thank you! Getting used to the rules here. Sorry.