Resque: multiple instances of resque-web on same server

[hackervera]

Regardless of what port I try to run this on, if I already have an instance running, resque-web complains that it's already running. I have two namespaces I would like to monitor. Is there a way to run two instances at once on different ports?

[FewKinG]

I'm interested in the same thing.
I have two different namespaces (tried to namespace by queue name suffix, but that doesn't work well because the workers don't accept wildcards for queue selection). So I used Redis namespacing. But now I have to start up the webinterface twice (once for each namespace). And now that doesn't work either. Any suggestions?

[FewKinG]

Okay nevermind. What doesn't work is to start two instances in the same app directory. But if you have different directories for your namespaces (as it usually is when you have development, staging and production environments) then you can boot up multiple instances on different ports on the same machine without problem.

[ahawkins]

@hone close this. It's been sorted out by the opener.

[gtd]

FewKinG is not the opener, and the explanation given does not make sense or work for me. I want to run resque-web for two different apps, they are most certainly in two different directories, and it does not work. I'm not sure what the implied relationship is between namespaces, directories and environments in that comment. Where do I define a namespace?

[FewKinG]

Okay, let me specify that:

Environment

The rails environment. I use development, staging and production.

Namespace

When you use resque, you can specify a namespace so that all jobs are created and processed under this namespace and won't interfere with jobs in different namespaces. I set up the namespace for the workers in an initializer (i.e. config/initializer/resque.rb):

require 'resque'
require 'resque_scheduler'

Resque.redis = 'localhost:6379'
Resque.redis.namespace = "resque:#{Rails.env}"

When I start resque-web, I specify a config file using the -L my_config_file.rb option. My config file looks like this:

require 'resque'
require 'resque_scheduler'

Resque.redis.namespace = ENV["RESQUE_NAMESPACE"] if ENV["RESQUE_NAMESPACE"]

That way I can specify the namespace by setting the environment variable RESQUE_NAMESPACE when starting the web-app.

My problem

What I tried to do was to start two instances of resque-web, because I wanted to monitor the queues for my development as well as my staging environment at the same time. It didn't work for me, when I tried to run both instances from the same rails app directory.

The solution

My actual setup (and what I guess is the more common case) is that the development environment resides on my development machine and the staging environment in a different place (in my case on the same machine though). This results in the two resque-web instances being run from different rails app directories, which worked perfectly for me.
Of course you have to make sure that both resque-web instances are running on different ports (using the -p option).

I hope I could make it clear that way. It also took me quite some time to figure all that out. If you still have trouble getting it to work, feel free to ask.

[gtd]

Thanks for that. My problem is different. I have two completely different apps, they run in the same environment (it's a service-oriented architecture). I don't use namespaces, I use separate redis databases which saves memory aside from being tidier. It's same redis host, same redis port, same redis namespace, just different database. Resque won't let me start two resque-webs on different ports which I believe is a legitimate bug. I'm under a big deadline right now, but I'll come back and propose a solution when I have time to dig into the code.

[boriskozak]

If you're having trouble starting two resque-web instances on different ports, just use the -P flag to point to a different pid for each instances

resque-web -p 4000 -P foo 
resque-web -p 4001 -P bar

[badrah]

Thanks a lot boriskozak .. That works like a charm.
The funny fact is that your suggestion is very logical .. Don't know why I didn't think of it that way :)

[FewKinG]

Oh wow, so that even works when starting two resque-web instances for the same rails app directory?
Seems logical to me too. Thanks :)