Sidekiq: Deprecation warnings for 'script' commands to redis when using redis-namespace

[plcstevens]

Ruby version:
Sidekiq / Pro / Enterprise version(s): Sidekiq Pro 3.4.3 / Sidekiq Enterprise 1.5.1

Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')

I am getting deprecation warnings on using 'scripts' command with Redis, mentioning that support for these will be gone in the next major release of redis-namespace. I have attempted to remove various configuration options from my initialiser, but nothing stops the deprecation warning appearing, leading me to believe this is a problem with using Sidekiq Enterprise and Redis namespace.

Any advice or suggestions would be greatly appreciated.

  * redis (3.3.3)
  * redis-namespace (1.5.3)
  * sidekiq (4.2.9)
  * sidekiq-ent (1.5.1)
  * sidekiq-pro (3.4.3)

url = ENV['REDIS_URL'] || 'redis://localhost:6379/5'
namespace = ENV['REDIS_NAMESPACE'] || "sidekiq-#{Rails.env}"

Sidekiq.remove_delay!

unless Rails.env.test?
  Sidekiq::Enterprise.unique!
  Sidekiq::Client.reliable_push!
end

Sidekiq.configure_client do |config|
  config.redis = { url: url, namespace: namespace }
end

Sidekiq.configure_server do |config|
  config.super_fetch!
  config.reliable_scheduler!

  config.redis = { url: url, namespace: namespace }

  config.periodic do |mgr|
    mgr.register('* * * * *', Worker)
  end
end

2017-03-01T16:05:06.213Z 12285 INFO: Running in ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin15]
2017-03-01T16:05:06.213Z 12285 INFO: Sidekiq Pro 3.4.3 / Sidekiq Enterprise 1.5.1, commercially licensed.
2017-03-01T16:05:06.223Z 12285 INFO: Gained leadership of the cluster
2017-03-01T16:05:06.224Z 12285 INFO: Starting processing, hit Ctrl-C to stop
2017-03-01T16:05:06.225Z 12285 INFO: SuperFetch activated
2017-03-01T16:05:06.230Z 12285 INFO: Managing 1 periodic jobs
Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')
Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')
Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')
Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')
Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')
Passing 'script' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /sidekiq-ent-1.5.1/lib/sidekiq-ent/scripting.rb:13:in `block (2 levels) in bootstrap')

[mperham]

Why are you using namespacing at all?

[plcstevens]

We run a number of different tasks/codebases on the same Redis Node/Instance and so use namespaces to separate our tasks/codebases. It avoids us having to examine which databases on Redis are already in use, and any clashes that could arise from that.

[mperham]

Just so you are aware, I can't support running multiple different Sidekiq Pro/Enterprise apps within one Redis DB. They might be running different versions and some features require coordination which can break with multiple apps, e.g. leader election. It might work 95% of the time but it's impossible to predict what will happen and subtle breakage is almost guaranteed over time.

I strongly urge people to use Redis DBs at minimum to isolate their apps. Namespaces are a hack.

[anothermh]

+1 to "Namespaces are a hack." I stopped using them for similar, difficult to troubleshoot problems. Segregate by database instead.

[n-rodriguez]

Hi! I have the same warning with Sidekiq Community. I use redis-namespace to isolate different services within the same DB. (and 1 DB per application)
For instance :

• sidekiq
• logster
• cache
• sessions
• websockets

Would you suggest to use 1 DB per service? It seems a bit overkill IMHO.

The warning :

Passing 'info' command to redis as is; administrative commands cannot be effectively namespaced and should be called on the redis connection directly; passthrough has been deprecated and will be removed in redis-namespace 2.0 (at /data/concerto/shared/bundle/ruby/2.3.0/gems/sidekiq-4.2.9/lib/sidekiq.rb:95:in `block in redis')

The gems I use :

* sidekiq (4.2.9)
* sidekiq-scheduler (2.1.2)
* redis (3.3.3)
* redis-namespace (1.5.3)

[mperham]

I have no plans to "fix" this. Either you peg an older version or you stop using redis-namespace. My opinion is very clear: redis-namespace is a hack and its use is discouraged. I believe you should use different databases for services and different instances for apps.

[n-rodriguez]

Thanks for your advices (http://www.mikeperham.com/2015/09/24/storing-data-with-redis/).

The trick here was to use ::Sidekiq.redis_info instead of ::Sidekiq.redis(&:info) (in https://github.com/lbeder/health-monitor-rails/blob/master/lib/health_monitor/providers/sidekiq.rb#L50).

And thank you for this very useful gem!

[plcstevens]

Hi @mperham,

Thank you for providing clear feedback around the usage of namespaces. We will look to move our usage away from namespaces towards separate databases/instances.

Cheers,

Phil

[Loremaster]

@mperham can you please then give me a recommendation how to separate db for development and test db correctly? I tried to use such code:

config/initializers/sidekiq.rb

unless Rails.env.production?
  Sidekiq.configure_server do |config|
    config.redis = { url: 'redis://localhost:6379/0', namespace: "sidekiq_app_name_#{Rails.env}" }
  end

  Sidekiq.configure_client do |config|
    config.redis = { url: 'redis://localhost:6379/0', namespace: "sidekiq_app_name_#{Rails.env}" }
  end
end

But that requested me to use redis-namespace which also shows deprecation message + you mentioned here that this is hack. Is there a proper way to separate redis dbs for test and development then? I don't want my tests to influence development database.

[mperham]

@Loremaster just use a different database for test.

  Sidekiq.configure_server do |config|
    config.redis = { db: Rails.env.test? ? 1 : 0 }
  end

  Sidekiq.configure_client do |config|
    config.redis = { db: Rails.env.test? ? 1 : 0 }
  end

[Loremaster]

@mperham thank you very much for that!!

[TheTeaNerd]

@mperham We have a legacy setup where we use only 1 namespace: 'resque'. I'd be :+1: on removing this from our setup but am concerned that we'd lose unprocessed jobs when deploying this change.

How would you suggest we migrate from an old namespace to no namespace in production?

[mperham]

@TheTeaNerd That namespace definitely has to go. 😆

I will write up something on how to migrate away from namespaces for my blog but it will take a few days.

[mperham]

http://www.mikeperham.com/2017/04/10/migrating-from-redis-namespace/