Sidekiq: TypeError (can't cast Rack::Session::SessionId to string): Postgres / Rails

[lacostenycoder]

Ruby version: 2.3.6
Sidekiq / Pro / Enterprise version(s): Sidekiq 5.1.3

Please include your initializer and any error message with the full backtrace.

config which has not changed recently

#confing/initializers/sidekiq.rb
require 'sidekiq'
require 'sidekiq/web'
require 'sidekiq-status/web'
require 'sidekiq-scheduler/web'
require 'sidekiq-status'
require 'sidekiq-limit_fetch'
require "sidekiq/middleware/server/influxdb" if ENV['INFLUX_DB_URL'].present?

Sidekiq::Extensions.enable_delay!

SIDEKIQ_NOTIFY_LIMITS = {
  'searchkick' => ENV['SIDEKIQ_SEARCHKICK_QUEUE_NOTIFY'],
  'default' => ENV['SIDEKIQ_DEFAULT_QUEUE_NOTIFY'],
}.freeze

Sidekiq.configure_client do |config|
  config.redis = { url: ENV['REDISS_URL'], network_timeout: 5 } if ENV['REDISS_URL']
  config.client_middleware do |chain|
    # accepts :expiration (optional)
    chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes # default
    chain.add Sidekiq::NotifyMiddleware
  end
end

Sidekiq.configure_server do |config|
  config.on(:startup) do
    if Rails.env.production?
      Sidekiq.schedule = YAML.load_file(Rails.root.join('config/sidekiq_scheduler.yml'))
      Sidekiq::Scheduler.reload_schedule!
      text = "Sidekiq/App started on #{ENV['BASE_PATH']} #{ENV['SERVER_NAME'].try(:upcase)} SERVER\n\n"
      text += `git log`
      SlackNotifier.new(text: text, channel: 'server_status').notify
    end
  end

  config.redis = { url: ENV['REDISS_URL'], network_timeout: 5 } if ENV['REDISS_URL']
  config.server_middleware do |chain|
    # accepts :expiration (optional)
    chain.add Sidekiq::Status::ServerMiddleware, expiration: 30.minutes # default
    chain.add Sidekiq::Cable::ServerMiddleware
  end
  config.client_middleware do |chain|
    # accepts :expiration (optional)
    chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes # default
  end

  if ENV['INFLUX_DB_URL'].present?
    config.server_middleware do |chain|
      except = [
          CleanupOldReportsWorker, DeleteOldPapertrailVersionsWorker, DynamicWorker,
          FakePatientsWorker, SeedsWorker
      ]
      chain.add Sidekiq::Middleware::Server::InfluxDB,
                influxdb_client: InfluxDB::Client.new(url: ENV['INFLUX_DB_URL']), # REQUIRED
                # series_name: 'sidekiq_jobs',                  # optional, default shown
                # retention_policy: nil,                        # optional, default nil
                # start_events: true,                           # optional, default true
                tags: { application: 'Thoroughcare' },          # optional, default {}
                except: except                                  # optional, default []
    end
  end
end

Backtrace errors

TypeError (can't cast Rack::Session::SessionId to string):
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/quoting.rb:35:in `rescue in type_cast'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/quoting.rb:23:in `type_cast'
activerecord (4.2.11) lib/active_record/connection_adapters/postgresql_adapter.rb:598:in `block in exec_cache'
activerecord (4.2.11) lib/active_record/connection_adapters/postgresql_adapter.rb:597:in `map'
activerecord (4.2.11) lib/active_record/connection_adapters/postgresql_adapter.rb:597:in `exec_cache'
activerecord (4.2.11) lib/active_record/connection_adapters/postgresql_adapter.rb:585:in `execute_and_clear'
activerecord (4.2.11) lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `block in select_all'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/query_cache.rb:83:in `cache_sql'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/query_cache.rb:68:in `select_all'
activerecord (4.2.11) lib/active_record/querying.rb:39:in `find_by_sql'
activerecord (4.2.11) lib/active_record/relation.rb:639:in `exec_queries'
activerecord (4.2.11) lib/active_record/relation.rb:515:in `load'
activerecord (4.2.11) lib/active_record/relation.rb:243:in `to_a'
activerecord (4.2.11) lib/active_record/relation/finder_methods.rb:500:in `find_nth_with_limit'
activerecord (4.2.11) lib/active_record/relation/finder_methods.rb:484:in `find_nth'
activerecord (4.2.11) lib/active_record/relation/finder_methods.rb:127:in `first'
activerecord-session_store (1.1.1) lib/active_record/session_store/session.rb:58:in `find_by_session_id'
activerecord-session_store (1.1.1) lib/action_dispatch/session/legacy_support.rb:47:in `find_session'
activerecord-session_store (1.1.1) lib/action_dispatch/session/legacy_support.rb:40:in `get_session_model'
activerecord-session_store (1.1.1) lib/action_dispatch/session/active_record_store.rb:83:in `block in write_session'
activerecord-session_store (1.1.1) lib/active_record/session_store/extension/logger_silencer.rb:47:in `silence_logger'
activerecord-session_store (1.1.1) lib/action_dispatch/session/active_record_store.rb:82:in `write_session'
activerecord-session_store (1.1.1) lib/action_dispatch/session/legacy_support.rb:26:in `set_session'
rack (1.6.12) lib/rack/session/abstract/id.rb:369:in `commit_session'
rack (1.6.12) lib/rack/session/abstract/id.rb:253:in `context'
rack (1.6.12) lib/rack/session/abstract/id.rb:247:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/cookies.rb:560:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
activerecord (4.2.11) lib/active_record/query_cache.rb:36:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
activerecord (4.2.11) lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.2.11) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
activesupport (4.2.11) lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
activesupport (4.2.11) lib/active_support/callbacks.rb:81:in `run_callbacks'
actionpack (4.2.11) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
railties (4.2.11) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.2.11) lib/rails/rack/logger.rb:20:in `block in call'
activesupport (4.2.11) lib/active_support/tagged_logging.rb:68:in `block in tagged'
activesupport (4.2.11) lib/active_support/tagged_logging.rb:26:in `tagged'
activesupport (4.2.11) lib/active_support/tagged_logging.rb:68:in `tagged'
railties (4.2.11) lib/rails/rack/logger.rb:20:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
request_store (1.4.1) lib/request_store/middleware.rb:19:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/request_id.rb:21:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
rack (1.6.12) lib/rack/methodoverride.rb:22:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
rack (1.6.12) lib/rack/runtime.rb:18:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/static.rb:120:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
rack (1.6.12) lib/rack/sendfile.rb:113:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
actionpack (4.2.11) lib/action_dispatch/middleware/ssl.rb:24:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
sentry-raven (2.7.4) lib/raven/integrations/rack.rb:51:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
railties (4.2.11) lib/rails/engine.rb:518:in `call'
railties (4.2.11) lib/rails/application.rb:165:in `call'
newrelic_rpm (5.7.0.350) lib/new_relic/agent/instrumentation/middleware_tracing.rb:92:in `call'
unicorn (5.4.0) lib/unicorn/http_server.rb:606:in `process_client'
unicorn-worker-killer (0.4.4) lib/unicorn/worker_killer.rb:52:in `process_client'
unicorn (5.4.0) lib/unicorn/http_server.rb:701:in `worker_loop'
unicorn (5.4.0) lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
unicorn (5.4.0) lib/unicorn/http_server.rb:142:in `start'
unicorn (5.4.0) bin/unicorn:126:in `<top (required)>'
/usr/local/bundle/bin/unicorn:23:in `load'
/usr/local/bundle/bin/unicorn:23:in `<top (required)>'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb:74:in `load'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb:74:in `kernel_load'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb:463:in `exec'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb:27:in `dispatch'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb:18:in `start'
/usr/local/bin/bundle:30:in `block in <main>'
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/friendly_errors.rb:124:in `with_friendly_errors'
/usr/local/bin/bundle:22:in `<main>'

Are you using an old version? Yes
Have you checked the changelogs to see if your issue has been fixed in a later version?
Yes it doesn't appear to have been an issue fixed.

https://github.com/mperham/sidekiq/blob/master/Changes.md
https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md
https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md

[mperham]

Unclear what this has to do with Sidekiq.

[jches]

@lacostenycoder I ran into this error as well - appears to be related to a change in rack that affects activerecord-session_store. I only see this happen under the sidekiq web interface, which seems odd.

Unfortunately a fix for rails 4.2 seems unlikely; from what I can tell AR-session_store no longer supports 4.2. However, I was able to get sidekiq web working by disabling sessions in the sidekiq initializer as described in the wiki: https://github.com/mperham/sidekiq/wiki/Monitoring#sessions-being-lost

[lacostenycoder]

@jches thanks a bunch that seems to have fixed the problem!!!

[bdavidxyz]

@jches thanks a lot. I'm not sure why they removed the paragraph from the wiki, but it's still accessible by looking at the revision/history

[turnspike]

I also had this issue on Rails 6.0.3.3 on localhost, here's the fix as per the above advice:

# config/initializers/sidekiq.rb
...
Sidekiq::Web.disable :sessions

[hussainakram]

For me this worked:

# config/initializers/sidekiq.rb
require 'sidekiq/web'
Sidekiq::Web.set :sessions, false

on Rails6.0.2.3 app