Sidekiq: Sidekiq.syncronous = true (Processing Jobs Automatically in Test Mode)

[ahawkins]

I'm proposing a new feature for sidekiq. I'd like to be able to test my app all the way through when doing integration tests. It's common for me to have jobs added to queues from other classes. It's very awkward to test this whole process because I have to assert that a job is in the queue, then find the right arguments, instantiate a new class and call it with those args. I think there is a better way. This feature is inspired by Resque, but resque fails with true syncronous support.

I'd like to implement this feature in Sidekiq that work this way:

• Jobs are called with the JSON decoded arguments. In resque, the arguments are not roundtriped with JSON. For example, if you enqueue a job with a DateTime/Time/Date the job will receive that object and not whatever the JSON parser spits out.
• Plugins work in syncronous mode. Resque callbacks don't work in when inline is set iirc. This may have changed, but it always annoyed me.
• Current behavior is unchanged.
• Provide sidekiq/rspec, sidekiq/test_unit, sidekiq/mini_test

Thoughts?

[KensoDev]

I am +1 on that.

I wrote a gem for Resque called perform_later, in test mode it works sync so you don't have to worry about the queue in test mode, it's just the regular code that will be queued in production or any other environment.

It's actually a configurable option so you can disable queuing in any ENV just by changing a YAML file setting

[mperham]

It would be very difficult to actually run the job through the full Sidekiq server codepath, including middleware. But we could certainly JSON marshal the payload and then manually call Worker#perform with the arguments as a poor man's integration test. Reasonable?

[ahawkins]

@mperham maybe not the full server code, but how about through plugins?

I think your "poor man's integration test" is a good start. I will work on it when I have a chance. I can provide a simple implementation then we can from there. Sound like a plan?

[mperham]

Yep, pull requests are always welcome.

[mperham]

require 'sidekiq/testing/inline' is now available on master

[phuongnd08]

Hi man, thanks for this. Though it would give me more choices if I can turn inline on/off depends on the type of the test. For some tests, I want to verify the size of the queue, for other I don't want them to run at all (as it may causes extensive loop during testing).

[rpocklin]

Works exactly how i'd expect, thanks. The only side effect is the schedule timing is untestable, which is ok 'cause I mainly want to test the result of the worker, so i've opted just to call Worker.new.perform(...) directly instead.

[phuongnd08]

Ok, so I used this code to on/off inline sidekiq for my specs, hope that any one come across may find it useful:

RSpec.configure do |config|
  config.before(:each, sidekiq: true) do
    load 'sidekiq/testing/inline.rb'
  end

  config.after(:each, sidekiq: true) do
    Sidekiq::Client.class_eval do
      singleton_class.class_eval do
        alias_method :raw_push, :raw_push_old
      end
    end
  end
end

[mhuggins]

The require 'sidekiq/testing/inline' solution doesn't work very well when you only want it enabled for specific tests or queues. @phuongnd08's solution is a slight improvement since it's isolated to specific tests, but it'd be much more ideal to enable inline testing for specific queues instead.

[andresakata]

For specific tests, you could mock like that:

allow(SomeWorker).to receive(:perform_async) do |something_id|
  SomeWorker.new.perform(something_id)
end

[RajRoR]

Thanks @andresakata!! :+1:

It's a wondering for me that sidekiq doesn't have a cleaner way to achieve this goal! Maybe I didn't find one?