Rspec-core: Access to arbitrary options metadata within "before" block

[wincent]

I'm trying to use Steak and Capybara to run some scenarios using a JavaScript-capable backend (Culerity) and others using the standard rack_test backend.

In Cucumber this was done using tags, but with Steak and RSpec 1.3 I'm told you can do something like this:

scenario 'foo', :js => true do
  # ... this one runs under culerity
end

scenario 'bar' do
  # ... this one runs under rack_test
end

This is enabled by this config:

Spec::Runner.configure do |config|    
  config.before :each do
    Capybara.current_driver = :culerity if options[:js]
  end

  config.after :each do
    Capybara.use_default_driver if options[:js]
  end
end

Under RSpec 2 this trick won't work; it seems that "options" isn't available in the scope of these before/after config blocks. I've been able to dig out the options via some very ugly "instance_variable_get" inspection:

RSpec.configure do |config|
  config.before :each do
    if self.running_example.instance_variable_get(:@options)[:js]
      Capybara.current_driver = :culerity
    end
  end

  config.after :each do
    if self.running_example.instance_variable_get(:@options)[:js]
      Capybara.use_default_driver
    end
  end
end

So this ticket is about getting access to the arbitrary metadata that can be passed in to "describe" and "it" blocks via the options hash.

Note that in the use case I'm talking about above, using the "filter_run" stuff won't give me what I want (that's for selecting a _subset_ of the examples to run, but I want to run _all_ of the examples, but switch Capybara drivers on the fly).

What do you think about providing access to the options via an accessor?

I've noticed that I can get at the options passed in to the RSpec::Core::Example subclass; ie:

it "should foo", :stuff => true do
  ...
end

But I can't seem to get at the stuff passed in to the RSpec::Core::ExampleGroup subclass; ie:

describe "foo", :thing => true do
  ...
end

In practice I don't really care too much about access to that metadata, because I want to switch drivers on a per-scenario basis (ie. at the "it/scenario" level). But if you agree to making the metadata available at that level, I guess it makes sense to make the metadata at the "describe/feature" level available too.

What do you think?

If I can get a "green light" on the idea I'm happy to put a patch together.

Cheers,
Wincent

[dchelimsky]

Green light! I'd like to have it work through the running_example variable though, not just options, to keep the local namespace uncluttered. Either of these would work for me:

RSpec.configure do |config|    
  config.before :each do
    Capybara.current_driver = :culerity if running_example[:js]
  end
end

RSpec.configure do |config|    
  config.before :each do
    Capybara.current_driver = :culerity if running_example.options[:js]
  end
end

Separate, but related, what do you think about renaming running_example to example?

[wincent]

Yes, makes sense to avoid cluttering the local namespace. Looking at the options:

1: if example[:foo]
2: if example.options[:foo]
3: if running_example[:foo]
4: if running_example.options[:foo]

Option "2" reads nicest to me because it is close to the English "if the example option foo is set". I'll go with that for now.

Will start playing and see if I can knock a patch together before the day is out.

[dchelimsky]

Great. Thanks!

[wincent]

Ok, have something now. Seeing as you can't attach patches to Github tickets, have pushed a few commits to forks here:

http://github.com/wincent/rspec-core/commits/core-ticket-42

And here:

http://github.com/wincent/rspec-rails/commits/core-ticket-42

Specifically we're talking about these commits:

http://github.com/wincent/rspec-core/commit/99f0428d53b98cee8a60250d1347224f3dbc5697
http://github.com/wincent/rspec-core/commit/2ed401684b68e404d75be051b932d592b260dc00
http://github.com/wincent/rspec-rails/commit/66c5df14c5f7e4d0f8e42f98009837077c680437
http://github.com/wincent/rspec-rails/commit/0c5801a412bfd3abaac5ed7222e4e1d34e4e78e7

The actual changes to the codebase were trivial (20 minutes of coding), but I've had quite a bit of trouble and spent several hours trying to keep the specs and features passing.

One of the problems is that it is quite fiddly if you make a change which requires simultaneous modifications across different repos. For example, the renaming of "running_example" to "example" in rspec-core requires changes in rspec-rails because the latter uses "running_example".

So you make the change in both repos at once, but you get failures in rspec-rails because it is using the system version of rspec-core rather than the local one.

The spec failures can be fixed by tweaking the Gemfile in the generated example Rails app so that it uses the local version of rspec-core instead of some other version that it might pull from elsewhere on the system. (That's what commit 0c5801a412bfd does.)

In this way you can make the change in both rspec-core and rspec-rails at the same time, and all specs keep passing.

The feature failures are another matter entirely. Cucumber insists on using some other version of rspec-core rather than the local one. I've spent hours trying to find out why, but I'm afraid I have to give up.

That in turn means I am not so confident about commit 66c5df14c5f7e. In particular I am not sure about the change to lib/rspec/rails/adapters.rb. I changed the instance variable name from @running_example to @example in the "method_name" method, but I don't even know what that method does, and the specs/features pass irrespective of what the instance variable is called.

So, take a look at the commits and perhaps you can see something that I can't that explains how to get Cucumber to use the right version of rspec-core.

In any case, with these commits we can do stuff like:

RSpec.configure do |config|
  config.before :each do
    Capybara.current_driver = :celerity if example.options[:js]
  end
end

There is one possible further modification that I was playing around with, but it is a bit more invasive so I haven't committed anything yet.

The thing is that in a "before :all" block there is obviously no "running example" at that point, so if the user tries to do something like this they'll bomb out because "example" is nil at this point:

config.before :all do
  do_something if example.options[:foo]
end

At that point we're executing within the scope of a RSpec::Core::ExampleGroup class, so if we ask for "example" we hit the "example" accessor and get nil back.

Note:

config.before :all do
  # people could do it this way, and it works...
  do_something if self.class.metadata[:foo]
  # but I expect lots will try "example.options" and fail anyway...
end

The idea I was playing with was in that context, returning a different non-nil object (probably a Metadata subclass) that responds to "options", and would return the user-supplied metadata passed in at the describe-block level. ie:

describe Thing, :js => true do
  before :all do
    example.options[:js] # => true
  end

  before :each do
    example.options[:foo] # => defined below
  end

  it "should behave", :foo => true do
  end
end

So you could define your metadata at any level, and access it at any level.

Not sure if this is opening up a can of worms, though, cause then we'd have to think about how this metadata should "trickle" down when nesting is involved, e.g:

describe Outer, :foo => 1 do
  describe Inner, :bar => 2 do
    before :all do
      # here should have access to :foo and :bar
    end

    before :each do
      # here should have access to :foo and :bar too
      # plus anything set at the "its" level
    end

    it "should behave", :foo => :override, :baz => 3 do
      # here :foo => override, :bar => 2, :baz => 3
    end
  end
end

[dchelimsky]

There's a lot to review in that comment and I'm pressed for time this minute, but re: the name change - we'll want to deprecate running_example as there are other consumers of that method in the wild. So in rspec-core, something like:

def example
  # code moved from running_example
end

def running_example
  RSpec.deprecate("running_example", "example")
  example
end

Then you don't have to worry about the other libs just yet and they can move independently.

[wincent]

Ah, yes, good idea. I'd seen the "RSpec.deprecate" in a couple places in the codebase but didn't think to use it.

[wincent]

Added the deprecation warning as suggested:

http://github.com/wincent/rspec-core/commit/272c3762fa0af775f0c7d23afffa3196bb7ad81c

[dchelimsky]

Merged - thanks!!!!!

[cmrd-senya]

Just for the record, since this issue is what I found using "rspec metadata before" search-engine query and since I couldn't find it documented anywhere.

This behaviour can be achieved in a up-to-date rspec version the following way:

before do |example|
  example.metadata[:foo] # => true
end

it "should behave", :foo => true do
end