Capybara: Quitting a dead remote session raises a fatal exception

[dylanlive]

Meta

Capybara Version: 2.10.1
Driver Information (and browser if relevant): selenium-webdriver 2.53.4, RSpec 3.4.0, Selenium Grid v.2.53.1

Summary

When a remote Selenium Grid session is disconnected unexpectedly, it's difficult to recover.

A Selenium::WebDriver::Error::UnknownError exception is raised upon exit, even after clearing the session_pool and creating a new instance of the driver (full exception below).

A new driver instance can be created, but after the suite is finished, the old driver instance still tries to exit.

Expected Behavior

When the test suite is finished running, at exit, it should not create an exception if the remote connection is already gone.

Actual Behavior

An exception is triggered.

This prevents the user from recovering by creating a new driver, as the old driver raises an exception at exit.

Steps to reproduce

1) Configure a remote driver to connect to Selenium Grid
2) Add this snippet to a test suite's spec_helper.rb so that the session_pool is cleared, and the example is retried (must be using rspec-retry to use run_with_retry...):

config.around(:each) do |example|
  example.run
  if example.exception
    Capybara.send(:session_pool).clear # Remove old session to recreate driver.
    example.run_with_retry # Re-run the example
  end
end

3) Let the test suite start running
4) Close the remote browser manually, or shut that node down

• An exception will be raised, but the code snippet above will clear the session pool and reconnect to the remote

5) Don't interfere when it reconnects. Examples will pass, but at the very end of the suite, when exiting, this is raised:

Finished in 50.3 seconds (files took 0.42497 seconds to load)
6 examples, 0 failures

/Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/response.rb:70:in `assert_ok': Error communicating with the remote browser. It may have died. (Selenium::WebDriver::Error::UnknownError)
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'intentionally_removed', ip: 'intentionally_removed', os.name: 'Linux', os.arch: 'amd64', os.version: '3.13.0-92-generic', java.version: '1.8.0_91'
Driver info: driver.version: EventFiringWebDriver
Capabilities [{applicationCacheEnabled=true, rotatable=false, handlesAlerts=true, databaseEnabled=true, version=45.0.2, platform=LINUX, nativeEvents=false, acceptSslCerts=true, webStorageEnabled=true, locationContextEnabled=true, browserName=firefox, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true}]
Session ID: de211c78-1dbb-4464-9514-c1a41105b338 (org.openqa.selenium.remote.UnreachableBrowserException)
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/response.rb:34:in `initialize'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/common.rb:78:in `new'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/common.rb:78:in `create_response'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/default.rb:90:in `request'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/http/common.rb:59:in `call'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/bridge.rb:649:in `raw_execute'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/bridge.rb:627:in `execute'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/remote/bridge.rb:220:in `quit'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/selenium-webdriver-2.53.4/lib/selenium/webdriver/common/driver.rb:175:in `quit'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.10.1/lib/capybara/selenium/driver.rb:242:in `quit'
    from /Users/dylan/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.10.1/lib/capybara/selenium/driver.rb:26:in `block in browser'

The code snippet added to spec_helper.rb was meant to recover from a Grid disconnection, but the exception becomes unavoidable when Capybara tries to exit.

I intend to add a rescue within the driver.quit method to account for Selenium::WebDriver::Error::UnknownError, similar to https://github.com/jnicklas/capybara/blob/master/lib/capybara/selenium/driver.rb#L243

[twalpole]

I'm reticent to add a rescue for UnknownError here since it could potentially hide real issues-- it would be much better is selenium actually responded with a specific error when communication has died. For your use case you could just do something along the lines of

Module  SeleniumPatch
   def quit
     super
  rescue Selenium::WebDriver::Error::UnknownError
  end
end

Capybara::Selenium::Driver.prepend SeleniumPatch

Additionally - fully emptying the session pool is a bad idea if you are or ever start to use multiple sessions at once - you'd be much better off just removing the current session from the session pool. That being said with the above patch you shouldn't need to touch the session pool at all, just call #quit on the current session and then it should automatically reconnect during retry.

[twalpole]

c64ca0f3cd26a57b20b72f71297a4e5f92cb7715 Should catch the error generated when the remote browser died. I'm cautious about swallowing other errors that may hide issues the user should be fixing. If there are more needing to be caught in custom situations you can easily prepend a module to swallow those errors too.