Capybara: Hang in `within_frame`, specifically `driver.switch_to_frame(:parent)` in nested iframes

[ioquatix]

Meta

Capybara Version: 3.31.0
Driver Information (and browser if relevant): chrome=80.0.3987.116

Expected Behavior

within_frame(...) should not hang.

Actual Behavior

within_frame(...) hangs.

Steps to reproduce

We had a spec that started failing recently.

def authorize_3d_secure(approve = true)
  # authorisation iframes are really hard to locate in the DOM
  outer_iframe_selector = "iframe[src*='authorize'], iframe[src*='three-ds-2-challenge']"
  # and on top of that, they have nested iframes?!
  inner_iframe_selector = "iframe"

  using_wait_time(15) do
    if has_selector?(outer_iframe_selector)
      within_frame(:css, outer_iframe_selector) do
        sleep 1

        if has_selector?(inner_iframe_selector)
          within_frame(:css, inner_iframe_selector) do

            if approve
              click_button("Complete")
            else
              click_button("Fail")
            end
          end
        end
      end
    end
  end
end

It has to navigate credit card "3D Secure" iframes which are two layers deep.

The code works fine, we navigate to the button. But it hangs when executing the ensure block of within_frame, specifically the line

driver.switch_to_frame(:parent)

I guess that at that point, the parent iframe is already gone, so it hangs indefinitely.

cc @madleech

Backtrace

selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/response.rb:72 in `assert_ok'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/response.rb:34 in `initialize'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/common.rb:88 in `new'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/common.rb:88 in `create_response'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/default.rb:114 in `request'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/http/common.rb:64 in `call'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/bridge.rb:167 in `execute'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/bridge.rb:567 in `execute'
selenium-webdriver-3.142.7/lib/selenium/webdriver/remote/w3c/bridge.rb:177 in `window_handle'
selenium-webdriver-3.142.7/lib/selenium/webdriver/common/driver.rb:197 in `window_handle'
capybara-3.31.0/lib/capybara/selenium/driver.rb:170 in `current_window_handle'
capybara-3.31.0/lib/capybara/selenium/driver.rb:155 in `switch_to_frame'
capybara-3.31.0/lib/capybara/session.rb:398 in `switch_to_frame'
capybara-3.31.0/lib/capybara/session.rb:431 in `within_frame'
capybara-3.31.0/lib/capybara/dsl.rb:58 in `block (2 levels) in <module
my_spec.rb in `block in authorize_3d_secure'
capybara-3.31.0/lib/capybara/session.rb:796 in `using_wait_time'
capybara-3.31.0/lib/capybara/dsl.rb:28 in `using_wait_time'

[twalpole]

From the stacktrace we can see that the hang is inside selenium, which probably means chromedriver is hanging (it also hangs if you try to full screen in headless mode) Not much we can do about it - chromedriver needs to be fixed. You could probably work around the issue temporarily by executing JS that clicks the button after a timeout. It wouldn’t be optimal but may solve your issue until chromedriver is fixed

[ioquatix]

After identifying the issue, I tried to hack around it which at least seems to work (at the expense of breaking/ignoring the scopes stack):

def authorize_3d_secure(approve = true)
  # authorisation iframes are really hard to locate in the DOM
  outer_iframe_selector = "iframe[src*='authorize'], iframe[src*='three-ds-2-challenge']"
  # and on top of that, they have nested iframes?!
  inner_iframe_selector = "iframe"

  using_wait_time(15) do
    if has_selector?(outer_iframe_selector)
      page.driver.switch_to_frame(find(outer_iframe_selector))

      if has_selector?(inner_iframe_selector)
        page.driver.switch_to_frame(find(inner_iframe_selector))

        if approve
          click_button "Complete"
        else
          click_button "Fail"
        end
      end
    end
  end
ensure
  page.driver.switch_to_frame(:top)
end

If this is an issue with chromedriver, should we report the issue upstream? Where?

[ioquatix]

Okay reported it here: https://groups.google.com/forum/#!topic/chromedriver-users/SWqlaLIUZNo

There is another discussion: https://groups.google.com/d/topic/chromedriver-users/nozKcX0W9hM/discussion

Which seems to imply that maybe the issue is not with chrome driver?

[ioquatix]

I will move this issue to selenium.

[twalpole]

If chromedriver isn’t returning or raising an error then it’s definitely a chromedriver problem - but yes, there’s not much Capybara can do about it

[ioquatix]

I think what's happening is:

1/ selecting outer iframe
2/ selecting inner iframe
3/ click button
4/ iframes all destroyed
5/ ensure block is trying to go to parent iframe which is gone, so it hangs waiting for the iframe to come back?

[ioquatix]

I think if the parent iframe was recreated, chromedriver would return?

[twalpole]

It wouldn’t be the same frame anymore so really wouldn’t be valid - if the parent frame is gone chromedriver should error imho

[ioquatix]

@twalpole That makes sense to me, but then how are we supposed to use the interface provided... if we know the iframe is going to be destroyed, the only way I can see to do it is the proposed implementation driver.switch_to_frame manually.

if the parent frame is gone chromedriver should error imho

At least in my case, I don't want to error, because the exact next line is to switch to the parent of that parent, which should be the top. So maybe it would make sense to error if you tried to do anything, but if you are just popping out of a nested context, even if it's destroyed, that should be okay.

[twalpole]

@ioquatix the issue is that the driver (chromedriver, geckodriver, etc) maintains the current context and Capybara needs tot track it too. If the driver doesn't provide any feedback that the context has been destroyed (via an error, etc) there is no way for Capybara to track what the current context is. We can't just pop back up two levels because the user may still be expecting to do something in the outer frame which we should then raise an error for.

If you are going to work around it in this case with switch_to_frame then you'll need to handle updating the scopes stack too, which is why in this case I'd work around it via delayed JS execution. This is definitely a bug in the responses we're getting/not getting from Chrome

[twalpole]

On a side note I do wonder why you're using the has_selector? calls - is this method ever called when the frames don't exist?

[ioquatix]

I didn’t write the original code but yeah maybe it was to work around the delayed creation of the iframes and/or trying to detect failures of the 3d secure I frame even appearing.

Agreed on all your other points. In an ideal case what should be the implementation of this method? No change from the original?

[twalpole]

@ioquatix within_frame or find will already wait for the creation of the iframes so you're not gaining anything from the has_selector? unless the method is ever called without there being iframes, but that seems unlikely. With your workaround it should probably be something along the lines of

def authorize_3d_secure(approve = true)
  # authorisation iframes are really hard to locate in the DOM
  outer_iframe_selector = "iframe[src*='authorize'], iframe[src*='three-ds-2-challenge']"
  # and on top of that, they have nested iframes?!
  inner_iframe_selector = "iframe"
  current_frame = current_scope

  using_wait_time(15) do
      page.driver.switch_to_frame(find(outer_iframe_selector))
      page.driver.switch_to_frame(find(inner_iframe_selector))

        if approve
          click_button "Complete"
        else
          click_button "Fail"
        end
      end
    end
  end
ensure
  page.driver.switch_to_frame(:top)
  raise "You called authorize_3d_secure from within an iframe - behavior is unpredictable" unless current_frame == current_scope
end

[twalpole]

To clarify one thing here -- the issue isn't that an iframe is destroyed - if it was just the current iframe being removed everything would work fine, it's the fact that two nested iframes are removed simultaneously, and apparently almost instantaneously, that causes the issue (if it was delayed even slightly there would be enough time to back out of the nested contexts before removal). You're workaround can be used in this case because you are going back to the top level which can be reset without needing any intermediate context. If you weren't going back to the top level, but instead other levels of nesting, then there may be issues.

[twalpole]

It might be worth implementing scopes snapshotting type behavior in order to allow resetting to a previous position - will need to think about that.

[ioquatix]

Something like push scope and restore scope.

[twalpole]

Closing this since it's definitely not a Capybara issue -- will think about whether it's worth adding features to make working around driver issues like this easier.

[ioquatix]

Okay great.

I guess I wouldn't think of this as a driver issue, but that the parent scope is no longer valid, so switching to it might not be valid, and what to do in that case. At least in my case I don't care, because I just want to get back to the top scope as you already know.

Thanks for your help.

[twalpole]

The driver issue is that it hangs --- nothing anyone can really do if chromedriver hangs because even if we Tim eout we won't be able to communicate with it again.

[ioquatix]

Are you certain it's hanging? If it is, you are right. I just wonder if it was waiting for the iframe to return from the dead

[twalpole]

Waiting forever for something that can’t happen (deleted element) is pretty much the definition of hanging.

[ioquatix]

But it's feasible that the iframe could be recreated, no? There is a difference between hanging because of a logic bug, and switch_to_frame(:parent) which at least in some situations might be valid..

[twalpole]

No it’s not possible to recreate the parent frame it would be a different element so the context would be different

[ioquatix]

Okay so if chromedriver fixes this issue, then it will throw an error when switching to parent frame?

[twalpole]

Possibly, or it may only throw an error if you try and interact with elements in the now deleted frame and let you cleanly go back up 2 levels like 79 used to let you

[ioquatix]

That makes sense.