Selenium: Firefox 59 fails to return dom elements

[itajaja]

• OS: OSX
• Selenium Version: 3.11.0 through protractor
• Browser: Firefox 59
• geckodriver-v0.20.0

since the latest firefox version (59) doing a POST /execute call that returns a DOM element fails with the following:

Failed: TypeError: cyclic object value
    Build info: version: '3.11.0', revision: 'e59cfb3', time: '2018-03-11T20:33:15.31Z'
    System info: xxx os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.3', java.version: '1.8.0_74'
    Driver info: driver.version: unknown

all other browsers, including firefox 58 don't have problems with this. I suppose that this happens because it tries to serialize an object with cyclical references.

[jimevans]

The W3C WebDriver Specification expressly calls out that attempting to return objects from JavaScript that have cyclic references is disallowed and should return an error. Sounds like Firefox now implements that part of the spec. For what it’s worth, the IE driver will act the same way, so saying “all other browsers” is slightly inaccurate.

[itajaja]

returning dom elements is a pretty common use case though. should we handle the serialization in userland?

[jimevans]

But returning a DOM element doesn't return a node reference in WebDriver. It returns a custom object that is a reference to a DOM element. Returning other types of DOM objects (like "document" or "window"), on the other hand, are especially problematic, and the driver explicitly excludes that case.

[jquense]

How are we supposed to implement custom element locators if we can't return DOM elements from a js run? I'm happy to adjust but it doesn't seem possible any other way?

[itajaja]

btw that would mean that all the protractor's API would have to be rethought? http://www.protractortest.org/#/api?view=ElementFinder or maybe I am missing something

[jimevans]

Folks, don't misunderstand me please. I'm not saying one can't return element references from JavaScript. That has always been the case, and will always continue to work. What I _am_ saying is that if you attempt to return an object from JavaScript that contains circular references, that will now fail. When you return an element reference from JavaScript, you _aren't_ returning the element itself, nor even an JSON serialization of the element. What gets passed back from the driver is just a simple object that represents the element that can be used in subsequent WebDriver calls.

I don't know Protractor well enough to know if they have something in their framework that attempts to return objects from JavaScript execution in the browser* that might contain circular references.

*I realize that Protractor is a JavaScript-based library, so it's all JavaScript execution. What I'm talking about here is JavaScript execution _in the browser_, as described by driver.executeScript.

[jquense]

Sorry I feel like maybe we are misunderstanding each other then. We are saying that returning DOM elements via executeScript is in fact throwing this error.

[jimevans]

And what I'm saying is that it's not globally broken, not for Firefox, not for geckodriver. Just yesterday, I ran the Selenium project's .NET test suite using geckodriver 0.20, against Firefox, and it was able to return elements from executeScript calls without difficulty, and without throwing an error.

[itajaja]

ok, then let me run additional tests and maybe add more reproducibility 👍

[itajaja]

ok, the problem is that react adds a __reactInternalInstance property on every dom element, and that makes the DOM element contain a circular reference. I am not sure what's the right way to do that. It seems like Firefox is having an expected behavior, but this makes really difficult to fix this problem without some selenium/webdriver change 🙁

[barancev]

Please provide a concise reproducible test case.

I've tried this sample using Selenium 3.11 Java binding, Firefox 61 (Nightly) and geckodriver 0.20 and it works well, no exceptions:

driver.get("https://foxhound87.github.io/mobx-react-form-demo/demo");
WebElement element = (WebElement) driver.executeScript("return document.getElementById('username--20')");
element.sendKeys("Test");

[itajaja]

here is a small repro:

const {Builder, By, Key, until} = require('selenium-webdriver');

(async function example() {
  let browser = new Builder()
    .forBrowser('firefox')
    .usingServer('http://localhost:4444/wd/hub')
    .build();
  try {
    await browser.get('http://todomvc.com/examples/scalajs-react/#/');
    const element = await browser.findElement(By.js(`
      return document.getElementsByTagName("input")
    `));
  } finally {
    await browser.quit();
  }
})();

using:

• webdriver-manager 12.0.6
• selenium-webdriver ^4.0.0-alpha.1
• firefox 59
• geckodriver-v0.20.0
• OSX

[itajaja]

note that if I change the js snippet to return document.getElementsByTagName("input")[0] then it works. as if the array is not cleaned properly maybe?

[pittgoose]

@barancev I'm confused why you say you're testing against Firefox 61 when the issue description, and all examples, say Firefox 59. If FF 61 isn't broken it might be safe to assume that they've already fixed this issue in subsequent builds, but I am also seeing this issue in FF 59.

[barancev]

@pittgoose
1) Actually, I've tested my sample with both FF 59 (release) and FF 61 (nightly)
2) No matter what version of the browser did I try or not, it's not an excuse to not provide a reproduction scenario for the issue :)

[barancev]

@itajaja
Thank you! The issue can be reproduced on this site.

[itajaja]

I am glad! so is this a problem with FF59 only or also with nightly?

[barancev]

Both FF 59 (release) and FF 61 (nightly) suffer of this issue.

[mbruning24]

Having the same issue, but using the by.repeater('obj in array') syntax with an angular application on FF 59. I believe this uses the same type of logic where it executes a script on the browser.

FF 59.0.1
geckodriver 0.20.0
Selenium 3.11.0
Protractor 5.3.0

EDIT downgraded to FF 58.0.2 and the problem went away.

[jleyba]

@itajaja getElementsByTagName returns a HTMLCollection, not an actual array. Notably:

Array.isArray(document.getElementsByTagName('input')) === false

Since it's not an array, Firefox is probably handling it like an object and finding a cycle (as it would with the DOM). I suspect it would work if you used return Array.from(document.getElementsByTagName("input"))?

[jquense]

We tried that @jleyba :/ originally out hunch was actually the opposite, that arrays weren't handled and HtmlCollection's were. But it turns out both aren't.

[Artur-]

Ran into the same issue but thought the problem was somewhere else as I have seen similar problems on Edge or IE11. Related to https://github.com/w3c/webdriver/issues/1241 and https://github.com/webcomponents/shadydom/issues/217

My testing indicates that Firefox 58.0.2 works, 59.0b3 and newer do not.

The only workaround I have found so far is to override the JSON serialization of the problematic element properties, e.g. element[key].toJSON = function(){return;}

[barancev]

Raised an issue on the driver: https://bugzilla.mozilla.org/show_bug.cgi?id=1447977

[monkpit]

I am seeing the same issue in a similar situation - getting elements with protractor on a react page. The problematic method in my code appears to be By.cssContainingText - looks like this is returning an array and iterating over each element to check if it contains the given text. I get the same cyclic object value error when running my tests, but I don't have the opportunity to use the [0] fix mentioned by @itajaja since the array is never directly exposed to me in my code.

Example (on a page running react, in Firefox 59):
element(By.cssContainingText('button', 'click me')).click()

[jerrythimothyj]

Hi, I am facing the same issue:

element(By.$(selector)) is throwing me Failed: TypeError: cyclic object value .

Any updates on this issue?

[whimboo]

Raised an issue on the driver: https://bugzilla.mozilla.org/show_bug.cgi?id=1447977

I was not able to reproduce the problem with any of the provided testcases so far. So if you are still able to see this problem with Firefox 61 onwards please comment here and add a reference to a publicly available web page, which reproduces the problem. Thanks.

[whimboo]

Ok, I'm now able to reproduce. All future updates will be visible in the above mentioned bug. I will close this issue when the problem has been fixed.

[lmtierney]

@whimboo So this issue can be closed?

[whimboo]

No, the bug hasn't been fixed yet. But I'm currently looking into it.

[lmtierney]

@whimboo but the bug is not with selenium is it? So we can close this issue in favor of the bugzilla one?

[whimboo]

Feel free to do so in this case.

[lmtierney]

Closed in favor of https://bugzilla.mozilla.org/show_bug.cgi?id=1447977

[andreastt]

I’ve just fixed this in https://bugzilla.mozilla.org/show_bug.cgi?id=1447977.

[barancev]

Awesome! Waiting for Nightly with this fix to test it.

[Risce]

I'm having the same issue in Protractor, Firefox 61 using element(by.buttonText("XXX"). I'm not able to click on it.

JavascriptError: TypeError: cyclic object value
Build info: version: '3.13.0', revision: '2f0d292', time: '2018-06-25T15:32:19.891Z'

[andreastt]

Well that build from last month won’t have the fix in it, will it?

[Risce]

This error can be reproduced with the basic Protractor example on www.protractortest.org, it is happening when you use Protractor locators.

[barancev]

@Risce Note that the fix has been landed 2 days ago and it's available in Firefox Nightly only at the moment.

[andreastt]

We can probably consider uplifting it to at least beta. I will ask.

[Risce]

@barancev @andreastt Thank you for your answer. I tried with the Firefox Nightly and works perfect. Thanks.

[Anojan]

@barancev @andreastt seems the issue still there in nightly as well? I ran my scripts against 62.0b8 (64-bit), and the selenium webdriver I use is 3.4.0
I get the same "JavascriptError: TypeError: cyclic object value" error

[andreastt]

@Anojan I can tell from the error message you get that you’re not using the latest Nightly. The error message in Nightly would not contain “TypeError”.

[whimboo]

62.0b8 means it is version 62.0 beta 8. If you want a nightly you have to download it from https://www.mozilla.org/en-US/firefox/channel/desktop/#nightly

[Anojan]

@andreastt @whimboo apologies guys, I was using beta version instead, my bad. @andreastt will the fix be applied to beta any soon?

[andreastt]

@Anojan I think we’re going to hold off uplifting it to beta because we have a suspicion it is causing intermittent timeouts on try.

[whimboo]

Please note that this patch landed on beta now. So with the next Firefox beta release it will be available to use.

[steffen-harbich-cognitum]

Did the firefox release 62 fix your problems? In my case, the line
((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot", element);
is still not working for polyfilled shadow DOM of Vaadin flow components:
org.openqa.selenium.JavascriptException: Cyclic object value

[whimboo]

@steffen-harbich-itc if shadowRoot is not working please file a new geckodriver issue. This is actually something new. Thanks.