Nightwatch: Doesn't work with selenium v3.5.0 (but works with v.3.4.0)

[subuta]

Thanks for awesome library!

We recently tried to use nighwatch with remote selenium(runs on modern.ie Windows env)
When we tried to use nightwatch with selenium v3.5.0, nightwatch not works correctly. response looks like below.

[Sample Test] Test Suite
==========================

Running:  some tests
INFO Request: POST http://192.168.99.100:4444/wd/hub/session 
 - data:  {"desiredCapabilities":{"browserName":"internet explorer","javascriptEnabled":true,"acceptSslCerts":true,"platform":"ANY","allowBlockedContent":true,"ignoreProtectedModeSettings":true,"name":"test"}} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":204}
INFO Response 200 POST http://192.168.99.100:4444/wd/hub/session (2223ms) { status: 0,
  sessionId: '8bc6d284-222b-487b-9bfb-13e1b1039233',
  value: 
   { acceptInsecureCerts: false,
     browserName: 'internet explorer',
     browserVersion: '11',
     pageLoadStrategy: 'normal',
     platformName: 'windows',
     'se:ieOptions': 
      { browserAttachTimeout: 0,
        elementScrollBehavior: 0,
        enablePersistentHover: true,
        'ie.browserCommandLineSwitches': '',
        'ie.ensureCleanSession': false,
        'ie.fileUploadDialogTimeout': 3000,
        'ie.forceCreateProcessApi': false,
        ignoreProtectedModeSettings: true,
        ignoreZoomSetting: false,
        initialBrowserUrl: 'http://localhost:39519/',
        nativeEvents: true,
        requireWindowFocus: false },
     setWindowRect: true,
     unhandledPromptBehavior: 'dismiss' } }
INFO Got sessionId from selenium 8bc6d284-222b-487b-9bfb-13e1b1039233
INFO Request: POST http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/url 
 - data:  {"url":"http://10.0.2.2/"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":30}
INFO Response 200 POST http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/url (1556ms) { state: 'success',
  sessionId: null,
  hCode: 24164963,
  value: null,
  class: 'org.openqa.selenium.remote.Response',
  status: 0 }
LOG     → Completed command url (1558 ms)
INFO Request: POST http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/elements 
 - data:  {"using":"css selector","value":"body"} 
 - headers:  {"Content-Type":"application/json; charset=utf-8","Content-Length":39}
INFO Response 200 POST http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/elements (264ms) { state: 'success',
  sessionId: null,
  hCode: 14282023,
  value: [ { 'element-6066-11e4-a52e-4f735466cecf': '20953718-0db8-4133-86d1-7245e5ba3ef4' } ],
  class: 'org.openqa.selenium.remote.Response',
  status: 0 }
INFO Request: GET http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/element/undefined/displayed 
 - data:   
 - headers:  {"Accept":"application/json"}
ERROR Response 500 GET http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/element/undefined/displayed (76ms) { state: 'stale element reference',
  sessionId: null,
  hCode: 14596883,
  value: 
   { additionalInformation: '\nDriver info: driver.version: unknown',
     localizedMessage: 'Error setting arguments for script\nFor documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html\nBuild info: version: \'3.5.0\', revision: \'8def36e068\', time: \'2017-08-10T23:00:22.093Z\'\nSystem info: host: \'IE11WIN7\', ip: \'192.168.99.100\', os.name: \'Windows 7\', os.arch: \'x86\', os.version: \'6.1\', java.version: \'1.8.0_144\'\nDriver info: driver.version: unknown',
     supportUrl: 'http://seleniumhq.org/exceptions/stale_element_reference.html',
     systemInformation: 'System info: host: \'IE11WIN7\', ip: \'192.168.99.100\', os.name: \'Windows 7\', os.arch: \'x86\', os.version: \'6.1\', java.version: \'1.8.0_144\'',
     cause: null,
     suppressed: [],
     message: 'Error setting arguments for script\nFor documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html\nBuild info: version: \'3.5.0\', revision: \'8def36e068\', time: \'2017-08-10T23:00:22.093Z\'\nSystem info: host: \'IE11WIN7\', ip: \'192.168.99.100\', os.name: \'Windows 7\', os.arch: \'x86\', os.version: \'6.1\', java.version: \'1.8.0_144\'\nDriver info: driver.version: unknown',
     hCode: 32286113,
     class: 'org.openqa.selenium.StaleElementReferenceException',
     buildInformation: null },
  class: 'org.openqa.selenium.remote.Response',
  status: 10 }

But when we try same test case with Selenium 3.4.0, it works as expected.

Is nightwatch compatible with selenium v3.5.0 or not?

[IEDriverServer changelog]
https://github.com/SeleniumHQ/selenium/blob/master/cpp/iedriverserver/CHANGELOG

this line looks like suspicious for me ...

To successfully use this version of the IE driver, you should be using
   version 3.5 or above of the language bindings. Those are the only versions
   of language bindings that will understand the payload required to be sent
   during the creation of a new session. Earlier versions of the language
   bindings are not guaranteed to be fully compatible with this release of
   the IE driver.

nightwatch.json(only relevant section)

{
  "src_folders" : ["__tests__/nightwatch/test"],
  "output_folder" : "reports",
  "custom_commands_path" : "",
  "custom_assertions_path" : "",
  "page_objects_path" : "",
  "globals_path" : "",

  "selenium" : {
    "start_process" : false
  },

  "test_settings" : {
    "default" : {
      "launch_url" : "http://10.0.2.2",
      "selenium_port"  : 4444,
      "selenium_host"  : "192.168.99.100",
      "silent": true,
      "desiredCapabilities": {
        "browserName": "internet explorer",
        "javascriptEnabled": true,
        "acceptSslCerts": true,
        "allowBlockedContent": true,
        "ignoreProtectedModeSettings": true
      }
    },

    "ie" : {
      "desiredCapabilities": {
        "browserName": "internet explorer",
        "javascriptEnabled": true,
        "acceptSslCerts": true,
        "allowBlockedContent": true,
        "ignoreProtectedModeSettings": true
      }
    }
  }
}

Sample tests

module.exports = {
  'should close modal when click close button': (browser) => {
    browser
        .url('http://10.0.2.2/xxx')
        .waitForElementVisible('body', 1000)
        .end()
  }
}

Versions

Nightwatch version: 0.9.16
Node.js version: v7.10.0
OS version: Windows 7 Enterprise SP1 (modern.ie / Virtualbox)
Browser version: Internet Explorer 11 (v11.0.9600.17420)
selenium-server-standalone version: v3.5.0
IEDriverServer version: v3.5.0(Win32)

Env

192.168.99.100:4444 -> selenium-standalone-server
10.0.2.2:80 -> Web server (at Virtualbox host)

[acierto]

Same for me, works fine with Chrome, but doesn't work with Firefox.
It opens the browser, you can see application well, but any kind of conditions are failing.

[hbobenicio]

I just downloaded the latest selenium-server-standalone (3.5.3) and tried to make a simple waitForElementVisible() / setValue() example to work. After a lot of
Timed out while waiting for element <body> to be visible for 6000 milliseconds. - expected "visible" but got: "not visible"
I found this issue. Then I just downgraded to selenium-server-standalone 3.4.0 and it just worked, finally.

It definitely doen't work with the selenium 3.5 version

nightwatch --version
nightwatch v0.9.16

[saibtech]

Is the samething working in firefox ??

[IceCreamYou]

I'll add to this that with Selenium 3.5.x, even though Nightwatch (0.9.16) commands seem to work fine in Chrome, the chromedriver (2.32) process doesn't get released when the Nightwatch tests are done.

Selenium 3.4.0 does seem to do better with Firefox (56, with marionette/geckodriver 0.18.0) but there are some things that still don't work with it (e.g. getLog(), moveToElement()/moveTo(), keys()).

[straris]

Same with selenium 3.6.0. & geckodriver 0.19
The browser can open a url but can't interact or find any elements. chromedriver works fine

[matthewfinn]

Finding the same issue with Selenium 3.5.0 & 3.6.0 with geckodriver 0.19.

Reverting to 3.4.0 works fine

[ghost]

Hi,

We tried also to work with Selenium 3.5+ and the Gecko driver version 0.19.

The strangest thing is that if you wrote this thing in your page object:

this.waitForElementPresent('@returnFlightRadioButton', this.api.globals.waitForSectionVisibleTimeout)
          .moveToElement('@returnFlightRadioButton' 0, 0)
          .click('@returnFlightRadioButton');

It doesn't work.

However, if you write the following:

this.waitForElementPresent('input[name=therightname]', this.api.globals.waitForSectionVisibleTimeout)
          .moveToElement('input[name=therightname]', 0, 0)
          .click('input[name=therightname]');

it works.

I am wondering if there is not some magic going on in the Nightwatch API which may explain why one way works and the other doesn't.

And it happens only with Firefox and the Gecko driver.

Any updates from the maintainers?

[straris]

@jackTheRipper your issue sounds different, where are you getting the @returnFlightRadioButton from?

[ghost]

@straris : I don't think it's different since I am experiencing the same issue as the one described here.

This is from my page object, my selectors are specified in my page object as it follows:

export default function searchbox() {
  return {
    selector: '.searchbox',
    elements: {
      returnFlightRadioButton: { selector: 'input[name=therightname]' },
    },
    commands: [{
      clickOnReturn() {
        this.waitForElementPresent('@returnFlightRadioButton', this.api.globals.waitForSectionVisibleBigTimeout)
          .moveToElement('@returnFlightRadioButton', 0, 0)
          .click('@returnFlightRadioButton');
      },
    }],
  };
};

[straris]

@jackTheRipper does it work for you with selenium 3.4.0?

[ghost]

Yes, it does.

But it does work with Selenium 3.5+ as well as soon as I am not using the @mySelector way for writing my page objects

[straris]

that is interesting, I am not using the selectors implementation, but might be an indication for the maintainers

[ghost]

@straris: are you using a Selenium Grid?

[straris]

yup

[ghost]

Which configuration are you using?

When running with Selenium 3.6, I am using Firefox 56 and Gecko driver 0.19.0.

I deploy my grid behind NGinx.

I access my websites using the HTTP basic authentication and a Firefox Profile.

Here is my configuration:

'firefox': {
      'desiredCapabilities': {
        'browserName': 'firefox',
        'javascriptEnabled': true,
        'acceptSslCerts': true,
        'firefox_profile': firefoxProfile
      }
    },

The Firefox profile is a base64 encoded string.

What are you using on your side?

[straris]

Same driver/selenium configuration, using docker for deployment.

const firefoxConf = { browserName: 'firefox', javascriptEnabled: true, acceptSslCerts: true };

[goatsy]

Hi, is this going to be fixed any time near in the future?
We have issues with the most simple tests (element is not visible) when we use Firefox.
The exact same test runs with Chrome.

Going back to 3.4.0 did not help so far.
Which geckodriver/firefox version should we use?

[LocalCommit]

Hi. To these one:

geckodriver 0.17.0
Firefox 54.0
Selenium v.3.4.0

[ryan-sandy]

I'm pretty sure it's related to this issue #1332.

[straris]

@ryan-sandy do you have any clues about it? It looks like #1332 is related to using gecko directly but this is not the case for this one.

[ryan-sandy]

@straris I'm not sure, as I am totally new to Selenium. My understanding was Selenium 3 called the geckodriver, but you could run geckodriver without selenium.

When I look at the output of Selenium in this issue, it matches the output of #1332.

INFO Response 200 POST http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/elements (264ms) { state: 'success',
  sessionId: null,
  hCode: 14282023,
  value: [ { 'element-6066-11e4-a52e-4f735466cecf': '20953718-0db8-4133-86d1-7245e5ba3ef4' } ],
  class: 'org.openqa.selenium.remote.Response',
  status: 0 }
INFO Request: GET http://192.168.99.100:4444/wd/hub/session/8bc6d284-222b-487b-9bfb-13e1b1039233/element/undefined/displayed 
 - data:   
 - headers:  {"Accept":"application/json"}

You can see the value is the same string reported in issue #1332.

[{ 'element-6066-11e4-a52e-4f735466cecf': 'blah ID' }]

I think it is an issue with Nightwatch expecting a different key from the value array. That's why downgrading works.

Again, I'm a new to Selenium/geckodriver so I could be totally wrong.

[quantuminformation]

Everything works fine on a mac, but when trying to run on Win7 64, we get this:
```bash
Starting selenium server... started - PID: 17228

[Layouts Test] Test Suite

Running: turbinesTestWithSettingSearchValue
Error retrieving a new session from the selenium server
Connection refused! Is selenium server started?
{ value:
{ stacktrace: 'org.openqa.selenium.WebDriverException: Timed out waiting for driver server to start.\nBuild info: version: \'3.7.1\', revision: \'8a0099a\', time: \'2017-11-06T21:07:36.161Z\'\nSystem info: host: \'W7D0596\', ip: \'172.21.1.54\', os.name: \'Windows 7\', os.arch: \'amd64\', os.version: \'6.1\', java.version: \'1.8.0_91\'\nDriver info: driver.version: unknown\r\n\tat org.openqa.selenium.remote.service.DriverService.waitUntilAvailable(DriverService.java:192)\r\n\tat ```

any ideas what we can do next?

[beatfactor]

This is fixed in v0.9.19.

[awatson1978]

I really wish there were somewhere that listed the Nightwatch/Selenium/Chromedriver version compatibilities. It's becoming obvious that there are only specific combinations that work; but that info isn't well documented. This and this are the best I can find. :/

Nightwatch
Selenium > v3.5.0, use Nightwatch v0.9.19
Selenium =< v3.4.0, use Nightwatch v0.9.16

Selenium / ChromeDriver Compatibility
Selenium Client version 2.53.1 - Supports ChromeDriver v2.35

ChromeDriver / Chrome Compatibility
ChromeDriver v2.35 - Supports Chrome v62-64
ChromeDriver v2.34 - Supports Chrome v61-63
ChromeDriver v2.33 - Supports Chrome v60-62
ChromeDriver v2.32 - Supports Chrome v59-61
ChromeDriver v2.31 - Supports Chrome v58-60
ChromeDriver v2.30 - Supports Chrome v58-60
ChromeDriver v2.29 - Supports Chrome v56-58
ChromeDriver v2.28 - Supports Chrome v55-57
ChromeDriver v2.27 - Supports Chrome v54-56
ChromeDriver v2.26 - Supports Chrome v53-55
ChromeDriver v2.25 - Supports Chrome v53-55
ChromeDriver v2.22 - Supports Chrome v49-52
ChromeDriver v2.20 - Supports Chrome v43-48

[SindreSvendby]

A update list for chrome -> chromedriver compatibility can always be found here:
https://sites.google.com/a/chromium.org/chromedriver/downloads

[StephanBijzitter]

@beatfactor using 0.9.20 I still get this error on AWS Lambda. What could be the reason?