Nightwatch: isVisible is calling getElementProperty on Safari, which is not supported for iOS

Created on 22 Jan 2020 · 27Comments · Source: nightwatchjs/nightwatch

Describe the bug

We're trialling cross-browser testing user Browserstack, which enables us to test against mobile browsers. There was a change made to isVisible in the API in October that's supposed to temporarily fix some issue in Safari (see here), but this is causing tests with Browserstack using an iOS to fail. With the error Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented. Interestingly, MacOS Safari via Browserstack _also_ fails with, though not with the same error (see debug-macos.log below).

Is this condition still necessary, and if so, what for? Is there a way we can work around it for browserstack, as both browsers work if the condition is removed.

Sample test

sampleTest.js

module.exports = {
    "Visible iOS test": function (browser) {
        browser
            .url("https://example.com/")

        browser.assert.visible('body div h1')
    }
}

Run with command

For iOS Safari

$ nightwatch test/sampleTest.js -e safari

For MacOS Safari

$ nightwatch test/sampleTest.js -e iphone

Verbose output

debug-iphone.log

Using insecure HTTP connection on port 80. Consider using SSL by setting port to 443 in your Nightwatch configution.

[Visible Test] Test Suite
=========================
ℹ Connected to hub-cloud.browserstack.com on port 80 (10454ms).
  Using: safari (undefined) on MAC 12.1 platform.

Running:  Visible iOS test

 Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented

 Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented

 Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented

 Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented

 Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented

✖ Testing if element <body div h1> is visible in 5000ms - expected "is visible" but got: "element could not be located" (5710ms)
    at Object.Visible iOS test (/Users/harleyf/IdeaProjects/nightwatch-demo/tests/visibleTest.js:6:24)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)


FAILED: 1 assertions failed and  5 errors (7.713s)

  See more info, video, & screenshots on Browserstack:
  [Browserstack URL]
_________________________________________________

TEST FAILURE: 5 errors during execution; 1 assertions failed, 0 passed (19.896s)

 ✖ visibleTest
 – Visible iOS test (7.713s)
   Testing if element <body div h1> is visible in 5000ms - expected "is visible" but got: "element could not be located" (5710ms)
       at Object.Visible iOS test (/Users/harleyf/IdeaProjects/nightwatch-demo/tests/visibleTest.js:6:24)
       at processTicksAndRejections (internal/process/task_queues.js:85:5)

  TimeoutError: An error occurred while running .isVisible() command on <body div h1>:
   {"value":false}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>:
   {"value":false}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>:
   {"value":false}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>:
   {"value":false}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>:
   {"value":false}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)

debug-macos.log

Using insecure HTTP connection on port 80. Consider using SSL by setting port to 443 in your Nightwatch configution.

[Visible Test] Test Suite
=========================
ℹ Connected to hub-cloud.browserstack.com on port 80 (10628ms).
  Using: Safari (12.1.2) on macOS platform.

Running:  Visible iOS test

✖ Testing if element <body div h1> is visible in 5000ms - expected "is visible" but got: "element could not be located" (5850ms)
    at Object.Visible iOS test (/Users/harleyf/IdeaProjects/nightwatch-demo/tests/visibleTest.js:6:24)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)


FAILED: 1 assertions failed and  4 errors (8.358s)

  See more info, video, & screenshots on Browserstack:
  https://automate.browserstack.com/builds/e2980acd9f7cc70b18943add625cedb7fad86965/sessions/9de9b04bf3be91090ce2b2dcf13a1772a5360156
_________________________________________________

TEST FAILURE: 4 errors during execution; 1 assertions failed, 0 passed (20.656s)

 ✖ visibleTest
 – Visible iOS test (8.358s)
   Testing if element <body div h1> is visible in 5000ms - expected "is visible" but got: "element could not be located" (5850ms)
       at Object.Visible iOS test (/Users/harleyf/IdeaProjects/nightwatch-demo/tests/visibleTest.js:6:24)
       at processTicksAndRejections (internal/process/task_queues.js:85:5)

  TimeoutError: An error occurred while running .isVisible() command on <body div h1>: undefined; undefined
   {"value":true}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>: undefined; undefined
   {"value":true}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>: undefined; undefined
   {"value":true}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)


  TimeoutError: An error occurred while running .isVisible() command on <body div h1>: undefined; undefined
   {"value":true}
       at processTicksAndRejections (internal/process/task_queues.js:85:5)

Configuration

nightwatch.json

{
    "src_folders": ["tests"],

    "selenium": {
        "start_process": false,
        "host": "hub-cloud.browserstack.com",
        "username": "username",
        "access_key": "access_key",
        "port": 80
    },

    "test_settings": {
        "iphone": {
            "desiredCapabilities": {
                "browserstack.user": "username",
                "browserstack.key": "access_key",
                "browserstack.local": "true",
                "device": "iPhone XS",
                "realMobile": "true",
                "os_version": "12"
            }
        },
        "safari": {
            "desiredCapabilities": {
                "browserstack.user": "username",
                "browserstack.key": "access_key",
                "browserstack.local": "true",
                "os": "OS X",
                "os_version": "Mojave",
                "browser": "Safari",
                "browser_version": "12.0",
                "resolution": "1920x1080"
            }
        }
    }
}

Your Environment

| Executable | Version |
| ---: | :--- |
| nightwatch --version | 1.3.2 |
| npm --version | 6.9.0 |
| yarn --version | 1.3.2 |
| node --version | 12.6.0 |

| Browser driver | Version |
| --- | --- |
| Browserstack | n/a (see config) |

| OS | Version |
| --- | --- |
| iOS | 12 |
| macOS Mojave | unknown |

Source

Topperfalkon

👍8

Most helpful comment

@WilliamABradley would you mind raising a separate issue for the waitForElementVisible failure? I've got an idea how to fix it, but the pathway seems to be a little different. However, the workaround @tellisnz-shift posted might work for you in the meantime.

Actually, never mind about the new issue, I'm working on a solution as I think it's a shared problem between isVisible, getValue, and the underlying _waitForDisplayed commands.

Topperfalkon on 10 Jun 2020

👍5

All 27 comments

Hi @Topperfalkon, I have the same problem. Did you find a solution ?

jean9696 on 29 Jan 2020

I patched the Nightwatch api locally to skip the conditional, but we're not currently using this for our CI tests, so that was fine enough to prove the issue.

The offending conditional is here: https://github.com/nightwatchjs/nightwatch/blob/master/lib/api/element-commands/isVisible.js#L53

If you need this for CI, I'd suggest taking a fork and removing the conditional for your usage.

I haven't tested it against macOS Safari on an actual device yet, it might be there's something specific with running it via Browserstack. I've not been able to do a more detailed investigation yet.

I would have offered a pull request to remove the conditional, but it was added recently so was hoping to understand from the Nightwatch team why it was added.

Topperfalkon on 29 Jan 2020

❤1

I hope the Nightwatch team is going to change that. Thank you for your reply ! 🙂
I ended up using waitForElementPresent instead of waitForElementVisible which is ok for now.

jean9696 on 29 Jan 2020

@Topperfalkon in the latest version, safaridriver doesn't include the /displayed endpoint, so if you try to run assert.visible() it fails. The best way to address this I guess it would be to check if Safari is local, using this.settings.webdriver.start_process in the same place where we check for browser name.

beatfactor on 29 Jan 2020

👍1

Ok, that's interesting. I would have thought Browserstack would be using safaridriver for their desktop sessions.

I'll use what you've suggested and do some more investigation, thanks @beatfactor!

Topperfalkon on 29 Jan 2020

I am facing the same issue when running tests on iOS emulator. "Error while running .getElementProperty() protocol action: Appium error: Method has not yet been implemented" and running the tests locally not on Browserstack. I can see temporary fix made for safari in the isVisible(), but this is not working. Any further suggestions?
I am running tests on
Safari 13.04
appium is 1.15.0.
nightwatch is 1.3.1

SowmyaMurthy on 4 Feb 2020

@SowmyaMurthy what is your configuration? I want to propose a fix for this, but I don't currently use Appium, so it might impact the fix

Topperfalkon on 5 Feb 2020

Actually, thinking about this further, this might break testing against older versions of safari too.

https://developer.apple.com/documentation/webkit/macos_webdriver_commands_for_safari_12_and_later

So we should only drop into the error-handling conditional when we know the safari version is 12 or above, and start_process is enabled.

I'm aiming to spend some time on this on Friday.

Topperfalkon on 5 Feb 2020

@beatfactor, @Topperfalkon do you have any updates about this issue?

azinchenko91 on 9 Mar 2020

@azinchenko91 I have a pull request that fixes the issue in most cases, but might have undesirable side effects. I'm trying to get more details from Browserstack about their Safari setup and some assistance in debugging the issues I'm still having with their service, but it's going slowly.

If you have an urgent need, you could always take my fork, or patch your installation of Nightwatch.

Topperfalkon on 9 Mar 2020

@Topperfalkon I had the same problem with waitForElementVisible. I have added your condition in the
waitForElementVisible.js and it is working now. Thank you so much

azinchenko91 on 9 Mar 2020

My PR for this issue is still open, but needs some more investigation into the behaviour with Browserstack and SauceLabs (I can replicate on both). I've had conversation with Browserstack's support and they claim it's working on their end, but I haven't yet been able to follow up. Anyone else with capacity to investigate further are more than welcome to take my PR and carry on where I left off

Topperfalkon on 27 Mar 2020

👍1

As the issue is not in the test infrastructure itself, it not only exists in Browserstack, but also in Sauce Labs.
Hope this could get fixed soon. I am blocked by this too.

robert-cui-tile on 17 Apr 2020

@robert-cui-tile I've not had a chance to investigate this further yet, Browserstack still seems to think there isn't a problem. I'll try and continue looking into it next week. In the meantime, if you're comfortable patching your copy of Nightwatch you can take the changes in my PR and apply them directly.

Topperfalkon on 19 Apr 2020

👍3

Should be fixed in v1.3.6

Topperfalkon on 30 May 2020

@Topperfalkon What is fixed in 1.3.6?

I am still getting Error while running .getElementProperty() protocol action: Unknown command from Safari 13 on MacOS Catalina via Browserstack

WilliamABradley on 4 Jun 2020

Are you definitely using 1.3.6? I can try and look at this later on.

Topperfalkon on 4 Jun 2020

I am using 1.3.6 and waitForElementVisible() still doesn't work on Safari on browserStack 2385

ituradastra on 8 Jun 2020

@Topperfalkon From package-lock.json:

    "nightwatch": {
      "version": "1.3.6",
      "resolved": "https://registry.npmjs.org/nightwatch/-/nightwatch-1.3.6.tgz",
      "integrity": "sha512-61kz2mw3Ng8Rrs2CDZv6aVB+bW+oNIFXL543L9kOvOqft3zVh2j08W8ww6BxGDzmWZe1HGRXNEI5U8+I4hO4KA==",
      "requires": {
        "assertion-error": "^1.1.0",
        "chai-nightwatch": "^0.4.0",
        "ci-info": "^2.0.0",
        "dotenv": "7.0.0",
        "ejs": "^2.7.4",
        "envinfo": "^7.5.1",
        "lodash.clone": "3.0.3",
        "lodash.defaultsdeep": "^4.6.1",
        "lodash.merge": "^4.6.2",
        "minimatch": "3.0.4",
        "minimist": "^1.2.5",
        "mkpath": "1.0.0",
        "mocha": "^6.2.2",
        "ora": "^4.0.3",
        "proxy-agent": "^3.1.1",
        "request": "^2.88.2",
        "request-promise": "^4.2.5",
        "semver": "^6.3.0",
        "strip-ansi": "^6.0.0"
      }
    },

WilliamABradley on 8 Jun 2020

Hmm... ok. I'll see if I've got time this afternoon to investigate further

Topperfalkon on 8 Jun 2020

I'm also getting this on SauceLabs running on emulated iOS Safari while trying to expand our current test set to mobile browsers. I've replaced waitForElementVisible with waitForElementPresent for now, but there's also an issue with assert.value where it seems that GetValue also swapped to getElementProperty in the same October change. I've swapped the assert.value assertions with attributeEquals(selector, 'value', expected, message) for now.

tellisnz-shift on 10 Jun 2020

Actually, never mind about the new issue, I'm working on a solution as I think it's a shared problem between isVisible, getValue, and the underlying _waitForDisplayed commands.

Topperfalkon on 10 Jun 2020

👍5

hi @Topperfalkon, do you have any suggestions on this issue? The only thing i did is replaced waitForElementVisible with waitForElementPresent, but it's not ideal.

overclocker21 on 25 Aug 2020

Hey @overclocker21, I opened a pull request a while back that should fix the issue, but has some caveats and might break functionality elsewhere, which I assume is why it hasn't been merged in yet. You can have a look through and either copy those changes or take my branch and use that in the meantime if you're happy with the potential issues it may have.

Topperfalkon on 26 Aug 2020

Hey guys, any updates about this one?
waitForElementVisible still not working on Safari, I have to use waitForElementPresent instead and works fine.

allanpfreitas on 1 Oct 2020

@allanpfreitas if you can, apply the changes from https://github.com/nightwatchjs/nightwatch/pull/2425 locally and use that. I've had good success with it so far.

MicahLC on 2 Oct 2020

I requested @beatfactor to apply these changes, I was hoping to see them in 1.5.0, but still the same, I'll wait for another updated version. But the 1.5.0 has good features similar to Selenium. Happy about that. :)