Selenium: Add support for W3C actions to node client

[noyainrain]

tl;dr Please implement the W3C actions API for the node client.

Using the latest selenium-webdriver node client (3.5) to drive a local instance of the latest Firefox (55) on Linux, calling the user interactions API results in the infamous UnknownCommandError: POST /session/[...]/moveto did not match a known command error.

To my understanding, Firefox / geckodriver has implemented the W3C actions API, while the node client at the moment only supports an older version of the API (having a quick look at http.js, the /actions endpoint is missing).

It seems this has already been implemented in other language bindings, for example the Java client. It would be awesome if we could get W3C action support for the node bindings to :) .

PS: Thanks for your great work on Selenium!

[kaiyoma]

I just ran into this as well. I'm running tests on Windows with Firefox 55 and a recent version of the gecko driver and I'm getting this error:

UnknownCommandError: POST /session/ff91f250-8eb1-44ba-94ae-1c55b5a96629/moveto did not match a known command

(Full bug report: https://github.com/mozilla/geckodriver/issues/958)

My tests work perfectly in Chrome and Chrome Headless, but not Firefox.

[dlgoodchild]

Just hit this too running selenium standalone firefox docker container.

image: "selenium/standalone-firefox-debug:3.6.0-bromine"

[jleyba]

I completely overhauled the actions API to support W3C. I'm not entirely sure how I feel about the API (or the API for any of the language bindings for that matter). It's kind of difficult to setup given how you can define parallel sequences across multiple devices.

Please take a look and let me know what you think -- note I'm not planning on publishing another npm release until 4.0, so you'll have to grab this from head.

I'm hoping for feedback on documentation [1] and overall usage [2].

[1] https://github.com/SeleniumHQ/selenium/blob/master/javascript/node/selenium-webdriver/lib/webdriver.js#L2522
[2] https://github.com/SeleniumHQ/selenium/blob/master/javascript/node/selenium-webdriver/test/actions_test.js

[noyainrain]

Hey @jleyba, sorry for the late reply, had some busy days.

First, thank you for implementing the new API, that's awesome! :+1:

After having a look at the W3C spec, I think the proposed API captures it pretty well. Some thoughts:

• input.Sequence might benefit from a perform() method (just a wrapper around ActionSequence.perform()), so we could still write one-liners for simple use cases, e.g. driver.actions().mouse().click(elem1).click(elem2).perform();.
• How would we make use of multiple pointers / fingers? Something like actions().touch(0).move(...) and actions().touch(1).move(...)?
• The idea of synchronize() is neat. The documentation for ActionSequence may be a bit clearer if it showed an example of pause() and then went on with something like "... This can get cumbersome. Rather than keeping track of the number of steps for the keyboard to pause..."

I see you removed support for the legacy version of ActionSequence. For our project, at the moment we only support testing on Chrome but not Firefox and this would just swap the problem. Would you consider keeping the legacy version around until the next major release (5), for example with an extra argument like actions(legacy=true)?

It would be even better for backward compatibility if the old ActionSequence interface could internally use the new W3C actions protocol if available and fall back otherwise. I know that this may involve extra effort better spent elsewhere, just saying it for completeness sake :) .

[noyainrain]

Just a short addition to the previous feedback:

I've refactored one of our UI tests to use the new API, ran it with a current local Firefox and everything works as expected :) .

Without the legacy version available, it does not work with any other common browser, as none has implemented the new spec yet:

• https://chromium.googlesource.com/chromium/src/+/master/docs/chromedriver_status.md
• https://docs.microsoft.com/en-us/microsoft-edge/webdriver
• https://developer.apple.com/library/content/documentation/NetworkingInternetWeb/Conceptual/WebDriverEndpointDoc/Commands/Commands.html

[jleyba]

Thanks for the feedback. I think I'm going to try to transparently support the legacy protocol (if W3C actions fail, it will translate to legacy and try again). I just started looking at it, but I'm hoping to have something to push within a few days.

[jleyba]

OK, I'm happier with this latest version. It's pretty similar to the legacy API, but still makes it possible to define parallel actions for the browsers that support it (right now just Firefox). The new API does support translating to mouse+keyboard actions to the legacy API, but it's not 100%, so you have to explicitly enable this behavior (see the new docs for details)

No browsers currently support touch/pen actions with W3C actions, so those aren't surfaced through the API yet.

[jleyba]

The new API is included in 4.0.0-alpha.1, which has been published to npm

[sahil1610]

@jleyba I am still facing the same issue with 4.0.0-alpha.1.

POST /session/{sessionId}/moveto did not match a known command

[biot023]

I'm seeing this with 4.0.0-alpha.1, too.
The error I'm getting is this:

node:13870) UnhandledPromiseRejectionWarning: UnsupportedOperationError:

Cannot POST /session

at parseHttpResponse (/home/mycomputer/work/home/dp/0_dev/node_modules/selenium-webdriver/lib/http.js:557:11)
at Executor.execute (/home/mycomputer/work/home/dp/0_dev/node_modules/selenium-webdriver/lib/http.js:468:26)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:160:7)

(node:13870) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:13870) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Ah, markdown is stripping some data out. Basically I'm getting an HTML file output that complains in a PRE tag that it "Cannot POST /session".
I'm not asking it to post to a /session url anywhere in my code.

[alexandergallen]

@jleyba does this also have an affect when using directConnect in protractor? or does it then skip all selenium related issues?

[jleyba]

@sahil1610 & @biot023 you have to use driver.actions({bridge:true}) to enable the fallback behavior. In hindsight, I should have made strict-W3C compliance opt-in. I'll fix that for the next release.

@alexandergallen Sorry, I have no idea what "directConnect in protractor" is.

[MagicPoulp]

jleyba,
With try/catch you could detect this error and write in the error message to suggest people to use {bridge: true} to fallback to the legacy API

W3C actions were 3 months ago only supported in firefox. Since actions are broken on HEAD for chrome (were 3 months ago), we need clearer error messages, or doc about the legacy fallback that is very nice.

[jleyba]

@MagicPoulp pull requests are always welcome

[fcheslack]

On 4.0.0a1
When using the
driver.actions({bridge:true})
most driver functions I am using now work as expected, however attempting to access browser logs
via driver.manage().logs().getAvailableLogTypes() or driver.manage().logs().get('browser')
I still get a
UnknownCommandError: GET /session/75b679f8-3544-46d2-94ec-16e442857fbb/log/types did not match a known command
or
UnknownCommandError: POST /session/655265ba-2324-4f3c-86bc-8894564098a0/log did not match a known command

[MagicPoulp]

"@MagicPoulp pull requests are always welcome"

My suggestion was to do mainly documentation. Something as fundamental as using actions was broken on HEAD.

WHen something does not work, it is common for people to have a look at the CHANGES. This is where I learnt about the W3C thing.

The fallback mechanism you provided is not 100% "plug-and-play". People still get errors when they don't know about it, you should document.

I had other issues with actions with actions involving SVG DOM objects not behaving the same across browsers.

--> suggestion to improve the quality of this repo (it is already of quality the Javascripy selenium is much cooler than in other frameworks 100%):
documentation, keeping backward compatibility (actions were broken on HEAD whilst they worked before), good commits names, commit time in chronological order (they are not in chronological order so I had difficulty to find the first commit where the error happened, and the nI could report you that it was broken on HEAD and then you made teh fallback mechanism)

[diemol]

This has been open for a while but I believe W3C actions should be already implemented in the JS bindings, do you know if this is true @Harsha509?

[JS31096]

Hi @diemol ,

Yes, W3C actions are implemented in the JavaScript bindings (Checked in alpha 4 versions, also we have good code samples tested in here https://www.selenium.dev/documentation/en/support_packages/mouse_and_keyboard_actions_in_detail/).

But, Am not sure if this thread contains discussion about any other issue (as i see lot of tagged links of issues from protractor).

This issue is also referenced in selenium-ide too in here https://github.com/SeleniumHQ/selenium-ide/issues/329, So @corevo comments also will be helpful in conforming this.

Thanks,
Harsha

[diemol]

Thanks, @Harsha509!
The issue is only about W3C actions for JS in this repo.
I'll close it then.