Protractor: Update dependency on selenium-webdriver to 3.7.0 or newer

[tbowmo]

I have been seeing a lot of random errors like

- Error: ECONNREFUSED connect ECONNREFUSED 127.0.0.1:62416
at ClientRequest.<anonymous> (C:\AzureAgentUI14\_work\4\s\src\Gandalf.Web\node_modules\selenium-webdriver\http\index.js:238:15)
at emitOne (events.js:116:13)
at ClientRequest.emit (events.js:211:7)
at Socket.socketErrorListener (_http_client.js:387:9)
at emitOne (events.js:116:13)

While exploring this error I found that it is selenium-webdriver that throws the error (as also noted in the error dump above). Protractor relies on [email protected].

However, in [email protected] they fixed this issue and implemented a more generic retry feature in case of any network issues with the webdriver socket.

Is there a possibility to update the dependencies on selenium-webdriver? As this would fix some of our problems with unstable tests.

[qiyigg]

I don't think there's a [email protected] for javascript, as far as I know, the next version is 4.0.0 which will break all the tests that are using control-flow.

[tbowmo]

ahh yes, I discovered that as well..

I solved it by using the patch-package npm package.. So we can apply a patch on our build agents on the fly, that forces selenium-webdriver to retry on the ECONNREFUSED error (as well as a couple of other errors).

[simonua]

@tbowmo, that sounds pretty interesting, and I may have a similar issue. Are you at liberty to share some of your solution, please? Thank you!

[tbowmo]

@simonua
I ended up using https://github.com/ds300/patch-package to patch the http client in selenium-webdriver. Basically just copying over this file to node_modules/selenium-webdriver/http, and create a patch using the patch-package module

You have to edit the index.js after you copied it over, as it doesn't do retries on ECONNREFUSED out of the box

[simonua]

@tbowmo, that's awesome! Thanks a lot for the information! I'll give it a go.

Cheers,

Simon

[rafalf]

@tbowmo can you post it how you patched it. I am having some problems

[tbowmo]

@rafalf what parts of the patching process are you having issues with?

[rafalf]

@tbowmo I copied the file, modified package and added the script line and then .. tried to run yarn to patch but got errors.
``` npx patch-package selenium-webdriver/
patch-package: you have both yarn.lock and package-lock.json
Defaulting to using npm
You can override this setting by passing --use-yarn or deleting
package-lock.json if you don't need it

☑ Creating temporary folder
☑ Building clean node_modules with npm
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: Jade has been renamed to pug, please install the latest version of pug instead of jade
npm WARN deprecated [email protected]: to-iso-string has been deprecated, use @segment/to-iso-string instead.
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN prepublish-on-install As of npm@5, prepublish scripts are deprecated.
npm WARN prepublish-on-install Use prepare for build steps and prepublishOnly for upload-only.
npm WARN prepublish-on-install See the deprecation note in npm help scripts for more information.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] prepublish: gulp prepublish
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] prepublish script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

16185 info lifecycle [email protected]~prepublish: [email protected]
16186 verbose lifecycle [email protected]~prepublish: unsafe-perm in lifecycle true
16187 verbose lifecycle [email protected]~prepublish: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/private/var/folders/vp/f4s9240s0wx2t0r91vpcqdkc0000gn/T/tmp-60871hvF7dbpUr4JB/node_modules/.bin:/usr/local/lib/node_modules/protractor/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
16188 verbose lifecycle [email protected]~prepublish: CWD: /private/var/folders/vp/f4s9240s0wx2t0r91vpcqdkc0000gn/T/tmp-60871hvF7dbpUr4JB
16189 silly lifecycle [email protected]~prepublish: Args: [ '-c', 'gulp prepublish' ]
16190 silly lifecycle [email protected]~prepublish: Returned: code: 1 signal: null
16191 info lifecycle [email protected]~prepublish: Failed to exec prepublish script
16192 verbose stack Error: [email protected] prepublish: gulp prepublish
16192 verbose stack Exit status 1
16192 verbose stack at EventEmitter. (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:285:16)
16192 verbose stack at EventEmitter.emit (events.js:182:13)
16192 verbose stack at ChildProcess. (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
16192 verbose stack at ChildProcess.emit (events.js:182:13)
16192 verbose stack at maybeClose (internal/child_process.js:947:16)
16192 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:236:5)
16193 verbose pkgid [email protected]
16194 verbose cwd /private/var/folders/vp/f4s9240s0wx2t0r91vpcqdkc0000gn/T/tmp-60871hvF7dbpUr4JB
16195 verbose Darwin 15.6.0
16196 verbose argv "/usr/local/Cellar/node/6.7.0/bin/node" "/usr/local/bin/npm" "i"
16197 verbose node v10.0.0
16198 verbose npm v5.6.0
16199 error code ELIFECYCLE
16200 error errno 1
16201 error [email protected] prepublish: gulp prepublish
16201 error Exit status 1
16202 error Failed at the [email protected] prepublish script.
16202 error This is probably not a problem with npm. There is likely additional logging output above.
16203 verbose exit [ 1, true ]
```

would you be able to post end to end how you did it

[tbowmo]

I can't really remember the exact steps that I did, but I believe that I forced it to use yarn (--use-yarn option), as that is our preferred package manager currently.

But install patch-package as per the description on their site, then update the files needed under selenium-webdriver, and then

yarn patch-package selenium-webdriver --use-yarn

That should create a patch file for selenium...

[rene-leanix]

Thanks for pointing us to the fix in selenium-webdriver, @tbowmo! All that's needed is the line added in this commit.

(I also wasn't able to use patch-package (with npm) for this, as it failed on a post-install script.)

[tbowmo]

@rene-leanix
In our case we needed to add ECONNREFUSED to the retry list as well, as we got frequent issues with that on our azure CI agents.

[simonua]

@tbowmo, same has been true for my organization.

[rafalf]

@simonua @rene-leanix @tbowmo How did you update this file?
I am on 5.4 protractor and looking at /usr/lib/node_modules/protractor/node_modules/selenium-webdriver/http/.index.js
and I dont have any of these functions:
shouldRetryRequest
isRetryableNetworkError

[tbowmo]

@rafalf I gave a recipe in reply no. 4 (if my calculations are right)

[rafalf]

this 3.6 index.js and there is no 276 line at all
https://github.com/SeleniumHQ/selenium/blob/selenium-3.6.0/javascript/node/selenium-webdriver/http/index.js

so I have no idea how did you add one line
can you elaborate

@rene-leanix

[rene-leanix]

Yes, you're right, @rafalf. My apologies! I didn't realise that this whole block was added in the transition from 3.6 to 3.7. So taking the index.js from selenium-3.7.0 and addingECONNREFUSED as suggested above is the only way forward for now, until Protractor updates the dependency.

[rafalf]

thanks