Openfoodnetwork: Chrome and chromedriver version compatibility

[mkllnk]

Description

We use headless Chrome and chromedriver to run our automated tests. The versions of the two need to be compatible. If you install both through your system's package manager they should be good. Semaphore provides compatible versions as well.

While most developers have Chrome installed, many new developers don't have chromedriver installed and need to do that extra step to set up. We started using the chromedriver-helper gem which installs the chromedriver automatically. That makes onboarding new developers or setting up a new computer much easier. Unfortunately, chromedriver-helper doesn't know which version it needs to install. It installs the latest by default which may not be compatible with the installed Chrome version.

Chromedriver-helper Installing the latest chromedriver version broke our Semaphore builds. If we want to have the convenience of chromedriver-helper, we need to find a configuration that will not break the build and is still convenient for developers.

Expected Behavior

• Semaphore can run specs successfully.
• Developers don't have to manually install chromedriver.

Actual Behavior

Semaphore fails.

Severity

The Semaphore build was modified to install a newer Chrome version which fixes the build for now. But that may not be the ideal solution.

Possible Fix

• Remove chromedriver-helper and not have the convenience of automated chromedriver installs.
• Install latest Chrome on Semaphore. This doesn't solve the problem for development environments with older Chrome, e.g. Debian stable.
• Configure chromedriver-helper to install specific version. This will need updating from time to time and we have to find a version that is compatible with all development environments and Semaphore.
• Can we configure chromedriver-helper to not install anything if the system provides chromedriver already?

[mkllnk]

I opened https://github.com/flavorjones/chromedriver-helper/issues/77 for the option to use the system's chromedriver if available. That would solve our problem, but I don't know how difficult it would be to implement. Maybe we can contribute some code to chromedriver-helper. That would be my preferred solution. The operating system's package manager does the version management.

[luisramos0]

Yeah, that issue you opened on chromedriver-helper is a good solution.
The other solution is to install a specific version in semaphore (did you do that? what version is it there now?) and lock chromedriver version to that semaphore version like this (2.44 is for chrome 69):

--- a/spec/spec_helper.rb
+++ b/spec/spec_helper.rb
@@ -48,7 +48,7 @@ Capybara.register_driver :chrome do |app|
  )
  Capybara::Selenium::Driver.new(app, browser: :chrome, options: options)
end
-
+Chromedriver.set_version “2.44”
Capybara.default_max_wait_time = 30 (edited)

link to find the appropriate version:
http://chromedriver.chromium.org/downloads

[sauloperez]

• Configure chromedriver-helper to install specific version. This will need updating from time to time and we have to find a version that is compatible with all development environments and Semaphore.

This seems like the best solution now for me. Like any other app dependency, it's specified in the Gemfile.

[luisramos0]

yes, I agree @sauloperez
but I dont see a way to do this in the gemfile file, only in the spec_helper file according to this

[Matt-Yorkley]

The recommended method is to set it in configs, so spec_helper should be fine. Sticking with version 2.44 of the driver will be compatible with Chrome 69 to 71, I think that's fine given how Semaphore doesn't update its apps very often.

[mkllnk]

If we lock the chromedriver version, it should work with Semaphore for a while. But it may randomly break the setup of devs when a new Chrome version is released. Chrome is updated regularly on most machines.

If we do it through a config file or environment variable, we can install the latest on dev machines and a specific version on Semaphore. Or can we even deactivate it via a config file?

[mkllnk]

The other solution is to install a specific version in semaphore (did you do that? what version is it there now?)

I just installed the latest Chrome on Semaphore because that didn't require a code change.

[luisramos0]

I agree @mkllnk
I am not sure what's the best solution for this.

[Matt-Yorkley]

Installing a specific Chrome version on Semaphore wouldn't be persisted between builds though right?

Semapahore exports a number of environment variables when building (the SEMAPHORE=true var seems a good bet) so we could check for that and conditionally set a Sempahore-specific version in the config?

https://semaphoreci.com/docs/available-environment-variables.html#variables-exported-in-builds-and-deploys

[mkllnk]

That's a good idea. We could also introduce a variable like CHROMEDRIVER_VERSION. So if Semaphore updated their Chrome, we can just change the Semaphore configuration and don't need to touch the code. It also means that I can use it locally for my Debian stable Chromium which is a bit outdated.

We can't control the chrome versions on all dev machines. So we need to make the chromedriver version adaptable to suite the local chrome version. An environment variable seems apt for that.

[Matt-Yorkley]

It would be nice if the gem could just pick a valid version. Maybe they'll add that feature.

I've upgraded to Buster despite Debian's advice, so my Chromium is _shiny_ :sunglasses:
https://wiki.debian.org/DontBreakDebian#Don.27t_suffer_from_Shiny_New_Stuff_Syndrome

[luisramos0]

ah. nice link.
i have learned something today! the good old RTFM has a nice version that i didnt know Read The Fantastic Manuals!

[sauloperez]

So if I followed your dicussion correctly, sounds like reading a new CHROMEDRIVER_VERSION env var with a default value valid for most dev boxes gives us the chance to also configure its value from Semaphore's interface.

What I like about it is that it decouples us from the particular env where it runs. The env var doesn't know about semaphore.

Is that it? Of course, we should give a short summary of the problem in a comment for next devs.

[mkllnk]

Yes, that's it. I would call it OFN_CHROMEDRIVER_VERSION because it's our custom code using that variable. There may be other projects with other versions. Or someone could think it's the system's version even though it isn't.

[mkllnk]

Currently we have the problem that the latest stable Chrome version installed on Semaphore is chrome 73. The latest chromedriver installed via chromedriver-helper supports only chrome 74.

[mkllnk]

I changed the Semaphore configuration now to install a specific chromedriver version that is compatible to the current chrome version. To keep it more stable I removed the part to install the latest chrome version. So we rely on Semaphore's chrome version 72 for now and need to adjust chromedriver when Semaphore updates their chrome version.

I hope that chromedriver-helper has the option to be deactivated on Semaphore by that point. We don't need to change it again then.

[luisramos0]

Thanks Maikel for taking care fo this!

[Matt-Yorkley]

There's been some updates to the background of this issue:

• The gem we're using for managing chromedriver has been deprecated and archived: https://github.com/flavorjones/chromedriver-helper/issues/83
• The maintainer recommends webdrivers gem as a superior replacement: https://github.com/flavorjones/chromedriver-helper/issues/83

[eelcodevlieger]

I locally replaced
gem 'chromedriver-helper'
with
gem 'webdrivers', '~> 3.8'

Then added to /spec/spec_helper.rb:
require 'webdrivers'

And replaced
WebMock.disable_net_connect!(:allow_localhost => true)
with

WebMock.enable!
WebMock.disable_net_connect!({
  allow_localhost: true,
  allow: 'chromedriver.storage.googleapis.com'
})

Temporarily added the following (don't think explicitly calling update is required)

Webdrivers.logger.level = :DEBUG
Webdrivers::Chromedriver.update

This showed the matching chromedriver was downloaded OK. I successfully ran some rspec tests locally to confirm the headless selenium tests work (via the ofn-devbox)

[luisramos0]

awesome!!! do you want to create a PR with that so we can see the full build running with it?

[eelcodevlieger]

Hi Luis, I've not had time to fully test it due to local database issues.
Opened PR https://github.com/openfoodfoundation/openfoodnetwork/pull/3828
just in case the tests might pass ( wishful :) )

[luisramos0]

nice, thanks! it's good to have the PR. maybe someone can pick it up now.