Openlibrary: Implement cross browser testing for better compatibility across the browsers.

[feat7]

Description

We don't have tests for cross-browser and devices. Due to this, there are few issues like "This doesn't work in XYZ browser and ABC device".

Expectation

The software should be compatible with different browsers. We need to implement cross-browser tests in the program.

Proposal & Constraints

I propose to use BrowserStack for integrating these tests. There are many open source platforms which are using BrowserStack.

Stakeholders

@mekarpeles @jdlrobson @hornc

[jdlrobson]

I'm not exactly sure what's being asked for here. I have a BrowserStack account but that requires manual verification/testing.

Are you suggesting adding some kind of automation? If so what would that look like?

[feat7]

@jdlrobson Yes. I am talking about automation. There are plugins available for Browserstack's integration with Travis CI.

[jdlrobson]

Which specific plugins should we add and what do we need to configure in the codebase to work with them? Do we need to write specific tests?

[feat7]

Yes. We will need to write the tests.
If I describe it, I would say we use a Node.js test framework, mocha for instance. Write down our selenium tests. Integrate it with Travis (just need to provide access key).
And that will pretty much do the work.
I have not seen the current testing environment here in the codebase yet so if we are using some other framework for testing that will do as well.
The important point is to write tests with selenium and integrate our browserstack with Travis.

[jdlrobson]

This is a big undertaking and given the current big projects i have one (modernising javascript and standardising our UI) I can't really commit /help out as much as I like.

It would obviously be a huge improvement if we could have it. If this is something you'll willing to lead on I think the first stage would be getting a browser test setup that someone can run locally working and passing and then moving on to the travis ci portion.

[feat7]

@jdlrobson Sure. I would want to work on this. I am labeling this issue as discussion. Others can write their inputs here.

[tfmorris]

I've edited the original issue to make it more readable. Feel free to delete the boilerplate which isn't applicable and make sure that you separate your new text from any remaining quoted boilerplate (>) with a blank line or it will all end up being quoted, making it difficult to tell your text from the boilerplate.

Returning to the topic at hand, we only have a few selenium tests (openlibrary/tests/integration) and I don't think they're currently run, so they're probably broken. Integration tests are, by their nature, kind of hairy beasts and for Open Library would require a number of services to be available (database, solr, cover service, loan service, etc).

Setting up BrowserStack, integrating it with Travis, etc is straightforward once we have some tests to execute. At a glance, it looks like the current tests run again dev.openlibrary.org which is better than nothing, but not very useful for development or automation with Travis (since our dev deploys are manual). The integration tests would need to run in a Dockerized environment.

If you want to tackle this, I think resurrecting the current Selenium tests would be a good start, then we can figure out how to Dockerize them.

[brad2014]

@jdlrobson, @feat7, would it be reasonable to have a 30 minute chat to identify a plan of action to make some progress on this (e.g. an initial setup and some tests?)

[feat7]

@brad2014 I am having a tough time these days. I can't work on this for some time.

[xayhewalo]

@cdrini @hornc Do you have anything to add about Dockerizing browser compatibility tests?

[perryjr]

May I please have a copy of tests/integration/auth.yaml ? The tests won't start without it :)