Nightwatch: Support running environments sequentially

[openjck]

This has been discussed before (#138) but I'd like to formally request the feature.

Nightwatch can run tests in multiple environments simultaneously. For example, the following command will run all tests in Firefox, Chrome, and Safari at the same time.

nightwatch --env firefox,chrome,safari

There are times when running tests sequentially is preferable: first in Firefox, then in Chrome, then in Safari. For example, the most basic BrowserStack Automate plan only allows tests to be run in one browser at a time. I'd like to request that Nightwatch support this with a --sequential flag (or similar).

[beatfactor]

Yes, but you can already do this by creating a small script which does something like nightwatch --env firefox && nightwatch --env chrome.

[openjck]

Good point. We do something like that in ensemble.

One downside is that this approach can be pretty verbose and repetitive, especially if the --config option needs to be passed.

For example, it ends up being:

nightwatch --config nightwatch.browserstack.conf.js --env ie && nightwatch --config nightwatch.browserstack.conf.js --env edge && nightwatch --config nightwatch.browserstack.conf.js --env firefoxWin7 && nightwatch --config nightwatch.browserstack.conf.js --env firefoxSierra && nightwatch --config nightwatch.browserstack.conf.js --env chromeWin7 && nightwatch --config nightwatch.browserstack.conf.js --env chromeSierra

whereas something like this would be nice:

nightwatch --sequential --config nightwatch.browserstack.conf.js --env ie,edge,firefoxWin7,firefoxSierr,chromeWin7,chromeSierra

That can still be scripted, and that's what we do in ensemble, but a --sequential flag would be convenient.

[rikbrowning]

Would be nice to have because then the end user would get a nice summary of results at the end, compared to however many summaries displayed throughout the execution.

[didaquis]

+1 to this funcionality!

[openjck]

I forgot to mention this earlier, but && doesn't work on Windows. That's why projects like npm-run-all exist.

Even then, npm-run-all requires one script for each command that should be run sequentially, so you end up needing to provide one script for each invocation of the nightwatch command, which gets really verbose. We have this problem in mozilla/ensemble.

[davidpelayo]

+1 To get this!