Protractor: multiCapabilities doesn't invoke parallel instances of WebDriver
Bug report
Using Protractor = 6.0.0-beta, multiCapabilities execute individual captibilities in sequential order rather than running the capabilities in parallel.
- Node Version:
v10.15.1 - Protractor Version:
6.0.0-beta - Selenium Standalone:
3.141.59 - Browser(s):
Chrome 72/ Firefox 64 - Chrome Driver:
2.46 - Operating System and Version
Mac OS 10.14.2
- Your protractor configuration file
// conf.js
exports.config = {
framework: 'jasmine',
seleniumAddress: 'http://localhost:4444/wd/hub',
multiCapabilities: [{
browserName: 'chrome',
specs: ['spec.js'],
}, {
browserName: 'chrome',
specs: ['spec.js'],
}]
};
- A relevant example test
// spec.js
describe('Protractor Demo App', () => {
it('should have a title', async () => {
await browser.get('http://juliemr.github.io/protractor-demo/');
await browser.driver.sleep(5000);
expect(await browser.getTitle()).toEqual('Super Calculator');
});
});
- Output from running the test
➜ standard git:(vr-async-await) ✗ sudo ./node_modules/protractor/bin/protractor config.js
.[13:10:24] I/testLogger -
------------------------------------
[13:10:24] I/testLogger - [chrome #01] PID: 11283
[chrome #01] Specs: /Users/vr/Documents/new-async-await/kb-automation/standard/spec.js
[chrome #01]
[chrome #01] [13:10:13] I/local - Starting selenium standalone server...
[chrome #01] [13:10:14] I/local - Selenium standalone server started at http://192.168.0.11:56758/wd/hub
[chrome #01] DEPRECATION: Setting specFilter directly on Env is deprecated, please use the specFilter option in `configure`
[chrome #01] Randomized with seed 98392
[chrome #01] Started
[chrome #01] .
[chrome #01]
[chrome #01]
[chrome #01] 1 spec, 0 failures
[chrome #01] Finished in 6.914 seconds
[chrome #01] Randomized with seed 98392 (jasmine --random=true --seed=98392)
[13:10:24] I/testLogger -
.[13:10:34] I/testLogger -
------------------------------------
[13:10:34] I/testLogger - [chrome #11] PID: 11309
[chrome #11] Specs: /Users/vr/Documents/new-async-await/kb-automation/standard/spec.js
[chrome #11]
[chrome #11] [13:10:24] I/local - Starting selenium standalone server...
[chrome #11] [13:10:25] I/local - Selenium standalone server started at http://192.168.0.11:62938/wd/hub
[chrome #11] DEPRECATION: Setting specFilter directly on Env is deprecated, please use the specFilter option in `configure`
[chrome #11] Randomized with seed 38299
[chrome #11] Started
[chrome #11] .
[chrome #11]
[chrome #11]
[chrome #11] 1 spec, 0 failures
[chrome #11] Finished in 6.914 seconds
[chrome #11] Randomized with seed 38299 (jasmine --random=true --seed=38299)
[13:10:34] I/testLogger -
[13:10:34] I/launcher - 0 instance(s) of WebDriver still running
[13:10:34] I/launcher - 0 instance(s) of WebDriver still running
[13:10:34] I/launcher - Running 0 instances of WebDriver
[13:10:34] I/launcher - chrome #01 passed
[13:10:34] I/launcher - chrome #11 passed
Able to reproduce this issue only in Protector 6.0.0-beta
vsravuri
All 26 comments
@cnishina
Could you please have a look at this issue? This issue is blocking me from running automation tests using Protractor = 6.0.0-beta branch.
vsravuri
on 8 Feb 2019
@cnishina @heathkit @juliemr
Could you please have a look at this issue? I am unable to run my tests with multiCapabilities using 6.0.0-beta build
vsravuri
on 23 Feb 2019
@cnishina
Any reason why you think multiCapabilities not working in Protractor 6.0.0?
vsravuri
on 29 Mar 2019
Same here
pkearney13
on 29 Mar 2019
Facing same problem with protractor version 6.0.0
jithinkmatthew
on 29 Mar 2019
@cnishina
Any update on this issue?
vsravuri
on 30 Mar 2019
Yikes, this appears to be an issue. Maybe keep using 5.4.2 until this is resolved. I'm not sure and will talk it over with Julie.
cnishina
on 30 Mar 2019
While removing q from Protractor, I did run into this (which might be suspect): https://github.com/angular/protractor/blob/master/lib/launcher.ts#L231
My problem with this was we called a method that returned a Promise without awaiting the promise. I ended up putting an await there. This might not be it... It might be:
https://github.com/angular/protractor/blob/master/lib/launcher.ts#L250
cnishina
on 30 Mar 2019
@cnishina
Appreciate your help for looking into this issue. I will continue using 5.4.2 until this issue is resolved.
vsravuri
on 30 Mar 2019
@juliemr @cnishina
Any plan to fix this issue in the beta release you are working on?
vsravuri
on 8 Apr 2019
@cnishina @juliemr
Both multiCapabilities & shardTestFiles (https://github.com/angular/protractor/issues/5232) doesn't work with Protractor 6.0.0
Eagerly waiting for next release of Protractor to unblock in running parallel tests.
vsravuri
on 7 May 2019
Similar issue here: https://github.com/angular/protractor/issues/5232
I have not had time to look at this.
cnishina
on 7 May 2019
Any updates on this?
gawicks
on 1 Jun 2019
I think this is the same issue, my error is saying the path to the spec file for chrome is not correct, but I believe it is looking at my project structure, it's obvious. So I believe the error is wrong, but it is the same issue.
Conf file:
In my multiCapabilities I have two different browsers with the argument
**specs: ./e2e/file-presentation-flow/*.e2e-spec.ts**
and
**specs: './e2e/smoke-tests/*.e2e-spec.ts**
respectively, but only Firefox browser launches. I expect both Chrome and Firefox to run parallel (2 instances of webDriver) with different spec files respectively.
multiCapabilities: [{
browserName: 'chrome',
chromeOptions: {
args: ['--no-sandbox', '--disable-gpu', '--disable-dev-shm-usage', '--use-fake-ui-for-media-stream',
'--ignore-certificate-errors', '--allow-insecure-localhost', '--incognito', '--headless', '--start-maximized', '--user-agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"'],
prefs: {
download: {
prompt_for_download: false,
default_directory: process.env.DOWNLOAD_DIRECTORY
},
specs: './e2e/file-presentation-flow/*.e2e-spec.ts',
'profile.content_settings.exceptions.clipboard': {
'https://files.globalmeet.com,*': { last_modified: Date.now(), setting: 1 }
},
}
},
},
{
browserName: 'firefox',
'moz:firefoxOptions': {
'args': ['--safe-mode', '--headless']
},
specs: './e2e/smoke-tests/*.e2e-spec.ts'
}
],
LOGS:
```[10:30:35] I/testLogger - [chrome #01] PID: 21345
[chrome #01] /home/ntelang/PGi/convergence-client/node_modules/protractor/built/runner.js:322
[chrome #01] throw new Error('Spec patterns did not match any files.');
[chrome #01] ^
[chrome #01]
[chrome #01] Error: Spec patterns did not match any files.
[chrome #01] at Runner.run (/home/ntelang/PGi/convergence-client/node_modules/protractor/built/runner.js:322:19)
[chrome #01] at process.on (/home/ntelang/PGi/convergence-client/node_modules/protractor/built/runnerCli.js:43:20)
[chrome #01] at emitTwo (events.js:126:13)
[chrome #01] at process.emit (events.js:214:7)
[chrome #01] at emit (internal/child_process.js:772:12)
[chrome #01] at _combinedTickCallback (internal/process/next_tick.js:141:11)
[chrome #01] at process._tickCallback (internal/process/next_tick.js:180:9)
[chrome #01]
[10:30:35] I/testLogger -
[10:30:35] E/launcher - Runner process exited unexpectedly with error code: 1
[10:30:35] I/launcher - 1 instance(s) of WebDriver still running
..[10:31:08] I/testLogger -
[10:31:08] I/testLogger - [firefox #11] PID: 21350
[firefox #11] Specs: /home/ntelang/PGi/convergence-client/e2e/smoke-tests/smokeTest.e2e-spec.ts
[firefox #11]
[firefox #11] [10:30:35] I/direct - Using FirefoxDriver directly...
[firefox #11] Spec started
```
telangn
on 18 Jun 2019
When should we expect 6.0.1? It looks like the development of a protractor has stopped completely. And the latest version 6.0.0 has a typescript error. that makes it totally unuseful. Can somebody release just that fix under 6.0.1, and then take care of the rest in a scope of 6.0.2?
Or else, if there's no more development of protractor, can it be announced, so that people start to look for alternatives?
wodddy
on 7 Aug 2019
Any update on this? Has someone found any workaround in the meantime? @cnishina
Even more important: Is the project abandoned?
froblesmartin
on 11 Sep 2019
I've found workaround for parallel sessions using bluebird promises.
Bluebird allows to control count of concurrent promises.
But you should know that it was not tested across all available multiCapabilities configurations.
In my case it just fixed configuration with shardTestFiles and maxInstances attributes.
launcher.js:
"use strict";
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
return new (P || (P = Promise))(function (resolve, reject) {
function fulfilled(value) {
try {
step(generator.next(value));
} catch (e) {
reject(e);
}
}
function rejected(value) {
try {
step(generator["throw"](value));
} catch (e) {
reject(e);
}
}
function step(result) {
result.done ? resolve(result.value) : new P(function (resolve) {
resolve(result.value);
}).then(fulfilled, rejected);
}
step((generator = generator.apply(thisArg, _arguments || [])).next());
});
};
Object.defineProperty(exports, "__esModule", {value: true});
/**
* The launcher is responsible for parsing the capabilities from the
* input configuration and launching test runners.
*/
const fs = require("fs");
const configParser_1 = require("./configParser");
const exitCodes_1 = require("./exitCodes");
const logger_1 = require("./logger");
const taskRunner_1 = require("./taskRunner");
const taskScheduler_1 = require("./taskScheduler");
const util_1 = require("./util");
const promiseBluebird = require('bluebird');
let logger = new logger_1.Logger('launcher');
let RUNNERS_FAILED_EXIT_CODE = 100;
/**
* Keeps track of a list of task results. Provides method to add a new
* result, aggregate the results into a summary, count failures,
* and save results into a JSON file.
*/
class TaskResults {
constructor() {
// TODO: set a type for result
this.results_ = [];
}
add(result) {
this.results_.push(result);
}
totalSpecFailures() {
return this.results_.reduce((specFailures, result) => {
return specFailures + result.failedCount;
}, 0);
}
totalProcessFailures() {
return this.results_.reduce((processFailures, result) => {
return !result.failedCount && result.exitCode !== 0 ? processFailures + 1 : processFailures;
}, 0);
}
saveResults(filepath) {
let jsonOutput = this.results_.reduce((jsonOutput, result) => {
return jsonOutput.concat(result.specResults);
}, []);
let json = JSON.stringify(jsonOutput, null, ' ');
fs.writeFileSync(filepath, json);
}
reportSummary() {
let specFailures = this.totalSpecFailures();
let processFailures = this.totalProcessFailures();
this.results_.forEach((result) => {
let capabilities = result.capabilities;
let shortName = (capabilities.browserName) ? capabilities.browserName : '';
shortName = (capabilities.logName) ?
capabilities.logName :
(capabilities.browserName) ? capabilities.browserName : '';
shortName += (capabilities.version) ? capabilities.version : '';
shortName += (capabilities.logName && capabilities.count < 2) ? '' : ' #' + result.taskId;
if (result.failedCount) {
logger.info(shortName + ' failed ' + result.failedCount + ' test(s)');
} else if (result.exitCode !== 0) {
logger.info(shortName + ' failed with exit code: ' + result.exitCode);
} else {
logger.info(shortName + ' passed');
}
});
if (specFailures && processFailures) {
logger.info('overall: ' + specFailures + ' failed spec(s) and ' + processFailures +
' process(es) failed to complete');
} else if (specFailures) {
logger.info('overall: ' + specFailures + ' failed spec(s)');
} else if (processFailures) {
logger.info('overall: ' + processFailures + ' process(es) failed to complete');
}
}
}
let allTasksCount;
let taskResults_ = new TaskResults();
/**
* Initialize and run the tests.
* Exits with 1 on test failure, and RUNNERS_FAILED_EXIT_CODE on unexpected
* failures.
*
* @param {string=} configFile
* @param {Object=} additionalConfig
*/
let initFn = function (configFile, additionalConfig) {
return __awaiter(this, void 0, void 0, function* () {
let configParser = new configParser_1.ConfigParser();
if (configFile) {
configParser.addFileConfig(configFile);
}
if (additionalConfig) {
configParser.addConfig(additionalConfig);
}
let config = configParser.getConfig();
logger_1.Logger.set(config);
logger.debug('Running with --troubleshoot');
logger.debug('Protractor version: ' + require('../package.json').version);
logger.debug('Your base url for tests is ' + config.baseUrl);
// Run beforeLaunch
yield util_1.runFilenameOrFn_(config.configDir, config.beforeLaunch);
// 1) If getMultiCapabilities is set, resolve that as
// `multiCapabilities`.
if (config.getMultiCapabilities && typeof config.getMultiCapabilities === 'function') {
if (config.multiCapabilities.length || config.capabilities) {
logger.warn('getMultiCapabilities() will override both capabilities ' +
'and multiCapabilities');
}
// If getMultiCapabilities is defined and a function, use this.
const waitMultiConfig = yield config.getMultiCapabilities();
config.multiCapabilities = waitMultiConfig;
config.capabilities = null;
}
// 2) Set `multicapabilities` using `capabilities`,
// `multicapabilities`, or default
if (config.capabilities) {
if (config.multiCapabilities.length) {
logger.warn('You have specified both capabilities and ' +
'multiCapabilities. This will result in capabilities being ' +
'ignored');
} else {
// Use capabilities if multiCapabilities is empty.
config.multiCapabilities = [config.capabilities];
}
} else if (!config.multiCapabilities.length) {
// Default to chrome if no capabilities given
config.multiCapabilities = [{browserName: 'chrome'}];
}
// 3) If we're in `elementExplorer` mode, throw an error and exit.
if (config.elementExplorer || config.framework === 'explorer') {
const err = new Error('Deprecated: Element explorer depends on the ' +
'WebDriver control flow, and thus is no longer supported.');
logger.error(err);
process.exit(1);
}
// 4) Run tests.
let scheduler = new taskScheduler_1.TaskScheduler(config);
if (!allTasksCount){
allTasksCount = scheduler.numTasksOutstanding();
}
process.on('uncaughtException', (exc) => {
let e = (exc instanceof Error) ? exc : new Error(exc);
if (config.ignoreUncaughtExceptions) {
// This can be a sign of a bug in the test framework, that it may
// not be handling WebDriver errors properly. However, we don't
// want these errors to prevent running the tests.
logger.warn('Ignoring uncaught error ' + exc);
return;
}
logger.error(e.message);
logger.error(e.stack);
if (e instanceof exitCodes_1.ProtractorError) {
let protractorError = e;
process.exit(protractorError.code);
} else {
process.exit(1);
}
});
process.on('unhandledRejection', (reason, p) => {
if (reason.stack.match('angular testability are undefined') ||
reason.stack.match('angular is not defined')) {
logger.warn('Unhandled promise rejection error: This is usually occurs ' +
'when a browser.get call is made and a previous async call was ' +
'not awaited');
}
logger.warn(p);
});
process.on('exit', (code) => {
if (code) {
logger.error('Process exited with error code ' + code);
} else if (scheduler.numTasksOutstanding() > 0) {
logger.error('BUG: launcher exited with ' + scheduler.numTasksOutstanding() + ' tasks remaining');
process.exit(RUNNERS_FAILED_EXIT_CODE);
}
});
// Run afterlaunch and exit
const cleanUpAndExit = (exitCode) => __awaiter(this, void 0, void 0, function* () {
try {
const returned = yield util_1.runFilenameOrFn_(config.configDir, config.afterLaunch, [exitCode]);
if (typeof returned === 'number') {
process.exit(returned);
} else {
process.exit(exitCode);
}
} catch (err) {
logger.error('Error:', err);
process.exit(1);
}
});
const totalTasks = scheduler.numTasksOutstanding();
let forkProcess = false;
if (totalTasks > 1) { // Start new processes only if there are >1 tasks.
forkProcess = true;
if (config.debug) {
throw new exitCodes_1.ConfigError(logger, 'Cannot run in debug mode with multiCapabilities, count > 1, or sharding');
}
}
const createNextTaskRunner = () => __awaiter(this, void 0, void 0, function* () {
return new Promise((resolve) => __awaiter(this, void 0, void 0, function* () {
const task = scheduler.nextTask();
if (task) {
const taskRunner = new taskRunner_1.TaskRunner(configFile, additionalConfig, task, forkProcess);
try {
const result = yield taskRunner.run();
if (result.exitCode && !result.failedCount) {
logger.error('Runner process exited unexpectedly with error code: ' + result.exitCode);
}
taskResults_.add(result);
task.done();
// yield createNextTaskRunner();
// If all tasks are finished
// if (scheduler.numTasksOutstanding() === 0) {
resolve();
// }
logger.info(scheduler.countActiveTasks() + ' instance(s) of WebDriver still running');
} catch (err) {
const errorCode = exitCodes_1.ErrorHandler.parseError(err);
logger.error('Error:', err.stack || err.message || err);
yield cleanUpAndExit(errorCode ? errorCode : RUNNERS_FAILED_EXIT_CODE);
}
} else {
resolve();
}
}));
});
const taskList = Array(allTasksCount).fill(() => createNextTaskRunner());
const maxConcurrentTasks = scheduler.maxConcurrentTasks();
yield promiseBluebird.map(taskList, task => task(), {concurrency: maxConcurrentTasks});
// for (let i = 0; i < maxConcurrentTasks; ++i) {
// yield createNextTaskRunner();
// }
logger.info('Running ' + scheduler.countActiveTasks() + ' instances of WebDriver');
// By now all runners have completed.
// Save results if desired
if (config.resultJsonOutputFile) {
taskResults_.saveResults(config.resultJsonOutputFile);
}
taskResults_.reportSummary();
let exitCode = 0;
if (taskResults_.totalProcessFailures() > 0) {
exitCode = RUNNERS_FAILED_EXIT_CODE;
} else if (taskResults_.totalSpecFailures() > 0) {
exitCode = 1;
}
yield cleanUpAndExit(exitCode);
// Start `const maxConcurrentTasks` workers for handling tasks in
// the beginning. As a worker finishes a task, it will pick up the next
// task from the scheduler's queue until all tasks are gone.
});
};
exports.init = initFn;
//# sourceMappingURL=launcher.js.map
abruslik
on 25 Sep 2019
@cnishina Keen to know where things stand with Protractor and future of version 6. Not seen any updates for few months?
Phonesis
on 14 Jan 2020
I was trying to contact some developers of this project and Angular 4 months ago and I had no reply. It really seems abandoned.
froblesmartin
on 14 Jan 2020
Yes, this is important. Please announce if protractor development is being continued or we need to look into alternatives?
Kungfoowiz
on 15 Jan 2020
Start using Cypress
telangn
on 15 Jan 2020
I don't see Cypress as a viable alternative atm. It only supports Chrome and doesn't work with BrowserStack. Also, it requires a paid license.
Phonesis
on 15 Jan 2020
@heathkit @cnishina @juliemr
Appreciate if you could provide some insight into the future of Protractor. Its' been more than 6 months since we have any update on Protractor 6.0. If the plan is to abandon protractor, kindly let us know so that we have enough room to look for the alternatives.
vsravuri
on 15 Jan 2020
While removing q from Protractor, I did run into this (which might be suspect): https://github.com/angular/protractor/blob/master/lib/launcher.ts#L231
My problem with this was we called a method that returned a Promise without awaiting the promise. I ended up putting an await there. This might not be it... It might be:
https://github.com/angular/protractor/blob/master/lib/launcher.ts#L250
here you wait for every runner, sequentially in the loop.
now:
for (let i = 0; i < maxConcurrentTasks; ++i) {
await createNextTaskRunner();
}
should be:
const runners = [];
for (let i = 0; i < maxConcurrentTasks; ++i) {
runners.push(createNextTaskRunner());
}
await Promise.all(runners); // wait for all, running simultaneously
and Promise((resolve) => ...) (https://github.com/angular/protractor/blob/master/lib/launcher.ts#L219)
should be just async function, so remove the line. Otherwise, no resolve() happens at the end of the method with outstanding tasks https://github.com/angular/protractor/blob/master/lib/launcher.ts#L235
yadimon
on 4 Mar 2020
@heathkit @cnishina @juliemr no longer work on Protractor, we can't speak to future plans. We continue to use Protractor internally, and there's a lot of other projects at Google that depend on it.
The Angular team would like to invest more in Protractor, and will have time now that they've landed Ivy. @IgorMinar will be better able to answer questions about the future of Protractor.
heathkit
on 4 Mar 2020
Is someone fixing this ? I need it badly. Our tests serially takes more than an hour. Please help.
rajjaiswalsaumya
on 27 Aug 2020
Related issues
luakri
·
3Comments
andyman3693
·
3Comments
jmcollin78
·
3Comments
davidkarlsen
·
3Comments
codef0rmer
·
3Comments
Most helpful comment
Any update on this? Has someone found any workaround in the meantime? @cnishina
Even more important: Is the project abandoned?