protractor 5.1.1 is unable to run test cases in sauce lab but working fine with 5.0.0 / 4 version perfectly

[aniruddhadas9]

Bug report

• Node Version: 6.9.1
• Protractor Version: 5.1.1
• Angular Version: 1.6.0
• Browser(s): saucelab chrome
• Operating System and Version OS X EI captain 10.11.6
• Your protractor configuration file find below
• A relevant example test: I was trying to test my test cases in sauce lab after upgrading it to 5.1.1 it did not work, but when I downgrade it to 4/5.0.0 it worked fine. It did not show any error but did not proceed with the tunnel created
• Output from running the test: attached below
• Steps to reproduce the bug change the protractor version and will not run
• The URL you are running your tests against (if relevant) not relevant

configuration ifle:

exports.config = {
  sauceUser: '<userid>',
  sauceKey: '<key>',
  sauceSeleniumAddress: 'localhost:4445/wd/hub',
  //webDriverProxy: '<proxyurl-commented nto required>',
  /*capabilities: {
   'browserName': 'chrome',
   'tunnel-identifier': 'shepherd',
   'build': 'build-12345',
   'name': 'Shepherd DEV functional',
   }*/

  multiCapabilities: [
    {
      'browserName': 'chrome',
      'tunnel-identifier': 'shepherd',
      'build': 'build-12345',
      'name': 'Shepherd DEV functional',
      'chromeOptions': {
        'args': ['--no-sandbox', '--test-type=browser'],
        'prefs': {
          'download': {
            'prompt_for_download': false,
            'default_directory': '/tmp'
          }
        }
      }
      // 'browserName': 'phantomjs',
      // 'phantomjs.binary.path': require('phantomjs-prebuilt').path,
      // 'phantomjs.ghostdriver.cli.args': ['--loglevel=DEBUG'],
      // 'phantomjs.page.settings.userName': 'tst_dm_adm',
      // 'phantomjs.page.settings.password': 'CAlsGSV#/HNyAX'
    }
  ],
  allScriptsTimeout: 50000,
  seleniumArgs: ['--ignore_ssl'],
  suites: {
    home: 'home/*.spec.js',
    ...
  },
  framework: 'jasmine2',
  jasmineNodeOpts: {
    // If true, print colors to the terminal.
    showColors: true,
    // Default time to wait in ms before a test fails.
    defaultTimeoutInterval: 50000,
    //remove protractor's default dot reporter
    print: function () {
    }
  },
  onPrepare: function () {
    var SpecReporter = require('jasmine-spec-reporter');
    // add jasmine spec reporter
    jasmine.getEnv().addReporter(new SpecReporter({
      displayStacktrace: 'all',
      prefixes: {success: '* ', failure: 'X ', pending: '* '}
    }));

    var jasmineReporters = require('jasmine-reporters');
    jasmine.getEnv().addReporter(
      new jasmineReporters.JUnitXmlReporter(null, true, true, 'testOutput')
    );
  }
};

* below is the console output after running protractor -c config.js*

C429349HHY3QD:shepherd-test RF886$ ./run-in-saucelab.sh 
Installing..
npm WARN [email protected] No description
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.
Updating webdriver..
[10:50:54] I/file_manager - creating folder /Users/RF886/aniruddhadas9/develop/herd/jan30/shepherd-test/node_modules/protractor/node_modules/webdriver-manager/selenium
[10:50:55] I/downloader - curl -o /Users/RF886/aniruddhadas9/develop/herd/jan30/shepherd-test/node_modules/protractor/node_modules/webdriver-manager/selenium/geckodriver-v0.15.0.tar.gz https://github.com/mozilla/geckodriver/releases/download/v0.15.0/geckodriver-v0.15.0-macos.tar.gz
[10:50:55] I/update - geckodriver: unzipping geckodriver-v0.15.0.tar.gz
[10:50:55] I/update - geckodriver: setting permissions to 0755 for /Users/RF886/aniruddhadas9/develop/herd/jan30/shepherd-test/node_modules/protractor/node_modules/webdriver-manager/selenium/geckodriver-v0.15.0
[10:50:56] I/downloader - curl -o /Users/RF886/aniruddhadas9/develop/herd/jan30/shepherd-test/node_modules/protractor/node_modules/webdriver-manager/selenium/chromedriver_2.28.zip https://chromedriver.storage.googleapis.com/2.28/chromedriver_mac64.zip
[10:50:56] I/downloader - curl -o /Users/RF886/aniruddhadas9/develop/herd/jan30/shepherd-test/node_modules/protractor/node_modules/webdriver-manager/selenium/selenium-server-standalone-3.3.1.jar https://selenium-release.storage.googleapis.com/3.3/selenium-server-standalone-3.3.1.jar
[10:50:58] I/update - chromedriver: unzipping chromedriver_2.28.zip
[10:50:58] I/update - chromedriver: setting permissions to 0755 for /Users/RF886/aniruddhadas9/develop/herd/jan30/shepherd-test/node_modules/protractor/node_modules/webdriver-manager/selenium/chromedriver_2.28
Running tests..
[10:51:00] I/launcher - Running 1 instances of WebDriver
[10:51:00] I/sauce - Using SauceLabs selenium server at https://localhost:4445/wd/hub

[NickTomlin]

Unfortunately there's not much that we can do with that information. Our own tests run on sauce; are you able to test using the example using your sauce creds and the latest version of protractor?

[aniruddhadas9]

I am able to do the test with 4 or 5.0.0 but when I upgrade to 5.1.1 it doesn't show any error but also it does not proceed. I don't see any test cases running sauce lab site.

I will spend some more time to debug and give you more information if I can.

[kieranbop]

Do we have any update on this?
I have been seeing the following since I updated:
Error: EPROTO write EPROTO 140735277850624:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:794:

[ashubham]

@NickTomlin I am facing the same issue as the OP.

@aniruddhadas9 Were you able to figure a workaround ?

[aniruddhadas9]

@ashubham @NickTomlin I just degraded to 5.0.0/4 as of now. that is the work-around as of now ;)
I think the protractor team is not able to track that with the information I am provided. please add as much information you can so that they can track it.

Meantime, I am working to give them more information.

[ashubham]

I tried with the configuration in the example:

exports.config = {
  sauceUser: process.env.SAUCE_USER,
  sauceKey: process.env.SAUCE_KEY,
  sauceSeleniumAddress: process.env.SAUCE_SELENIUM_ADDRESS,
  // Capabilities to be passed to the webdriver instance.
  capabilities: {
    'browserName': 'chrome'
  },

  // Framework to use. Jasmine is recommended.
  framework: 'jasmine',

  // Spec patterns are relative to the current working directory when
  // protractor is called.
  specs: ['example_spec.js'],

  // Options to be passed to Jasmine.
  jasmineNodeOpts: {
    defaultTimeoutInterval: 30000
  }
};

I put my sauce opts as the env variables.

./bin/protractor ./conf.js --capabilities.tunnelIdentifier=tunnelId --capabilities.seleniumVersion="3.1.1"

LMK what exactly you might need @NickTomlin

[wswebcreation]

@ashubham and @aniruddhadas9

Strange that the tests are failing for you both. I have the following setup and it works:

• protractor: 5.1.1
• Node: 6.9.2
• OSX 10.12.3
• Sauce Connect 4.4.3 (higher version doesn't pull up the connection)

Config:

'use strict';
const path = require('path');
const config = require('./protractor.shared.conf').config;

const SAUCE_USERNAME = '<username>';
const SAUCE_ACCESS_KEY = '<key>';

config.baseUrl = `http://localhost:5555`;

config.seleniumAddress = 'http://localhost:4445/wd/hub';

config.multiCapabilities = [
    {
        browserName: 'chrome',
        platform: "Windows 10",
        version: "latest",
        screenResolution: "1400x1050",
        name: '<Name>',
        username: SAUCE_USERNAME,
        accessKey: SAUCE_ACCESS_KEY,
        'tunnel-identifier': '<tunnel-identifier>',
        shardTestFiles: true
    }
];
exports.config = config;

Biggest difference in the configurations is that:

• I don't use sauceUser: '<userid>'
• I don't use sauceKey: '<key>'
• I don't use sauceSeleniumAddress: 'localhost:4445/wd/hub'

Can you check with my configuration if it works? If so we might have an issue with the default connection that we should use of Protractor for Sauce Labs

Grtz

Wim

[aniruddhadas9]

@wswebcreation sure. Let me try these configs

[DOrlov77]

The bug root is the recent fix, see https://github.com/angular/protractor/commit/3edd62eccccb67ec7cb71b8c3d8b2c2921a6f7ca#commitcomment-22055547
The workaround is to use username/accessKey/tunnel-identifier per capability indeed.

[aniruddhadas9]

The way @wswebcreation mentioned worked for me now. Actually the protractor version 4 and 5 takes configuration in different ways.

if you are using protractor 4 then configure your sauce lab the way I did and if you are using protractor 5 or higher then use protractor config the way @wswebcreation in his post

[kenisteward]

@aniruddhadas9 this should be closed since you have your answer.

[wswebcreation]

@kenisteward

You are right, it works with this "workaround". I do think there is an issue, I need to dive into it. The issue could also be that the documentation isn't correct. That's why I think it's better to keep this open

[wswebcreation]

After some debugging I put myself on the wrong track, see the linked issue. The solution it pretty simple.

Cause:
Between Protractor 4.0.14 and Protractor 5.0.0 there has been a change in the driverProviders. There is now a condition here that will NOT go to the Sauce Labs runner if you provide a custom seleniumAddress.

Solution:
You need to change your seleniumAddress-property to sauceSeleniumAddress, then it will work. So don't keep the seleniumAddress in your config file!
Protractor will take care of all the connection between your tests and Sauce Labs, even update the instances in Sauce Labs with the correct status.

I'm going to close the issue for now. Feel free to add more questions in this issue.

[kenisteward]

@wswebcreation I'm trying this out tonight. i hope it works.

In other news you guys take PR"s right? I see in Sauce.ts it does not print out the session id so that if you are running in say jenkins / travis ci that it would asssociate the build to your job.

See here
https://wiki.saucelabs.com/display/DOCS/Setting+Up+Reporting+between+Sauce+Labs+and+Jenkins

I currently do this onComplete()
console.log("SauceOnDemandSessionId=id job-name=browser-job-name

[wswebcreation]

@kenisteward

Hope it works to. Last thing you are right about. I now do this manually, printing doesn't do the trick, you need to update it yourself.

A PR would be welcome

[kenisteward]

@wswebcreation unfortunately it did not work out.

As it turns out the webDriverProxy setting is used not only in Sauce.ts but it is also used in the driverprovider to initialize the driver.

Because of this when i set that i'm able to get out of my proxy to send saucelabs requests but i'm unable to connect to my localhosted (in jenkins) selenium relayed driver.

gives me an access denied error because it looks like the password is not being appended to the username before the @ on the driver url. I have no clue why the proxy woudl be taking that out.

[wswebcreation]

@kenisteward ,

Can you place your config here, so we can check it?

[kenisteward]

const conf = require('./e2e/config/conf.e2e.json');
const { SpecReporter } = require('jasmine-spec-reporter');
const jasmineReporters = require('jasmine-reporters');

const shartTestFiles = false;
const maxInstances = 1;

exports.config = {
  SELENIUM_PROMISE_MANAGER: 0,
  sauceBuild: process.env.JENKINS_BUILD_NUMBER || 'localTestBuildNumber__00',
  sauceUser: process.env.SAUCE_USERNAME || 'test',
  sauceKey: process.env.SAUCE_ACCESS_KEY || 'test',
  sauceSeleniumUseHttp: true,
  sauceSeleniumAddress:  'localhost:4445/wd/hub', // using this with jenkins and Selenium relay 443 is not open for my env.
  webDriverProxy: http://my.proxy.org:3122,
  multiCapabilities: [
    {
      count: 1,
      "name": "Windows 7 chrome 57",
      "tunnel-identifier": "shepherd-jenkins",
      "platform": "Windows 7",
      "browserName": "chrome",
      chromeOptions: {
        prefs: {
          'credentials_enable_service': false,
          'profile': {
            'password_manager_enabled': false
          }
        },
        args: [
          '--disable-cache',
          '--disable-application-cache',
          '--disable-offline-load-stale-cache',
          '--disk-cache-size=0',
          '--v8-cache-options=off'
        ]
      },
      "version": "57",
      shartTestFiles
    }
  ],
  allScriptsTimeout: 11000,
  seleniumArgs: ['--ignore_ssl'],
  specs: [
    './e2e/app/data-objects/**/*.e2e-spec.ts',
  ],
  /*suites: [
    './e2e/!**!/!*.e2e-spec.ts'
  ],*/
  // directConnect: true,
  baseUrl: conf.baseUrl8443,
  framework: 'jasmine2',
  jasmineNodeOpts: {
    showColors: true,
    defaultTimeoutInterval: 30000,
    print: function () {
    }
  },

  onPrepare() {
    let capabilities
    require('ts-node').register({
      project: 'e2e/tsconfig.e2e.json'
    });


    jasmine.getEnv().addReporter(new SpecReporter({
      spec: { displayStacktrace: true }
    }));

    return browser.getProcessedConfig().then((c) => {
      capabilities = c.capabilities;
      // used to change the namespace for the data.
      process.env.CURRENT_BROWSER = c.capabilities.browserName.replace(new RegExp(' ', 'g'), '-');
      jasmine.getEnv().addReporter(
        new jasmineReporters.JUnitXmlReporter({
          filePrefix: c.capabilities.browserName,
          consolidateAll: true,
          modifySuiteName: function (generatedSuiteName, suite) {
            // this will produce distinct suite names for each capability,
            // e.g. 'firefox.login tests' and 'chrome.login tests'
            return c.capabilities.browserName + '.' + generatedSuiteName;
          }
        }));

      return browser.driver.get(c.baseUrl).then(function () {
        browser.sleep(2000);
      });
    });
  },

  onComplete() {
    return browser.getProcessedConfig().then(function (c) {
      return browser.getSession().then(function (session) {
        // required to be here so saucelabs picks up reports to put in jenkins
        console.log('SauceOnDemandSessionID=' + session.getId() + ' job-name=' + c.capabilities.name);
      });
    });
  }
};

also having trouble just getting 1 instance of a browser to run in sauce labs
the issue being for some reason It takes the accesskey out of the selenium address

[wswebcreation]

@kenisteward

I got a few questions, can you please answer them:

1. did it work with the workaround?
2. is the webDriverProxy a new problem?
3. Have you also looked at this

Tnx in advance

[kenisteward]

1. what work around?
2. webDriverProxy is the issue in my use case solely because i don't need to use the proxy to connect to my driver instance but i do need it for the sauceLabs call. I'd prefer there just be 2 different settings like it was before.
3. I saw it but i don't know how it can help me

[wswebcreation]

Hi @kenisteward

The workaround I referred to is this, but I understand your problem is the webDriverProxy. Looks like your issue is a different issue then the initial issue of @aniruddhadas9 . Will check it the coming days

For the sauceAgent, I thought this will give you an extra solution for the proxy problem based on what I read from the docs. How I read it is that this should not interfere with your Jenkins setup

[sighrobot]

FWIW I used the config above by @wswebcreation as my template and it worked with Protractor 5.3.2.

The important things for me were the _de-Saucing_, if you will, of the three options listed specifically by @wswebcreation.

Another thing to note is that the newly generically named options must be included in _each_ browser config object, as opposed to do the top level. For me, it looks something like this:

const COMMON = {
    platform: 'Windows 7',
    username: process.env.USERNAME,
    accessKey: process.env.ACCESS_KEY,
    build: process.env.BUILD,
    name: process.env.COMMIT + ' (' + process.env.BRANCH + ')',
    'tunnel-identifier': process.env.TUNNEL_ID,
};

exports.config = {
    multiCapabilities: [
        Object.assign(COMMON, {
            browserName: 'chrome',
        }),
        Object.assign(COMMON, {
            browserName: 'firefox',
            version: '58.0',
        }),
    ],

    ...
};

Finally, at first I was getting back this error when running through SauceLabs remotely:

Error: Invalid URL: sauce_a1b2c3d:4445/wd/hub

After I prepended my seleniumAddress with http://, it worked perfectly.

Thanks you @wswebcreation!