Protractor: Calling `pending` in a spec marks the spec as failed.

[sveneh]

See the attached testcase

https://gist.github.com/sveneh/9c589d321e6fc31f7460

When I run this in [email protected] using [email protected] I get this:

>protractor pending.conf.js
Starting selenium standalone server...
[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://192.168.88.100:60036/wd/hub
Started
F*

Failures:
1) a spec with pending should be pending
  Message:
    Failed: => marked PendingThis is pending.
  Stack:
    Error: Failed: => marked PendingThis is pending.
Pending:

1) a spec with pending should be also pending
  No reason given

2 specs, 1 failure, 1 pending spec
Finished in 0.006 seconds
Shutting down selenium standalone server.
[launcher] 0 instance(s) of WebDriver still running
[launcher] chrome #1 failed 1 test(s)
[launcher] overall: 1 failed spec(s)
[launcher] Process exited with error code 1

running the same spec in pure [email protected]:

>jasmine 
Started
**

Pending:

1) a spec with pending should be pending
  This is pending.


2) a spec with pending should be also pending
  No reason given

2 specs, 0 failures, 2 pending specs
Finished in 0.002 seconds

Protractor should not fail specs that are marked as pending.

[cstraynor]

I'm experiencing this too. It would be good to get resolved so I can have accurate reports to delineate tests actually failing and tests being skipped.

[theandrewlane]

Someone please fix this!! In the meantime, to mark as pending without using pending("reason") use xit('some spec description', function (){

[lynchblue]

+1 That's a pretty cool jasmine2 feature and it's a pity that we cannot use it with protractor. Is anyone taking care of this?

[ctapobep]

+1

[juliemr]

See https://github.com/angular/jasminewd/issues/27#issuecomment-144882283

[bruderol]

+1 this is really anoying! I want to be able to put tests to pending with correct messages in log.

[bruderol]

As a workaround I introduced an own pending to use in my protractor tests (overwriting the jasmine pending), maybe that is helpful for somebody else as a workaround:

            // in my protractor onPrepare code:
            var _pending = pending;
            pending = function(pendingMessage) {

                return {

                    it: function(description, itFunction) {
                        var consoleColorYellowStart = '\u001B[33m';
                        var consoleColorYellowEnd = '\u001B[0m';
                        return xit(description + '\n  ' + consoleColorYellowStart + 'Marked as Pending: ' + pendingMessage + consoleColorYellowEnd, itFunction);
                    },

                    describe: function(description, describeFunction) {
                        return describe(description, function() {
                            _pending(pendingMessage);
                            return describeFunction();
                        });
                    },

                    here: function() {
                        return _pending(pendingMessage);
                    }

                };

            };

Usage:
pending('my pending comment').describe('my describe block', function() { ... });
or
pending('my pending comment').it('my it block', function() { ... });
or (just in case you still need the old pure jasmine pending implementation somewhere else, with all advantages and disadvantages)
pending('my pending comment').here();

[ask1987]

+1

[lgarest]

Still experiencing this, meanwhile the workaround works like a charm @bruderol thanks!

[moneytree-doug]

Any updates on this issue?

[mattc41190]

Also feeling some burn here +1

[rcbop]

+1 also happening here. :(

[d-werner]

+1

[ArturKwiatkowski]

+1

[bruderol]

In the meantime I found out that there is a possibility to set comments on tests ignored with xit and therefore even improved my workaround a little bit:

            // in my protractor onPrepare code:
            var _pending = pending;
            pending = function(pendingMessage) {

                return {

                    it: function(description, itFunction) {
                         var spec = xit(desription, itFunction);
                         spec.pend(pendingMessage);
                         return spec;                        
                    },

                    describe: function(description, describeFunction) {
                        return describe(description, function() {
                            _pending(pendingMessage);
                            return describeFunction();
                        });
                    },

                    here: function() {
                        return _pending(pendingMessage);
                    }

                };

            };

Usage:
pending('my pending comment').describe('my describe block', function() { ... });
or
pending('my pending comment').it('my it block', function() { ... });
or (just in case you still need the old pure jasmine pending implementation somewhere else, with all advantages and disadvantages)
pending('my pending comment').here();

[stheikki]

also experiencing this, is this issue currently being worked on?

[dverbovyi]

+1. Also faced with this issue

[HugoDoyon]

+1 Should be priorize, this is a very usefull feature.

[juliemr]

Closing in favor of https://github.com/angular/jasminewd/issues/32

Note that this is pending a change to Jasmine.

[sgibson21]

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

[andregs]

This hack works for me while we don't have https://github.com/angular/jasminewd/issues/32

// In protractor `onPrepare`
jasmine.Suite.prototype.pend = function (message) {
  this.markedPending = true;
  this.children.forEach(spec => spec.pend(message));
};

xdescribe('Foo', function(){
  // bar
}).pend('reason');

[Sankannavar]

Am new to protractor
Was trying to skip some testcases on failure and continue remaining during batch run
I tried using pending but its not working inside it block
ex: it('Step 1:', function () {
});
it('Step 1:', function () {
if(true)
{
pending('my pending comment').it('my it block', function() { });
}
}); here am getting as failed
but same working outside of it block

it('Step 1:', function () {

});

pending('my pending comment').it('my it block', function() {  });

it('Step 2:', function () {

});

Please some one help me. thanks in advance.

[Kitsula]

Short workaround:
(environment.postDeployment ? xit : it)('should something', () => { expect(true).toBeTruthy(); });

[aschlei]

This issue should be fixed in jasmine 2.9 (See https://github.com/jasmine/jasmine/commit/f4caf27). However, protractor is still using jasmine 2.8.

[ViieeS]

I have the latest version of Jasmine v3.3.0 and still get this behaviour... 😕

[tymfear]

I have the latest version of Jasmine v3.3.0 and still get this behaviour... 😕

If you don't use Protractor 6.x, then, whichever version of Jasmine you got installed, you will use the version used by jasminewd.

[ViieeS]

@tymfear oh, sorry don't test it with protractor. It's about unit tests.

[nitin-kapoor]

I am using pending function like below but its marking as failed
Even the spec report treating pending function as failed.

[nitin-kapoor]

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

I have used this implementation and it's awesome!
But the issue is that even if we are using a pending function in the spec then jasmine is marking its status as failed only. I got to know while debugging(screenshot below) However I can add a little hack in this by adding checks for specific test cases by checking its description or id but this can cause issues later.

Can you please suggest something.

Test case shich is marked as pending

[Garcia-JuanLuis]

This issue should be fixed in jasmine 2.9 (See jasmine/jasmine@f4caf27). However, protractor is still using jasmine 2.8.

Hi, thank you for your support. I've updated to Protractor 7.0.0 and Jasmine 3.6.3.
When I use pending('Force skip') ==> I get an error that says "Failed: undefinedForce skip".
NOTE: I also see a message that says "no tests were found" which is pretty interesting.

[Garcia-JuanLuis]

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

I have used this implementation and it's awesome!
But the issue is that even if we are using a pending function in the spec then jasmine is marking its status as failed only. I got to know while debugging(screenshot below) However I can add a little hack in this by adding checks for specific test cases by checking its description or id but this can cause issues later.

Can you please suggest something.

Test case shich is marked as pending

I extended the jasmine-spec-reporter (https://www.npmjs.com/package/jasmine-spec-reporter)

var SpecReporter = require('jasmine-spec-reporter');
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
    if ( spec.failedExpectations[i].message.toLowerCase().indexOf('pending') >= 0) pending = true;
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);
  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

Specs marked as pending are displayed as pending in the report.

This does not work for me; however, it could be the way I'm implementing it.
Would you place this code in its own class and then import it as a reporter?

I put the code under the Protractor config file, "onPrepare()" function... and I am getting two "results" in one window.

Failures:
1) Testing Skip Test shoud be skipped
Message:
Failed: => marked PendingSkipping This test
Stack:
Error: Failed: => marked PendingSkipping This test
at C:\regUSA\Utilities\ProtractorTestnode_modulesjasminewd2\index.js:64:48
at ControlFlow.emit (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\events.js:62:21)
at ControlFlow.shutdown_ (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:2674:10)
at shutdownTask_.MicroTask (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:2599:53)

1 spec, 1 failure
Finished in 0.012 seconds

---

• Pending *

---

1) Testing Skip Test shoud be skipped
No reason given

Executed 0 of 1 spec INCOMPLETE (1 PENDING) in 0.012 sec.
[15:17:09] I/local - Shutting down selenium standalone server.
[15:17:09] I/launcher - 0 instance(s) of WebDriver still running
[15:17:09] I/launcher - chrome #01 failed 1 test(s)
[15:17:09] I/launcher - overall: 1 failed spec(s)
[15:17:09] E/launcher - Process exited with error code 1

[Garcia-JuanLuis]

I took the implementation provided by sgibson21 and created a new project with Jasmine 3.6.3 and Protractor 7.0.0.
Still, I was getting failures when using pending(); so, I went ahead and started to play around with the implementation. Following is what my confi.js file looks like:
`// conf.js
let Jasmine2HtmlReporter = require('protractor-jasmine2-html-reporter');
var SpecReporter = require('jasmine-spec-reporter').SpecReporter;

let htmlReporter = new Jasmine2HtmlReporter({
savePath: 'reports',
fileName: 'e2e-report',
takeScreenshotsOnlyOnFailures: true,
cleanDestination: false,
fileNameDateSuffix: true
});

exports.config = {
framework: 'jasmine',
specs: ['spec.js'],

onPrepare() {
var PendingSpecReporter = SpecReporter;

PendingSpecReporter.prototype.specDone = function (spec) {
  this.metrics.stopSpec(spec);
  var pending = false;
  var pendingReason;
  for(var i = 0; i < spec.failedExpectations.length; i++) {
      var index = spec.failedExpectations[i].message.toLowerCase().indexOf('pending');
    if ( index >= 0){
        pendingReason = spec.failedExpectations[i].message.substring(index+7); // 7 = the length of the word "pending"
        pending = true;
    }
  }
  if (spec.status === 'pending' || pending) {
    this.metrics.pendingSpecs++;
    this.display.pending(spec);

    spec['failedExpectations'] = [];
    spec['passedExpectations'] = [];
    spec['status'] = 'pending';

    if(spec['pendingReason'] === 'undefined' || spec['pendingReason'] === ''){
        spec['pendingReason'] = pendingReason;
    }

  } else if (spec.status === 'passed') {
    this.metrics.successfulSpecs++;
    this.display.successful(spec);
  } else if (spec.status === 'failed') {
    this.metrics.failedSpecs++;
    this.display.failed(spec);
  }
};

module.exports = PendingSpecReporter;

jasmine.getEnv().addReporter(new PendingSpecReporter({ spec: { displayStacktrace: 'pretty' } }));
    // Assign the test reporter to each running instance
jasmine.getEnv().addReporter(htmlReporter);

let jasmineReporter = require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmineReporter.JUnitXmlReporter({
  consolidateAll: true,
  savePath: 'reports',
  filePrefix: 'e2e-report',
}));

}
}`

Following is what my specs look like
`// spec.js
describe('Testing custom results', function() {
xit('shoud be skipped because we are using "x" in front of the spec', function() {
pending('Force skip');
expect(true).toBe(true);
});

it('shoud be skipped using custom implementation', function() {  
  expect(true).toBe(true);
  pending('My custom pending reason!');

});

it('shoud pass!', function() {
expect(true).toBe(true);
});

it('shoud fail!', function() {
expect(true).toBe(false);
});

});`

Following is what the XML report looks like
`

-

-

-

-

-

-

- at UserContext. (C:\regUSA\Utilities\ProtractorTest\spec.js:18:17)
at C:\regUSA\Utilities\ProtractorTestnode_modulesjasminewd2\index.js:112:25
at new ManagedPromise (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:1077:7)
at ControlFlow.promise (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:2505:12)
at schedulerExecute (C:\regUSA\Utilities\ProtractorTestnode_modulesjasminewd2\index.js:95:18)
at TaskQueue.execute_ (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:3084:14)
at TaskQueue.executeNext_ (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:3067:27)
at asyncRun (C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:2974:25)
at C:\regUSA\Utilities\ProtractorTestnode_modules\selenium-webdriverlib\promise.js:668:7]]>

`

Following is what the HTML report looks like
``