Nightwatch: Test steps call order undocumented

[mightyiam]

The order in which test steps will run is not clearly documented (if at all).

If this is a test file:

module.exports = {
  'step one': function (browser) {
    browser
      // do stuff
  },
  'step two': function (browser) {
    browser
      // do stuff
      .end()
  }
}

Then I do not understand what determines the order in which these test steps will be called.

Is it alphabetical?

BTW I'm experiencing an issue where only my first step gets called.

[beatfactor]

I'm not seeing a clear issue or a request here. You should clearly report a bug or request a change in behaviour.

[mightyiam]

@beatfactor OK the issue is not a question.

The issue is that documentation does not explain how the order at which the test steps get called is determined.

[mightyiam]

What determines that step one is called before step two and not after it?

Perhaps it is clear to some but not to me. And I do not see any mention of this in the documentation.

[beatfactor]

I don't think it has to. The example suggests the execution order is top to bottom.

[mightyiam]

I think it does. Because that assumption is too implicit.

For example, the most popular answer to this question could be considered good advice. And so I would assume that a popular node module would not rely on the order at which the properties of an object were specified. Does it?

[mightyiam]

This is a more popular question.

And I'm not sure that for...in is used for this, but for that matter, MDN states that the order is arbitrary.

[mightyiam]

Apparently, there is much controversy on this.

See the V8 issue.

I still think that documentation should mention it, to prevent any kind of confusion.

[beatfactor]

Maybe a better practice would be to write independent test cases, i.e. to not rely on previous test cases and so I think that example should be removed from the documentation.

[mightyiam]

Really? I'm surprised.

So, deprecate test steps, you mean?

[beatfactor]

There's nothing to deprecate, this is not a feature built by nightwatch it's how javascript works. The order will always be the same as what node.js and V8 do by default with object keys and I think that's pretty clear, unless otherwise stated.

[mightyiam]

It may be clear to anyone who knows what node.js and V8 do by default with object keys. But I didn't. And I kind of shouldn't be expected to, since that behavior is not ECMAScript spec, but an implementation artifact.

I won't tell you how to write code. I just would like the next guy to not wonder about this as I have.

[beatfactor]

Well, it is an open-source project so you are free to "tell me how to write code" by sending in a pull request if you feel that something should be changed.

[sknopf]

I like the idea of removing the steps all together from the documentation. As @beatfactor said it's commonly considered good practice to write independent tests but it also complicates things like retry logic if Nightwatch needs to know whether more test case steps than just the failing test need to be rerun upon retry. Seems like given that and some controversy over how stable order in Object.keys currently is it's better to not encourage new Nightwatch users to use it.

On May 15, 2015, at 6:47 AM, Shahar Or [email protected] wrote:

It may be clear to anyone who knows what node.js and V8 do by default with object keys. But I didn't. And I kind of shouldn't be expected to, since that behavior is not ECMAScript spec, but an implementation artifact.

I won't tell you how to write code. I just would like the next guy to not wonder about this as I have.

—
Reply to this email directly or view it on GitHub.

[mightyiam]

Steps is a conceptual and visual hierarchical level.

If we discourage that, do we provide a different means of such a hierarchical level?

[beatfactor]

Adding a secondary interface, like BDD would address that.

On Fri, May 15, 2015 at 3:06 PM, Shahar Or [email protected] wrote:

Steps is a conceptual and visual hierarchical level.

If we discourage that, do we provide a different means of such a
hierarchical level?

—
Reply to this email directly or view it on GitHub
https://github.com/beatfactor/nightwatch/issues/449#issuecomment-102392531
.

[sknopf]

Actually using page objects would address that as well.

On Fri, May 15, 2015 at 9:16 AM, Andrei Rusu [email protected]
wrote:

Adding a secondary interface, like BDD would address that.

On Fri, May 15, 2015 at 3:06 PM, Shahar Or [email protected]
wrote:

Steps is a conceptual and visual hierarchical level.

If we discourage that, do we provide a different means of such a
hierarchical level?

—
Reply to this email directly or view it on GitHub
<
https://github.com/beatfactor/nightwatch/issues/449#issuecomment-102392531

.

—
Reply to this email directly or view it on GitHub
https://github.com/beatfactor/nightwatch/issues/449#issuecomment-102393935
.

[mightyiam]

Thank you. BTW this is a duplicate of #298.