Mocha: Run tests in definition order

[segrey]

It'd be great if mocha would run tests in its definition order. Let's consider this example:

var assert = require('assert');
var runOrder = 1;

describe('Mocha test runner', () => {
  it('1st test', function () {
    assertRunOrder(1)
  });
  describe('inner suite', () => {
    it('2nd test', () => {
      assertRunOrder(2)
    });
  });
  it('3rd test', () => {
    assertRunOrder(3)
  });
});

function assertRunOrder(expectedRunOrder) {
  assert.equal(runOrder++, expectedRunOrder);
}

Actual test run order:

1. Mocha test runner > inner suite> 2nd test
2. Mocha test runner > 1st test
3. Mocha test runner > 3rd test

Expected test run order:

1. Mocha test runner > 1st test
2. Mocha test runner > inner suite> 2nd test
3. Mocha test runner > 3rd test

[albertogasparin]

Have you tried with mocha-steps? (not sure about the behaviour with nested describe though).

[segrey]

@albertogasparin Sorry, missed your question. It doesn't work either: 3rd test is executed as the second one.

[takayukioda]

Needed some change, but I succeed to make them run in expected order
wrap each it by describe.

describe('Mocha test runner', () => {
  describe('inner suite for 1st', () => {
    it('1st test', () => {
      assertRunOrder(1)
    });
  });
  describe('inner suite for 2nd', () => {
    it('2nd test', () => {
      assertRunOrder(2)
    });
  });
  describe('inner suite for 3rd', () => {
    it('3rd test', () => {
      assertRunOrder(3)
    });
  });
});

  Mocha test runner
    inner suite for 1st
      ✓ 1st test
    inner suite for 2nd
      ✓ 2nd test
    inner suite for 3rd
      ✓ 3rd test


  3 passing (9ms)

[sberney]

mocha-steps , while useful, does almost nothing. It does not change the execution order of the tests. Instead it programmatically marks all following suites and tests as pending when a test invoked with step() fails.

The issue here is that tests don't run in definition order. I too really wish I could use mocha to easily run a smoke test sequence. In the midst of the smoke test, I run a big BDD suite I want to run first, before the smoke test continues. The thing I'm smoke testing manages lifecycles, so the suite ought to succeed at one point in time, but not after that.

I've tried to find other tools to do this, but I haven't encountered anything satisfactory. And I would need a second parallel set of expensive tooling for the alternate test execution framework. (I also have regular, order agnostic test files run through mocha). Mocha would be perfect for it, if I could control test execution order somehow.

[plroebuck]

Seriously doubt this will be changed. Mocha runs suites (which run tests). If you want to manage your run order, @takayukioda's scheme above achieves that.

@sberney, in your case, write your sequence as three distinct suites:

• smoke-test-phase1
• smoke-test-phase2 (BDD)
• smoke-test-phase3

[sberney]

@plroebuck I also seriously doubt this will be changed, but, for the record, this is a deal breaker.

If I had found or do find any test runner libraries (regardless of its ecosystem maturity, which of course wouldn't match mocha's) that ran mocha-compatible BDD-style tests, but just in order, I would switch to that in a heartbeat.

Your suggestion isn't realistic when you get into complicated nesting scenarios, and I don't feel like explaining this to junior programmers.

The point of describe(), context(), it(), is to naturally describe the API; in a declarative sense. Changing my declarative description wouldn't make sense without changing the API I'm describing, which isn't happening.

Mocha isn't very well developed for smoke test suites; but that's fine with me. It's great at what it's great at.

[plroebuck]

@sberney, I don't know your situation. Your initial comment only suggested 3 steps, which isn't all that complex. I only suggested wrapping the first and last parts of your smoke test for grouping to achieve your desired effect. Perhaps for more complex scenarios, I would have unique hierarchially-valued strings in the file headers. Simple shell script could then grab whatever was grepped for, creating a temp spec appropriately wrapped in an untitled context (which hopefully doesn't offend your BDD sensibilities as much) and run that through Mocha.

Mocha's design just doesn't have the concept of pausing a running suite to work on another one.