Mocha: Describe block with async function behaving weirdly

[shashi-raja]

Mocha 3.5
Node 8.4

describe('Test Suite 1', async function(){
  let email = await faker.internet.email();
  it('should print email', function(){
     email.should.not.be.a('string')
   )}
})

Running this test should give

Test Suite 1
   ✓should print email

But its giving
0 passing

Also modifying the above code to

describe('Test Suite 1', async function(){
  let email;
  it('should print email', async function(){
     email = await faker.internet.email()
     email.should.not.be.a('string')
   )}
})

Runs the test but doesn't print the name of describe i.e Test Suite 1
✓should print email
Removing the async from describe and putting it in it works fine

 describe('Test Suite 1', function(){
  it('should print email',async  function(){
     let email = await faker.internet.email();
     email.should.not.be.a('string')
   )}
})

Test Suite 1
   ✓should print email

Also .only and .skip do not work when async function is included in the describe block

[ScottFreeCode]

Async functions are syntactic sugar for returned promise chains, and Mocha's describe blocks do not support (as in waiting for resolution of) returned promises. Failing to warn about it is perhaps less helpful than it could be, but the behavior is pretty much expected under the current design.

For cases where the names of the tests or how many tests there are can only be determined asynchronously, there is the --delay option, although that still requires the entire describe to occur after the asynchronous info is obtained (it still must be called synchronously inside describe before the describe returns).

If you don't need the data to define the tests but only to run them, you can use async before/beforeEach hooks to get it (that would be the obvious way to do these trivial examples), or even async tests with the initial methods to obtain the data written into the start of the test.

If you can come up with a real use case where --delay's limitations are needlessly awkward but it can't just be done with before/beforeEach since the definition of tests depends on it, then a case could be made for adding promise support (from which async function support would automatically be derived) to describe.

[caphale]

@ScottFreeCode , I have similar problem noted above.

Describe('some statement', function(){

before(function(done){

some async function;
done();
});
async.eachSeries(test,function(test,call){
it ('should test---', function(){

});

})

})

It block does not wait till before is complete. ' it' races around .

Now remove the async function , (dynamically forming tests). Before works as expected.

[ScottFreeCode]

Hi @caphale,

I get errors if I copy-paste this code into a new test file and try to run it, so I can't exactly debug what's happening. I don't even know whether by "Now remove the async function..." you mean the asynchronous code in the before or the async.eachSeries -- although it's more important that both might be incorrect, but I cannot necessarily tell how so given my lack of knowledge of where async.eachSeries comes from and what code structure has been replaced with "some async function;" in the before.

I can take some guesses, but these are just guesses:

• Does the before block really look something like this?
  js before(function(done) { thisFunctionHappensToBeAsync() // or ;(async function() { // setup steps here })() done() })
  If so, either move done inside the async function, or change it to this:
  js before(thisFunctionHappensToBeAsync) // or before(async function() { // setup steps here })
  The slightly longer "why" is that done tells Mocha that you've finished executing your asynchronous steps and before has completed running; if you call done after you've made the function calls that schedule the asynchronous setup but before those asynchronous steps actually complete, Mocha cannot know that it has to wait for them to complete (because you're telling Mocha that they already have); instead you'd need to call done in (at the end of) the asynchronous steps.
• I'm not familiar with async.eachSeries, but I assume from the name that the callback is not called synchronously, so those its are presumably being created outside the describe suite (since as previously mentioned it must be called before describe returns and not in their own asynchronous callbacks, whether describe is called in an asynchronous callback or not). The following points need to be kept in mind when designing the correct test structure:
  
  
  
  • If you don't need the asynchronous data to determine the number of tests or their names, then call describe and it synchronously instead and get the asynchronous data in an asynchronous or promise-returning before (or even directly in an asynchronous or promise-returning test).
  
  
  • If you do need to use asynchronously retrieved data to determine the number or names of its, you must use whatever asynchronous API you have to gather all the data and in a single, final callback call describe and all the its inside it followed by run(), then add the --delay option to running Mocha.
  
  

---

P.S. If you wrap your code in:
```js
(code here)
```
...GitHub will format it much better.

[bennypowers]

I want to fetch test data and use it to partially apply a validation function. doing it in beforeEach would defeat the purpose, since I wouldn't have the data outside the scope of the test

suite('muh data', async () => {
  const DATA = await fetch('data.json').then(jsonify);

  // checks that only path has changed
  // original -> path -> obj -> assertion
  const validatePropChanges = curry((original, path, obj) {/* yadda yadda */});

  const validate = validatePropChanges(DATA);

  test('incrementFoo only increments foo', () => {
    const cloned = clone(DATA);
    incrementFoo(cloned);
    cloned.foo.should.equal(DATA.foo + 1);
    validate('foo', DATA)
  });

[Bamieh]

I'd use this approach

const validatePropChanges = curry((original, path, obj) {/* yadda yadda */});

suite('muh data', function() {
   let validate;
   let DATA;
   before(async function() {
     const DATA = await fetch('data.json').then(jsonify);
     validate = validatePropChanges(DATA);
   })


  test('incrementFoo only increments foo', () => {
    const cloned = clone(DATA);
    incrementFoo(cloned);
    cloned.foo.should.equal(DATA.foo + 1);
    validate('foo', DATA)
  });

[stale[bot]]

I am a bot that watches issues for inactivity.
This issue hasn't had any recent activity, and I'm labeling it stale. In 14 days, if there are no further comments or activity, I will close this issue.
Thanks for contributing to Mocha!

[outsideris]

It's not stale because of #3243

[ofrobots]

In TypeScript projects, the lack of ability to use async describe functions leads to the following:

I want to write:

describe('express middleware', async () => {
  const {logger, mw} = await elb.middleware({level: 'info'});
  // ...
}

What I need to write:

describe('express middleware', () => {
  type ThenArg<T> = T extends Promise<infer U> ? U : T;
  type ElbThenType = ThenArg<ReturnType<typeof elb.middleware>>;
  let logger: ElbThenType['logger'];
  let mw: ElbThenType['mw'];

  before(async () => {
    ({logger, mw} = await elb.middleware({level: 'info'}));
  });

It would be quite handy if describe were to wait for the promise to resolve if one was returned. Is that possibility, or is there something fundamental that prevents this from ever happening.

[pronebird]

Is there any plan to fix this? I need async describe to spin off mongo before creating the app and running tests.

[plroebuck]

Why not spin up MongoDB in root-level before(), or use --delay with this.run()?

[pronebird]

@plroebuck --delay could work (ps I realized I haven't seen this option 😮). That's the scenario I have:

describe("Suite", async () => {
  const mongoURI = await spinOffMongoAndReturnConnectionString();
  const configuration = {
    ...readConfigFromEnv(),
    mongoURI
  };
  const expressApp = createTestApp(configuration);

  it('should do magic', async () => {
    const res = await chai.request(expressApp).put('/some/url').send({ ... });
    res.body.should.be.equal(true);
  });
})

ps: I would gladly move mongo setup to global hook that runs once before everything happens.

[plroebuck]

Maybe...

let dbconn;

before(async () => {
  dbconn = await spinOffMongoAndReturnConnectionString();
});

describe("Suite", () => {
  let expressApp;

  before(() => {
    const configuration = {
      ...readConfigFromEnvSync(),
      dbconn
    };
    expressApp = createTestAppSync(configuration);
  });

  it('should do magic', async () => {
    const res = await chai.request(expressApp).put('/some/url').send({ ... });
    res.body.should.be.equal(true);
  });

  after(() => {
    expressApp = null;
  });
})

after(() => {
  dbconn = null;
});

[pronebird]

@plroebuck Thanks. Surely that works except that Flow doesn't know how mocha works so it will likely force me to check if expressApp is null in each occurrence. Doh.. :/

ps: on the side note, chai.request() is not typed properly either so Flow swallows this error so f* it let's move on. :goberserk:

[plroebuck]

LOL...

  it('should do magic', async () => {
    // $FlowMocha
    const res = await chai.request(expressApp).put('/some/url').send({ ... });
    res.body.should.be.equal(true);
  });

".flowconfig"

suppress_comment= \\(.\\|\n\\)*\\$FlowMocha

[alex-at-cascade]

I struggled with this same problem for a while (certain tests mysteriously not running) until I found this issue posted. I had just assumed describe would wait for promises the same way it does. I can work around it, but it would be wonderful if describe could just recognize async functions and behave accordingly.

[plroebuck]

@alex-at-cascade, well at least you'll get a warning from the upcoming release...

[WORMSS]

Just to add to the discussion that our company got caught out by the non-async nature of describe when a developer changed from require to import to match the rest of the system. Sadly this caused no errors, and was only during the addition of .only() by myself that the test reported 0 Passed when I knew there should be 38 that either passed or failed.`

describe.only('aqs.utils', async () => {
  await import('./equals');
  await import('./gas');
  await import('./notEquals');
  await import('./startsWith');
  await import('./wordSearch');
});

We went back to the original and now works as expected, but I think we would prefer to use import

describe.only('aqs.utils', () => {
  require('./equals');
  require('./gas');
  require('./notEquals');
  require('./startsWith');
  require('./wordSearch');
});

[ondratra]

Because I want to prevent my code from having Hadouken indent (http://i.imgur.com/BtjZedW.jpg), I wanted to use this design:

test.ts

import mySubtest1 from './subtests/component1'

describe('1st level describe', async () => {
    const prerequisites = await serverINeedToRunBeforeTests()

    describe('2nd level describe', mySubtest1(prerequisites))
})

./subtests/component1.ts

export default testComponent1(prerequisites) {
    // test the component
    it('does what it should', () => {})
}

In this scenario before() can't help me so I am forced to rewrite the code and use --delay. Delay is not optimal in such situation because it brings unnecessary waiting when is set delay high while it can start things prematurely when delay is set low. Async here is elegant solution.

@ScottFreeCode Is there any obstacle in implementing async support for describe?

[plroebuck]

@ondratra, why can't before() work for you?

"test.ts"

import testComponent1 from './subtests/component1';

describe('1st level describe', () => {
  let prerequisites;

  before(async () => {
    prerequisites = await serverINeedToRunBeforeTests();
  });

  describe('2nd level describe', () => {
    testComponent1(prerequisites);
  });

  after(() => {
    disconnectFromServerINeedToRunBeforeTests();
    prerequisites = null;
  });
});

[ondratra]

@plroebuck after more thinking and testing I found out your design solves the issue. thx

[junaid1840]

I have found another way to get things done.
May be this could work for someone.

describe('Parent', () => {
    let array: any = [];
    before(async () => {
        array = await someAsyncDataFetchFunction();
        asyncTests();
    });
    it('Dummy test to run before()',async () => {
        expect(0).to.equal(0); // You can use this test to getting confirm whether data fetch is completed or not.
    });
    function asyncTests() {
        array.forEach((currentValue: any) => {
            describe('Child', async () => {
                it('Test '+ currentValue ,() => {
                    expect(currentValue).to.equal(true);
                })
            })
        });
    }
});

That's how I achieved the assertion on every element of the array. (Array data is being fetch asynchronously).
Kindly correct me if I'm doing it wrong.

[WORMSS]

You will fall into the same problem I was having junaid, those inner

        array.forEach((currentValue: any) => {
            describe('Child', async () => {
                it('Test '+ currentValue ,() => {
                    expect(currentValue).to.equal(true);
                })
            })
        });

Will not be contained within the "child" describe and will actually be outside the scope of the tests. Don't get me wrong, they will run, but they won't be counted towards where you think they will be.

You would presume it would be

Parent
  [x] Dummy
  Child
    [x] Test 1
    [x] Test 2
    [x] Test 3

but it will actually be

Parent
  [x] Dummy
  Child
[x] Test 1
[x] Test 2
[x] Test 3

Subtitle but different.

[danielbodart]

Is there any plan to fix this in the long term?

[jcollum]

Yeah this behavior is a pain, resulted in a lot of weird race conditions for me.

What worked for me: if you need async, do it in your before blocks.