Mocha: Adding -p,--prepare option to allow async initialization before loading test files

[enobufs]

I came across a situation where async initialization before loading all test files has become necessary. The --require option allows us to some initialization tasks but this is synchronous. It would be really great if mocha offers a new option which allows us to give a path to an initialization files like this:

// mocha.ops
-r test/require
--prepare test/init_a
--prepare test/init_b

The init_x files would have something like this:

module.exports = function (cb) {
    // do some async stuff
    cb();
}

or;

module.exports = function () {
    // do some async stuff
    return Promise.resolve().then(function () { /* do something */ });
}

My project started without needing such async initialization for testing. Now it is needed, but we already have a number of test cases that references some variables before before handlers are executed. Modifying all these test cases to do all the initialization in the before handlers are not realistic/doable at this point.

We have experimented the proposed option, in this branch, which works very well. This change is done off of 2.2.5 tag.

Please let me know if this has already been addressed or discussed elsewhere. If this feature is found useful to other users (I am sure it is), I will be more than happy to send a PR for an appropriate branch/revision.

[willfrew]

Unless I'm misunderstanding your use case, I think this could be achieved with either:

1. Root level hooks
2. Delayed root suite

I had the same problem and found this issue, then solved it using root level hooks. Hope this helps.

[danielstjules]

A root level before hook should solve this for you, as @willfrew mentioned :)

[enobufs]

Oops, sorry, I missed the earlier post by @willfrew ...
I tried root level before a while ago, and it did not work because:

(1) Many test cases access modules at module level like this:

var bar = require('foo').bar;
describe('my test', function () {...});

Where the bar property won't be available until an async initialization has been complete, and not available when mocha load this file.
I know we could move the line inside before handler to be sure that the bar is accessed after a required initialization, but again, there are a number of tests cases we already have, applying this change everywhere is currently not easy.

(2) The root before handler is called only if the file that contains the root before.

This becomes a problem if you want to run tests partially, by either specifying a file or directory for mocha. If the path does not include the root before, it won't work. (which means you need to have root before in every file, which is not ideal. Or is there a way to force mocha to load a specific file even you are passing a specific path to a file or folder to run only those tests? (like mocha.opts that is loaded all the time regardless of the path you are passing to mocha)

[danielstjules]

In regards to 1 - How does the application in question normally work if there's a race around the use of properties that are being defined async? One solution to that would be to make foo.bar return a promise that resolves with the actual value. Then you're not having to bootstrap everything, but only the modules you care about in each test file's before hook.

[enobufs]

The application we have is a traditional web application that uses express. It is designed so that all initializations of sub-components (DB, Redis, etc) are complete before the system starts listening on http server port. These sub-components have some dependencies, too (some caching feature won't become available until redis connection is established, etc.), so we internally have multiple levels of initialization phases to organize this dependency graph.

Many of our test cases uses mocking/stubbing (sinon, or custom) to skip all these init sequence, but we also use mocha for integration tests where we want to initialize all with real code rather than stubs.

Original mocha used to work for us, but as the system grows, this problem has become evident, we needed a hack with a hope that upstream mocha provides similar option like -p,--prepare I proposed so that we can ditch our forked ver.

The exposed property foo.bar is not always a function. It could be an object created as a result of earlier initialization. (DB instance, redis client with specific db index, etc). You can imagine the following idiom:

var foo = require('foo');
foo.use(require('mod_a'));
foo.use(require('mod_b'));
foo.use(require('mod_c'));
  :
foo.init(function () {
    foo.bar.doSomething();
});

[enobufs]

I have published a module to help this situation, but hacky... I'd still wish the same feature is provided natively.
https://www.npmjs.com/package/mocha-prepare

[antoinebrault]

+1
Also, "suite.file" is fucked up when using --delay
See https://github.com/mochajs/mocha/issues/2063

Im trying to generate dynamic describes based on the result of a couple of database calls.

[jbcpollak]

thanks @enobufs your package works perfectly

[calvinl]

+1 I just came across this when discovering the need to sync my test database before running the test suite, but @enobufs's package works indeed!

[enobufs]

It's been close to 3 years since I made mocha-prepare our of my need. It is great to know many other people see the value of the feature, BUT, I still wish that mochajs provide the same feature natively.

The mocha-prepare, as it is being a hack, has its own limitation as reported here - to which I do not have a solution. It would be great to see mochajs maintainers/contributors would reconsider incorporating an equivalent feature hopefully soon.

[JSBach-BYU]

@enobufs I found your package today due to your comments here. It is EXACTLY what I needed for my project.
Thanks for your contribution!!

[krzkaczor]

@enobufs nice work man! I am working on code generation tool for typescript and I wanted to programmatically run it at the very beginning of the test suite because later tests cases use generated code. I couldn't use before/after hooks because then generation would happen too late and ts-node would scream about compilation errors.

[plroebuck]

Doesn't use of --file cover this already?

[krzkaczor]

@plroebuck In my case it didn't work — i think all files are eagerly parsed by loader first and thus I was getting tsc errors (my before hook was generating new ts files).

[plroebuck]

Would have assumed a suitably written init file run with --file option capable of delaying processing of test script...

[krzkaczor]

@plroebuck, in my opinion, a better solution would be to fully utilize require option. Imagine that the required file could export function (potentially async), then mocha just waits till it's execution is done. This solves all the problems.

[plroebuck]

Was thinking more of a Promise.all(promises).then({}).catch({}) style solution. @krzkaczor, not necessarily disagreeing with your opinion, but would prefer a workable solution that didn't add more complexity to Mocha (i.e., documention vs. code changes). Wouldn't the Promise.all as last line of init script approach handle this issue?

[AndreyYevtukhov]

In some project, I need to:

1. read prepared test data from files
2. process it
3. generate tests, based on processed data
4. run tests (there are few .specs files, that need async operations to be done BEFORE test start)

I've tried to do it using:

1. -r (--require) <file> - 0 result, it doesn't support async operations
2. --file <file> - 0 result, works as -require
3. --delay and run() - works, but... I have multiple .specs files. I'm using --grep to run different .specs combinations. In some cases this file won't be included, so no async prepare operations wouldn't be done, and tests wouldn't be created correctly... Maybe some example of using ROOT-LEVEL HOOKS and DELAYED ROOT SUITE for dynamically created tests (that need async initialization) will help me understand how to use them correctly here...
4. mocha-prepare - works perfectly. All data is prepared before any tests suites are generated, and I can run different .specs combinations using --grep without any errors

---

[outsideris]

Usually, I use npm test to run mocha test.
I think pretest in npm scripts is able to solve the problem and it support async operations.

"scripts": {
  "pretest": "node prepare.js",
  "test": "mocha"
 },

[enobufs]

For things you can run in a different process, yes.

[plroebuck]

@enobufs , have glanced your package a couple times, but I guess still fail to see why using some type of Promise.all() scheme doesn't work with --file or --require. Do you have a PR that doesn't monkey-patch Mocha (i.e., a maintainer-level solution)?

Do any of you have an MCVE that we could use for testing?

[enobufs]

@plroebuck, it was explained by @AndreyYevtukhov :

-r (--require) <file> - 0 result, it doesn't support async operations
--file <file> - 0 result, works as -require

The async ops in the file you specify with -r,--file won't be able to block subsequent Mocha.run() call - (mocha test files will be loaded before your async calls complete)

[plroebuck]

@enobufs rather than monkey-patching Mocha.prototype.run(), could it be implemented using runner.on('waiting', onWaitingCB) with --delay?
Sorry about all the questions, but inexperienced with this aspect of the codebase...

[sparkida]

I think the Mocha wiki article "Using Mocha programmatically" is what a lot of people are looking for.

[enobufs]

Just to be clear...

This issue is about the command line script, mocha.js.

[plroebuck]

@enobufs wrote:

This issue is about the command line script, _mocha.

FTFY

[plroebuck]

@antoinebrault, @sa-0001, no one provided an MCVE for #2063 before the #$@&%*! stale bot closed it. Anyone have one that we could test against?

[plroebuck]

@enobufs, still nothing we can test against from you either. Give us something that replicates the problem to test with. If you have an implementation suggestion that _doesn't_ involve monkey-patching Mocha.run() or swallowing --watch, it would be welcome.

[enobufs]

@plroebuck I made a proposal (in my fork) for this issue, along with a test case for it to demonstrate what the issues are and how it can be solved.

https://github.com/enobufs/mocha-issue-1810-test

Please review.

[boneskull]

See https://github.com/enobufs/mocha-issue-1810-test/pull/1 for an example of how to use existing Mocha functionality to do this.

@enobufs please close this issue if my suggestion addresses your use case; I'm having trouble understanding if it doesn't.

[enobufs]

@boneskull I made a PR for you to show you why --file wouldn't work. https://github.com/boneskull/mocha-issue-1810-test/pull/1

[enobufs]

@boneskull Let me take that back. Now I think --file works. (I was so much concerned about initializing modules before loading in the spec... )

[enobufs]

@boneskull It appears to me that --file option was introduced in v5.0.0. before()/after() for the file also became available in the version? (I will make a note for mocha-prepare to discourage the use of mocha-prepare)

[enobufs]

My test shows, --file option before v6.0.0 did not work as expected.

Before v6.0.0, the file specified by --file won't load if you try to run specific files.

[iredjee]

@enobufs thanks for great mocha-prepare, I think it helped a lot of people )

Any idea how to leverage async usage of --file option? As I understand it's possible, but I can't figure out how, and documentation is not so detailed about this point.

I want to mention one more issue. Some details here. Briefly, there are some plugins to visualize tree of tests in VSCode for example, and they programmatically use Mocha.loadFiles. Eventually this method fails if you need to run some async preparation before requiring files. So it would be nice to have option to run async function before starting Mocha.run and/or Mocha.loadFiles

[AndreyYevtukhov]

Found another solution for myself here:

https://stackoverflow.com/questions/22465431/how-can-i-dynamically-generate-test-cases-in-javascript-node

maybe it'll help ^^

[dbettini]

@enobufs @boneskull It seems you're saying that the new behavior is the same as mocha-prepare. However, the test files are STILL loaded before the root before promise resolves(from file used by the --file flag), but their hooks/tests are not run. mocha-prepare, on the other hand, loads the test files AFTER prepare hook calls done. Tested on [email protected]

[Mihailoff]

In newer versions (v8 in my case), there is a --delay option available https://mochajs.org/#delayed-root-suite

If you need to perform asynchronous operations before any of your suites are run, you may delay the root suite. Run mocha with the --delay flag. This will attach a special callback function, run(), to the global context:

// init.js
promise.then(() => {
  run()
})

before((done) => {
  // waits until run()
  done()
})