Mocha: global setup / teardown hooks

[boneskull]

With the addition of root hook plugins and parallel mode, an important use case may have been omitted.

In serial mode, a root _before all_ (before()) hook can perform async setup (e.g., starting a test server) and a root _after all_ (after()) can perform async teardown (closing the server). This is not perfect, given Mocha could crash and the server would not get cleaned up, but I think it's behavior people expect.

In parallel mode, this behavior not available. Root hooks "before all" and "after all" run _once per file_. If you want to run something once and only once, you're kind of stuck.

One could abuse mochaHooks by defining a Promise-returning function, and doing setup there...

exports.mochaHooks = async () => {
  // do something once only
});

... but there's no associated "teardown".

The only real workaround that I'm aware of is via script, e.g., {"scripts": {"test": "startServer && mocha; stopServer"}}. Top-level await suffers from the same problem as the above example--no associated teardown.

An idea for this would be to implement two new "hooks" in mochaHooks, e.g., begin and end.

• These would be aggregated before Mocha begins running tests, similar to the behavior of the other hooks
• The hooks are guaranteed (within reason) to run _once and only once_.
• These hooks can share a context _with each other_, but nothing else. You cannot access the current test context from begin, for instance. You _could_ define this.foo in begin and access it in end, I suppose.
• Standard Node.js-style callbacks and Promise-returning functions allowed
• They will always run in the main process

Alternatives:

• Instead of using mochaHooks, use a different property.
• Instead of having a single property, use mochaBegin and mochaEnd properties.
• Name it something else, but be careful not to conflate it with existing hooks or interfaces (setup is going to be very confusing for those using the tdd interface, for example).

cc @nicojs, who may have opinions about how this should work.

[boneskull]

FWIW I think it may be confusing to use mochaHooks for this, since there isn't a 1:1 mapping between e.g., begin and any other hook.

[boneskull]

@mochajs/core Any ideas? @jan-molak?

It would be helpful to have someone with this actual use case. Might not figure this out until somebody has a problem with how it works and complains about it. :smile:

[rickcarrier]

I see a need for a begin() and end() hooks that run once for all describes in parallel mode. I'm currently doing some setup multiple times in the beforeAll() because there isn't a better option that allows me to set up once

[boneskull]

yes, I think this is needed, but unsure what the API should look like. open to ideas.

won’t have time to implement anything for a couple weeks at least

[craigtaub]

Been looking into how Jest does things. Setups and teardowns provided by Jest:

• Once-global - creates state only available for this context, not tests. You can add basic properties to the process.env global here, but not resources (limitation of nodejs, type-casts any props to a string).
• Single-use per-file - creates and shares state inside the current suite context
• Shared per-file - creates and shares state inside the current suite context

There is an example for getting it to work for Puppeteer (here) which uses 2 of the above.

Essentially nothing I found providing a global setup and teardown for setting and sharing state between processes.
They have many GH issues for this. Most of the discussion around the json serialisation of resources problem.

There might be some libs out there which could to help us, but needs more investigation. I'll see if I can find anything on AVA.
Hope this helps.

[boneskull]

Thanks. I think a mochaGlobalSetup and mochaGlobalTeardown would be appropriate then. I'll see if we can share the context object between them.

[boneskull]

I _think_--but am not sure--that we should _not_ re-run these when watching files.

[jgehrcke]

I think a mochaGlobalSetup and mochaGlobalTeardown would be appropriate then

:+1: This has little mental overhead and aligns with existing concepts. I was looking for something like this just today.

When we look at https://mochajs.org/#run-cycle-overview that visualization I think immediately makes the point that we need a hook at a hierarchy level closer to the root of things (I looked at this picture today and wondered: why would there be no "global hook"?, then found this issue here.)

The https://mochajs.org/#root-hook-plugins section was a bit repelling as of the complexity, and caveats.

I spent many years writing tests in the Python ecosystem using pytest. I must say that I miss the concept of _scoped_ "fixtures" in the JavaScript testing ecosystem. I do not like the word "fixture" too much, but let's think of it as a dependency. An individual test can require a dependency. But a test suite (test module) can, too. But also the entire test session can require/request a dependency. What we think of as "global" here would be global within a "test session" in pytest. So, in pytest, if you define a "session-scoped fixture" then the session would "consume it" (run its code), _once_ (because there is only one session, the current session, think "test run", comprised of many test modules), and _before_ running the first test in the first module. For testing complex systems we've always made heavy use of this concept; with quite a bit of business logic being executed as part of global test suite (well, session) setup and teardown.

[boneskull]

The https://mochajs.org/#root-hook-plugins section was a bit repelling as of the complexity, and caveats.

Yes, if we add something like this we will reduce the complexity there (at least in terms of documentation). If you want to run a hook ("fixture") _once_, use mochaGlobalSetup; if you want to run it before all tests in a suite _in a given file_, use a before hook; if you want to run it before _each_ test, use a beforeEach hook. There may not be any practical difference between mochaGlobalSetup and a before _in Mocha's default serial mode_, but if they are used as prescribed, the user can expect consistent behavior in serial and parallel mode.

(hope that made sense)

[sarbbottam]

Looking forward to a resolution to a global setup that runs only once for all the test files.

[boneskull]

This will be released in v8.2.0. I do not know yet when v8.2.0 will be released.

[rickcarrier]

@boneskull Any estimate when v8.2.0 will be released? I'm trying to prioritize some work based on when this fix will be available. Thanks!