Jest: What is the official way to run jest programmatically?

[IgorAufricht]

My question is how to run jest programmatically. There are some examples on the internet:
https://stackoverflow.com/a/30562836/1652273
https://stackoverflow.com/a/33669530/1652273
https://github.com/facebook/jest/issues/3848

All of them reference requiring jest or jest-cli and calling runCLI(), but it does not seem to be documented anywhere (I haven't found anything in the jest docs nor is there any documentation for the jest-cli package).

Is this the official supported way? Thanks.

[IgorAufricht]

I guess my main question is: is the runCli() a part of public API?

For example this commit https://github.com/facebook/jest/commit/200b032d170054dc3860f3cd869a94201c54dd19 changed the runCli() - it removed the onComplete callback and replaced it with a Promise, which is not reflected in the commit message and/or the changelog.

[thymikee]

I think there's no official way, that's why it's not referenced in the docs and changelog.
Other way to do it would be jest-editor-support package by @orta

@cpojer what do you think about an "officially supported" way of invoking Jest as a node process?
@IgorAufricht would you be interesting in writing about this?

[IgorAufricht]

To add a bit of context - I'm adding jest into our grunt-based build process. I basically have these options:

1. Run jest outside grunt (e.g. the build script could be something like npm run jest && grunt build)
2. Use the deprecated grunt-jest
3. Run jest from grunt as a process (e.g. using grunt-run)
4. Run jest programmatically from a grunt task
   From these options the nicest would be the last one, hence my question.

If there is currently not an official way to run jest from node, then adding it would involve more than just a documentation change (probably some more thinking about the public API, adding tests and documentation, etc.). While I personally think it might be useful to add an official way to run jest programmatically, it's up to the jest maintainers to decide whether to add support for this or not.

@thymikee I'm not entirely sure what you're asking (writing about what?). But yes, I'm happy to help.

[cpojer]

@thymikee yeah, I'm not opposed, Jest already has an API that can be used but it requires people to read the source to understand how it works, so it's not optimal. The other thing is that we may choose to break the API from major to major.

[orta]

FWIW, I'd love this - I think exposing a consistent JS API to run tests in jest-editor-support would mean that jest-x deps APIs can break between majors, but editor-support could provide a facade for whatever has to actually happen under the hood.

[cpojer]

I'm gonna close this because the issue is a question. If you'd like to use Jest's programmatic API, check out the main module of jest-cli. If you'd like it to do things that it doesn't currently do, please send as a well-tested PR that adds or changes behavior.

[tquinlan1992]

const jest = require('jest');

jest.run(argv);

[slavafomin]

I'm also interested in running jest programmatically from node.js, sadly it's not documented anywhere and the API doesn't look very thoughtful for this use case.

[SimenB]

It's not. That's something we want to improve though (although a bit down the priority list at the moment)

[slavafomin]

@SimenB I've just found out that Jest is working pretty fine in this scenario actually, the only problem is the documentation and typings support.

I've posted a small example on how to use Jest in this scenario:
— How to run Jest programmatically in node.js (Jest JavaScript API)

@cpojer Is there an issue where the progress on this feature could be tracked?

[SimenB]

No issue currently. We'll try to split up jest-cli for the next major and clean up the programmatic API at least a little bit (so that yargs and argv stuff is removed). After that, we'll probably create an issue trying to gather feedback for what use cases people have for using Jest programmatically in order to decide how to best move forward

[slavafomin]

@SimenB that sounds great. Thank you for your hard work!

We are calling Jest as part of our build framework inside of the existing node.js process. The current implementation works great for us, we get the console output as well as results object with more information regarding tests outcome.

My only suggestion would be to have a better naming for the API and a typings support, so the module could be used in the TypeScript environment directly.

[SimenB]

We're going to be migrating to TS (#7554), so typings will definitely be there 🙂

[brendoncrawford]

Just in case anybody is wondering, I was able to successfully do this:

npm install --save-dev jest jest-cli

const jest = require("jest-cli/build/cli");
jest
  .runCLI(
    {json: false},
    [process.cwd()]
  )
  .catch((error) => {
    console.log("Error:");
    console.log(error);
  })
  .then(() => {
    console.log("Done");
  });

EDIT: Fixed typo.

[SimenB]

Teeny, tiny start here: #7696. Not ready for consumption (don't rely on us not breaking its API without bumping major), but it will happen at some point 🙂

[jasonwr]

Just in case anybody is wondering, I was able to successfully do this:

npm install --save-dev jest jest-cli

const jest = require("jest-cli/build/cli"); Jest .runCLI( {json: false}, [process.cwd()] ) .catch((error) => { console.log("Error:"); console.log(error); }) .then(() => { console.log("Done"); });

Small typo Jest should be jest

[SimenB]

_Please_ don't reach into the build directory - that's bound to break.

We've created a @jest/core package in master (the PR linked above - will be part of the next release, whenever that happens) that will evolve into the programmatic API. It's API should be considered experimental (aka, don't come complaining if it breaks outside of a major), but it's moving forward at least 🙂

[jasonwr]

_Please_ don't reach into the build directory - that's bound to break.

We've created a @jest/core package in master (the PR linked above - will be part of the next release, whenever that happens) that will evolve into the programmatic API. It's API should be considered experimental (aka, don't come complaining if it breaks outside of a major), but it's moving forward at least 🙂

@SimenB hey that sounds really good. Along with the new code a tutorial would be very nice. Thank you.

[SimenB]

Whenever we've landed on an API we're happy with, we'll document it properly as well 🙂

[jasonwr]

Whenever we've landed on an API we're happy with, we'll document it properly as well 🙂

@SimenB also I'd be happy to be your use case tester as I'm sure others here would be. 💯

[simonsen78]

@SimenB I would also love some documentation on this area. Looking forward to it! :+1:

[SimenB]

The core team will be meeting up in London later this month, and I've written this issue down as one of the things I'd like to discuss at that time. So hopefully we'll have a plan for programmatic API after that 🙂

[another-guy]

@SimenB ❤️ it's very nice to hear. Is there any way to affect the priority on this issue?

[SimenB]

Until we gave a plan (which we'll hopefully have in 3 weeks or so), no.

In the meantime, maybe you can provide som API suggestions? What do you want from a programmatic API?

The first thing that comes to mind as difficult (the way things currently are) is to trigger events in watch mode - the only interface is internal FS watchers and watch plugins, nothing that can be interacted with from the outside. But is that something you need, or is just spinning up Jest and awaiting the promise it returns enough?

So if we can focus on "run jest" and "run tests multiple times (and interact with a running version of Jest)" as separate things, that'd probably be good.

For one-time runs, do you need an argv parser? Should you be able to provide partial config, or mix in defaults yourself?

[SimenB]

Can open this for tracking purposes

[another-guy]

@SimenB Thank you for being open to proposals! I'd definitely consider offering input on the API shape but I don't feel qualified at this point.

My interest is coming from a specific use case, however. You can read about it on Medium: End-to-End Testing VS Code Extensions via Jest: Setting Things Up.

My only suggestion for now would be to provide first-class typings please. The existing typings felt outdated to me. When I'm calling runCLI() in my code, I'm passing the first argument as any. And I had to dig through the jest-cli and jest-config code base to see what I need to pass in. Even if documentation is in a perfect shape, many developers are editor's code-completion driven. 🤷‍♂️

If I was a lib contributor, I'd try to make jest-config responsible for providing API for

• reading config objects from files,
• creating config objects from scratch (by hand),
• merging config objects hierarchically.

While jest-cli to be responsible for exposing APIs that allow triggering Jest executions:

• via raw CLI parameters,
• as well as by passing config objects.

Hopefully, it makes sense. I'm probably missing other important functions, but this seems to be minimal set I may need. 🤔

---

P.S. For Watch functionality you may want to get more input from someone else. I'm unsure here.

[SimenB]

Thanks for the feedback!

My only suggestion for now would be to provide first-class typings please.

Jest 24.3.0 has been rewritten in TypeScript, so all modules provide full typings

[another-guy]

@SimenB then I'm missing something because as I said

I had to dig through the jest-cli and jest-config code base to see what I need to pass in.

...as in _typings were misleading in my case_ and I had to fall back to runCli(config as any, ...) in my code

[SimenB]

Oh sorry, Jest 24.3 was released earlier today so I assumed you'd tested with an older version.

As mentioned, we don't really have a public API meant for consumption yet, but typings should be in place for all modules. You'll probably have to jump through a few hoops to be able to use it properly though (we only really have a CLI API and yargs handles creating the object passed to runCLI)

[another-guy]

@SimenB the type info showed here is missing a config field... Either I am doing something very wrong then, or the typings are not quite right. 🤷‍♂️ Sorry, I'm in the office now and can't offer more details than below.

  "devDependencies": {
    "@types/jest": "^24.0.9",
    "@types/jest-cli": "^23.6.0",
    "@types/node": "^11.10.4",
    "jest": "^24.1.0",
    "jest-cli": "^24.1.0",
    "coveralls": "^3.0.3",
    "prettier": "^1.16.4",
    "rimraf": "^2.6.3",
    "ts-jest": "^24.0.0",
    "ts-node": "^8.0.2",
    "tslint": "^5.13.1",
    "tslint-config-prettier": "^1.18.0",
    "typescript": "^3.3.3333",
    "vscode": "^1.1.30"
  }

The dependencies were up to date with latest (at the time of coding -- about a week ago).

[SimenB]

As mentioned, we released a version of jest with bundled typings _today_. You're not looking at Jest's official typings, I don't know where your typings are from

[simonsen78]

I actually managed to get Jest running programatically using the API by piecing together bits and pieces of code from various places on the internet.

As it stands now, it's working perfectly for me. Can't really think of anything I'd need at the moment.

All I need/needed was a basic example on a wiki page, that could have spared me some time. But I guess that will arrive when the API goes public?

I like Jest more and more the deeper I dive, thank you for sharing a great piece of software! 👍

[SimenB]

Talked with @orta about this today, and we concluded that we don't really know how this should look. 😀 I'll just dump my notes from our conversation here so they're not lost (no concrete API suggestions here really, just some rambling about use cases and how they might work).

We figured that the thing that makes sense is:

• Invoke jest with a partial of the configuration, and it returns a stream of events (once the stream ends, the jest run is complete)
  
  
  
  • not sure if this should be rx, node core event emitters or something else entirely
  
  
• Expose some way of loading Jest's configuration (a la babel's loadOptions, but not as fancy) that can be modified, then passed to jest
  
  
  
  • should mix in our own stuff with the partially supplied config as well
  
  
• jest needs to expose its reporters in a way that allows you to print the results as they come back in a way that pretty much looks like jest

Things out of scope for a first iteration:

• watch mode (not sure how to allow you to send signals to the watch mode to make use of it)
• run jest as a server, and ask it to execute tests to avoid the initial startup cost
• CRA currently wraps Jest, we should support their use case without them needing to spawn the Jest CLI

[wadethestealth]

Moving forward, which is the module that is supposed to support programtic calls jest-cli or jest-core?

[SimenB]

it'll be @jest/core. Still no news on this btw, I haven't had the time to design or discuss anything more here

[lamhieu-vk]

in my toolkit have a test runner based on Jest, currently I using like below:

import jest from 'jest';

const config = {}; // something
args.push('--config', JSON.stringify(config));

await jest.run(args);

I suggest syntax to use like this, same webpack:

import * as jest from 'jest/core';

const runner = jest(config);

// we can add some event listeners before run
runner.on('event', callbackOfEvent);

runner.run(callback); // run once
runner.watch(callback); // run watch

what do you think about this?

[SimenB]

instantiating jest then calling run is an idea I can get behind! We probably won't add events or watch for a first go, but it will leave the door open for it to be added without breaking the API. I like it!

[kiprasmel]

Up - any updates on this?
The lack of documentation sucks..

I don't know how to reproduce the same behaviour as running jest --config=./path/to/config and just ended up using execSync (from child_process) to call the npm script for running tests lul

https://stackoverflow.com/a/32872753
https://nodejs.org/api/child_process.html#child_process_child_process_execsync_command_options

[joscha]

we have code like this:

async function getListOfTests(partition: Partition): Promise<string[]> {
  // unfortunately the jest node interface is quite rudimentary at the moment,
  // there is no way to redirect the output to a different stream whilst using
  // a public API, so we need to capture it
  // See https://github.com/facebook/jest/issues/5048 and
  // https://github.com/facebook/jest/pull/7696
  // so for the time being we hijack stdout
  // TODO(joscha): remove this workaround once https://github.com/facebook/jest/issues/5048
  //  is closed
  const output = await capture(async () => await runCLI({
    _: [],
    $0: '',
    listTests: true,
    json: true,
    testPathPattern: partition.testPathPattern,
    testPathIgnorePatterns: partition.testPathIgnorePatterns,
  }, [rootDir]));

  return (JSON.parse(output) as string[])
  // we sort here to keep the list as stable as we can
      .sort((a: string, b: string) => a.localeCompare(b));
}

with

/**
 * Captures writes to a given node process for the time of the execution of a given function
 *
 * Caveats: monkey-patches process.stdout, output that is bigger than the node heap will cause an
 * overflow because it will be kept in memory instead of piping it through
 */
async function capture(
    fn: () => Promise<any>,
    p: WriteStream = process.stdout,
): Promise<string> {
  const originalWrite = p.write;
  let output = '';
  try {
    p.write = (chunk: Uint8Array | string) => {
      if (typeof chunk === 'string') {
        output += chunk;
      }
      return true;
    };
    await fn();
  } catch (e) {
    throw e;
  } finally {
    p.write = originalWrite;
  }
  return output;
}

if it helps someone as a workaround. We use it to shard our tests on CI automatically using https://github.com/joscha/ShardyMcShardFace to achieve something like in https://github.com/facebook/jest/issues/6270

[pauldraper]

instantiating jest then calling run is an idea I can get behind!

For comparison, see Jasmine

import * as Jasmine from 'jasmine';

const jasmine = new Jasmine();
jasmine.loadConfigFile('spec/support/jasmine.json'); // or jasmine.loadConfig
jasmine.configureDefaultReporter(...); // by default, console
jasmine.execute();

https://jasmine.github.io/setup/nodejs.html