Jest: Make jest small

[styfle]

🚀 Feature Proposal

Make jest small again (less than 30 MB)

Motivation

It appears jest is the largest when comparing other test harnesses.

[email protected] was small...it looks like [email protected] is when the size exploded.
Now [email protected] is even larger.

Example

|package|install size|
|-------:|-----------|
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |

Click a badge above to see a history of the install size.

Pitch

Every repo needs to install jest to run it's own tests.

One repo with 50 MB is maybe not too bad but your app is split into 200 microservices, that's 10 GB of jest!

[wtgtybhertgeghgtwtg]

It's reporting differently for [email protected].

But, yeah, there are definitely some downstream dependencies that could be trimmed.

[SimenB]

Happy to trim deps if anyone is willing to figure out which we can drop without feature or performance regressions 🙂 Or if our .npmignores are not aggressive enough

[wtgtybhertgeghgtwtg]

There's not that much can be done on jest's side, just downstream stuff that's already been rejected (get handlebars to stop using uglify, get jsdom to stop using request, get people to stop bundling CLI's).
https://github.com/facebook/jest/issues/6040 might help, though, but that's waiting on micromatch.

[SimenB]

The 46 mb jump was rough, but seems like a bug in counting. It's back to the level of 22, so maybe close this?

[styfle]

I updated the table in the original post above.

It looks like jest is still the largest so I think the issue is still relevant.

Update: also see jest dependency graph which might help identify unnecessary dependencies

[bugzpodder]

@styfle how do I add the package size badge in a github comment? Would like to use that elsewhere. ( https://github.com/facebook/create-react-app/issues/3880 )

Also, when I just try to add the [email protected] it shows up as 61MB on my system.

> du -hd1 node_modules/ | sort -h
844K    node_modules//async
888K    node_modules//uglify-js
912K    node_modules//jest-util
924K    node_modules//source-map-support
972K    node_modules//babel-runtime
1.2M    node_modules//escodegen
1.4M    node_modules//node-notifier
1.8M    node_modules//cssstyle
2.2M    node_modules//ajv
2.9M    node_modules//jsdom
3.3M    node_modules//handlebars
3.8M    node_modules//fsevents
4.8M    node_modules//lodash
7.3M    node_modules//core-js
 61M    node_modules/
> cat package.json 
{
  "name": "j2301",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "jest": "23.0.1"
  }
}

[styfle]

@bugzpodder If you click on the badge from the result page, it will display the markdown.

it shows up as 61MB on my system

Package Phobia is actually counting the raw bytes so it will be the smallest possible install size in a real world scenario. When you run npm install on your machine, there are many js files in a package that are smaller than the cluster/block size so they are actually increased to take up the full block.

For example, Windows default is 4 KB so a file smaller than 4 KB will take up 4 KB like this image.

Update: I created https://github.com/facebook/create-react-app/pull/4534 so you can see the difference between that PR and this PR

[rickhanlonii]

It looks like jest is still the largest so I think the issue is still relevant.

I think Jest does more out of the box than the others listed, so is this is a fair expectation?

[styfle]

@rickhanlonii Good question!

I haven't used jest besides going through the first few sections of the docs which appears to be pretty common among these frameworks: asserts, matching, async, setup/teardown.

I think the mocking feature and snapshot feature are the only additional parts that are shipped with jest that are not shipped with other test frameworks.

Were those features added in [email protected] when the size spiked?

[SimenB]

We've got built-in support for babel transpilation, code coverage, fake timers, parallelisation across CPUs and really powerful VCS integration which combined with the dependency tree only runs the test related to files changed from a given commit, instead of the entire suite.

That last one (in addition to clear errors) is probably my favorite.

(and that's not even mentioning the watch mode (see https://github.com/jest-community/jest-watch-typeahead), and custom runners, allowing you to run e.g. eslint in the same watch process)

[thymikee]

We also ship JSDOM by default.

[styfle]

Very cool!

I didn't see VCS integration listed in the docs, it sounds glorious 🙌

Back to my original comment...

[email protected] was small...it looks like [email protected] is when the size exploded.

I see now that the CHANGELOG.md shows that this is the first version published to npm so I image the versions prior are another package with the same name.

That being said I still think there's room for improvement. But for now I will continue to use the competition to test micro services.

[qwertie]

I came to the issues page to say something similar: it's okay with me if Jest is huge-by-default, but couldn't there be a dramatically smaller jest-lite? And hey, do I really need Babel in my TypeScript project?

Or, if the Jest core APIs are designed to be drop-in compatible with some other API, I would be more comfortable using that other API until such time as I have a project even remotely close to 1MB of source code, which might justify having a 44MB "on disk" test framework with 418 non-jest dependencies.

[thymikee]

And hey, do I really need Babel in my TypeScript project

If you want to have mocking through jest.mock() working with import statements, you pretty much have to use it.

[qwertie]

I've never used mocking but if I did, I'd be happy to settle for the kind of mocking that doesn't parse my source code.

| | |
|-|-|
| Based on size | I guess these don't. |
| TestDouble | |
| Sinon | |

[thymikee]

Maybe it makes sense to make that configurable with allowJestMock or some other flag? cc @SimenB @cpojer @aaronabramov

[SimenB]

We need babel for code coverage as well.

I honestly don't understand why people are so averse to large dependencies in a server side dev only dependency

[cpojer]

I don't think this issue is actionable as it is. I do not care for the absolute size of Jest compared to other test runners. If we can provide a neat solution for people that's slightly bigger, I'm ok with that. It won't bloat production apps. If we can find low-hanging fruit and are able to remove a large chunk of our largest deps (20%+), I'm on board with that. I'm not willing to make any product changes in Jest to save a few bytes here and there though.

[qwertie]

Maybe "retain source-level compatibility with a much smaller test library" (jasmine?) is an actionable item?

Let the other library be the Preact to your React.

[wtgtybhertgeghgtwtg]

You're gonna run into a lot of these things even if you go to another test framework. If you pull in nyc for coverage, you're gonna get babel and handlebars and it ends up not being a whole lot smaller than jest.

I honestly don't understand why people are so averse to large dependencies in a server side dev only dependency

Complex dependency trees can present some problems other than just size. For example, all the work that went into getting micromatch@3 working with jest, since a dependency of a dependency of a dependency had a dynamic require. It's also a pain in a corporate environment, where a deep dependency can get flagged by a proxy. That nearly kept us from using create-react-app at work. It can also cause slowdown, like globby has been experiencing.

Also, it just makes me mad.

[milesj]

To continue with what @wtgtybhertgeghgtwtg said, it also has a lot to do with transitive dependencies, any conflicting versions, and possible stale/outdated versions. Large deps exacerbate this issue a lot of the time.

[qwertie]

@wtgtybhertgeghg I question the assumptions that (1) everyone wants a code coverage tool regardless of its cost and (2) that the tool should be installed per-repo rather than globally.

(Ideally (2) could be fixed in npm itself by having some sort of "recommended global dependency" feature where a project.json can ask for a big tool to be installed globally so that when you have many small repos they can share the same large tool, without completely losing dependency information as occurs today where --global tools are not listed in package.json at all. Oh well...)

[milesj]

IMO, I think the best approach would be to split jest up into a few more packages. Like jest-coverage which has istanbul as a peer dep, and allows us to configure/upgrade istanbul according to our app. The same could be said for jsdom and the other major deps, because as of right now, we have to wait for Jest to upgrade them in a major to pull in new changes.

Also has the added benefit of reducing size as consumers pick and choose what they want.

[Magnitus-]

Judging by this thread, I think it's clear that Jest is primarily a frontend test library with everything baked in.

For frontend, an integrated transpiler makes sense I guess (I really try to avoid it for my own projects, but every company I worked for uses one, I guess it is fashionable), but when you're trying to test some small model library in Node.js for the backend, suddenly all those things that are baked in for the frontend don't really make sense.

Call me paranoid, but from a security perspective, I worry when I see a seemingly unconstrained dependency tree I got to wade though in order to use a library (and in this case, the dependency tree is not really analytically tractable I'm afraid).

Overall, seems like a really nice full featured batteries-included frontend testing library though (just not really for me).

[styfle]

@Magnitus- Good point! Check out this spiderweb https://npm.anvaka.com/#/view/2d/jest/

[wtgtybhertgeghgtwtg]

Test transpilation actually doesn't account for that much of the dependency tree. A good chunk of the babel stuff is being used in other parts, like for inline snapshots or formatting messages. I think micromatch accounts for more dependencies here than babel does.

[TrySound]

I agree that big dependency tree makes node running slow. In some cases it's good to bundle dependencies like rollup does to achieve as fast as it possible start.

/cc @jonschlinkert

[wtgtybhertgeghgtwtg]

The problem with bundling is that it creates duplication. While it'll be smaller and less to require and parse if that library and only that library is using those bundled dependencies, it'll be larger and more to parse if something else that's being used does. While it may be an option for CLI's (prettier, rollup), it'd likely be a large net increase in size for users if jest tried to do something like that.

[TrySound]

Well, I think bundling all jest-* packages in one may increase start time and reduce amount of dependencies. I'm not sure jest packages are widely used somewhere except jest itself.

[wtgtybhertgeghgtwtg]

I think we may be talking about different things. A lot of the packages jest uses (yargs, read-pkg, micromatch, rimraf, request, readable-stream, lodash) are widely used. Bundling them would likely cause net increase in size for stacks that use jest.

[TrySound]

I'm talking that it's not necessary to bundle all dependencies. Parts of the tree may be squashed. It should be a big win.

The other point is that source code of packages may be bundled to reduce node pressure. And all jest stuff may be considered as one package. Non jest stuff may still live in node modules.

[jonschlinkert]

@TrySound thanks for the cc.

I totally agree with the spirit of this issue and the goals you're trying to accomplish. Regarding micromatch specifically, our top priorities are reducing dependencies, initialization time, and bundle size.

I have been working on the next release of micromatch, and it's almost ready to publish - it already has parity with the current version, and is much faster in benchmarks and initialization time. I've also reduced code footprint down to two total dependencies, and neither dependency has any dependencies of its own.

My goal is to publish micromatch v4.0 this week, but I'm moving to a new home at the moment so it will be as soon as I can get to it. Hope this helps achieve at least part of the goal you have here.

edit: related tracking issue on micromatch

[natealcedo]

@Magnitus- I would actually like to argue against Jest being mainly used as a front end testing framework. I actually use it alot at work for both front-end and server side code. I love that it's batteries included. Granted if you want something light you could always go down the path of using Mocha.

On that note, my pain point with using something like Mocha though is that if your project gets sufficiently large enough, you end up having a tonne of other dependencies as well that come out of the box with Jest. (Chai, sinon, rewire, istanbul, etc). In a sense, you're building your own test-environment from scratch.

I like to think about it similar to how the react community was initially: everyone building their own boiler plates. When create-react-app came out most of these boilerplates started dwindling down because most developers don't really want to fiddle with configuration. They want to be productive cause that's what they get paid to be and that's what Jest aims to be from what I've seen. Delightful Javascript Testing

Having said that, I don't really see an issue with Jest being a large-ish dependency. Most of us have pretty sweet rigs anyways. :)

[mikl]

Wow, yes, I just added Jest to a Node.js project that does not use Babel (backend only), and just adding Jest added ~5000 lines to my package.lock. All of Babel and oodles and oodles of strange dependencies like AJV, Husky, is-number and its dozens of siblings, request with its own zillion dependencies, uglify-js, etc. etc.

I guess much of that won't even get loaded under normal circumstances, but it feels a bit wasteful to burden npms servers with all that.

[wtgtybhertgeghgtwtg]

Very little of that is because of babel, and even less of it is under the control of jest.

• ajv is from request, which jsdom uses (although they're willing to change that).
• is-number and friends are from micromatch (which should heavily deduplicate them in the next version).
• uglify-js comes from the CLI part of handlebars (which comes from istanbul-reports which comes from istanbul-api), and they are not interested in moving that to a different package.

husky isn't a dependency of jest, I don't know why you're seeing that.

[mikl]

@wtgtybhertgeghgtwtg I guess most of the cruft comes from upstream dependencies, which makes it a lot harder to fix. Thanks for you for your efforts nonetheless.

[techieshark]

One note on @styfle's point:

One repo with 50 MB is maybe not too bad but your app is split into 200 microservices, that's 10 GB of jest!

I assume the author is aware, but other folks who find this issue may be able to minimize this problem by using pnpm which aims to be a disk-space efficient package manager.

[styfle]

@techieshark Yes pnpm could help some in this case. But I would imagine the reason for using microservices is to have independent dependencies with independent versions for each microservice. So your mileage may vary (so to speak) with pnpm.

[probil]

It's getting bigger

| version | size |
| --- | --- |
| 24.8.0 | |

jest-lite is 2 times smaller

| version | size |
| --- | --- |
| 1.0.0-alpha.4 | |

[SimenB]

24.6 had the typescript migration, so the tarballs now includes both a type definition and type definition sourcemap (essentially doubling the number of files we ship, although they're not as big as actual JS code). So it's 100% expected, though I guess unfortunate, that the size went up. It also added a bunch of @types/ dependencies

---

Our stance hasn't changed since https://github.com/facebook/jest/issues/6266#issuecomment-399017224 - PRs reducing size without reducing features or making the code more complex is very much welcome. We'll be landing micromatch 4 in Jest 25 which should remove some of the size - there might be other changes we could make as well

[cztomsik]

Ok so maybe this is dumb proposition but jsdom could be peerDependency, and error could be thrown if it is requested and not present in the current project. That could strip quite a lot of deps for node.js projects (microservices, etc.)

[jeysal]

That would be possible, yes, but I would only consider it if node becomes the default environment (which I'd like to do for performance reasons). Right now it'd hurt the 'works out of the box' experience too much.

[cztomsik]

I personally don't mind setting node in my jest config in exchange of less dependencies.

[SimenB]

We have precedence for that (--detect-leaks only works if you manually install weak-napi), but I'm not sure it makes sense as JSDOM's such an integral part of Jest's "batteries included" approach. Relying on people to install the correct versions of JSDOM seems gnarly and error prone when it's such a common use case.
(It absolutely does not make sense as long as jsdom is the default, but that'll hopefully change in Jest 26 anyways)

[StringEpsilon]

You could make node-notifier a peer dependency for test result notifications.

That would net 6 packages less and 5.5mb install size.

It also doesn't strike me of an "batteries included" feature. Still breaking though.

[SimenB]

Yeah, that would be easier! It's actually already optional, so easier code change than others

[cpojer]

Jest 26 will begin to lastingly reduce Jest's package weight: https://github.com/facebook/jest/pull/9950

[styfle]

Nice work! I can already see a small difference already in the 26 alpha:

|version|install size|
|-------:|-----------|
| [email protected] | |
| [email protected] | |
| [email protected] | |
| [email protected] | |

[ibezkrovnyi]

Disclaimer: I know that flat mode of organizing dependencies is most used,
I just wanted to notice that beside size of de-duplicated tree of dependencies there is also complexity for package manager to manipulate such huge graph of dependencies.

To check that I have added only [email protected] to pakage.json and did npm i --legacy-bundling:

| numbers
--- | ---
number of package.json files inside jest/node_modules | 2163
size of jest folder | 127,468,507 bytes

Did quick comparison to other packages we use (sorted):
package name | number of total dependencies before de-duplication
--- | ---
[email protected] | 2163
[email protected] | 673
[email protected] | 896
@babel/[email protected] | 515
[email protected] | 300
(subjective) we also have package with a lot of dependencies related to webdriverio, selenium, etc | 2133

Total number of dependencies before de-duplication we have in ui projects is over 10,000 (and similar number of added packages is reported by [email protected] after successful npm install).
package-lock.json size is ~5 mb

• npm dedupe already hangs
• npm install takes ages to complete

adding --max-old-space-size doesn't help

Unfortunately we can't change package manager, but even if yarn/pnpm is able to manage such dependency graphs much faster, even then complexity of keeping de-duplicated tree of dependencies is still there.

[probil]

@ibezkrovnyi Thanks for the idea regarding npm i --legacy-bundling.

I've just checked the latest version and seems like situation improved in jest@26 but not drastically

# jest v25.5.0
npm init --yes
npm i -D [email protected]
du -sh ./node_modules  

+ [email protected]
added 506 packages from 346 contributors and audited 506 packages in 18.644s
found 0 vulnerabilities

 53M    ./node_modules
----------------

# jest v26.0.1
npm init --yes
npm i -D [email protected]
du -sh ./node_modules/

+ [email protected]
added 504 packages from 346 contributors and audited 504 packages in 20.415s
found 0 vulnerabilities

 50M    ./node_modules/

Here is the list of "leaders":

u -shc ./node_modules/* | sort -rh | head -10
 50M    total
6.3M    ./node_modules/node-notifier
5.4M    ./node_modules/@babel
4.8M    ./node_modules/lodash
3.6M    ./node_modules/jsdom
1.2M    ./node_modules/@types
1.1M    ./node_modules/ajv
1.1M    ./node_modules/acorn
996K    ./node_modules/snapdragon
948K    ./node_modules/@jest

[cpojer]

Let's remove node-notifier and make it an optional plugin. I don't know of anyone who uses it, and the people who do can install a separate package to hook it in. Does anyone want to work on a PR to make it its own package that will be used only when installed (and explicitly enabled)?

Note it is already an optional dependency (and at FB we blackhole it to an empty package).

[cpojer]

@SimenB is removing sane in https://github.com/facebook/jest/pull/10048.

[SimenB]

Note it is already an optional dependency (and at FB we blackhole it to an empty package).

Yeah, it should just be a matter of moving it from optionalDependencies to devDependencies and tweaking the error message to tell people to install it. We do this for weak-napi already

[probil]

Here is approx. files count per dependency (top 10):

$ du -a | cut -d/ -f2 | sort | uniq -c | sort -nr | head -10
1051 lodash
 528 @babel
 485 jsdom
 217 @jest
 144 resolve
 144 @types
 126 cssstyle
 100 ajv
  99 node-notifier
  86 jest-snapshot

BTW, can't jsdom be made optional or peer? It's FE only, 13Mb in node_modules, uses deprecated [email protected]

[cpojer]

@probil JSDOM will become optional in Jest 28: https://jestjs.io/blog/2020/05/05/jest-26

Right now it's the default, so Jest 27 will make the node env the default, and then 28 will stop shipping it.

Note that for the monorepo at Facebook where we develop React Native, we blackhole jest-environment-jsdom to an empty module.

[probil]

@cpojer Cool! Thanks for update