Create-react-app: After upgrade to CRA 2.0, tests run very slow.

[Ektomorfik]

Is this a bug report?

I think so.

After upgrade to CRA 2.0 Jest tests run much slower than before.
I think it could be made by Jest 23

Did you try recovering your dependencies?

yes, but problem still exists

Which terms did you search for in User Guide?

Running Tests

Environment

Environment:
OS: Windows 10
Node: 8.11.2
Yarn: 1.10.1
npm: 6.4.1
Watchman: Not Found
Xcode: N/A
Android Studio: Not Found

Packages: (wanted => installed)
react: ^16.5.2 => 16.5.2
react-dom: ^16.5.2 => 16.5.2
react-scripts: 2.0.3 => 2.0.3

Steps to Reproduce

Tests in previous version reloaded less than 1s. After upgrade to CRA 2.0 tests reload in about 3-4sec. Problec occurs even if I create new empty project

1. create-react-app app
2. npm run test

Expected Behavior

It should be faster :(

Actual Behavior

Reproducible Demo

(Paste the link to an example project and exact instructions to reproduce the issue.)

[Timer]

Did you use the Node test environment previously? We switched the default to JSDOM which has some overhead.

[Ektomorfik]

I used default settings

[Ektomorfik]

Maybe it would help a bit. Some time ago, I setup next.js project with Jest 23 and I had the same issue. I solved problem by downgrade Jest to 22.x.x.

[gaearon]

cc @rogeliog is there a reason to expect slowdown in 23?

[rogeliog]

I don’t think so, any ideas @SimenB @thymikee?

[SimenB]

We've had reports of slowdown on windows, see e.g. https://github.com/facebook/jest/issues/6783

We haven't been able to pin it down, though, unfortunately...

[thymikee]

IIRC this was slowing down watch mode a bit: https://github.com/facebook/jest/pull/6647

[rogeliog]

@SimenB do you know if the issue is only affecting windows?
@Ektomorfik do they also run slower when you run them in "non watch mode"_(i.e CI=true npm run test)_

[bugzpodder]

I've had cases where I saw a big slow down on mac and not on watch mode. Two snapshot tests took over 30 seconds to run (and time varies on different runs). Will test it a bit more today and see if i can file something.

[rogeliog]

I set up a repo locally where I run 100 test suites to benchmark jest@21, jest@22 and jest@23. It seems that... All tests suites have this exact code.

describe('Simple assertions', () => {
  it('has a simple assertion', () => {
    expect(1).toBe(1);
  });

  it('using matchers', () => {
    expect({
      username: 'foo',
      password: '123456'
    }).toMatchObject(expect.objectContaining({
      username: 'foo'
    }));
  });
})

And I am running them all in non-watch mode. Jest@23 is running slower in my benchmark. After doing a lot of git bisect and I see a big performance hit introduced in this PR https://github.com/facebook/jest/pull/5932... Checking out a version right before that PR performs way better, any thoughts @cpojer?

[SimenB]

Hmmm, might be related to https://github.com/facebook/jest/issues/6694

Maybe we should look into caching module lookups?

[rogeliog]

By "related", you mean that facebook/jest#5932 might be the part of the root cause of facebook/jest#6694?

[SimenB]

Kinda. My point was that resolution is probably slow as we do it for every single modules in every single test. So anything adding work to the resolution by definition is super-hot code, and thus suspectible to performance regressions. Especially FS stuff as that's notoriously slow on Windows

[Ektomorfik]

@rogeliog - yes I tried do that. Problem still occurs :(

[someden]

Not only Windows. Test also run very slow inside node container, example:

$ cd ~/MyProject
$ docker run -p 3000:3000 -it --rm -v "$(pwd)":"$(pwd)" -w "$(pwd)" node:latest bash
$ yarn test
 PASS  src/__tests__/actions.test.js (17.432s)
 PASS  src/__tests__/reducers.test.js (18.061s)
...

[rogeliog]

I've been investigating the issue and created https://github.com/facebook/jest/issues/7110, but I don't think that is the specific reason for CRA... I'll keep investigating it.

[Ektomorfik]

@rogeliog Thanks so much :)
@Timer - Is there opportunity to speed it up? I mean eg. downgrade jest, custom config etc?

[mcmillion]

@someden are you also noticing long startup times in a node container?

I'm in the process of upgrading to Babel 7 and I'm now running across this. Outside the container tests are fast (<1s a piece), inside the suite takes 20-30s to start, and then each test takes 15-20s.

[someden]

@mcmillion Yep

[empz]

Fresh create-react-app (2.1.1).

 PASS  src/App.test.js
  √ renders without crashing (41ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        6.447s
Ran all test suites.

Watch Usage: Press w to show more.

So while the test actually seems to run pretty fast (41ms), the setup takes 6 seconds. Re-running doesn't make it faster.

Windows 10 x64, I7 4690K, SSD...

[lukaszczernal]

We also struggle with slow tests if we run:
react-scripts-ts test --env=jsdom

but if we run them with coverage flag they are way faster
react-scripts-ts test --env=jsdom --coverage

I understand that it should be the opposite :-)

We are running:
"react": "^16.2.0", "react-dom": "^16.2.0", "react-redux": "^5.0.6", "react-router-dom": "^4.2.2", "react-scripts-ts": "2.17.0", "enzyme-adapter-react-16": "^1.1.1",

[johnbowdenatfacet]

Confirmed, jest is running ~18 seconds with --coverage as opposed to ~28 seconds without... for 60 odd tests. Why so slow? And ~12 seconds with CI=true ...

[petetnt]

Could someone check if the issue still persists with Jest 24, shipped with CRA@3

[marius-ionescu]

Hey, have just tested this now and yes, there is a huge improvement:
(running in band)
w 2.1.8
Test Suites: 3 skipped, 98 passed, 98 of 101 total
Tests: 35 skipped, 379 passed, 414 total
Time: 646.448s
(running with coverage does bring some improvement but not that much)

w 3.0.0
Test Suites: 3 skipped, 98 passed, 98 of 101 total
Tests: 35 skipped, 379 passed, 414 total
Time: 139.081s
I've got some statistics without running in band, but it's mostly around the same for 3.0.0.

So the bump to 3.0.0 seems to have fixed this issue for windows(windows 10, w node 11.9.0)

[petetnt]

That's awesome @marius-ionescu!

For everyone else: closing this as improved on 3.0.0, but do still post if having similar issues and we can re-open this one.

[SimenB]

A bunch of performance improvements landed in 24.6.0, happy to see it translates to real gains!
https://github.com/facebook/jest/blob/4d3c1a187bd429fd8611f6b0f19e4aa486fa2a85/CHANGELOG.md#performance-2

[bugzpodder]

Thanks SimenB