React-testing-library: Previous render sometimes leaking into next test

[ljosberinn]

• @testing-library/react version: 10.3.0
• Testing Framework and version: jest 26.0.1
• DOM Environment: jsdom 16.2.2

Relevant code or config:

https://codesandbox.io/s/great-dawn-s229i?file=/src/ServiceWorker.test.tsx

it('renders a toast onupdatefound triggering site reload on click', async () => {
    const mockReload = jest.fn();

    Object.defineProperty(window, 'location', {
      value: {
        reload: mockReload,
      },
      writable: true,
    });

    makeMockRegister({
      addEventListener: makeMockAddEventListener(),
      registration: jest.fn(),
      shouldReject: false,
    });

    render(<ServiceWorker />);

    const toast = await screen.findByRole('alert');

    fireEvent.click(toast);

    expect(mockReload).toHaveBeenCalledTimes(1);
  });

What you did:

Trying to write tests in the order the ServiceWorker executes.

What happened:

screen.findByRole('alert') fails because multiple elements with that query are in the DOM.

Reproduction:

https://codesandbox.io/s/great-dawn-s229i?file=/src/ServiceWorker.test.tsx

Problem description:

If I disable all other tests except that one, it works.

Changing the order of the tests works too; the test marked as problematic test currently is the 5th/last.

It works locally if its the 4th:

Sandbox does not like the 4th...

...but as 3rd

Adding cleanup or manually calling unmount() does not help at all.

[JacobMGEvans]

I remember something like this happening a long while back in codesandbox, I think they had an issue of mine closed saying they fixed it. I will dig it up and see if it's related.

[JacobMGEvans]

Nope, it was a crash occurring. https://github.com/codesandbox/codesandbox-client/issues/3149

Has this only been attempted in codesandbox or in a separate environment too?

[ljosberinn]

We already talked but for visibility: the sandbox is just a reproduction of what I have locally; in this project.

[r3wt]

i've had this problem as well. we just change all of our tests to const {unmount} = render(...); then call unmount() at end of test.

[ljosberinn]

I tried that too and it didn't help, sadly.

[JacobMGEvans]

Do you have timers or anything async that isn't being resolved in the tests, because the test blocks will continue to execute which will cause them to leak into JSDOM when another test starts running?

example not having await before act() or waitFor() etc...

[ljosberinn]

So the initial issue technically still persists. However I was able to rewrite the test entirely and I don't have this issue anymore.

I suspect it has to be something to do with using Object.defineProperty and/or my usage, because when I used it in another test, I had the same issue of previous renders leaking into another test again. Removing said Object.defineProperty solved it however.

[mboettcher]

I'm running into the same issue that the previous render leaks into the next test. Sounds to me like a very bad thing because this could lead to false-positive tests.

[puneetgkaur]

I am running into the same issue. Version: 10.4.8.
If I run my test in isolation it succeeds, otherwise it fails.

[eps1lon]

For everybody commenting they have the same issue: Please include:

• exact @testing-library/react version
• exact react-dom version
• your test code

There's definitely a potential issue with React 17 (caused by deferred effect cleanup functions). Timers that aren't properly cleaned up can also cause this issue.

Overall these kind of issues are very hard to track down so simple "I have the same issue" are not helpful.

[FernandoBasso]

I have been having a similar problem. Here's a simple, reproducible case:

unit-tests-renders-leaking.zip

It is simple app that lazy loads two routes, Home and About. The tests are very short and simple too. There are only two tests, actually.

If I run only the first test in App.spec.js, then it works. If I run only the second test, it works. The problem arises when I run both, in which case it seems the render from the first test is leaking into the second one, like if they are not being run in isolation.

I tried manually calling cleanup() and even unmount() after each unit, but the results are the same.

Pasting the relevant code here:

App.spec.js:

describe("<App />", () => {
  it("should render the loading fallback", () => {
    const { container } = render(<App />);
    expect(container.textContent).toContain('Loading...');
  });

  it("should route and render the home page", async () => {
    const { debug, container } = render(<App />);
    expect(container.textContent).toContain('Loading...');

    await waitFor(() => {
      expect(container.textContent).toContain('Home Page');
      debug();
    });
  });
});

App.js

import React, { lazy, Suspense } from "react";
import { BrowserRouter as Router, Route, Switch } from "react-router-dom";
import "./styles.css";

const Home = lazy(() => import(/* webpackChunkName: 'Home' */ "./Home"));
const About = lazy(() => import(/* webpackChunkName: 'About' */ "./About"));

export default function App() {
  return (
    <div className="App">
      <Router>
        <Suspense fallback={<div>Loading...</div>}>
          <Switch>
            <Route exact path="/" component={Home} />
            <Route path="/about" component={About} />
          </Switch>
        </Suspense>
      </Router>
    </div>
  );
}

[MatanBobi]

Thanks @FernandoBasso for the code sample!
From what I can see, the DOM is being cleared between tests and the elements are being unmounted. The issue lies within the dynamic import and the lazy implementation.
On the first test, React's lazy implementation calls the dynamic import statement, at that time the lazy has a -1 status means Uninitialized. At the second render the status is already Resolved meaning that the Home Component is already there and no fallback is shown.
I tried to use jest.resetModules but that didn't help.
Though I'm not sure what's the meaning of this test (to render the App component), it's just testing React's lazy mechanism isn't it?
I'm not sure this is something we can change in our code, @nickmccurdy @kentcdodds @eps1lon does any of you have an idea about this one?

[eps1lon]

I suspect that it has to do with the lazy components. In the first test React tries to resolve them but they won't resolve in time so a fallback is displayed. In the second test they're already resolved so no fallback needs to be displayed.

I guess you'd have to reset the modules before each test. I think jest has an API for that.

[MatanBobi]

I suspect that it has to do with the lazy components. In the first test React tries to resolve them but they won't resolve in time so a fallback is displayed. In the second test they're already resolved so no fallback needs to be displayed.

I guess you'd have to reset the modules before each test. I think jest has an API for that.

I tried using jest.resetModules but that didn't work.. Maybe because it's dynamically imported it wasn't cleared from the modules registry? 🤔

[bigfoot31]

I think you guys are onto something. I too used lazy components in a project and had issues with react-testing library. Never managed to solve the problem. Hacked around it by writing test cases in different files. In a previous project without lazy components, I had no issues.

[r3wt]

fwiw, we use React.lazy and Suspense as well throughout all of our projects where we've had these issues.

[ljosberinn]

My initial issue was related to service worker registration, which also didn't use lazy or Suspense, but only fired the notification once the promise resolved, so it's promise-related for sure.

[eps1lon]

I suspect that it has to do with the lazy components. In the first test React tries to resolve them but they won't resolve in time so a fallback is displayed. In the second test they're already resolved so no fallback needs to be displayed.
I guess you'd have to reset the modules before each test. I think jest has an API for that.

I tried using jest.resetModules but that didn't work.. Maybe because it's dynamically imported it wasn't cleared from the modules registry?

We need to also re-require the modules when resetting:

describe("<App />", () => {
  let React;
  let cleanup;
  let render;
  let waitFor;
  let App;

  beforeEach(() => {
    jest.resetModules();

    App = require("./App").default;
    React = require("react");
    ({ cleanup, render, waitFor } = require("@testing-library/react"));
  });

  afterEach(() => {
    cleanup();
  });

  it("should render the loading fallback", () => {
    const { container } = render(<App />);
    expect(container.textContent).toContain("Loading...");
  });

  it("should route and render the home page", async () => {
    const { container } = render(<App />);
    expect(container.textContent).toContain("Loading...");

    await waitFor(() => {
      expect(container.textContent).toContain("Home Page");
    });
  });
});

Alternatively you need a single test for everything that follows a lazy component being resolved.

We only need to re-evaluate App.js in this specific case but this leads to multiple React modules throughout the component tree which is why I reset all react related modules.

The problem is that the lazy components itself are created when we evaluate App.js. In our first render their state is still Uninitialized. Though after that test their state is Resolved. Now in our second test React encounters lazy components that are already resolved and no longer needs to display a fallback.

It's not an issue with @testing-library/react but react itself. If you have an issue with testing React.lazy please open an issue in the React repository explaining your use case.