Storybook: provide better example for async snapshot testing

[macrozone]

Is your feature request related to a problem? Please describe.

the readme of storyshots mentions a way to do async snapshot testing, e.g. if you use apollo or siilar: https://github.com/storybookjs/storybook/tree/master/addons/storyshots/storyshots-core#storyshots-for-async-rendered-components

There are two problems with that example:

• it uses enzyme, while default storybook uses react-test-renderer. If you just migrate to that solution, all your snapshot will be different
• enzyme-to-json will render all properties (at least im my case). This is not what you want, you want to diff the resulting html. it also seems to run out of memory on larger stories. I think this is because i use a decorator that provides the apolloclient

Describe the solution you'd like

I'd like to see a way to configure this async rendering with the default storybook renderer

Describe alternatives you've considered

Houff... i tried many things... Best solution so far is this (using diffable-html and enzym's html())

import initStoryshots, { Stories2SnapsConverter } from "@storybook/addon-storyshots";
import { shallow, mount } from "enzyme";
import { act } from "react-dom/test-utils";
import "jest-styled-components";
import toDiffableHtml from "diffable-html";
import preloadAll from "jest-next-dynamic";

beforeAll(async () => {
  await preloadAll();
});

const waitForNextTick = () => new Promise((resolve) => setTimeout(resolve));

initStoryshots({
  asyncJest: true,
  test: async ({ story, context, done }) => {
    const converter = new Stories2SnapsConverter();
    const snapshotFilename = converter.getSnapshotFileName(context);
    const storyElement = story.render();
    const tree = mount(storyElement);
    await act(async () => {
      await waitForNextTick();
    });
    tree.update();
    if (snapshotFilename) {
      expect(toDiffableHtml(tree.html())).toMatchSpecificSnapshot(snapshotFilename);
    }
    done();
  },
});

I hope that if async rendering lands in react, these problems will be solved...

Are you able to assist bring the feature to reality?

yes, but i want to gather better solutions here (if any) first.

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[krishnaglick]

Would love a better solution for this!

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

no! bad stale-bot!

...

ok, you are just doing your job... but... has no one in the whole storybook-comunity found a better way for asyns snaphost testing? :-/

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[krishnaglick]

Bumping this, bloody stalebot.

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

no! stalebot! we are not done here!

[macrozone]

i want to bump this because i can't believe that there is no better way. I am looking for a solution for over a year now and i am very astonished that no one came up with a solution.

my workaround has a lot of downsides, e.g. jest-styled-components does not work with it and the diff is slow and hard to parse.

[krishnaglick]

https://github.com/storybookjs/storybook/tree/master/addons/storyshots/storyshots-puppeteer
Looks like this could work for it. I spent a little bit of time trying to but ran into trouble. May revisit when I've got more time.

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[krishnaglick]

Bump

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

shoo stale bot

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

sorry, @stale, its still an issue! And i also still have not found a good solution. Waiting a tick all the time also increases time for all snapshots, so in bigger projects, this add significant time for tests

[sbaruth]

Would love to see a solution to this as well.

[tobiaswahyudi]

+1 to the stalebot shooing community!

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

dear stalebot, usually i would send in PRs, but i really have no clue how to do it better.

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

stalebot, please #stayhome

[chengyin]

I got this working with react-test-renderer, a simplified version of my config:

import initStoryshots, {
  Stories2SnapsConverter,
} from '@storybook/addon-storyshots';
import { act, create } from 'react-test-renderer';

const wait = () =>
  act(
    () =>
      new Promise(resolve => {
        setTimeout(resolve, 10);
      }),
  );

const converter = new Stories2SnapsConverter();

const runTest = async (story, context) => {
  const filename = converter.getSnapshotFileName(context);

  if (!filename) {
    return;
  }

  const storyElement = story.render();
  let tree;
  act(() => {
    tree = create(storyElement);
  });

  await wait();
  expect(tree.toJSON()).toMatchSpecificSnapshot(filename);

  tree.unmount();
};

initStoryshots({
  asyncJest: true,
  test: ({ story, context, done }) => {
    runTest(story, context).then(done);
  },
});

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[krishnaglick]

Not done with this yet... Still need to try the above async solution. Or for storybook to give us one.

[krishnaglick]

I get a ton of errors about running act twice using the above sample.

[chengyin]

What are your react and react-test-renderer versions?

[krishnaglick]

Test Renderer: 16.13.1
React: 16.13.1

[krishnaglick]

I was able to chop down on my issue and things appear to be working. Biggest complaint is wait's timeout is 1000, but I'll take it.

[chengyin]

The timeout is probably tied to the specific implementation. We have been using 10ms and it has been working for us. I don't recall any wrong results since implemented (end of April).

[ericsoco]

@krishnaglick or other googlers, to prevent the act warnings you have to be sure to wrap the delay Promise in act(); otherwise, component updates can fire after the test fn exits but before jest cleans up the async test.

This is the adaptation of @chengyin's suggestion that's working for me:

const converter = new Stories2SnapsConverter();

async function wait(delay) {
  return act(
    () =>
      new Promise(resolve => {
        setTimeout(resolve, delay);
      })
  );
}

// Align with snapshotWithOptions implementation
// https://github.com/storybookjs/storybook/blob/master/addons/storyshots/storyshots-core/src/test-bodies.ts#L23
function assertSnapshot(renderer, filename) {
  if (filename) {
    expect(renderer.toJSON()).toMatchSpecificSnapshot(filename);
  } else {
    expect(renderer).toMatchSnapshot();
  }
}

async function runAsyncTest(story, context) {
  const filename = converter.getSnapshotFileName(context);

  const storyElement = story.render();
  let renderer;
  act(() => {
    renderer = create(storyElement);
  });

  // For Flow's benefit
  if (!renderer) return;

  // Let one render cycle pass before rendering snapshot
  await wait(0);
  assertSnapshot(renderer, filename);

  renderer.unmount();
}

// Run synchronous tests
initStoryshots({
  storyKindRegex: /^((?!.*?__async__).)*$/,
  test: multiSnapshotWithOptions({}),
});

// Run async tests
initStoryshots({
  storyKindRegex: /__async__/,
  test: ({story, context, done}) => runAsyncTest(story, context).then(done),
});

[krishnaglick]

Oh, thanks for the ping.
I discovered what the issue was: one of my storybook tests was using react-meta-tags, which appears to use react-test-renderer or some other utility under the hood to render metatags. That was firing and conflicting. Once I mocked that module things worked out!

[sdevalapurkar-bcgov]

I got this working with react-test-renderer, a simplified version of my config:

import initStoryshots, {
  Stories2SnapsConverter,
} from '@storybook/addon-storyshots';
import { act, create } from 'react-test-renderer';

const wait = () =>
  act(
    () =>
      new Promise(resolve => {
        setTimeout(resolve, 10);
      }),
  );

const converter = new Stories2SnapsConverter();

const runTest = async (story, context) => {
  const filename = converter.getSnapshotFileName(context);

  if (!filename) {
    return;
  }

  const storyElement = story.render();
  let tree;
  act(() => {
    tree = create(storyElement);
  });

  await wait();
  expect(tree.toJSON()).toMatchSpecificSnapshot(filename);

  tree.unmount();
};

initStoryshots({
  asyncJest: true,
  test: ({ story, context, done }) => {
    runTest(story, context).then(done);
  },
});

Thank you! This works great

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[macrozone]

no solution worked good for me so far, so no stale bot, go away

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[jdelStrother]

Stay out of this, stalebot.

[jdelStrother]

For what it's worth, we were seeing this error/warning when using useEffect -

    Warning: An update to MeasuredDiv ran an effect, but was not wrapped in act(...).

    When testing, code that causes React state updates should be wrapped into act(...):

but all our storybook stories are wrapped in <StrictMode>. Removing StrictMode sidesteps the issue, which is probably good enough for our snapshots, but it would be nice to see an official mechanism for supporting act/useEffect.

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[annaloukianova]

Our current snapshot setup is maybe a little funky,

initStoryshots({
  test: multiSnapshotWithOptions({
    createNodeMock: (node) => document.createElement(node.type),
    integrityOptions: { cwd: __dirname },
  }),
  stories2snapsConverter: new Stories2SnapsConverter({
    snapshotsDirName: './snapshots',
    snapshotExtension: '.snap.js',
  }),
});

Any ideas on how to use async while also passing in various snapshot options (without forking stoyshots)?

[krishnaglick]

If all you're looking to do is control the names you can do:

const storyPath = converter.getSnapshotFileName(context);
// Do things to storyPath
expect(tree!.toJSON()).toMatchSpecificSnapshot(storyPath);

[annaloukianova]

I'm using a custom stories2snapsconverter which seems to take care of that, but as far as I can tell I need the node mock (probably integrity options too, I'm not sure) and I can't seem to get that working with the above solution (copied for clarity:)

initStoryshots({
  asyncJest: true,
  test: ({ story, context, done }) => {
    runTest(story, context).then(done);
  },
});

[stale[bot]]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[rhermes62]

This is still an issue and would be nice to have an official solution as a part of StoryShots so sorry stale bot.

[hubgit]

Does this seem reasonable?

import initStoryshots, { Stories2SnapsConverter } from '@storybook/addon-storyshots'
import { act, create } from 'react-test-renderer'

const converter = new Stories2SnapsConverter()

initStoryshots({
  asyncJest: true,
  test: async ({ story, context, done }) => {
    const filename = converter.getSnapshotFileName(context)

    if (!filename) {
      return
    }

    // render the component
    const renderer = create(story.render())

    // wait for state changes, wrapped in act
    await act(() => new Promise((resolve) => setTimeout(resolve)))

    expect(renderer.toJSON()).toMatchSpecificSnapshot(filename)

    renderer.unmount()

    done()
  },
})

[rodgoeliz]

Does this seem reasonable?

import initStoryshots, { Stories2SnapsConverter } from '@storybook/addon-storyshots'
import { act, create } from 'react-test-renderer'

const converter = new Stories2SnapsConverter()

initStoryshots({
  asyncJest: true,
  test: async ({ story, context, done }) => {
    const filename = converter.getSnapshotFileName(context)

    if (!filename) {
      return
    }

    // render the component
    const renderer = create(story.render())

    // wait for state changes, wrapped in act
    await act(() => new Promise((resolve) => setTimeout(resolve)))

    expect(renderer.toJSON()).toMatchSpecificSnapshot(filename)

    renderer.unmount()

    done()
  },
})

Seems okey by me, thank you. you are awesome.
I think these are the types.

import initStoryshots, { Stories2SnapsConverter } from '@storybook/addon-storyshots';
import { StoryContext } from '@storybook/react/types-6-0';
import { act, create, ReactTestRenderer } from 'react-test-renderer';

const wait = () => act(() => new Promise((resolve) => setTimeout(resolve, 10)));

const converter = new Stories2SnapsConverter();

const runTest = async (story: StoryContext, context: StoryContext) => {
  const filename = converter.getSnapshotFileName(context);

  if (!filename) return;

  const storyElement = story.render();
  let tree: ReactTestRenderer | undefined;
  act(() => {
    tree = create(storyElement) as ReactTestRenderer;
  });

  await wait();

  expect(tree?.toJSON()).toMatchSpecificSnapshot(filename);
  tree?.unmount();
};

initStoryshots({
  asyncJest: true,
  test: ({ story, context, done }) => {
    runTest(story, context).then(done);
  },
});

[macrozone]

problem is, that toJSON is not a good idea as it may contain also code from the decorator of the story (and in my case completly breaks because of memory problem).

the default does not seem to use toJSON, i could not figure out how the default works. We just need to have the default snapshot mechanism with the additional wait()