Vue-test-utils: Removing sync mode

[eddyerburgh]

After searching for an alternative solution, we've decided to remove synchronous updating (sync mode) from Vue Test Utils.

__tl;dr__

We will remove sync mode in the next beta version. Test code will change from this:

it('render text', (done) => {
    const wrapper = mount(TestComponent)
    wrapper.trigger('click')
    wrapper.text().toContain('some text')
})

To this:

it('render text', async () => {
    const wrapper = mount(TestComponent)
    wrapper.trigger('click')
    await Vue.nextTick()
    wrapper.text().toContain('some text')
})

Background

By default, Vue batches updates to run asynchronously (on the next "tick"). This is to prevent unnecessary DOM re-renders, and watcher computations (see the docs for more details).

When we started Vue Test Utils, we decided to make updates run synchronously. The reasoning was that synchronous tests are easier to write, and have improved error handling.

For context, the decision was made just after async/await had been released in node 7.6, so asynchronous tests often looked like this:

it('render text', (done) => {
    const wrapper = mount(TestComponent)
    wrapper.trigger('click')
    Vue.nextTick(() => {
        wrapper.text().toContain('some text')
        wrapper.trigger('click')
        Vue.nextTick(() => {
            wrapper.text().toContain('some different text')
            done()
        })
    })
})

Because of this decision Vue Test Utils runs updates synchronously by default. That was a mistake.

Why?

Sync mode causes bugs that don't exist when Vue runs normally. This is frustrating for users, and bad functionality for a testing framework that's intended to give you confidence that your code will work in production.

We went through three different approaches to implement sync mode. Each had problems. The final attempt was to reimplement synchronous updates in Vue core, but there were still bugs caused by synchronous updates.

Solution

The solution is to remove sync mode entirely from Vue Test Utils and rely on the user explicitly waiting for updates to be applied:

it('render text', async () => {
    const wrapper = mount(TestComponent)

    wrapper.trigger('click')
    await Vue.nextTick()

    wrapper.text().toContain('some text')

    wrapper.trigger('click')
    await Vue.nextTick()
    wrapper.text().toContain('some different text')
})

This will be a big change for many test suites. You have two choices:

1. Refactor tests to run asynchrnously
2. Keep Vue Test Utils locked at beta.28

We recommend that you update your tests in order to benefit from future features of Vue Test Utils. This will also make your tests more robust since Vue will perform updates in the same way as it does in production.

To make the migration as smooth as possible, so we'll provide documentation and guides to help you write tests asynchronously with Vue.nextTick.

Finally, I'm sorry for the work this change will require. I was a large part of the driving force for running in sync mode by default, and I underestimated the problems that it would cause.

Going forward, this will improve the stability of Vue Test Utils, and we'll be able to release a stable API as v1.

[dominik-bln]

Thanks for the explanation and better now than after v1.

Just as a thought, wouldn't recommending something like this be better to make it also work when localVue is used:

await wrapper.vm.$nextTick()

Or is that irrelevant?

[eddyerburgh]

You can use either, they're the same function

[ghost]

As much as I liked it, thank you for sharing the detailed explanation and changing it in the beta rather than causing more refactoring work later. Also thank you for creating an issue prior to changing it! ❤️

[sagalbot]

@eddyerburgh can you please adjust the release https://github.com/vuejs/vue-test-utils/releases/tag/v1.0.0-beta.29 and place this under breaking changes? There's BREAKING CHANGES but it's not listed under there and I didn't notice the text at the top.

[brennj]

Hey @eddyerburgh! I could be talking complete nonsense, but would it be possible to await that next tick in the wrapper.trigger to keep the test code cleaner?

i.e. instead of

wrapper.trigger('click');
await Vue.nextTick();

do

await wrapper.trigger('click');

[eddyerburgh]

Yes that's a good idea, I was thinking the same thing 👍

[ghost]

can you please adjust the release https://github.com/vuejs/vue-test-utils/releases/tag/v1.0.0-beta.29 and place this under breaking changes?

@sagalbot @eddyerburgh I don't think this was included in v1.0.0-beta.29: https://github.com/vuejs/vue-test-utils/compare/v1.0.0-beta.29...dev

https://github.com/vuejs/vue-test-utils/pull/1141 was merged on 9th of February while v1.0.0-beta.29 was already tagged on the 2nd.

[markrian]

await wrapper.trigger('click');

I feel like that's conflating two things: triggering the event, and flushing to the DOM. What if a tester wanted to do more than just trigger a click event before the next tick?

I'd suggest instead either sticking with the original trigger/await nextTick pattern, or adding a new method, something like:

await wrapper.triggerAndNextTick('click');

... but with a less terrible name!

[brennj]

Fair point. I would imagine there could be some interesting points of inspiration to take from the async utilities/patterns of dom-testing-library that is popular with the React flavor of react-testing-library. https://testing-library.com/docs/api-async

[satyavh]

What a change; this also broke wrapper.setProps({})
So before:

wrapper.setProps({
   prop: someValue,
});
expect(wrapper.vm.prop).toEqual(someValue);

would trigger changes (dom, watchers, computed, etc) in the component, but now you have to do

wrapper.setProps({
   prop: someValue,
});

await wrapper.vm.$nextTick();
expect(wrapper.vm.prop).toEqual(someValue);

Imo this change should be reflected in the docs asap!

[garyo]

Is this change in 1.0.0-beta.29? I'm fighting #1163 and trying to figure out the best way forward.

[ghost]

Is this change in 1.0.0-beta.29?

@garyo No, it is not—see also my comment above: https://github.com/vuejs/vue-test-utils/issues/1137#issuecomment-464737969

[dietergeerts]

What a mess, release notes are telling lies then.... I had to update the test-utils, and now I'm getting all sorts of errors on tests that where succeeding before. There is something really wrong with the DOM updates when ran with test-utils. Are there other things I can check to resolve these issues?

[ghost]

What a mess, release notes are telling lies then....

@dietergeerts I don't see this mentioned in the release notes. 🤔

Are there other things I can check to resolve these issues?

It probably helps to create a new issue with reproduction steps and the error message.

[dietergeerts]

https://github.com/vuejs/vue-test-utils/releases/tag/v1.0.0-beta.29

Use Vue async mode, this has caused bugs and will be reverted (see #1137)

It's very confusing

[ghost]

@eddyerburgh I agree with @dietergeerts that this entry in the release notes could have some clarification. It reads as if Use Vue async mode is handled by this issue while it is actually https://github.com/vuejs/vue-test-utils/pull/1062.

[dietergeerts]

Just to keep you guys up-to-date: When reverting to beta.28, all my problems are resolved.
The issue I had was that certain test cases where failing because other test cases where included in the run, for no apparent reason. it's very strange. We always use createLocalVue to have each test really isolated.

By using the test utils, are there any connection on global level?

[vvanpo]

await wrapper.trigger('click');

I feel like that's conflating two things: triggering the event, and flushing to the DOM. What if a tester wanted to do more than just trigger a click event before the next tick?

@markrian Would they not still be able to do that?

wrapper.trigger('click')
doSomethingElse()
await localVue.nextTick()

If they don't await the trigger, then everything should work as before, no?

[markrian]

@vvanpo They would still be able to do that, sure. But conceptually, it's mixing two different concepts. The trigger method, I would argue, is simply responsible for dispatching events, and (probably) should know nothing about the task queue/tick mechanism. IMO, the two concepts should remain separate, for simplicity.

[ChristophWalter]

Are there any updates on this? We just had a lot of troubles when testing with Vuetify and finally got the tests stable and independently after disabling the sync mode. This could have saved us a lot of effort...

[lmiller1990]

This is quite confusing now - when do we need to do await? For _every_ DOM update? I'm having a bit of trouble getting my head around what this change means - see here for an example of some confusion. When using vue-router, and doing router.push, this will change the DOM (obviously). So should I be doing await router.push()? Or is this a race condition (as the OP of that issue talks about).

[eddyerburgh]

Sync mode has now been removed in beta.30

[tmorehouse]

With beta.30 I'm having difficultly testing the <transition> afterLeave hook, All other hooks I can test, but for some reason can't get the afterLeave handler tested (i.e. we emit an event in the afterLeave hook, and no matter how many nextTick's we wait, that emitted event never appears as emitted (expect(wrapper.emitted('event-after-leave')).toBeDefined()). All other transition hook events fire ok (beforeEnter, afterEnter, etc) Note: we had the transition stub set to false in order to test the hooks previously)

Not sure if this is related to the sync changes or not.

[al-the-x]

We pinned to beta.29 to resolve until we could await all the right places. Totally appreciate the lib and contributions, and the removal of "sync mode" is a better design decision, TBH. A major version bump for this BC-breaking change would have been nicer.

We got hit with the [email protected] bug at the same time, so it was a Fun Time ™️

[RyanAtViceSoftware]

Trusting your users to be able to understand async testing and recommending tools like https://www.npmjs.com/package/wait-for-expect would probably go a long way to avoid some of the headaches caused here.

[stuheiss]

@eddyerburgh you say "sync mode has been released" - do you mean removed, reenabled, or something else? I was under the impression that sync mode was removed as of beta.29

[ghost]

@stuheiss sync mode was removed in v1.0.0-beta.30 (it's still available in v1.0.0-beta.29) and the docs have been updated.

@eddyerburgh should this issue be closed now?

[xiaosu]

With beta.30 I'm having difficultly testing the <transition> afterLeave hook, All other hooks I can test, but for some reason can't get the afterLeave handler tested (i.e. we emit an event in the afterLeave hook, and no matter how many nextTick's we wait, that emitted event never appears as emitted (expect(wrapper.emitted('event-after-leave')).toBeDefined()). All other transition hook events fire ok (beforeEnter, afterEnter, etc) Note: we had the transition stub set to false in order to test the hooks previously)

Not sure if this is related to the sync changes or not.

How do you stub transition in your tests?

[tmorehouse]

@xiaosu we were disabling the beta.29 transition stub (to use the real transition):

const wrapper = mount(SomeComponent, {
  stubs: {
    transition: false
  }
})

But now that transition is not subbed at all, it should just work as before, but the after-* hooks no longer run under beta.30

[kierans]

Is there a way to pin docs online at a certain version?

The reason I ask is that when using Vuetify, tests can blow up when running in sync mode usually with an error like TypeError: Cannot read property '$scopedSlots' of undefined. This is because when testing form validation DOM nodes are inserted/removed depending on the validation outcome. When testing synchronously some variable is undefined, whereas by waiting for the nextTick the DOM batch is processed and all is well. For those having the same issue with Vuetify see https://github.com/vuetifyjs/vuetify/issues/9151 or https://github.com/vuetifyjs/vuetify/issues/9820.

However when reading through the docs of @vue/test-utils I couldn't find any reference to sync yet my repo was still at beta.29. This was confusing for me when different forums posts on how to fix my tests from blowing up was to set a mounting option that didn't exist according to the docs. It took more googling to find this issue.

I support the idea of removing sync updates because it goes against the grain of Vue, and Node JS in general, so I'm glad that it's gone. However I think it would be helpful for people to be able to view the docs online for the version of the library they're using. Especially for breaking changes like this.

[dobromir-hristov]

The only way I see is to point to that version on Netlify.