Following the forum thread, using this issue to kick off work on an official unit testing utility lib. We are planning to built on top of @eddyerburgh 's Avoriaz, since it seems to be the most mature one currently, and Edd has kindly offered to work on it. Of course, we would also like to pull in ideas from other community solutions and make this a collaborative effort, so I'm pinging some community members that might also be interested in contributing here: @callumacrae, @asselin, @jackmellis, @codebryo, @BosNaufal.
I think the first step is outlining what feature set we are aiming for - here's some high level goals:
in the meanwhile, feel free to provide general feedback on some of the needs you've run into while unit testing Vue.
Also @vuejs/collaborators for any potential input.
The main question for me is should the test utils use a wrapper API, augmented API or simple API?
Wrapper API
const wrapper = mount(Component)
// Access vm
wrapper.vm.someMethod()
// Helper methods
wrapper.find('div')
Augmented API
const augmentedVm = mount(Component)
// Access vm
augmentedVm.someMethod()
// Helper methods
augmentedVm.$find('div')
Simple API
// Access vm
getMountedVm(Component)
// Helper methods
findElement('div', Component)
avoriaz uses the wrapper API. Personally I think it's the best API for this library - it avoids possible collisions that can occur with the augmented API and produces readable and chainable code.
What are people thoughts?
@eddyerburgh I prefer wrapper. Simple API can be tedious to import all the methods.
Thank @eddyerburgh 's great library! I have read avoriaz' doc and it is well done.
Another field worth testing is the events a component emits. For example, a custom input component will likely emit an update event.
It is reasonable for a user to expect utility for testing custom event in the standard toolkit.
@HerringtonDarkholme Good idea. What would the API look like for that? wrapper.didEmit('update')?
On another note. If we add a find method - should find return an array of wrappers, or a container class?
A container class could default to the first item:
// Array
wrapper.find('div')[0].dispatch('click')
// Container
wrapper.find('div').dispatch('click')
And throw an error message if no element is found:
// Array
wrapper.find('div')[0].is('div') // throws Uncaught TypeError: Cannot read property 'is' of undefined
// Container
wrapper.find('div').is('div') // throws Error, no element matching tag div was found
The downside to a container is it needs a custom API to access wrappers at different indices.
// Array
wrapper.find('div')[1].is('div')
// Container
wrapper.find('div').at(1).is('div')
@pearofducks suggested having two find methods:
wrapper.find('div') - returns the first wrapper found, like querySelector
wrapper.findAll('div') - returns an array of wrappers, like querySelectorAll
What do people think?
@eddyerburgh can't the returned wrapper behave like jQuery collections? They have methods but are somewhat array-like (can be accessed via indices and even have iteration methods). That said, .at(1) isn't that bad.
@eddyerburgh call me old school, but I still feel that jQuery is the most convenient dom traversal library.
For wrapper.find('.my-element').is('div'), I would expect it means "for all elements with class my-element, they are HTMLDivElement". For wrapper.find('div').dispatch('click'), I would expect click events are dispatched to all divs.
That means, any predicate or method is applied on all elements in the collection. If the user want _one_ specific element to test they can simply use at(index). Also, filter or some, like those utility functions in underscore, can be helpful.
What do you think?
@yyx990803 Yes, it could return an object with properties 0,1,2 etc. But you will get type errors if a wrapper doesn't exist at an index:
wrapper.find('div')[3].find('p') // throws Uncaught TypeError: Cannot read property 'is' of undefined
If we used a method like at to access the wrapper, we could throw a custom error message
wrapper.find('div').at(3).find('p') // throws No div tag exists at index 3
It would be nice to throw as many custom error messages as possible. But I'm not sure if it justifies having a more complicated API.
@HerringtonDarkholme I hadn't considered treating the wrapper like that, but it could definitely work.
We should go with an API that's most natural for users. I think a lot of users will be familiar with the jQuery API, so maybe we should use a similar one as you suggest.
I get the feeling I'm the only person who doesn't do a lot of DOM traversal in their Unit Tests!
Most of my tests focus around the output of computed properties and methods using different data/prop values. I would leave checking the html layout of my page down to integration testing. Is this not how people test their Vue components?
Personally this is where I feel the wrapper API adds one-too-many layers between the user and the code-under-test. Is there some way we could either integrate or separate these two areas?
On top of the above list (which is all very sensible stuff) I would add the following to my wishlist:
Vue.use(plugin) without it polluting the global environment and leaking between tests. Some sort of isolated Vue constructor perhaps? <input> element)I like the idea of just rendering to the virtual dom (but it still be explorable). But would this cause issues with components that do this.$el.querySelector(..) etc. ?
@jackmellis This is interesting. Add ability to mock/inject methods, vuex store & lifecycle hooks to my wishlist.
@eddyerburgh yeah, I think a jQuery-like-object would be natural for most users, but you make a good point about [1] vs. .at(1) - I'm actually leaning towards .at(1) because of the error handling aspect.
@jackmellis good points - with the wrapper API you still have access to the actual vm inside though, so you can just use the wrapper API for stubbing purposes.
I've made a draft API to get some ideas down - https://github.com/eddyerburgh/vue-test-utils-proposal/blob/master/API.md
@eddyerburgh Great! I like the consistency of is, did, has prefix to indicate they are assertion method!
I have several questions:
the slot option in mount and shallow seems to accept a ComponentOption. But in VueJS runtime slot contains an array of VNode. Does it mean vue-test-util will inflate option to vnode? Or should we change the slot option to accept VNode instead?
It seems global is a little bit confusing to me. The doc refers global as Vue.globalMethod. $route is called instance method. Maybe another word like interceptis better? (inject is already used in VueJS)
provide/inject is an advanced feature but also desirable here. A provide option must be useful.
didEmit returns a boolean. But sometimes we also need to test event's payload. So returning the event might be better?
@jackmellis I actually like to do unit tests with rendering. Tools like Jest that support a snapshot feature are great for checking the output.
@eddyerburgh when mentioning jQuery, a thing that I felt was most useful and a big win for jquerys popularity is sizzle. What about leveraging sizzle for finding so people can so things like div:first and stuff? Besides that, the container method seems a little nicer as it could be extended with more functionality longterm.
API:
I like the cleaner mount approach of the proposal as this allows for a lot of flexibility.
Regarding the didEmit event, I would also add a function to test the payload, so it could be a second method didEmitWith for example.
The more methods we can provide, that people can even guess when working with the better.
Regarding setProps - https://github.com/eddyerburgh/vue-test-utils-proposal/blob/master/API.md#setpropsprops
It's always hard to make the right call on this kind of things. For one, it would probably fit a lot of cases that it automatically updates but sometimes it's maybe desired to not update for some reason, then it would help to have it manually.
So if the workflow would look like this:
wrapper.setProps({some: 'data'}).update()
I would prefer it as it leaves all control up to the user and while testing code, the more control you have over things the better.
I want also to note that update in Vue is async by default.
A waitForUpdate like test util appears in Vue source repo too.
https://github.com/vuejs/vue/blob/dev/test/unit/features/component/component-slot.spec.js#L41
Yes so wouldn't the updating of props require you to wait for the next tick anyway? To me this implies manually calling update is not necessary, as long as you somehow wait for the async updates to occurr. Otherwise, would you expect wrapper.setData to require an update before any changes are reflected in the component?
@HerringtonDarkholme
re: update:
It would be good to have a synchronous update function for tests. From reading the docs, async updates are important to avoid re renders. This isn't as important in tests. I think having a synchronous update function is more useful than avoiding unnecessary renders.
In avoriaz we force render the function by updating it with a re rendered vm.
function update(vm) {
vm._update(vm._render());
}
This might be a naive way of doing so. Perhaps someone with better knowledge of Vue core could advise.
@codebryo Agreed, it would be great to use sizzle. I tried to implement with avoriaz, but couldn't think of a way to make it work with vNodes and components. Although I'm sure there is a way to use it.
I noticed the async-ness issue. In Vue's own unit tests, I want to ensure the exact same behavior with user code, so we preserved the async-ness in tests. But for userland component testing, especially unit-testing, I think having to deal with the async-ness would be more of a annoyance. Vue used to have a global config option async which can make all updates sync, but I removed it to prevent users from relying on it too much. Maybe we can bring it back as an internal API for testing purposes.
@codebryo can you give an example of a case where you don't want the component to update after changing props?
#shallow - Default depth can be 1 but it should be configurable.
@eddyerburgh I actually did not test sizzle with vNodes yet, but I am sure it should be doable somehow :D. Maybe it's a thing for later though. When the container supports all major things it's not so critical.
The proposed solution for the update is good enough for testing imo.
@yyx990803 a crazy example I saw when reviewing a clients work: (Sorry I can't share the code)
ref input element in the dom that gets filled through external JS and kicks of an extra method call based on the result.Well, that code is arguable not great, but it was a case that heavily relied on the async update behavior.
As that is probably a very special case and if everyone feels autoupdating of the html is the exptected behavior it's good as well :)
Okay so the Vuenit library has some mocking utilities for Vuex and Vue-Router. I've spent some time extracting them into standalone packages so they can be library-agnostic. Can they be integrated into vue-test-utils?
https://github.com/jackmellis/mock-vue-router
https://github.com/jackmellis/mock-vuex
@codebryo if that's not a common case it should be trivial to just temporarily turn async on during specific tests.
@jackmellis that's great - although I'm wondering what's preventing your from injecting a real router/store created during the test?
@yyx990803
Vuex/router require a fair amount of configuration to get going, the mock libraries don't require any, it will just set some stubbed/dummy config for anything you omit.
The real libraries carry with them quite a bit of overhead that you often don't need or care about when you just want an object that looks like vue-router, for example.
The mock libraries also contain some useful methods for testing, like vuex where you can do store.expect('dispatch', 'myAction') to easily test that an action is dispatched.
In my experience, I've spent too much time trying to set up vuex and vue-router when setting up my component, when really i don't need 90% its functionality. The mock libraries came out of me writing many many tests that required too much setup to get working and thinking 'I wish I just had a dummy $router object available'
It would be nice to throw as many custom error messages as possible. But I'm not sure if it justifies having a more complicated API.
having to write .at(1) instead of [1] is a fair tradeoff for the error handling ๐
@HerringtonDarkholme About the waitForUpdate, I took vue's helper and changed it a bit for vue-mdc tests. I made it support returning promises inside of the callbacks(I think...) https://github.com/posva/vue-mdc/blob/master/test/helpers/wait-for-update.js#L10
Good job on the api draft ๐
Edit: What do you think of adding an exist check:
expect(wrapped.find('.foo')).to.exist()
// or
wrappend.find('.foo').should.exist()
You probably know https://github.com/nathanboktae/chai-dom/, that's where I got the idea from
@posva exists is a good idea ๐
Although a chai plugin should be a separate project in my opinion. exists should be added as a wrapper method, and a chai plugin project can add the assertion you added.
// with vue-test-utils
wrapper.find('div').exists() // returns true
// with vue-test-utils-chai-plugin
expect(wrapped.find('.foo')).to.exist()
@jackmellis I can't follow 100% on the store. Mocking is usually quite easy to do for the functions you wanne test ,and if you want to include certain functionality (like a vue component working with a Vuex store) it's nice that it works when needed.
For the $router though I can see a lot of value in the generally mocked version.
@posva exists is great.
@eddyerburgh Do you think when using the container and find methods, that something to generally trigger interactions would be helpful?
wrapper.find('button').trigger('click')
that would then allow things like:
expect(mockedFn).toHaveBeenCalled() and similar.
@codebryo in the draft there is dispatch which does what you want.
At the moment it works for native events or custom events, but I'm wondering whether it should be split into two methods:
wrapper.dispatch('click')
wrapper.emit('custom')
I also think trigger might be a better name than dispatch ๐ค
@eddyerburgh I have actually mistaken dispatch in this case for the Vue - Vuex communication (what does not make sense at this point) or the already deprecated $dispatch.
For some reason dispatching interactions never clicked with me. So for me personally trigger makes kind of more sense:
// list to touch events on mobile
trigger('touch')
// trigger potential more complex events
trigger('keyPress', 'e')
So considering dispatch is mostly in use to fire actions in the Vuex store it kind of feels weird to combine it with elements logic.
@yyx990803 @eddyerburgh Glad to see this kicked off! I've been away for the US holiday weekend, but have (just a few :) thoughts to add:
IMO, I don't think we should shy away from rendering to a real DOM, particularly since there are implementations like JSDOM that are pretty lightweight. My thinking is that if someone wants to traverse the DOM to test if it rendered correctly, or to stimulate a component by triggering events on specific nodes, then we should recommend rendering to a DOM. This gets us completely out of the business of querying/traversing DOM, and lets test writers use their preferred traversal and assertion libraries (e.g. chai-dom, chai-jquery, Sizzle, etc.). This would make the API much smaller; we could eliminate contains, find, html, is, isEmpty, text, and trigger, and only have to provide 1 method to retrieve the root DOM node. The only methods we'd have to provide on the wrapped component are methods that deal specifically with Vue component stuff.
Conceptually, I believe the mount method should allow instantiating a component with all the same options that can be passed into a 'real' component instantiation. So 2 changes that I think should be made to mount are:
slots should also accept a string of HTMLdidEmit method). The functions in most cases would probably be Sinon spys and could leverage the full power of Sinon assertions. IMO, this is better because 1. It makes testing of emitted events and passed in callback functions use the same style of code 2. The spys can test not just that an event was emitted, but also that it was emitted with the correct parameters too.A couple of specifics on the API:
isVueInstance. Wouldn't that always return true?setData is wise. A component's data is private to itself. It's OK for a test to inspect it, but not for a test to change it externally. One method that's missing is a manual way to force Vue's nextTick. This is required for components that queue something to happen with Vue.nextTick() in response to a stimulus, and the unit test needs to make the callback runs before asserting something.
Going along with the above, I think that the setProps API should not automatically force an update/render. One of the uses for a unit test library is to be able to reproduce bugs. One class of bugs is related to async update behavior. IMO we should have the capability to exactly reproduce the steps of a bug, which means that we need to be able to e.g. update props on interacting components, force a tick to happen, update some more props, etc.
Should there be a setSlots API to change the value of a component's slots? I'm a bit fuzzy on how Vue works here-- can the value of a slot change, or will it always be the same after instantiation, but a component in a slot can render different HTML?
One thing that I think might be useful is to have a Mock/Spy Component ala Sinon. The idea is that you could pass this MockComponent to the component under test in a slot. It would have methods on it to tell you how many times it was instantiated (for testing v-for in a component), be able to emit events, etc. What do you guys think?
@asselin
IMO, I don't think we should shy away from rendering to a real DOM, particularly since there are implementations like JSDOM that are pretty lightweight.
There are times (e.g. router components/pages) when a component may contain many children but I'd be testing the component in units, shallow render makes sense, tests would be faster.
test writers use their preferred traversal and assertion libraries (e.g. chai-dom, chai-jquery, Sizzle, etc.)
But I'd like to use chai-dom (I'm familiar), ๐ on that, we should be able to use preferred traversal and assertion libraries.
slots should also accept a string of HTML
That would be nice when a component accepts many named slots. ๐
...event names and functions (this would replace the didEmit method). The functions in most cases would probably be Sinon spys
Using spys is good but didEmit is also required, sometimes I just care an event is emitted; other times I may want to check event payload is correct.
IMO keep both. ๐
to have a Mock/Spy Component ala Sinon
๐ it is great.
@asselin thanks for your thoughts, a lot of what you're saying rings true with me:
This gets us completely out of the business of querying/traversing DOM, and lets test writers use their preferred traversal and assertion libraries
I absolutely agree with this, there are so many ways to traverse the dom, it feels a little bit like we're basically just rewriting jQuery with a couple of vue-specifics. However, we are definitely in the minority on this so I'm not gonna fight for it!
slots should also accept a string of HTML
Again agreed, slots are often not just components. I would suggest that the slots option should allow either a string (with any number of slots), a hash of slot names and strings, and a hash or slot names and objects.
{
slots : {
footer : Compnent,
header : '<h1>Just some html content</h1>'
}
}
To test emitted events, it should accept a hash of event names and functions (this would replace the didEmit method).
I think this would be useful but I think it should compliment didEmit, not replace it.
I'm not sure I understand the purpose of isVueInstance. Wouldn't that always return true?
Using the traversal methods you can return a html element that is still wrapped in all of these helper methods. So you can search for an element with a given class, and then check if that element is actually the root of a component.
I don't think including setData is wise. A component's data is private to itself. It's OK for a test to inspect it, but not for a test to change it externally.
Disagree with this one. The whole point of unit tests is to observe the behaviour of something under given conditions. I don't want to go through a whole tangle of other methods that I'm not testing in order to get my data correct for my test scenario, when I could just go wrapper.setData({loading : true}).
One method that's missing is a manual way to force Vue's nextTick.
I believe this is the wrapper.update() method
One thing that I think might be useful is to have a Mock/Spy Component ala Sinon.
This is an absolute yes, I use this kind of component-stubbing all the time to simulate events emitted by a complex child component. Something like this
One thing I'd like to sort out is VDOM vs DOM and full vs shallow. They are actually orthogonal:
Shallow means we don't render child components (or only render child components to a certain depth). However this can be done by rendering to either VDOM or real DOM.
VDOM: simply render a component's VDOM tree with vm._render(). The tree consists of Vue's VNode format and are pure JavaScript objects. This is shallow by default, and very cheap/fast. The downside is we need to implement custom support for querying and value retrieval of the tree (not too hard actually, we just need a CSS selector matching library)
DOM: actually complete the full render, turning the VDOM tree into real DOM trees. This is obviously heavier, but it actually works with both a real browser and JSDOM. JSDOM is heavier than Vue's VDOM but still much lighter than an actual browser. It also provides all the querying/traversal/value retrieval API that the DOM already has, so it might be a good tradeoff. In addition, test runners like Jest has JSDOM enabled by default, so that makes testing using this approach very straightforward. The downside is shallow rendering is more tricky to implement.
I think both are valuable, the question is how hard it is to keep the API of both modes as close as possible.
String slots and stubbed components is a great idea, I'll add them to the proposal.
@asselin We should not use a real DOM in shallow. Even JSDOM is expensive. A big problem with Vue unit tests currently is how long they take to run.
@jackmellis @asselin We won't be traversing the DOM, we'll be traversing the virtual DOM. This means wrappers of DOM nodes are actually wrappers of vNodes.
I see traversal methods as a fundamental part of the vue-test-utils.
They give a clean API for common Vue specific assertions:
wrapper.find(MyComponent).at(1).hasProp('bar')
wrapper.find(MyComponent).length
wrapper.contains(MyComponent)
And for common behavior, like triggering events. We can trigger events with custom libraries right now. But it's verbose, and not accessible to people who don't know about the libraries.
// with simulant
const event = simulant( 'click');
const target = vm.$el.getElementById( 'target' );
simulant.fire( target, event );
// with vue-test-utils
wrapper.find('div').at(2).trigger('click')
I've made a draft API to get some ideas down - https://github.com/eddyerburgh/vue-test-utils-proposal/blob/master/API.md
This looks really good!
The only thing I'd change is the didEmit object - it seems like an abstraction too far for the library, and means you can't see how many times the event was emitted and what the payload was. Seems to me that it would be better to have the ability to specify event listeners and leave the actual testing logic in userland (e.g. with sinon).
Two approaches to that:
@yyx990703 @codebryo re vuex, it's a little beside the point by now I think, but I created a quick gist to show the difference between using Vuex and mock-vuex based on an example from the Avoriaz docs.
https://gist.github.com/jackmellis/1bdc016e3c1f69b057284ca4a7dc762c
Also consider tests where you want to test a computed property or method that returns different values based on state changes.
https://gist.github.com/jackmellis/0c903896879eafdd17628a8f11a3e662
If you change the state of a Vuex instance, you'll get warnings about mutating the state, but in your tests you don't mind mutating the state outside of the component, in fact it's often necessary.
@yyx990803 @eddyerburgh re shallow rendering. I've got an implementation for shallow rendering to the DOM by intercepting child components and replacing them with a dummy component. It works fine in my experience. I think saying "you can only shallow render to the VDOM" is a bit presumptuous as I have had many components that need to access the DOM but I also don't want to fully render their children during tests.
I think this discussion does make me wonder exactly what are the most common tests people write? As I mentioned earlier, I don't do a lot of "does component contain child component" or "does component render element with this class" tests. Most of my tests are "does computed property return correct value", "does calling this method send an ajax request", "does clicking this button call this method (without actually testing the method itself)". Obviously I do test that certain things are rendered in certain scenarios, but most of my logic is within methods and computed properties so that's what I test. Perhaps part of this design should be understanding common test cases?
@jackmellis
I don't think including setData is wise.
Disagree with this one. ... I don't want to go through a whole tangle of other methods that I'm not testing in order to get my data correct for my test scenario
OK, good point, I agree.
I think this discussion does make me wonder exactly what are the most common tests people write?
Do you have any components and unit tests that you could post publicly as a starting point for 1. generating the use cases we need to cover 2. when we're comparing API options, visually seeing what the effect would be on unit tests? I'll go dig around and see what I can come up with that I can share (and certainly if anyone else has anything, feel free to chime in).
They're old (~8 months) and incomplete, but here's some unit tests I wrote with vue-test: https://gist.github.com/callumacrae/237f3cc589b967ccd169e83b5ec5b03b
TIL it has been 8 months since I've had enough time to write a unit test ๐ณ
@asselin There are lots of examples of people using avoriaz to test on github - https://github.com/search?utf8=%E2%9C%93&q=avoriaz+mount&type=Code.
And some tests for a clock component I wrote - https://github.com/eddyerburgh/vue-digital-clock/blob/master/test/unit/specs/Clock.spec.js
@znck @eddyerburgh @yyx990803 Before dismissing JSDOM as too slow, I think a performance comparison is warranted. What I'm afraid of is if we go the route of including DOM/vDOM traversal functions, the API is either going to be incomplete, or there's going to be a LOT of code to write to make it complete (see http://airbnb.io/enzyme/docs/api/shallow.html and related sections for example, to see how many APIs Enzyme provides to React developers to traverse DOM/vDOM for unit tests).
The question really boils down to this: Will a new API provide a significant benefit over the existing general purpose solutions? There's a lot of downside to consider:
IMO, it'd be better to tell folks they can use whatever existing library they prefer (QSA, Sizzle, jQuery), and then concentrate on the Vue-specific things.
@asselin ill have to see how much I'm allowed to borrow as I'm working on a closed source project, I may be able to just do some pseudo code based on things I've tested.
In the meantime, I have some example test scenarios for very simple components,some just require dom traversal but some go beyond that. The best example is probably this one:
https://github.com/jackmellis/vuenit-examples/blob/master/src/components/create-a-thing.vue
https://github.com/jackmellis/vuenit-examples/blob/master/spec/components/create-a-thing.spec.js
Where i want to test dispatches and ajax posts. Note that I've tested the button click that kicks it all off separately.
So how would you normally Test such a component? It's only a simple form but it definitely goes beyond dom traversal!
@asselin a performance comparison is a good idea.
You raise some valid points.
Will a new API provide a significant benefit over the existing general purpose solutions?
I think yes.
In answer to your concerns:
Also, custom traversing the VDOM will be faster than fully rendering to a DOM. This is a huge advantage in my eyes.
If we've collected a minimal list of features/API requirements, then we should start the implementation. I think we would get better feedback/insights if we've something concrete to try on.
I've got some time over the next few days, I can volunteer to start pulling together some code (unless @eddyerburgh you want to start). We can start with Avoriaz, and I can add the stuff @callumacrae and I did on vue-test and vue-cordova-template around events and webpack-mocha-jsdom setup.
I think creating illustrative use cases from some of the unit tests folks have volunteered above is also a high priority as we continue to discuss the API.
@yyx990803 What is your preferred workflow? As this is still a bit of exploration, do you want us to start the work in another repo, and then move it here after it stabilizes a bit, or work in here?
Thanks!
@asselin If we're pulling in avoriaz, I'd like to do it ๐ . It'd probably be quicker as I already know the codebase.
use cases sounds like a good idea, although I've not got experience using them. Can you kick off the discussion in an issue and explain what form they should be in and how to use them?
@jackmellis with avoriaz, I'd mock the dependencies using globals:
test('submit sends router update', async t => {
const thing = {id: 7}
const $http = {
post: sinon.stub().resolves({})
}
const $store = {
dispatch: sinon.stub().resolves(thing)
}
const $router = {
push: sinon.stub()
}
const wrapper = mount(CreateAThing, {
globals: { $http, $store, $router }
})
await wrapper.vm.submit()
t.true($router.push.calledWith(`/things/${thing.id}`))
})
@eddyerburgh Try make it usable from the very beginning. So, we would be able to get quick feedback.
Let me know when it's at a point that I can contribute some code / help out :)
@eddyerburgh
Regarding performance, I now use Jest for our code base and run quite a lot of unit tests, that use JSDOM. As everything runs on node it's really fast. So I don't have any performance issues that would cross out JSDOM as tool. Had problems with mocha/chai and phantomjs, but that's also not working with vDOM as phantom won't support it.
For tests, I am happy to provide a bunch of different specs.
In general how to work with Jest it would look something like this for mocking functions:
// mock the get function of a $http object
$http.get = jest.fn(() => 42)
test('add magic number', () => {
let result = wrapper.vm.magicSum(23)
expect($http.get).toHaveBeenCalled()
expect(result).toBe(23 + 42)
})
// some HTML heavy components
test('component', () => {
let wrapper = mount(Comp)
expect(wrapper.html).toMatchSnapshot()
wrapper.setData('foo': 'bar').update()
expect(wrapper.html).toMatchSnapshot() // should now contain the new rendered html
})
Hi guys,
Glad to see this happening. Just last week I just started putting together notes and use cases for what I'd like to make sure that I know how to test in vue as preparation for upgrading an application to vue 2. I'm bringing in concepts from my past that aren't super vue-like here and there, so any feedback would be appreciated.
In particular the models and controllers categories are a bit strange in vue, but I think that there is still model and controller like behavior that needs to be testable, even if it's in a vue and even if vue takes care of some of it automatically. It doesn't currently categorize unit vs end to end either. I'm just thinking in general, what are things that often need to be tested. It seems to me that the end to end stuff is kind of like normal, and that it's the unit tests that need the most help now, but the end goal is just a good blend of tests.
And finally I'm wondering whether any of these libraries or this newly planned one will consider it a goal to work with 1.0.28? I would be happy to help on weekends with that if that is the case.
Thanks for your efforts!
I think e2e testing is probably completely out of the scope of this library - there's nothing library specific in that kind of testing, usually.
Secondly, whether any of these libraries or this newly planned one will consider it a goal to work with 1.0.28? I would be happy to help on weekends with that if that is the case.
vue-test supports Vue 1, not sure about Avoriaz. However, I'll be deprecating it in favour of vue-test-utils when it's done.
Yes what I meant was that some of those things in the list are not going to be relevant as I was thinking of an overall application not just vue components, though since vue takes on some model and controller responsibilities, the traditional list of things just for unit tests will be too small (as this group of libraries shows by their methods already) I think vue-test's description is right on for the goal, a component testing library. Testing drag ordering or drag selecting a group of children are good targets for how a component and it's children need to be able to be tested, which is why I'd also like to be able to test the real dom.
I've added the mount method from the API proposal.
find returns a WrapperArray object.
A WrapperArray has all the methods of a Wrapper. By default, it will call methods on every Wrapper inside the array:
import { compileToFunctions } from 'vue-template-compiler'
const compiled = compileToFunctions('<div><p class="p-class"></p><p id="p-id"></p></div>')
const wrapper = mount(compiled)
wrapper.find('p').hasClass('p-class') // returns false
wrapper.find('p').hasAttribute('id', 'p-id') // returns false
wrapper.find('p').isEmpty() // returns true
You can select a Wrapper by using at
wrapper.find('div').at(0) // returns Wrapper
wrapper.find('p').at(0).hasClass('p-class') // returns true
WrapperArray contains a length property:
wrapper.find('div').length // returns number of wrappers in WrapperArray
Some methods can only be called on a single Wrapper:
wrapper.find('p').find('div') // throws Error - must be called on a single wrapper, use at(i) to access a wrapper
wrapper.find('p').at(0) // returns WrapperArray
find uses sizzle to match elements. This mean you can use any valid sizzle selector in the find method:
wrapper.find('div > .foo:first-of-type ~ #bar [name="value"]')
Mount is missing the didEmit method. It will be difficult to implement as a wrapper method.
TO DO
docs:publish scriptWrapperArrayVue.use. Maybe a Vue.clean method?EventListener object?@eddyerburgh would like to help on this project. Is there a best place to pitch in?
Here is a project where we have a ton of coverage on the components with some inspiration tools from a mix of element-ui, vue's internal and probably a half dozen other places.
Utils we have, provides about all we ned to accomplish stuff
https://github.com/doximity/vue-genome/blob/master/test/unit/util.js
Usage examples
https://github.com/doximity/vue-genome/blob/master/test/unit/specs/components/autocomplete/index.spec.js
@eddyburgh looks great so far!
I think the find/html/name/text methods should be available without having to call at first. It would feel much more intuitive to write wrapper.find('#p-id').text(), especially as you will often only be looking to retrieve a single element anyway. In that case, perhaps wrapper methods should indeed act on the first matched element?
The at method helps with error messaging for sure, but I would still like to expose index properties to make it easier to treat the result as an array. I think it should also have array-like methods like forEach, map, reduce, etc.
I'm not convinced about the EventListener API, it looks convoluted for what it is. The simplest (and current) solution for checking for events is surely wrapper.vm.$on('event', spy) and spy.calledWith(payload) which works fine for most cases. It would be cool if we could intercept Vue's event emitting system somehow and be able to assert that any event has been emitted by a component, but something tells me that would require some changes to Vue itself.
The Vue.use situation is a major issue for me. At the minute the only way you can register a plugin with is on a global scale, there's no way to isolate this to a specific component, meaning you have to have it in your entire test suite. This definitely makes it harder to keep unit tests atomic, especially when working with asynchronous tests where a Vue.clean method at the end of the test would cause even more issues. I'm not sure of a solution to this, except a change to the core Vue process.
@Austio PRs dealing with any items from the TO DO would be helpful ๐
@jackmellis I agree that wrapper.find('#p-id').text() should default to the first item.
But I think the behavior should be consistent. I think it would be confusing for users if some methods call every item, and some default to the first item.
The at method helps with error messaging for sure, but I would still like to expose index properties to make it easier to treat the result as an array.
I think it's best to keep the API simple. What use cases can you see that would benefit from array properties?
I think it should also have array-like methods like forEach, map, reduce, etc.
Agreed, although where does it stop? Do we include all array methods?
I'm not convinced about the EventListener API, it looks convoluted for what it is.
Ok, maybe we could add an on method? Or should we just leave it up to users to use wrapper.vm.$on?
The Vue.use situation is a major issue for me
I'm going to make another issue for this
It's certainly not a deal breaker, I just thought if you are returning an array-like object, people will expect it to have an array api, with indexed elements etc.
On seconds thoughts, I think it's more important to have the iteration methods forEach, map, filter, and indexOf/includes (would these 2 need to accept css/component selectors?)
Have you ever looked at how backbone achieves this for collections? I think they did a really great job on it.
Was looking at API to build out. Wanted to discuss but I don't think we should have a setData method. To me, the whole reason of having components is to pass or provide props.
What scenario would require setData?
@Austio I'm not sure I follow. I would say 98% of my unit tests require me to set my component's data before making any assertions.
The simplest example I can think of is a component that shows an error handler when something goes wrong. The errorMessage property is only set during some complicated method that calls out to various other parts of my codebase:
<template>
<div>
<error-component v-if="errorMessage" :message="errorMessage"/>
<!-- some actual component stuff -->
</div>
</template>
<script>
export default {
data(){
return { errorMessage : '' };
},
methods : {
doSomeStuff(){
this.$http('/api').then(response => {
return this.$store.dispatch('someAction', response.data)
})
.then(() => {
return calloutToSomeExternalAsyncCode();
})
.catch(err => {
this.errorMessage = err.message;
});
}
}
}
</script>
Now obviously I will test that method at some point, but testing whether the error message component is rendered shouldn't need me to set up a mock http, a mock vuex store, intercept an external method, etc. Not when my test can be as simple as:
t.false(wrapper.contains(ErrorComponent);
wrapper.setData({errorMessage : 'uhoh'});
t.true(wrapper.contains(ErrorComponent);
that means I can focus my tests on one thing at a time, which is kind of the point of them being units.
@jackmellis @eddyerburgh
My opinion is that when testing a component (which is essentially a function) you should treat it as a black box and test the inputs and side effects. Inputs being DOM events, props and slots, side effects being things like emitted events or dom changes.
Setting the data directly is reaching into the functions black box and at that point you are testing how something is happening instead of what is happening. From a app design perspective, I also think setting internal data directly in tests encourages coupling all of the concerns into a single component instead of separating them.
I can go into more detail if needed but may make sense to split this into a separate issue for discussion? Already nearly 60 comments on this one :)
You see i see unit testing components (especially complex higher-level ones) should be whiteboxed. But i don't think black box testing of entire components is bad, I just think it shows that this library should not force its users to test in a certain way, as every application and component and user is different. Vue-test-utils should be as unopinionated as possible... in my opinion!
Any particular reason you're choosing to use Sizzle instead of querySelectorAll for .find()? It seems to me like the only reason to do that would be to add support for browsers that Vue itself doesn't support.
@callumacrae Good point, I used it because I saw there are inconsistencies between browsers implementing matches.
But you make a valid point. It would be better to not have sizzle as a dependency. The tests still pass in chrome and phantom using elm.matches. Maybe we should use matches instead
Hm, didn't know about browser inconsistencies. Are you talking about the API differences (matches vs matchesSelector) or behavioural differences?
The API differences, but we can use the polyfill on the mdn page
Sizzle or an alternative pure-js selector matching lib might be needed if we end up implementing the same interface for pure-vdom shallow render. I don't think the dependency would be a big problem because the tests are run in Node most of the time.
Btw regarding whether the ArrayWrapper should support methods on a single element - how about we make the API similar to querySelector and querySelectorAll - i.e.
.find() returns a single wrapper.findAll() returns an array wrapperI think locating a single element would be much more common, and if we can't make the single wrapper and array wrapper share all methods, then it would actually be simpler to just separate the API methods.
@yyx990803 Agreed on find and findAll. I'll implement it later ๐
@callumacrae there are some selectors like :first or :visible that are supported by sizzle but not querySelector API, so sizzle is very convenient imo.
@yyx990803 absolutely agree. That would then make the .at() function clearer as one would probably not expect it to be used with the .find() method.
@eddyerburgh nothing more to say than great work already :) <3
About selector API:
After skimming sizzle's source, I think sizzle make deep assumption that it is run on DOM environment. To reuse sizzle for its special selector would require many native dom methods implemented on VNode. Also, selectors like :visibledo not make sense in a shallow rendering. Sizzle itself might not be a good candidate in vue-test-util situation.
We might need something like https://github.com/chrisdickinson/cssauron, which provide a css parser and hooks for user to implement different Node traversal method. (The library itself seems not actively maintained, though)
There are some interesting ideas in vue-unit, created by @wrseward. I especially like build() and buildShallow() and the port of vue's waitForUpdate(). Are there plans on introducing these features?
@nachocab, build looks great. I'd be happy to add it to the library ๐
The update function should mean there's no need to add waitForUpdate, but maybe there are use cases where it would be needed
@eddyerburgh I've tried using update (as documented here), but I don't think it works as a replacement for Vue.nextTick().then(() => ...).then(done, done). This is my use case: https://stackoverflow.com/a/44443847/355567
Ahh, I didn't know vue-unit existed, looks like a pretty well thought out lib too.
/cc @wrseward it would be great to have some feedback from you and join forces with us :)
Would love to see something to mock global event bus related things.
I consider the event hub pattern to be an escape hatch for certain edge cases rather than a recommended pattern for general use.
It brings the same, if not worse, problems that $dispatch() did, and I regularly see people on SO and our forum encountering problems with it.
For that reason, I think we should not offer any API tailored to this pattern.
What are the rest of you thinking about the event hub?
I haven't ever found I need it (there's almost always an alternative solution). On the other hand, if there is a design pattern that is featured in the official documentation, and that is sometimes unavoidable, I think it would only be fair to consider providing a way to test it.
In my opinion, this library, while not encouraging bad practices, should not punish people for using event buses and the like. The purpose is to make it easy to test vue apps in general, not to force a specific design pattern. Vue's impartiality on this (unlike, say, react) is what makes it so appealing to newcomers.
a design pattern that is featured in the official documentation
It is currently mentioned in the migration guide only, but yes, it is being mentioned.
My experience with reading and/or answering many questions on SO and our own forum is that people tend to use this pattern for the simplest problems because it seems to be an easy solution - people tend not to see the problems of reasoning about the apps event and data flow that event busses bring into their apps.
For that reason I think we should discourage the use in the documentation, or at least write few warning words next to it. If e can do that, then support in the test tools would be fine with me. Otherwise, we would further encourage a pattern that, in my experience from working with the community, is already vastly over- or even abused.
@LinusBorg Can you give some examples of event hub usage that could hurt or block rather than help?
Honestly, I barely had the need to use it because it couples components but I think if you define it as a property on the Vue prototype it's pretty easy to mock it or use it without many restrictions
I don't think that this issue is the right place to go into too much detail. The general grievances i have with the event bus is that it requires a lot of mental overhead for any but the most simple scenarios (and the simple ones generally can be solved easily through normal props).
The developer has to keep track of which events might change state where and when, because the application structure doesn't really tell you. The "when" is especially problematic because you essentially need to keep in mind when which component might be destroyed or re-created and could potentially miss an update from the event bus that it needs.
All of this is based in the fact that it's, well, event-based and not state-based like the rest of the patterns we use in Vue.
So of course the Event bus is not a problem (but not necessary either) if you use it to communicate between two not too distant components that are guaranteed to always exist together , at the same time. But it rarely stays with that, nd tends to become a mess.
@yyx990803 @eddyerburgh I would be very happy to collaborate on this! Thank you :+1:
I think there are some great ideas here and want some time read over everything before giving feedback
One thing I'd love to see is utilities that allow the testing of single file components without Webpack. When writing unit tests Webpack definitely gets in the way of simple easy test scripts such as those using tape.
To get things running with Avoriaz originally I had to spend a great deal of time writing a webpack file runner (similar to babel-node) that webpacks a file (or glob) and runs it in a node process. I'd really like to be able to avoid that. I'm not sure how much effort might be involved in supporting this.
@chrisnicola yes this is an area I feel strongly about. In this day and age, passing testing code through webpack in order to test it in a node environment feels like a hack. This isn't limited to Vue but Single File Components makes it harder to overcome. I would suggest looking at require-extension-hooks.
I would love it if the official docs could do away with karma/webpack advice, is the introduction of an official testing utility the right opportunity?
@jackmellis @chrisnicola Seeing your comments inspired me to clean up some work I had done earlier to make unit testing a little better. While it's not what you're asking for, what I did was remove karma & phantomJS, and replace it with mocha-webpack and JSDOM. See issue https://github.com/vuejs-templates/webpack/issues/875 and PR https://github.com/vuejs-templates/webpack/pull/874.
With this change, you still need webpack, but npm run unit will execute faster, and I added npm run unit:watch which will use webpack's watcher to rerun unit tests in the background whenever you make a code change.
I think mocha-webpack is great, but is quite complex (I looked at the code when I was figuring out how I was going to get tape working with webpack) and obviously only works if you use mocha.
I ended up creating https://www.npmjs.com/package/wprun to support general running of test files through webpack. It still isn't my ideal solution however. I'm going to look at require-extension-hooks a bit more I think.
I recommend using Jest with jest-vue-preprocessor. Jest caches compiled components, runs faster than mocha-webpack and has better support.
There are two problems with Jest
@eddyerburgh one other huge pitfall with jest is the inability to get a debugger in the code. This has been an issue for like 2ish years right now and it makes it very difficult for less experienced js developers to do unit testing. We recently switched out component testing from jest back to karma for this very reason.
I think that the concept of testing outside of webpack is great, but I don't think it should be a focus of vue test utils. I think in leiu of that we should provide direct hookin to the vue-cli generators and setup tests for hacknews example so that others have real examples.
This is fixed as of node 8.4
On Tue, Aug 29, 2017 at 11:21 AM, Austin notifications@github.com wrote:
@eddyerburgh https://github.com/eddyerburgh one other huge pitfall with
jest is the inability to get a debugger in the code. This has been an issue
for like 2ish years right now and it makes it very difficult for less
experienced js developers to do unit testing. We recently switched out
component testing from jest back to karma for this very reason.I think that the concept of testing outside of webpack is great, but I
don't think it should be a focus of vue test utils. I think in leiu of that
we should provide direct hookin to the vue-cli generators and setup tests
for hacknews example so that others have real examples.โ
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/vuejs/vue-test-utils/issues/1#issuecomment-325698820,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACounSNq0UIDSBdUPA4Yljox05V5Usmks5sdCyFgaJpZM4NooPe
.
For reference my launch.json looks like this:
{
// Use IntelliSense to learn about possible Node.js debug attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit:
https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Jest Tests",
"program": "${workspaceRoot}/node_modules/.bin/jest",
"args": [
"-i"
],
"runtimeExecutable": "/path/to/node-8.4.0",
"internalConsoleOptions": "openOnSessionStart",
"outFiles": [
"${workspaceRoot}/dist/**/*"
]
}
]
}
On Tue, Aug 29, 2017 at 11:22 AM, Avi Block atblock@gmail.com wrote:
This is fixed as of node 8.4
On Tue, Aug 29, 2017 at 11:21 AM, Austin notifications@github.com wrote:
@eddyerburgh https://github.com/eddyerburgh one other huge pitfall
with jest is the inability to get a debugger in the code. This has been an
issue for like 2ish years right now and it makes it very difficult for less
experienced js developers to do unit testing. We recently switched out
component testing from jest back to karma for this very reason.I think that the concept of testing outside of webpack is great, but I
don't think it should be a focus of vue test utils. I think in leiu of that
we should provide direct hookin to the vue-cli generators and setup tests
for hacknews example so that others have real examples.โ
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/vuejs/vue-test-utils/issues/1#issuecomment-325698820,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACounSNq0UIDSBdUPA4Yljox05V5Usmks5sdCyFgaJpZM4NooPe
.
@blocka you are able to get a debugger in using webstorm (or rubymine) or with visual studio code, but that has been true for a while. I just tested out 8.4.0 on my node server (that uses jest for testing) and the debugger does not work out of the box.
one other huge pitfall with jest is the inability to get a debugger in the code.
This statement is false
https://github.com/vuejs/vue-test-utils/issues/1#issuecomment-325760355
Well, i think it really depends on perspective, my experience is that the people on my team who are migrating from backbone to vue had a highly difficult time with it. They had to learn several layers concepts to even debug the code, which resulted in less test coverage because testing to them was hard. The alternative after we moved back to karma was they threw in a debugger and could see the entire closure context and had access to all the normal vue tools, happiness increased and so did code coverage.
Although I personally think jest is superior, in my experience it is not a good tool for beginner js programmers or programmers who are not coming from a more js heavy framework like react and that is why i think it is a pitfall.
I'm not sure what this has to do with backbone. Personally, if not for jest I would probably not be testing my components at all. The fact that to get any testing done you need mocha, and karma, and sinon, and god knows what else...I just couldn't make sense out of it.
Comes along jest, and bam! Just one tool, does all I need. So I found jest much simpler to use and set up.
We should create another issue to discuss test frameworks. There are no plans currently to add test framework specific code to vue-test-utils. Right now, it's framework agnostic - and I think it should stay that way.
Yeah I can't help but feel this got a bit off topic. We shouldn't really be forcing users to use one testing framework over another, we should just be ensuring there is a nice and __easy__ way to test Vue components everywhere. So (grits teeth) those wanting to use karma can do so, but there should also be a well documented framework-agnostic way of being able to at least load a vue component from within node.
@jackmellis, thank you, that is exactly right.
I get Jest is great, regardless, I also have no intention of using it in my projects. I strongly disagree with the approach React and Angular have taken of "anointing" a particular testing framework.
Since the beta is released, I'm closing this issue.
We can continue discussing features in the issue tracker
So, there is a plan to implement Array-like methods for WrapperArray(.map, .forEach, .filter),? I find them extremely useful.
@saintplay not at the moment, if you want to make a feature request you can make a new issue.
My opinion is we shouldn't add those methods. You can access the array of wrappers from a WrapperArray already:
const wrapperArr = wrapper.findAll('div')
wrapperArr.wrappers.forEach(w => {
w.is('div') // true
})
Oh! I didn't notice that, is this in the documentation? @eddyerburgh That array will do the job
Most helpful comment
Ahh, I didn't know
vue-unitexisted, looks like a pretty well thought out lib too./cc @wrseward it would be great to have some feedback from you and join forces with us :)