Vuetify: [Feature Request] Support end to end testing (e2e) with data-test property on all interactive elements of all components

[WhatFreshHellIsThis]

Problem to solve

Currently it's very difficult to do end to end testing with a Vuetify app as it's extremely hard to select some elements programatically by selector. For example using Cypress to test an application with Vuetify controls such as v-autocomplete or v-date-picker or v-time-picker involves lengthy, convoluted and fragile selectors prone to breaking on Vuetify updates.

Proposed solution

Add a data-test property on all Vuetify components that are user interactive so that specfiying a value will in turn allow e2e tests to control all interactive aspects of components programmatically. For example an autocomplete needs a way to identify all user interactive elements including the menu joined together under the specified data-test property, so the overall component might have a data-test property value set to "user", then the input portion might have a data-test="user-input", the popup menu items would be data-test="user-menu-1" second item data-test="user-menu-2", the prepend icon might have a data-test="user-prepend-icon" etc. Note that this is most needed in particular in multi-element controls such as the date and time picker, autocomplete etc. If nothing is specified for data-test property then it need not set any of the component or it's children but if it is then anything a user can interact with should be tagged with this property to support testing.

[YipingRuan]

Does this work for your case?
https://vuetifyjs.com/en/getting-started/unit-testing/#e-2-e-tests

[WhatFreshHellIsThis]

No unfortunately it does not because of the reason I wrote about above; multi-element objects like date / time pickers and autocompletes etc only have that attribute set at the top level leaving no alternative but to pick through css selectors attempting to find the "4" in the time picker for 4 o'clock selection, the 4th result returned in the autocomplete menu list (which doesn't even get any data-cy passed to it from the parent autocomplete now making it orphaned from it from a testing point of view) etc. If those controls flowed the data-cy down into the child elements e.g. on a time picker: data-cy="[parent data-cy value]:clock-4" or date picker: data-cy="parent data-cy value]:day-15" or something along those lines as I indicated on all user interactive elements then it would be deterministic to test drive them even between big updates to the components.

[WhatFreshHellIsThis]

After more attempts and a lot of wasted time just trying to implement a simple reliable smoke test I've come to the view that there is just no practical way that Vuetify and Cypress will ever reliably work together. I think there are fundamental differences that rule out any non trivial e2e testing without far more fundamental changes to Vuetify than even the issues I already brought up. Sure you could make a quick example for docs that appears to work but put into production on a large application and things go sideways quickly. It's not only the lack of ability to target and control Vuetify elements but so many more issues around how the dom is changing so completely and quickly for even what seem like the simplest of processes like making and verifying selections in auto completes and Cypress just not able to deal with that. Neither side is doing anything wrong afaict, it's just a fundamental issue in interoperability.

[jasonatball]

I attempted to use the data-test attribute in a v-select and it doesn't appear to be passed down at all. I'm starting to share @WhatFreshHellIsThis concerns that Cypress might not be able to test my Vuetify app. I've also tried NightWatch but found Cypress much easier to use. However, without proper support for data attributes I'm not sure how to introspect the menu that opens in a v-select. Let me know if you want this opened as a separate issue.

[Mikilll94]

@WhatFreshHellIsThis @jasonatball
I agree that Vuetify is not optimized for E2E testing. However, for v-select I am just setting this prop
:menu-props="{ contentClass: 'cy-mySelect'}"

Thanks to it I have access to the list of items which was detached from the DOM.

[jasonatball]

I was able to create this Cypress command to select the drop down:

Cypress.Commands.add('chooseVSelect', (inputLabel, selection) => {
  cy.get('label')
    .contains(new RegExp(`^${inputLabel}$`, 'i'))
    .click({ force: true })

  cy.get('.v-list-item__title')
    .contains(new RegExp(`^${selection}$`, 'i'))
    .click()
})

[WhatFreshHellIsThis]

Anything is possible once for sure but in my experience it's not reliable and fails regularly over time. I thought I had it figured out so many times only to have my tests start failing randomly again and again.

[Mikilll94]

I have found this issue when doing E2E tests with Cypress:
https://github.com/cypress-io/cypress/issues/8376

[squaredsquid]

I agree that Vuetify is not optimized for E2E testing. However, for v-select I am just setting this prop
:menu-props="{ contentClass: 'cy-mySelect'}"

I'd like to emphasize that this works for me in a selenium based testing setup. This feature should probably be documented more prominently until there is are better options for suporting E2E testing.

It also works for v-menu and v-autocomplete, that have the same issue of using a menu, that is detached from component DOM nodes.

In addition there is also the possibility to style the list items in v-select and v-autocomplete using the slot item. There you can add custom attributes/markup to the v-list-item-content. But this results in a considerably less readable template.