Cypress: Selector Playground does not work for me on specific Salesforce based sites

[JasonFairchild]

I couldn't find any other issue like the one I seem to be having, when browsing selector playground issues that have already been created. I don't really know how to investigate this further. It happens every time I try to click this selector playground button (boxed in red). So it has never worked for me on the sites I need to test. But it does work on google.com. So my only vague idea so far is that it could be related to salesforce somehow, since to build our sites we use lightning and aura web components.

Current behavior:

https://files.gitter.im/cypress-io/cypress/eHIk/image.png

cypress_runner.js:162659 [mobx] Encountered an uncaught exception that was thrown by a reaction or observer component, in: 'Reaction[Autorun@32]' TypeError: Cannot read property 'length' of undefined
    at superMatcher (cypress_runner.js:14146)
    at Sizzle.select (cypress_runner.js:14342)
    at Function.Sizzle [as find] (cypress_runner.js:12510)
    at jQuery.fn.init.find (cypress_runner.js:14541)
    at getOrCreateSelectorHelperDom (cypress_runner.js:177581)
    at Object.addOrUpdateSelectorPlaygroundHighlight (cypress_runner.js:177609)
    at AutIframe._clearHighlight (cypress_runner.js:176393)
    at AutIframe.toggleSelectorHighlight (cypress_runner.js:176584)
    at cypress_runner.js:177129
    at reactionRunner (cypress_runner.js:162926)
    at trackDerivedFunction (cypress_runner.js:162090)
    at Reaction../node_modules/mobx/lib/mobx.js.Reaction.track (cypress_runner.js:162630)
    at Reaction.onInvalidate (cypress_runner.js:162907)
    at Reaction../node_modules/mobx/lib/mobx.js.Reaction.runReaction (cypress_runner.js:162596)
    at runReactionsHelper (cypress_runner.js:162733)
    at reactionScheduler (cypress_runner.js:162711)

Here is the line of code it references in the stack trace. If the answer to my problem is contained in the comments/messages, I don't understand how, so maybe someone could just explain it to me, if this is the case.

Desired behavior:

I would like the playground selector to work as described by its documentation. It very obviously is not working on these sites. I'll understand if this is an issue that cypress can't fix if it is only specific to certain sites. Maybe it really is an issue contained primarily in cypress code though? Or maybe I can find help on how I can resolve it on my end.

Steps to reproduce: (app code and test code)

simply run a test that only visits this URL:

1. cy.visit('https://help.doterra.com/')
2. Open the playground selector
3. Click on the arrow button as if to select elements (the one boxed in red from the first screen shot)
4. mouse off of that button and immediately I always see the mobx console error, if it hasn't occurred even earlier
5. At this point the selector playground seems stuck and the error will happen over and over trying to click that button

If I need to create a more contained environment to reproduce it in, I can do that, but I figured this should work since that is a public site. It would be very interesting to me if someone else could not reproduce it this way.

Versions

Cypress package version: 3.7.0
Cypress binary version: 3.7.0

Chrome Version 79.0.3945.88 (Official Build) (64-bit)

[jennifer-shehane]

I've confirmed this error throws with hovering over the AUT using the selector playground on the following test:

it('selector playground error', () => {
  cy.visit('https://help.doterra.com/')
})

This error throws when calling this .find from within jQuery from this piece of code:

https://github.com/cypress-io/cypress/blob/develop/packages/runner/src/lib/dom.js#L156:L156

I'm not sure why this is happening.

[JasonFairchild]

@jennifer-shehane Thank you for confirming that. Its interesting to me that the line of code you linked has "$(shadowRoot)." in it. Is it easy to say why? I've been dealing with shadow root principles for a few weeks now, but as far as I understand this site shouldn't be involved much in shadow DOM. Its primarily built from salesforce aura components which should not use any type of shadow DOM, and should be a good deal easier to work with/less restrictive when it comes to DOM querying. This probably doesn't matter for why that is in the code, but I thought I'd throw it out there, in case.

Let me know if you need any more help from me. I can confirm that I have reproduced this same issue on another doterra site that is built using SF components, so I really wouldn't be surprised if it was related to something SF has implemented, I just don't understand what yet...

[JasonFairchild]

I believe I understand this issue a lot better now. And I'm doubtful there is much cypress would want to do about it, if they even could.

TL;DR
Salesforce takes a good deal of liberty in its implementation of synthetic Shadow DOM. So it patches the DOM traversing methods so that they filter out elements they don't want to be found by said methods. I believe the selector playground must use these methods in its functionality to find DOM elements. So this is why the feature doesn't work on many salesforce sites (along with any other cypress commands that try to find elements they filter).

Here is a part of the code where they prototype redefine these methods (and other methods in other places, from what I understand). Its a part of their locker service https://developer.salesforce.com/docs/component-library/tools/locker-service-viewer
There are legit reasons that they do this, for the use cases that lightning web components often have. I just don't understand them very well. I guess generally, they don't want people inadvertently or purposefully messing up components that are made to be used over and over by people that often are not super javascript savvy. But it sure presents challenges for e2e testing. For either selenium or cypress.

function apply$2() {
        function elemFromPoint(left, top) {
            const element = elementFromPoint.call(this, left, top);
            if (isNull(element)) {
                return element;
            }
            return retarget(this, pathComposer(element, true));
        }
        // https://github.com/Microsoft/TypeScript/issues/14139
        Document.prototype.elementFromPoint = elemFromPoint;
        // Go until we reach to top of the LWC tree
        defineProperty(Document.prototype, 'activeElement', {
            get() {
                let node = DocumentPrototypeActiveElement.call(this);
                if (isNull(node)) {
                    return node;
                }
                while (!isUndefined(getNodeOwnerKey(node))) {
                    node = parentElementGetter.call(node);
                    if (isNull(node)) {
                        return null;
                    }
                }
                if (node.tagName === 'HTML') {
                    // IE 11. Active element should never be html element
                    node = this.body;
                }
                return node;
            },
            enumerable: true,
            configurable: true,
        });
        // The following patched methods hide shadowed elements from global
        // traversing mechanisms. They are simplified for performance reasons to
        // filter by ownership and do not account for slotted elements. This
        // compromise is fine for our synthetic shadow dom because root elements
        // cannot have slotted elements.
        // Another compromise here is that all these traversing methods will return
        // static HTMLCollection or static NodeList. We decided that this compromise
        // is not a big problem considering the amount of code that is relying on
        // the liveliness of these results are rare.
        defineProperty(Document.prototype, 'getElementById', {
            value() {
                const elm = getElementById.apply(this, ArraySlice.call(arguments));
                if (isNull(elm)) {
                    return null;
                }
                return isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm) ? elm : null;
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
        defineProperty(Document.prototype, 'querySelector', {
            value() {
                const elements = querySelectorAll.apply(this, ArraySlice.call(arguments));
                const filtered = collectionFind(elements, elm => isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm));
                return !isUndefined(filtered) ? filtered : null;
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
        defineProperty(Document.prototype, 'querySelectorAll', {
            value() {
                const elements = querySelectorAll.apply(this, ArraySlice.call(arguments));
                const filtered = collectionFilter(elements, elm => isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm));
                return createStaticNodeList(filtered);
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
        defineProperty(Document.prototype, 'getElementsByClassName', {
            value() {
                const elements = getElementsByClassName.apply(this, ArraySlice.call(arguments));
                const filtered = collectionFilter(elements, elm => isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm));
                return createStaticHTMLCollection(filtered);
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
        defineProperty(Document.prototype, 'getElementsByTagName', {
            value() {
                const elements = getElementsByTagName.apply(this, ArraySlice.call(arguments));
                const filtered = collectionFilter(elements, elm => isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm));
                return createStaticHTMLCollection(filtered);
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
        defineProperty(Document.prototype, 'getElementsByTagNameNS', {
            value() {
                const elements = getElementsByTagNameNS.apply(this, ArraySlice.call(arguments));
                const filtered = collectionFilter(elements, elm => isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm));
                return createStaticHTMLCollection(filtered);
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
        defineProperty(
        // In Firefox v57 and lower, getElementsByName is defined on HTMLDocument.prototype
        getOwnPropertyDescriptor(HTMLDocument.prototype, 'getElementsByName')
            ? HTMLDocument.prototype
            : Document.prototype, 'getElementsByName', {
            value() {
                const elements = getElementsByName.apply(this, ArraySlice.call(arguments));
                const filtered = collectionFilter(elements, elm => isUndefined(getNodeOwnerKey(elm)) || isGlobalPatchingSkipped(elm));
                return createStaticNodeList(filtered);
            },
            writable: true,
            enumerable: true,
            configurable: true,
        });
    }

[JasonFairchild]

So right now, what we have decided to do is save the native definition for these methods, before SF overrides them, and then reapply the native definition. We will only ever do this in test environments. We have this script run before each of our test sites is built. It is a very brute force implementation (reapply every 100ms) that we are just beginning to experiment with:

<script>
    let dfs = {};
    dfs['querySelector'] = document.querySelector;
    dfs['getElementById'] = document.getElementById;
    dfs['querySelectorAll'] = document.querySelectorAll;
    dfs['getElementsByClassName'] = document.getElementsByClassName;
    dfs['getElementsByTagName'] = document.getElementsByTagName;
    dfs['getElementsByTagNameNS'] = document.getElementsByTagNameNS;
    dfs['getElementsByName'] = document.getElementsByName;
    setInterval(function(){
        document.querySelector = dfs['querySelector'];
        document.getElementById = dfs['getElementById'];
        document.querySelectorAll = dfs['querySelectorAll'];
        document.getElementsByClassName = dfs['getElementsByClassName'];
        document.getElementsByTagName = dfs['getElementsByTagName'];
        document.getElementsByTagNameNS = dfs['getElementsByTagNameNS'];
        document.getElementsByName = dfs['getElementsByName'];
    }, 100);
</script>

With the native method definitions there, I haven't found a time where the playground selector will error and it seems to work completely to find selectors as I'd expect. So that is the main reason why I think this is the root problem of the issue I was experiencing.

We are planning to try this to facilitate testing (and maybe a few other capabilities) and we are hopeful that this will not introduce any significant environment differences between where we test and prod. We think we may be fine, particularly with how we use SF LWC. But I would be very interested in others opinions in what SF is doing with their locker service and how else we might deal with it.

I think we could of course, just follow Salesforce's way of querying. Emulating this in custom cypress commands:
document.querySelector('LWC_component_selector').shadowRoot.querySelector('element_selector')

And I probably will try that as soon as document-fragments stop causing cypress to have this recursion error #5528 But that might be a lot of work for nested LWC components that are many levels deep, as I'm pretty sure we'd have to drill through every parent shadow level one at a time. So we'll see.

I will leave this issue open a little while longer to see about other's opinions. But if someone else thinks it should be closed, I understand.

[gabbersepp]

@JasonFairchild do you have tried to reproduce the issue with the callstack size? I was wondering if Salesforce override some methods but I have never proofed it. Now as you have found out that they indeed are redefining some methods, this may be the solution? So it would be nice if you can apply your solution to @jennifer-shehane small example linked in https://github.com/cypress-io/cypress/issues/4373 and tell us if the problem gets solved

[JasonFairchild]

@gabbersepp Hi. So, I have confirmed that I still get the maximum call stack size exceeded through cypress when traversing around Salesforce's document-fragments, even when using the native definitions of the methods in that code snippet I shared. So I don't think it has to do with methods they have overridden, but I could be wrong, as I know I haven't reclaimed all the methods they may have messed with. But I get the feeling that regardless of what we do, I'm quite certain there is no changing the fact that the salesforce LWC's live in document-fragments the way they do.

Yet it should be easy to apply that brute force way of reclaiming those methods from SF and other methods, if we just find out what they are, to any example. I guess my point here is I'm less confident it will resolve this particular problem (even though it definitely resolves some other problems of using cypress with SF).

There is also this idea potentially: In a recent beta release, SF is allowing a way that this whole "locker service" (basically the entirety of their method overriding implementation) can be turned off. But this feature is harder to get access too. You need certain SF orgs that they allow beta testing on. My company has such an org and we'll be starting to test this out soon, but it may be harder to share with others. It can be read about more here. https://releasenotes.docs.salesforce.com/en-us/spring20/release-notes/rn_networks_lightning_locker_toggle.htm

Regardless, I have been starting to think about how I might test #4373 I definitely have incentive to get this issue resolved soon =). As a starting point, I did try to apply your fix locally, but I need to learn more about how to actually do that (I couldn't find lines of code in our node modules for cypress matching the ones you changed in your PR https://github.com/cypress-io/cypress/pull/5528/files). Do I need to fork the cypress repo? And in general, it might help if someone could point me in the direction of some sample tests that cypress likes to have added. Are they more like unit tests, or e2e tests that use cypress or both?

[JasonFairchild]

I have some other projects to work on for now that don't involve using cypress. So I'll have to revisit this issue and 4373 another time. Some updates may come from Salesforce that make it all easier in the meantime anyway.

For now I will close this issue as I still think it is something that Cypress couldn't really account for. But hopefully this info can remain available if others come along that use Lightning web components and community site building off the SF platform. Thanks for the help previously as well.

[sergeyblohin]

@JasonFairchild, hi, I encountered the same problem when working with SalesForce.
A new experimental feature experimentalShadowDomSupport helped me.

cy
  .get('one-app-nav-bar')
  .shadow()
  .find('one-app-nav-bar-item-root')
  .shadow()
  .find('a[title="Home"]')