Selenium: Not supported exceptions when use findelements() on shadow-root - affects By.name, By.tagName, By.xpath

[slawkow]

Meta -

OS:
Windows 7
Selenium Version:
3.6.0
Browser:
Chrome, but not relevant in fact
Browser Version:
not relevant

Expected Behavior -

When used: By.name("list") than the element will be found with the 'name' attribute, not with 'tagName'. And of course will be found and possible to interract without errors.
That affect both attributes - name and tagName should be possible to use on these WebElements.

For xpath - I expect that xpath will be correct processed.

Actual Behavior -

for name attribute (1):
org.openqa.selenium.WebDriverException: unknown error: a.getElementsByTagName is not a function

for tagName attribute (2):
org.openqa.selenium.WebDriverException: unknown error: b.getElementsByTagName is not a function

for Xpath (3):
org.openqa.selenium.InvalidSelectorException: invalid selector: Unable to locate an element with the xpath expression .//shop-list because of the following error:
NotSupportedError: Failed to execute 'evaluate' on 'Document': The node provided is '#document-fragment', which is not a valid context node type.

Steps to reproduce -

Site: https://shop.polymer-project.org/
This site is with Polymer written, so for every component there comes shadow root (Shadow DOM) - that's why I need to use expandRootElement method.

public WebElement expandRootElement(WebElement element) {
        WebElement ele = (WebElement) ((JavascriptExecutor) getDriver())
                .executeScript("return arguments[0].shadowRoot",element);
        return ele;
}

public WebElement getShadowRoot() {
        WebElement root = getDriver().waitForElement(By.tagName("shop-app"));
        return expandRootElement(root);
}

getShadowRoot().findElement(By.partialLinkText("Men's Outerwear")).click(); // it's OK, Selenium clicks that

WebElement shopList = getShadowRoot().findElement(By.name("list")); // here comes exception (1)
WebElement shopList = getShadowRoot().findElement(By.tagName("shop-list")); // here comes exception (2)
WebElement shopList = getShadowRoot().findElement(By.xpath(".//shop-list")); // here exception for xpath (3)

[arviedelgado]

For exception 1 and 2:
ShadowRoot elements does not have the getElementsByTagName function.
Finding elements By.tagName in Selenium will try to call this function and will throw an error.

As workaround, ShadowRoot elements has getElementById, querySelector and querySelectorAll functions.
Finding elements By.Id and By.CssSelector should work.

For exception 3:
XPath calls cannot return nodes like HTMLDocument and ShadowRoot - which is also a document.

For selenium, I think ShadowRoot should be handled like an IFrame and requires .SwitchTo as it has its own context.

[arviedelgado]

Now, for your code to work. Here is a sample using C# to get the links of each product on the page.

Output:

Starting ChromeDriver 2.33.506120 (e3e53437346286c0bc2d2dc9aa4915ba81d9023f) on port 60062
Only local connections are allowed.
https://shop.polymer-project.org/detail/mens_outerwear/Men+s+Tech+Shell+Full-Zip
https://shop.polymer-project.org/detail/mens_outerwear/Anvil+L+S+Crew+Neck+-+Grey
https://shop.polymer-project.org/detail/mens_outerwear/Green+Flex+Fleece+Zip+Hoodie
https://shop.polymer-project.org/detail/mens_outerwear/Android+Nylon+Packable+Jacket
https://shop.polymer-project.org/detail/mens_outerwear/YouTube+Ultimate+Hooded+Sweatshirt
https://shop.polymer-project.org/detail/mens_outerwear/Grey+Heather+Fleece+Zip+Hoodie
https://shop.polymer-project.org/detail/mens_outerwear/Vastrm+Hoodie
https://shop.polymer-project.org/detail/mens_outerwear/Recycled+Plastic+Bottle+Hoodie+-+Green
https://shop.polymer-project.org/detail/mens_outerwear/Rowan+Pullover+Hood
https://shop.polymer-project.org/detail/mens_outerwear/Men+s+Voyage+Fleece+Jacket
https://shop.polymer-project.org/detail/mens_outerwear/Eco-Jersey+Chrome+Zip+Up+Hoodie
https://shop.polymer-project.org/detail/mens_outerwear/Android+Colorblock+Hooded+Pullover
https://shop.polymer-project.org/detail/mens_outerwear/Tri-blend+Full-Zip+Hoodie
https://shop.polymer-project.org/detail/mens_outerwear/Fleece+Full-Zip+Hoodie
https://shop.polymer-project.org/detail/mens_outerwear/Jacquard-Knit+Full-Zip+Fleece
https://shop.polymer-project.org/detail/mens_outerwear/YouTube+Unisex+Flex+Fleece+Zip+Hoodie
Press any key to continue . . .

Code:

using System;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Chrome;

namespace SeleniumIssue4971
{
    class Program
    {
        static void Main(string[] args)
        {
            RemoteWebDriver driver = new ChromeDriver();
            driver.Navigate().GoToUrl("https://shop.polymer-project.org/list/mens_outerwear");
            var links = GetRoot(driver, "shop-app", "shop-list").FindElementsByCssSelector("ul > li > a");
            foreach (var link in links)
            {
                Console.WriteLine(link.GetAttribute("href"));
            }
        }

        static RemoteWebElement GetRoot(RemoteWebDriver driver, params string[] selectors)
        {
            var root = (RemoteWebElement)driver.ExecuteScript("return document");
            foreach (var selector in selectors)
            {
                root = (RemoteWebElement)driver.ExecuteScript("return arguments[0].querySelector(arguments[1]).shadowRoot", root, selector);
            }
            return root;
        }
    }
}