Accessing Shadow DOM tree with Selenium

Question

Is it possible to access elements within a Shadow DOM using Selenium/Chrome webdriver?

Using the normal element search methods doesn't work, as is to be expected. I've seen references to the switchToSubTree spec on w3c, but couldn't locate any actual docs, examples, etc.

Anyone had success with this?

Answer 1

The accepted answer is no longer valid and some of the other answers have some drawbacks or are not practical (the /deep/ selector doesn't work and is deprecated, document.querySelector('').shadowRoot works only with the first shadow element when shadow elements are nested), sometimes the shadow root elements are nested and the second shadow root is not visible in document root, but is available in its parent accessed shadow root. I think is better to use the selenium selectors and inject the script just to take the shadow root:

def expand_shadow_element(element):
  shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root

outer = expand_shadow_element(driver.find_element_by_css_selector("#test_button"))
inner = outer.find_element_by_id("inner_button")
inner.click()

To put this into perspective I just added a testable example with Chrome's download page, clicking the search button needs open 3 nested shadow root elements:

import selenium
from selenium import webdriver
driver = webdriver.Chrome()


def expand_shadow_element(element):
  shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root

driver.get("chrome://downloads")
root1 = driver.find_element_by_tag_name('downloads-manager')
shadow_root1 = expand_shadow_element(root1)

root2 = shadow_root1.find_element_by_css_selector('downloads-toolbar')
shadow_root2 = expand_shadow_element(root2)

root3 = shadow_root2.find_element_by_css_selector('cr-search-field')
shadow_root3 = expand_shadow_element(root3)

search_button = shadow_root3.find_element_by_css_selector("#search-button")
search_button.click()

Doing the same approach suggested in the other answers has the drawback that it hard-codes the queries, is less readable and you cannot use the intermediary selections for other actions:

search_button = driver.execute_script('return document.querySelector("downloads-manager").shadowRoot.querySelector("downloads-toolbar").shadowRoot.querySelector("cr-search-field").shadowRoot.querySelector("#search-button")')
search_button.click()

Answer 2

Until Selenium supports shadow DOM out of the box, you can try the following workaround in Java. Create a class that extends By class:

import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.SearchContext;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.WrapsDriver;
import org.openqa.selenium.internal.FindsByCssSelector;

import java.io.Serializable;
import java.util.List;

public class ByShadow {
    public static By css(String selector) {
        return new ByShadowCss(selector);
    }

    public static class ByShadowCss extends By implements Serializable {

        private static final long serialVersionUID = -1230258723099459239L;

        private final String cssSelector;

        public ByShadowCss(String cssSelector) {
            if (cssSelector == null) {
                throw new IllegalArgumentException("Cannot find elements when the selector is null");
            }
            this.cssSelector = cssSelector;
        }

        @Override
        public WebElement findElement(SearchContext context) {
            if (context instanceof FindsByCssSelector) {
                JavascriptExecutor jsExecutor;
                if (context instanceof JavascriptExecutor) {
                    jsExecutor = (JavascriptExecutor) context;
                } else {
                    jsExecutor = (JavascriptExecutor) ((WrapsDriver) context).getWrappedDriver();
                }
                String[] subSelectors = cssSelector.split(">>>");
                FindsByCssSelector currentContext = (FindsByCssSelector) context;
                WebElement result = null;
                for (String subSelector : subSelectors) {
                    result = currentContext.findElementByCssSelector(subSelector);
                    currentContext = (FindsByCssSelector) jsExecutor.executeScript("return arguments[0].shadowRoot", result);
                }
                return result;
            }

            throw new WebDriverException(
                    "Driver does not support finding an element by selector: " + cssSelector);
        }

        @Override
        public List<WebElement> findElements(SearchContext context) {
            if (context instanceof FindsByCssSelector) {
                JavascriptExecutor jsExecutor;
                if (context instanceof JavascriptExecutor) {
                    jsExecutor = (JavascriptExecutor) context;
                } else {
                    jsExecutor = (JavascriptExecutor) ((WrapsDriver) context).getWrappedDriver();
                }
                String[] subSelectors = cssSelector.split(">>>");
                FindsByCssSelector currentContext = (FindsByCssSelector) context;
                for (int i = 0; i < subSelectors.length - 1; i++) {
                    WebElement nextRoot = currentContext.findElementByCssSelector(subSelectors[i]);
                    currentContext = (FindsByCssSelector) jsExecutor.executeScript("return arguments[0].shadowRoot", nextRoot);
                }
                return currentContext.findElementsByCssSelector(subSelectors[subSelectors.length - 1]);
            }

            throw new WebDriverException(
                    "Driver does not support finding elements by selector: " + cssSelector);
        }

        @Override
        public String toString() {
            return "By.cssSelector: " + cssSelector;
        }
    }
}

And you can use it without writing any additional functions or wrappers. This should work with any kind of framework. For example, in pure Selenium code this would look like this:

WebElement searchButton =
    driver.findElement(ByShadow.css(
        "downloads-manager >>> downloads-toolbar >>> cr-search-field >>> #search-button"));

or if you use Selenide:

SelenideElement searchButton =
    $(ByShadow.css("downloads-manager >>> downloads-toolbar >>> cr-search-field >>> #search-button"));

Answer 3

It should also be noted that the Selenium binary Chrome driver now supports Shadow DOM (since Jan 28, 2015) : http://chromedriver.storage.googleapis.com/2.14/notes.txt

Answer 4

Unfortunately it looks like the webdriver spec does not support this yet.

My snooping uncovered :

http://www.w3.org/TR/webdriver/#switching-to-hosted-shadow-doms

https://groups.google.com/forum/#!msg/selenium-developers/Dad2KZsXNKo/YXH0e6eSHdAJ

Answer 5

I am using C# and Selenium and managed to find an element inside a nestled shadow DOM using java script. This is my html tree:

html tree

I want the url on the last line and to get it I first select the "downloads-manager" tag and then the first shadow root right below it. Once inside the shadow root I want to find the element closest to the next shadow root. That element is "downloads-item". With that selected I can enter the second shadow root. From there I select the img item containing the url by id = "file-icon". At last I can get the attribute "src" which contains the url I am seeking.

The two lines of C# code that does the trick:

IJavaScriptExecutor jse2 = (IJavaScriptExecutor)_driver;
var pdfUrl = jse2.ExecuteScript("return document.querySelector('downloads-manager').shadowRoot.querySelector('downloads-item').shadowRoot.getElementById('file-icon').getAttribute('src')");

Answer 6

I found a much easier way to get the elements from Shadow Dom. I am taking the same example given above, for search icon of Chrome Download Page.

IWebDriver driver;

public IWebElement getUIObject(params By[] shadowRoots)
        {
            IWebElement currentElement = null;
            IWebElement parentElement = null;
            int i = 0;
            foreach (var item in shadowRoots)
            {
                if (parentElement == null)
                {
                    currentElement = driver.FindElement(item);
                }
                else
                {
                    currentElement = parentElement.FindElement(item);
                }
                if(i !=(shadowRoots.Length-1))
                {
                    parentElement = expandRootElement(currentElement);
                }
                i++;
            }
            return currentElement;
        }

 public IWebElement expandRootElement(IWebElement element)
        {
            IWebElement rootElement = (IWebElement)((IJavaScriptExecutor)driver)
        .ExecuteScript("return arguments[0].shadowRoot", element);
            return rootElement;
        }

Google Chrome Download Page

Now as shown in image we have to expand three shadow root elements in order to get our search icon. To to click on icon all we need to do is :-

  [TestMethod]
        public void test()
        {
           IWebElement searchButton= getUIObject(By.CssSelector("downloads-manager"),By.CssSelector("downloads-toolbar"),By.Id("search-input"),By.Id("search-buton"));
            searchButton.Click();
        }

So just one line will give you your Web Element, just need to make sure you pass first shadow root element as first argument of the function "getUIObject" second shadow root element as second argument of the function and so on, finally last argument for the function will be the identifier for your actual element (for this case its 'search-button')