Shadow-dom support for selenium

Question

I am working on an automation project which uses shadow dom extensively. So every time using execute_script function to access shadow root

for eg.

root = driver.execute_script('return document.querySelector(".flex.vertical.layout").shadowRoot')

Then use the root to access the elements within. Since we have shadow roots at many levels, this is annoying me a lot. is there any better solution exist than this to access elements within shadow root?

I am using Chrome 2.20 driver.

Answer 1

Since you use often that you may create a function, then the above becomes:

def select_shadow_element_by_css_selector(selector):
  running_script = 'return document.querySelector("%s").shadowRoot' % selector
  element = driver.execute_script(running_script)
  return element

shadow_section = select_shadow_element_by_css_selector(".flex.vertical.layout")
shadow_section.find_element_by_css(".flex")

on the resulting element you can put any of the methods:

find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

To find multiple elements (these methods will return a list):

find_elements_by_name
find_elements_by_xpath
find_elements_by_link_text
find_elements_by_partial_link_text
find_elements_by_tag_name
find_elements_by_class_name
find_elements_by_css_selector

later edit:

Sometime the shadow host elements are hidden withing shadow trees that's why the best way to do it is to use the selenium selectors to find the shadow host elements and inject the script just to take the shadow root: :

def expand_shadow_element(element):
  shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root

#the above becomes 
shadow_section = expand_shadow_element(find_element_by_tag_name("neon-animatable"))
shadow_section.find_element_by_css(".flex")

To put this into perspective I just added a testable example with Chrome's download page, clicking the search button needs open 3 nested shadow root elements:

import selenium
from selenium import webdriver
driver = webdriver.Chrome()


def expand_shadow_element(element):
  shadow_root = driver.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root

selenium.__file__
driver.get("chrome://downloads")
root1 = driver.find_element_by_tag_name('downloads-manager')
shadow_root1 = expand_shadow_element(root1)

root2 = shadow_root1.find_element_by_css_selector('downloads-toolbar')
shadow_root2 = expand_shadow_element(root2)

root3 = shadow_root2.find_element_by_css_selector('cr-search-field')
shadow_root3 = expand_shadow_element(root3)

search_button = shadow_root3.find_element_by_css_selector("#search-button")
search_button.click()

Answer 2

By googling I found another workaround for this problem - which is using The "/deep/ combinator".

for example, I was able to access all the shadow roots elements by

driver.find_elements_by_css_selector('body/deep/.layout.horizontal.center')

This will have access to the element with compound class name "layout horizontal center" regardless of the number of shadow roots it has.

But this only working in chrome driver and I see notes "/deep/" is a deprecated approach.

Answer 3

You can write extension methods to operate on IWebElement to expand the root as below.

        public static class SeleniumExtension
{
    public static IWebElement ExpandRootElement(this IWebElement element, IWebDriver driver)
    {
        return (IWebElement)((IJavaScriptExecutor)driver)
                .ExecuteScript("return arguments[0].shadowRoot", element);
    }
}

You can use above extension method to traverse through the elements hierarchy to reach the element of intrest..

        By downloads_manager_ShadowDom= By.TagName("downloads-manager");
        By downloadToolBarShadowDom = By.CssSelector("downloads-toolbar");
        By toolBarElement = By.CssSelector("cr-toolbar");

    IWebElement ToolBarElement = driver.FindElement(downloads_manager_ShadowDom).ExpandRootElement(driver)
                          .FindElement(downloadToolBarShadowDom).ExpandRootElement(driver)
                          .FindElement(toolBarElement);

Answer 4

Trying to have this automated on Chrome I came up with an inelegant solution of recursively searching through each shadow dom explicitly using:

driver.executeScript(scriptToRun, cssSelector);

Here's the javascript (passed as a string):

  function recursiveSearch(element, target) {
    let result = element.querySelector(target);
    if (result) { return result; }
    let subElements = element.querySelectorAll("*");
    for (let i = 0; i < subElements.length; i++) {
      let subElement = subElements[i];
      if (subElement && subElement.shadowRoot) {
        let result = recursiveSearch(subElement.shadowRoot, target);
        if (result) return result;
      }
    }
  }
  return recursiveSearch(document, arguments[0]);

Since the contents of a shadowRoot might be empty initially one can use driver.wait and until.elementIsVisible to avoid returning a null element.

Async example:

    return await driver.wait(until.elementIsVisible(await driver.wait(async () => {
    return await driver.executeScript(scriptToRun, cssSelector);
  }, timeOut)));

Alternatively

My previous solution was to traverse the elements with shadowdoms explicitly, but is less autonomous. Same as above but with this script:

  let element = document.querySelector(arguments[0][0]);
  let selectors = arguments[0].slice(1);
  for (i = 0; i < selectors.length; i++) {
    if (!element || !element.shadowRoot) {return false;}
    element = element.shadowRoot.querySelector(selectors[i]);
  }
  return element;

Where selectors would be something like:

['parentElement1', 'parentElement2', 'targetElement']

Sidenote

I found that running my automation tests on Firefox Quantum 57.0 doesn't suffer from hidden shadow doms, and any element can be found with a simple:

driver.findElement(searchQuery);

Answer 5

Maybe you may use IJavaScriptExecutor?

IWebDriver driver;
IJavaScriptExecutor jsExecutor = (IJavaScriptExecutor)driver;
jsExecutor.ExecuteScript('yourShadowDom.func()');

Answer 6

The WebDriver spec still doesn't have anything specific to say about Shadow DOM.

Nor the Selenium project pages - which is understandable, as they closely follow the spec. Yet there is some low-level code in existence.

So, the short answer is: no, there is no special support in the spec, in Selenium's WebDriver API or implementation code at present.

Yes, the capability seems to exist in ChromeDriver 2.14 (as a wrapper around Chrome). However, as far as I can tell there are no Selenium or WebDriver-level bindings to let you use it.

But for more detail and possible workarounds, see also: Accessing Shadow DOM tree with Selenium, also: Accessing elements in the shadow DOM, and especially: Finding elements in the shadow DOM