Why would using Selenium webdriver to execute JS f

Question

Docker environment: Docker Container, ubuntu:18.10, geckodriver-v0.23.0-linux64, selenium-3.14.1

Local environment: Mac OSX, geckodriver-v0.23.0-linux64, selenium-3.14.1

Why would executing JS work fine locally on my Mac, but not in a Docker container?

Python function that is triggering error on nfl.com:

def scroll_down(driver, value):
    """ Scroll down some """
    driver.execute_script("window.scrollBy(0,"+str(value)+")")

Traceback from nfl.com

Traceback (most recent call last):
  File "/app/foo/foo.py", line 159, in <module>
    main()
  File "/app/foo/foo.py", line 150, in main
    detect_full_html_loaded(driver)
  File "/app/foo/foo.py", line 55, in detect_full_html_loaded
    scroll_down(driver, 1000)
  File "/app/foo/foo.py", line 45, in scroll_down
    driver.execute_script("window.scrollBy(0,"+str(value)+")")
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
    'args': converted_args})['value']
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette

Python function that is triggering error on nytimes.com and sony.com:

def capture_screenshot(assets_dir, url, sleep, driver):
    """ Take simple screenshot of above-the-fold """
    driver.get(url)
    time.sleep(sleep)
    driver.set_window_size(1400, 700)
    Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
    print('Created: "' + assets_dir  + 'screenshot.png' + '"')

Traceback on nytimes.com:

Traceback (most recent call last):
      File "/app/foo/foo.py", line 159, in <module>
        main()
      File "/app/foo/foo.py", line 152, in main
        capture_element_pic(input_file, assets_dir, data['finalUrl'], sel, sleep, driver)
      File "/app/foo/foo.py", line 89, in capture_element_pic
        driver.set_window_size(1400, driver.execute_script("return document.body.parentNode.scrollHeight"))
      File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1090, in set_window_size
        self.set_window_rect(width=int(width), height=int(height))
      File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1182, in set_window_rect
        "height": height})['value']
      File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
        self.error_handler.check_response(response)
      File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
        raise exception_class(message, screen, stacktrace)
    selenium.common.exceptions.WebDriverException: Message: [Exception... "Failure"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: chrome://marionette/content/browser.js :: get rect :: line 254"  data: no]

Traceback from sony.com

Traceback (most recent call last):
  File "/app/foo/foo.py", line 159, in <module>
    main()
  File "/app/foo/foo.py", line 149, in main
    capture_screenshot(assets_dir, data['finalUrl'], sleep, driver)
  File "/app/foo/foo.py", line 82, in capture_screenshot
    Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
  File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png
    return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
AttributeError: 'NoneType' object has no attribute 'encode'

Again, this works fine locally.

Answer 1

It was a resource related issue like @Andersson suggested. Adding an appropriate shm_size in docker-compose.yml fixed the issue in both Chrome and Firefox Gecko webdrivers.

https://docs.docker.com/compose/compose-file/#shm_size