How do I use Selenium Webdriver on headless Chrome

Question

I'm learning to use Selenium for basic things like taking a screenshot, scraping, and testing and would like to use it with headless Chrome, which is now stable as of Chrome 59.

I have been able to take a screenshot using the 'selenium-webdriver' gem and chromedriver, but not headless.

Here is the ruby script that I am running which hangs after starting to initialize the driver

require 'rubygems'
require 'selenium-webdriver'

Selenium::WebDriver.logger.level = :debug
p 'initializing driver'
driver = Selenium::WebDriver.for :chrome, switches: %w[--headless --disable-gpu --screenshot --hide-scrollbars]
p 'navigating to Google'
driver.navigate.to "http://google.com"  
driver.save_screenshot("./screen.png")
driver.quit

and the output from the logs:

:> ruby rubytest.rb
"initializing driver"
2017-06-07 15:55:43 DEBUG Selenium Executing Process 

["/Users/name/Documents/scrapings/python/env/bin/chromedriver", "--port=9515"]
2017-06-07 15:55:43 DEBUG Selenium polling for socket on ["127.0.0.1", 9515]
Starting ChromeDriver 2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b) on port 9515
Only local connections are allowed.
2017-06-07 15:55:43 INFO Selenium -> POST session
2017-06-07 15:55:43 INFO Selenium    >>> http://127.0.0.1:9515/session | {"desiredCapabilities":{"browserName":"chrome","version":"","platform":"ANY","javascriptEnabled":true,"cssSelectorsEnabled":true,"takesScreenshot":false,"nativeEvents":false,"rotatable":false,"chromeOptions":{"args":["--headless","--disable-gpu","--screenshot","--hide-scrollbars"]}}}
2017-06-07 15:55:43 DEBUG Selenium      > {"Accept"=>"application/json", "Content-Type"=>"application/json; charset=utf-8", "Content-Length"=>"284"}
[RUBY BACKTRACE TO DRIVER INITIALIZATION]

I have tried using the JavaScript and Python drivers with similar code and nothing works. When I try this with Python, the error message is

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.5 x86_64)

Answer 1

I've written a blog post about how to do it. Summing up:

1) Make sure you have Chrome version 57+ on Linux, 59+ on macOS or 60+ on Windows (the last one hasn't been released yet, you have to use the beta, aka "Canary");

2) Add/update the gem selenium-webdriver;

3) Make sure you’re using ChromeDriver version 2.30 or higher;

4) Add the following driver to your spec_helper.rb or rails_helper.rb:

Capybara.register_driver :chrome do |app|
  Capybara::Selenium::Driver.new app, browser: :chrome,
    options: Selenium::WebDriver::Chrome::Options.new(args: %w[headless disable-gpu])
end

Capybara.javascript_driver = :chrome

Answer 2

I found this blog post useful for setting up headless chrome with selenium in ruby

require "selenium-webdriver"

# configure the driver to run in headless mode
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
driver = Selenium::WebDriver.for :chrome, options: options

driver.navigate.to "https://www.google.com"

# resize the window and take a screenshot
driver.manage.window.resize_to(800, 800)
driver.save_screenshot "screenshot.png"

Answer 3

Managed to work through this in the end via various docs, blog posts and gists.

caps = Selenium::WebDriver::Remote::Capabilities.chrome("desiredCapabilities" => {"takesScreenshot" => true}, "chromeOptions" => {"binary" => "/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary"})

driver = Selenium::WebDriver.for :chrome, desired_capabilities: caps, switches: %w[--headless --no-sandbox --disable-gpu --remote-debugin-port=9222 --screen-size=1200x800]

You need to use a very recent version of Chrome (I'm using Canary) and tell Selenium the path to the binary. You also need to set the desired capabilities for 'takesScreenshot' to true.