Preventing timeout when connecting to a URL

Question

I want to see the time taken to access a url using Benchmark in the code below. I also tried to do the same thing without benchmark. That is, get time at start of test and end of test, subtract the two to get the time. Both methods end in the same timeout error.

require 'open-uri'
require 'benchmark'

response = nil
puts "opening website with benchmark..."
puts Benchmark.measure{
  response = open('http://mywebsite.com')
}

puts "Done !"
status = response.status
puts status

Error:

opening website with benchmark...
C:/ruby/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill': execution expired (Timeout::Error)
    from C:/ruby/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
    from C:/ruby/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
    from C:/ruby/lib/ruby/1.8/net/protocol.rb:126:in `readline'
    from C:/ruby/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
    from C:/ruby/lib/ruby/1.8/net/http.rb:2017:in `read_new'
    from C:/ruby/lib/ruby/1.8/net/http.rb:1051:in `request'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:248:in `open_http'
    from C:/ruby/lib/ruby/1.8/net/http.rb:543:in `start'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:242:in `open_http'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:162:in `catch'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:162:in `open_loop'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:518:in `open'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:30:in `open'
    from C:/code/test.rb:7
    from C:/ruby/lib/ruby/1.8/benchmark.rb:293:in `measure'
    from C:/code/test.rb:6

When I try to connect to this URL in my browser, it takes about 2-3 minutes to access, all the time.

I searched google, but found no useful answers to my problem. I know that I have to change the timeout setting for something, but not able to figure out which one. Can someone please help ?

Answer 1

Use the :read_timeout option, specified in seconds, e.g.,

open('foo.com', :read_timeout => 10)

http://ruby-doc.org/stdlib-1.8.7/libdoc/open-uri/rdoc/OpenURI/OpenRead.html

Answer 2

The BufferedIO class that Net::HTTP uses for the connection that open-uri then uses for the request has a read_timeout attribute set to 60 seconds.

The Net::HTTP class provides a setter read_timeout for that.

Unfortunately, the way open-uri sets up that request doesn't provide you with a way to get at that setting before the request or the easily override the default of 60.

You will probably need to use Net::HTTP yourself.

link = URI.parse(url)
request = Net::HTTP::Get.new(link.path)
response = Net::HTTP.start(link.host, link.port) {|http|
  http.read_timeout = 100 #Default is 60 seconds
  http.request(request)
}

Code stolen from this answer

edit: or upgrade to 1.9 which supports the :read_timeout = x option Dave Newton noted.