Ruby : open causes a deadlock

Question

First of all, I am a beginner in Ruby and not very familiar with the way Ruby manages the code and gets it to work so I do expect the issue being me no knowing how to use ruby correctly.
The issue I am having is a deadlock, but I am not using any threads in my program. Plus, the error only happens about once every 1000 to 1500 function calls, making it very difficult to pinpoint and correct.
Here is the complete error message when the issue occurs:

/usr/lib/ruby/2.3.0/timeout.rb:95:in `join': No live threads left. Deadlock? (fatal)
    from /usr/lib/ruby/2.3.0/timeout.rb:95:in `ensure in block in timeout'
    from /usr/lib/ruby/2.3.0/timeout.rb:95:in `block in timeout'
    from /usr/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
    from /usr/lib/ruby/2.3.0/net/http.rb:878:in `connect'
    from /usr/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
    from /usr/lib/ruby/2.3.0/net/http.rb:852:in `start'
    from /usr/lib/ruby/2.3.0/open-uri.rb:319:in `open_http'
    from /usr/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
    from /usr/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
    from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
    from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
    from /usr/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
    from /usr/lib/ruby/2.3.0/open-uri.rb:717:in `open'
    from /usr/lib/ruby/2.3.0/open-uri.rb:35:in `open'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/utils.rb:85:in `get_pic'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:87:in `page_link'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:116:in `chapter_link'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:142:in `chapter'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:57:in `block in MF_manga_missing_chapters'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `reverse_each'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `MF_manga_missing_chapters'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:80:in `MF_update'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:5:in `update_manga'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:15:in `block in update_all'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `each'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `update_all'
    from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:22:in `update'
    from ./MangaScrap.rb:28:in `<main>'

the link to the complete program is here: https://github.com/Hellfire01/MangaScrap
The issue happens to the 3 different methods that use open, here is the one that crashed this time:

# conect to link and download picture
def get_pic(link)
  safe_link = link.gsub(/[\[\]]/) { '%%%s' % $&.ord.to_s(16) }
  tries ||= 20
  begin
    page = open(safe_link, "User-Agent" => "Ruby/#{RUBY_VERSION}")
  rescue URI::InvalidURIError => error
    puts "Warning : bad url"
    puts link
    puts "message is : " + error.message
    return nil
  rescue => error
    if tries > 0
    tries -= 1
    sleep(0.2)
    retry
    else
      puts 'could not get picture ' + safe_link + ' after ' + $nb_tries.to_s + ' tries'
      puts "message is : " + error.message
      return nil
    end
  end
  sleep(0.2)
  return page
end

Here is the link of the file : https://github.com/Hellfire01/MangaScrap/blob/master/sources/utils.rb

What I would like to know :
- How can I fix this error ?
- If I can not fix this error, are there alternatives to open-uri that I can use ?

Any help is most welcome

Answer 1

You're not catching all exceptions here. When nothing is specified after rescue, it means that you're catching StandardError which is not at the root of exceptions' hierarchy.

If you want to make sure you're catching all exceptions and retry opening an url (or whatever behavior you'd like), what you want to do is: rescue Exception => error