First of all, I am a beginner in Ruby and not very familiar with the way Ruby manages the code and gets it to work so I do expect the issue being me no knowing how to use ruby correctly.
The issue I am having is a deadlock, but I am not using any threads in my program. Plus, the error only happens about once every 1000 to 1500 function calls, making it very difficult to pinpoint and correct.
Here is the complete error message when the issue occurs:
/usr/lib/ruby/2.3.0/timeout.rb:95:in `join': No live threads left. Deadlock? (fatal)
from /usr/lib/ruby/2.3.0/timeout.rb:95:in `ensure in block in timeout'
from /usr/lib/ruby/2.3.0/timeout.rb:95:in `block in timeout'
from /usr/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
from /usr/lib/ruby/2.3.0/net/http.rb:878:in `connect'
from /usr/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
from /usr/lib/ruby/2.3.0/net/http.rb:852:in `start'
from /usr/lib/ruby/2.3.0/open-uri.rb:319:in `open_http'
from /usr/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
from /usr/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
from /usr/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
from /usr/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
from /usr/lib/ruby/2.3.0/open-uri.rb:717:in `open'
from /usr/lib/ruby/2.3.0/open-uri.rb:35:in `open'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/utils.rb:85:in `get_pic'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:87:in `page_link'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:116:in `chapter_link'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_download.rb:142:in `chapter'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:57:in `block in MF_manga_missing_chapters'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `reverse_each'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:45:in `MF_manga_missing_chapters'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/mangafox/MF_update.rb:80:in `MF_update'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:5:in `update_manga'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:15:in `block in update_all'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `each'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:14:in `update_all'
from /home/mat/travail_perso/RUBY/MangaScrapp_github/sources/update.rb:22:in `update'
from ./MangaScrap.rb:28:in `<main>'
the link to the complete program is here: https://github.com/Hellfire01/MangaScrap
The issue happens to the 3 different methods that use open, here is the one that crashed this time:
# conect to link and download picture
def get_pic(link)
safe_link = link.gsub(/[\[\]]/) { '%%%s' % $&.ord.to_s(16) }
tries ||= 20
begin
page = open(safe_link, "User-Agent" => "Ruby/#{RUBY_VERSION}")
rescue URI::InvalidURIError => error
puts "Warning : bad url"
puts link
puts "message is : " + error.message
return nil
rescue => error
if tries > 0
tries -= 1
sleep(0.2)
retry
else
puts 'could not get picture ' + safe_link + ' after ' + $nb_tries.to_s + ' tries'
puts "message is : " + error.message
return nil
end
end
sleep(0.2)
return page
end
Here is the link of the file : https://github.com/Hellfire01/MangaScrap/blob/master/sources/utils.rb
What I would like to know :
- How can I fix this error ?
- If I can not fix this error, are there alternatives to open-uri that I can use ?
Any help is most welcome