I am trying to scrape imdb using pythons scrapy. however I am not being able to get the rating info from the page as shown below:
I am using the below code:
from scrapy.spiders import Spider
from scrapy.selector import Selector
from imdb.items import ImdbItem
class ImdbSpider(Spider):
name = "imdb"
allowed_domains = ["imdb.com"]
start_urls = [
"http://www.imdb.com/title/tt0068646/reviews?ref_=%20best",
]
def parse(self, response):
sel = Selector(response)
ratings = sel.xpath('//div[contains(@id,"tn15content")]/div/img')
items = []
for rating in ratings:
item = ImdbItem()
item['rating'] = rating.xpath('/@alt').extract()
items.append(item)
return items
I am sorry if this is a very basic question but I am very new to python and web scraping and can't really figure out how to achieve so would someone kindly guide me??
The
/
is extra, use: