How to target data attribute with Scrapy

2020-04-11 13:15发布

I'm using Scrapy library to crawl a webpage.

But I have a problem. I do not know how to target data attribute.

I have an link with data attribute and href as follows:

<a data-item-name="detail-page-link" href="this-is-some-link">

What I want is the value of href. If a had class I could do it as follows:

response.css('.some-class::attr(href)') 

But the problem is that I do not know how to target data-item-name attribute.

Any advice?

标签: python scrapy
2条回答
Juvenile、少年°
2楼-- · 2020-04-11 14:18

I'm not sure, if you can do this with the css method, but with the xpath method you should be able to do:

response.xpath("//a[@data-item-name]/@href")
查看更多
聊天终结者
3楼-- · 2020-04-11 14:19

Using scrapy css selector, you can do :

response.css('a[data-item-name="detail-page-link"]::attr(href)').extract() 
查看更多
登录 后发表回答