here is my spider
from scrapy.spider import BaseSpider
from scrapy.selector import Selector
from sample1.items import ppppkartItem
class ppppkartSpider(BaseSpider):
name = "ppppkart"
allowed_domains = ["ppppkart.com"]
start_urls = ["http://www.ppppkart.com/mobilesotracker=nmenu_sub_electronics_0_Mobiles"]
def parse(self, xmlresponse):
sel = Selector(xmlresponse)
sites = sel.xpath('//html/body/div/div[2]/div/div[2]/div[2]/div/div[2]')
items = []
for site in sites:
item = ppppkartItem()
item['image'] = site.xpath('.//a/img/@src').extract()
item['price'] = site.xpath('.//span/text()').extract()
item['title'] = site.xpath('.//a/text()').extract()
item['link'] = site.xpath('.//a/@href').extract()
items.append(item)
return items
here is my item
`
from scrapy.item import Item, Field
class ppppkartItem(Item):
price = Field()
title = Field()
link = Field()
image = Field()
here is my result
[{"image": ["http://img8a.ppppcart.com/image/mobile/q/f/r/apple-iphone-5c-imadpnhyw2qnxkh5.jpeg", "http://img7a.ppppcart.com/image/mobile/j/z/n/htc-one-max-imadqrqeyceghdba.jpeg",
the whole result is in single row , i need column wise result what can i do..thanks in advance
i want result to be like
image:img result; price:price result; title:title result; link:link result;
image:2nd img result; price:2nd price result; title:2nd title result; link:2nd link result;
There are few ways of doing this task:
You can use Scrapy built if functionality to output item into csv:
scrapy crawl [your spider name] -o items.csv -t csv
For more information read here: http://doc.scrapy.org/en/latest/topics/feed-exports.html#topics-feed-exports
You can create your own pipeline and use item exporter. then you will be able to have more control over what happens with your item, i.e. you can drop some items etc. For more information about this two topics read: http://doc.scrapy.org/en/latest/topics/item-pipeline.html and http://doc.scrapy.org/en/latest/topics/exporters.html