scrapy csv output having all results in single row

2019-09-09 02:59发布

here is my spider

from scrapy.spider import BaseSpider
from scrapy.selector import Selector
from sample1.items import ppppkartItem

class ppppkartSpider(BaseSpider):
   name = "ppppkart"
   allowed_domains = ["ppppkart.com"]
   start_urls = ["http://www.ppppkart.com/mobilesotracker=nmenu_sub_electronics_0_Mobiles"]

   def parse(self, xmlresponse):
       sel = Selector(xmlresponse)
       sites = sel.xpath('//html/body/div/div[2]/div/div[2]/div[2]/div/div[2]')
       items = []
       for site in sites:
           item = ppppkartItem()
           item['image'] = site.xpath('.//a/img/@src').extract()
           item['price'] = site.xpath('.//span/text()').extract()
           item['title'] = site.xpath('.//a/text()').extract()
           item['link'] = site.xpath('.//a/@href').extract() 
           items.append(item)
       return items     

here is my item

`

from scrapy.item import Item, Field

class ppppkartItem(Item):
    price = Field()
    title = Field()
    link = Field()
    image = Field()

here is my result

[{"image": ["http://img8a.ppppcart.com/image/mobile/q/f/r/apple-iphone-5c-imadpnhyw2qnxkh5.jpeg", "http://img7a.ppppcart.com/image/mobile/j/z/n/htc-one-max-imadqrqeyceghdba.jpeg", 

the whole result is in single row , i need column wise result what can i do..thanks in advance

i want result to be like

image:img result; price:price result; title:title result; link:link result;
image:2nd img result; price:2nd price result; title:2nd title result; link:2nd link result;

1条回答
Lonely孤独者°
2楼-- · 2019-09-09 03:58

There are few ways of doing this task:

  1. You can use Scrapy built if functionality to output item into csv:

    scrapy crawl [your spider name] -o items.csv -t csv

    For more information read here: http://doc.scrapy.org/en/latest/topics/feed-exports.html#topics-feed-exports

  2. You can create your own pipeline and use item exporter. then you will be able to have more control over what happens with your item, i.e. you can drop some items etc. For more information about this two topics read: http://doc.scrapy.org/en/latest/topics/item-pipeline.html and http://doc.scrapy.org/en/latest/topics/exporters.html

查看更多
登录 后发表回答