I am a scrapy newbie and have written below spider. I want write to xml or csv with each row in csv or each item in xml as name,tele,addr.
I am using command: scrapy crawl abc -o items.csv -t csv
I am looking for output:
name,addr,tele
n1,a1,t1
n2,a2,t2
n3,a3,t3
But I get:
name,addr,tele
n1,n2,n3 a1,a2,a3 t1,t2,t3
Spider Code
import scrapy
from abc.items import abcItem
from scrapy.contrib.loader import ItemLoader
class abcSpider(scrapy.Spider):
name = "abc"
allowed_domains = ["abc.com"]
start_urls = ["abc.com/"]
def parse(self, response):
items = []
l = ItemLoader(item=abcItem(), response=response)
l.add_xpath('name', '//section[@class="abcrp"]/a/@title')
l.add_xpath('tele', '//p[@class="abcw"]/a/@href')
l.add_xpath('addr', '//span[@class="dn"]/text()')
return l.load_item()
items code
import scrapy
class abcItem(scrapy.Item):
name = scrapy.Field()
addr = scrapy.Field()
tele = scrapy.Field()
I was able to resolve this. I used a for loop on a outer tag, that contained my name, addr and tele tags