Scrapy: Default values for items & fields. What is

Question

As far as I could find out from the documentation and various discussions on the net, the ability to add default values to fields in a scrapy item has been removed.

This doesn't work

category = Field(default='null')

So my question is: what is a good way to initialize fields with a default value?

I already tried to implement it as a item pipeline as suggested here, without any success. https://groups.google.com/forum/?fromgroups=#!topic/scrapy-users/-v1p5W41VDQ

Answer 1

figured out what the problem was. the pipeline is working (code follows for other people's reference). my problem was, that I am appending values to a field. and I wanted the default method work on one of these listvalues... chose a different way and it works. I am now implementing it with a custom setDefault processor method.

class DefaultItemPipeline(object):

def process_item(self, item, spider):
    item.setdefault('amz_VendorsShippingDurationFrom', 'default')
    item.setdefault('amz_VendorsShippingDurationTo', 'default')
    # ...
    return item