Iterating over a dictionary in python and strippin

问题:

I am working with the web scraping framework Scrapy and I am a bit of a noob when it comes to python. So I am wondering how do I iterate over all of the scraped items which seem to be in a dictionary and strip the white space from each one.

Here is the code I have been playing with in my item pipeline.:

for info in item:
   info[info].lstrip()

But this code does not work, because I cannot select items individually. So I tried to do this:

for key, value item.items():
   value[1].lstrip()

This second method works to a degree, but the problem is that I have no idea how then to loop over all of the values.

I know this is probably such an easy fix, but I cannot seem to find it. Any help would be greatly appreciated. :)

回答1:

Not a direct answer to the question, but I would suggest you look at Item Loaders and input/output processors. A lot of your cleanup can be take care of here.

An example which strips each entry would be:

class ItemLoader(ItemLoader):

    default_output_processor = MapCompose(unicode.strip)

回答2:

In a dictionary comprehension (available in Python >=2.7):

clean_d = { k:v.strip() for k, v in d.iteritems()}

回答3:

What you should note is that lstrip() returns a copy of the string rather than modify the object. To actually update your dictionary, you'll need to assign the stripped value back to the item.

For example:

for k, v in your_dict.iteritems():
    your_dict[k] = v.lstrip()

Note the use of .iteritems() which returns an iterator instead of a list of key value pairs. This makes it somewhat more efficient.

I should add that in Python3, .item() has been changed to return "views" and so .iteritems() would not be required.

回答4:

Try

for k,v in item.items():
   item[k] = v.replace(' ', '')

or in a comprehensive way as suggested by monkut:

newDic = {k,v.replace(' ','') for k,v in item.items()}

回答5:

Although @zquare had the best answer for this question, I feel I need to chime in with a Pythonic method that will also account for dictionary values that are not strings. This is not recursive mind you, as it only works with one dimensional dictionary objects.

d.update({k: v.lstrip() for k, v in d.items() if isinstance(v, str) and v.startswith(' ')})

This updates the original dictionary value if the value is a string and starts with a space.

UPDATE: If you want to use Regular Expressions and avoid using starts with and endswith. You can use this:

import re
rex = re.compile(r'^\s|\s$')
d.update({k: v.strip() for k, v in d.items() if isinstance(v, str) and rex.search(v)})

This version strips if the value has a leading or trailing white space character.

回答6:

I use the following. You can pass any object as an argument, including a string, list or dictionary.

# strip any type of object
def strip_all(x):
  if isinstance(x, str): # if using python2 replace str with basestring to include unicode type
    x = x.strip()
  elif isinstance(x, list):
    x = [strip_all(v) for v in x]
  elif isinstance(x, dict):
    for k, v in x.iteritems():
      x.pop(k)  # also strip keys
      x[ strip_all(k) ] = strip_all(v)
  return x

回答7:

Assuming you would like to strip the values of yourDict creating a new dict called newDict:

newDict = dict(zip(yourDict.keys(), [v.strip() if isinstance(v,str) else v for v in yourDict.values()]))

This code can handle multi-type values, so will avoid stripping int, float, etc.