generator vs. list comprehension

问题:

I have something, when run as a list comprehension, runs fine.

It looks like,

[myClass().Function(things) for things in biggerThing]

Function is a method, and it builds a list. The method itself doesn't return anything, but lists get manipulated within.

Now when I change it to a generator ,

(myClass().Function(things) for things in biggerThing)

It doesn't manipulate the data like I would expect it to. In fact, it doesn't seem to manipulate it at all.

What is the functional difference between a list comprehension and a generator?

回答1:

Generators are evaluated on the fly, as they are consumed. So if you never iterate over a generator, its elements are never evaluated.

So, if you did:

for _ in (myClass().Function(things) for things in biggerThing):
    pass

Function would run.

---

Now, your intent really isn't clear here.

Instead, consider using map:

map(myClass().Function, biggerThing)  

Note that this will always use the same instance of MyClass

If that's a problem, then do:

for things in BiggerThing:
    myClass().Function(things)

回答2:

Generators are lazy evaluated. You need to process a generator in order to your function be evaluated. One can use collections.deque to consume a generator:

import collections
generator = (myClass().Function(thing) for thing in biggerThing) 
collections.deque(generator , maxlen=0)

And consider using @staticmethod or @classmethod, or change to

myfunc = myClass().Function
generator = (myfunc(thing) for thing in biggerThing) 
collections.deque(generator , maxlen=0)

to reduce new instance of myClass creation for each thing processing.

update, performance

1. collections vs iteration

def l():
    for x in range(100):
       y = x**2
      yield y

def consume(it):
    for i in it:
        pass

>>> timeit.timeit('from __main__ import l, consume; consume(l())', number=10000)
0.4535369873046875
>>> timeit.timeit('from __main__ import l, collections; collections.deque(l(), 0)', number=10000)
0.24533605575561523

2. instance vs class vs static methods

class Test(object):
    @staticmethod
    def stat_pow(x):
        return x**2
    @classmethod
    def class_pow(cls, x):
        return x**2
    def inst_pow(self, x):
        return x**2

def static_gen():
    for x in range(100):
        yield Test.stat_pow(x)

def class_gen():
    for x in range(100):
        yield Test.class_pow(x)

def inst_gen():
    for x in range(100):
        yield Test().inst_pow(x)

>>> timeit.timeit('from __main__ import static_gen as f, collections; collections.deque(f(), 0)', number=10000)
0.5983021259307861
>>> timeit.timeit('from __main__ import class_gen as f, collections; collections.deque(f(), 0)', number=10000)
0.6772890090942383
>>> timeit.timeit('from __main__ import inst_gen as f, collections; collections.deque(f(), 0)', number=10000)
0.8273470401763916

回答3:

When you create a generator, you are only able to use each element once. It's like I'm creating a batch of cookies that I'm eating as I go. They serve their purpose (make me happy), but they're gone once you use them.

List comprehensions create lists, and they will allow you to access that data structure forever (ostensibly). You can also use all the list methods on them (very useful). But the idea is that it creates an actual data structure (something that holds data for you).

Check out this post right here: Generators vs. List Comprehensions

回答4:

Generator won't execute the function, until you call next() on the generator.

 >>>def f():
 ...    print 'Hello'
 >>>l = [f() for _ in range(3)]
 Hello
 Hello
 Hello
 >>>g = (f() for _ in range(3)) # nothing happens 
 >>>
 >>>next(g)
 Hello

回答5:

List comprehension:

• List can be indexed. eg.,. [0, 1, 2, 3, 4][0]

• A created List can be used any number of times.

• An empty list occupies 72 bytes, and for each item adds occupies 8 bytes extra.

Generators:

• Generators cant be indexed

• A generator can be used only once.

• A generator occupies much lesser memory(80 bytes).

Please note that in case of generator, the content inside is emptied, once it is used.

>>> sys.getsizeof([])
72
>>> list1 = [x for x in range(0, 5)]
>>> sys.getsizeof(list1)
136
>>>
>>> generator1 = (x for x in range(0,100))
>>> sys.getsizeof(generator1)
80
>>> generator1 = (x for x in range(0,5))
>>> sys.getsizeof(generator1)
80
>>> list(generator1)
[0, 1, 2, 3, 4]
>>> list(generator1)
[]
>>>