Python: generator expression vs. yield

Question

In Python, is there any difference between creating a generator object through a generator expression versus using the yield statement?

Using yield:

def Generator(x, y):
    for i in xrange(x):
        for j in xrange(y):
            yield(i, j)

Using generator expression:

def Generator(x, y):
    return ((i, j) for i in xrange(x) for j in xrange(y))

Both functions return generator objects, which produce tuples, e.g. (0,0), (0,1) etc.

Any advantages of one or the other? Thoughts?

---

Thanks everybody! There is a lot of great information and further references in these answers!

Answer 1

In usage, note a distinction between a generator object vs a generator function.

A generator object is use-once-only, in contrast to a generator function, which can be reused each time you call it again, because it returns a fresh generator object.

Generator expressions are in practice usually used "raw", without wrapping them in a function, and they return a generator object.

E.g.:

def range_10_gen_func():
    x = 0
    while x < 10:
        yield x
        x = x + 1

print(list(range_10_gen_func()))
print(list(range_10_gen_func()))
print(list(range_10_gen_func()))

which outputs:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Compare with a slightly different usage:

range_10_gen = range_10_gen_func()
print(list(range_10_gen))
print(list(range_10_gen))
print(list(range_10_gen))

which outputs:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[]
[]

And compare with a generator expression:

range_10_gen_expr = (x for x in range(10))
print(list(range_10_gen_expr))
print(list(range_10_gen_expr))
print(list(range_10_gen_expr))

which also outputs:

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[]
[]

Answer 2

In this example, not really. But yield can be used for more complex constructs - for example it can accept values from the caller as well and modify the flow as a result. Read PEP 342 for more details (it's an interesting technique worth knowing).

Anyway, the best advice is use whatever is clearer for your needs.

P.S. Here's a simple coroutine example from Dave Beazley:

def grep(pattern):
    print "Looking for %s" % pattern
    while True:
        line = (yield)
        if pattern in line:
            print line,

# Example use
if __name__ == '__main__':
    g = grep("python")
    g.next()
    g.send("Yeah, but no, but yeah, but no")
    g.send("A series of tubes")
    g.send("python generators rock!")