Difference between generator expression and generator function

python, python-3.x

Solution

The functions you provided have completely different semantics in the general case.

The first one, with `yield from`, passes the control to the iterable. This means that calls to `send()` and `throw()` during the iteration will be handled by the iterable and not by the function you are defining.

The second function only iterates over the elements of the iterable, and it will handle all the calls to `send()` and `throw()`. To see the difference check this code:

In [8]: def action():
   ...:     try:
   ...:         for el in range(4):
   ...:             yield el
   ...:     except ValueError:
   ...:         yield -1
   ...:         

In [9]: def f():
   ...:     yield from action()
   ...:     

In [10]: def g():
    ...:     return (el for el in action())
    ...: 

In [11]: x = f()

In [12]: next(x)
Out[12]: 0

In [13]: x.throw(ValueError())
Out[13]: -1

In [14]: next(x)
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-14-5e4e57af3a97> in <module>()
----> 1 next(x)

StopIteration: 

In [15]: x = g()

In [16]: next(x)
Out[16]: 0

In [17]: x.throw(ValueError())
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-17-1006c792356f> in <module>()
----> 1 x.throw(ValueError())

<ipython-input-10-f156e9011f2f> in <genexpr>(.0)
      1 def g():
----> 2     return (el for el in action())
      3 

ValueError: 

In fact, due to this reason, `yield from` probably has a higher overhead than the genexp, even though it is probably irrelevant.

Use `yield from` only when the above behaviour is what you want or if you are iterating over a simple iterable that is not a generator (so that `yield from` is equivalent to a loop + simple `yield`s).

Stylistically speaking I'd prefer:

def h():
    for el in range(4):
        yield el

Instead of `return`ing a genexp or using `yield from` when dealing with generators.

In fact the code used by the generator to perform the iteration is almost identical to the above function:

In [22]: dis.dis((i for i in range(4)).gi_code)
  1           0 LOAD_FAST                0 (.0)
        >>    3 FOR_ITER                11 (to 17)
              6 STORE_FAST               1 (i)
              9 LOAD_FAST                1 (i)
             12 YIELD_VALUE
             13 POP_TOP
             14 JUMP_ABSOLUTE            3
        >>   17 LOAD_CONST               0 (None)
             20 RETURN_VALUE

As you can see it does a `FOR_ITER` + `YIELD_VALUE`. note that the argument (`.0`), is `iter(range(4))`. The bytecode of the function also contains the calls to `LOAD_GLOBAL` and `GET_ITER` that are required to lookup `range` and obtain its iterable. However this actions must be performed by the genexp too, just not inside its code but before calling it.

Problem

Original source