Why is a Python generator confusing its scope with global in an exec'd script?

python, python-2.7, python-exec

Solution

Dodgy indeed.

From the doc:

If two separate objects are given as globals and locals, the code will be executed as if it were embedded in a class definition.

(Note that when you run `exec(..., globals(), locals())` at module level, this does not apply, because `globals() is locals()`, i.e. not two separate objects).

This indicates you can simply reproduce the problem by running this script:

class A(object):

  items = (
    'foo/bar',
    'foo/baz',
    'foof',
    'barf/fizz',
  )

  whitelist = (
    'foo/',
  )

  for key in items:
    try:
      # Not terribly efficient, but who cares; computers are fast.
      next(True for prefix in whitelist if key.startswith(prefix))
      # Found!
      print(key)
    except StopIteration:
      pass

"Ok, but why do I get the error here?"

So glad you asked. The answer is here.

Problem

Ok, so I'm working in an environment where a config script for a tool is an `exec`'d python script. The exec call is something like this: outer.py: ``` exec(open("inner.py").read(), globals(), {}) ``` Now, I want to do some relatively basic iteration within the `exec`'d script. In this case, doing work when some values aren't in the whitelist: inner.py: ``` items = ( 'foo/bar', 'foo/baz', 'foof', 'barf/fizz', ) whitelist = ( 'foo/', ) for key in items: try: # Not terribly efficient, but who cares; computers are fast. next(True for prefix in whitelist if key.startswith(prefix)) # Do some work here when the item doesn't match the whitelist. except StopIteration: print("%10s isn't in the whitelist!" % key) ``` Running `python inner.py` yields the expected result: ``` foof isn't in the whitelist! barf/fizz isn't in the whitelist! ``` Here's the strange part: Running `python outer.py` seems to show that the interpreter is confused about the scope of the generator: ``` Traceback (most recent call last): File "outer.py", line 1, in <module> exec(open("inner.py").read(), globals(), {}) File "<string>", line 15, in <module> File "<string>", line 15, in <genexpr> NameError: global name 'key' is not defined ``` Some other notes: You can `print(key)` just fine inside the `for` loop (before the generator is run). Replacing the empty dict with `locals()` in the `exec` line of `outer.py` resolves the issue, but that code is out of my control. I'm running the OS X built Python 2.7.2 (Mountain Lion).

Original source

Related problems