Why is Python 3.x's super() magic?

Question

In Python 3.x, `super()` can be called without arguments: ``` class A(object): def x(self): print("Hey now") class B(A): def x(self): super().x() ``` ``` >>> B().x() Hey now ``` In order to make this work, some compile-time magic is performed, one consequence of which is that the following code (which rebinds `super` to `super_`) fails: ``` super_ = super class A(object): def x(self): print("No flipping") class B(A): def x(self): super_().x() ``` ``` >>> B().x() Traceback (most recent call last): File "", line 1, in File "", line 3, in x RuntimeError: super(): __class__ cell not found ``` Why is `super()` unable to resolve the superclass at runtime without assistance from the compiler? Are there practical situations in which this behaviour, or the underlying reason for it, could bite an unwary programmer? ... and, as a side question: are there any other examples in Python of functions, methods etc. which can be broken by rebinding them to a different name?

Accepted Answer

The new magic `super()` behaviour was added to avoid violating the D.R.Y. (Don't Repeat Yourself) principle, see PEP 3135. Having to explicitly name the class by referencing it as a global is also prone to the same rebinding issues you discovered with `super()` itself:

```
class Foo(Bar):
    def baz(self):
        return super(Foo, self).baz() + 42

Spam = Foo
Foo = something_else()

Spam().baz()  # liable to blow up
```

The same applies to using class decorators where the decorator returns a new object, which rebinds the class name:

```
@class_decorator_returning_new_class
class Foo(Bar):
    def baz(self):
        # Now `Foo` is a *different class*
        return super(Foo, self).baz() + 42
```

The magic `super()` `__class__` cell sidesteps these issues nicely by giving you access to the original class object.

The PEP was kicked off by Guido, who initially envisioned `super` becoming a keyword, and the idea of using a cell to look up the current class was also his. Certainly, the idea to make it a keyword was part of the first draft of the PEP.

However, it was in fact Guido himself who then stepped away from the keyword idea as 'too magical', proposing the current implementation instead. He anticipated that using a different name for `super()` could be a problem:

My patch uses an intermediate solution: it assumes you need `__class__` whenever you use a variable named `'super'`. Thus, if you (globally) rename `super` to `supper` and use `supper` but not `super`, it won't work without arguments (but it will still work if you pass it either `__class__` or the actual class object); if you have an unrelated variable named `super`, things will work but the method will use the slightly slower call path used for cell variables.

So, in the end, it was Guido himself that proclaimed that using a `super` keyword did not feel right, and that providing a magic `__class__` cell was an acceptable compromise.

I agree that the magic, implicit behaviour of the implementation is somewhat surprising, but `super()` is one of the most mis-applied functions in the language. Just take a look at all the misapplied `super(type(self), self)` or `super(self.__class__, self)` invocations found on the Internet; if any of that code was ever called from a derived class you'd end up with an infinite recursion exception. At the very least the simplified `super()` call, without arguments, avoids that problem.

As for the renamed `super_`; just reference `__class__` in your method as well and it'll work again. The cell is created if you reference either the `super` or `__class__` names in your method:

```
>>> super_ = super
>>> class A(object):
...     def x(self):
...         print("No flipping")
... 
>>> class B(A):
...     def x(self):
...         __class__  # just referencing it is enough
...         super_().x()
... 
>>> B().x()
No flipping
```

Why is Python 3.x's super() magic?

Solution

Problem

Related problems