Memory leak when invoking __iadd__ via __get__ without using temporary

Question

trying modify a decorator not to use a `weakref`, I stumbled across the following behaviour: ``` import weakref class descriptor(object): def __get__(self, instance, owner): return proxy(instance) class proxy(object): def __init__(self, instance): self.instance = instance def __iadd__(self, other): return self class A(object): descr = descriptor() def is_leaky(test_fn): a = A() wr = weakref.ref(a) test_fn(a) del a return wr() is not None def test1(a): tmp = a.descr tmp += object() def test2(a): a.descr += object() print(is_leaky(test1)) # gives False print(is_leaky(test2)) # gives True!!! ``` This seems very odd to me, as I'd expect both cases to behave the same. Furthermore, from my understanding of reference counting and object lifetime, I was convinced that in both cases the object should be freed. I have tested it both on python2.7 and python3.3. Is this a bug or intentional behaviour? Is there a way to get both calls to have the expected results (free the object in question)? I d…

Accepted Answer

The two codepaths are not equivalent.

In-place add acts on two operators, the assignment target and the item added. In `test1` that is `temp`, a local variable, and the in-place addition is translated to following:

```
temp = temp.__iadd__(object())
```

and since you return `self` and `temp` refers to the same object, this becomes `temp = temp` and that reference is cleaned up after the function exits.

In `test2`, you complicated matters, since now descriptors are getting involved again:

```
a.descr += object() 
```

becomes:

```
a.descr = A.__dict__['descr'].__get__(a, A).__iadd__(object())
```

so you assign the result of `A.__dict__['descr'].__get__(a, A)` to the instance attribute `a.descr`; the descriptor has no `__set__()` method and is not consulted.

But, and here is the catch, the `proxy` object holds a reference to `a` itself, `a.descr.instance` is a reference to `a`! You created a circular reference.

This reference keeps the object alive long enough to show through the weak reference, but as soon as the garbage collection process runs and breaks this cycle, `a` will be gone anyway.

Moral of this story? Don't use `__iadd__` in combination with a non-data descriptor; include both `__get__` and `__set__` because you need to control what happens when the result is assigned back.

Memory leak when invoking iadd via get without using temporary

Solution

Problem