Performance discrepancy: obj.__setitem__(x,y) vs. obj[x] = y?

performance, python

Solution

Hmm that's interesting. If I run a pared-down version of your stuff:

setup="""
def getbrack(a, b):
    return a[b]

def getitem(a, b):
    return a.__getitem__(b)

def setbrack(a, b, c):
    a[b] = c

def setitem(a, b, c):
    return a.__setitem__(b, c)

a = {2: 3}
"""

`setitem` and `getitem` are both slower than their corresponding `setbrack` and `getbrack`:

>>> timeit.timeit("getbrack(a, 2)", setup, number=10000000)
1.1424450874328613
>>> timeit.timeit("getitem(a, 2)", setup, number=10000000)
1.5957350730895996
>>> timeit.timeit("setbrack(a, 2, 3)", setup, number=10000000)
1.4236340522766113
>>> timeit.timeit("setitem(a, 2, 3)", setup, number=10000000)
2.402789831161499

However, if I run exactly your test, then I get your same results - `GET 'Brackets'` is slower than `GET 'Methods'`.

This means it has something to do with the classes you're using and not bracket vs. setitem per se.

Now, if I modify the test to not refer back to `self`...

brackets = '''d = {}

class AttrDict2(dict):
    def __getattr__(self, key):
        return d[key]
    def __setattr__(self, key, val):
        d[key] = val

o = AttrDict2()
o.att = 1'''

methods = '''d = {}

class AttrDict2(dict):
    def __getattr__(self, key):
        return d.__getitem__(key)
    def __setattr__(self, key, val):
        d.__setitem__(key, val)

o = AttrDict2()
o.att = 1'''

Then I again get the behavior that brackets is always faster than methods. So perhaps it has something to do with how `self[]` works in a `dict` subclass?

Problem

I was writing up a simple `dict` subclass with attribute access, and I stumbled across something that seemed strange when I was optimizing. I had originally written the `__getattr__` and `__setattr__` methods as simple aliases to `self[key]` etc, but then I thought it should be faster to call `self.__getitem__` and `self.__setitem__` directly, since they would presumably be called under the hood by the `[key]` notation. Out of curiosity, I timed both implementations, and discovered some surprises. Below are the two implementations: not much going on here, as you can see. ``` # brackets class AttrDict(dict): def __getattr__(self, key): return self[key] def __setattr__(self, key, val): self[key] = val # methods class AttrDict(dict): def __getattr__(self, key): return self.__getitem__(key) def __setattr__(self, key, val): self.__setitem__(key, val) ``` Intuitively, I expected the second implementation to be slightly faster, since it presumably skips the step of going from the bracket notation into a function call. However, that's not exactly what my `timeit` results showed. ``` >>> methods = '''\ ... class AttrDict(dict): ... def __getattr__(self, key): ... return self.__getitem__(key) ... def __setattr__(self, key, val): ... self.__setitem__(key, val) ... o = AttrDict() ... o.att = 1 ... ''' >>> brackets = '''\ ... class AttrDict(dict): ... def __getattr__(self, key): ... return self[key] ... def __setattr__(self, key, val): ... self[key] = val ... ... o = AttrDict() ... o.att = 1 ... ''' >>> getting = 'foo = o.att' >>> setting = 'o.att = 1' ``` The code above is all just setup. Here are the tests: ``` >>> for op in (getting, setting): ... print('GET' if op == getting else 'SET') ... for setup in (brackets, methods): ... s = 'Brackets:' if setup == brackets else 'Methods:' ... print(s, min(timeit.repeat(op, setup, number=1000000, repeat=20))) ... GET Brackets: 1.109725879526195 Methods: 1.050940903987339 SET Brackets: 0.44571820606051915 Methods: 0.7166479863124096 >>> ``` As you can see, using `self.__getitem__` is very slightly faster than `self[key]`, but `self.__setitem__` is significantly slower than `self[key] = val`. This seems very odd--I know function call overhead can be large, but if that were the issue I'd expect to see the bracket notation faster in both cases, which is not happening here. I looked into it a little further; here are the `dis` results: ``` >>> exec(brackets) >>> dis.dis(AttrDict.__getattr__) 3 0 LOAD_FAST 0 (self) 3 LOAD_FAST 1 (key) 6 BINARY_SUBSCR 7 RETURN_VALUE >>> dis.dis(AttrDict.__setattr__) 5 0 LOAD_FAST 2 (val) 3 LOAD_FAST 0 (self) 6 LOAD_FAST 1 (key) 9 STORE_SUBSCR 10 LOAD_CONST 0 (None) 13 RETURN_VALUE >>> exec(methods) >>> dis.dis(AttrDict.__getattr__) 3 0 LOAD_FAST 0 (self) 3 LOAD_ATTR 0 (__getitem__) 6 LOAD_FAST 1 (key) 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 12 RETURN_VALUE >>> dis.dis(AttrDict.__setattr__) 5 0 LOAD_FAST 0 (self) 3 LOAD_ATTR 0 (__setitem__) 6 LOAD_FAST 1 (key) 9 LOAD_FAST 2 (val) 12 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 15 POP_TOP 16 LOAD_CONST 0 (None) 19 RETURN_VALUE ``` The only thing I can think of is that maybe the `POP_TOP` instruction has significant overhead compared to the rest of the calls, but can it really be that much? It's the only thing that stands out here... Can anyone see what's going on to make `__setitem__` so much slower than its bracket-cousin, relative to `__getitem__`? Potentially relevant info: CPython 3.3.2 32-bit on win32

Original source