Python: Easily access deeply nested dict (get and set)

python

Solution

Attribute Tree

The problem with your first specification is that Python can't tell in `__getitem__` if, at `my_obj.a.b.c.d`, you will next proceed farther down a nonexistent tree, in which case it needs to return an object with a `__getitem__` method so you won't get an `AttributeError` thrown at you, or if you want a value, in which case it needs to return `None`.

I would argue that in every case you have above, you should expect it to throw a `KeyError` instead of returning `None`. The reason being that you can't tell if `None` means "no key" or "someone actually stored `None` at that location". For this behavior, all you have to do is take `dotdictify`, remove `marker`, and replace `__getitem__` with:

def __getitem__(self, key):
    return self[key]

Because what you really want is a `dict` with `__getattr__` and `__setattr__`.

There may be a way to remove `__getitem__` entirely and say something like `__getattr__ = dict.__getitem__`, but I think this may be over-optimization, and will be a problem if you later decide you want `__getitem__` to create the tree as it goes like `dotdictify` originally does, in which case you would change it to:

def __getitem__(self, key):
    if key not in self:
        dict.__setitem__(self, key, dotdictify())
    return dict.__getitem__(self, key)

I don't like the `marker` business in the original `dotdictify`.

Path Support

The second specification (override `get()` and `set()`) is that a normal `dict` has a `get()` that operates differently from what you describe and doesn't even have a `set` (though it has a `setdefault()` which is an inverse operation to `get()`). People expect `get` to take two parameters, the second being a default if the key isn't found.

If you want to extend `__getitem__` and `__setitem__` to handle dotted-key notation, you'll need to modify `doctictify` to:

class dotdictify(dict):
    def __init__(self, value=None):
        if value is None:
            pass
        elif isinstance(value, dict):
            for key in value:
                self.__setitem__(key, value[key])
        else:
            raise TypeError, 'expected dict'

    def __setitem__(self, key, value):
        if '.' in key:
            myKey, restOfKey = key.split('.', 1)
            target = self.setdefault(myKey, dotdictify())
            if not isinstance(target, dotdictify):
                raise KeyError, 'cannot set "%s" in "%s" (%s)' % (restOfKey, myKey, repr(target))
            target[restOfKey] = value
        else:
            if isinstance(value, dict) and not isinstance(value, dotdictify):
                value = dotdictify(value)
            dict.__setitem__(self, key, value)

    def __getitem__(self, key):
        if '.' not in key:
            return dict.__getitem__(self, key)
        myKey, restOfKey = key.split('.', 1)
        target = dict.__getitem__(self, myKey)
        if not isinstance(target, dotdictify):
            raise KeyError, 'cannot get "%s" in "%s" (%s)' % (restOfKey, myKey, repr(target))
        return target[restOfKey]

    def __contains__(self, key):
        if '.' not in key:
            return dict.__contains__(self, key)
        myKey, restOfKey = key.split('.', 1)
        target = dict.__getitem__(self, myKey)
        if not isinstance(target, dotdictify):
            return False
        return restOfKey in target

    def setdefault(self, key, default):
        if key not in self:
            self[key] = default
        return self[key]

    __setattr__ = __setitem__
    __getattr__ = __getitem__

Test code:

>>> life = dotdictify({'bigBang': {'stars': {'planets': {}}}})
>>> life.bigBang.stars.planets
{}
>>> life.bigBang.stars.planets.earth = { 'singleCellLife' : {} }
>>> life.bigBang.stars.planets
{'earth': {'singleCellLife': {}}}
>>> life['bigBang.stars.planets.mars.landers.vikings'] = 2
>>> life.bigBang.stars.planets.mars.landers.vikings
2
>>> 'landers.vikings' in life.bigBang.stars.planets.mars
True
>>> life.get('bigBang.stars.planets.mars.landers.spirit', True)
True
>>> life.setdefault('bigBang.stars.planets.mars.landers.opportunity', True)
True
>>> 'landers.opportunity' in life.bigBang.stars.planets.mars
True
>>> life.bigBang.stars.planets.mars
{'landers': {'opportunity': True, 'vikings': 2}}

Problem

Original source

Related problems

• Access nested dictionary items via a list of keys?
• Recursively access dict via attributes as well as index access?