Convert nested Python dict to object?

Question

I'm searching for an elegant way to get data using attribute access on a dict with some nested dicts and lists (i.e. javascript-style object syntax).

For example:

>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}

Should be accessible in this way:

>>> x = dict2obj(d)
>>> x.a
1
>>> x.b.c
2
>>> x.d[1].foo
bar

I think, this is not possible without recursion, but what would be a nice way to get an object style for dicts?

Answer 1

I had some problems with __getattr__ not being called so I constructed a new style class version:

class Struct(object):
    '''The recursive class for building and representing objects with.'''
    class NoneStruct(object):
        def __getattribute__(*args):
            return Struct.NoneStruct()

        def __eq__(self, obj):
            return obj == None

    def __init__(self, obj):
        for k, v in obj.iteritems():
            if isinstance(v, dict):
                setattr(self, k, Struct(v))
            else:
                setattr(self, k, v)

    def __getattribute__(*args):
        try:
            return object.__getattribute__(*args)
        except:            
            return Struct.NoneStruct()

    def __repr__(self):
        return '{%s}' % str(', '.join('%s : %s' % (k, repr(v)) for 
(k, v) in self.__dict__.iteritems()))

This version also has the addition of NoneStruct that is returned when an attribute is called that is not set. This allows for None testing to see if an attribute is present. Very usefull when the exact dict input is not known (settings etc.).

bla = Struct({'a':{'b':1}})
print(bla.a.b)
>> 1
print(bla.a.c == None)
>> True

Answer 2

If you want to access dict keys as an object (or as a dict for difficult keys), do it recursively, and also be able to update the original dict, you could do:

class Dictate(object):
    """Object view of a dict, updating the passed in dict when values are set
    or deleted. "Dictate" the contents of a dict...: """

    def __init__(self, d):
        # since __setattr__ is overridden, self.__dict = d doesn't work
        object.__setattr__(self, '_Dictate__dict', d)

    # Dictionary-like access / updates
    def __getitem__(self, name):
        value = self.__dict[name]
        if isinstance(value, dict):  # recursively view sub-dicts as objects
            value = Dictate(value)
        return value

    def __setitem__(self, name, value):
        self.__dict[name] = value
    def __delitem__(self, name):
        del self.__dict[name]

    # Object-like access / updates
    def __getattr__(self, name):
        return self[name]

    def __setattr__(self, name, value):
        self[name] = value
    def __delattr__(self, name):
        del self[name]

    def __repr__(self):
        return "%s(%r)" % (type(self).__name__, self.__dict)
    def __str__(self):
        return str(self.__dict)

Example usage:

d = {'a': 'b', 1: 2}
dd = Dictate(d)
assert dd.a == 'b'  # Access like an object
assert dd[1] == 2  # Access like a dict
# Updates affect d
dd.c = 'd'
assert d['c'] == 'd'
del dd.a
del dd[1]
# Inner dicts are mapped
dd.e = {}
dd.e.f = 'g'
assert dd['e'].f == 'g'
assert d == {'c': 'd', 'e': {'f': 'g'}}

Answer 3

Here is another way to implement SilentGhost's original suggestion:

def dict2obj(d):
  if isinstance(d, dict):
    n = {}
    for item in d:
      if isinstance(d[item], dict):
        n[item] = dict2obj(d[item])
      elif isinstance(d[item], (list, tuple)):
        n[item] = [dict2obj(elem) for elem in d[item]]
      else:
        n[item] = d[item]
    return type('obj_from_dict', (object,), n)
  else:
    return d

Answer 4

If your dict is coming from json.loads(), you can turn it into an object instead (rather than a dict) in one line:

import json
from collections import namedtuple

json.loads(data, object_hook=lambda d: namedtuple('X', d.keys())(*d.values()))

See also How to convert JSON data into a Python object.

Answer 5

>>> def dict2obj(d):
        if isinstance(d, list):
            d = [dict2obj(x) for x in d]
        if not isinstance(d, dict):
            return d
        class C(object):
            pass
        o = C()
        for k in d:
            o.__dict__[k] = dict2obj(d[k])
        return o


>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
>>> x = dict2obj(d)
>>> x.a
1
>>> x.b.c
2
>>> x.d[1].foo
'bar'

Answer 6

I ended up trying BOTH the AttrDict and the Bunch libraries and found them to be way too slow for my uses. After a friend and I looked into it, we found that the main method for writing these libraries results in the library aggressively recursing through a nested object and making copies of the dictionary object throughout. With this in mind, we made two key changes. 1) We made attributes lazy-loaded 2) instead of creating copies of a dictionary object, we create copies of a light-weight proxy object. This is the final implementation. The performance increase of using this code is incredible. When using AttrDict or Bunch, these two libraries alone consumed 1/2 and 1/3 respectively of my request time(what!?). This code reduced that time to almost nothing(somewhere in the range of 0.5ms). This of course depends on your needs, but if you are using this functionality quite a bit in your code, definitely go with something simple like this.

class DictProxy(object):
    def __init__(self, obj):
        self.obj = obj

    def __getitem__(self, key):
        return wrap(self.obj[key])

    def __getattr__(self, key):
        try:
            return wrap(getattr(self.obj, key))
        except AttributeError:
            try:
                return self[key]
            except KeyError:
                raise AttributeError(key)

    # you probably also want to proxy important list properties along like
    # items(), iteritems() and __len__

class ListProxy(object):
    def __init__(self, obj):
        self.obj = obj

    def __getitem__(self, key):
        return wrap(self.obj[key])

    # you probably also want to proxy important list properties along like
    # __iter__ and __len__

def wrap(value):
    if isinstance(value, dict):
        return DictProxy(value)
    if isinstance(value, (tuple, list)):
        return ListProxy(value)
    return value

See the original implementation here by https://stackoverflow.com/users/704327/michael-merickel.

The other thing to note, is that this implementation is pretty simple and doesn't implement all of the methods you might need. You'll need to write those as required on the DictProxy or ListProxy objects.