Encoding nested python object in JSON

Question

I want to encode objects in JSON. But, I can not figure out how to make the output without the string escaping.

import json

class Abc:
    def __init__(self):
        self.name="abc name"
    def toJSON(self):
        return json.dumps(self.__dict__, cls=ComplexEncoder)

class Doc:
    def __init__(self):
        self.abc=Abc()
    def toJSON(self):
        return json.dumps(self.__dict__, cls=ComplexEncoder)

class ComplexEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Abc) or isinstance(obj, Doc):
            return obj.toJSON()
        else:
            return json.JSONEncoder.default(self, obj)

doc=Doc()
print doc.toJSON()

The result is (the dumps returns a string representation, that's why the " are escaped)

{"abc": "{\"name\": \"abc name\"}"}

I want something a little bit different. The expected result is

{"abc": {"name": "abc name"}"}

But I don't see how to... Any hint ?

thanks in advance.

Answer 1

my previous sample, with another nested object and your advices :

import json

class Identity:
    def __init__(self):
        self.name="abc name"
        self.first="abc first"
        self.addr=Addr()
    def reprJSON(self):
        return dict(name=self.name, firstname=self.first, address=self.addr) 

class Addr:
    def __init__(self):
        self.street="sesame street"
        self.zip="13000"
    def reprJSON(self):
        return dict(street=self.street, zip=self.zip) 

class Doc:
    def __init__(self):
        self.identity=Identity()
        self.data="all data"
    def reprJSON(self):
        return dict(id=self.identity, data=self.data) 

class ComplexEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj,'reprJSON'):
            return obj.reprJSON()
        else:
            return json.JSONEncoder.default(self, obj)

doc=Doc()
print "Str representation"
print doc.reprJSON()
print "Full JSON"
print json.dumps(doc.reprJSON(), cls=ComplexEncoder)
print "Partial JSON"
print json.dumps(doc.identity.addr.reprJSON(), cls=ComplexEncoder)

produces the expected result :

Str representation
{'data': 'all data', 'id': <__main__.Identity instance at 0x1005317e8>}
Full JSON
{"data": "all data", "id": {"name": "abc name", "firstname": "abc first", "address": {"street": "sesame street", "zip": "13000"}}}
Partial JSON
{"street": "sesame street", "zip": "13000"}

Thanks.

Answer 2

To avoid repetition of code like in Fred Laurent's answer I overloaded the __iter__() method as follows. This also permits to 'jsonize' list elements, datetime and decimal with no extra dependencies, just use dict().

import datetime
import decimal


class Jsonable(object):
    def __iter__(self):
        for attr, value in self.__dict__.iteritems():
            if isinstance(value, datetime.datetime):
                iso = value.isoformat()
                yield attr, iso
            elif isinstance(value, decimal.Decimal):
                yield attr, str(value)
            elif(hasattr(value, '__iter__')):
                if(hasattr(value, 'pop')):
                    a = []
                    for subval in value:
                        if(hasattr(subval, '__iter__')):
                            a.append(dict(subval))
                        else:
                            a.append(subval)
                    yield attr, a
                else:
                    yield attr, dict(value)
            else:
                yield attr, value

class Identity(Jsonable):
    def __init__(self):
        self.name="abc name"
        self.first="abc first"
        self.addr=Addr()

class Addr(Jsonable):
    def __init__(self):
        self.street="sesame street"
        self.zip="13000"

class Doc(Jsonable):
    def __init__(self):
        self.identity=Identity()
        self.data="all data"


def main():
    doc=Doc()
    print "-Dictionary- \n"
    print dict(doc)
    print "\n-JSON- \n"
    print json.dumps(dict(doc), sort_keys=True, indent=4)

if __name__ == '__main__':
    main()

The output:

-Dictionary- 

{'data': 'all data', 'identity': {'first': 'abc first', 'addr': {'street': 'sesame street', 'zip': '13000'}, 'name': 'abc name'}}

-JSON- 

{
    "data": "all data", 
    "identity": {
        "addr": {
            "street": "sesame street", 
            "zip": "13000"
        }, 
        "first": "abc first", 
        "name": "abc name"
    }
}

Hope it helps! Thanks

Answer 3

So, the immediate problem is that you're passing the json module a JSON value, which will get encoded as just another string in the JSON value.

The broader problem is that you're greatly overcomplicating this.

Drawing on JSON datetime between Python and JavaScript, I'd go with something closer to this:

import json

class Abc:
    def __init__(self):
        self.name="abc name"
    def jsonable(self):
        return self.name

class Doc:
    def __init__(self):
        self.abc=Abc()
    def jsonable(self):
        return self.__dict__

def ComplexHandler(Obj):
    if hasattr(Obj, 'jsonable'):
        return Obj.jsonable()
    else:
        raise TypeError, 'Object of type %s with value of %s is not JSON serializable' % (type(Obj), repr(Obj))

doc=Doc()
print json.dumps(doc, default=ComplexHandler)

which gets you:

~$ python nestjson.py 
{"abc": "abc name"}
~$ 

This can be made cleaner/saner/safer (in particular, just grabbing __dict__ isn't generally a recommended thing to do outside debugging/troubleshooting), but it should get the point across. All you need, fundamentally, is a way to get a json-compatible object (whether that's a simple string or number, or a list or dict) out of each "node" in the tree. That object should not be an already-JSON-serialized object, which is what you were doing.

Answer 4

This is what you're looking for: https://github.com/jsonpickle/jsonpickle

It does nested serialization of Python objects and can easily be extended to serialize custom types.

Answer 5

I could not add this as a comment and adding as answer. Fred's final sample was useful for me.I was told jsonpickle does this, but could not get the module to install and run properly. So used the code here. Minor tweak though, I had way too many variables to add by hand to some of the objects. So this little loop simplified things:

def reprJSON(self):
    d = dict()
    for a, v in self.__dict__.items():
        if (hasattr(v, "reprJSON")):
            d[a] = v.reprJSON()
        else:
            d[a] = v
    return d

It can be used in any object that has a subclass that is too busy to hand encode. Or can be made a helper for all classes. This also works for the full JSON presentation of member arrays that contain other classes (as long as they implement reprJSON() of course).