Python Lambda function that gets invoked for a dynamodb stream has JSON that has DynamoDB format (contains the data types in JSON). I would like to covert DynamoDB JSON to standard JSON. PHP and nodejs have Marshaler that can do this. Please let me know if there are similar or other options for Python.
DynamoDB_format = `{"feas":
{"M": {
"fea": {
"L": [
{
"M": {
"pre": {
"N": "1"
},
"Li": {
"N": "1"
},
"Fa": {
"N": "0"
},
"Mo": {
"N": "1"
},
"Ti": {
"S": "20160618184156529"
},
"Fr": {
"N": "4088682"
}
}
}
]
}
}
}
}`
Update: There is a library now: https://pypi.org/project/dynamodb-json/
Here is an improved version of indiangolfer's answer.
While indiangolfer's solution works for the question, this improved version might be more useful for others who stumble upon this thread.
def unmarshal_dynamodb_json(node):
data = dict({})
data['M'] = node
return _unmarshal_value(data)
def _unmarshal_value(node):
if type(node) is not dict:
return node
for key, value in node.items():
# S – String - return string
# N – Number - return int or float (if includes '.')
# B – Binary - not handled
# BOOL – Boolean - return Bool
# NULL – Null - return None
# M – Map - return a dict
# L – List - return a list
# SS – String Set - not handled
# NN – Number Set - not handled
# BB – Binary Set - not handled
key = key.lower()
if key == 'bool':
return value
if key == 'null':
return None
if key == 's':
return value
if key == 'n':
if '.' in str(value):
return float(value)
return int(value)
if key in ['m', 'l']:
if key == 'm':
data = {}
for key1, value1 in value.items():
if key1.lower() == 'l':
data = [_unmarshal_value(n) for n in value1]
else:
if type(value1) is not dict:
return _unmarshal_value(value)
data[key1] = _unmarshal_value(value1)
return data
data = []
for item in value:
data.append(_unmarshal_value(item))
return data
It is improved in the following ways:
handles more data types, including lists, which were not handled correctly previously
handles lowercase and uppercase keys
Edit: fix recursive object bug
I couldn't find anything out in the wild. So, I decided to port the PHP implementation of dynamodb json to standard json that was published here. I tested this in a python lambda function processing DynamoDB stream. If there is a better way to do this, please let me know.
(PS: This is not a complete port of PHP Marshaler)
The JSON in the question gets transformed to:
{
"feas":{
"fea":[
{
"pre":"1",
"Mo":"1",
"Ti":"20160618184156529",
"Fa":"0",
"Li":"1",
"Fr":"4088682"
}
]
}
}
def unmarshalJson(node):
data = {}
data["M"] = node
return unmarshalValue(data, True)
def unmarshalValue(node, mapAsObject):
for key, value in node.items():
if(key == "S" or key == "N"):
return value
if(key == "M" or key == "L"):
if(key == "M"):
if(mapAsObject):
data = {}
for key1, value1 in value.items():
data[key1] = unmarshalValue(value1, mapAsObject)
return data
data = []
for item in value:
data.append(unmarshalValue(item, mapAsObject))
return data
