How to merge two json string in Python?

Question

I recently started working with Python and I am trying to concatenate one of my JSON String with existing JSON String. I am also working with Zookeeper so I get the existing json string from zookeeper node as I am using Python kazoo library.

# gets the data from zookeeper
data, stat = zk.get(some_znode_path)
jsonStringA = data.decode("utf-8")

if I print jsonStringA it gives me like this -

{"error_1395946244342":"valueA","error_1395952003":"valueB"}

But if I do print json.loads(jsonString) then it prints out like this -

{u'error_1395946244342': u'valueA', u'error_1395952003': u'valueB'}

Here jsonStringA will have my existing JSON String. Now I have another key-value pair which I need to add in the exiting jsonStringA -

Below is my Python code -

# gets the data from zookeeper
data, stat = zk.get(some_znode_path)
jsonStringA = data.decode("utf-8")

timestamp_in_ms = "error_"+str(int(round(time.time() * 1000)))
node = "/pp/tf/test/v1"
a,b,c,d = node.split("/")[1:]
host_info = "h1"
local_dc = "dc3"
step = "step2"

My existing jsonStringA will be like this after extracting from zookeeper -

{"error_1395946244342":"valueA","error_1395952003":"valueB"}

Now I need to append this key-value pair in the jsonStringA -

"timestamp_in_ms":"Error Occured on machine "+host_info+" in datacenter "+ local_dc +" on the "+ step +" of process "+ c +"

So in short I need to merge below key-value pair -

"error_1395952167":"Error Occured on machine h1 in datacenter dc3 on the step2 of process test"

So final JSON String will look like this -

{"error_1395946244342":"valueA","error_1395952003":"valueB","error_1395952167":"Error Occured on machine h1 in datacenter dc3 on the step2 of process test"}

Is this possible to do?

Answer 1

As of Python 3.5, you can merge two dicts with:

merged = {**dictA, **dictB}

(https://www.python.org/dev/peps/pep-0448/)

So:

jsonMerged = {**json.loads(jsonStringA), **json.loads(jsonStringB)}
asString = json.dumps(jsonMerged)

etc.

Answer 2

You can load both json strings into Python Dictionaries and then combine. This will only work if there are unique keys in each json string.

    import json

    a = json.loads(jsonStringA)
    b = json.loads(jsonStringB)
    c = dict(a.items() + b.items())
    # or c =  dict(a, **b)

Answer 3

What do you mean by merging? JSON objects are key-value data structure. What would be a key and a value in this case? I think you need to create new directory and populate it with old data:

d = {}
d["new_key"] = jsonStringA[<key_that_you_did_not_mention_here>] + \ 
               jsonStringB["timestamp_in_ms"]

Merging method is obviously up to you.

Answer 4

Merging json objects is fairly straight forward but has a few edge cases when dealing with key collisions. The biggest issues have to do with one object having a value of a simple type and the other having a complex type (Array or Object). You have to decide how you want to implement that. Our choice when we implemented this for json passed to chef-solo was to merge Objects and use the first source Object's value in all other cases.

This was our solution:

from collections import Mapping
import json


original = json.loads(jsonStringA)
addition = json.loads(jsonStringB)

for key, value in addition.iteritems():
    if key in original:
        original_value = original[key]
        if isinstance(value, Mapping) and isinstance(original_value, Mapping):
            merge_dicts(original_value, value)
        elif not (isinstance(value, Mapping) or 
                  isinstance(original_value, Mapping)):
            original[key] = value
        else:
            raise ValueError('Attempting to merge {} with value {}'.format(
                key, original_value))
    else:
        original[key] = value

You could add another case after the first case to check for lists if you want to merge those as well, or for specific cases when special keys are encountered.

Answer 5

To append key-value pairs to a json string, you can use dictA.update(dictB).

For your case, this will look like this:

dictA = json.loads(jsonStringA)
dictB = json.loads('{"error_1395952167":"Error Occured on machine h1 in datacenter dc3 on the step2 of process test"}')

dictA.update(dictB)
jsonStringA = json.dumps(dictA)

Answer 6

Assuming a and b are the dictionaries you want to merge:

c = {key: value for (key, value) in (a.items() + b.items())}

To convert your string to python dictionary you use the following:

import json
my_dict = json.loads(json_str)

---

Update: full code using strings:

# test cases for jsonStringA and jsonStringB according to your data input
jsonStringA = '{"error_1395946244342":"valueA","error_1395952003":"valueB"}'
jsonStringB = '{"error_%d":"Error Occured on machine %s in datacenter %s on the %s of process %s"}' % (timestamp_number, host_info, local_dc, step, c)

# now we have two json STRINGS
import json
dictA = json.loads(jsonStringA)
dictB = json.loads(jsonStringB)

merged_dict = {key: value for (key, value) in (dictA.items() + dictB.items())}

# string dump of the merged dict
jsonString_merged = json.dumps(merged_dict)

But I have to say that in general what you are trying to do is not the best practice. Please read a bit on python dictionaries.

---

Alternative solution:

jsonStringA = get_my_value_as_string_from_somewhere()
errors_dict = json.loads(jsonStringA)

new_error_str = "Error Ocurred in datacenter %s blah for step %s blah" % (datacenter, step)
new_error_key = "error_%d" % (timestamp_number)

errors_dict[new_error_key] = new_error_str

# and if I want to export it somewhere I use the following
write_my_dict_to_a_file_as_string(json.dumps(errors_dict))

And actually you can avoid all these if you just use an array to hold all your errors.