Change values in dict of nested dicts using items

Question

How would you modify/create keys/values in a dict of nested dicts based on the values of a list, in which the last item of the list is a value for the dict, and the rest of items reefer to keys within dicts? This would be the list:

list_adddress = [ "key1", "key1.2", "key1.2.1", "value" ]

This would only be a problem in situations like when parsing command line arguments. It's obvious that modifying/creating this value within a script would be pretty easy using dict_nested["key1"]["key1.2"]["key1.2.1"]["value"].

This would be a nested dict of dicts:

dict_nested = { 
    "key1": {
                "key1.1": { 
                            "...": "...",
                },
                "key1.2": { 
                            "key1.2.1": "change_this",
                },
            },

    "key2": {
                "...": "..."
            },
}

I guess that in this case, something like a recursive function or a list comprehension would be required.

def ValueModify(list_address, dict_nested):
    ...
    ...
    ValueModify(..., ...)

Also, if items in list_address would reefer to keys in non-existing dictionaries, they should be created.

Answer 1

One-liner:

keys, (newkey, newvalue) = list_address[:-2], list_address[-2:]
reduce(dict.__getitem__, keys, dict_nested)[newkey] = newvalue

Note: dict.get and operator.getitem would produce wrong exceptions here.

An explicit for-loop as in Joel Cornett's answer might be more readable.

If you want to create non-existing intermediate dictionaries:

reduce(lambda d,k: d.setdefault(k, {}), keys, dict_nested)[newkey] = newvalue

If you want to override existing intermediate values that are not dictionaries e.g., strings, integers:

from collections import MutableMapping

def set_value(d, keys, newkey, newvalue, default_factory=dict):
    """
    Equivalent to `reduce(dict.get, keys, d)[newkey] = newvalue`
    if all `keys` exists and corresponding values are of correct type
    """
    for key in keys:
        try:
            val = d[key]
        except KeyError:
            val = d[key] = default_factory()
        else:
            if not isinstance(val, MutableMapping):
                val = d[key] = default_factory()
        d = val
    d[newkey] = newvalue

Example

list_address = ["key1", "key1.2", "key1.2.1", "key1.2.1.1", "value"]
dict_nested = {
    "key1": {
                "key1.1": {
                            "...": "...",
                },
                "key1.2": {
                            "key1.2.1": "change_this",
                },
            },

    "key2": {
                "...": "..."
            },
}

set_value(dict_nested, list_address[:-2], *list_address[-2:])
assert reduce(dict.get, list_address[:-1], dict_nested) == list_address[-1]

Tests

>>> from collections import OrderedDict
>>> d = OrderedDict()
>>> set_value(d, [], 'a', 1, OrderedDict) # non-existent key
>>> d.items()
[('a', 1)]
>>> set_value(d, 'b', 'a', 2) # non-existent intermediate key
>>> d.items()
[('a', 1), ('b', {'a': 2})]
>>> set_value(d, 'a', 'b', 3) # wrong intermediate type
>>> d.items()
[('a', {'b': 3}), ('b', {'a': 2})]
>>> d = {}
>>> set_value(d, 'abc', 'd', 4)
>>> reduce(dict.get, 'abcd', d) == d['a']['b']['c']['d'] == 4
True
>>> from collections import defaultdict
>>> autovivify = lambda: defaultdict(autovivify)
>>> d = autovivify()
>>> set_value(d, 'abc', 'd', 4)
>>> reduce(dict.get, 'abcd', d) == d['a']['b']['c']['d'] == 4
True
>>> set_value(1, 'abc', 'd', 4) #doctest:+IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
TypeError:
>>> set_value([], 'abc', 'd', 4) #doctest:+IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
TypeError:
>>> L = [10]
>>> set_value(L, [0], 2, 3)
>>> L
[{2: 3}]

Answer 2

I think this works like you're after.

def ValueModify(l, d):
  if l[0] not in d:
    d[l[0]] = dict()
  if isinstance(d[l[0]], dict):
    ValueModify(l[1:], d[l[0]])
  else:
    d[l[0]] = l[1]

I'm using isinstance, which is type checking, and not generally something you do in Python, but it does set the value as expected.

-- Edited --

Added in the missing key check to set nested values if the original nested_dict is not fully populated.

Answer 3

address_list = ["key1", "key1.1", "key1.2", "value"]

def set_value(dict_nested, address_list):
    cur = dict_nested
    for path_item in address_list[:-2]:
        try:
            cur = cur[path_item]
        except KeyError:
            cur = cur[path_item] = {}
    cur[address_list[-2]] = address_list[-1]

Answer 4

Here's a recursive solution.

def unravel(d, keys):
    i = keys[0]
    keys = keys[1:]
    tmpDict = d[i]
    if type(tmpDict) != type({}):
        return tmpDict
    else:
        return unravel(tmpDict, keys)

Answer 5

To insert a new key-value pair or update the value of a pair:

import copy
def update_nested_map(d, u, *keys):
    d = copy.deepcopy(d)
    keys = keys[0]
    if len(keys) > 1:
        d[keys[0]] = update_nested_map(d[keys[0]], u, keys[1:])
    else:
        d[keys[0]] = u
    return d

test:

    >>> d = {'m': {'d': {'v': {'w': 1}}}}

    >>> update_nested_map(d, 999, ['m', 'd', 'v', 'w'])
    {'m': {'d': {'v': {'w': 999}}}}

    >>> update_nested_map(d, 999, ['m', 'd', 'v', 'z'])
    {'m': {'d': {'v': {'z': 999, 'w': 1}}}}

    >>> update_nested_map(d, 999, ['m', 'd', 'l'])
    {'m': {'d': {'v': {'w': 1}, 'l': 999}}}

    >>> update_nested_map(d, 999, ['m','d'])
    {'m': {'d': 999}}