Chained, nested dict() get calls in python

Question

I'm interrogating a nested dictionary using the dict.get('keyword') method. Currently my syntax is...

M = cursor_object_results_of_db_query

for m in M:
    X = m.get("gparents").get("parent").get("child")
    for x in X:
        y = x.get("key")

However, sometimes one of the "parent" or "child" tags doesn't exist, and my script fails. I know using get() I can include a default in the case the key doesn't exist of the form...

get("parent", '') or
get("parent", 'orphan') 

But if I include any Null, '', or empty I can think of, the chained .get("child") fails when called on ''.get("child") since "" has no method .get().

The way I'm solving this now is by using a bunch of sequential try-except around each .get("") call, but that seems foolish and unpython---is there a way to default return "skip" or "pass" or something that would still support chaining and fail intelligently, rather than deep-dive into keys that don't exist?

Ideally, I'd like this to be a list comprehension of the form:

[m.get("gparents").get("parent").get("child") for m in M]

but this is currently impossible when an absent parent causes the .get("child") call to terminate my program.

Answer 1

Since these are all python dicts and you are calling the dict.get() method on them, you can use an empty dict to chain:

[m.get("gparents", {}).get("parent", {}).get("child") for m in M]

By leaving off the default for the last .get() you fall back to None. Now, if any of the intermediary keys is not found, the rest of the chain will use empty dictionaries to look things up, terminating in .get('child') returning None.

Answer 2

How about using a small helper function?

def getn(d, path):
    for p in path:
        if p not in d:
            return None
        d = d[p]
    return d

and then

[getn(m, ["gparents", "parent", "child"]) for m in M]

Answer 3

Another approach is to recognize that if the key isn't found, dict.get returns None. However, None doesn't have an attribute .get, so it will throw an AttributeError:

for m in M:
    try:
       X = m.get("gparents").get("parent").get("child")
    except AttributeError:
       continue

    for x in X:
        y = x.get("key")
        #do something with `y` probably???

Just like Martijn's answer, this doesn't guarantee that X is iterable (non-None). Although, you could fix that by making the last get in the chain default to returning an empty list:

 try:
    X = m.get("gparents").get("parent").get("child",[])
 except AttributeError:
    continue

---

Finally, I think that probably the best solution to this problem is to use reduce:

try:
    X = reduce(dict.__getitem__,["gparents","parent","child"],m)
except (KeyError,TypeError):
    pass
else:
    for x in X:
       #do something with x

The advantage here is that you know if any of the gets failed based on the type of exception that was raised. It's possible that a get returns the wrong type, then you get a TypeError. If the dictionary doesn't have the key however, it raises a KeyError. You can handle those separately or together. Whatever works best for your use case.

Answer 4

I realise I'm a bit late for the part but here's the solution I came up with when faced with a similar problem:

def get_nested(dict_, *keys, default=None):
    if not isinstance(dict_, dict):
        return default
    elem = dict_.get(keys[0], default)
    if len(keys) == 1:
        return elem
    return get_nested(elem, *keys[1:], default=default)

For example:

In [29]: a = {'b': {'c': 1}}
In [30]: get_nested(a, 'b', 'c')
Out[30]: 1
In [31]: get_nested(a, 'b', 'd') is None
Out[31]: True