Make sure all dicts in a list have the same keys

Question

I have a list with dictionaries like

[{'x': 42}, {'x': 23, 'y': 5}]

and want to make sure all dicts have the same keys, with values of None if the key was not present in the original dict. So the list above should become

[{'x': 42, 'y': None}, {'x': 23, 'y': 5}]

What's the most beautiful and pythonic way to do this? Current approach:

keys = reduce(lambda k, l: k.union(set(l)), [d.keys() for d in my_list], set())
new_list = [dict.fromkeys(keys, None) for i in xrange(len(my_list))]
for i, l in enumerate(my_list):
    new_list[i].update(l)

But especially the first two lines seem kind of clumsy. Ideas?

Answer 1

This creates a new list of dictionaries, all of them with complete keys:

>>> import itertools as it
>>> l = [{'x': 42}, {'x': 23, 'y': 5}]
>>> all_keys = set(it.chain.from_iterable(l))
>>> [dict((k, a.get(k, None)) for k in all_keys) for a in l]
[{'x': 42, 'y': None}, {'x': 23, 'y': 5}]

Answer 2

The easiest way to do this:

from itertools import chain

dicts = [{'x': 42}, {'x': 23, 'y': 5}]

keys = set(chain.from_iterable(dicts))
for item in dicts:
     item.update({key: None for key in keys if key not in item})

Giving us:

[{'y': None, 'x': 42}, {'y': 5, 'x': 23}]

We make a set from all the keys in all the dictionaries, then we loop through the dicts updating with any values they don't have.

An alternative to using itertools.chain.from_iterable() would be be to do reduce(or_, [dict.keys() for dict in dicts]), using functools.reduce() (in 3.x, the reduce() builtin in 2.x) and operator.or_, although I feel this is less readable.

If you wanted to create a new list, rather than updating the old one, simply replace the for loop with:

newdicts = [{key: item.get(key, None) for key in keys} for item in dicts]

Answer 3

>>> from itertools import chain 
>>> l = [{'x': 42}, {'x': 23, 'y': 5}]
>>> all_keys = set(chain.from_iterable(l))   
>>> for d in l:
        d.update((k,None) for k in all_keys-d.viewkeys())


>>> l
[{'y': None, 'x': 42}, {'y': 5, 'x': 23}]