How to validate structure (or schema) of dictionar

Question

I have a dictionary with config info:

my_conf = {
    'version': 1,

    'info': {
        'conf_one': 2.5,
        'conf_two': 'foo',
        'conf_three': False,
        'optional_conf': 'bar'
    }
}

I want to check if the dictionary follows the structure I need.

I'm looking for something like this:

conf_structure = {
    'version': int,

    'info': {
        'conf_one': float,
        'conf_two': str,
        'conf_three': bool
    }
}

is_ok = check_structure(conf_structure, my_conf)

Is there any solution done to this problem or any library that could make implementing check_structure more easy?

Answer 1

Without using libraries, you could also define a simple recursive function like this:

def check_structure(struct, conf):
    if isinstance(struct, dict) and isinstance(conf, dict):
        # struct is a dict of types or other dicts
        return all(k in conf and check_structure(struct[k], conf[k]) for k in struct)
    if isinstance(struct, list) and isinstance(conf, list):
        # struct is list in the form [type or dict]
        return all(check_structure(struct[0], c) for c in conf)
    elif isinstance(struct, type):
        # struct is the type of conf
        return isinstance(conf, struct)
    else:
        # struct is neither a dict, nor list, not type
        return False

This assumes that the config can have keys that are not in your structure, as in your example.

---

Update: New version also supports lists, e.g. like 'foo': [{'bar': int}]

Answer 2

You can build structure using recursion:

def get_type(value):
    if isinstance(value, dict):
        return {key: get_type(value[key]) for key in value}
    else:
        return str(type(value))

And then compare required structure with your dictionary:

get_type(current_conf) == get_type(required_conf)

Example:

required_conf = {
    'version': 1,
    'info': {
        'conf_one': 2.5,
        'conf_two': 'foo',
        'conf_three': False,
        'optional_conf': 'bar'
    }
}

get_type(required_conf)

{'info': {'conf_two': "<type 'str'>", 'conf_one': "<type 'float'>", 'optional_conf': "<type 'str'>", 'conf_three': "<type 'bool'>"}, 'version': "<type 'int'>"}

Answer 3

@tobias_k beat me to it (both in time and quality probably) but here is another recursive function for the task that might be a bit easier for you (and me) to follow:

def check_dict(my_dict, check_against):
    for k, v in check_against.items():
        if isinstance(v, dict):
            return check_dict(my_dict[k], v)
        else:
            if not isinstance(my_dict[k], v):
                return False
    return True

Answer 4

The nature of dictionaries, if they are being used in python and not exported as some JSON, is that the order of the dictionary need not be set. Instead, looking up keys returns values (hence a dictionary).

In either case, these functions should provide you with what your looking for for the level of nesting present in the samples you provided.

#assuming identical order of keys is required

def check_structure(conf_structure,my_conf):
    if my_conf.keys() != conf_structure.keys():
        return False

    for key in my_conf.keys():
        if type(my_conf[key]) == dict:
            if my_conf[key].keys() != conf_structure[key].keys():
                return False

    return True

#assuming identical order of keys is not required

def check_structure(conf_structure,my_conf):
    if sorted(my_conf.keys()) != sorted(conf_structure.keys()):
        return False

    for key in my_conf.keys():
        if type(my_conf[key]) != dict:
            return False
        else:
            if sorted(my_conf[key].keys()) != sorted(conf_structure[key].keys()):
                return False

    return True

This solution would obviously need to be changed if the level of nesting was greater (i.e. it is configured to assess the similarity in structure of dictionaries that have some values as dictionaries, but not dictionaries where some values these latter dictionaries are also dictionaries).

Answer 5

You may use schema (PyPi Link)

schema is a library for validating Python data structures, such as those obtained from config-files, forms, external services or command-line parsing, converted from JSON/YAML (or something else) to Python data-types.

from schema import Schema, And, Use, Optional, SchemaError

def check(conf_schema, conf):
    try:
        conf_schema.validate(conf)
        return True
    except SchemaError:
        return False

conf_schema = Schema({
    'version': And(Use(int)),
    'info': {
        'conf_one': And(Use(float)),
        'conf_two': And(Use(str)),
        'conf_three': And(Use(bool)),
        Optional('optional_conf'): And(Use(str))
    }
})

conf = {
    'version': 1,
    'info': {
        'conf_one': 2.5,
        'conf_two': 'foo',
        'conf_three': False,
        'optional_conf': 'bar'
    }
}

print(check(conf_schema, conf))