Which is the best way to allow configuration optio

Question

I have a Python application which needs quite a few (~30) configuration parameters. Up to now, I used the OptionParser class to define default values in the app itself, with the possibility to change individual parameters at the command line when invoking the application.

Now I would like to use 'proper' configuration files, for example from the ConfigParser class. At the same time, users should still be able to change individual parameters at the command line.

I was wondering if there is any way to combine the two steps, e.g. use optparse (or the newer argparse) to handle command line options, but reading the default values from a config file in ConfigParse syntax.

Any ideas how to do this in an easy way? I don't really fancy manually invoking ConfigParse, and then manually setting all defaults of all optinos to the appropriate values...

Answer 1

Update: This answer still has issues; for example, it cannot handle required arguments, and requires an awkward config syntax. Instead, ConfigArgParse seems to be exactly what this question asks for, and is a transparent, drop-in replacement.

One issue with the current is that it will not error if the arguments in the config file are invalid. Here's a version with a different downside: you'll need to include the -- or - prefix in the keys.

Here's the python code (Gist link with MIT license):

# Filename: main.py
import argparse

import configparser

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('--config_file', help='config file')
    args, left_argv = parser.parse_known_args()
    if args.config_file:
        with open(args.config_file, 'r') as f:
            config = configparser.SafeConfigParser()
            config.read([args.config_file])

    parser.add_argument('--arg1', help='argument 1')
    parser.add_argument('--arg2', type=int, help='argument 2')

    for k, v in config.items("Defaults"):
        parser.parse_args([str(k), str(v)], args)

    parser.parse_args(left_argv, args)
print(args)

Here's an example of a config file:

# Filename: config_correct.conf
[Defaults]
--arg1=Hello!
--arg2=3

Now, running

> python main.py --config_file config_correct.conf --arg1 override
Namespace(arg1='override', arg2=3, config_file='test_argparse.conf')

However, if our config file has an error:

# config_invalid.conf
--arg1=Hello!
--arg2='not an integer!'

Running the script will produce an error, as desired:

> python main.py --config_file config_invalid.conf --arg1 override
usage: test_argparse_conf.py [-h] [--config_file CONFIG_FILE] [--arg1 ARG1]
                             [--arg2 ARG2]
main.py: error: argument --arg2: invalid int value: 'not an integer!'

The main downside is that this uses parser.parse_args somewhat hackily in order to obtain the error checking from ArgumentParser, but I am not aware of any alternatives to this.