ConfigParser and String interpolation with env var

问题:

it's a little bit I'm out of python syntax and I have a problem in reading a .ini file with interpolated values.

this is my ini file:

[DEFAULT]
home=$HOME
test_home=$home

[test]
test_1=$test_home/foo.csv
test_2=$test_home/bar.csv

Those lines

from ConfigParser import SafeConfigParser

parser = SafeConfigParser()
parser.read('config.ini')

print parser.get('test', 'test_1')

does output

$test_home/foo.csv

while I'm expecting

/Users/nkint/foo.csv

EDIT:

I supposed that the $ syntax was implicitly included in the so called string interpolation (referring to the manual):

On top of the core functionality, SafeConfigParser supports interpolation. This means values can contain format strings which refer to other values in the same section, or values in a special DEFAULT section.

But I'm wrong. How to handle this case?

回答1:

First of all according to the documentation you should use %(test_home)s to interpolate test_home. Moreover the key are case insensitive and you can't use both HOME and home keys. Finally you can use SafeConfigParser(os.environ) to take in account of you environment.

from ConfigParser import SafeConfigParser
import os


parser = SafeConfigParser(os.environ)
parser.read('config.ini')

Where config.ini is

[DEFAULT]
test_home=%(HOME)s

[test]
test_1=%(test_home)s/foo.csv
test_2=%(test_home)s/bar.csv

回答2:

If you want to expand some environment variables, you can do so using os.path.expandvars before parsing a StringIO stream:

import ConfigParser
import os
import StringIO

with open('config.ini', 'r') as cfg_file:
    cfg_txt = os.path.expandvars(cfg_file.read())

config = ConfigParser.ConfigParser()
config.readfp(StringIO.StringIO(cfg_txt))

回答3:

You can write custom interpolation in case of Python 3:

import configparser
import os


class EnvInterpolation(configparser.BasicInterpolation):
    """Interpolation which expands environment variables in values."""

    def before_get(self, parser, section, option, value, defaults):
        return os.path.expandvars(value)


cfg = """
[section1]
key = value
my_path = $PATH
"""

config = configparser.ConfigParser(interpolation=EnvInterpolation())
config.read_string(cfg)
print(config['section1']['my_path'])

回答4:

ConfigParser.get values are strings, even if you set values as integer or True. But ConfigParser has getint, getfloat and getboolean.

settings.ini

[default]
home=/home/user/app
tmp=%(home)s/tmp
log=%(home)s/log
sleep=10
debug=True

config reader

>>> from ConfigParser import SafeConfigParser
>>> parser = SafeConfigParser()
>>> parser.read('/home/user/app/settings.ini')
>>> parser.get('defauts', 'home')
'/home/user/app'
>>> parser.get('defauts', 'tmp')
'/home/user/app/tmp'
>>> parser.getint('defauts', 'sleep')
10
>>> parser.getboolean('defauts', 'debug')
True

Edit

Indeed you could get name values as environ var if you initialize SafeConfigParser with os.environ. Thanks for the Michele's answer.

回答5:

the trick for proper variable substitution from environment is to use the ${} syntax for the environment variables:

[DEFAULT]
test_home=${HOME}

[test]
test_1=%(test_home)s/foo.csv
test_2=%(test_home)s/bar.csv

回答6:

It seems in the last version 3.5.0, ConfigParser was not reading the env variables, so I end up providing a custom Interpolation based on the BasicInterpolation one.

class EnvInterpolation(BasicInterpolation):
    """Interpolation as implemented in the classic ConfigParser,
    plus it checks if the variable is provided as an environment one in uppercase.
    """

    def _interpolate_some(self, parser, option, accum, rest, section, map,
                          depth):
        rawval = parser.get(section, option, raw=True, fallback=rest)
        if depth > MAX_INTERPOLATION_DEPTH:
            raise InterpolationDepthError(option, section, rawval)
        while rest:
            p = rest.find("%")
            if p < 0:
                accum.append(rest)
                return
            if p > 0:
                accum.append(rest[:p])
                rest = rest[p:]
            # p is no longer used
            c = rest[1:2]
            if c == "%":
                accum.append("%")
                rest = rest[2:]
            elif c == "(":
                m = self._KEYCRE.match(rest)
                if m is None:
                    raise InterpolationSyntaxError(option, section,
                                                   "bad interpolation variable reference %r" % rest)
                var = parser.optionxform(m.group(1))
                rest = rest[m.end():]
                try:
                    v = os.environ.get(var.upper())
                    if v is None:
                        v = map[var]
                except KeyError:
                    raise InterpolationMissingOptionError(option, section, rawval, var) from None
                if "%" in v:
                    self._interpolate_some(parser, option, accum, v,
                                           section, map, depth + 1)
                else:
                    accum.append(v)
            else:
                raise InterpolationSyntaxError(
                    option, section,
                    "'%%' must be followed by '%%' or '(', "
                    "found: %r" % (rest,))

The difference between the BasicInterpolation and the EnvInterpolation is in:

   v = os.environ.get(var.upper())
   if v is None:
       v = map[var]

where I'm trying to find the var in the enviornment before checking in the map.