Export with alphabetical sort in Python ConfigPars

Question

Is there any solution to force the RawConfigParser.write() method to export the config file with an alphabetical sort?

Even if the original/loaded config file is sorted, the module mixes the section and the options into the sections arbitrarily, and is really annoying to edit manually a huge unsorted config file.

PD: I'm using python 2.6

Answer 1

I was looking into this for merging a .gitmodules doing a subtree merge with a supermodule -- was super confused to start with, and having different orders for submodules was confusing enough haha.

Using GitPython helped alot:

from collections import OrderedDict
import git

filePath = '/tmp/git.config'
# Could use SubmoduleConfigParser to get fancier
c = git.GitConfigParser(filePath, False)
c.sections()
# http://stackoverflow.com/questions/8031418/how-to-sort-ordereddict-in-ordereddict-python
c._sections = OrderedDict(sorted(c._sections.iteritems(), key=lambda x: x[0]))
c.write()
del c

Answer 2

I was able to solve this issue by sorting the sections in the ConfigParser from the outside like so:

config = ConfigParser.ConfigParser({}, collections.OrderedDict)
config.read('testfile.ini')
# Order the content of each section alphabetically
for section in config._sections:
    config._sections[section] = collections.OrderedDict(sorted(config._sections[section].items(), key=lambda t: t[0]))

# Order all sections alphabetically
config._sections = collections.OrderedDict(sorted(config._sections.items(), key=lambda t: t[0] ))

# Write ini file to standard output
config.write(sys.stdout)

Answer 3

This is my solution for writing config file in alphabetical sort:

class OrderedRawConfigParser( ConfigParser.RawConfigParser ):
"""
Overload standart Class ConfigParser.RawConfigParser
"""
def __init__( self, defaults = None, dict_type = dict ):
    ConfigParser.RawConfigParser.__init__( self, defaults = None, dict_type = dict )

def write(self, fp):
    """Write an .ini-format representation of the configuration state."""
    if self._defaults:
        fp.write("[%s]\n" % DEFAULTSECT)
        for key in sorted( self._defaults ):                
            fp.write( "%s = %s\n" % (key, str( self._defaults[ key ] ).replace('\n', '\n\t')) )                 
        fp.write("\n")
    for section in self._sections:
        fp.write("[%s]\n" % section)
        for key in sorted( self._sections[section] ): 
            if key != "__name__":
                fp.write("%s = %s\n" %
                         (key, str( self._sections[section][ key ] ).replace('\n', '\n\t')))    
        fp.write("\n")    

Answer 4

Three solutions:

1. Pass in a dict type (second argument to the constructor) which returns the keys in your preferred sort order.
2. Extend the class and overload write() (just copy this method from the original source and modify it).
3. Copy the file ConfigParser.py and add the sorting to the method write().

See this article for a ordered dict or maybe use this implementation which preserves the original adding order.

Answer 5

The first method looked as the most easier, and safer way.

But, after looking at the source code of the ConfigParser, it creates an empty built-in dict, and then copies all the values from the "second parameter" one-by-one. That means it won't use the OrderedDict type. An easy work around can be to overload the CreateParser class.

class OrderedRawConfigParser(ConfigParser.RawConfigParser):
    def __init__(self, defaults=None):
        self._defaults = type(defaults)() ## will be correct with all type of dict.
        self._sections = type(defaults)()
        if defaults:
            for key, value in defaults.items():
                self._defaults[self.optionxform(key)] = value

It leaves only one flaw open... namely in ConfigParser.items(). odict doesn't support update and comparison with normal dicts.

Workaround (overload this function too):

    def items(self, section):
        try:
            d2 = self._sections[section]
        except KeyError:
            if section != DEFAULTSECT:
                raise NoSectionError(section)
            d2 = type(self._section)()  ## Originally: d2 = {}
        d = self._defaults.copy()
        d.update(d2)  ## No more unsupported dict-odict incompatibility here.
        if "__name__" in d:
            del d["__name__"]
        return d.items()

Other solution to the items issue is to modify the odict.OrderedDict.update function - maybe it is more easy than this one, but I leave it to you.

PS: I implemented this solution, but it doesn't work. If i figure out, ConfigParser is still mixing the order of the entries, I will report it.

PS2: Solved. The reader function of the ConfigParser is quite idiot. Anyway, only one line had to be changed - and some others for overloading in an external file:

def _read(self, fp, fpname):

    cursect = None
    optname = None
    lineno = 0
    e = None
    while True:
        line = fp.readline()
        if not line:
            break
        lineno = lineno + 1
        if line.strip() == '' or line[0] in '#;':
            continue
        if line.split(None, 1)[0].lower() == 'rem' and line[0] in "rR":
            continue
        if line[0].isspace() and cursect is not None and optname:
            value = line.strip()
            if value:
                cursect[optname] = "%s\n%s" % (cursect[optname], value)
        else:
            mo = self.SECTCRE.match(line)
            if mo:
                sectname = mo.group('header')
                if sectname in self._sections:
                    cursect = self._sections[sectname]
                ## Add ConfigParser for external overloading
                elif sectname == ConfigParser.DEFAULTSECT:
                    cursect = self._defaults
                else:
                    ## The tiny single modification needed
                    cursect = type(self._sections)() ## cursect = {'__name__':sectname}
                    cursect['__name__'] = sectname
                    self._sections[sectname] = cursect
                optname = None
            elif cursect is None:
                raise ConfigParser.MissingSectionHeaderError(fpname, lineno, line) 
                ## Add ConfigParser for external overloading.
            else:
                mo = self.OPTCRE.match(line)
                if mo:
                    optname, vi, optval = mo.group('option', 'vi', 'value')
                    if vi in ('=', ':') and ';' in optval:
                        pos = optval.find(';')
                        if pos != -1 and optval[pos-1].isspace():
                            optval = optval[:pos]
                    optval = optval.strip()
                    if optval == '""':
                        optval = ''
                    optname = self.optionxform(optname.rstrip())
                    cursect[optname] = optval
                else:
                    if not e:
                        e = ConfigParser.ParsingError(fpname)
                        ## Add ConfigParser for external overloading
                    e.append(lineno, repr(line))
    if e:
        raise e

Trust me, I didn't wrote this thing. I copy-pasted it entirely from ConfigParser.py

So overall what to do?

1. Download odict.py from one of the links previously suggested
2. Import it.
3. Copy-paste these codes in your favorite utils.py (which will create the OrderedRawConfigParser class for you)
4. cfg = utils.OrderedRawConfigParser(odict.OrderedDict())
5. use cfg as always. it will stay ordered.
6. Sit back, smoke a havanna, relax.

PS3: The problem I solved here is only in Python 2.5. In 2.6 there is already a solution for that. They created a second custom parameter in the __init__ function, which is a custom dict_type.

So this workaround is needed only for 2.5