At what point does a config file become a programm

Question

I have been mulling over config files and their relationship to code for a while now and depending on the day and direction of the wind my opinions seem to change. More and more though I keep coming back to the realization I first had while learning Lisp: there is little difference between data and code. This seems doubly true for config files. When looked at in the right light a Perl script is little more than a config file for perl. This tends to have fairly heavy consequences for tasks such as QA and divisions of labor like who should be responsible for changing config files.

The creep from config file to full fledged language is generally slow and seems to be driven by the desire to have a generic system. Most projects seem to start out small with a few config items like where to write logs, where to look for data, user names and passwords, etc. But then they start to grow: features start to be able to be turned on or off, the timings and order of operations start to be controlled, and, inevitably, someone wants to start adding logic to it (e.g. use 10 if the machine is X and 15 if the machine is Y). At a certain point the config file becomes a domain specific language, and a poorly written one at that.

Now that I have rambled on to set the stage, here are my questions:

1. What is the true purpose of a config file?
2. Should an attempt be made to keep config files simple?
3. Who should be responsible for making changes to them (developers, users, admins, etc.)?
4. Should they be source controlled (see question 3)?

As I said earlier my answers to these questions shift constantly, but right now I am thinking:

1. to allow a non-programmers to change large chunks of behaviour quickly
2. yes, anything that is not coarsely grained should be in code
3. users should be responsible for config files and programmers should be responsible for a configuration layer between config files and code that gives more fine grained control of the application
4. no, but the finer grained middle layer should be

Answer 1

The code of our applications becomes less important... There is scripting, there are all kind of attributes that define the behaviour of classes, methods, method arguments and properties. Users can define database triggers and database constraints. There can be very complicated config files. Sometimes the user can define XSLT stylsheets to manipulate input and output because our systems need to be open (SOA). And there is stuff like BizzTalk that needs complex configuration too. Users can define complex workflows.

We have to write better code to deal with this complex environment, so the code of our applications becomes more important...

Answer 2

Yes, config files should be simple. They should contain no 'logic' themselves - think of them as a list of expressions in if statements, not the conditional statements in their entirety.

They're there to allow the user to decide which of the options coded within the application should be used, so don't try to make them complicated, it'll end up being self-defeating - you may end up writing simple config files to control how the original config file should be configured otherwise!

Answer 3

Very interesting questions!

I tend to limit my config files to a very simple "key=value" format, because I fully agree with you that config files can very quickly become full-blown programs. For example, anyone who has ever tried to "configure" OpenSER knows the feeling you are talking about: it's not configuration, it's (painful) programming.

When you need your application to be very "configurable" in ways that you cannot imagine today, then what you really need is a plugins system. You need to develop your application in a way that someone else can code a new plugin and hook it into your application in the future.

So, to answer your questions:

1. What is the true purpose of a config file?

   I would say, to allow the people who will install your application to be able to tweek some deployment-related parameters, such as host name, number of threads, names of the plugins you need, and the deployment-parameters for those plugins (check out FreeRadius's configuration for an example of this principle), etc.. Definitely not the place to express business logic.

2. Should an attempt be made to keep config files simple?

   Definitely. As you suggested, "programming" in a config file is horrible. I believe it should be avoided.

3. Who should be responsible for making changes to them (developers, users, admins, etc.)?

   In general, I would say admins, who deploy the application.

4. Should they be source controlled (see question 3)?

   I usually don't source-control the configuration files themselves, but I do source-control a template configuration file, with all the parameters and their default values, and comments describing what they do. For example, if a configuration file is named database.conf, I usually source-control a file named database.conf.template. Now of course I am talking about what I do as a developer. As an admin, I may want to source-control the actual settings that I chose for each installation. For example, we manage a few hundred servers remotely, and we need to keep track of their configurations: we chose to do this with source-control.

---

Edit: Although I believe the above to be true for most applications, there are always exceptions, of course. Your application may allow its users to dynamically configure complex rules, for example. Most email clients allow the users to define rules for the management of their emails (for example, "all emails coming from 'john doe' and not having me in the To: field should be discarded"). Another example is an application that allows the user to define a new complex commercial offer. You may also think about applications like Cognos which allow their users to build complex database reports. The email client will probably offer the user a simple interface to define the rules, and this will generate a complex configuration file (or even perhaps a bit of code). On the other hand, the user-defined configuration for the commercial offers might be saved in a database, in a structured way (neither a simple key=value structure nor a portion of code). And some other applications might even allow the user to code in python or VB, or some other automation-capable language. In other words... your mileage may vary.

Answer 4

I have seen python programs where the config file is code. If you don't need to do anything special (conditionals, etc.) it doesn't look much different from other config styles. e.g. I could make a file config.py with stuff like:

num_threads = 13
hostname = 'myhost'

and the only burden on the user, compared with (say) INI files, is that they need to put '' around strings. No doubt you could do the same thing in other interpreted languages. It gives you unlimited ability to complicate your config file if necessary, at the risk of possibly scaring your users.

Answer 5

Ok. You will have some users which want a really simple config, you should give it to them. At the same time, you will have constant requests of "Can you add this? How do I do in the config file?", I don't see why you can't support both groups.

The project I am currently working on uses Lua for its configuration file. Lua is a scripting language, and it works quite well in this scenario. There is available an example of our default configuration.

You'll note that it is mainly key=value statements, where value can be any of Lua's built-in types. The most complicated thing there are lists, and they aren't really complicated (it's just a matter of syntax).

Now I'm just waiting for someone to ask how to set their server's port to a random value every time they start it up...

Answer 6

I believe your question is very relevant given the move to "fluent interfaces". Many developers have "seen the light" with respect to XML configured applications. Using XML can be very verbose and difficult to edit correctly (especially if no schema is provided). Having a fluent interface allows the developer to configure the application in a domain-specific language with the assistance of some key-value pairs from a plain text configuration file (or perhaps command-line parameters). It also makes it very easy to setup and configure new instances of the application for testing or whatever.

Here are my answers to your question:

• What is the true purpose of a config file?

A config file is a way to allow the user to customize the behavior of their program at run-time.

• Should an attempt be made to keep config files simple?

Ideally, I would think that config files should at least be supplemented by a fluent interface to configure the program (this is useful in many respects). If you do require a config file then it should be kept very simple, nothing other than key-value pairs.

• Who should be responsible for making changes to them (developers, users, admins, etc.)?

I think the answer to this depends on your organization. It should be the responsibility of the person deploying the software to ensure that it is properly configured.

• Should they be source controlled (see question 3)?

I will steal this answer from someone else :) I like the idea of storing a template configuration in source control and modifying it for each local user's needs. Chances are one developer's config file is another developer's nightmare so it is best to leave things that vary by user out of source control. Having a template is also a nice way to let the person deploying the application (or other developers) see exactly what values are valid for the config file.