At what point does a config file become a programm

Question

I have been mulling over config files and their relationship to code for a while now and depending on the day and direction of the wind my opinions seem to change. More and more though I keep coming back to the realization I first had while learning Lisp: there is little difference between data and code. This seems doubly true for config files. When looked at in the right light a Perl script is little more than a config file for perl. This tends to have fairly heavy consequences for tasks such as QA and divisions of labor like who should be responsible for changing config files.

The creep from config file to full fledged language is generally slow and seems to be driven by the desire to have a generic system. Most projects seem to start out small with a few config items like where to write logs, where to look for data, user names and passwords, etc. But then they start to grow: features start to be able to be turned on or off, the timings and order of operations start to be controlled, and, inevitably, someone wants to start adding logic to it (e.g. use 10 if the machine is X and 15 if the machine is Y). At a certain point the config file becomes a domain specific language, and a poorly written one at that.

Now that I have rambled on to set the stage, here are my questions:

1. What is the true purpose of a config file?
2. Should an attempt be made to keep config files simple?
3. Who should be responsible for making changes to them (developers, users, admins, etc.)?
4. Should they be source controlled (see question 3)?

As I said earlier my answers to these questions shift constantly, but right now I am thinking:

1. to allow a non-programmers to change large chunks of behaviour quickly
2. yes, anything that is not coarsely grained should be in code
3. users should be responsible for config files and programmers should be responsible for a configuration layer between config files and code that gives more fine grained control of the application
4. no, but the finer grained middle layer should be

Answer 1

I have a different philosophy about config files. Data about how an application should be run is still data, and therefore belongs in a data store, not in code (a config file IMO is code). If end users need to be able to change the data, then the application should provide an interface to do so.

I only use config files to point at data stores.

Answer 2

Recently I was working upon a project and I realised that I wanted to have conditionals inside my configuration file - which had previously just been a pretty simple one of the form:

key = val
key2 = val
name = `hostname`

I didn't want to write a mini-language, because unless I did it very carefully I couldn't allow the flexibility that would be useful.

Instead I decided that I'd have two forms:

1. If the file started with "#!" and was executable I'd parse the result of running it.

2. Otherwise I'd read it as-is

This means that I can now allow people to write "configuration files" that look like this:

 #!/usr/bin/perl
if ( -x /bin/foo ) 
{
   print <<EOF;
foo=me
bar=you
EOF
}
else
{
   print <<EOF;
foo=bar
bar=foo
EOF
}

This way I get the power of a dynamic configuration file if the user wants to use it, and the simplicity of not having to write my own mini-language.

Answer 3

Config file: "What is my purpose?"
You: "Configure the butter."
Config file: "Ok..."
Config file: "What is my purpose?"
You: "You configure butter."
Config file: "Oh my god."

1. There is no "true purpose" of a configuration file. Its whatever makes sense for your application. In general, things that differ (or might differ) between machines and don't change in the middle of your application run should probably be in a configuration file. Defaults, ports, and addresses for other services are all great candidates. Keys and secrets are also great candidates but should be handled separately from your normal config for security reasons. I disagree that the purpose of a config file is to allow quick changes to be made. The purpose should be to allow flexibility in the setup of your application. If a config file is a quick easy to way to allow that flexibility, so much the better - but you should not be intending your config files to be frequently changing.

2. Yes and no. Should you atempt to make your application's code simple? Yes. You should attempt to make everything you write simple and to the point. No more complicated than it needs to be. Same is true of your config. However, this is very application specific. Hardcoding what should be in config because it would make your config "too complicated" is bad design. In fact, trying to "keep things simple" is why config files end up being a giant mess. Sometimes the simplest move is to modularize. This is why your configuration files should be written in a well known general purpose programming langauge - not some terrible configuration language (read: all "configuration languages" suck).

3. Again, who should be modifying config files is completely application dependent. But I agree with miniquark, whoever is deploying the application should be in charge of the configuration.

4. Source control everything you can. Source control is great. You can roll stuff back super easily and you have a full history of the changes you've made and a record of who made those changes. So why not?

Answer 4

You could turn to theory of computation to define what counts as a programming language. If your configuration file format is Turing Complete then it reasonably counts as a programming language. By this definition, a file format to describe levels of Sokoban counts as a programming language (see here). There are other levels of complexity below Turing Complete that may also count, such as Regular Grammars and Pushdown Automata.

Another way to look at it is that many config files are only capable of data markup, whereas a proper programming language must be able to implement algorithms. For example, JSON is a config file format, whereas ECMA Script is a programming language.

Answer 5

Here's my thoughts:

1. To allow the runtime behavior of an application to be modified easily. This can be by programmers or non programmers, depending on the needs. This can be during development, but I often view configuration files as a way to help make a program more flexible at any point.

2. Yes. I think config files should be as simple as possible, given the constraint that you may need various options to control different behaviors of your runtime. I prefer grouping configuration settings, and simplifying them as much as possible.

3. Depends on what and why the change is being made. If users are going to be changing it, a front-end should be made to hide them from the details. The same is often true of non-developers in general.

4. I often source control the "default" configuration, but have a way to override this per system for the actual runtime.

As for adding logic to the config file - I'd avoid this. I think it's better to just have the configuration file switch on the logic in your application. Behavior in config files leads to a lack of maintainability and understanding, in my experience. I strongly prefer keeping configuration files as simple as possible.

Answer 6

Config files invariably inch their way to becoming ugly, illogical "full fledged programming languages". It takes art and skill to design good programming languages, and config languages turned programming language tend to be horrendous.

A good approach is to use a nicely designed language, say python or ruby, and use it to create a DSL for your configuration. That way your configuration language can remain simple on the surface but in actually be the full fledged programming language.