What is the standard way to handle web.config file

I am trying to determine a good way to handle web.config files that differ between environments. My workstation, development server, staging server, and production server have slightly different settings in the web.config file. I would like to keep the different web.config files under version control. In addition to the the history, it makes moving files between servers easy. What is the best way to handle this situation?

Here is my current strategy/procedure:

My project contains 4 configuration files: web.config, web_local.config, web_dev.config, web_stg.config, and web_prod.config.
Using SVN settings, the web.config is ignored.
The files are moved between servers using SVN to check-in and check-out the files.
After using SVN to update to the latest copy, the web.config file on each system is replaced with the appropriate web_####.config. (I do this by copying the file to a new dir, renaming it web.config, and dragging it back to the original directory to replace the working copy.)

This works but it does not feel like it's the best way to approach things.
Here are the problems that come to mind:

The last step where the appropriate .config file is copied to a new directory, renamed, and used to replace the current seems clunky and prone to error. Two of these sites are on the same machine and it is easier than one might think to use the incorrect config file for the replacement.
The web_####.config files should be hidden from public access by server settings, but it still feels inappropriate to be pulling down all of the web_####.config files from SVN for each project when only one is needed per site. It would be better if I could get only the needed file from SVN. I can manually delete the unused files after I update the SVN directory, but this causes SVN to red check the directory because of the missing files. I want a green checked directory with only the needed files if possible. Can each directory be customized to download a different set of files?

What strategy/technique/procedure are other people using to handle this situation?