I have found that in the Machine.config you can add your connection strings so that you don't need your connection strings in each running apps config file. The down side to this is that you have number of framework versions * 2 machine.config file. So for version 4.* you have 2 alone. This took my research on to find that you can externalize the connection string using the configSource attribute in the connectionString element of the machine.config
<connectionStrings configSource="Config\connectionStrings.config" />
However, one realizes that the config source must be in a subfolder of where the the machine.config file is located. So how do you have one, machine wide, location to store all your connection strings?
I couldn't find anywhere on the internet that had this answer. So I am going to answer this myself after some experimentation.
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config>mklink /D Config C:\Test
<connectionStrings configSource="Config\connectionStrings.config" />
Now you have one location for all your connection strings.