Sed to loop through file and replace placeholder v

Question

I was hoping to use bash to loop through a file and turn all placeholder varialbes into real variables:

• $PLACEHOLDER_USER -> $USER
• $PLACEHOLDER_STATE -> $STATE

It needs to work with any variable starting with placeholder and turn it into its real variable. This is the code I have so far:

$FILE="/mytest.conf"
sed -i "s/$var1/$var2/g" "$FILE"

However I'm not sure how I make it loop through the entire file, and I'm not sure how I can make it with any variable which starts with $PLACEHOLDER_.

Answer 1

The default action of sed is to read and print every line of the input file. You can modify this behavior in various ways by writing a sed script. A typical script would be something like

sed -i "s/\\\$PLACEHOLDER_USER/$USER/g;s/\\\$PLACEHOLDER_STATE/$STATE/g" file

Pay attention to the quoting here; the double quotes allow the shell to replace $USER and $STATE with their values from the environment, while the backslashed dollar signs will not be substituted. So the shell performs some substitutions, and by the time sed actually runs, the script has become

sed -i 's/\$PLACEHOLDER_USER/fred/g;s/\$PLACEHOLDER_STATE/Arkansas/g' file

(I supplied single quotes here to emphasize that no further substitution will take place.)

In the more general case, sed has no access to your environment variables, but you can write a shell script which generates a sed script from your variables.

env | sed 's%^\([^=]*\)=\(.*\)%s/\\\$PLACEHOLDER_\1/\2/g%' | sed -f - file

This is somewhat tricky. The output from the first sed script is another sed script which is read by another sed instance with sed -f -. (This is not supported on all platforms, but should at least work on Linux. If yours does not support this, you can work around it by writing the script to a temporary file, or using a command substitution.)

env lists your environment variables. The first sed script will prefix each variable name with \$PLACEHOLDER_ and generate a snippet of sed to replace any occurrence of that with its value from the environment. (Caveat: If the value contains regex metacharacters, you will need a significantly more complex script. If the values can contain slashes, you need to replace the slash delimiter with another delimiter; you can use any ASCII character you like.)