Have sed operate on variable instead of standard i

Question

There are many questions asking about how to use a variable within a sed pattern, for example: sed "s/re/$VAR/" file, but I want to perform a substitution on a variable. So far, I have been using echo to do this:

echo "$VAR" | sed 's/re/new/'

But, though this works, it seems messy. Is there a simpler/more elegant way of doing this in bash or zsh?

Answer 1

If you don't want to use sed, you can use "parameter expansion" for search and replace within a parameter; quoting the Bash manual:

 ${parameter/pattern/string} 

The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string.

If pattern begins with /, all matches of pattern are replaced with string. Normally only the first match is replaced. If pattern begins with #, it must match at the beginning of the expanded value of parameter. If pattern begins with %, it must match at the end of the expanded value of parameter.

If string is null, matches of pattern are deleted and the / following pattern may be omitted. If parameter is @ or *, the substitution operation is applied to each positional parameter in turn, and the expansion is the resultant list. If parameter is an array variable subscripted with @ or *, the substitution operation is applied to each member of the array in turn, and the expansion is the resultant list.

Parameter expansion is not limited to this kind of substitution, there are many more, see the manual.

A few examples

Leaving away quoting for readability.

1. Simple variables

   $ var='abcabc'
   $ echo ${var/a/X}   # Replace first 'a'
   Xbcabc
   $ echo ${var//a/X}  # Replace all 'a'
   XbcXbc
   $ echo ${var/a}     # Remove 'a'
   bcabc
   $ echo ${var//a}    # Remove all 'a'
   bcbc
   $ echo ${var/#b}    # Try to remove 'b' from start of string - no-op
   abcabc
   $ echo ${var/#a}    # Remove 'a' from start of string
   bcabc
   $ echo ${var/%b}    # Try to remove 'b' from end of string - no-op
   abcabc
   $ echo ${var/%c}    # Remove 'c' from end of string
   abcab
   

2. Positional parameters

   $ set -- abcabc defdef abcabc # Set $1, $2 and $3
   $ echo ${@/a/X}               # Replace first 'a' for each parameter
   Xbcabc defdef Xbcabc
   $ echo ${@//a/X}              # Replace all 'a' for each parameter
   XbcXbc defdef XbcXbc
   $ echo ${@/?/X}               # Replace first occurrence of any character
   Xbcabc Xefdef Xbcabc
   $ echo ${@//?/X}              # Replace all characters
   XXXXXX XXXXXX XXXXXX
   

3. Arrays

   $ arr=(abcabc defdef abcabc) # Create array
   $ echo ${arr[@]/a/X}         # Replace first 'a' in each element
   Xbcabc defdef Xbcabc
   $ echo ${arr[@]//a/X}        # Replace all 'a' in each element
   XbcXbc defdef XbcXbc
   $ echo ${arr[@]/?/X}         # Replace first character in each element
   Xbcabc Xefdef Xbcabc
   $ echo ${arr[@]//?/X}        # Replace all characters in all elements
   XXXXXX XXXXXX XXXXXX
   

4. Combinations

   These can also be combined:

   $ arr=(patxxx xxpatxx xxxpat)  # Create array
   $ echo ${arr[@]/pat/X}         # Replace 'pat' in each element
   Xxxx xxXxx xxxX
   $ echo ${arr[@]/%pat/X}        # Replace 'pat' if it matches at the end
   patxxx xxpatxx xxxX
   $ echo ${arr[@]/#pat/X}        # Replace 'pat' if it matches at the beginning
   Xxxx xxpatxx xxxpat
   

5. Extended pattern matching

   Together with extended patterns, parameter expansion for search and replace becomes quite powerful:

   $ printf -v var "%b" ' have_spaces\t'     # Has space and tab
   $ echo "$var" | cat -A                    # Show whitespace
    have_spaces^I$
   $ shopt -s extglob                        # Turn on extended pattern matching
   $ echo "${var//+([[:space:]])}" | cat -A  # Remove all whitespace
   have_spaces$
   

   The example is somewhat contrived as the same problem could be solved simpler – the point is that it's pretty flexible.

Answer 2

You can do it like this:

sed 's/re/new' <<< "$var"

If you want to do that and update the variable you can do this:

var=$(sed 's/re/new' <<< "$var")