How do I manipulate $PATH elements in shell script

Question

Is there a idiomatic way of removing elements from PATH-like shell variables?

That is I want to take

PATH=/home/joe/bin:/usr/local/bin:/usr/bin:/bin:/path/to/app/bin:.

and remove or replace the /path/to/app/bin without clobbering the rest of the variable. Extra points for allowing me put new elements in arbitrary positions. The target will be recognizable by a well defined string, and may occur at any point in the list.

I know I've seen this done, and can probably cobble something together on my own, but I'm looking for a nice approach. Portability and standardization a plus.

I use bash, but example are welcome in your favorite shell as well.

---

The context here is one of needing to switch conveniently between multiple versions (one for doing analysis, another for working on the framework) of a large scientific analysis package which produces a couple dozen executables, has data stashed around the filesystem, and uses environment variable to help find all this stuff. I would like to write a script that selects a version, and need to be able to remove the $PATH elements relating to the currently active version and replace them with the same elements relating to the new version.

---

This is related to the problem of preventing repeated $PATH elements when re-running login scripts and the like.

---

• Previous similar question: How to keep from duplicating path variable in csh
• Subsequent similar question: What is the most elegant way to remove a path from the $PATH variable in Bash?

Answer 1

This is easy using awk.

Replace

{
  for(i=1;i<=NF;i++) 
      if($i == REM) 
          if(REP)
              print REP; 
          else
              continue;
      else 
          print $i; 
}

Start it using

function path_repl {
    echo $PATH | awk -F: -f rem.awk REM="$1" REP="$2" | paste -sd:
}

$ echo $PATH
/bin:/usr/bin:/home/js/usr/bin
$ path_repl /bin /baz
/baz:/usr/bin:/home/js/usr/bin
$ path_repl /bin
/usr/bin:/home/js/usr/bin

Append

Inserts at the given position. By default, it appends at the end.

{ 
    if(IDX < 1) IDX = NF + IDX + 1
    for(i = 1; i <= NF; i++) {
        if(IDX == i) 
            print REP 
        print $i
    }
    if(IDX == NF + 1)
        print REP
}

Start it using

function path_app {
    echo $PATH | awk -F: -f app.awk REP="$1" IDX="$2" | paste -sd:
}

$ echo $PATH
/bin:/usr/bin:/home/js/usr/bin
$ path_app /baz 0
/bin:/usr/bin:/home/js/usr/bin:/baz
$ path_app /baz -1
/bin:/usr/bin:/baz:/home/js/usr/bin
$ path_app /baz 1
/baz:/bin:/usr/bin:/home/js/usr/bin

Remove duplicates

This one keeps the first occurences.

{ 
    for(i = 1; i <= NF; i++) {
        if(!used[$i]) {
            print $i
            used[$i] = 1
        }
    }
}

Start it like this:

echo $PATH | awk -F: -f rem_dup.awk | paste -sd:

Validate whether all elements exist

The following will print an error message for all entries that are not existing in the filesystem, and return a nonzero value.

echo -n $PATH | xargs -d: stat -c %n

To simply check whether all elements are paths and get a return code, you can also use test:

echo -n $PATH | xargs -d: -n1 test -d

Answer 2

Reposting my answer to What is the most elegant way to remove a path from the $PATH variable in Bash? :

#!/bin/bash
IFS=:
# convert it to an array
t=($PATH)
unset IFS
# perform any array operations to remove elements from the array
t=(${t[@]%%*usr*})
IFS=:
# output the new array
echo "${t[*]}"

or the one-liner:

PATH=$(IFS=':';t=($PATH);unset IFS;t=(${t[@]%%*usr*});IFS=':';echo "${t[*]}");

Answer 3

The first thing to pop into my head to change just part of a string is a sed substitution.

example: if echo $PATH => "/usr/pkg/bin:/usr/bin:/bin:/usr/pkg/games:/usr/pkg/X11R6/bin" then to change "/usr/bin" to "/usr/local/bin" could be done like this:

## produces standard output file

## the "=" character is used instead of slash ("/") since that would be messy, # alternative quoting character should be unlikely in PATH

## the path separater character ":" is both removed and re-added here, # might want an extra colon after the last path

echo $PATH | sed '=/usr/bin:=/usr/local/bin:='

This solution replaces an entire path-element so might be redundant if new-element is similar.

If the new PATH'-s aren't dynamic but always within some constant set you could save those in a variable and assign as needed:

PATH=$TEMP_PATH_1; # commands ... ; \n PATH=$TEMP_PATH_2; # commands etc... ;

Might not be what you were thinking. some of the relevant commands on bash/unix would be:

pushd popd cd ls # maybe l -1A for single column; find grep which # could confirm that file is where you think it came from; env type

..and all that and more have some bearing on PATH or directories in general. The text altering part could be done any number of ways!

Whatever solution chosen would have 4 parts:

1) fetch the path as it is 2) decode the path to find the part needing changes 3) determing what changes are needed/integrating those changes 4) validation/final integration/setting the variable

Answer 4

Just a note that bash itself can do search and replace. It can do all the normal "once or all", cases [in]sensitive options you would expect.

From the man page:

${parameter/pattern/string}

The pattern is expanded to produce a pattern just as in pathname expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. If Ipattern begins with /, all matches of pattern are replaced with string. Normally only the first match is replaced. If pattern begins with #, it must match at the beginning of the expanded value of parameter. If pattern begins with %, it must match at the end of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following pattern may be omitted. If parameter is @ or *, the substitution operation is applied to each positional parameter in turn, and the expansion is the resultant list. If parameter is an array variable subscripted with @ or *, the substitution operation is applied to each member of the array in turn, and the expansion is the resultant list.

You can also do field splitting by setting $IFS (input field separator) to the desired delimiter.

Answer 5

There are a couple of relevant programs in the answers to "How to keep from duplicating path variable in csh". They concentrate more on ensuring that there are no repeated elements, but the script I provide can be used as:

export PATH=$(clnpath $head_dirs:$PATH:$tail_dirs $remove_dirs)

Assuming you have one or more directories in $head_dirs and one or more directories in $tail_dirs and one or more directories in $remove_dirs, then it uses the shell to concatenate the head, current and tail parts into a massive value, and then removes each of the directories listed in $remove_dirs from the result (not an error if they don't exist), as well as eliminating second and subsequent occurrences of any directory in the path.

This does not address putting path components into a specific position (other than at the beginning or end, and those only indirectly). Notationally, specifying where you want to add the new element, or which element you want to replace, is messy.