Using getopts in bash shell script to get long and

Question

I wish to have long and short forms of command line options invoked using my shell script.

I know that getopts can be used, but like in Perl, I have not been able to do the same with shell.

Any ideas on how this can be done, so that I can use options like:

./shell.sh --copyfile abc.pl /tmp/
./shell.sh -c abc.pl /tmp/

In the above, both the commands mean the same thing to my shell, but using getopts, I have not been able to implement these?

Answer 1

An improved solution:

# translate long options to short
# Note: This enable long options but disable "--?*" in $OPTARG, or disable long options after  "--" in option fields.
for ((i=1;$#;i++)) ; do
    case "$1" in
        --)
            # [ ${args[$((i-1))]} == ... ] || EndOpt=1 ;;& # DIRTY: we still can handle some execptions...
            EndOpt=1 ;;&
        --version) ((EndOpt)) && args[$i]="$1"  || args[$i]="-V";;
        # default case : short option use the first char of the long option:
        --?*) ((EndOpt)) && args[$i]="$1"  || args[$i]="-${1:2:1}";;
        # pass through anything else:
        *) args[$i]="$1" ;;
    esac
    shift
done
# reset the translated args
set -- "${args[@]}"

function usage {
echo "Usage: $0 [options] files" >&2
    exit $1
}

# now we can process with getopt
while getopts ":hvVc:" opt; do
    case $opt in
        h)  usage ;;
        v)  VERBOSE=true ;;
        V)  echo $Version ; exit ;;
        c)  source $OPTARG ;;
        \?) echo "unrecognized option: -$opt" ; usage -1 ;;
        :)
        echo "option -$OPTARG requires an argument"
        usage -1
        ;;
    esac
done

shift $((OPTIND-1))
[[ "$1" == "--" ]] && shift

Answer 2

I only write shell scripts now and then and fall out of practice, so any feedback is appreciated.

Using the strategy proposed by @Arvid Requate, we noticed some user errors. A user who forgets to include a value will accidentally have the next option's name treated as a value:

./getopts_test.sh --loglevel= --toc=TRUE

will cause the value of "loglevel" to be seen as "--toc=TRUE". This can be avoided.

I adapted some ideas about checking user error for CLI from http://mwiki.wooledge.org/BashFAQ/035 discussion of manual parsing. I inserted error checking into handling both "-" and "--" arguments.

Then I started fiddling around with the syntax, so any errors in here are strictly my fault, not the original authors.

My approach helps users who prefer to enter long with or without the equal sign. That is, it should have same response to "--loglevel 9" as "--loglevel=9". In the --/space method, it is not possible to know for sure if the user forgets an argument, so some guessing is needed.

1. if the user has the long/equal sign format (--opt=), then a space after = triggers an error because an argument was not supplied.
2. if user has long/space arguments (--opt ), this script causes a fail if no argument follows (end of command) or if argument begins with dash)

In case you are starting out on this, there is an interesting difference between "--opt=value" and "--opt value" formats. With the equal sign, the command line argument is seen as "opt=value" and the work to handle that is string parsing, to separate at the "=". In contrast, with "--opt value", the name of the argument is "opt" and we have the challenge of getting the next value supplied in the command line. That's where @Arvid Requate used ${!OPTIND}, the indirect reference. I still don't understand that, well, at all, and comments in BashFAQ seem to warn against that style (http://mywiki.wooledge.org/BashFAQ/006). BTW, I don't think previous poster's comments about importance of OPTIND=$(( $OPTIND + 1 )) are correct. I mean to say, I see no harm from omitting it.

In newest version of this script, flag -v means VERBOSE printout.

Save it in a file called "cli-5.sh", make executable, and any of these will work, or fail in the desired way

./cli-5.sh  -v --loglevel=44 --toc  TRUE
./cli-5.sh  -v --loglevel=44 --toc=TRUE
./cli-5.sh --loglevel 7
./cli-5.sh --loglevel=8
./cli-5.sh -l9

./cli-5.sh  --toc FALSE --loglevel=77
./cli-5.sh  --toc=FALSE --loglevel=77

./cli-5.sh   -l99 -t yyy
./cli-5.sh   -l 99 -t yyy

Here is example output of the error-checking on user intpu

$ ./cli-5.sh  --toc --loglevel=77
ERROR: toc value must not have dash at beginning
$ ./cli-5.sh  --toc= --loglevel=77
ERROR: value for toc undefined

You should consider turning on -v, because it prints out internals of OPTIND and OPTARG

#/usr/bin/env bash

## Paul Johnson
## 20171016
##

## Combines ideas from
## https://stackoverflow.com/questions/402377/using-getopts-in-bash-shell-script-to-get-long-and-short-command-line-options
## by @Arvid Requate, and http://mwiki.wooledge.org/BashFAQ/035

# What I don't understand yet: 
# In @Arvid REquate's answer, we have 
# val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
# this works, but I don't understand it!


die() {
    printf '%s\n' "$1" >&2
    exit 1
}

printparse(){
    if [ ${VERBOSE} -gt 0 ]; then
        printf 'Parse: %s%s%s\n' "$1" "$2" "$3" >&2;
    fi
}

showme(){
    if [ ${VERBOSE} -gt 0 ]; then
        printf 'VERBOSE: %s\n' "$1" >&2;
    fi
}


VERBOSE=0
loglevel=0
toc="TRUE"

optspec=":vhl:t:-:"
while getopts "$optspec" OPTCHAR; do

    showme "OPTARG:  ${OPTARG[*]}"
    showme "OPTIND:  ${OPTIND[*]}"
    case "${OPTCHAR}" in
        -)
            case "${OPTARG}" in
                loglevel) #argument has no equal sign
                    opt=${OPTARG}
                    val="${!OPTIND}"
                    ## check value. If negative, assume user forgot value
                    showme "OPTIND is {$OPTIND} {!OPTIND} has value \"${!OPTIND}\""
                    if [[ "$val" == -* ]]; then
                        die "ERROR: $opt value must not have dash at beginning"
                    fi
                    ## OPTIND=$(( $OPTIND + 1 )) # CAUTION! no effect?
                    printparse "--${OPTARG}" "  " "${val}"
                    loglevel="${val}"
                    shift
                    ;;
                loglevel=*) #argument has equal sign
                    opt=${OPTARG%=*}
                    val=${OPTARG#*=}
                    if [ "${OPTARG#*=}" ]; then
                        printparse "--${opt}" "=" "${val}"
                        loglevel="${val}"
                        ## shift CAUTION don't shift this, fails othewise
                    else
                        die "ERROR: $opt value must be supplied"
                    fi
                    ;;
                toc) #argument has no equal sign
                    opt=${OPTARG}
                    val="${!OPTIND}"
                    ## check value. If negative, assume user forgot value
                    showme "OPTIND is {$OPTIND} {!OPTIND} has value \"${!OPTIND}\""
                    if [[ "$val" == -* ]]; then
                        die "ERROR: $opt value must not have dash at beginning"
                    fi
                    ## OPTIND=$(( $OPTIND + 1 )) #??
                    printparse "--${opt}" " " "${val}"
                    toc="${val}"
                    shift
                    ;;
                toc=*) #argument has equal sign
                    opt=${OPTARG%=*}
                    val=${OPTARG#*=}
                    if [ "${OPTARG#*=}" ]; then
                        toc=${val}
                        printparse "--$opt" " -> " "$toc"
                        ##shift ## NO! dont shift this
                    else
                        die "ERROR: value for $opt undefined"
                    fi
                    ;;

                help)
                    echo "usage: $0 [-v] [--loglevel[=]<value>] [--toc[=]<TRUE,FALSE>]" >&2
                    exit 2
                    ;;
                *)
                    if [ "$OPTERR" = 1 ] && [ "${optspec:0:1}" != ":" ]; then
                        echo "Unknown option --${OPTARG}" >&2
                    fi
                    ;;
            esac;;
        h|-\?|--help)
            ## must rewrite this for all of the arguments
            echo "usage: $0 [-v] [--loglevel[=]<value>] [--toc[=]<TRUE,FALSE>]" >&2
            exit 2
            ;;
        l)
            loglevel=${OPTARG}
            printparse "-l" " "  "${loglevel}"
            ;;
        t)
            toc=${OPTARG}
            ;;
        v)
            VERBOSE=1
            ;;

        *)
            if [ "$OPTERR" != 1 ] || [ "${optspec:0:1}" = ":" ]; then
                echo "Non-option argument: '-${OPTARG}'" >&2
            fi
            ;;
    esac
done



echo "
After Parsing values
"
echo "loglevel  $loglevel" 
echo "toc  $toc"

Answer 3

In case you don't want the getopt dependency, you can do this:

while test $# -gt 0
do
  case $1 in

  # Normal option processing
    -h | --help)
      # usage and help
      ;;
    -v | --version)
      # version info
      ;;
  # ...

  # Special cases
    --)
      break
      ;;
    --*)
      # error unknown (long) option $1
      ;;
    -?)
      # error unknown (short) option $1
      ;;

  # FUN STUFF HERE:
  # Split apart combined short options
    -*)
      split=$1
      shift
      set -- $(echo "$split" | cut -c 2- | sed 's/./-& /g') "$@"
      continue
      ;;

  # Done with options
    *)
      break
      ;;
  esac

  # for testing purposes:
  echo "$1"

  shift
done

Of course, then you can't use long style options with one dash. And if you want to add shortened versions (e.g. --verbos instead of --verbose), then you need to add those manually.

But if you are looking to get getopts functionality along with long options, this is a simple way to do it.

I also put this snippet in a gist.

Answer 4

I have been working on that subject for quite a long time... and made my own library which you will need to source in your main script. See libopt4shell and cd2mpc for an example. Hope it helps !

Answer 5

I don't have enough rep yet to comment or vote his solution up, but sme's answer worked extremely well for me. The only issue I ran into was that the arguments end up wrapped in single-quotes (so I have an strip them out).

I also added some example usages and HELP text. I'll included my slightly extended version here:

#!/bin/bash

# getopt example
# from: https://stackoverflow.com/questions/402377/using-getopts-in-bash-shell-script-to-get-long-and-short-command-line-options
HELP_TEXT=\
"   USAGE:\n
    Accepts - and -- flags, can specify options that require a value, and can be in any order. A double-hyphen (--) will stop processing options.\n\n

    Accepts the following forms:\n\n

    getopt-example.sh -a -b -c value-for-c some-arg\n
    getopt-example.sh -c value-for-c -a -b some-arg\n
    getopt-example.sh -abc some-arg\n
    getopt-example.sh --along --blong --clong value-for-c -a -b -c some-arg\n
    getopt-example.sh some-arg --clong value-for-c\n
    getopt-example.sh
"

aflag=false
bflag=false
cargument=""

# options may be followed by one colon to indicate they have a required argument
if ! options=$(getopt -o abc:h\? -l along,blong,help,clong: -- "$@")
then
    # something went wrong, getopt will put out an error message for us
    exit 1
fi

set -- $options

while [ $# -gt 0 ]
do
    case $1 in
    -a|--along) aflag=true ;;
    -b|--blong) bflag=true ;;
    # for options with required arguments, an additional shift is required
    -c|--clong) cargument="$2" ; shift;;
    -h|--help|-\?) echo -e $HELP_TEXT; exit;;
    (--) shift; break;;
    (-*) echo "$0: error - unrecognized option $1" 1>&2; exit 1;;
    (*) break;;
    esac
    shift
done

# to remove the single quotes around arguments, pipe the output into:
# | sed -e "s/^'\\|'$//g"  (just leading/trailing) or | sed -e "s/'//g"  (all)

echo aflag=${aflag}
echo bflag=${bflag}
echo cargument=${cargument}

while [ $# -gt 0 ]
do
    echo arg=$1
    shift

    if [[ $aflag == true ]]; then
        echo a is true
    fi

done

Answer 6

getopt and getopts are different beasts, and people seem to have a bit of misunderstanding of what they do. getopts is a built-in command to bash to process command-line options in a loop and assign each found option and value in turn to built-in variables, so you can further process them. getopt, however, is an external utility program, and it doesn't actually process your options for you the way that e.g. bash getopts, the Perl Getopt module or the Python optparse/argparse modules do. All that getopt does is canonicalize the options that are passed in — i.e. convert them to a more standard form, so that it's easier for a shell script to process them. For example, an application of getopt might convert the following:

myscript -ab infile.txt -ooutfile.txt

into this:

myscript -a -b -o outfile.txt infile.txt

You have to do the actual processing yourself. You don't have to use getopt at all if you make various restrictions on the way you can specify options:

• only put one option per argument;
• all options go before any positional parameters (i.e. non-option arguments);
• for options with values (e.g. -o above), the value has to go as a separate argument (after a space).

Why use getopt instead of getopts? The basic reason is that only GNU getopt gives you support for long-named command-line options.¹ (GNU getopt is the default on Linux. Mac OS X and FreeBSD come with a basic and not-very-useful getopt, but the GNU version can be installed; see below.)

For example, here's an example of using GNU getopt, from a script of mine called javawrap:

# NOTE: This requires GNU getopt.  On Mac OS X and FreeBSD, you have to install this
# separately; see below.
TEMP=`getopt -o vdm: --long verbose,debug,memory:,debugfile:,minheap:,maxheap: \
             -n 'javawrap' -- "$@"`

if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi

# Note the quotes around `$TEMP': they are essential!
eval set -- "$TEMP"

VERBOSE=false
DEBUG=false
MEMORY=
DEBUGFILE=
JAVA_MISC_OPT=
while true; do
  case "$1" in
    -v | --verbose ) VERBOSE=true; shift ;;
    -d | --debug ) DEBUG=true; shift ;;
    -m | --memory ) MEMORY="$2"; shift 2 ;;
    --debugfile ) DEBUGFILE="$2"; shift 2 ;;
    --minheap )
      JAVA_MISC_OPT="$JAVA_MISC_OPT -XX:MinHeapFreeRatio=$2"; shift 2 ;;
    --maxheap )
      JAVA_MISC_OPT="$JAVA_MISC_OPT -XX:MaxHeapFreeRatio=$2"; shift 2 ;;
    -- ) shift; break ;;
    * ) break ;;
  esac
done

This lets you specify options like --verbose -dm4096 --minh=20 --maxhe 40 --debugfi="/Users/John Johnson/debug.txt" or similar. The effect of the call to getopt is to canonicalize the options to --verbose -d -m 4096 --minheap 20 --maxheap 40 --debugfile "/Users/John Johnson/debug.txt" so that you can more easily process them. The quoting around "$1" and "$2" is important as it ensures that arguments with spaces in them get handled properly.

If you delete the first 9 lines (everything up through the eval set line), the code will still work! However, your code will be much pickier in what sorts of options it accepts: In particular, you'll have to specify all options in the "canonical" form described above. With the use of getopt, however, you can group single-letter options, use shorter non-ambiguous forms of long-options, use either the --file foo.txt or --file=foo.txt style, use either the -m 4096 or -m4096 style, mix options and non-options in any order, etc. getopt also outputs an error message if unrecognized or ambiguous options are found.

NOTE: There are actually two totally different versions of getopt, basic getopt and GNU getopt, with different features and different calling conventions.² Basic getopt is quite broken: Not only does it not handle long options, it also can't even handle embedded spaces inside of arguments or empty arguments, whereas getopts does do this right. The above code will not work in basic getopt. GNU getopt is installed by default on Linux, but on Mac OS X and FreeBSD it needs to be installed separately. On Mac OS X, install MacPorts (http://www.macports.org) and then do sudo port install getopt to install GNU getopt (usually into /opt/local/bin), and make sure that /opt/local/bin is in your shell path ahead of /usr/bin. On FreeBSD, install misc/getopt.

A quick guide to modifying the example code for your own program: Of the first few lines, all is "boilerplate" that should stay the same, except the line that calls getopt. You should change the program name after -n, specify short options after -o, and long options after --long. Put a colon after options that take a value.

Finally, if you see code that has just set instead of eval set, it was written for BSD getopt. You should change it to use the eval set style, which works fine with both versions of getopt, while the plain set doesn't work right with GNU getopt.

¹Actually, getopts in ksh93 supports long-named options, but this shell isn't used as often as bash. In zsh, use zparseopts to get this functionality.

²Technically, "GNU getopt" is a misnomer; this version was actually written for Linux rather than the GNU project. However, it follows all the GNU conventions, and the term "GNU getopt" is commonly used (e.g. on FreeBSD).