Correct way to get allowed arguments from Argument

Question

Question: What is the intended / official way of accessing possible arguments from an existing argparse.ArgumentParser object?

Example: Let's assume the following context:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)

Here I'd like to get the following list of allowed arguments:

['-h', '--foo', '--help', '-f']

I found the following workaround which does the trick for me

parser._option_string_actions.keys()

But I'm not happy with it, as it involves accessing a _-member that is not officially documented. Whats the correct alternative for this task?

Answer 1

First a note on the argparse docs - it's basically a how-to-use document, not a formal API. The standard for what argparse does is the code itself, the unit tests (test/test_argparse.py), and a paralyzing concern for backward compatibility.

There's no 'official' way of accessing allowed arguments, because users usually don't need to know that (other than reading the help/usage).

But let me illustrate with a simple parser in an iteractive session:

In [247]: parser=argparse.ArgumentParser()
In [248]: a = parser.add_argument('pos')
In [249]: b = parser.add_argument('-f','--foo')

add_argument returns the Action object that it created. This isn't documented, but obvious to any one who has created a parser interactively.

The parser object has a repr method, that displays major parameters. But it has many more attributes, which you can see with vars(parser), or parser.<tab> in Ipython.

In [250]: parser
Out[250]: ArgumentParser(prog='ipython3', usage=None, description=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=True)

The Actions too have repr; the Action subclass is determined by the action parameter.

In [251]: a
Out[251]: _StoreAction(option_strings=[], dest='pos', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [252]: b
Out[252]: _StoreAction(option_strings=['-f', '--foo'], dest='foo', nargs=None, const=None, default=None, type=None, choices=None, help=None, metavar=None)

vars(a) etc can be used to see all attributes.

A key parser attribute is _actions, a list of all defined Actions. This is the basis for all parsing. Note it includes the help action that was created automatically. Look at option_strings; that determines whether the Action is positional or optional.

In [253]: parser._actions
Out[253]: 
[_HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None),
 _StoreAction(option_strings=[], dest='pos',....),
 _StoreAction(option_strings=['-f', '--foo'], dest='foo', ...)]

_option_string_actions is a dictionary, mapping from option_strings to Actions (the same objects that appear in _actions). References to those Action objects appear all over the place in argparse code.

In [255]: parser._option_string_actions
Out[255]: 
{'--foo': _StoreAction(option_strings=['-f', '--foo'],....),
 '--help': _HelpAction(option_strings=['-h', '--help'],...),
 '-f': _StoreAction(option_strings=['-f', '--foo'], dest='foo',...),
 '-h': _HelpAction(option_strings=['-h', '--help'], ....)}

In [256]: list(parser._option_string_actions.keys())
Out[256]: ['-f', '--help', '-h', '--foo']

Note that there is a key for each - string, long or short; but there's nothing for pos, the positional has an empty option_strings parameter.

If that list of keys is what you want, use it, and don't worry about the _. It does not have a 'public' alias.

I can understand parsing the help to discover the same; but that's a lot of work to just avoid using a 'private' attribute. If you worry about the undocumented attribute being changed, you should also worry about the help format being changed. That isn't part of the docs either.

help layout is controlled by parser.format_help. The usage is created from information in self._actions. Help lines from information in

    for action_group in self._action_groups:
        formatter.add_arguments(action_group._group_actions)

(you don't want to get into action groups do you?).

There is another way of getting the option_strings - collect them from the _actions:

In [258]: [a.option_strings for a in parser._actions]
Out[258]: [['-h', '--help'], [], ['-f', '--foo']]

===================

Delving in to code details a bit:

parser.add_argument creates an Action, and then passes it to parser._add_action. This is the method the populates both .actions and action.option_strings.

    self._actions.append(action)
    for option_string in action.option_strings:
        self._option_string_actions[option_string] = action

Answer 2

I don't think there is a "better" way to achieve what you want.

---

If you really don't want to use the _option_string_actions attribute, you could process the parser.format_usage() to retrieve the options, but doing this, you will get only the short options names.

If you want both short and long options names, you could process the parser.format_help() instead.

This process can be done with a very simple regular expression: -+\w+

import re

OPTION_RE = re.compile(r"-+\w+")
PARSER_HELP = """usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]

optional arguments:
  -h, --help         show this help message and exit
  --foo FOO, -f FOO  a random options
  --bar BAR, -b BAR  a more random option
"""

options = set(OPTION_RE.findall(PARSER_HELP))

print(options)
# set(['-f', '-b', '--bar', '-h', '--help', '--foo'])

---

Or you could first make a dictionnary which contains the argument parser configuration and then build the argmuent parser from it. Such a dictionnary could have the option names as key and the option configuration as value. Doing this, you can access the options list via the dictionnary keys flattened with itertools.chain:

import argparse
import itertools

parser_config = {
    ('--foo', '-f'): {"help": "a random options", "type": str},
    ('--bar', '-b'): {"help": "a more random option", "type": int, "default": 0}
}

parser = argparse.ArgumentParser()
for option, config in parser_config.items():
    parser.add_argument(*option, **config)

print(parser.format_help())
# usage: test_args_2.py [-h] [--foo FOO] [--bar BAR]
# 
# optional arguments:
#   -h, --help         show this help message and exit
#   --foo FOO, -f FOO  a random options
#   --bar BAR, -b BAR  a more random option

print(list(itertools.chain(*parser_config.keys())))
# ['--foo', '-f', '--bar', '-b']

This last way is what I would do, if I was reluctant to use _option_string_actions.

Answer 3

I have to agree with Tryph's answer.

Not pretty, but you can retrieve them from parser.format_help():

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--foo', '-f', type=str)
goal = parser._option_string_actions.keys()

def get_allowed_arguments(parser):
    lines = parser.format_help().split('\n')
    line_index = 0
    number_of_lines = len(lines)
    found_optional_arguments = False
    # skip the first lines until the section 'optional arguments'
    while line_index < number_of_lines:
        if lines[line_index] == 'optional arguments:':
            found_optional_arguments = True
            line_index += 1
            break
        line_index += 1
    result_list = []
    if found_optional_arguments:
        while line_index < number_of_lines:
            arg_list = get_arguments_from_line(lines[line_index])
            if len(arg_list) == 0:
                break
            result_list += arg_list
            line_index += 1
    return result_list

def get_arguments_from_line(line):
    if line[:2] != '  ':
        return []
    arg_list = []
    i = 2
    N = len(line)
    inside_arg = False
    arg_start = 2
    while i < N:
        if line[i] == '-' and not inside_arg:
            arg_start = i
            inside_arg = True
        elif line[i] in [',',' '] and inside_arg:
            arg_list.append(line[arg_start:i+1])
            inside_arg = False
        i += 1
    return arg_list

answer = get_allowed_arguments(parser)

There's probably a regular expressions alternative to the above mess...

Answer 4

This started as a joke answer, but I've learned something since - so I'll post it.

Assume, we know the maximum length of an option allowed. Here is a nice answer to the question in this situation:

from itertools import combinations

def parsable(option):
    try:
        return len(parser.parse_known_args(option.split())[1]) != 2
    except:
        return False

def test(tester, option):
    return any([tester(str(option) + ' ' + str(v)) for v in ['0', '0.0']])

def allowed_options(parser, max_len=3, min_len=1):
    acceptable = []
    for l in range(min_len, max_len + 1):
        for option in combinations([c for c in [chr(i) for i in range(33, 127)] if c != '-'], l):
            option = ''.join(option)
            acceptable += [p + option for p in ['-', '--'] if test(parsable, p + option)]
    return acceptable

Of course this is very pedantic as the question doesn't require any specific runtime. So I'll ignore that here. I'll also disregard, that the above version produces a mess of output because one can get rid of it easily.

But more importantly, this method detected the following interesting argparse "features":

• In in the OP example, argparse would also allow --fo. This has to be a bug.
• But further, in the OP example again, argparse would also allow -fo (ie. setting foo to o without space or anything). This is documented and intended, but I didn't know it.

Because of this, a correct solution is a bit longer and would look something like this (only parsable changes, I'll omit the other methods):

def parsable(option):
    try:
        default = vars(parser.parse_known_args(['--' + '0' * 200])[0])
        parsed, remaining = parser.parse_known_args(option.split())
        if len(remaining)  == 2:
            return False
        parsed = vars(parsed)
        for k in parsed.keys():
            try:
                if k in default and default[k] != parsed[k] and float(parsed[k]) != 0.0:
                    return False  # Filter '-fx' cases where '-f' is the argument and 'x' the value.
            except:
                return False
        return True
    except:
        return False

Summary: Besides all the restrictions (runtime and fixed maximum option length), this is the only answer that correctly respects the real parser behavior - however buggy it may even be. So here you are, a perfect answer that is absolutely useless.