We Keep Coding

Is there a way to disable positional arguments given that a flag is true with Python argparse?

I am building a command line tool which should work as follows:
mytool [-h] [-c|--config FILE] [-l|--list] ACTION
positional arguments:
ACTION the action to be performed
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-l, --list show list of configured actions and exit
-c, --config CONFIG use CONFIG instead of the default configuration file
I am using argparse to parse the command line and I am facing what seems to be a limitation of the library. Let me explain through some use-cases.
Use-case 1
$ mytool -h
$ mytool -c path/to/file -h
$ mytool -l -h
$ mytool some-action -h
All the above invocations of mytool shall print the help message and exit, exactly as it is shown above, more importantly showing ACTIONS to be mandatory.
Use-case 2
$ mytool -l
$ mytool -c path/to/file --list
$ mytool --list some-action
$ mytool --list --config path/to/file
All the above invocations must list the configured actions given the content of the configuration files, and exit. The allowed values of ACTION depend on the content of the configuration file, they are not simply hard-coded in the program.
Notice that even if an action is given, it is ignored because -l|--list has a higher precendance, similar to how -h works against other flags and arguments.
Also, please note that solutions such as this, which implement custom argparse.Action sub-classes won't work because the action of the listing flag needs the value of the configuration flag, so the parsing must complete (at least partially) for the listing to begin.
Finally, the absence of the otherwise required positional argument ACTION does not cause the parser to abort with an error.
Use-case 3
In the absence of -l|--list the parser works as expected:
$ mytool -c path/to/file # exits with error, positional argument missing
$ mytool some-action # ok
In simple words, I am trying to make the -l|--list flag "disable" the mandatory enforcing of the positional argument ACTION. Moreover I am looking for a solution that allows me to perform the listing (the action of the -l|--list flag) after the parsing has (at least partially) completed, so the value of the -c|--config flag is available.

-h works the way it does because it uses a special action, argparse._HelpAction. (Simiarly, -v uses argparse._VersionAction.) This action causes the script to exit while parsing, so no folllowing arguments are processed. (Note that any side effects produced while parsing previous arguments may still occur; the only notion of precedence parse_args has is that arguments are processed from left to right.)
If you want -l to similarly show available actions and exit, you need to define your own custom action. For example,
class ListAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string):
print("Allowable actions are...")
print("foo")
print("bar")
print("baz")
parser.exit()
p = argparse.ArgumentParser()
p.add_argument("-l", "--list", action=ListAction)
...
args = p.parse_args()
In particular, p.parse_args(["-l", "-h"]) will list actions without displaying the help message, and p.parse_args(["-h", "-l"]) will print the help message but not list actions. Which one gets processed and terminates your script depends solely on which one appears first in the argument list.

As shown in the usage, ACTION will be required, unless the user uses '-h' or '-v', which exit in the middle of parsing.
You could define ACTION as nargs='?', or as a flagged argument, in which case it is optional.
I was going to suggest giving ACTION choices, which will then appear in the help. But if they depend on '--config' value that could be more awkward (though not impossible).
'-l' could have a custom action class that behaves like 'version', but prints the desired list and exits. But then you can't provide an action as well.
Creating arguments that depend on each other in some way is awkward, though not impossible. It easiest to handle interdependencies after parsing, when all arguments have been read. Keep in mind that argparse tries to accept arguments in any order, '-c' could be before, or after '-l', or Action.
In theory though your custom list action, could check the Namespace for a 'config' value, and base its action on that value. That would work only if you can count on the user providing '-c' before '-l'; argparse won't enforce that for you.
As a general rule it's best to think of argparse as a tool for finding out what the user wants, with limited assistance in policing that, and an ability to automatically format usage and help. Don't expect it to help with complicated interactions and logical mixes of arguments.

Argparse: Ignore dashes in unknown arguments or collect values (potentially starting with dashes) until the next known command

I have a Python script that will later call multiple Bash scripts with supprocess.run. When calling the Python script, the user should be able to specify lists of arguments (some of which might start with hyphens) for the Bash scripts like
python script.py \
--bash-args1 --param1 val1 --param2 val2 \
--bash-args2 bla --param3 val3 --blu
Argparse should parse this into Namespace(bash_args1=['--param1', 'val1', '--param2', 'val2'], bash_args2=['bla', '--param3', 'val3', '--blu']). Is there a canonical way of achieving this? I cannot use nargs=argparse.REMAINDER or parser.parse_known_args because I need to collect the arguments for more than one Bash script and a simple nargs='+' will fail if the secondary arguments start with dashes.
I guess I would need one of the following:
Either something similar to REMAINDER that causes argparse to collect all strings up to the next known argument
an option that tells argparse to ignore dashes in unknown arguments when using nargs='+'.

For posterity:
There are a few ways of working around this limitation of argparse. I wrapped one up and published it on PyPI. You can use it just like argparse. The only difference is that there is an extra option you can supply as nargs parameter in add_argument. When this option is used, the parser collects all unknown arguments (regardless of whether they start with a hyphen or not) until the next known argument. For more info check out the repo on github.

Pytest CLI: how can i ignore test case by name?

User can run test cases like below
python /test/submit.py --pytest-args "test/cases/"
this will be run as
pytest.main("test/cases/")`
submit.py argparse parse argument expects ('--pytest-args', default='', type=str)
submit.py is some module, where one of the def [like initiate_test()] will split --pytest-args by space, and runs pytest based on same split --pytest-args.
Now I want to pass --pytest-args in such a way that it ignore test_name.
But -ignore or -k is not doing what I expect.
Below is how I tried -ignore or -k.
-ignore seems like it is limited to directory & module.
python /test/submit.py --pytest-args "test/cases/ --ignore=test/cases/test.py::test_class::test_name"
while -k=test_name does selective run, but -k!=test_name did not do deselection.
`python /test/submit.py --pytest-args "test/cases/ -k!=test_name"`
Also I tried shell parameter like a='-k not (test_name)' with
python /test/submit.py --pytest-args "test/cases/ $a"
this does not work as there is space in $a.
Note: I don’t have access to test.py or submit.py, so I cannot use markers. So solution should be with respect to CLI.

Use python /test/submit.py --pytest-args "test/cases/ -k-test_name
Note the the test to be skipped starts with a - when passing to -k argument.
One can also spell out the command line argument as -k 'not test_name'.

Parsing limited switches with python argparse

Is there a way to parse only limited number of switches in a function using argparse? Say, my command is:
python sample.py -t abc -r dfg -h klm -n -p qui
And I want argparse to parse from -t to -h and leave the remaining, also show help for these only.
Next I want to parse any switch after -h into another function and see corresponding help there.
Is this behavior possible in argparse? Also is there a way I can modify sys.arg it is using internally?
Thanks.

python sample.py -t abc -r dfg -h klm -n -p qui
And I want argparse to parse from -t to -h and leave the remaining, also show help for these only. Next I want to parse any switch after -h into another function and see corresponding help there.
There are some issues with your specification:
Is -h the regular help? If so it has priority, producing the help without parsing the other arguments. The string after -h suggests you are treating it like a normal user define argument, which would then require initiating the parser with help turned off. But then how would you ask for help?
What sets the break between the two parsings/help? The number of arguments, the -h flag (regardless of order), or the id of the flags. Remember argparse accepts flagged arguments in any order.
You could define one parser that knows about -t and -r, and another that handles -n and -p. Calling each with parse_known_args lets it operate without raising a unknown argument error.
You can also modify the sys.argv. parse_args (and the known variant), takes an optional argv argument. If that is none, then it uses sys.argv[1:]. So you could either modify sys.argv itself (deleting items), or you could pass a subset of sys.argv to the parser.
parser1.parse_known_args(sys.argv[1:5])
parser2.parse_known_args(['-n','one','-o','two'])
parser3.parse_args(sys.argv[3:])
Play with those ideas, and come back to us if there are further questions.

You can always modify sys.args and put anything you wish there.
As for your main question, you can have two parsers. One of them will have arguments -t to -h, the second -n and -p. Then you can use argparse's parse_known_args() method on each parser, which will parse only the arguments defined for each of them.

docopt positional mandatory arguments keeping delimiters in name?

I'm trying a very simple python script with only positional arguments, handled by docopt.
#!/usr/bin/env python
opt_spec = """Test
Usage: docopt_test (import | export <output_file> <output_format>)
docopt_test (-h | --help)
docopt_test (-v | --version)
Options:
-h --help Show this screen.
-v --version Show version.
"""
from docopt import docopt
if __name__ == '__main__':
arguments = docopt(opt_spec, version='Test 1.0')
print(arguments)
When run it will print:
./docopt_test.py export file.xml xml
{'--help': False,
'--version': False,
'<output_file>': 'file.xml',
'<output_format>': 'xml',
'export': True,
'import': False}
The problem is that the output_file and output_format arguments retain the < and > delimiters in the name, making calls like args['output_file'] impossible. Removing the delimiters from the usage string changes the semantics, making the options into keywords.
Is there a way to solve this without having to resort to usage like args['<output_file>']?

I think that's considered part of dispatching, which may have all kinds of language-specific implications. There's an experimental project for docopt-dispatch, which seems to handle it nicely with Python annotations. I have a program called snippets where I remap the argument names within my own style of dispatch but I do so by calling args just the way you did.