I'm trying to create a command like
prog [-h] [-i ID [ID ...]] | -x [SOMETHING]
{cmd1,cmd2,cmd3}...
So basically at the top level I have a parser that has a mutual exlusive group for the -i and -x options, and then following those (and possibly other) options, I have a command that I want to run. Each command has their own set of options that they use. I can get the commands working fine with the add_subparsers(), but the problem I'm running into is when I try to add an argument to the root parser that has nargs='+'. When I do that, it slurps up all of the arguments for -i thinking that the command is an argument and not an ID.
Is there a way around this? It seems like it would have to look through the arguments to -i looking for a command word and then tell argparse that it should resume parsing at that point.
I had to read your description several times, but I think this is the problem:
prog -i id1 id2 cmd1 -foo 3 ....
and it gives some sort of warning about not finding {cmd1,cmd2,cmd3}. The exact error may differ because in some versions subparsers aren't actually required.
In any case, the arguments to -i are ['id1','id2','cmd1'], everything up to the next - flag. To the main parser, the subparsers argument is just another positional one (with choices). When allocating strings to -i it does not check whether the string matches one of the cmds. It just looks at whether it starts with - or not.
The only way you can use an nargs='+' (or '*') in the context is to include some other flagged argument, e.g.
prog -i id1 id2 -x 3 cmd1 --foo ...
I realize that goes against your mutually_exclusive group.
The basic point is non flag strings are allocated based on position, not value. For a variable nargs you have to have some sort of explicit list terminator.
From the sidebar
Argparse nargs="+" is eating positional argument
It's similar except that your next positional is the subparsers cmd.
==============
A positional with '+' will work right before a subparsers cmd
usage: prog [-h] foo [foo ...] {cmd1,cmd2} ...
In [160]: p1.parse_args('1 22 3 cmd1'.split())
Out[160]: Namespace(cmd='cmd1', foo=['1', '22', '3'])
But that's because strings for foo and cmd are allocated with one regex pattern test.
In
usage: prog [-h] [--bar BAR [BAR ...]] {cmd1,cmd2} ...
strings are allocated to bar without reference to the needs of the following positional, cmd. As shown in the suggested patches for http://bugs.python.org/issue9338, changing this behavior is not a trivial change. It requires an added look-ahead trial-and-error loop.
Related
This script will print env vars.
Using Python 3.9.
The goal is to able to run any subcommands if desired. The error I am getting is that if any additional short flags are added, the "ignore environment" arg is trying to parse it. I dont want this. Additional short flags go to anything assigned after --eval.
parser.py
import argparse, os
def parseargs(p):
p.usage = '%(prog)s [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]'
p.add_argument(
"-i",
"--ignore-environment",
action="store_const",
const=dict(),
dest="env",
help="start with an empty environment",
default=os.environ,
)
p.add_argument(
"--export",
nargs=1,
help="Set argument with --export NAME=VALUE"
)
p.add_argument(
"--eval",
nargs="+",
help="Run any commands with newly updated environment, "
"--eval COMMAND ARGS"
)
return p
Execution as follows
>>> p = argparse.ArgumentParser()
>>> parseargs(p) # assigns arguments to parser
>>> p.parse_args('--export FOO=bar --eval cat test.py'.split()) # This is ok and works correctly. cat is the bash command
Namespace([os.environs..], eval=['cat', 'test.py'], export=['FOO=bar'])
>>>p.parse_args('--export FOO=bar --eval ls -l'.split()) # This is fails
error: unrecognized arguments: -l
How do I get "-l" to be overlook by "-i/ignore environment" but passed to eval, like using cat test.py. I have tried using sub_parser but to no avail. The same result occurs.
The problem is that parse_args tries to identify possible options lexically before ever considering the semantics of any actual option.
Since an option taking a variable number of arguments pretty much has to be the last option used alway, consider making --eval a flag which is used to tell your program how to interpret the remaining positonal arguments. Then ls and -l can be offset by --, preventing parse_args from thinking -l is an undefined option.
p.add_argument(
"--eval",
action='store_true',
help="Run any commands with newly updated environment, "
)
# zero or more, so that you don't have to provide a dummy argument
# when the lack of --eval makes a command unnecessary.
# Wart: you can still use --eval without specifying any commands.
# I don't believe argparse alone is capable of handling this,
# at least not in a way that is simpler than just validating
# arguments after calling parse_args().
p.add_argument('cmd_and_args', nargs='*')
Then your command line could look like
>>> p.parse_args('--export FOO=bar --eval -- ls -l'.split())
or even
>>> p.parse_args('--eval --export FOO=bar -- ls -l'.split())
Later, you'll use the boolean value of args.eval to decide how to treat the list args.cmd_and_args.
Important: One wrinkle with this is that you are attaching these options to arbitrary pre-existing parsers, which may have their own positional arguments defined, so getting this to play nice with the original parser might be difficult, if not impossible.
The other option is to take a single argument to be parsed internally.
p.add_arguments("--eval")
...
args = p.parse_args()
cmd_and_args = shlex.split(args.eval) # or similar
Then
>>> p.parse_args(['--export', 'FOO=bar', '--eval', 'ls -l'])
(Note that using str.split isn't going to work for a command line like --export FOO=bar --eval "ls -l".)
From the Argparse documentation:
If you have positional arguments that must begin with - and don’t look like negative numbers, you can insert the pseudo-argument '--' which tells parse_args() that everything after that is a positional argument [...]
So in your case, there are no changes you can make to how you add or define the arguments, but the string you provide to be parsed should have -- preceding the arguments to the eval option, as such:
--export FOO=bar --eval ls -- -l
I want to recreate [-A [-b value]] where in command would look like this:
test.py -A -b 123
Seems really simple but I can't get it right. My latest attempt has been:
byte = subparser.add_parser("-A")
byte.add_argument("-b", type=int)
While the add_parser command accepts '-A', the parser cannot use it. Look at the help:
usage: ipython3 [-h] {-A} ...
positional arguments:
{-A}
optional arguments:
-h, --help show this help message and exit
A subparser is really a special kind of positional argument. To the main parser, you have effectively defined
add_argument('cmd', choices=['-A'])
But to the parsing code, '-A' looks like an optional's flag, as though you had defined
add_argument('-A')
The error:
error: argument cmd: invalid choice: '123' (choose from '-A')
means that it has skipped over the -A and -b (which aren't defined for the main parser), and tried to parse '123' as the first positional. But it isn't in the list of valid choices.
So to use subparsers, you need specify 'A' as the subparser, not '-A'.
I have this code parsing command line arguments:
def handleCmdLineArgs(self):
parser = argparse.ArgumentParser()
parser.add_argument('-j','--juice', help='juice', default="")
parser.add_argument('-bx','--box', help='box', default="")
args,unknown = parser.parse_known_args()
When I run a command line with an argument that starts with j argparse AFTER the -j argument argparse will replace the -j argument with the remainder of the word:
Example:
program.py -j orange -jungle
argparse will return args.juice = "ungle" instead of the desired "orange"
I have created a workaround but I'm curious if anyone else has seen this or knows the reason why it is happening? Or is this maybe a bug in argparse?
This is expected behaviour. For single-dash options the space is optional. So these two are equivalent:
program.py -jorange
program.py -j orange
See the Option value syntax section of the documentation:
For short options (options only one character long), the option and its value can be concatenated:
>>> parser.parse_args(['-xX'])
Namespace(foo=None, x='X')
If you want to pass in orange -jungle as the value, you need to use quoting on the command line:
program.py -j "orange -jungle"
If you want to pass in additional positional arguments that just happen to start with a -, use -- to signal the end of the option flags:
program.py -j orange -- -jungle
See the Arguments containing - section:
If you have positional arguments that must begin with - and don’t look like negative numbers, you can insert the pseudo-argument '--' which tells parse_args() that everything after that is a positional argument:
>>> parser.parse_args(['--', '-f'])
Namespace(foo='-f', one=None)
I created an cli specification with docopt which works great, however for some reason I have to rewrite it to argparse
Usage:
update_store_products <store_name>...
update_store_products --all
Options:
-a --all Updates all stores configured in config
How to do that?
What is important I don't want to have something like this:
update_store_products [--all] <store_name>...
I think it would be rather something like this:
update_store_products (--all | <store_name>...)
I tried to use add_mutually_exclusive_group, but I got error:
ValueError: mutually exclusive arguments must be optional
First off, you should include the shortest code necessary to reproduce the error in the question itself. Without it an answer is just a shot in the dark.
Now, I'm willing to bet your argparse definitions look a bit something like this:
parser = ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('--all', action='store_true')
group.add_argument('store_name', nargs='*')
The arguments in a mutually exclusive group must be optional, because it would not make much sense to have a required argument there, as the group could then only have that argument ever. The nargs='*' alone is not enough – the required attribute of the created action will be True – to convince the mutex group that the argument is truly optional. What you have to do is add a default:
parser = ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('--all', action='store_true')
group.add_argument('store_name', nargs='*', default=[])
This will result in:
[~]% python2 arg.py
usage: arg.py [-h] (--all | store_name [store_name ...])
arg.py: error: one of the arguments --all store_name is required
[~]% python2 arg.py --all
Namespace(all=True, store_name=[])
[~]% python2 arg.py store1 store2 store3
Namespace(all=False, store_name=['store1', 'store2', 'store3'])
I am using the OptionParser from optparse module to parse my command that I get using the raw_input().
I have these questions.
1.) I use OptionParser to parse this input, say for eg. (getting multiple args)
my prompt> -a foo -b bar -c spam eggs
I did this with setting the action='store_true' in add_option() for '-c',now if there is another option with multiple argument say -d x y z then how to know which arguments come from which option? also if one of the arguments has to be parsed again like
my prompt> -a foo -b bar -c spam '-f anotheroption'
2.) if i wanted to do something like this..
my prompt> -a foo -b bar
my prompt> -c spam eggs
my prompt> -d x y z
now each entry must not affect the other options set by the previous command. how to accomplish these?
For part 2: you want a new OptionParser instance for each line you process. And look at the cmd module for writing a command loop like this.
You can also solve #1 using the nargs option attribute as follows:
parser = OptionParser()
parser.add_option("-c", "", nargs=2)
parser.add_option("-d", "", nargs=3)
optparse solves #1 by requiring that an argument always have the same number of parameters (even if that number is 0), variable-parameter arguments are not allowed:
Typically, a given option either takes
an argument or it doesn’t. Lots of
people want an “optional option
arguments” feature, meaning that some
options will take an argument if they
see it, and won’t if they don’t. This
is somewhat controversial, because it
makes parsing ambiguous: if "-a" takes
an optional argument and "-b" is
another option entirely, how do we
interpret "-ab"? Because of this
ambiguity, optparse does not support
this feature.
You would solve #2 by not reusing the previous values to parse_args, so it would create a new values object rather than update.