I am using the OptionParser from optparse module to parse my command that I get using the raw_input().
I have these questions.
1.) I use OptionParser to parse this input, say for eg. (getting multiple args)
my prompt> -a foo -b bar -c spam eggs
I did this with setting the action='store_true' in add_option() for '-c',now if there is another option with multiple argument say -d x y z then how to know which arguments come from which option? also if one of the arguments has to be parsed again like
my prompt> -a foo -b bar -c spam '-f anotheroption'
2.) if i wanted to do something like this..
my prompt> -a foo -b bar
my prompt> -c spam eggs
my prompt> -d x y z
now each entry must not affect the other options set by the previous command. how to accomplish these?
For part 2: you want a new OptionParser instance for each line you process. And look at the cmd module for writing a command loop like this.
You can also solve #1 using the nargs option attribute as follows:
parser = OptionParser()
parser.add_option("-c", "", nargs=2)
parser.add_option("-d", "", nargs=3)
optparse solves #1 by requiring that an argument always have the same number of parameters (even if that number is 0), variable-parameter arguments are not allowed:
Typically, a given option either takes
an argument or it doesn’t. Lots of
people want an “optional option
arguments” feature, meaning that some
options will take an argument if they
see it, and won’t if they don’t. This
is somewhat controversial, because it
makes parsing ambiguous: if "-a" takes
an optional argument and "-b" is
another option entirely, how do we
interpret "-ab"? Because of this
ambiguity, optparse does not support
this feature.
You would solve #2 by not reusing the previous values to parse_args, so it would create a new values object rather than update.
Related
This script will print env vars.
Using Python 3.9.
The goal is to able to run any subcommands if desired. The error I am getting is that if any additional short flags are added, the "ignore environment" arg is trying to parse it. I dont want this. Additional short flags go to anything assigned after --eval.
parser.py
import argparse, os
def parseargs(p):
p.usage = '%(prog)s [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]'
p.add_argument(
"-i",
"--ignore-environment",
action="store_const",
const=dict(),
dest="env",
help="start with an empty environment",
default=os.environ,
)
p.add_argument(
"--export",
nargs=1,
help="Set argument with --export NAME=VALUE"
)
p.add_argument(
"--eval",
nargs="+",
help="Run any commands with newly updated environment, "
"--eval COMMAND ARGS"
)
return p
Execution as follows
>>> p = argparse.ArgumentParser()
>>> parseargs(p) # assigns arguments to parser
>>> p.parse_args('--export FOO=bar --eval cat test.py'.split()) # This is ok and works correctly. cat is the bash command
Namespace([os.environs..], eval=['cat', 'test.py'], export=['FOO=bar'])
>>>p.parse_args('--export FOO=bar --eval ls -l'.split()) # This is fails
error: unrecognized arguments: -l
How do I get "-l" to be overlook by "-i/ignore environment" but passed to eval, like using cat test.py. I have tried using sub_parser but to no avail. The same result occurs.
The problem is that parse_args tries to identify possible options lexically before ever considering the semantics of any actual option.
Since an option taking a variable number of arguments pretty much has to be the last option used alway, consider making --eval a flag which is used to tell your program how to interpret the remaining positonal arguments. Then ls and -l can be offset by --, preventing parse_args from thinking -l is an undefined option.
p.add_argument(
"--eval",
action='store_true',
help="Run any commands with newly updated environment, "
)
# zero or more, so that you don't have to provide a dummy argument
# when the lack of --eval makes a command unnecessary.
# Wart: you can still use --eval without specifying any commands.
# I don't believe argparse alone is capable of handling this,
# at least not in a way that is simpler than just validating
# arguments after calling parse_args().
p.add_argument('cmd_and_args', nargs='*')
Then your command line could look like
>>> p.parse_args('--export FOO=bar --eval -- ls -l'.split())
or even
>>> p.parse_args('--eval --export FOO=bar -- ls -l'.split())
Later, you'll use the boolean value of args.eval to decide how to treat the list args.cmd_and_args.
Important: One wrinkle with this is that you are attaching these options to arbitrary pre-existing parsers, which may have their own positional arguments defined, so getting this to play nice with the original parser might be difficult, if not impossible.
The other option is to take a single argument to be parsed internally.
p.add_arguments("--eval")
...
args = p.parse_args()
cmd_and_args = shlex.split(args.eval) # or similar
Then
>>> p.parse_args(['--export', 'FOO=bar', '--eval', 'ls -l'])
(Note that using str.split isn't going to work for a command line like --export FOO=bar --eval "ls -l".)
From the Argparse documentation:
If you have positional arguments that must begin with - and don’t look like negative numbers, you can insert the pseudo-argument '--' which tells parse_args() that everything after that is a positional argument [...]
So in your case, there are no changes you can make to how you add or define the arguments, but the string you provide to be parsed should have -- preceding the arguments to the eval option, as such:
--export FOO=bar --eval ls -- -l
I'm creating a command-line interface for my script. I'd like the user to be able to provide options either in the command-line itself, or in a configuration file.
Certain options have reasonable defaults. The logic I want is this:
If the option is neither in the command line nor in the config file, use the default.
If the option is in the config file only, or the command line only, use the option.
If the option is in both the config file and the command line, the explicit command line takes precedence.
For example, here is my docstring:
"""Usage: my-script.py [options] <some_arg>
Options:
-c --config=<file> Configuration file, if it exists.
[default: settings.yaml]
-d --delay=<delay> Delay before scraping each message, to avoid rate
limiting. Delays by a gaussian distribution with
average <delay> and standard deviation <delay>/2.
[default: 1]
"""
And so, without a settings.yaml I want:
$ python my-script.py foo # delay is "1"
$ python my-script.py -d 5 foo # delay is "5"
And with a settings.yaml of delay: 10 I want:
$ python my-script.py foo # delay is "10"
$ python my-script.py -d 5 foo # delay is "5"
The issue I run into is that docopt just gives me either a {"--delay": "1"} or {"--delay": "5"}. I have no way of knowing whether it comes from a default. Further, I want to specify the default in the docstring - this is much nicer from the user's perspective.
Is there any good docopt-y, Pythonic way to accomplish this? I was thinking to check the parsed arguments against the default arguments, but then the user would not be able to override a config file argument with a command-line argument if it happens to match the argument value.
Here's how I'm loading the config file:
args = docopt(__doc__, version='Yahoo! Groups Backup-er 0.1',
options_first=True)
cfg_args = {}
if args['--config'] != 'settings.yaml' and not os.path.exists(args['--config']):
sys.exit("Specified config file '%s' does not exist." % args['--config'])
if os.path.exists(args['--config']):
settings = yaml.load(open(args['--config']))
command_line_args = args
args = {}
for key, val in settings.items():
args['--%s' % key] = val
args.update(command_line_args)
Yes, the syntax to check is:
args = docopt(__doc__)
if args["--flag"]:
print "With --flag\n"
Please read this article for details.
Is there a way to parse only limited number of switches in a function using argparse? Say, my command is:
python sample.py -t abc -r dfg -h klm -n -p qui
And I want argparse to parse from -t to -h and leave the remaining, also show help for these only.
Next I want to parse any switch after -h into another function and see corresponding help there.
Is this behavior possible in argparse? Also is there a way I can modify sys.arg it is using internally?
Thanks.
python sample.py -t abc -r dfg -h klm -n -p qui
And I want argparse to parse from -t to -h and leave the remaining, also show help for these only. Next I want to parse any switch after -h into another function and see corresponding help there.
There are some issues with your specification:
Is -h the regular help? If so it has priority, producing the help without parsing the other arguments. The string after -h suggests you are treating it like a normal user define argument, which would then require initiating the parser with help turned off. But then how would you ask for help?
What sets the break between the two parsings/help? The number of arguments, the -h flag (regardless of order), or the id of the flags. Remember argparse accepts flagged arguments in any order.
You could define one parser that knows about -t and -r, and another that handles -n and -p. Calling each with parse_known_args lets it operate without raising a unknown argument error.
You can also modify the sys.argv. parse_args (and the known variant), takes an optional argv argument. If that is none, then it uses sys.argv[1:]. So you could either modify sys.argv itself (deleting items), or you could pass a subset of sys.argv to the parser.
parser1.parse_known_args(sys.argv[1:5])
parser2.parse_known_args(['-n','one','-o','two'])
parser3.parse_args(sys.argv[3:])
Play with those ideas, and come back to us if there are further questions.
You can always modify sys.args and put anything you wish there.
As for your main question, you can have two parsers. One of them will have arguments -t to -h, the second -n and -p. Then you can use argparse's parse_known_args() method on each parser, which will parse only the arguments defined for each of them.
Consider the following script:
import argparse
parser1 = argparse.ArgumentParser()
parser1.add_argument('-a')
args1 = parser1.parse_args()
parser2 = argparse.ArgumentParser()
parser2.add_argument('-b')
args2 = parser2.parse_args()
I have several questions:
Is parse_args a one-time method or is there a way to clear the
arguments before adding new ones? (e.g. something like
args1.clear() or parser1.clear())
The result of this script is unusable. Although this script accepts
the -a argument, it does not accept any value for 'a'. Nor does it
accept any -b argument. Is there some way to make any of the arguments really work?
This is my actual scenario: I have 2 scripts. Both import the same
file which has initialization code (load config files, create
loggers, etc.), lets call it init.py This init.py file also parses
the arguments only because it needs one value from it. The problem
is that I need one of the scripts to accept other arguments as well.
Since init.py does something with one argument, I cannot wait with
parse_args. How can I make it work?
Edit:
Here is the output of my script:
[prompt]# python2.7 myscript.py -a
usage: a.py [-h] [-a A]
myscript.py: error: argument -a: expected one argument
[prompt]# python2.7 myscript.py -a 1
Namespace(a='1')
usage: a.py [-h] [-b B]
myscript.py: error: unrecognized arguments: -a 1
Your scenario is quite unclear, but I guess what you're looking for is parse_known_args
Here I guessed that you called init.py from the other files, say caller1.py and caller2.py
Also suppose that init.py only parses -a argument, while the original script will parse the rest.
You can do something like this:
in init.py put this in do_things method:
parser = argparse.ArgumentParser()
parser.add_argument('-a')
parsed = parser.parse_known_args(sys.argv)
print 'From init.py: %s' % parsed['a']
In caller1.py:
init.do_things(sys.argv)
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parsed = parser.parse_known_args(sys.argv)
print 'From caller1.py: %s' % parsed['b']
If you call caller1.py as follows: python caller1.py -a foo -b bar, the result will be:
From init.py: foo
From caller1.py: bar
But if your scenario is not actually like this, I would suggest to use #Michael0x2a answer, which is just to use single ArgumentParser object in caller1.py and pass the value appropriately for init.py
This doesn't really make sense, because for all intents and purposes, the parser object is stateless. There's nothing to clear, since all it does is takes in the console arguments, and returns a Namespace object (a pseudo-dict) without ever modifying anything in the process.
Therefore, you can consider parse_args() to be idempotent. You can repeatedly call it over and over, and the same output will occur. By default, it will read the arguments from sys.argv, which is where the console arguments are stored.
However, note that you can pipe in custom arguments by passing in a list to the parse_args function so that the parser will using something other then sys.argv as input.
I'm not sure what you mean. If you call python myscript.py -a 15, args1 will equal Namespace(a='15'). You can then do args1['a'] to obtain the value of 15. If you want to make the flag act as a toggle, call parser.add_argument('-a', action='store_true'). Here is a list of all available actions.
I would try and confine all the console/interface code into a single module and into a single parser. Basically, remove the code to parse the command line from init.py and the second file into an independent little section. Once you run the parser, which presents a unified interface for everything in your program, pass in the appropriate variables to functions inside init.py. This has the added advantage of keeping the UI separate and more easily interchangeable with the rest of the code.
First of all, I know optparse is deprecated since version 2.7, but I only have Python 2.3 available in the machine I'm working.
The question is how to know the order in which the options were given in the command line, for instance:
python example.py -f foo -b bar
will execute first the option f and then the option b and
python example.py -b bar -f foo
will do the opposite.
The only solution I came up with after reading optargs documentation is to use the callback action to store the option and detect the position relative to the other options, since the options object doesn't seem to follow any particular order.
Do you know another (maybe better) solution to this problem?
It's against convention to have option flags that trigger different behaviors depending on the order. But if you really want check for the order, you can just look in sys.argv
#assuming both -f and -b are given in cmdline and you need to check for order
index_f = sys.argv.find("-f")
index_b = sys.argv.find("-b")
if index_f < index_b:
# do something if -f is before -b