How do i achieve the following command by overriding argpase.Action method.(overriding __call__() method)
python program.py --delete --file filename
With this definition:
parser = argparse.ArgumentParser()
parser.add_argument('--delete', action='store_true')
parser.add_argument('--file') # default 'store' action
your input should produce args that looks like:
namespace(delete=True, file='filename')
It's up to your code to make use of the boolean args.delete and the string args.file.
The argparse documentation discusses writing a custom Action class, but I don't think you need to do that here. Remember, the primary task of the parser is parsing, figuring out what the user wants in the command line. It is not the executer. It does not act on that information.
Related
I have a legacy Python application which uses some options in its CLI, using argparse, like:
parser = argparse.ArgumentParser()
parser.add_argument('-f', default='foo')
Now I need to remove this option, since now its value cannot be overwritten by users but it has to assume its default value (say 'foo' in the example). Is there a way to keep the option but prevent it to show up and be overwritten by users (so that I can keep the rest of the code as it is)?
It's not entirely clear what you can do, not with the parser. Can you edit the setup? Or just modify results of parsing?
If you can edit the setup, you could replace the add_argument line with a
parser.setdefaults(f='foo')
https://docs.python.org/3/library/argparse.html#parser-defaults
The -f won't appear in the usage or help, but it will appear in the args
Or you could leave it in, but suppress the help display
parser.add_argument('-f', default='foo', help=argparse.SUPPRESS)
https://docs.python.org/3/library/argparse.html#help
Setting the value after parsing is also fine.
Yes you can do that. After the parser is parsed (args = parser.parse_args()) it is a NameSpace so you can do this:
parser = argparse.ArgumentParser()
args = parser.parse_args()
args.foo = 'foo value'
print(args)
>>> Namespace(OTHER_OPTIONS, foo='foo value')
I assumed that you wanted to add test to your parser, so your original code will still work, but you do not want it as an option for the user.
I think it doesn't make sense for the argparse module to provide this as a standard option, but there are several easy ways to achieve what you want.
The most obvious way is to just overwrite the value after having called parse_args() (as already mentioned in comments and in another answer):
args.f = 'foo'
However, the user may not be aware that the option is not supported anymore and that the application is now assuming the value "foo". Depending on the use case, it might be better to warn the user about this. The argparse module has several options to do this.
Another possibility is to use an Action class to do a little magic. For example, you could print a warning if the user provided an option that is not supported anymore, or even use the built-in error handling.
import argparse
class FooAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if values != 'foo':
print('Warning: option `-f` has been removed, assuming `-f foo` now')
# Or use the built-in error handling like this:
# parser.error('Option "-f" is not supported anymore.')
# You could override the argument value like this:
# setattr(namespace, self.dest, 'foo')
parser = argparse.ArgumentParser()
parser.add_argument('-f', default='foo', action=FooAction)
args = parser.parse_args()
print('option=%s' % args.f)
You could also just limit the choices to only "foo" and let argparse create an error for other values:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-f', default='foo', choices=['foo'])
args = parser.parse_args()
print('option=%s' % args.f)
Calling python test.py -f bar would then result in:
usage: test.py [-h] [-f {foo}]
test.py: error: argument -f: invalid choice: 'bar' (choose from 'foo')
The below paste contains relevant snippets from three separate Python files. The first is a script called from the command line which instantiates CIPuller given certain arguments. What happens is that the script gets called with something like:
script.py ci (other args to be swallowed by argparse).
The second is part of a subclass called Puller. The third is part of a subclass of Puller called CIPuller.
This works wonderfully, as the correct subclass is called, and any user using the wrong other args gets to see the correct args for their given subclass, plus the generic arguments from the superclass. (Although I was made aware offline that perhaps I should use argparse sub-commands for this.)
I'm stuck trying to write tests for these classes. Currently, I need an ArgumentParser to instantiate the classes, but in testing I'm not instantiating things from the command line, hence my ArgumentParser is useless.
I tried creating an ArgumentParser in the test harness to pass to CIPuller's constructor in the test code, but if I use add_argument there, argparse understandably complains about double (duplicate) arguments when it calls add_argument in the CIPuller constructor.
What would be a suitable design to test these classes with arguments?
#!/usr/bin/env python
from ci_puller import CIPuller
import argparse
import sys
# Using sys.argv[1] for the argument here, as we don't want to pass that onto
# the subclasses, which should receive a vanilla ArgumentParser
puller_type = sys.argv.pop(1)
parser = argparse.ArgumentParser(
description='Throw data into Elasticsearch.'
)
if puller_type == 'ci':
puller = CIPuller(parser, 'single')
else:
raise ValueError("First parameter must be a supported puller. Exiting.")
puller.run()
class Puller(object):
def __init__(self, parser, insert_type):
self.add_arguments(parser)
self.args = parser.parse_args()
self.insert_type = insert_type
def add_arguments(self,parser):
parser.add_argument(
"-d", "--debug",
help="print debug info to stdout",
action="store_true"
)
parser.add_argument(
"--dontsend",
help="don't actually send anything to Elasticsearch",
action="store_true"
)
parser.add_argument(
"--host",
help="override the default host that the data is sent to",
action='store',
default='kibana.munged.tld'
)
class CIPuller(Puller):
def __init__(self, parser, insert_type):
self.add_arguments(parser)
self.index_prefix = "code"
self.doc_type = "cirun"
self.build_url = ""
self.json_url = ""
self.result = []
super(CIPuller, self).__init__(parser, insert_type)
def add_arguments(self, parser):
parser.add_argument(
'--buildnumber',
help='CI build number',
action='store',
required=True
)
parser.add_argument(
'--testtype',
help='Job type per CI e.g. minitest / feature',
choices=['minitest', 'feature'],
required=True
)
parser.add_argument(
'--app',
help='App e.g. sapi / stats',
choices=['sapi', 'stats'],
required=True
)
Unittesting for argparse is tricky. There is a test/test_argparse.py file that is run as part of the overall Python unittest. But it has a complicated custom testing harness to handle most cases.
There are three basic issues, 1) calling parse_args with test values, 2) testing the resulting args, 3) testing for errors.
Testing the resulting args is relatively easy. And the argparse.Namespace class has simple __eq__ method so you can test one namespace against another.
There are two ways of testing inputs. One is to modify the sys.argv. Initially sys.argv has strings meant for the tester.
self.args = parser.parse_args()
tests sys.argv[1:] as a default. So if you change sys.argv you can test custom values.
But you can also give parse_args a custom list. The argparse docs uses this in most of its examples.
self.args = parser.parse_args(argv=myargv)
If myarg is None it uses sys.argv[1:]. Otherwise it uses that custom list.
Testing errors requires either a custom parse.error method (see docs) or wrapping the parse_args in a try/except block that can catch a sys.exit exception.
How do you write tests for the argparse portion of a python module?
python unittest for argparse
Argparse unit tests: Suppress the help message
Unittest with command-line arguments
Using unittest to test argparse - exit errors
I am using Argparse to parse shell input to my Python function.
The tricky part is that this script first reads in a file that partially determines what kind of arguments are available to Argparse (it's a JSON file containing criteria by which the user can specify what data to output).
But before these arguments are added to my parser, I would like to read in some arguments relating to the file reading itself. (e.g. whether to fix the formatting of the input file). Kinda like this:
test.py (fix_formatting=True, **more arguments added later)
When I try to run args = parser.parse_args() twice, after the initial input and after adding more keys, things fall apart: Argparse quite predictably complains that some of the user input are unrecognized arguments:. I thought I might use subparsers to that end.
So I tried variations of (following the example in the docs as best as I could):
def main():
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='sub-command help')
settingsparser = subparsers.add_parser('settings') #i want a subparser called 'settings'
settingsparser.add_argument('--fix_formatting', action='store_true') #this subparser shall have a --fix_formatting
Then I try to parse only the "settings" part like so:
settings=parser.parse_args(['settings'])
This seems to work. But then I add my other keys and things break:
keys=['alpha','beta','gamma','delta']
for key in keys:
parser.add_argument("--"+key, type=str, help="X")
args = parser.parse_args()
If I parse any input for any of the arguments from keys, Argparse complains that I make an invalid choice: [...] (choose from 'settings'). Now I don't understand why I have to choose from "settings"; the docs say that the parse
will only contain attributes for the main parser and the subparser that was selected by the command line (and not any other subparsers)
what is my error of understanding here?
and if this is the wrong approach, how would one go about parsing one bit of input before another bit?
Any help is much appreciated!
parse_args calls parse_known_args. This returns the args namesparse along with a list of strings (from sys.argv) that it could not process (extras). parse_args raises this error if this list is not empty.
https://docs.python.org/3/library/argparse.html#partial-parsing
Thus parse_known_args is useful if you want to parse some of the input.
sys.argv remains unchanged. Subsequent calls to a parser (whether it was the original one or not) use that again, unless you pass the extras.
I don't think subparsers help you here. They aren't meant for delayed or two stage parsing. I'd suggest playing with the documentation examples for subparsers first.
To the main parser, the subparsers look like
subparsers = parser.add_argument('cmd', choices=['select',...])
In other words, it adds a positional argument where the choices are the subparser names that you define. That may help you see why it expects you to name select. Positionals are normally required.
(there's an exception to this in recent versions, https://stackoverflow.com/a/22994500/901925)
I'm using argparse with optional parameter, but I want to avoid having something like this : script.py -a 1 -b -a 2
Here we have twice the optional parameter 'a', and only the second parameter is returned. I want either to get both values or get an error message.
How should I define the argument ?
[Edit]
This is the code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', dest='alpha', action='store', nargs='?')
parser.add_argument('-b', dest='beta', action='store', nargs='?')
params, undefParams = self.parser.parse_known_args()
append action will collect the values from repeated use in a list
parser.add_argument('-a', '--alpha', action='append')
producing an args namespace like:
namespace(alpha=['1','3'], b='4')
After parsing you can check args.alpha, and accept or complain about the number of values. parser.error('repeated -a') can be used to issue an argparse style error message.
You could implement similar functionality in a custom Action class, but that requires understanding the basic structure and operation of such a class. I can't think anything that can be done in an Action that can't just as well be done in the appended list after.
https://stackoverflow.com/a/23032953/901925 is an answer with a no-repeats custom Action.
Why are you using nargs='?' with flagged arguments like this? Without a const parameter this is nearly useless (see the nargs=? section in the docs).
Another similar SO: Python argparse with nargs behaviour incorrect
I want to use nose and coverage in my project. When I run nose with --with-coverage argument, my programs argument-parsing module goes nuts because "--with-coverage" isn't a real argument according to it.
How do I turn the argparse off, but during testing only? Nose says all my tests fail because of the bad argument.
I actually just ran into this issue myself the other day. You don't need to "disable" your parsing module or anything. What you can do is change the module that uses argparse to ignore those arguments it receives that it doesn't recognize. That way they can still be used by other scripts (for example if your command-line call passes secondary arguments to another program execution).
Without your code, I'll assume you're using the standard parse_args() method on your argparse.ArgumentParser instance. Replace it with parse_known_args() instead.
Then, whenever you subsequently reference the parsed-arguments Namespace object, you'll need to specify and element, specifically 0. While parse_args() returns the args object alone, parse_known_args() returns tuple: the first element is the parsed known arguments, and the latter element contains the ignored unrecognized arguments (which you can later use/pass in your Python code, if necessary).
Here's the example change from my own project:
class RunArgs(object):
'''
A placeholder for processing arguments passed to program execution.
'''
def __init__(self):
self.getArgs()
#self.pause = self.args.pause # old assignment
self.pause = self.args[0].pause # new assignment
#...
def __repr__(self):
return "<RunArgs(t=%s, #=%s, v=%s)>" % (str(x) for x in (self.pause,self.numreads,self.verbose))
def getArgs(self):
global PAUSE_TIME
global NUM_READS
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--pause', required=False,
type=self.checkPauseArg, action='store', default=PAUSE_TIME)
parser.add_argument('-n', '--numreads', required=False,
type=self.checkNumArg, action='store', default=NUM_READS)
parser.add_argument('-v', '--verbose', required=False,
action='store_true')
#self.args = parser.parse_args() # old parse call
self.args = parser.parse_known_args() # new parse call
#...
I've read that you can use nose-testconfig, or otherwise use mock to replace the call (not test it). Though I'd agree with #Ned Batchelder, it begs questioning the structure of the problem.
As a workaround, instead of running nose with command-line arguments, you can have a .noserc or nose.cfg in the current working directory:
[nosetests]
verbosity=3
with-coverage=1
Though, I agree that parse_known_args() is a better solution.
It sounds like you have tests that run your code, and then your code uses argparse which implicitly pulls arguments from sys.argv. This is a bad way to structure your code. Your code under test should be getting arguments passed to it some other way so that you can control what arguments it sees.
This is an example of why global variables are bad. sys.argv is a global, shared by the entire process. You've limited the modularity, and therefore the testability, of your code by relying on that global.