I am interested in hook extra arguments parsed using argparse in one class to another method in another class which already has few arguments parsed using argparse module.
Project 1
def x():
parser = argparse.ArgumentParser()
parser.add_argument('--abc')
Project 2
def y():
parser = argparse.ArgumentParser()
parser.add_argument('--temp1')
parser.add_argument('--temp2')
When I run x(), I want to add the "--abc" argument to the list of argument y() has which is "temp1", "temp2" at runtime. Is inheritance the best way to go and defining the constructors accordingly ? Could someone provide some sample code snippet ?
Thanks !
argparse implements a parents feature that lets you add the arguments of one parser to another. Check the documentation. Or to adapt your case:
parser_x = argparse.ArgumentParser(add_help=False)
parser_x.add_argument('--abc')
parser_y = argparse.ArgumentParser(parents=[parser_x])
parser_y.add_argument('--temp1')
parser_y.add_argument('--temp2')
parser_y.print_help()
prints:
usage: ipython [-h] [--abc ABC] [--temp1 TEMP1] [--temp2 TEMP2]
optional arguments:
-h, --help show this help message and exit
--abc ABC
--temp1 TEMP1
--temp2 TEMP2
The add_help=False is needed to avoid a conflict between the -h that parser_x would normally add with the one that parser_y gets.
Another way is to let x add its argument to a predefined parser:
def x(parser=None):
if parser is None:
parser = argparse.ArgumentParser()
parser.add_argument('--abc')
return parser
def y():
....
return parser
parsery = y()
parserx = x(parsery)
It might also be useful to know that add_argument returns a reference to the argument (Action object) that it created.
parser = argparse.ArgumentParser()
arg1 = parser.add_argument('--abc')
Do this in a shell and you'll see that arg1 displays as:
_StoreAction(option_strings=['--abc'], dest='abc', nargs=None,
const=None, default=None, type=None, choices=None,
help=None, metavar=None)
arg1 is an object that you can place in lists, dictionaries. You could even, in theory, add it to another parser. That's in effect what the parents mechanism does (i.e. copy action references from the parent to the child).
You can inspire yourself from Django's management commands. They are basically setup as follow:
The entry point is run_from_argv which calls create_parser, parse the command line, extract the parsed arguments and provide them to execute;
The create_parser method creates an argparse parser and uses add_argument to prepopulate default options available for all commands. This function then calls the add_arguments method of the class which is meant to be overloaded by subclasses;
The execute method is responsible to handle the various behaviours associated to the default options. It then calls handle which is meant to be overloaded by subclasses to handle the specific options introduced by add_arguments.
Your requirements are not completely clear but I think that in your case you don't need to bother with an execute method. I’d go with:
import argparse
import sys
class BaseParser:
def create_parser(self, progname):
parser = argparse.ArgumentParser(prog=progname)
parser.add_argument('--temp1')
parser.add_argument('--temp2')
self.add_arguments(parser)
return parser
def add_arguments(self, parser):
pass # to be optionnally defined in subclasses
def parse_command_line(self, argv):
parser = create_parser(argv[0])
options = parser.parse_args(argv[1:])
parsed_options = vars(options)
self.handle(**parsed_options) # HAS TO be defined in subclasses
class X(BaseParser):
def add_arguments(self, parser):
parser.add_argument('--abc')
def handle(self, **options):
abc = options['abc']
temp1 = options['temp1']
temp2 = options['temp2']
# do stuff with thoses variables
class Y(BaseParser):
def handle(self, **options):
temp1 = options['temp1']
temp2 = options['temp2']
# do stuff
x = X()
y = Y()
args = sys.argv
x.parse_command_line(args)
y.parse_command_line(args)
You could simplify the code further if X is a subclass of Y.
Related
The situation
One module (let's call it A) implements a "primary" argparse parser, that parses known arguments common for all children classes (using parser.parse_known_args(argv)). Then, it passes remaining arguments to another object that it's calling. Let's assume it may be either object of class B or C. Both B and C have their own argparse parsers which parse remaining arguments (using parser.parse_args(argv)). They take different arguments, specific to class B or C.
Example snippet from class A:
parser = argparse.ArgumentParser(
description="Parent parser",
formatter_class=argparse.RawTextHelpFormatter,
allow_abbrev=False
)
parser.add_argument('--argument_A', action="append", default=None,
help="Help of first argument of parser A")
known, remaining_args = parser.parse_known_args(argv)
my_obj = self.create_obj(b_or_c, remaining_args)
Example snippet from class B:
parser = argparse.ArgumentParser(
description="Class B parser",
formatter_class=argparse.RawTextHelpFormatter,
allow_abbrev=False
)
parser.add_argument('--argument_B', action="append", default=None,
help="Help of first argument of parser B")
B_arguments_parsed = parser.parse_args(argv)
Example snippet from class C:
parser = argparse.ArgumentParser(
description="Class C parser",
formatter_class=argparse.RawTextHelpFormatter,
allow_abbrev=False
)
parser.add_argument('--argument_C', action="append", default=None,
help="Help of first argument of parser C")
C_arguments_parsed = parser.parse_args(argv)
While implementing passing the arguments and parsing them in the right places was easy, I didn't find a simple solution to print proper help.
The question
How do I implement help message so that my parent parser (from class A) prints it's own help and help from parser B or C?
I would like to see help message from parser A and, depending on which object was selected, parser B or C.
What's a good way to handle lots of parameters using standard python modules & techniques when creating a function in a module that can be called from the command line or imported and called programmatically?
For example:
# my_thing.py
import argparse
def my_thing(
param1=None, param2=None,
param3=None, param4=None,
param5=None, param6=None,
param7=None, param8=None):
# Do something with all those parameters
pass
def main():
parser = argparse.ArgumentParser()
# add arguments
args = parser.parse_args()
my_thing(
param1=args.param1, param2=args.param2,
param3=args.param3, param4=args.param4,
param5=args.param5, param6=args.param6,
param7=args.param7, param8=args.param8):
if __name__ == "__main__":
main()
or maybe this...
# my_thing.py
import argparse
def my_thing(params):
# Do something with all those parameters
pass
def main():
parser = argparse.ArgumentParser()
# add arguments
args = parser.parse_args()
params = {
"param1":args.param1, "param2":args.param2,
"param3":args.param3, "param4":args.param4,
"param5":args.param5, "param6":args.param6,
"param7":args.param7, "param8":args.param8}
my_thing(params)
if __name__ == "__main__":
main()
It may not be the BEST way, but you could store all the parameters in a dictionary, or order them in a list.
#dictionary
def my_thing(dict):
param_1 = dict['param_1']
param_i = dict['param_i']
# list
def my_thing(list_of_param):
param_1 = list_of_param[0]
...param_i = list_of_param[i]...
A better way would be to create a wrapper object to encapsulate the parameters, but none of those really help for ease of creating new instances.
To create new instances quickly it may help to store the parameters in a .txt or .csv file and parse the file for the different parameters. This would make it easy to run in the command line because you could easily add the file as one of the arguments.
python my_script.py my_parameters.txt
You can actually use a third option, passing the __dict__ attribute of the Namespace object that parser.parse_args() returns, into my_thing. object.__dict__ accesses the underlying dictionary that all objects use to store their attributes. In this case, the attributes of the Namespace object are the command line arguments that are provide to the script.
# my_thing.py
import argparse
def my_thing(params):
print(params)
def main():
parser = argparse.ArgumentParser()
# add arguments
args = parser.parse_args()
my_thing(args.__dict__)
if __name__ == "__main__":
main()
How about using keyword-only arguments?
For example:
import argparse
def my_thing(*, param1, param2, param3):
# Do something with all those parameters
pass
def main():
parser = argparse.ArgumentParser()
# add arguments
args = parser.parse_args()
# see https://stackoverflow.com/q/16878315/5220128
my_thing(**vars(args))
if __name__ == "__main__":
main()
I'm using argprase to create an option, and it's a very specific option to do one specific job. The script currently has roughly 30 knobs, and most aren't used regularly.
I'm creating an option:
opt.add_argument('-opt',help="Some Help", help=argparse.SUPPRESS)
But i want there to be two ways to show the help for the script:
my_script -help
my_script -help-long
I want the -help-long to also show all the hidden args. I couldn't find a way to do this.
Is there a way to implement this behavior?
I don't think there's a builtin way to support this. You can probably hack around it by checking sys.argv directly and using that to modify how you build the parser:
import sys
show_hidden_args = '--help-long' in sys.argv
opt = argparse.ArgumentParser()
opt.add_argument('--hidden-arg', help='...' if show_hidden_args else argparse.SUPPRESS)
opt.add_argument('--help-long', help='Show all options.', action='help')
args = opt.parse_args()
Of course, if writing this over and over is too inconvenient, you can wrap it in a helper function (or subclass ArgumentParser):
def add_hidden_argument(*args, **kwargs):
if not show_hidden_args:
kwargs['help'] = argparse.SUPPRESS
opt.add_argument(*args, **kwargs)
And you'll probably want to add a non-hidden --help-long argument so that users know what it supposedly does...
This is a variation on #mgilson's answer, looking in sys.argv to see whether we should suppress some help or not
import argparse
import sys
def hide_args(arglist):
for action in arglist:
action.help=argparse.SUPPRESS
hidelist=[]
parser = argparse.ArgumentParser()
a1 = parser.add_argument('--foo',help='normal')
a2 = parser.add_argument('--bar',help='hidden')
hidelist.append(a2)
if '-h' in sys.argv[1:]:
hide_args(hidelist)
args = parser.parse_args()
Here I've chosen to interpret --help as asking for a long help; -h for the short. I could have added a separate --longhelp argument instead.
1207:~/mypy$ python3 stack37303960.py --help
usage: stack37303960.py [-h] [--foo FOO] [--bar BAR]
optional arguments:
-h, --help show this help message and exit
--foo FOO normal
--bar BAR hidden
for a short help
1207:~/mypy$ python3 stack37303960.py -h
usage: stack37303960.py [-h] [--foo FOO]
optional arguments:
-h, --help show this help message and exit
--foo FOO normal
add_argument returns a pointer to the Action object that it created. Here I save selected ones in the hidelist. Then I conditionally iterate through that list and change the help to SUPPRESS. Many of the attributes of an Action can be changed after the initial creation (experiment in an interactive session).
The parser also maintains a list of actions. The default help is the first one on the parser._actions list. It uses this list both for parsing and formatting the help.
In [540]: parser._actions[0]
Out[540]: _HelpAction(option_strings=['-h', '--help'], dest='help', nargs=0, const=None, default='==SUPPRESS==', type=None, choices=None, help='show this help message and exit', metavar=None)
You could achieve something by subclassing ArgumentParser and _HelpAction:
class LongHelp(argparse._HelpAction):
def __init__(self,*args, **kwargs):
super().__init__(*args, **kwargs)
def __call__(cls, parser, namespace, values, option_string):
print(parser.long_help)
class ArgParserWithLongHelp(argparse.ArgumentParser):
def __init__(self):
super().__init__(self)
self.long_help = {}
self.add_argument("--long-help", action=LongHelp )
def add_argument(self, *args, **kwargs):
if kwargs.get('long_help'):
self.long_help.update({k:kwargs['long_help'] for k in args})
kwargs.pop('long_help')
super().add_argument(*args, **kwargs)
opt = ArgParserWithLongHelp()
opt.add_argument('-opt', help=argparse.SUPPRESS, long_help='Some extra help')
args = opt.parse_args()
I want to have an argument --foobar using Python argparse, so that whenever this argument appears, the program prints a particular string and exits. I don't want to consume any other arguments, I don't want to check other arguments, nothing.
I have to call add_argument somehow, and then perhaps, from parse_args() get some information and based on that, print my string.
But even though I successfully used argparse before, I am surprised to find I have trouble with this one.
For example, none of the nargs values seem to do what I want, and none of the action values seem to fit. They mess up with the other arguments, which I want to ignore once this one is seen.
How to do it?
Use a custom action= parameter:
import argparse
class FoobarAction(argparse.Action):
def __init__(self, option_strings, dest, **kw):
self.message = kw.pop('message', 'Goodbye!')
argparse.Action.__init__(self, option_strings, dest, **kw)
self.nargs = 0
def __call__(self, parser, *args, **kw):
print self.message
parser.exit()
p = argparse.ArgumentParser()
p.add_argument('--ip', nargs=1, help='IP Address')
p.add_argument('--foobar',
action=FoobarAction,
help='Abort!')
p.add_argument('--version',
action=FoobarAction,
help='print the version number and exit!',
message='1.2.3')
args = p.parse_args()
print args
Reference: https://docs.python.org/2.7/library/argparse.html#action-classes
EDIT:
It looks like there is already an action= that does exactly what FoobarAction does. action='version' is the way to go:
import argparse
p = argparse.ArgumentParser()
p.add_argument('--foobar',
action='version',
version='Goodbye!',
help='Abort!')
args = p.parse_args()
print args
I'm just going to post this here, if it helps then great!
import argparse
parser = argparse.ArgumentParser(description='')
parser.add_argument('-foobar', '--foobar', help='Description for foobar argument',
required=False)
args = vars(parser.parse_args())
if args['foobar'] == 'yes':
foobar()
Usage:
python myscrip.py -foobar yes
Use action='store_true' (see the docs).
arg_test.py:
import argparse
import sys
p = argparse.ArgumentParser()
p.add_argument('--foobar', action='store_true')
args = p.parse_args()
if args.foobar:
print "foobar"
sys.exit()
Usage:
python arg_test.py --foobar
Result:
foobar
I want to know which options were explicitly passed through command-line.
Consider the following argparse setup in test.py:
parser.add_argument("--foo", default=True, action="store_true")
parser.add_argument("--bar", default=False, action="store_true")
When I execute ./test.py --foo --bar, I shall get foo=True, bar=True in the Namespace.
In this case, --foo and --bar were passed explicitly through command-line.
When I execute ./test.py --bar, I shall still get foo=True, bar=True in the Namespace.
So, I need to find which args were actually passed while executing through command-line (in the 2nd case : --bar), without sacrificing the defaults functionality.
One approach is to search in argv, but it's not efficient and doesn't look elegant.
I want to know, if there is any argparse api or any other better approach which shall allow me to do this?
Simply set no default. If the variable is not set, the user did not pass it. After checking that, you can handle the default yourself.
With a 'store_true' action, the builtin default is 'False'
With
parser.add_argument('--foo',action='store_true')
no input produces
Namespace(foo=False)
while '--foo' produces
Namespace(foo=True)
with
parser.add_argument('--foo',action='store_true', default=True)
it is always foo=True. That argument is useless. DO NOT set your own default when using 'store_true' or 'store_false'.
If you want to know whether the user gave you a --foo or not, use the first form, and check whether the namespace value is true or not. If in later code you need foo to be True regardless of what the user gave you, set it explicitly, after you have used argparse.
The answers and comments here recommended to not set defaults, and then handle the defaults on my own within the code. However, the add_argument calls aren't completely under my control, so this wasn't really an option.
Initially I went with checking the presence of the options in sys.argv. This approach quickly proved inefficient, bug-prone and not at all scalable.
Finally, I ended up with this which seems to be working just fine:
class _Reflection(object):
def __init__(self, source, reflection, name=None):
self.source = source
self.reflection = reflection
self.name = name
def __getattr__(self, attribute):
self.attribute = attribute
return _Reflection(self.source.__getattribute__(attribute), self.reflection.__getattribute__(attribute), name=attribute)
def __call__(self, *args, **kwargs):
source_output = self.source(*args, **kwargs)
if self.name == 'add_argument':
# if the method being called is 'add_argument',
# over-ride the 'default' argument's value to 'None' in our secondary argparser.
kwargs['default'] = None
reflection_output = self.reflection(*args, **kwargs)
return _Reflection(source_output, reflection_output)
class ReflectionArgumentParser(object):
def create(self, *args, **kwargs):
self.parser = argparse.ArgumentParser(*args, **kwargs)
self._mirror = argparse.ArgumentParser(*args, **kwargs)
return _Reflection(self.parser, self._mirror)
def parse_args(self, *args, **kwargs):
return self.parser.parse_args(*args, **kwargs)
def filter_defaults(self, *args, **kwargs):
return self._mirror.parse_args(*args, **kwargs)
mirrorParser = ReflectionArgumentParser()
parser = mirrorParser.create()
parser.add_argument('-f', '--foo', default=False, action="store_true")
parser.add_argument('-b', '--baz', default=0, action="store_const", const=10)
parser.add_argument('bar', nargs='*', default='bar')
print mirrorParser.parse_args([])
# Outputs: Namespace(bar='bar', baz=0, foo=False)
print mirrorParser.filter_defaults([])
# Outputs: Namespace(bar=[], baz=None, foo=None)
print mirrorParser.filter_defaults('--foo -b lorem ipsum'.split())
# Outputs: Namespace(bar=['lorem', 'ipsum'], baz=10, foo=True)
I have tried this implementation with argument-groups and subparsers.
This doesn't deal with set_defaults method, however the additions required are trivial.
This is possible using the ArgumentParser.get_default(dest) method.
Basically, you iterate over all of the parsed arguments and collect which ones are not equal to the default value:
args = parser.parse_args()
non_default = {arg: value for (arg, value) in vars(args).iteritems() if value != parser.get_default(arg)}
Although this doesn't work in your specific example because --foo is a do-nothing argument (it sets the variable to the default value).