We Keep Coding

Click password option only if argument equals something

In click, I'm defining this command:
#click.command('time', short_help='Timesheet Generator')
#click.argument('time_command', type=click.Choice(['this', 'last']))
#click.argument('data_mode', type=click.Choice(['excel', 'exchange']), default='exchange')
#click.option('--password', prompt=True, hide_input=True, confirmation_prompt=False)
#pass_context
def cli(ctx, time_command, data_mode, password):
The issue I have is that I only want the password to prompt if the data_mode argument equals exchange. How can I pull this off?

We can remove the need for a prompt if another parameter does not match a specific value by building a custom class derived from click.Option, and in that class over riding the click.Option.handle_parse_result() method like:
Custom Class:
import click
def PromptIf(arg_name, arg_value):
class Cls(click.Option):
def __init__(self, *args, **kwargs):
kwargs['prompt'] = kwargs.get('prompt', True)
super(Cls, self).__init__(*args, **kwargs)
def handle_parse_result(self, ctx, opts, args):
assert any(c.name == arg_name for c in ctx.command.params), \
"Param '{}' not found for option '{}'".format(
arg_name, self.name)
if arg_name not in opts:
raise click.UsageError(
"Illegal usage: `%s` is a required parameter with" % (
arg_name))
# remove prompt from
if opts[arg_name] != arg_value:
self.prompt = None
return super(Cls, self).handle_parse_result(ctx, opts, args)
return Cls
Using Custom Class:
To use the custom class, pass the cls parameter to click.option decorator like:
#click.option('--an_option', cls=PromptIf('an_argument', 'an_arg_value'))
pass in the name of the parameter to examine for the desired value, and the value to check for.
How does this work?
This works because click is a well designed OO framework. The #click.option() decorator usually instantiates a click.Option object but allows this behavior to be overridden with the cls parameter. So it is a relatively easy matter to inherit from click.Option in our own class and over ride the desired methods.
In this case we over ride click.Option.handle_parse_result() and disable the need to prompt if the other specified parameter does not match the desired.
Note: This answer was inspired by this answer.
Test Code:
#click.command()
#click.argument('an_argument', type=click.Choice(['excel', 'exchange']),
default='exchange')
#click.option('--password', hide_input=True, confirmation_prompt=False,
cls=PromptIf('an_argument', 'exchange'))
def cli(an_argument, password):
click.echo(an_argument)
click.echo(password)
cli('exchange'.split())

You can try splitting this up to multiple commands.
For example, time would be the entry point command. Either time_excel or time_exchange would then be invoked by time based on the value of data_mode. One could have a password prompt while the other wouldn't.
See Invoking Other Commands in Click's documentation.

Python Click: make option depend on previous option

Is there an idiomatic way, using the Python Click library, to create a command where one option depends on a value set by a previous option?
A concrete example (my use case) would be that a command takes an option of type click.File as input, but also an encoding option which specifies the encoding of the input stream:
import click
#click.command()
#click.option("--encoding", type=str, default="utf-8")
#click.option("--input",
type=click.File("r", encoding="CAN I SET THIS DYNAMICALLY BASED ON --encoding?"))
def cli(encoding, input):
pass
I guess it would have to involve some kind of deferred evaluation using a callable, but I'm not sure if it's even possible given the current Click API.
I've figured out I can do something along the following lines:
import click
#click.command()
#click.pass_context
#click.option("--encoding", type=str, default="utf-8")
#click.option("--input", type=str, default="-")
def cli(ctx, encoding, input):
input = click.File("r", encoding=encoding)(input, ctx=ctx)
But it somehow feels less readable / maintainable to decouple the option decorator from the semantically correct type constraint that applies to it, and put str in there instead as a dummy. So if there's a way to keep these two together, please enlighten me.
A proposed workaround:
I guess I could use the click.File type twice, making it lazy in the decorator so that the file isn't actually left opened, the first time around:
#click.option("--input", type=click.File("r", lazy=True), default="-")
This feels semantically more satisfying, but also redundant.

It is possible to inherit from the click.File class and override the .convert() method to allow it to gather the encoding value from the context.
Using a Custom Class
It should look something like:
#click.command()
#click.option("--my_encoding", type=str, default="utf-8")
#click.option("--in_file", type=CustomFile("r", encoding_option_name="my_encoding"))
def cli(my_encoding, in_file):
....
CustomFile should allow the user to specify whichever name they want for the parameter from which the encoding value should be collected, but there can be a reasonable default such as "encoding".
Custom File Class
This CustomFile class can be used in association with an encoding option:
import click
class CustomFile(click.File):
"""
A custom `click.File` class which will set its encoding to
a parameter.
:param encoding_option_name: The 'name' of the encoding parameter
"""
def __init__(self, *args, encoding_option_name="encoding", **kwargs):
# enforce a lazy file, so that opening the file is deferred until after
# all of the command line parameters have been processed (--encoding
# might be specified after --in_file)
kwargs['lazy'] = True
# Python 3 can use just super()
super(CustomFile, self).__init__(*args, **kwargs)
self.lazy_file = None
self.encoding_option_name = encoding_option_name
def convert(self, value, param, ctx):
"""During convert, get the encoding from the context."""
if self.encoding_option_name not in ctx.params:
# if the encoding option has not been processed yet, wrap its
# convert hook so that it also retroactively modifies the encoding
# attribute on self and self.lazy_file
encoding_opt = [
c for c in ctx.command.params
if self.encoding_option_name == c.human_readable_name]
assert encoding_opt, \
"option '{}' not found for encoded_file".format(
self.encoding_option_name)
encoding_type = encoding_opt[0].type
encoding_convert = encoding_type.convert
def encoding_convert_hook(*convert_args):
encoding_type.convert = encoding_convert
self.encoding = encoding_type.convert(*convert_args)
self.lazy_file.encoding = self.encoding
return self.encoding
encoding_type.convert = encoding_convert_hook
else:
# if it has already been processed, just use the value
self.encoding = ctx.params[self.encoding_option_name]
# Python 3 can use just super()
self.lazy_file = super(CustomFile, self).convert(value, param, ctx)
return self.lazy_file

Hook on arguments in argparse python

I am interested in hook extra arguments parsed using argparse in one class to another method in another class which already has few arguments parsed using argparse module.
Project 1
def x():
parser = argparse.ArgumentParser()
parser.add_argument('--abc')
Project 2
def y():
parser = argparse.ArgumentParser()
parser.add_argument('--temp1')
parser.add_argument('--temp2')
When I run x(), I want to add the "--abc" argument to the list of argument y() has which is "temp1", "temp2" at runtime. Is inheritance the best way to go and defining the constructors accordingly ? Could someone provide some sample code snippet ?
Thanks !

argparse implements a parents feature that lets you add the arguments of one parser to another. Check the documentation. Or to adapt your case:
parser_x = argparse.ArgumentParser(add_help=False)
parser_x.add_argument('--abc')
parser_y = argparse.ArgumentParser(parents=[parser_x])
parser_y.add_argument('--temp1')
parser_y.add_argument('--temp2')
parser_y.print_help()
prints:
usage: ipython [-h] [--abc ABC] [--temp1 TEMP1] [--temp2 TEMP2]
optional arguments:
-h, --help show this help message and exit
--abc ABC
--temp1 TEMP1
--temp2 TEMP2
The add_help=False is needed to avoid a conflict between the -h that parser_x would normally add with the one that parser_y gets.
Another way is to let x add its argument to a predefined parser:
def x(parser=None):
if parser is None:
parser = argparse.ArgumentParser()
parser.add_argument('--abc')
return parser
def y():
....
return parser
parsery = y()
parserx = x(parsery)
It might also be useful to know that add_argument returns a reference to the argument (Action object) that it created.
parser = argparse.ArgumentParser()
arg1 = parser.add_argument('--abc')
Do this in a shell and you'll see that arg1 displays as:
_StoreAction(option_strings=['--abc'], dest='abc', nargs=None,
const=None, default=None, type=None, choices=None,
help=None, metavar=None)
arg1 is an object that you can place in lists, dictionaries. You could even, in theory, add it to another parser. That's in effect what the parents mechanism does (i.e. copy action references from the parent to the child).

You can inspire yourself from Django's management commands. They are basically setup as follow:
The entry point is run_from_argv which calls create_parser, parse the command line, extract the parsed arguments and provide them to execute;
The create_parser method creates an argparse parser and uses add_argument to prepopulate default options available for all commands. This function then calls the add_arguments method of the class which is meant to be overloaded by subclasses;
The execute method is responsible to handle the various behaviours associated to the default options. It then calls handle which is meant to be overloaded by subclasses to handle the specific options introduced by add_arguments.
Your requirements are not completely clear but I think that in your case you don't need to bother with an execute method. I’d go with:
import argparse
import sys
class BaseParser:
def create_parser(self, progname):
parser = argparse.ArgumentParser(prog=progname)
parser.add_argument('--temp1')
parser.add_argument('--temp2')
self.add_arguments(parser)
return parser
def add_arguments(self, parser):
pass # to be optionnally defined in subclasses
def parse_command_line(self, argv):
parser = create_parser(argv[0])
options = parser.parse_args(argv[1:])
parsed_options = vars(options)
self.handle(**parsed_options) # HAS TO be defined in subclasses
class X(BaseParser):
def add_arguments(self, parser):
parser.add_argument('--abc')
def handle(self, **options):
abc = options['abc']
temp1 = options['temp1']
temp2 = options['temp2']
# do stuff with thoses variables
class Y(BaseParser):
def handle(self, **options):
temp1 = options['temp1']
temp2 = options['temp2']
# do stuff
x = X()
y = Y()
args = sys.argv
x.parse_command_line(args)
y.parse_command_line(args)
You could simplify the code further if X is a subclass of Y.

How do I get the user specified list of optional command line args not containing the defaults using argparse in python?

I want to know which options were explicitly passed through command-line.
Consider the following argparse setup in test.py:
parser.add_argument("--foo", default=True, action="store_true")
parser.add_argument("--bar", default=False, action="store_true")
When I execute ./test.py --foo --bar, I shall get foo=True, bar=True in the Namespace.
In this case, --foo and --bar were passed explicitly through command-line.
When I execute ./test.py --bar, I shall still get foo=True, bar=True in the Namespace.
So, I need to find which args were actually passed while executing through command-line (in the 2nd case : --bar), without sacrificing the defaults functionality.
One approach is to search in argv, but it's not efficient and doesn't look elegant.
I want to know, if there is any argparse api or any other better approach which shall allow me to do this?

Simply set no default. If the variable is not set, the user did not pass it. After checking that, you can handle the default yourself.

With a 'store_true' action, the builtin default is 'False'
With
parser.add_argument('--foo',action='store_true')
no input produces
Namespace(foo=False)
while '--foo' produces
Namespace(foo=True)
with
parser.add_argument('--foo',action='store_true', default=True)
it is always foo=True. That argument is useless. DO NOT set your own default when using 'store_true' or 'store_false'.
If you want to know whether the user gave you a --foo or not, use the first form, and check whether the namespace value is true or not. If in later code you need foo to be True regardless of what the user gave you, set it explicitly, after you have used argparse.

The answers and comments here recommended to not set defaults, and then handle the defaults on my own within the code. However, the add_argument calls aren't completely under my control, so this wasn't really an option.
Initially I went with checking the presence of the options in sys.argv. This approach quickly proved inefficient, bug-prone and not at all scalable.
Finally, I ended up with this which seems to be working just fine:
class _Reflection(object):
def __init__(self, source, reflection, name=None):
self.source = source
self.reflection = reflection
self.name = name
def __getattr__(self, attribute):
self.attribute = attribute
return _Reflection(self.source.__getattribute__(attribute), self.reflection.__getattribute__(attribute), name=attribute)
def __call__(self, *args, **kwargs):
source_output = self.source(*args, **kwargs)
if self.name == 'add_argument':
# if the method being called is 'add_argument',
# over-ride the 'default' argument's value to 'None' in our secondary argparser.
kwargs['default'] = None
reflection_output = self.reflection(*args, **kwargs)
return _Reflection(source_output, reflection_output)
class ReflectionArgumentParser(object):
def create(self, *args, **kwargs):
self.parser = argparse.ArgumentParser(*args, **kwargs)
self._mirror = argparse.ArgumentParser(*args, **kwargs)
return _Reflection(self.parser, self._mirror)
def parse_args(self, *args, **kwargs):
return self.parser.parse_args(*args, **kwargs)
def filter_defaults(self, *args, **kwargs):
return self._mirror.parse_args(*args, **kwargs)
mirrorParser = ReflectionArgumentParser()
parser = mirrorParser.create()
parser.add_argument('-f', '--foo', default=False, action="store_true")
parser.add_argument('-b', '--baz', default=0, action="store_const", const=10)
parser.add_argument('bar', nargs='*', default='bar')
print mirrorParser.parse_args([])
# Outputs: Namespace(bar='bar', baz=0, foo=False)
print mirrorParser.filter_defaults([])
# Outputs: Namespace(bar=[], baz=None, foo=None)
print mirrorParser.filter_defaults('--foo -b lorem ipsum'.split())
# Outputs: Namespace(bar=['lorem', 'ipsum'], baz=10, foo=True)
I have tried this implementation with argument-groups and subparsers.
This doesn't deal with set_defaults method, however the additions required are trivial.

This is possible using the ArgumentParser.get_default(dest) method.
Basically, you iterate over all of the parsed arguments and collect which ones are not equal to the default value:
args = parser.parse_args()
non_default = {arg: value for (arg, value) in vars(args).iteritems() if value != parser.get_default(arg)}
Although this doesn't work in your specific example because --foo is a do-nothing argument (it sets the variable to the default value).

Argparse Subparsers, and linking to Classes

We have a simple Python program to manage various types of in-house servers, using argparse:
manage_servers.py <operation> <type_of_server>
Operations are things like check, build, deploy, configure, verify etc.
Types of server are just different types of inhouse servers we use.
We have a generic server class, then specific types that inherit from that:
class Server
def configure_logging(self, loggin_file):
...
def check(self):
...
def deploy(self):
...
def configure(self):
...
def __init__(self, hostname):
self.hostname = hostname
logging = self.configure_logging(LOG_FILENAME)
class SpamServer(Server):
def check(self):
...
class HamServer(Server):
def deploy(self):
...
My question is how to link that all up to argparse?
Originally, I was using argparse subparses for the operations (check, build, deploy) and another argument for the type.
subparsers = parser.add_subparsers(help='The operation that you want to run on the server.')
parser_check = subparsers.add_parser('check', help='Check that the server has been setup correctly.')
parser_build = subparsers.add_parser('build', help='Download and build a copy of the execution stack.')
parser_build.add_argument('-r', '--revision', help='SVN revision to build from.')
...
parser.add_argument('type_of_server', action='store', choices=types_of_servers,
help='The type of server you wish to create.')
Normally, you'd link each subparse to a method - and then pass in the type_of_server as an argument. However, that's slightly backwards due to the classes- I need to create an instance of the appropriate Server class, then call the operation method inside of that.
Any ideas of how I could achieve the above? Perhaps a different design pattern for Servers? Or a way to still use argparse as is?
Cheers,
Victor

Just use the parser.add_subparsers(dest=... argument with a mapping of type_of_server to classes:
subparsers = parser.add_subparsers(dest='operation', help='The operation that you want to run on the server.')
...
server_types = dict(spam=SpamServer, ham=HamServer)
args = parser.parse_args()
server = server_types[args.type_of_server]()
getattr(server, args.operation)(args)