We Keep Coding

Click: how do I apply an action to all commands and subcommands but allow a command to opt-out (part duex)?

Building on my initial question, I'd like to be able to have the body of a parent group run before I run my callback.
I have a case where I'd like to automatically run a common function, check_upgrade(), for most of my click commands and sub-commands, but there are a few cases where I don't want to run it. I was thinking I could have a decorator that one can add (e.g. #bypass_upgrade_check) for commands where check_upgrade() should not run.
For example:
def do_upgrade():
print("Performing upgrade")
bypass_upgrade_check = make_exclude_hook_group(do_upgrade)
#click.group(cls=bypass_upgrade_check())
#click.option('--arg1', default=DFLT_ARG1)
#click.option('--arg2', default=DFLT_ARG2)
#click.pass_context
def cli(ctx, arg1, arg2):
config.call_me_before_upgrade_check(arg1, arg2)
#bypass_upgrade_check
#cli.command()
def top_cmd1():
click.echo('cmd1')
#cli.command()
def top_cmd2():
click.echo('cmd2')
#cli.group()
def sub_cmd_group():
click.echo('sub_cmd_group')
#bypass_upgrade_check
#sub_cmd_group.command()
def sub_cmd1():
click.echo('sub_cmd1')
#sub_cmd_group.command()
def sub_cmd2():
click.echo('sub_cmd2')
I'd like things to function like explained in the initial question, but instead of executing do_upgrade() before executing the body of cli(), I'd like it to call:
cli() --> do_upgrade() --> top_cmd1()
for example. Or for a nested command:
cli() --> sub_cmd_group() --> do_upgrade() --> sub_cmd1()
So I guess another way to phrase the question is: is it possible to have the functionality from the original question, but have the callback be called right before the subcommand itself runs instead of being called before any of the Group blocks run?
The reason I need this is because the arguments passed in to the top-level CLI command indicate the server address to check for an upgrade. I need this information to process do_upgrade(). I can't pass this information directly to do_upgrade() because this server information is also used elsewhere in the application. I can query it from do_upgrade() with something like config.get_server().

In a similar fashion to the original question, one way to solve this is to build a custom decorator that pairs with a custom click.Group class. The added complication is to hook the Command.invoke() instead of the Group.invoke() so that the callback will be invoked immediately preceding the Command.invoke() and thus will be invoked after any Group.invoke():
Custom Decorator Builder:
import click
def make_exclude_hook_command(callback):
""" for any command that is not decorated, call the callback """
hook_attr_name = 'hook_' + callback.__name__
class HookGroup(click.Group):
""" group to hook context invoke to see if the callback is needed"""
def group(self, *args, **kwargs):
""" new group decorator to make sure sub groups are also hooked """
if 'cls' not in kwargs:
kwargs['cls'] = type(self)
return super(HookGroup, self).group(*args, **kwargs)
def command(self, *args, **kwargs):
""" new command decorator to monkey patch command invoke """
cmd = super(HookGroup, self).command(*args, **kwargs)
def hook_command_decorate(f):
# decorate the command
ret = cmd(f)
# grab the original command invoke
orig_invoke = ret.invoke
def invoke(ctx):
"""call the call back right before command invoke"""
parent = ctx.parent
sub_cmd = parent and parent.command.commands[
parent.invoked_subcommand]
if not sub_cmd or \
not isinstance(sub_cmd, click.Group) and \
getattr(sub_cmd, hook_attr_name, True):
# invoke the callback
callback()
return orig_invoke(ctx)
# hook our command invoke to command and return cmd
ret.invoke = invoke
return ret
# return hooked command decorator
return hook_command_decorate
def decorator(func=None):
if func is None:
# if called other than as decorator, return group class
return HookGroup
setattr(func, hook_attr_name, False)
return decorator
Using the decorator builder:
To use the decorator we first need to build the decorator like:
bypass_upgrade = make_exclude_hook_command(do_upgrade)
Then we need to use it as a custom class to click.group() like:
#click.group(cls=bypass_upgrade())
...
And finally, we can decorate any commands or sub-commands to the group that need to not use the callback like:
#bypass_upgrade
#my_group.command()
def my_click_command_without_upgrade():
...
How does this work?
This works because click is a well designed OO framework. The #click.group() decorator usually instantiates a click.Group object but allows this behavior to be over-ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Group in our own class and over ride the desired methods.
In this case, we build a decorator that sets an attribute on any click function that does not need the callback called. Then in our custom group, we overide both the group() and the command() decorators so that we can we monkey patch invoke() on the command and if the command that is about to be executed has not been decorated, we call the callback.
Test Code:
def do_upgrade():
click.echo("Performing upgrade")
bypass_upgrade = make_exclude_hook_command(do_upgrade)
#click.group(cls=bypass_upgrade())
#click.pass_context
def cli(ctx):
click.echo('cli')
#bypass_upgrade
#cli.command()
def top_cmd1():
click.echo('cmd1')
#cli.command()
def top_cmd2():
click.echo('cmd2')
#cli.group()
def sub_cmd_group():
click.echo('sub_cmd_group')
#bypass_upgrade
#sub_cmd_group.command()
def sub_cmd1():
click.echo('sub_cmd1')
#sub_cmd_group.command()
def sub_cmd2():
click.echo('sub_cmd2')
if __name__ == "__main__":
commands = (
'top_cmd1',
'top_cmd2',
'sub_cmd_group sub_cmd1',
'sub_cmd_group sub_cmd2',
'--help',
)
import sys, time
time.sleep(1)
print('Click Version: {}'.format(click.__version__))
print('Python Version: {}'.format(sys.version))
for cmd in commands:
try:
time.sleep(0.1)
print('-----------')
print('> ' + cmd)
time.sleep(0.1)
cli(cmd.split())
except BaseException as exc:
if str(exc) != '0' and \
not isinstance(exc, (click.ClickException, SystemExit)):
raise
Results:
Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> sub_cmd_group sub_cmd2
cli
sub_cmd_group
Performing upgrade
sub_cmd2
-----------
> top_cmd1
cli
cmd1
-----------
> top_cmd2
cli
Performing upgrade
cmd2
-----------
> sub_cmd_group sub_cmd1
cli
sub_cmd_group
sub_cmd1
-----------
> sub_cmd_group sub_cmd2
cli
sub_cmd_group
Performing upgrade
sub_cmd2
-----------
> --help
Usage: test.py [OPTIONS] COMMAND [ARGS]...
Options:
--arg1 TEXT
--arg2 TEXT
--help Show this message and exit.
Commands:
sub_cmd_group
top_cmd1
top_cmd2

Python Click multiple command names

Is it possible to do something like this with Python Click?
#click.command(name=['my-command', 'my-cmd'])
def my_command():
pass
I want my command lines to be something like:
mycli my-command
and
mycli my-cmd
but reference the same function.
Do I need to do a class like AliasedGroup?

AliasedGroup is not what you are after, since it allows a shortest prefix match, and it appears you need actual aliases. But that example does provide hints in a direction that can work. It inherits from click.Group and overides some behavior.
Here is a one way to approach what you are after:
Custom Class
This class overides the click.Group.command() method which is used to decorate command functions. It adds the ability to pass a list of command aliases. This class also adds a short help which references the aliased command.
class CustomMultiCommand(click.Group):
def command(self, *args, **kwargs):
"""Behaves the same as `click.Group.command()` except if passed
a list of names, all after the first will be aliases for the first.
"""
def decorator(f):
if isinstance(args[0], list):
_args = [args[0][0]] + list(args[1:])
for alias in args[0][1:]:
cmd = super(CustomMultiCommand, self).command(
alias, *args[1:], **kwargs)(f)
cmd.short_help = "Alias for '{}'".format(_args[0])
else:
_args = args
cmd = super(CustomMultiCommand, self).command(
*_args, **kwargs)(f)
return cmd
return decorator
Using the Custom Class
By passing the cls parameter to the click.group() decorator, any commands added to the group via the the group.command() can be passed a list of command names.
#click.group(cls=CustomMultiCommand)
def cli():
"""My Excellent CLI"""
#cli.command(['my-command', 'my-cmd'])
def my_command():
....
Test Code:
import click
#click.group(cls=CustomMultiCommand)
def cli():
"""My Excellent CLI"""
#cli.command(['my-command', 'my-cmd'])
def my_command():
"""This is my command"""
print('Running the command')
if __name__ == '__main__':
cli('--help'.split())
Test Results:
Usage: my_cli [OPTIONS] COMMAND [ARGS]...
My Excellent CLI
Options:
--help Show this message and exit.
Commands:
my-cmd Alias for 'my-command'
my-command This is my command

Here is a simpler way to solve the same thing:
class AliasedGroup(click.Group):
def get_command(self, ctx, cmd_name):
try:
cmd_name = ALIASES[cmd_name].name
except KeyError:
pass
return super().get_command(ctx, cmd_name)
#click.command(cls=AliasedGroup)
def cli():
...
#click.command()
def install():
...
#click.command()
def remove():
....
cli.add_command(install)
cli.add_command(remove)
ALIASES = {
"it": install,
"rm": remove,
}

Since this question has been asked, a click-aliases library has been created.
It works a bit like the other answers except that you don’t have to declare the command class by yourself:
import click
from click_aliases import ClickAliasedGroup
#click.group(cls=ClickAliasedGroup)
def cli():
pass
#cli.command(aliases=['my-cmd'])
def my_command():
pass

Optional argument in command with click

I am trying to accomplish something not very standard for CLI parsing with Click and it only works partially:
main CLI has multiple sub-commands (in sample below 'show' and 'check')
both those commands might have optional argument, but the argument is preceding them not following
I decided to handle that argument in the group "above" it and pass the value in the context
Sample:
import click
#click.group()
#click.argument('hostname', required=False)
#click.pass_context
def cli(ctx, hostname=None):
""""""
ctx.obj = hostname
click.echo("cli: hostname={}".format(hostname))
#cli.command()
#click.pass_obj
def check(hostname):
click.echo("check: hostname={}".format(hostname))
#cli.command()
#click.pass_obj
def show(hostname):
click.echo("check: hostname={}".format(hostname))
if __name__ == '__main__':
cli()
The part WITH the hostname works:
> pipenv run python cli.py localhost check
cli: hostname=localhost
check: hostname=localhost
> pipenv run python cli.py localhost show
cli: hostname=localhost
check: hostname=localhost
But the part WITHOUT the hostname DOES NOT:
> pipenv run python cli.py show
Usage: cli.py [OPTIONS] [HOSTNAME] COMMAND [ARGS]...
Error: Missing command.
Anybody has an idea about the direction I should start looking into?

This can be done by over riding the click.Group argument parser like:
Custom Class:
class MyGroup(click.Group):
def parse_args(self, ctx, args):
if args[0] in self.commands:
if len(args) == 1 or args[1] not in self.commands:
args.insert(0, '')
super(MyGroup, self).parse_args(ctx, args)
Using Custom Class:
Then to use the custom group, pass it as the cls argument to the group decorator like:
#click.group(cls=MyGroup)
#click.argument('hostname', required=False)
#click.pass_context
def cli(ctx, hostname=None):
....
How?
This works because click is a well designed OO framework. The #click.group() decorator usually instantiates a click.Group object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Group in our own class and over ride desired methods.
In this case we over ride click.Group.parse_args() and if the first parameter matches a command and the second parameter does not, then we insert an empty string as the first parameter. This puts everything back where the parser expects it to be.

Using click.CommandCollection drops group options

import click
#click.group()
#click.option('--username')
def cli1(username):
click.echo(username)
#cli1.command()
def something():
click.echo('howdy')
#click.group()
def cli2():
pass
#cli2.command()
def somethingelse():
click.echo('doody')
cli = click.CommandCollection(sources=[cli1, cli2])
if __name__ == '__main__':
cli()
I would expect that this would allow me to pass --username to something, but when I run this script:
python script.py something --username hi
I get:
Error: no such option: --username
It seems like using the CommandCollection is breaking my options. Has anyone else dealt with this before? There is an open ticket in the click repo for this that hasn't been touched since 2015 and has no solution.

With a bit of new plumbing this can be done.
How??
You can inherit from click.Group and then pass the created class to click.group() like:
#click.group(cls=GroupWithCommandOptions)
In the new class, the options on the group can be applied to the command for parsing, and then during command invocation, the group function can be called with the appropriate options.
New Group Class:
import click
class GroupWithCommandOptions(click.Group):
""" Allow application of options to group with multi command """
def add_command(self, cmd, name=None):
""" Hook the added command and put the group options on the command """
click.Group.add_command(self, cmd, name=name)
# add the group parameters to the command
for param in self.params:
cmd.params.append(param)
# hook the command's invoke with our own
cmd.invoke = self.build_command_invoke(cmd.invoke)
self.invoke_without_command = True
def build_command_invoke(self, original_invoke):
def command_invoke(ctx):
""" insert invocation of group function """
# separate the group parameters
ctx.obj = dict(_params=dict())
for param in self.params:
name = param.name
ctx.obj['_params'][name] = ctx.params[name]
del ctx.params[name]
# call the group function with its parameters
params = ctx.params
ctx.params = ctx.obj['_params']
self.invoke(ctx)
ctx.params = params
# now call (invoke) the original command
original_invoke(ctx)
return command_invoke
Test Code:
# Pass new group class to our group which needs options
#click.group(cls=GroupWithCommandOptions)
#click.option('--username')
def cli1(username):
click.echo(username)
#cli1.command()
def something():
click.echo('howdy')
#click.group()
def cli2():
pass
#cli2.command()
def somethingelse():
click.echo('doody')
cli = click.CommandCollection(sources=[cli1, cli2])
if __name__ == '__main__':
cli('something --username hi'.split())
Results:
hi
howdy

Save a command line option's value in an object with Python's Click library

I want to parse some command line arguments with Python's Click library and save the provided values in an object.
My first guess would be to do it like this:
import click
class Configuration(object):
def __init__(self):
# configuration variables
self.MyOption = None
# method call
self.parseCommandlineArguments()
#click.command()
#click.option('--myoption', type=click.INT, default=5)
def parseCommandlineArguments(self, myoption):
# save option's value in the object
self.MyOption = myoption
# create an instance
configuration = Configuration()
print(configuration.MyOption)
However, this does not work, instead I get:
TypeError: parseCommandlineArguments() takes exactly 2 arguments (1 given)
Apparently, passing self to the decorated function is not the correct way to do it. If I remove self from the method arguments then I can e.g. do print(myoption) and it will print 5 on the screen but the value will not be known to any instances of my Configuration() class.
What is the correct way to handle this? I assume it has something to do with context handling in Click but I cannot get it working based on the provided examples.

If I'm understanding you correctly, you want a command line tool that will take configuration options and then do something with those options. If this is your objective then have a look at the example I posted. This example uses command groups and passes a context object through each command. Click has awesome documentation, be sure to read it.
import click
import json
class Configuration(object):
"""
Having a custom context class is usually not needed.
See the complex application documentation:
http://click.pocoo.org/5/complex/
"""
my_option = None
number = None
is_awesome = False
uber_var = 900
def make_conf(self):
self.uber_var = self.my_option * self.number
pass_context = click.make_pass_decorator(Configuration, ensure=True)
#click.group(chain=True)
#click.option('-m', '--myoption', type=click.INT, default=5)
#click.option('-n', '--number', type=click.INT, default=0)
#click.option('-a', '--is-awesome', is_flag=True)
#pass_context
def cli(ctx, myoption, number, is_awesome):
"""
this is where I will save the configuration
and do whatever processing that is required
"""
ctx.my_option = myoption
ctx.number = number
ctx.is_awesome = is_awesome
ctx.make_conf()
pass
#click.command('save')
#click.argument('output', type=click.File('wb'))
#pass_context
def save(ctx, output):
"""save the configuration to a file"""
json.dump(ctx.__dict__, output, indent=4, sort_keys=True)
return click.secho('configuration saved', fg='green')
#click.command('show')
#pass_context
def show(ctx):
"""print the configuration to stdout"""
return click.echo(json.dumps(ctx.__dict__, indent=4, sort_keys=True))
cli.add_command(save)
cli.add_command(show)
After this is installed your can run commands like this:
mycli -m 30 -n 40 -a show
mycli -m 30 -n 40 -a save foo.json
mycli -m 30 -n 40 -a show save foo.json
The complex example is an excellent demo for developing a highly configurable multi chaining command line tool.