I have a function which is wrapped as a command using click. So it looks like this:
#click.command()
#click.option('-w', '--width', type=int, help="Some helping message", default=0)
[... some other options ...]
def app(width, [... some other option arguments...]):
[... function code...]
I have different use cases for this function. Sometimes, calling it through the command line is fine, but sometime I would also like to call directly the function
from file_name import app
width = 45
app(45, [... other arguments ...])
How can we do that? How can we call a function that has been wrapped as a command using click? I found this related post, but it is not clear to me how to adapt it to my case (i.e., build a Context class from scratch and use it outside of a click command function).
EDIT: I should have mentioned: I cannot (easily) modify the package that contains the function to call. So the solution I am looking for is how to deal with it from the caller side.
You can call a click command function from regular code by reconstructing the command line from parameters. Using your example it could look somthing like this:
call_click_command(app, width, [... other arguments ...])
Code:
def call_click_command(cmd, *args, **kwargs):
""" Wrapper to call a click command
:param cmd: click cli command function to call
:param args: arguments to pass to the function
:param kwargs: keywrod arguments to pass to the function
:return: None
"""
# Get positional arguments from args
arg_values = {c.name: a for a, c in zip(args, cmd.params)}
args_needed = {c.name: c for c in cmd.params
if c.name not in arg_values}
# build and check opts list from kwargs
opts = {a.name: a for a in cmd.params if isinstance(a, click.Option)}
for name in kwargs:
if name in opts:
arg_values[name] = kwargs[name]
else:
if name in args_needed:
arg_values[name] = kwargs[name]
del args_needed[name]
else:
raise click.BadParameter(
"Unknown keyword argument '{}'".format(name))
# check positional arguments list
for arg in (a for a in cmd.params if isinstance(a, click.Argument)):
if arg.name not in arg_values:
raise click.BadParameter("Missing required positional"
"parameter '{}'".format(arg.name))
# build parameter lists
opts_list = sum(
[[o.opts[0], str(arg_values[n])] for n, o in opts.items()], [])
args_list = [str(v) for n, v in arg_values.items() if n not in opts]
# call the command
cmd(opts_list + args_list)
How does this work?
This works because click is a well designed OO framework. The #click.Command object can be introspected to determine what parameters it is expecting. Then a command line can be constructed that will look like the command line that click is expecting.
Test Code:
import click
#click.command()
#click.option('-w', '--width', type=int, default=0)
#click.option('--option2')
#click.argument('argument')
def app(width, option2, argument):
click.echo("params: {} {} {}".format(width, option2, argument))
assert width == 3
assert option2 == '4'
assert argument == 'arg'
width = 3
option2 = 4
argument = 'arg'
if __name__ == "__main__":
commands = (
(width, option2, argument, {}),
(width, option2, dict(argument=argument)),
(width, dict(option2=option2, argument=argument)),
(dict(width=width, option2=option2, argument=argument),),
)
import sys, time
time.sleep(1)
print('Click Version: {}'.format(click.__version__))
print('Python Version: {}'.format(sys.version))
for cmd in commands:
try:
time.sleep(0.1)
print('-----------')
print('> {}'.format(cmd))
time.sleep(0.1)
call_click_command(app, *cmd[:-1], **cmd[-1])
except BaseException as exc:
if str(exc) != '0' and \
not isinstance(exc, (click.ClickException, SystemExit)):
raise
Test Results:
Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> (3, 4, 'arg', {})
params: 3 4 arg
-----------
> (3, 4, {'argument': 'arg'})
params: 3 4 arg
-----------
> (3, {'option2': 4, 'argument': 'arg'})
params: 3 4 arg
-----------
> ({'width': 3, 'option2': 4, 'argument': 'arg'},)
params: 3 4 arg
I tried with Python 3.7 and Click 7 the following code:
import click
#click.command()
#click.option('-w', '--width', type=int, default=0)
#click.option('--option2')
#click.argument('argument')
def app(width, option2, argument):
click.echo("params: {} {} {}".format(width, option2, argument))
assert width == 3
assert option2 == '4'
assert argument == 'arg'
app(["arg", "--option2", "4", "-w", 3])
app(["arg", "-w", 3, "--option2", "4" ])
app(["-w", 3, "--option2", "4", "arg"])
All the app calls are working fine!
This use-case is described in the docs.
Sometimes, it might be interesting to invoke one command from another command. This is a pattern that is generally discouraged with Click, but possible nonetheless. For this, you can use the Context.invoke() or Context.forward() methods.
cli = click.Group()
#cli.command()
#click.option('--count', default=1)
def test(count):
click.echo('Count: %d' % count)
#cli.command()
#click.option('--count', default=1)
#click.pass_context
def dist(ctx, count):
ctx.forward(test)
ctx.invoke(test, count=42)
They work similarly, but the difference is that Context.invoke() merely invokes another command with the arguments you provide as a caller, whereas Context.forward() fills in the arguments from the current command.
If you just want to call the underlying function, you can directly access it as click.Command.callback. Click stores the underlying wrapped Python function as a class member. Note that directly calling the function will bypass all Click validation and any Click context information won't be there.
Here is an example code that iterates all click.Command objects in the current Python module and makes a dictionary of callable functions out from them.
from functools import partial
from inspect import getmembers
import click
all_functions_of_click_commands = {}
def _call_click_command(cmd: click.Command, *args, **kwargs):
result = cmd.callback(*args, **kwargs)
return result
# Pull out all Click commands from the current module
module = sys.modules[__name__]
for name, obj in getmembers(module):
if isinstance(obj, click.Command) and not isinstance(obj, click.Group):
# Create a wrapper Python function that calls click Command.
# Click uses dash in command names and dash is not valid Python syntax
name = name.replace("-", "_")
# We also set docstring of this function correctly.
func = partial(_call_click_command, obj)
func.__doc__ = obj.__doc__
all_functions_of_click_commands[name] = func
A full example can be found in binance-api-test-tool source code.
Here is a solution to call a click function with a dictionary of options:
Sometimes, it might be interesting to invoke one command from another
command. This is a pattern that is generally discouraged with Click,
but possible nonetheless. For this, you can use the Context.invoke()
or Context.forward() methods.
They work similarly, but the difference is that Context.invoke()
merely invokes another command with the arguments you provide as a
caller, whereas Context.forward() fills in the arguments from the
current command. Both accept the command as the first argument and
everything else is passed onwards as you would expect.
Example:
cli = click.Group()
#cli.command()
#click.option('--opt1', default=1)
#click.option('--opt2', default=2)
def test(opt1, opt2):
print(opt1)
print(opt2)
#cli.command()
#click.pass_context
def dist(ctx):
args = {"opt1":3, "opt2": 4}
ctx.invoke(test, **args)
if __name__ == "__main__":
dist()
Related
I'm writing a small script to learn Python. The script prints a chess tournament table for N players. It has a simple CLI with a single argument N. Now I'm trying the following approach:
import argparse
def parse_args(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Tournament tables")
parser.add_argument('N', help="number of players (2 at least)", type=int)
args = parser.parse_args(argv)
if args.N < 2:
parser.error("N must be 2 at least")
return args.N
def main(n: int) -> None:
print(F"Here will be the table for {n} players")
if __name__ == '__main__':
main(parse_args())
But this seems to have a flaw. The function main doesn't check n for invalid input (as it's the job of CLI parser). So if somebody calls main directly from another module (a tester for example), he may call it with lets say 0, and the program most likely crashes.
How should I properly handle this issue?
I'm considering several possible ways, but not sure what is the best.
Add a proper value checking and error handling to main. This option looks ugly to me, as it violates the DRY principle and forces main to double the job of CLI.
Just document that main must take only n >= 2, and its behaviour is unpredicted otherwise. Possibly to combine with adding an assertion check to main, like this:
assert n >= 2, "n must be 2 or more"
Perhaps such a function should not be external at all? So the whole chosen idiom is wrong and the script's entry point should be rewritten another way.
???
You could have main do all the checking aind raise ArgumentError if something is amiss. Then catch that exception and forward it to the parser for display. Something along these lines:
import argparse
def run_with_args(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Tournament tables")
parser.add_argument('N', help="number of players (2 at least)", type=int)
args = parser.parse_args(argv)
try:
main(args.N)
except argparse.ArgumentError as ex:
parser.error(str(ex))
def main(n: int) -> None:
if N < 2:
raise argparse.ArgumentError("N must be 2 at least")
print(F"Here will be the table for {n} players")
if __name__ == '__main__':
run_with_args()
If you don't want to expose argparse.ArgumentError to library users of main, you can also create a custom exception type instead of it.
A common way of running argparse when wanting to test functions/CLI is to have the main function take a the sys.argv list and then call parse_args from within main like so:
arg.py
import argparse
def parse_args(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Tournament tables", prog="prog")
parser.add_argument("N", help="number of players (2 at least)", type=int)
args = parser.parse_args(argv)
if args.N < 2:
parser.error("N must be 2 at least")
return args
def main(argv: list[str] | None = None) -> None:
args = parse_args(argv)
print(f"Here will be the table for {args.N} players")
if __name__ == "__main__":
main()
This way a test can call main with a hypothetical CLI:
test_main.py
import pytest
from arg import main
def test_main(capsys):
with pytest.raises(SystemExit):
main(["0"])
out, err = capsys.readouterr()
assert err.splitlines()[-1] == "prog: error: N must be 2 at least"
I’ve been using Pydantic liberally for enforcing data typing at runtime, within my code itself. Your N>=2 is easily enforced with a validator.
It’s a very robust, extremely widely used, library and very fast as it’s more of a data ingestion validator than a type checker.
And you could write the call as follows. How you call main is entirely up to you: direct call, argparse, click…
class ParamChecker(BaseModel):
n : int
#validator('n')
def n2(cls, v):
if v <2:
raise ValueError('must be 2+')
return v
def main(n: int) -> None:
params = ParamChecker(**locals())
Pydantic also gets you informative, if not really end user friendly, error messages.
What I need
I need an ArgumentParser, with a conflict handling scheme, that resolves some registered set of duplicate arguments, but raises on all other arguments.
What I tried
My initial approach (see also the code example at the bottom) was to subclass ArgumentParser, add a _handle_conflict_custom method, and then instantiate the subclass with ArgumentParser(conflict_handler='custom'), thinking that the _get_handler method would pick it up.
The Problem
This raises an error, because the ArgumentParser inherits from _ActionsContainer, which provides the _get_handler and the _handle_conflict_{strategy} methods, and then internally instantiates an _ArgumentGroup (that also inherits from _ActionsContainer), which in turn doesn't know about the newly defined method on ArgumentParser and thus fails to get the custom handler.
Overriding the _get_handler method is not feasible for the same reasons.
I have created a (rudimentary) class diagram illustrating the relationships, and therefore hopefully the problem in subclassing ArgumentParser to achieve what I want.
Motivation
I (think, that I) need this, because I have two scripts, that handle distinct parts of a workflow, and I would like to be able to use those separately as scripts, but also have one script, that imports the methods of both of these scripts, and does everything in one go.
This script should support all the options of the two individual scripts, but I don't want to duplicate the (extensive) argument definitions, so that I would have to make changes in multiple places.
This is easily solved by importing the ArgumentParsers of the (part) scripts and using them as parents, like so combined_parser = ArgumentParser(parents=[arg_parser1, arg_parser2]).
In the scripts I have duplicate options, e.g. for the work directory, so I need to resolve those conflicts.
This could also be done, with conflict_handler='resolve'.
But because there are a lot of possible arguments (which is not up to our team, because we have to maintain compatibility), I also want the script to raise an error if something gets defined that causes a conflict, but hasn't been explicitly allowed to do so, instead of quietly overriding the other flag, potentially causing unwanted behavior.
Other suggestions to achieve these goals (keeping both scripts separate, enabling use of one script that wraps both, avoiding code duplication and raising on unexpected duplicates) are welcome.
Example Code
from argparse import ArgumentParser
class CustomParser(ArgumentParser):
def _handle_conflict_custom(self, action, conflicting_actions):
registered = ['-h', '--help', '-f']
conflicts = conflicting_actions[:]
use_error = False
while conflicts:
option_string, action = conflicts.pop()
if option_string in registered:
continue
else:
use_error = True
break
if use_error:
self._handle_conflict_error(action, conflicting_actions)
else:
self._handle_conflict_resolve(action, conflicting_actions)
if __name__ == '__main__':
ap1 = ArgumentParser()
ap2 = ArgumentParser()
ap1.add_argument('-f') # registered, so should be resolved
ap2.add_argument('-f')
ap1.add_argument('-g') # not registered, so should raise
ap2.add_argument('-g')
# this raises before ever resolving anything, for the stated reasons
ap3 = CustomParser(parents=[ap1, ap2], conflict_handler='custom')
Other questions
I am aware of these similar questions:
python argparse subcommand with dependency and conflict
argparse conflict when used with two connected python3 scripts
Handling argparse conflicts
... and others
But even though some of them provide interesting insights into argparse usage and conflicts, they seem to address issues that are not related to mine.
While I agree that FMc's approach is probably the better one in terms of long term viability, I have found a way to override a custom handler into the ArgumentParser.
The key is to override the _ActionsContainer class which actually defines the handler functions. Then to override the base classes that the ArgumentParser and _ArgumentGroup inherit from.
In the case below, I've simply added a handler that ignores any conflicts, but you could add any custom logic you want.
import argparse
class IgnorantActionsContainer(argparse._ActionsContainer):
def _handle_conflict_ignore(self, action, conflicting_actions):
pass
argparse.ArgumentParser.__bases__ = (argparse._AttributeHolder, IgnorantActionsContainer)
argparse._ArgumentGroup.__bases__ = (IgnorantActionsContainer,)
parser = argparse.ArgumentParser(conflict_handler="ignore")
parser.add_argument("-a", type=int, default=1)
parser.add_argument("-a", type=int, default=2)
parser.add_argument("-a", type=int, default=3)
parser.add_argument("-a", type=int, default=4)
print(parser.parse_args())
Running python custom_conflict_handler.py -h prints:
usage: custom_conflict_handler.py [-h] [-a A] [-a A] [-a A] [-a A]
optional arguments:
-h, --help show this help message and exit
-a A
-a A
-a A
-a A
Running python custom_conflict_handler.py prints:
Namespace(a=1)
Running python custom_conflict_handler.py -a 5 prints:
Namespace(a=5)
For a various reasons -- notably the needs of testing -- I have adopted the
habit of always defining argparse configuration in the form of a data
structure, typically a sequence of dicts. The actual creation of the
ArgumentParser is done in a reusable function that simply builds the parser
from the dicts. This approach has many benefits, especially for more complex
projects.
If each of your scripts were to shift to that model, I would think that you
might be able to detect any configuration conflicts in that function and raise
accordingly, thus avoiding the need to inherit from ArgumentParser and mess
around with understanding its internals.
I'm not certain I understand your conflict-handling needs very well, so the
demo below simply hunts for duplicate options and raises if it sees one, but I
think you should be able to understand the approach and assess whether it might
work for your case. The basic idea is to solve your problem in the realm
of ordinary data structures rather than in the byzantine world of argparse.
import sys
import argparse
from collections import Counter
OPTS_CONFIG1 = (
{
'names': 'path',
'metavar': 'PATH',
},
{
'names': '--nums',
'nargs': '+',
'type': int,
},
{
'names': '--dryrun',
'action': 'store_true',
},
)
OPTS_CONFIG2 = (
{
'names': '--foo',
'metavar': 'FOO',
},
{
'names': '--bar',
'metavar': 'BAR',
},
{
'names': '--dryrun',
'action': 'store_true',
},
)
def main(args):
ap = define_parser(OPTS_CONFIG1, OPTS_CONFIG2)
opts = ap.parse_args(args)
print(opts)
def define_parser(*configs):
# Validation: adjust as needed.
tally = Counter(
nm
for config in configs
for d in config
for nm in d['names'].split()
)
for k, n in tally.items():
if n > 1:
raise Exception(f'Duplicate argument configurations: {k}')
# Define and return parser.
ap = argparse.ArgumentParser()
for config in configs:
for d in config:
kws = dict(d)
xs = kws.pop('names').split()
ap.add_argument(*xs, **kws)
return ap
if __name__ == '__main__':
main(sys.argv[1:])
There is an answer that's marginally less hacky than Hans's approach. You can simply subclass argparse.ActionsContainer and argparse.ArgumentGroup and make sure you inherit from ActionsContainer after argparse.ArgumentParser, this way it'll be later in the MRO and will take precedence. Here's an example:
import argparse
from typing import Iterable, Any
class ArgumentGroup(argparse._ArgumentGroup):
def _handle_conflict_custom(
self,
action: argparse.Action,
conflicting_actions: Iterable[tuple[str, argparse.Action]],
) -> None:
...
class ActionsContainer(argparse._ActionsContainer):
def _handle_conflict_custom(
self,
action: argparse.Action,
conflicting_actions: Iterable[tuple[str, argparse.Action]],
) -> None:
...
def add_argument_group(self, *args: Any, **kwargs: Any) -> ArgumentGroup:
group = ArgumentGroup(self, *args, **kwargs)
self._action_groups.append(group)
return group
class ArgumentParser(argparse.ArgumentParser, ActionsContainer):
...
Based on FMcs approach I have created something a little more elaborate, I know this isn't code review, but feedback is still welcome. Also, maybe it helps someone to see this fleshed out a bit more.
import argparse
from collections import Counter, OrderedDict
from typing import List, Dict, Any
from copy import deepcopy
class OptionConf:
def __init__(self):
self._conf = OrderedDict() # type: Dict[str, List[Dict[str, Any]]]
self._allowed_dupes = list() # type: List[str]
def add_conf(self, command, *conf_args, **conf_kwargs):
if command not in self._conf:
self._conf[command] = []
conf_kwargs['*'] = conf_args
self._conf[command].append(conf_kwargs)
def add_argument(self, *conf_args, **conf_kwargs):
self.add_conf('add_argument', *conf_args, **conf_kwargs)
def register_allowed_duplicate(self, flag):
self._allowed_dupes.append(flag)
def generate_parser(self, **kwargs):
argument_parser = argparse.ArgumentParser(**kwargs)
for command, conf_kwargs_list in self._conf.items():
command_func = getattr(argument_parser, command)
for conf_kwargs in conf_kwargs_list:
list_args = conf_kwargs.pop('*', [])
command_func(*list_args, **conf_kwargs)
conf_kwargs['*'] = list_args
return argument_parser
def _get_add_argument_conf_args(self):
for command, kwargs_list in self._conf.items():
if command != 'add_argument':
continue
return kwargs_list
return []
def resolve_registered(self, other):
if self.__class__ == other.__class__:
conf_args_list = self._get_add_argument_conf_args() # type: List[Dict[str, Any]]
other_conf_args_list = other._get_add_argument_conf_args() # type: List[Dict[str, Any]]
# find all argument names of both parsers
all_names = []
for conf_args in conf_args_list:
all_names += conf_args.get('*', [])
all_other_names = []
for other_conf_args in other_conf_args_list:
all_other_names += other_conf_args.get('*', [])
# check for dupes and throw if appropriate
found_allowed_dupes = []
tally = Counter(all_names + all_other_names)
for name, count in tally.items():
if count > 1 and name not in self._allowed_dupes:
raise Exception(f'Duplicate argument configurations: {name}')
elif count > 1:
found_allowed_dupes.append(name)
# merge them in a new OptionConf, preferring the args of self (AS OPPOSED TO ORIGINAL RESOLVE)
new_opt_conf = OptionConf()
for command, kwargs_list in self._conf.items():
for kwargs in kwargs_list:
list_args = kwargs.get('*', [])
new_opt_conf.add_conf(command, *list_args, **kwargs)
for command, kwargs_list in other._conf.items():
for kwargs in kwargs_list:
# if it's another argument, we remove dupe names
if command == 'add_argument':
all_names = kwargs.pop('*', [])
names = [name for name in all_names if name not in found_allowed_dupes]
# and only add if there are names left
if names:
new_opt_conf.add_argument(*deepcopy(names), **deepcopy(kwargs))
# put names back
kwargs['*'] = all_names
else:
# if not, we just add it
list_args = kwargs.pop('*', [])
new_opt_conf.add_conf(command, *deepcopy(list_args), **deepcopy(kwargs))
# put list args back
kwargs['*'] = list_args
return new_opt_conf
raise NotImplementedError()
if __name__ == '__main__':
opts_conf = OptionConf()
opts_conf.add_argument('pos_arg')
opts_conf.add_argument('-n', '--number', metavar='N', type=int)
opts_conf.add_argument('-i', '--index')
opts_conf.add_argument('-v', '--verbose', action='store_true')
opts_conf2 = OptionConf()
opts_conf2.add_argument('-n', '--number', metavar='N', type=int)
opts_conf2.add_argument('-v', action='store_true')
opts_conf.register_allowed_duplicate('-n')
opts_conf.register_allowed_duplicate('--number')
try:
resolved_opts = opts_conf.resolve_registered(opts_conf2)
except Exception as e:
print(e) # raises on -v
opts_conf.register_allowed_duplicate('-v')
resolved_opts = opts_conf.resolve_registered(opts_conf2)
ap = resolved_opts.generate_parser(description='does it work?')
ap.parse_args(['-h'])
When I launch my code I would like to choose a function (from a set of functions) which will be used. Unfortunately I have many loops and the code is very expensive so the following pseudocode is is highly deprecated:
import argparse
import mystuff
code body in which my_variable gets created
if args.my_index == 1:
def my_func(my_variable):
return my_variable + 1
if args.my_index == 2:
def my_func(my_variable):
return my_variable**2 +1
having used the following command:
$ python3 my_code.py --index 1
I was thinking about promoting the function to an external class module, maybe using the properties of class initialization.
You can register your functions inside a container like a tuple. Then your can retrieve them by index. the .index attribute of your ArgumentParser object is going to be 1 more than the tuple indices:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--index', type=int)
args = parser.parse_args()
my_variable = 10
funcs = (lambda x: x + 1, lambda x: x ** 2 + 1, lambda x: x ** 3 + 1)
if args.index is not None:
print(funcs[args.index - 1](my_variable))
This way when you execute your script using python3 my_code.py --index 1, the .index is 1, so you need to get the first item of the tuple which is args.index - 1.
output: 11
If by any chance your functions follow a specific pattern(like my_variable ** n + 1 here) you can define a generic function that handles it without registering all the functions:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--index', type=int)
args = parser.parse_args()
my_variable = 10
def func(x):
return my_variable ** x + 1
if args.index is not None:
print(func(args.index))
Here's an example how you might use a base class with your functions as classes inheriting from it to automatically build your commandline (using the __subclasses__() method on the base class) and do the calculation, without writing some ridiculous sequence of if statements.
For more info on subclasses see here in the Python documentation.
import argparse
# base class for all your functions to inherit from
class mystuff():
pass
# function classes inherit from mystuff, provide at least a docstring and a calculate() method.
class fun1(mystuff):
'''
Calculate fun1
'''
def calculate(self,value):
return value*2
class fun2(mystuff):
'''
Calculate fun2
'''
def calculate(self,value):
return value*23
if __name__=="__main__":
parser = argparse.ArgumentParser(description="Allow calling functions each defined in their own class")
# find all the classes to be used as commandline arguments
for fun in mystuff.__subclasses__():
# print( f"Creating argument for {fun.__name__}" )
parser.add_argument( f'--{fun.__name__}', default=0, dest='function', action='store_const', const=fun, help=fun.__doc__)
parser.add_argument('variable', type=int, help="Value to pass to calculation")
args = parser.parse_args()
# create an instance and call its calculate function passing the variable value
result = args.function().calculate(args.variable)
print( f"Calling {args.function.__name__} with variable {args.variable} gives result {result}" )
Getting the usage:
fun1.py -h
gives:
usage: fun1.py [-h] [--fun1] [--fun2] variable
Allow calling functions each defined in their own class
positional arguments:
variable Value to pass to calculation
optional arguments:
-h, --help show this help message and exit
--fun1 Calculate fun1
--fun2 Calculate fun2
invoking one of the functions:
fun1.py --fun1 235
gives result:
Calling fun1 with variable 235 gives result 470
I'm sure this could be made more sophisticated, perhaps removing the need for a -- before each function name by using subparsers.
I have a function which is wrapped as a command using click. So it looks like this:
#click.command()
#click.option('-w', '--width', type=int, help="Some helping message", default=0)
[... some other options ...]
def app(width, [... some other option arguments...]):
[... function code...]
I have different use cases for this function. Sometimes, calling it through the command line is fine, but sometime I would also like to call directly the function
from file_name import app
width = 45
app(45, [... other arguments ...])
How can we do that? How can we call a function that has been wrapped as a command using click? I found this related post, but it is not clear to me how to adapt it to my case (i.e., build a Context class from scratch and use it outside of a click command function).
EDIT: I should have mentioned: I cannot (easily) modify the package that contains the function to call. So the solution I am looking for is how to deal with it from the caller side.
You can call a click command function from regular code by reconstructing the command line from parameters. Using your example it could look somthing like this:
call_click_command(app, width, [... other arguments ...])
Code:
def call_click_command(cmd, *args, **kwargs):
""" Wrapper to call a click command
:param cmd: click cli command function to call
:param args: arguments to pass to the function
:param kwargs: keywrod arguments to pass to the function
:return: None
"""
# Get positional arguments from args
arg_values = {c.name: a for a, c in zip(args, cmd.params)}
args_needed = {c.name: c for c in cmd.params
if c.name not in arg_values}
# build and check opts list from kwargs
opts = {a.name: a for a in cmd.params if isinstance(a, click.Option)}
for name in kwargs:
if name in opts:
arg_values[name] = kwargs[name]
else:
if name in args_needed:
arg_values[name] = kwargs[name]
del args_needed[name]
else:
raise click.BadParameter(
"Unknown keyword argument '{}'".format(name))
# check positional arguments list
for arg in (a for a in cmd.params if isinstance(a, click.Argument)):
if arg.name not in arg_values:
raise click.BadParameter("Missing required positional"
"parameter '{}'".format(arg.name))
# build parameter lists
opts_list = sum(
[[o.opts[0], str(arg_values[n])] for n, o in opts.items()], [])
args_list = [str(v) for n, v in arg_values.items() if n not in opts]
# call the command
cmd(opts_list + args_list)
How does this work?
This works because click is a well designed OO framework. The #click.Command object can be introspected to determine what parameters it is expecting. Then a command line can be constructed that will look like the command line that click is expecting.
Test Code:
import click
#click.command()
#click.option('-w', '--width', type=int, default=0)
#click.option('--option2')
#click.argument('argument')
def app(width, option2, argument):
click.echo("params: {} {} {}".format(width, option2, argument))
assert width == 3
assert option2 == '4'
assert argument == 'arg'
width = 3
option2 = 4
argument = 'arg'
if __name__ == "__main__":
commands = (
(width, option2, argument, {}),
(width, option2, dict(argument=argument)),
(width, dict(option2=option2, argument=argument)),
(dict(width=width, option2=option2, argument=argument),),
)
import sys, time
time.sleep(1)
print('Click Version: {}'.format(click.__version__))
print('Python Version: {}'.format(sys.version))
for cmd in commands:
try:
time.sleep(0.1)
print('-----------')
print('> {}'.format(cmd))
time.sleep(0.1)
call_click_command(app, *cmd[:-1], **cmd[-1])
except BaseException as exc:
if str(exc) != '0' and \
not isinstance(exc, (click.ClickException, SystemExit)):
raise
Test Results:
Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> (3, 4, 'arg', {})
params: 3 4 arg
-----------
> (3, 4, {'argument': 'arg'})
params: 3 4 arg
-----------
> (3, {'option2': 4, 'argument': 'arg'})
params: 3 4 arg
-----------
> ({'width': 3, 'option2': 4, 'argument': 'arg'},)
params: 3 4 arg
I tried with Python 3.7 and Click 7 the following code:
import click
#click.command()
#click.option('-w', '--width', type=int, default=0)
#click.option('--option2')
#click.argument('argument')
def app(width, option2, argument):
click.echo("params: {} {} {}".format(width, option2, argument))
assert width == 3
assert option2 == '4'
assert argument == 'arg'
app(["arg", "--option2", "4", "-w", 3])
app(["arg", "-w", 3, "--option2", "4" ])
app(["-w", 3, "--option2", "4", "arg"])
All the app calls are working fine!
This use-case is described in the docs.
Sometimes, it might be interesting to invoke one command from another command. This is a pattern that is generally discouraged with Click, but possible nonetheless. For this, you can use the Context.invoke() or Context.forward() methods.
cli = click.Group()
#cli.command()
#click.option('--count', default=1)
def test(count):
click.echo('Count: %d' % count)
#cli.command()
#click.option('--count', default=1)
#click.pass_context
def dist(ctx, count):
ctx.forward(test)
ctx.invoke(test, count=42)
They work similarly, but the difference is that Context.invoke() merely invokes another command with the arguments you provide as a caller, whereas Context.forward() fills in the arguments from the current command.
If you just want to call the underlying function, you can directly access it as click.Command.callback. Click stores the underlying wrapped Python function as a class member. Note that directly calling the function will bypass all Click validation and any Click context information won't be there.
Here is an example code that iterates all click.Command objects in the current Python module and makes a dictionary of callable functions out from them.
from functools import partial
from inspect import getmembers
import click
all_functions_of_click_commands = {}
def _call_click_command(cmd: click.Command, *args, **kwargs):
result = cmd.callback(*args, **kwargs)
return result
# Pull out all Click commands from the current module
module = sys.modules[__name__]
for name, obj in getmembers(module):
if isinstance(obj, click.Command) and not isinstance(obj, click.Group):
# Create a wrapper Python function that calls click Command.
# Click uses dash in command names and dash is not valid Python syntax
name = name.replace("-", "_")
# We also set docstring of this function correctly.
func = partial(_call_click_command, obj)
func.__doc__ = obj.__doc__
all_functions_of_click_commands[name] = func
A full example can be found in binance-api-test-tool source code.
Here is a solution to call a click function with a dictionary of options:
Sometimes, it might be interesting to invoke one command from another
command. This is a pattern that is generally discouraged with Click,
but possible nonetheless. For this, you can use the Context.invoke()
or Context.forward() methods.
They work similarly, but the difference is that Context.invoke()
merely invokes another command with the arguments you provide as a
caller, whereas Context.forward() fills in the arguments from the
current command. Both accept the command as the first argument and
everything else is passed onwards as you would expect.
Example:
cli = click.Group()
#cli.command()
#click.option('--opt1', default=1)
#click.option('--opt2', default=2)
def test(opt1, opt2):
print(opt1)
print(opt2)
#cli.command()
#click.pass_context
def dist(ctx):
args = {"opt1":3, "opt2": 4}
ctx.invoke(test, **args)
if __name__ == "__main__":
dist()
I have to either store the command line argument in a variable or assign a default value to it.
What i am trying is the below
import sys
Var=sys.argv[1] or "somevalue"
I am getting the error out of index if i don't specify any argument. How to solve this?
Var=sys.argv[1] if len(sys.argv) > 1 else "somevalue"
The builtin argparse module is intended for exactly these sorts of tasks:
import argparse
# Set up argument parser
ap = argparse.ArgumentParser()
# Single positional argument, nargs makes it optional
ap.add_argument("thingy", nargs='?', default="blah")
# Do parsing
a = ap.parse_args()
# Use argument
print a.thingy
Or, if you are stuck with Python 2.6 or earlier, and don't wish to add a requirement on the backported argparse module, you can do similar things manually like so:
import optparse
opter = optparse.OptionParser()
# opter.add_option("-v", "--verbose") etc
opts, args = opter.parse_args()
if len(args) == 0:
var = "somevalue"
elif len(args) == 1:
var = args[0]
else:
opter.error("Only one argument expected, got %d" % len(args))
print var
Good question.
I think the best solution would be to do
try:
var = sys.argv[1]
except IndexError:
var = "somevalue"
Try the following with a command-line-processing template:
def process_command_line(argv):
...
# add your option here
parser.add_option('--var',
default="somevalue",
help="your help text")
def main(argv=None):
settings, args = process_command_line(argv)
...
print settings, args # <- print your settings and args
Running ./your_script.py with the template below and your modifications above prints {'var': 'somevalue'} []
For an example of a command-line-processing template see an example in Code Like a Pythonista: Idiomatic Python (http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html#command-line-processing):
#!/usr/bin/env python
"""
Module docstring.
"""
import sys
import optparse
def process_command_line(argv):
"""
Return a 2-tuple: (settings object, args list).
`argv` is a list of arguments, or `None` for ``sys.argv[1:]``.
"""
if argv is None:
argv = sys.argv[1:]
# initialize the parser object:
parser = optparse.OptionParser(
formatter=optparse.TitledHelpFormatter(width=78),
add_help_option=None)
# define options here:
parser.add_option( # customized description; put --help last
'-h', '--help', action='help',
help='Show this help message and exit.')
settings, args = parser.parse_args(argv)
# check number of arguments, verify values, etc.:
if args:
parser.error('program takes no command-line arguments; '
'"%s" ignored.' % (args,))
# further process settings & args if necessary
return settings, args
def main(argv=None):
settings, args = process_command_line(argv)
# application code here, like:
# run(settings, args)
return 0 # success
if __name__ == '__main__':
status = main()
sys.exit(status)