We Keep Coding

Python Click: ignore missing argument error if a flag is set

I want to make a simple CLI program that would accept --edit-config option to open its config in an editor. However, I can't figure out how to make Click ignore Missing Argument error if I specify the --edit-config flag only.
import click
#click.group(invoke_without_command=True)
#click.argument("name")
#click.option("--edit-config", is_flag=True)
def cli(name, edit_config):
# If we run it like
# $ myprog --edit-config | Missing Argument: NAME
if edit_config:
click.echo("Editing config")
...
exit(0)
# But this is the main functionality
# $ myprog John
click.echo(f"Hello, {name}")
#cli.command()
def command_a():
...
...
The first command causes Missing Argument error. There can be workarounds, like nargs=-1 or required=False, however, it seems that this makes the help page to either go completely or the name argument to be marked as optional in the help message, which is not true in the majority of use cases.
Is there is a simple way to make Click not raise Missing Argument error if a certain option is specified?

After reading the docs further I've found this section, which provides a way of implementing --edit-config or any other option that completely alters the execution flow.
def edit_config(ctx, param, value):
if not value or ctx.resilient_parsing:
return
click.echo("Editing config")
ctx.exit()
#click.group(invoke_without_command=True)
#click.argument("name")
#click.option("--edit-config", is_flag=True, callback=edit_config,
is_eager=True, expose_value=False)
def cli(name):
print(f"Hello, {name}!")
#cli.command()
def howareyou():
print(f"How are you?")
$ myapp John
Hello, John!
$ myapp John howareyou
Hello, John!
How are you?
$ myapp --edit-config
Editing config
$ myapp --help
Usage: myapp [OPTIONS] NAME COMMAND [ARGS]...
Options:
--edit-config
--help Show this message and exit.
Commands:
howareyou

How do you define control flow in a click CLI in Python based on a single option and pass the rest to another command?

This question is about the Python click package and relates to control flow based on arguments passed to the CLI.
I'm trying to build a master CLI to sit at the top directory of my repo, which will control many different modes of operation from one central place. When you invoke this command, it will take perhaps n options, where the first will determine which module to invoke, and then pass along the n-1 args to another module. But I want each module's commands and options to be defined within its respective module, not the main CLI controller, as I'm trying to keep the main controller simple and each module nicely abstracted away.
Here is a simple example of what I want:
import click
#click.command()
#click.option('--foo-arg', default='asdf')
def foo(foo_arg):
click.echo(f"Hello from foo with arg {foo_arg}")
#click.command()
#click.option('--bar-arg', default='zxcv')
def bar(bar_arg):
click.echo(f"Hello from bar with arg {bar_arg}")
#click.command()
#click.option('--mode', type=click.Choice(["foo", "bar"]))
def cli(mode):
if mode == "foo":
foo()
if mode == "bar":
bar()
if __name__ == '__main__':
cli()
In this example, foo() and bar() should be thought of as modules that are buried within the repo and which may also require a large number of CLI options, which I don't want to overload the main cli.py with. I feel like foo()'s CLI options should live within foo for contextual reasons. The following is how I want it to work.
Example 1:
python -m cli --mode foo --foo-arg asdf
should produce
Hello from foo with arg asdf
Example 2:
python -m cli --mode bar --bar-arg zxcv
should produce
Hello from bar with arg zxcv
Example 3:
python -m cli --mode foo --bar-arg qwer
should fail since foo() doesn't have the --bar-arg option.
Disclaimer: I know that I could register foo and bar as separate commands (invoked via python -m cli foo --foo-arg asd, i.e. with foo instead of --foo). However, due to a reason beyond the scope of this question, I need foo or bar to be specified by the --mode option identifier. This is a limitation of a tool that interacts with my app, which is unfortunately outside my control.
Is there a way to parse the args and make control flow possible based on a subset of args and then pass the remaining ones to a subsequent module, while not defining every module's options as decorators on def cli()?

Using an option to invoke a subcommand can be achieved by using click.MultiCommand and a custom parse_args() method like:
Custom Class
def mode_opts_cmds(mode_opt_name, namespace=None):
class RemoteCommandsAsModeOpts(click.MultiCommand):
def __init__(self, *args, **kwargs):
super(RemoteCommandsAsModeOpts, self).__init__(*args, **kwargs)
self.mode_opt_name = '--{}'.format(mode_opt_name)
opt = next(p for p in self.params if p.name == mode_opt_name)
assert isinstance(opt.type, click.Choice)
choices = set(opt.type.choices)
self.commands = {k: v for k, v in (
namespace or globals()).items() if k in choices}
for command in self.commands.values():
assert isinstance(command, click.Command)
def parse_args(self, ctx, args):
try:
args.remove(self.mode_opt_name)
except ValueError:
pass
super(RemoteCommandsAsModeOpts, self).parse_args(ctx, args)
def list_commands(self, ctx):
return sorted(self.commands)
def get_command(self, ctx, name):
return self.commands[name]
return RemoteCommandsAsModeOpts
Using the Custom Class
To make use of the custom class, invoke the mode_opts_cmds() function to create the custom class and then use the cls parameter to pass that class to the click.group() decorator.
#click.group(cls=mode_opts_cmds('mode'))
#click.option('--mode', type=click.Choice(["foo", "bar"]))
def cli(mode):
"""My wonderful cli"""
How does this work?
This works because click is a well designed OO framework. The #click.group() decorator usually instantiates a click.Group object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Group in our own class and over ride desired methods.
In this case we over ride click.Group.parse_args() so that when the command line is parsed we simply remove --mode and then the command line parses just like a normal sub-command.
To make this a library function
If the custom class is going to reside in a library, then instead of using the default globals() from the library file, the namespace will need to be passed in. The custom class creator method would then need to be called with something like:
#click.group(cls=mode_opts_cmds('mode', namespace=globals()))
Test Code
import click
#click.command()
#click.option('--foo-arg', default='asdf')
def foo(foo_arg):
"""the foo command is ok"""
click.echo(f"Hello from foo with arg {foo_arg}")
#click.command()
#click.option('--bar-arg', default='zxcv')
def bar(bar_arg):
"""bar is my favorite"""
click.echo(f"Hello from bar with arg {bar_arg}")
#click.group(cls=mode_opts_cmds('mode'))
#click.option('--mode', type=click.Choice(["foo", "bar"]))
def cli(mode):
"""My wonderful cli command"""
if __name__ == "__main__":
commands = (
'--mode foo --foo-arg asdf',
'--mode bar --bar-arg zxcv',
'--mode foo --bar-arg qwer',
'--mode foo --help',
'',
)
import sys, time
time.sleep(1)
print('Click Version: {}'.format(click.__version__))
print('Python Version: {}'.format(sys.version))
for command in commands:
try:
time.sleep(0.1)
print('-----------')
print('> ' + command)
time.sleep(0.1)
cli(command.split())
except BaseException as exc:
if str(exc) != '0' and \
not isinstance(exc,
(click.ClickException, SystemExit)):
raise
Test Results
Click Version: 7.0
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> --mode foo --foo-arg asdf
Hello from foo with arg asdf
-----------
> --mode bar --bar-arg zxcv
Hello from bar with arg zxcv
-----------
> --mode foo --bar-arg qwer
Usage: test.py foo [OPTIONS]
Try "test.py foo --help" for help.
Error: no such option: --bar-arg
-----------
> --mode foo --help
Usage: test.py foo [OPTIONS]
the foo command is ok
Options:
--foo-arg TEXT
--help Show this message and exit.
-----------
>
Usage: test.py [OPTIONS] COMMAND [ARGS]...
My wonderful cli command
Options:
--mode [foo|bar]
--help Show this message and exit.
Commands:
bar bar is my favorite
foo the foo command is ok

You could use Callbacks for validations: https://click.palletsprojects.com/en/7.x/options/#callbacks-for-validation
Also, use multi chaining commands, so each argument will be passed to the command it needs.
https://click.palletsprojects.com/en/7.x/commands/#multi-command-chaining

Call python function based on command line argument

I have a script with several functions:
def a():
pass
def b():
pass
def c():
pass
Which by design will be invoked depending on cmd line argument. I can create several if statements which will evaluate which function should run:
if args.function == "a":
a()
elif args.function == "b":
b()
elif args.function == "c":
c()
But is there a better way to do this?

You could make a dictionary like so
d = {"a" : a,
"b" : b}
and then dispatch
d[args.function]()

Perhaps you are looking for a library like click? It lets you easily add command-line subcommands with a decorator.
import click
#click.group()
def cli():
pass
#cli.command()
def a():
print("I am a")
#cli.command()
def b():
print("Je suis b")
if __name__ == '__main__':
cli()
Sample output:
bash$ ./ick.py --help
Usage: ick.py [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
a
b
bash$ ./ick.py a
I am a

Try using eval
eval(function_name_passed_as_argument + "()")
def a():
pass
def b():
pass
eval(args.function + "()")
This doesn't require the use of if-else logic. Function name passed as argument will be executed directly.

You make a dictionary as already pointed out, but how are you going to handle a bad input? I would create a default method and use the dict.get method:
def fallback():
print('That command does not exist')
# add any code you want to be run for
# a bad input here...
functions = {
'a': a,
'b': b
}
Then call the function by retrieving it:
functions.get(args.function.lower(), fallback)()

Python has several built-in functions that we can utilize for instance Argparse, this method pretty common in python for command line programmable application.
The basics:
import argparse
parser = argparse.ArgumentParser()
parser.parse_args()
By this method, you can have something like this:
$ python3 prog.py -v
verbosity turned on
$ python3 prog.py --help
usage: prog.py [-h] [-v]
optional arguments:
-h, --help show this help message and exit
-v, --verbose increase output verbosity

Optional argument in command with click

I am trying to accomplish something not very standard for CLI parsing with Click and it only works partially:
main CLI has multiple sub-commands (in sample below 'show' and 'check')
both those commands might have optional argument, but the argument is preceding them not following
I decided to handle that argument in the group "above" it and pass the value in the context
Sample:
import click
#click.group()
#click.argument('hostname', required=False)
#click.pass_context
def cli(ctx, hostname=None):
""""""
ctx.obj = hostname
click.echo("cli: hostname={}".format(hostname))
#cli.command()
#click.pass_obj
def check(hostname):
click.echo("check: hostname={}".format(hostname))
#cli.command()
#click.pass_obj
def show(hostname):
click.echo("check: hostname={}".format(hostname))
if __name__ == '__main__':
cli()
The part WITH the hostname works:
> pipenv run python cli.py localhost check
cli: hostname=localhost
check: hostname=localhost
> pipenv run python cli.py localhost show
cli: hostname=localhost
check: hostname=localhost
But the part WITHOUT the hostname DOES NOT:
> pipenv run python cli.py show
Usage: cli.py [OPTIONS] [HOSTNAME] COMMAND [ARGS]...
Error: Missing command.
Anybody has an idea about the direction I should start looking into?

This can be done by over riding the click.Group argument parser like:
Custom Class:
class MyGroup(click.Group):
def parse_args(self, ctx, args):
if args[0] in self.commands:
if len(args) == 1 or args[1] not in self.commands:
args.insert(0, '')
super(MyGroup, self).parse_args(ctx, args)
Using Custom Class:
Then to use the custom group, pass it as the cls argument to the group decorator like:
#click.group(cls=MyGroup)
#click.argument('hostname', required=False)
#click.pass_context
def cli(ctx, hostname=None):
....
How?
This works because click is a well designed OO framework. The #click.group() decorator usually instantiates a click.Group object but allows this behavior to be over ridden with the cls parameter. So it is a relatively easy matter to inherit from click.Group in our own class and over ride desired methods.
In this case we over ride click.Group.parse_args() and if the first parameter matches a command and the second parameter does not, then we insert an empty string as the first parameter. This puts everything back where the parser expects it to be.

Passing command line arguments in python by pytest

I am able to pass command line arguments when running
python <filename>.py arg1
But when am trying to pass the command line arguments for running pytest it fails and gives error as below. Can you please advise.
pytest <filename>.py arg1
ERROR: file not found: arg1
EDIT:
For example am thinking of using it this way assuming I have passed an argument and am reading it via sys.argv:
import sys
arg = sys.argv[3]
def f():
return 3
def test_function():
assert f() == arg

Your pytest <filename>.py arg1 command is trying to call pytest on two modules <filename>.py and arg1 , But there is no module arg1.
If you want to pass some argument before running pytest then run the pytest from a python script after extracting your variable.
As others suggested though you would probably want to parameterize your tests in some other way, Try:Parameterized pytest.
# run.py
import pytest
import sys
def main():
# extract your arg here
print('Extracted arg is ==> %s' % sys.argv[2])
pytest.main([sys.argv[1]])
if __name__ == '__main__':
main()
call this using python run.py filename.py arg1

Here's the method I just cooked up from reading the parameterized pytest docs and hacking for a while... I don't know how stable or good it is going to be overall since I just got it working.
I did however check that HTML coverage generation works with this method.
add a file to your test directory for configuring the command-line args you want to pass:
tests\conftest.py
# this is just so we can pass --server and --port from the pytest command-line
def pytest_addoption(parser):
''' attaches optional cmd-line args to the pytest machinery '''
parser.addoption("--server", action="append", default=[], help="real server hostname/IP")
parser.addoption("--port", action="append", default=[], help="real server port number")
and then add a test file, with this special pytest_generate_tests function which is called when collecting a test function
tests\test_junk.py
def pytest_generate_tests(metafunc):
''' just to attach the cmd-line args to a test-class that needs them '''
server_from_cmd_line = metafunc.config.getoption("server")
port_from_cmd_line = metafunc.config.getoption("port")
print('command line passed for --server ({})'.format(server_from_cmd_line))
print('command line passed for --port ({})'.format(port_from_cmd_line))
# check if this function is in a test-class that needs the cmd-line args
if server_from_cmd_line and port_from_cmd_line and hasattr(metafunc.cls, 'real_server'):
# now set the cmd-line args to the test class
metafunc.cls.real_server = server_from_cmd_line[0]
metafunc.cls.real_port = int(port_from_cmd_line[0])
class TestServerCode(object):
''' test-class that might benefit from optional cmd-line args '''
real_server=None
real_port = None
def test_valid_string(self):
assert self.real_server!=None
assert self.real_port!=None
def test_other(self):
from mypackage import my_server_code
if self.real_server != None:
assert "couldn\'t find host" not in my_server_code.version(self.real_server, self.real_port)
then run (with HTML coverage, for example) with:
pytest tests\test_junk.py --server="abc" --port=123 --cov-report html --cov=mypackage

It seems monkeypatch also works.
Example:
import sys
def test_example(monkeypatch):
monkeypatch.setattr(sys, 'argv', ['/path/to/binary', 'opt1', '...'])
assert f() == '...'
def test_another():
# sys.argv is not modified here
assert f() != '...'