We Keep Coding

Python: How to make command line arguments available for objects?

I am relatively new to Python and I am trying to solve the following problem at the moment. I am writing a script which parses command line arguments with argparse. Besides some other things I have a verbosity flag, which I want to make available for some objects. Currently I am passing it to the constructor and I am wondering if there is a more elegant way for doing this.
class MyClass(object):
def __init__(self, verbosity=False):
self.verbosity = verbosity
def do_something(self):
if self.verbosity:
print("Doing something ...")
.
import argparse
import myclass
def main():
parser = argparse.ArgumentParser(description='Testing.')
parser.add_argument('-v', '--verbosity', action='store_true',
help="Produce verbose output")
args = parser.parse_args()
object = myclass.MyClass(verbosity=args.verbosity)
object.do_something()
if __name__ == "__main__":
main()
Any help would really be appreciated.

I would say what you're doing is the cleanest approach, at least in this simple case. You could pass the args namespace that argparse gives you to the MyClass initializer, but then you'll need to construct a similar object if you want to run tests on your class, or to use it as a library from another script.
If you want the same command line options to be available from multiple classes or scripts, you could create a config.py that looks something like this:
import argparse
__all__ = ['options']
parser = argparse.ArgumentParser(description='Testing.')
parser.add_argument('-v', '--verbosity', action='store_true',
help="Produce verbose output")
args = parser.parse_args()
options = vars(args)
Then from your script you can do from config import options and get the parsed argument from options['verbosity']. You could also change the value of options - for something like:
if self.too_much_output():
options['verbosity'] = False
and have that change be available to any code that's using options.
(The idea to use vars(args) comes from this answer to another argparse-related question.)

Unittest with command-line arguments

From what I understand from another SO post, to unittest a script that takes command line arguments through argparse, I should do something like the code below, giving sys.argv[0] as arg.
import unittest
import match_loc
class test_method_main(unittest.TestCase):
loc = match_loc.main()
self.assertEqual(loc, [4])
if __name__ == '__main__':
sys.argv[1] = 'aaaac'
sys.argv[2] = 'ac'
unittest.main(sys.argv[0])
This returns the error:
usage: test_match_loc.py [-h] text patterns [patterns ...]
test_match_loc.py: error: the following arguments are required: text, patterns
I would like to understand what is going on here deeper. I understand
if __name__ == '__main__':
main()
says that if this is being executed by the 'main', highest level, default interpreter, to just automatically run the 'main' method. I'm assuming
if __name__ == '__main__':
unittest.main()
just happens to be the way you say this for running unittest scripts.
I understand when any script is run, it automatically has an argv object, a vector collecting all the items on the command line.
But I do not understand what unittest.main(sys.arg[0]) would do. What does 'unittest.main' do with arguments? How can I pre-set the values of sys.argv - doesn't it automatically reset every time you run a script? Furthermore, where does this object 'sys.argv' exist, if outside of any script? Finally, what is the correct way to implement tests of command-line arguments?
I am sorry if my questions are vague and misguided. I would like to understand all the components relevant here so I can actually understand what I am doing.
Thank you very much.

Just by playing around with a pair of simple files, I find that modifying sys.argv in the body of the caller module affects the sys.argv that the imported module sees:
import sys
sys.argv[1] = 'aaaac'
sys.argv[2] = 'ac'
class test_method_main(unittest.TestCase):
...
But modifying sys.argv in the main block as you do, does not show up in the imported one. We could dig into the documentation (and code) to see exactly why, but I think it's enough to just identify what works.
Here's what I reconstructed from your previous question of the imported module - with a few diagnostic prints
import argparse
import sys
def main():
print(sys.argv)
parser = argparse.ArgumentParser(
description='Takes a series of patterns as fasta files'
' or strings and a text as fasta file or string and'
' returns the match locations by constructing a trie.')
parser.add_argument('text')
parser.add_argument('patterns', nargs='+')
args = parser.parse_args()
print(args)
return 1
You could also test a parser with your own list of strings, recognising that parse_args uses sys.argv[1:] if its argument is missing or None:
def main(argv=None):
print(argv)
...
args = parser.parse_args(argv)
print(args)
return 1
loc = match_loc.main(['abc','ab']) # and in the caller
Even though I was able to construct a working test case, you really should have given enough information that I didn't need to guess or dig around.

Is there a way to add an already created parser as a subparser in argparse?

Normally, to add a subparser in argparse you have to do:
parser = ArgumentParser()
subparsers = parser.add_subparser()
subparser = subparsers.add_parser()
The problem I'm having is I'm trying to add another command line script, with its own parser, as a subcommand of my main script. Is there an easy way to do this?
EDIT: To clarify, I have a file script.py that looks something like this:
def initparser():
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar')
return parser
def func(args):
#args is a Namespace, this function does stuff with it
if __name__ == '__main__':
initparser().parse_args()
So I can run this like:
python script.py --foo --bar
I'm trying to write a module app.py that's a command line interface with several subcommands, so i can run something like:
python app.py script --foo --bar
Rather than copy and pasting all of the initparser() logic over to app.py, I'd like to be able to directly use the parser i create from initparser() as a sub-parser. Is this possible?

You could use the parents parameter
p=argparse.ArgumentParser()
s=p.add_subparsers()
ss=s.add_parser('script',parents=[initparser()],add_help=False)
p.parse_args('script --foo sst'.split())
ss is a parser that shares all the arguments defined for initparser. The add_help=False is needed on either ss or initparser so -h is not defined twice.

You might want to take a look at the shlex module as it sounds to me like you're trying to hack the ArgumentParser to do something that it wasn't actually intended to do.
Having said that, it's a little difficult to figure out a good answer without examples of what it is, exactly, that you're trying to parse.

I think your problem can be addressed by a declarative wrapper for argparse. The one I wrote is called Argh. It helps with separating definition of commands (with all arguments-related stuff) from assembling (including subparsers) and dispatching.

This is a way old question, but I wanted to throw out another alternative. And that is to think in terms of inversion of control. By this I mean the root ArgumentParser would manage the creation of the subparsers:
# root_argparser.py
from argparse import ArgumentParser, Namespace
__ARG_PARSER = ArgumentParser('My Script')
__SUBPARSERS = __ARG_PARSER.add_subparsers(dest='subcommand')
__SUBPARSERS.required = True
def get_subparser(name: str, **kwargs) -> ArgumentParser:
return __SUBPARSERS.add_parser(name, **kwargs)
def parse_args(**kwargs) -> Namespace:
return __ARG_PARSER.parse_args(**kwargs)
# my_script.py
from argparse import ArgumentParser
from root_argparse import get_subparser
__ARG_PARSER = get_subparser('script')
__ARG_PARSER.add_argument('--foo')
__ARG_PARSER.add_argument('--bar')
def do_stuff(...):
...
# main.py
from root_argparse import parse_args
import my_script
if __name__ == '__main__':
args = parse_args()
# do stuff with args
Seems to work okay from some quick testing I did.

Python: argument parser that handles global options to sub-commands properly

argparse fails at dealing with sub-commands receiving global options:
import argparse
p = argparse.ArgumentParser()
p.add_argument('--arg', action='store_true')
s = p.add_subparsers()
s.add_parser('test')
will have p.parse_args('--arg test'.split()) work,
but fails on p.parse_args('test --arg'.split()).
Anyone aware of a python argument parser that handles global options to sub-commands properly?

You can easily add this argument to both parsers (main parser and subcommand parser):
import argparse
main = argparse.ArgumentParser()
subparser = main.add_subparsers().add_parser('test')
for p in [main,subparser]:
p.add_argument('--arg', action='store_true')
print main.parse_args('--arg test'.split()).arg
print main.parse_args('test --arg'.split()).arg
Edit: As #hpaulj pointed in comment, there is also parents argument which you can pass to ArgumentParser constructor or to add_parser method. You can list in this value parsers which are bases for new one.
import argparse
base = argparse.ArgumentParser(add_help=False)
base.add_argument('--arg', action='store_true')
main = argparse.ArgumentParser(parents=[base])
subparser = main.add_subparsers().add_parser('test', parents=[base])
print main.parse_args('--arg test'.split()).arg
print main.parse_args('test --arg'.split()).arg
More examples/docs:
looking for best way of giving command line arguments in python, where some params are req for some option and some params are req for other options
Python argparse - Add argument to multiple subparsers (I'm not sure if this question is not overlaping with this one too much)
http://docs.python.org/dev/library/argparse.html#parents

Give docopt a try:
>>> from docopt import docopt
>>> usage = """
... usage: prog.py command [--test]
... prog.py another [--test]
...
... --test Perform the test."""
>>> docopt(usage, argv='command --test')
{'--test': True,
'another': False,
'command': True}
>>> docopt(usage, argv='--test command')
{'--test': True,
'another': False,
'command': True}

There's a ton of argument-parsing libs in the Python world. Here are a few that I've seen, all of which should be able to handle address the problem you're trying to solve (based on my fuzzy recollection of them when I played with them last):
opster—I think this is what mercurial uses, IIRC
docopt—This one is new, but uses an interesting approach
cliff—This is a relatively new project by Doug Hellmann (PSF member, virtualenvwrapper author, general hacker extraordinaire) is a bit more than just an argument parser, but is designed from the ground up to handle multi-level commands
clint—Another project that aims to be "argument parsing and more", this one by Kenneth Reitz (of Requests fame).

Here's a dirty workaround --
import argparse
p = argparse.ArgumentParser()
p.add_argument('--arg', action='store_true')
s = p.add_subparsers()
s.add_parser('test')
def my_parse_args(ss):
#parse the info the subparser knows about; don't issue an error on unknown stuff
namespace,leftover=p.parse_known_args(ss)
#reparse the unknown as global options and add it to the namespace.
if(leftover):
s.add_parser('null',add_help=False)
p.parse_args(leftover+['null'],namespace=namespace)
return namespace
#print my_parse_args('-h'.split()) #This works too, but causes the script to stop.
print my_parse_args('--arg test'.split())
print my_parse_args('test --arg'.split())
This works -- And you could modify it pretty easily to work with sys.argv (just remove the split string "ss"). You could even subclass argparse.ArgumentParser and replace the parse_args method with my_parse_args and then you'd never know the difference -- Although subclassing to replace a single method seems overkill to me.
I think however, that this is a lit bit of a non-standard way to use subparsers. In general, global options are expected to come before subparser options, not after.

The parser has a specific syntax: command <global options> subcommand <subcommand ptions>, you are trying to feed the subcommand with an option and but you didn't define one.

Specifying default filenames with argparse, but not opening them on --help?

Let's say I have a script that does some work on a file. It takes this file's name on the command line, but if it's not provided, it defaults to a known filename (content.txt, say). With python's argparse, I use the following:
parser = argparse.ArgumentParser(description='my illustrative example')
parser.add_argument('--content', metavar='file',
default='content.txt', type=argparse.FileType('r'),
help='file to process (defaults to content.txt)')
args = parser.parse_args()
# do some work on args.content, which is a file-like object
This works great. The only problem is that if I run python myscript --help, I get an ArgumentError if the file isn't there (which I guess makes sense), and the help text is not shown. I'd rather it not try to open the file if the user just wants --help. Is there any way to do this? I know I could make the argument a string and take care of opening the file myself later (and I've been doing that), but it would be convenient to have argparse take care of it.

You could subclass argparse.FileType:
import argparse
import warnings
class ForgivingFileType(argparse.FileType):
def __call__(self, string):
try:
super(ForgivingFileType,self).__call__(string)
except IOError as err:
warnings.warn(err)
parser = argparse.ArgumentParser(description='my illustrative example')
parser.add_argument('--content', metavar='file',
default='content.txt', type=ForgivingFileType('r'),
help='file to process (defaults to content.txt)')
args = parser.parse_args()
This works without having to touch private methods like ArgumentParser._parse_known_args.

Looking at the argparse code, I see:
ArgumentParser.parse_args calls parse_known_args and makes sure that there isn't any pending argument to be parsed.
ArgumentParser.parse_known_args sets default values and calls ArgumentParser._parse_known_args
Hence, the workaround would be to use ArgumentParser._parse_known_args directly to detect -h and, after that, use ArgumentParser.parse_args as usual.
import sys, argparse
parser = argparse.ArgumentParser(description='my illustrative example', argument_default=argparse.SUPPRESS)
parser.add_argument('--content', metavar='file',
default='content.txt', type=argparse.FileType('r'),
help='file to process (defaults to content.txt)')
parser._parse_known_args(sys.argv[1:], argparse.Namespace())
args = parser.parse_args()
Note that ArgumentParser._parse_known_args needs a couple of parameters: the arguments from the command line and the namespace.
Of course, I wouldn't recommend this approach since it takes advantage of the internal argparse implementation and that might change in the future. However, I don't find it too messy, so you still might want to use it if you think maintenance risks pay off.

Use stdin as default:
parser.add_argument('file', default='-', nargs='?', type=argparse.FileType('r'))

Perhaps you could define your own type or action in the add_argument call that checks if the file exists, and returns a file handle if it does and None (or something else) otherwise.
This would require you to write some code of yourself as well though, but if the default value can not always be used you probably have to do some checking sooner or later. Like Manny D argues you might want to reconsider your default value.