We Keep Coding

When does argparse not complain about this missing argument?

There is probably an obvious answer to this question, but I've looked at it for a bit without figuring it out. This is some old Python code using argparse. I haven't used argparse recently, so I may have forgotten some nuance.
#test.py
def load_crossval_dataset(args):
schema, samplenum, permuted, search = args.schema, args.samplenum, args.permuted, args.search
print "schema", schema
print "samplenum", samplenum
print "permuted", permuted
print "search", search
import argparse
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
subparsers = parser.add_subparsers()
# create the parser for the "crossval" command
parser_crossval = subparsers.add_parser('crossval', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser_crossval.add_argument('schema', help='name of schema')
parser_crossval.add_argument("-n", "--samplenum", action="store", type=int, dest="samplenum", help="number of samples to do crossvalidation on")
parser_crossval.add_argument("-p", "--permuted", action="store_true", dest="permuted", help="permuted dataset", default=False)
parser_crossval.add_argument("-s", "--search", action="store_true", dest="search", help="model search", default=False)
parser_crossval.set_defaults(func=load_crossval_dataset)
args = parser.parse_args()
args.func(args)
Let us invoke this as:
python test.py
usage: test.py [-h] {crossval} ...
test.py: error: too few arguments
Now as
python test.py crossval -h
usage: test.py crossval [-h] [-n SAMPLENUM] [-p] [-s] schema
positional arguments:
schema name of schema
optional arguments:
-h, --help show this help message and exit
-n SAMPLENUM, --samplenum SAMPLENUM
number of samples to do crossvalidation on (default: None)
-p, --permuted permuted dataset (default: False)
-s, --search model search (default: False)
Now as
python test.py crossval -n 1 -s True
schema True
samplenum 1
permuted False
search True
Question: why does argparse not complain about the missing schema argument, and why does it set it to True?

At a glance, the -s option is boolean - so its presence implies True and it requires no argument. So when you say python test.py crossval -n 1 -s True, the True gets parsed as being the schema argument since the -s switch doesn't require a value.
This much can in fact be gleaned from the usage string in the help text:
usage: test.py crossval [-h] [-n SAMPLENUM] [-p] [-s] schema
The [-s] indicates that it's a nullary option, unlike -n which is listed as [-n SAMPLENUM] since it requires an argument (SAMPLENUM).
Edit:
This behavior is stated in the Python 2.7 Documentation for argparse, which I infer is the version you are using in your example since you are using the statement- rather than function-form of print. To quote section 15.4.3.2:
'store_true' and 'store_false' - These are special cases of 'store_const' using for storing the values True and False respectively. In addition, they create default values of False and True respectively.

The option -s doesn't take an argument (store_const, store_true and store_false actions don't take an argument — this could stand to be clarified in the documentation). So in python test.py crossval -n 1 -s True, the argument True is a positional argument of crossval, not an argument of -s; thus it's the value of schema.
python test.py crossval -n 1 -s correctly complains about the missing argument to test.py crossval.

Passing in multiple options for argparse in Python

I have been looking at argparse documentation but I am still confused how to use it.
I made a python script to get issues from either pmd, checkstyle, or findbugs after a code analysis. Theses issues are also categorized into severities such as major, blocker, and critical.
So I want to be able to pass in two arguments in the form python script.py arg1 arg2 where arg1 would be a combination of p,c,f which stands for pmd, checkstyle, or findbug and arg2 would be a combination of m,c,b which stands for major, critical, and blocker.
So for instance, if I write python script.py pf cb in the terminal, I would get pmd and findbugs issues of critical and blocker severity.
It would be awesome if someone can give me a general structure of how this should go.
Thanks.

Perhaps you'd rather let the user specify the flags more verbose, like this?
python script.py --checker p --checker f --level c --level b
If so, you can use append
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--checker', action='append')
>>> parser.add_argument('--level', action='append')
Then you get a parameter checker and level, both as lists to iterate over.
If you really want to use the combined flags:
for c in arg1:
run_checker(c, arg2)
Assuming that you just pass the severity levels to the checker in some way.

You could try setting boolean flags if each option is present in an argument:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("pcf", help="pmd, checkstyle, or findbug")
parser.add_argument("mcb", help="major, critical, and blocker")
args = parser.parse_args()
# Use boolean flags
pmd = 'p' in args.pcf
checkstyle = 'c' in args.pcf
findbug = 'f' in args.pcf
major = 'm' in args.mcb
critical = 'c' in args.mcb
blocker = 'b' in args.mcb
This would work using python script.py pf cb.
Also, just a helpful hint. If you place the following at the top of your python file and then make it executable with chmod +x script.py you can then call the file directly (using a *NIX operating system):
#!/usr/bin/env python
import argparse
...
Now run with ./script.py pf cb or even put it in your path and call script.py pf cb

https://stackoverflow.com/users/1401034/ewan is a good solution given your specification. But a slight variation gives users more options, and might be clearer.
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--pmd', action='store_true')
parser.add_argument('-c', '--checkstyle', action='store_true')
parser.add_argument('-f', '--findbug', action='store_true')
parser.add_argument('-m', '--major', action='store_true')
parser.add_argument('-d', '--critical', action='store_true')
parser.add_argument('-b', '--blocker', action='store_true')
args = parser.parse_args()
print args
Sample runs:
1840:~/mypy$ python2.7 stack24265375.py -cfmd
Namespace(blocker=False, checkstyle=True, critical=True, findbug=True, major=True, pmd=False)
1841:~/mypy$ python2.7 stack24265375.py -h
usage: stack24265375.py [-h] [-p] [-c] [-f] [-m] [-d] [-b]
optional arguments:
-h, --help show this help message and exit
-p, --pmd
-c, --checkstyle
-f, --findbug
-m, --major
-d, --critical
-b, --blocker
1842:~/mypy$ python2.7 stack24265375.py --critical --major
Namespace(blocker=False, checkstyle=False, critical=True, findbug=False, major=True, pmd=False)
In this case, there are 6 optional, boolean flags. The short, single letter versions can be strung together in any combination. Except the necessary '-', this form can look just like the one using 2 positional arguments. But they can be mixed and matched in any combination. And the long form is self documenting.
The help display might be clearer if these 6 arguments were divided into 2 argument groups.
parser = argparse.ArgumentParser()
group1 = parser.add_argument_group('issue')
group1.add_argument('-p', '--pmd', action='store_true')
group1.add_argument('-c', '--checkstyle', action='store_true')
group1.add_argument('-f', '--findbug', action='store_true')
group2 = parser.add_argument_group('severity')
group2.add_argument('-m', '--major', action='store_true')
group2.add_argument('-d', '--critical', action='store_true')
group2.add_argument('-b', '--blocker', action='store_true')
args = parser.parse_args()
print args
with the help:
usage: stack24265375.py [-h] [-p] [-c] [-f] [-m] [-d] [-b]
optional arguments:
-h, --help show this help message and exit
issue:
-p, --pmd
-c, --checkstyle
-f, --findbug
severity:
-m, --major
-d, --critical
-b, --blocker

Python 2.7 argparse: How to nest optional mutally exclusive arguments properly?

My Program should include the following options, properly parsed by argparse:
purely optional: [-h, --help] and [-v, --version]
mutually exclusive: [-f FILE, --file FILE] and [-u URL, --url URL]
optional if --url was chosen: [-V, --verbose]
required if either --file or --url was chosen: [-F, --format FORMAT]
The desired usage pattern would be:
prog.py [-h] [-v] [-f FILE (-F FORMAT) | -u URL [-V] (-F FORMAT) ]
with the -F requirement applying to both members of the mutually exclusive group.
Not sure if it rather be a positional.
So it should be possible to run:
prog.py -u "http://foo.bar" -V -F csv
and the parser screaming in case i forgot the -F (as he's supposed to).
What i've done so far:
parser = ArgumentParser(decription='foo')
group = parser.add_mutually_exclusive_group()
group.add_argument('-f','--file', nargs=1, type=str, help='')
group.add_argument('-u','--url', nargs=1, type=str, help='')
parser.add_argument('-V','--verbose', action='store_true', default=False, help='')
parser.add_argument('-F','--format', nargs=1, type=str, help='')
Since it has a 'vanilla mode' to run without command line arguments, all arguments must be optional.
How can i implement points 3. and 4. into my code?
EDIT:
I tried -f and -u as subparsers, as described here, but subcommands seem to be treated like positionals and the parser gives me an error: too few arguments if i run it without arguments.

Use of nargs=2 and tuple metavar approximates your goal
parser = argparse.ArgumentParser(prog='PROG')
group = parser.add_mutually_exclusive_group()
group.add_argument('-f','--file', nargs=2, metavar=('FILE','FORMAT'))
group.add_argument('-u','--url', nargs=2, metavar=('URL','FORMAT'))
parser.add_argument('-V','--verbose', action='store_true',help='optional with url')
which produces:
usage: PROG [-h] [-f FILE FORMAT | -u URL FORMAT] [-V]
optional arguments:
-h, --help show this help message and exit
-f FILE FORMAT, --file FILE FORMAT
-u URL FORMAT, --url URL FORMAT
-V, --verbose optional with url
This requires the format along with filename or url, it just doesn't require the -F. As others noted -V can be ignored in the -f case.
I tried -f and -u as subparsers, as described here, but subcommands seem to be treated like positionals and the parser gives me an error: too few arguments if i run it without arguments.
In the latest version(s) subcommands are no longer treated as required positionals. This was, as best I can tell, a side effect of changing the error message to be more informative. Instead of _parse_known_args doing a:
if positionals:
self.error(_('too few arguments'))
it scans _actions to see which are required, and then lists them by name in the error message. This is discussed in http://bugs.python.org/issue9253 . I know this change is in development (3.4), and may also be in 3.3.

These points can enforced in optparse using a callback method when a certain option is present.
However, in argparse these are not available.
You can add a subparser for the url and the file sub-option, and parse these seperatly.
from the help:
Note that the object returned by parse_args() will only contain attributes for
the main parser and the subparser that was selected by the command line
(and not any other subparsers). So in the example above, when the a command
is specified, only the foo and bar attributes are present, and when the b command
is specified, only the foo and baz attributes are present.
But I would just properly document the usage, and just ignore the arguments that are not
applicable.
e.g. let these two command lines behave exactly the same:
prog.py -f FILE -V
prog.py -f FILE

Print program usage example with argparse module

I am trying to learn how to use python's argparse module. Currently my python script is:
parser = argparse.ArgumentParser(description='My first argparse attempt',
add_help=True)
parser.add_argument("-q", action ="store", dest='argument',
help="First argument")
output = parser.parse_args()
And it gives the output as :
usage: test.py [-h] [-q ARGUMENT]
My first argparse attempt
optional arguments:
-h, --help show this help message and exit
-q ARGUMENT First argument
Now, lets suppose I want my -h or --help argument to print a usage example also. Like,
Usage: python test.py -q "First Argument for test.py"
My purpose is to print the above usage example along with the default content of -h argument so that the user can get a basic idea of how to use the test.py python script.
So, is this functionality inbuilt in the argparse module. If no than what is the correct way to approach this problem.

Use parser.epilog to display something after the generated -h text.
parser = argparse.ArgumentParser(
description='My first argparse attempt',
epilog='Example of use')
output = parser.parse_args()
prints:
My first argparse attempt
optional arguments:
-h, --help show this help message and exit
Example of use

In python, how to get subparsers to read in parent parser's argument?

Here is an example code:
import argparse
parser=argparse.ArgumentParser()
parser.add_argument('-main_arg')
subparser=parser.add_subparser()
a=subparser.add_parser('run')
a.add_argument('required_sub_arg')
a.add_argument('arg_a')
b=subparser.add_parser('b')
parser.parse_args()
I want it to read in -main_arg if I enter program run required_sub_arg -main_arg -arg_a
Right now, it doesn't recognize -main_arg as a valid argument.

PSA to recent readers
As this question still has visits in 2018, before doing anything this complex with argparse, please consider using docopt or click instead. It will improve both your sanity and that of anyone who might read or modify your code. Thank you.
Original answer
As is, you have a few issues.
First, parser.parse_args is a method that returns a namespace of parser's arguments, so you should do something like
args = parser.parse_args()
Then args.main_args to get-main_arg from a call like
program -main_arg run required_sub_arg -arg_a
Your issue with main_arg is that you have created a argument to parser named main_arg, and you make a call like
program run required_sub_arg -main_arg -arg_a
that refers to an argument to a named main_arg. Since a doesn't have such an argument, it is invalid.
In order to refer to a parser's argument from one of its subparser, you have to make said subparser inherit the arguments of its parent. This is done with
a=parser.add_subparser('run', parents=[parser])
You have mistaken subparser for child parser. See http://docs.python.org/dev/py3k/library/argparse.html and https://code.google.com/p/argparse/issues/detail?id=54 for more informations.

For anyone else using argparse that arrives here looking for a way to display "common" sub-parser arguments in the "main" help screen, here's one approach:
import argparse
common = argparse.ArgumentParser(add_help=False)
common.add_argument('--shared', action='store_true', help='some shared arg')
parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument('--parent', action='store_true', help='parent only arg')
subparsers = parser.add_subparsers()
run = subparsers.add_parser('run', parents=[common])
run.add_argument('--fast', action='store_true', help='run only arg')
parser.epilog = "--- Arguments common to all sub-parsers ---" \
+ common.format_help().replace(common.format_usage(), '')
args = parser.parse_args()
Main help:
$ program.py -h
usage: program.py [-h] {run} ...
positional arguments:
{run}
optional arguments:
-h, --help show this help message and exit
--parent parent only arg
--- Arguments common to all sub-parsers ---
optional arguments:
--shared some shared arg
run sub-parser help:
$ program.py run -h
usage: program.py run [-h] [--shared]
optional arguments:
-h, --help show this help message and exit
--shared some shared arg
--fast run only arg
To address the actual question, since the accepted answer doesn't run for me, here's some additional information on why it doesn't seem possible to truly share argparse arguments with the same name across both parent and child/sub-parser parsers.
First, the problem with the following code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-main_arg')
subparsers = parser.add_subparsers()
run = subparsers.add_parser('run', parents=[parser])
args = parser.parse_args()
Is that it leads to the following error, because both parent parser and sub-parser run define the -h/--help argument (by default).
Argparse.ArgumentError: argument -h/--help: conflicting option strings: -h, --help
While this error can be avoided by suppressing the -h/--help option (with add_help=False) on either the parent or the child, it's nice to have the help option at both levels.
Another potential way to avoid conflicting help options is to move common arguments to a shared parser, common:
import argparse
common = argparse.ArgumentParser(add_help=False)
common.add_argument('-main_arg', action='store_true')
parser = argparse.ArgumentParser(parents=[common])
subparsers = parser.add_subparsers()
run = subparsers.add_parser('run', parents=[common])
args = parser.parse_args()
print(args)
While this appears to work on the surface, in practice, it doesn't work as intended:
$ program.py run # OK
Namespace(main_arg=False)
$ program.py run -main_arg # OK
Namespace(main_arg=True)
$ program.py -main_arg run # BAD: expected main_arg to be True
Namespace(main_arg=False)
The behavior observed when parsing program.py -main_arg run illustrates a key relationship: a parent argparser and its sub-parsers are independent parsers, where the parent parses all arguments up to the sub-parser "command" positional argument, and then the selected sub-parser parses the remaining arguments in the same Namespace as the parent with no regard for attributes that may have been set by the parent.