We Keep Coding

Python argparse: Too few arguments

I'm trying to use the argparse library in Python. I want to have the user do something like:
python my_script.py csv_name.csv [--dryrun]
where --dryrun is an optional parameter.
I then have the user enter an API key and secret key. When I run my code, I get past entering the API and secret keys and then I get:
usage: my_script.py [-h] csv dryrun
salesforceImporter.py: error: too few arguments
Here's my code:
def main():
api_key = getpass.getpass(prompt='Enter API Key: ')
secret_key = getpass.getpass(prompt='Enter Secret Key: ')
parser = argparse.ArgumentParser()
parser.add_argument("csv")
parser.add_argument("dryrun")
args = parser.parse_args()
validate_csv_name(args.csv)
is_dry_run = args.dryrun == '--dryrun'
Any idea where I'm going wrong?
Thanks!

When you use the following syntax:
parser.add_argument("csv")
parser.add_argument("dryrun")
You're adding these as positional -- required -- arguments. Only arguments with a leading dash or two are optional.
See the docs here:
The add_argument() method must know whether an optional argument, like -f or --foo, or a positional argument, like a list of filenames, is expected. The first arguments passed to add_argument() must therefore be either a series of flags, or a simple argument name. For example, an optional argument could be created like:
>>> parser.add_argument('-f', '--foo')

To add an optional --dry-run argument, you may use the following snippet:
parser.add_argument('--dry-run', action='store_true')
Calling your script using python my_script.py csv_name.csv --dry-run will result to args.dry_run being True. Not putting the option will result to it being False

arparse python argument from directory

I have a file structure like this:
project/
main_prog.py
tools/
script.py
md_script/
__init__.py
md_script.py
I search in tools for local python modules. In this example it's md_script. And i want to use it as positional argument like install in my code, but when i use it, I'v got an error:
./jsh.py md_script
usage: jsh.py [-h] {install,call,list,log,restore} ... [md_script]
jsh.py: error: invalid choice: 'md_script' (choose from 'install', 'call', 'list', 'log', 'restore')
python3.4 on ubuntu14.10
Here is my code:
parser = argparse.ArgumentParser(prog='jsh.py',
description='Some help.', epilog='Example of usage: some help')
subparsers = parser.add_subparsers()
parser_install = subparsers.add_parser('install', help = 'Install new project.')
parser_install.add_argument('install', nargs='?', help = 'Name of project to be installed')
if os.path.isdir(full/path/to/tools/):
name_arg = next(os.walk(full/path/to/tools))[1]
tools_arg = parser.add_argument_group('Tools', 'Modules from tools')
for element in name_arg:
tools_arg.add_argument(element, nargs='?', help='md_script description')
args = parser.parse_args()
try:
if not len(sys.argv) > 1:
parser.print_help()
elif 'install' in args:
do_some_stuff
elif element in args:
do_some_md_script_stuff
else:
parser.print_help()

The usage line shows what's wrong:
usage: jsh.py [-h] {install,call,list,log,restore} ... [md_script]
You need to use something like
jsh.py install md_script
You specified subparsers, so you have to give it a subparser name.
From the usage it also looks like you created other subparsers, call, list, etc that you don't show in the code.
You also define positional arguments after creating subparser. That's where the [md_script] comes from. Be careful about making a lot of nargs='?' positionals (including the argument for the install subparser). This could make things confusing for your users. In fact it seems to confusing you. Remember that subparser is in effect a positional argument (one that requires 1 string).
I'd suggest experimenting with a simplier parser before creating one this complicated.
So from your comments and examples I see that you goal is let the user name a module, so your script can invoke it in some way or other. For that populating the subparsers with these names makes sense.
I wonder why you also create an optional positional argument with the same name:
module_pars = subparsers.add_parser(element, help = 'Modules from tools')
module_pars.add_argument(element, nargs='?', help=element+' description')
Is that because you are using the presence of the attribute as evidence that this subparser was invoked?
elif element in args:
do_some_md_script_stuff
The argparse documentation has a couple of other ideas.
One particularly effective way of handling sub-commands is to combine the use of the add_subparsers() method with calls to set_defaults() so that each subparser knows which Python function it should execute.
and
However, if it is necessary to check the name of the subparser that was invoked, the dest keyword argument to the add_subparsers() call will work:
These avoid the messiness of a '?' positional argument, freeing you to use subparser arguments for real information.
subparsers = parser.add_subparsers(dest='module')
....
for element in name_arg:
# module_pars = 'parser_'+element # this does nothing
module_pars = subparsers.add_parser(element, help = 'Modules from tools')
module_pars.set_defaults(func = do_some_md_script_stuff)
# or module_pars.set_default(element='I am here')
module_pars.add_argument('real_argument')
Now you can check:
if args.module='md_script':
do_some_md_script_stuff(args)
or
if hasattr(args, 'func'):
func(args)
With the alternative set_defaults, your original test should still work:
if element in args:
do_some_md_script_stuff

I did it like this. It's exactly what I want to.
if os.path.isdir(TOOLS_PATH):
name_arg = next(os.walk(TOOLS_PATH))[1]
for element in name_arg:
module_pars = 'parser_'+element
module_pars = subparsers.add_parser(element, help = 'Modules from tools')
module_pars.add_argument(element, nargs='?', help=element+' description')
I didn't test it, because i dont have a test module, but ./jsh.py md_script goes into elif element in args: print('md_script') and print string. So it looks like it works.
Thanks for all replies.
Edit: I tested it. In add_argument i must change nargs='?' for nargs='*' to catch more than one argument.
And to catch arguments from command line I used this:
elif args:
for element in name_arg:
if element in args:
element_arg = sys.argv[2:]
done_cmd,msg = opt_exec_module(element,*element_arg)
my_logger(done_cmd,msg)
Not very elegant but it works.

Override the positional and optional arguments with another argument in command line (argparse python module)

I am using argparser to parse the command line arguments.
Now, I have something like
./script.py 1112323 0 --salary 100000 -- age 34
Here first two are positional arguments and rest are optional.
Now, I want to have a feature such that when the user gives a filename as input in command line, then it should override these above arguments and take the arguments from header of the file. I meam when user gives sth like
id|sequence|age|name|........... (header of the file with first two cols as positional arguments and rest positional)
On giving this in command line:
./script.py -f filename
it should not complain of above positional arguments.
Is this feasible over my current implementation?

You will most likely need to implement this check yourself. Make both arguments (positional and -f) optional (required=False and nargs="*") and then implement your custom check and use the error method of ArgumentParser. To make it easier for user mention the correct usage in help string.
Something like this:
parser = ArgumentParser()
parser.add_argument("positional", nargs="*", help="If you don't provide positional arguments you need use -f")
parser.add_argument("-f", "--file", required=False, help="...")
args = parser.parse_args()
if not args.file and not args.positional:
parser.error('You must use either -f or positional argument')

argparse key=value parameters

This first link has the same question in the first section, but it is unanswered
(python argparse: parameter=value). And this second question is similar, but I can't seem to get it working for my particular case
( Using argparse to parse arguments of form "arg= val").
So my situation is this -- I am re-writing a Python wrapper which is used by many other scripts (I would prefer not to modify these other scripts). Currently, the Python wrapper is called with command line arguments of the form --key=value for a number of different arguments, but was parsed manually. I would like to parse them with argparse.
N.B. The argument names are unwieldy, so I am renaming using the dest option in add_argument.
parser = argparse.ArgumentParser(description='Wrappin Ronnie Reagan')
parser.add_argument("--veryLongArgName1", nargs=1, dest="arg1", required=True)
parser.add_argument("--veryLongArgName2", nargs=1, dest="arg2")
parser.add_argument("--veryLongArgName3", nargs=1, dest="arg3")
userOpts = vars(parser.parse_args())
Which, while apparently parsing the passed command lines correctly, displays this as the help:
usage: testing_argsparse.py [-h] --veryLongArgName1 ARG1
[--veryLongArgName2 ARG2]
[--veryLongArgName3 ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
But what I want is that all parameters are specified with the --key=value format, not --key value. i.e.
usage: testing_argsparse.py [-h] --veryLongArgName1=ARG1
[--veryLongArgName2=ARG2]
[--veryLongArgName3=ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required

testing_argsparse.py --veryLongArgName1=foo
works. argparse module accepts both --veryLongArgName1=foo and --veryLongArgName1 foo formats.
What exact command line arguments are you trying to pass to argparse that's causing it to not work?

A little late but for anyone with a similar request as the OP you could use a custom HelpFormatter.
class ArgFormatter(argparse.HelpFormatter):
def _format_args(self, *args):
result = super(ArgFormatter, self)._format_args(*args)
return result and '%%%' + result
def _format_actions_usage(self, *args):
result = super(ArgFormatter, self)._format_actions_usage(*args)
return result and result.replace(' %%%', '=')
This can then be passed to ArgumentParser to give the wanted behavior.
parser = argparse.ArgumentParser(
description='Wrappin Ronnie Reagan',
formatter_class=ArgFormatter)
This intercepts the args (ARG1, ARG2, ...) and adds a custom prefix which is later replaced (along with the unwanted space) for an = symbol. The and in the return statements makes sure to only modify the result if it's non-empty.

Argparse: ignore multiple positional arguments when optional argument is specified

I'm trying to make argparse ignore the fact that two normally required positional arguments shouldn't be evaluated when an optional argument (-l) is specified.
Basically I'm trying to replicate the behavior of --help: when you specify the -h, all missing required arguments are ignored.
Example code:
parser = argparse.ArgumentParser(description="Foo bar baz")
parser.add_argument('arg1', help='arg1 is a positional argument that does this')
parser.add_argument('arg2', help='arg2 is a positional argument that does this')
parser.add_argument('-l', '--list', dest='list', help='this is an optional argument that prints stuff')
options, args = parser.parse_args()
if options.list:
print "I list stuff"
And of course, if I run it now, I get :
error: too few arguments
I tried different things like nargs='?', but couldn't get anything working.
This question is quite similar but wasn't answered.

Unfortunately, argparse isn't quite flexible enough for this. The best you can do is to make arg1 and arg2 optional using nargs="?" and check yourself whether they are given if needed.
The internal help action is implemented by printing the help message and exiting the program as soon as -h or --help are encountered on the command line. You could write a similar action yourself, something like
class MyAction(argparse.Action):
def __call__(self, parser, values, namespace, option_string):
print "Whatever"
parser.exit()
(Warning: untested code!)
There are definite downsides to the latter approac, though. The help message will unconditionally show arg1 and arg2 as compulsory arguments. And parsing the command line simply stops when encountering -l or --list, ignoring any further arguments. This behaviour is quite acceptable for --help, but is less than desirable for other options.

I ran into this issue and decided to use subcommands. Subcommands might be overkill, but if you find your program not using some of the positional arguments in many instances (as I did), then subcommands might be a good solution.
For your given example, I'd use something like the following:
parser = argparse.ArgumentParser(description="Foo bar baz")
subparsers = parser.add_subparsers(description='available subcommands')
parser_main = subparsers.add_parser('<main_command_name>')
parser_main.add_argument('arg1', help='arg1 is a positional argument that does this')
parser_main.add_argument('arg2', help='arg2 is a positional argument that does this')
parser_list = subparsers.add_parser('list', help='this is a subcommand that prints stuff')
options, args = parser.parse_args()
I left out some details that you might want to include (like set_defaults(func=list)), which are mentioned in the argparse documentation.

I think the nargs='*' is helpful.
Positional arguments is ignorable, then you can use if to check the positional arguments is true or false.
http://docs.python.org/library/argparse.html#nargs

The cleanest approach I've been able to find so far is to split the parsing into two stages. First check for -l/--list:
parser = argparse.ArgumentParser(description="Foo bar baz")
parser.add_argument('-l', '--list', dest='list', action='store_true',
help='this is an optional argument that prints stuff')
options, remainder = parser.parse_known_args()
Now, since you used parse_known_args, you won't get an error up to here, and you can decide what to do with the remainder of the arguments:
if options.list:
print "I list stuff"
else:
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('arg1', help='arg1 is a positional argument that does this')
parser.add_argument('arg2', help='arg2 is a positional argument that does this')
options = parser.parse_args(remainder)
You may want to set the usage option in the first parser to make the help string a bit nicer.

I may have found a solution here. True, it is a dirty hack, but it works.
Note: all the following applies to Python 3.3.2.
As per the answer here, parse_args checks which actions are required and throws an error if any of them are missing. I propose to override this behavior.
By subclassing ArgumentParser we can define a new ArgumentParser.error method (original here) that will check whether the error was thrown because some arguments are missing and take necessary action. Code follows:
import argparse
import sys
from gettext import gettext as _
class ArgumentParser(argparse.ArgumentParser):
skip_list = []
def error(self, message):
# Let's see if we are missing arguments
if message.startswith('the following arguments are required:'):
missingArgs = message.split('required: ')[1].split(', ')
newArgs = [] # Creating a list of whatever we should not skip but is missing
for arg in missingArgs:
if arg not in self.skip_list:
newArgs.append(arg)
else:
self.skip_list.remove(arg) # No need for it anymore
if len(newArgs) == 0:
return # WARNING! UNTESTED! MAY LEAD TO SPACETIME MELTDOWN!
else: # Some required stuff is still missing, so we show a corrected error message
message = _('the following arguments are required: %s') % ', '.join(newArgs)
self.print_usage(sys.stderr) # Original method behavior
args = {'prog': self.prog, 'message': message}
self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
The new method first checks whether the error is because arguments are missing from the command line (see here for the code that generates the error). If so, the method gets the names of the arguments from the error message and puts them into a list (missingArgs).
Then, we iterate over this list and check which arguments should be skipped, and which are still required. To determine which arguments to skip, we compare them against skip_list. It is a field in our ArgumentParser subclass that should contain the names of the arguments to skip even when they are required by the parser. Please note that arguments that happen to be in skip_list are removed from it when they are found.
If there are still required arguments that are missing from the command line, the method throws a corrected error message. If all the missing arguments should be skipped, however, the method returns.
WARNING! The original definition of ArgumentParser.error states that if it is overridden in a subclass it should not return, but rather exit or raise an exception. Therefore, what is shown here is potentially unsafe and may cause your program to crash, your computer to catch fire or worse - IT MAY EVAPORATE ALL YOUR TEA. However, it seems like in this particular case (missing required arguments) it is safe to return from the method. But it might not be. You have been warned.
In order to fill skip_list we could use code like this:
class SpecialHelp(argparse._HelpAction):
def __call__(self, parser, namespace, values, option_string=None):
parser.print_help()
print()
for action in parser._actions:
if action != self and action.required:
parser.skip_list.append(argparse._get_action_name(action))
This particular class imitates the built-in help action, but instead of exiting it inserts all the remaining required arguments into skip_list.
Hope my answer helps and best of luck.

It may be ugly, but that is what I normally do.
def print_list():
the_list = ["name1", "name2"]
return "{0}".format(the_list)
...
parser.add_argument("-l", "--list", action='version',
version=print_list(), help="print the list")