I am writing a program that takes a list of urls as a parameter. Optionally, I would like to also be able to include a file containing a list of additional urls. Thus the program help would look like a combination of the below:
usage: [-h] url [url ...]
or
usage: [-h] --input-file file [ url [url ...]]
I am currently programming it as:
usage: [-h] [--input-file file] [url [url ...]]
where both usage of --input-file and specifying at least one url is optional. I am then checking myself if either the input-file or list of urls is specified.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-i','--input-file', help="File of urls to be read")
parser.add_argument('urls',nargs='*')
args = parser.parse_args()
if not args.input_list and not args.urls:
parser.error('must have either an input-file or a url')
Is there a way in argparse where I can force the user to specify a url if no input-file is given, but make it optional if it is?
If you specify a fromfile-prefix-chars (such as #), the user could specify a file to read these urls from. The lines of this file are read and added to the sys.argv just as though they had been typed in the command line.
http://docs.python.org/dev/library/argparse.html#fromfile-prefix-chars
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('urls',nargs='+') # require 1 or more url
args = parser.parse_args()
these all work (where urls.txt has one url string per line)
parser.parse_args('four.txt'.split())
parser.parse_args('#urls.txt four.txt'.split())
parser.parse_args('#urls.txt'.split())
while no file or url gives an error
parser.parse_args(''.split())
The default help does not say anything about allowing that #url.txt file name, but you could add that in a custom description string.
PS - Your post-argparse test is simpler than anything you could specify in argparse. A required mutually-exclusive-group involving the -i argument and the positional might work, but it doesn't do much more than your test. I wrote 'might' because I'd have to double whether a '*' positional works in such a group. But this is one of the first cases I've seen where that fromfile_prefix_chars option would be genuinely useful.
ArgumentParser.add_mutually_exclusive_group() should help you accomplish your goal. Have a look at the example at https://docs.python.org/2/library/argparse.html#mutual-exclusion or the code below:
~#: cat urls.py
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-infile', dest='file', help='File of URLs to be read')
group.add_argument('-urls', dest='urls', nargs='+', help='One or more URLs')
args = parser.parse_args()
~#: python urls.py
usage: urls.py [-h] (-infile FILE | -urls URLS [URLS ...])
urls.py: error: one of the arguments -infile -urls is required
Related
So I'm writing this very small program to do http get and post requests. The requests are as follows:
requestApp.py help
requestApp.py help get
requestApp.py help post
requestApp.py get [-v] [-h key:value] URL
requestApp.py post [-v] [-h key:value] [-d inline-data] [-f file] URL
As you can see, the -v, -h, -d, -f, URL arguments are optional. The get and post arguments are non-optional. I'll show you the snippet of my program that is relevant to this situation:
parser = argparse.ArgumentParser(description='httpc is a curl-like application but supports HTTP protocol only.')
parser.add_argument('command', type=str, help=help_output())
parser.add_argument('url', action='store_true', help='The URL that will be provided to perform the requested command.')
parser.add_argument('-v', '--verbose', action='store_true')
The command argument will be help, get, or post, and the url argument is self explanatory. My question is related to the second and third commands above, namely:
requestApp.py help get
requestApp.py help post
How can I make sure that when typing help get, the get will not be registered in the URL (same for help post). In addition, when I do include a URL, I want it to be stored inside of the URL argument. Would I have to manually evaluate the arguments passed through if statements? Or there is a better way to do it?
Perhaps the closest an argparse solution can come, at least without going the subparser route, is:
import argparse
import sys
print(sys.argv)
parser = argparse.ArgumentParser()
parser.add_argument('-k', '--keyvalue')
parser.add_argument('-v', '--verbose', action='store_true')
parser.add_argument('-d', '--data')
parser.add_argument('-f', '--file')
parser.add_argument('pos1', choices = ['help', 'get', 'post'])
parser.add_argument('pos2')
args = parser.parse_args()
print(args)
The resulting help is:
1744:~/mypy$ python stack54383659.py get aurl -h
['stack54383659.py', 'get', 'aurl', '-h']
usage: stack54383659.py [-h] [-k KEYVALUE] [-v] [-d DATA] [-f FILE]
{help,get,post} pos2
positional arguments:
{help,get,post}
pos2
optional arguments:
-h, --help show this help message and exit
-k KEYVALUE, --keyvalue KEYVALUE
-v, --verbose
-d DATA, --data DATA
-f FILE, --file FILE
The fit isn't perfect. For example you can give just help, but you can provide just -h. The 2nd positional value can be any string, 'get', a valid url or something else. Your own code will have to valid that. The key:value bit requires your own parsing.
In the argparse way of parsing the optionals can occur in any order. The two positionals have to occur in the given order (relative to each other).
In newer Pythons I can change the last positional to be 'optional', and use the new intermixed parser. That would allow me to give just 'help' (or just 'get'):
parser.add_argument('pos2', nargs='?')
args = parser.parse_intermixed_args()
intermixed is needed if the two positional values are separated by flags. For some complex reasons, the regular parsing may consume the '?' argument prematurely leaving you with an extra unrecognized string.
Another approach is to define all the flagged arguments, and use parse_known_args. The non-flag values will be in the extras list, which you can parse as you like. Older parsers like optparse did essentially that. argparse added a limited ability to handle positional arguments as well, but strictly by position, not by value.
It is quite complicated to do this using argparse here is how to do it using docopt, docopt parses the usage pattern and returns a dictionary :
"""
Usage:
requestApp help [get|post]
requestApp get [-v] [-k=key:value] <URL>
requestApp post [-v] [-k=key:value] [-d=data] [-f file] <URL>
Options:
-v --verbose This is verbose mode
-d=data This option does this
-k=key:value This one does that
-f file This one is magic
"""
from docopt import docopt
ARGS = docopt(__doc__)
For example with requestApp.py post -k hello:world -f myfile.txt google.com docopt will return:
{
"--verbose": false,
"-d": None,
"-f": "myfile.txt",
"-k": "hello:world",
"<URL>": "google.com",
"get": false,
"help": false,
"post": true
}
Then you can do:
if ARGS['help']:
if ARGS['get']: pass # requestApp help get
else if ARGS['post']: pass # requestApp help post
else: pass # requestApp help
exit()
if ARGS['get']: pass # requestApp get
else if ARGS['post']: pass # requestApp post
if ARGS['--verbose']: print("this is just the beginning")
-h is a reserved option by default (for help) that makes docopt returns the usage pattern and exit.
docopt will return the usage pattern to stdout and exit if you try illegal commands such as requestApp help unicorn
I'm building a command line argparser for my program and I try to give much detail in the -h option
I have the following code:
import argparse
legal_actions = ['act1', 'act2', 'act3']
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='available commands')
parser_cmd = subparsers.add_parser("cmd")
parser_cmd.add_argument("-a", "--action", type=str, metavar="", choices=legal_actions, required=True,
help='list of actions: {%(choices)s}')
parser_cmd.add_argument("nargs", type=str, nargs='*',
help="the rest of the arguments required to perform an action")
parser_cmd.set_defaults(func=cmd_handler)
python prog.py cmd -h will result the following prints in the command line
usage: cmd [-h] -a [nargs [nargs ...]]
positional arguments:
nargs the rest of the arguments required to perform an action
optional arguments:
-h, --help show this help message and exit
-a , --action list of actions: {act1, act2, act3}
Every action requires a different number of arguments, so I want to add something that will describe the actions (from the list of actions) like:
actions availble:
act1: requires 2 arguments (arg1, arg2)
act2: requires 0 arguments ()
act3: requires 1 arguments (arg1)
And I want it to have any link with the above "optional arguments", so it'll be easy to see the "acts" are under the -a option
If you want to add more information, you can use the epilog-parameter:
from argparse import RawDescriptionHelpFormatter # This is used to enable newlines in epilogs and descriptions(\n)
from argparse import ArgumentParser
description = 'Some description of program'
epilog = 'actions availble:\n\t'
epilog += 'act1: requires 2 arguments (arg1, arg2)\n\t'
epilog += 'act2: requires 0 arguments ()\n\t'
epilog += 'act3: requires 1 arguments (arg1)'
parser = argparse.ArgumentParser(description=description, epilog=epilog,
formatter_class=RawTextHelpFormatter)
This will print out
actions availble:
act1: requires 2 arguments (arg1, arg2)
act2: requires 0 arguments ()
act3: requires 1 arguments (arg1)
At the end of help output. The epilog-parameter can also be included in the add_parser() when using add_subparsers():
This object has a single method, add_parser(), which takes a command name and any ArgumentParser constructor arguments, and returns an ArgumentParser object that can be modified as usual.
NOTE: the default formatter will ignore newlines, so take a look at Python argparse: How to insert newline in the help text? where this is addressed, which describe how to replace the formatter รก la:
ArgumentParser(..., formatter_class=RawDescriptionHelpFormatter)
Read more about the epilog-parameter in docs.
Trying to make my script more generic so I added some flags. My problem is the help only works if you type -h , obviously. I want to envoke -h when no flags are selected.
For example:
python 0_log_cleaner.py
Traceback (most recent call last):
File "0_log_cleaner.py", line 51, in <module>
getFiles(options.path,options.org_phrase,options.new_phrase,options.org_AN,options.new_AN,options.dst_path)
File "0_log_cleaner.py", line 37, in getFiles
for filename in os.listdir(path):
TypeError: coercing to Unicode: need string or buffer, NoneType found
but if I add -h I get:
python 0_log_cleaner.py -h
Usage: Example:
python 0_log_cleaner.py --sp original_logs/ --dp clean_logs/ --od CNAME --nd New_CNAME --oan 10208 --nan NewAN
Options:
-h, --help show this help message and exit
--sp=PATH Path to the source logs ie original_logs/
--dp=DST_PATH Path to where sanitized logs will be written to ie
clean_logs
--od=ORG_PHRASE original domain name ie www.clientName.com, use the command
-od clientName
--nd=NEW_PHRASE domain name to replace -od. ie -od clientName -nd domain
makes all log that use to be www.clientName.com into
www.domain.com
--oan=ORG_AN original AN number
--nan=NEW_AN AN number to replace original. ie -oan 12345 -nan AAAA1
replaces all instances of the AN number 12345 with AAAA1
EDIT 3 ANSWER
sample of my code to produce ^
import argparse
import sys
usage = "Description of function"
parser = argparse.ArgumentParser(description=usage)
parser.add_argument("--sp", dest="path", help='Path to the source logs ie logs/')
...
...(additional add arugments)
args = parser.parse_args()
def getFiles(path,org_phrase,new_phrase,org_AN,new_AN,dst_path):
if not len(sys.argv) > 1:
parser.print_help()
else:
run your logic
borrowed from here : Argparse: Check if any arguments have been passed
Here's how the final code looks like:
import argparse
import sys
usage = "Description of function"
parser = argparse.ArgumentParser(description=usage)
parser.add_argument("--sp", dest="path", help='Path to the source logs ie logs/')
...
...(additional add arugments)
args = parser.parse_args()
def getFiles(path,org_phrase,new_phrase,org_AN,new_AN,dst_path):
if not len(sys.argv) > 1:
parser.print_help()
else:
run your logic
If someone is still interested in a (very simple) solution:
parser = argparse.ArgumentParser()
parser.add_argument("jfile", type=str, help="Give the JSON file name.")
parser.add_argument("--output", type=str, help="Type in the final excel files name.")
try:
args = parser.parse_args()
return args
except:
parser.print_help()
My professor wanted the script to force the -h / --help page even when there are too few arguments. Instead of going like "python SCRIPT.py -h".
So what I did here was like: "Try to parse the arguments. And if it works, give them back to the main methode. Otherwise, if you fail (except), print the help(). Okay? Nice". ;)
Without knowing the method you are parsing with, I will assume the following (comment me if I am wrong or edit your question with some code on how you handle your parsing):
You are parsing everything and putting it in a variable. let parsed be that variable.
You are checking parsed for the existence of any of your option flags.
You probably not checking for the non-existence of arguments:
parsed = '' <- empty string
# or if you are using a list:
# parsed = []
if parsed: <- if parsed is not empty ("" or []) returns true
Do your stuff here, because you have options now
else: <- Differently options were not provided
Invoke the same method that you invoke when the option is -h
Also as #dhke suggests, consider using argparse if you are not using it already!
EDIT #1:
Translated for your specific case:
args = parser.parse_args() <-- ending line of your provided code
if not args:
parser.print_help()
else:
Do your stuff
I want to parse the cmdline option to be -option key=value type in argparse.
For example:
script.py -project prj1=rev1
Generally:
script.py -project prj1 --> OK
script.py -project=prj1 --> OK
script.py -project prj1=rev1 --> How to flag that argument value should be in str=str format.
script.py -project=prj1,prj2 --> How to flag that we need comma separated strings.
In the above, -project is the option. proj1=rev1 in this way I want the argument to be present. It should flag an error if it is not in the proper format and print the help message. I can use regular expression once I collect the project value using (\w)=(\w). If not in the above format can flag an error. But is there a way to filter out this and flag an error at parsing the cmdline arguments itself?
You can take advantage of this fact from the argparse documentation:
type= can take any callable that takes a single string argument and returns the converted value:
For example, to support the first format (--project prj1=rev1) you could do something like:
import os
import sys
import argparse
def handle_kv_string(val):
if '=' in val:
return val.split('=')
else:
raise argparse.ArgumentTypeError('Must specify k=v')
def parse_args():
p = argparse.ArgumentParser()
p.add_argument('--project',
type=handle_kv_string)
return p.parse_args()
def main():
args = parse_args()
print args
if __name__ == '__main__':
main()
This gets you, with valid arguments:
$ ./argtest --project foo=bar
Namespace(project=['foo', 'bar'])
And with invalid arguments:
$ ./argtest --project foo
usage: argtest.py [-h] [--project PROJECT]
argtest.py: error: argument --project: Must specify k=v
You could apply a similar solution to your second example.
I am creating a python script and for parsing the arguments I would need this:
the script will accept three parameters, only one always mandatory, the second one will only be mandatory depending on certain values of the first one and the third one may or may not appear.
This is my try:
class pathAction(argparse.Action):
folder = {'remote':'/path1', 'projects':'/path2'}
def __call__(self, parser, args, values, option = None):
args.path = values
print "ferw %s " % args.component
if args.component=='hos' or args.component=='hcr':
print "rte %s" % args.path
if args.path and pathAction.folder.get(args.path):
args.path = pathAction.folder[args.path]
else:
parser.error("You must enter the folder you want to clean: available choices[remote, projects]")
def main():
try:
# Arguments parsing
parser = argparse.ArgumentParser(description="""This script will clean the old component files.""")
parser.add_argument("-c", "--component", help="component to clean", type=lowerit, choices=["hos", "hcr", "mdw", "gui"], required=True)
parser.add_argument("-p", "--path", help="path to clean", action = pathAction, choices = ["remote", "projects"])
parser.add_argument("-d", "--delete", help="parameter for deleting the files from the filesystem", nargs='*', default=True)
args = parser.parse_args()
if works well except one case: if i have -c it should complain because there is no -p however it does not
Can you help me please?
Thanks
You can add some custom validation like this:
if args.component and not args.path:
parser.error('Your error message!')
Your special action will be used only if there is a -p argument. If you just give it a -c the cross check is never used.
Generally checking for interactions after parse_args (as Gohn67 suggested) is more reliable, and simpler than with custom actions.
What happens if your commandline was '-p remote -c ...'? pathAction would be called before the -c value is parsed and set. Is that what you want? Your special action only works if -p is given, and is the last argument.
Another option is to make 'component' a subparser positional. By default positionals are required. path and delete can be added to those subparsers that need them.
import argparse
parser = argparse.ArgumentParser(description="""This script will clean the old component files.""")
p1 = argparse.ArgumentParser(add_help=False)
p1.add_argument("path", help="path to clean", choices = ["remote", "projects"])
p2 = argparse.ArgumentParser(add_help=False)
p2.add_argument("-d", "--delete", help="parameter for deleting the files from the filesystem", nargs='*', default=True)
sp = parser.add_subparsers(dest='component',description="component to clean")
sp.add_parser('hos', parents=[p1,p2])
sp.add_parser('hcr', parents=[p1,p2])
sp.add_parser('mdw', parents=[p2])
sp.add_parser('gui', parents=[p2])
print parser.parse_args()
sample use:
1848:~/mypy$ python2.7 stack21625446.py hos remote -d 1 2 3
Namespace(component='hos', delete=['1', '2', '3'], path='remote')
I used parents to simplify adding arguments to multiple subparsers. I made path a positional, since it is required (for 2 of the subparsers). In those cases --path just makes the user type more. With nargs='*', --delete has to belong to the subparsers so it can occur last. If it's nargs was fixed (None or number) it could be an argument of parser.