In Python, how can we find out the command line arguments that were provided for a script, and process them?
For some more specific examples, see Implementing a "[command] [action] [parameter]" style command-line interfaces? and How do I format positional argument help using Python's optparse?.
import sys
print("\n".join(sys.argv))
sys.argv is a list that contains all the arguments passed to the script on the command line. sys.argv[0] is the script name.
Basically,
import sys
print(sys.argv[1:])
The canonical solution in the standard library is argparse (docs):
Here is an example:
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument("-f", "--file", dest="filename",
help="write report to FILE", metavar="FILE")
parser.add_argument("-q", "--quiet",
action="store_false", dest="verbose", default=True,
help="don't print status messages to stdout")
args = parser.parse_args()
argparse supports (among other things):
Multiple options in any order.
Short and long options.
Default values.
Generation of a usage help message.
Just going around evangelizing for argparse which is better for these reasons.. essentially:
(copied from the link)
argparse module can handle positional
and optional arguments, while
optparse can handle only optional
arguments
argparse isn’t dogmatic about
what your command line interface
should look like - options like -file
or /file are supported, as are
required options. Optparse refuses to
support these features, preferring
purity over practicality
argparse produces more
informative usage messages, including
command-line usage determined from
your arguments, and help messages for
both positional and optional
arguments. The optparse module
requires you to write your own usage
string, and has no way to display
help for positional arguments.
argparse supports action that
consume a variable number of
command-line args, while optparse
requires that the exact number of
arguments (e.g. 1, 2, or 3) be known
in advance
argparse supports parsers that
dispatch to sub-commands, while
optparse requires setting
allow_interspersed_args and doing the
parser dispatch manually
And my personal favorite:
argparse allows the type and
action parameters to add_argument()
to be specified with simple
callables, while optparse requires
hacking class attributes like
STORE_ACTIONS or CHECK_METHODS to get
proper argument checking
There is also argparse stdlib module (an "impovement" on stdlib's optparse module). Example from the introduction to argparse:
# script.py
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument(
'integers', metavar='int', type=int, choices=range(10),
nargs='+', help='an integer in the range 0..9')
parser.add_argument(
'--sum', dest='accumulate', action='store_const', const=sum,
default=max, help='sum the integers (default: find the max)')
args = parser.parse_args()
print(args.accumulate(args.integers))
Usage:
$ script.py 1 2 3 4
4
$ script.py --sum 1 2 3 4
10
If you need something fast and not very flexible
main.py:
import sys
first_name = sys.argv[1]
last_name = sys.argv[2]
print("Hello " + first_name + " " + last_name)
Then run python main.py James Smith
to produce the following output:
Hello James Smith
The docopt library is really slick. It builds an argument dict from the usage string for your app.
Eg from the docopt readme:
"""Naval Fate.
Usage:
naval_fate.py ship new <name>...
naval_fate.py ship <name> move <x> <y> [--speed=<kn>]
naval_fate.py ship shoot <x> <y>
naval_fate.py mine (set|remove) <x> <y> [--moored | --drifting]
naval_fate.py (-h | --help)
naval_fate.py --version
Options:
-h --help Show this screen.
--version Show version.
--speed=<kn> Speed in knots [default: 10].
--moored Moored (anchored) mine.
--drifting Drifting mine.
"""
from docopt import docopt
if __name__ == '__main__':
arguments = docopt(__doc__, version='Naval Fate 2.0')
print(arguments)
One way to do it is using sys.argv. This will print the script name as the first argument and all the other parameters that you pass to it.
import sys
for arg in sys.argv:
print arg
#set default args as -h , if no args:
if len(sys.argv) == 1: sys.argv[1:] = ["-h"]
I use optparse myself, but really like the direction Simon Willison is taking with his recently introduced optfunc library. It works by:
"introspecting a function
definition (including its arguments
and their default values) and using
that to construct a command line
argument parser."
So, for example, this function definition:
def geocode(s, api_key='', geocoder='google', list_geocoders=False):
is turned into this optparse help text:
Options:
-h, --help show this help message and exit
-l, --list-geocoders
-a API_KEY, --api-key=API_KEY
-g GEOCODER, --geocoder=GEOCODER
I like getopt from stdlib, eg:
try:
opts, args = getopt.getopt(sys.argv[1:], 'h', ['help'])
except getopt.GetoptError, err:
usage(err)
for opt, arg in opts:
if opt in ('-h', '--help'):
usage()
if len(args) != 1:
usage("specify thing...")
Lately I have been wrapping something similiar to this to make things less verbose (eg; making "-h" implicit).
As you can see optparse "The optparse module is deprecated with and will not be developed further; development will continue with the argparse module."
Pocoo's click is more intuitive, requires less boilerplate, and is at least as powerful as argparse.
The only weakness I've encountered so far is that you can't do much customization to help pages, but that usually isn't a requirement and docopt seems like the clear choice when it is.
import argparse
parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('integers', metavar='N', type=int, nargs='+',
help='an integer for the accumulator')
parser.add_argument('--sum', dest='accumulate', action='store_const',
const=sum, default=max,
help='sum the integers (default: find the max)')
args = parser.parse_args()
print(args.accumulate(args.integers))
Assuming the Python code above is saved into a file called prog.py
$ python prog.py -h
Ref-link: https://docs.python.org/3.3/library/argparse.html
You may be interested in a little Python module I wrote to make handling of command line arguments even easier (open source and free to use) - Commando
Yet another option is argh. It builds on argparse, and lets you write things like:
import argh
# declaring:
def echo(text):
"Returns given word as is."
return text
def greet(name, greeting='Hello'):
"Greets the user with given name. The greeting is customizable."
return greeting + ', ' + name
# assembling:
parser = argh.ArghParser()
parser.add_commands([echo, greet])
# dispatching:
if __name__ == '__main__':
parser.dispatch()
It will automatically generate help and so on, and you can use decorators to provide extra guidance on how the arg-parsing should work.
I recommend looking at docopt as a simple alternative to these others.
docopt is a new project that works by parsing your --help usage message rather than requiring you to implement everything yourself. You just have to put your usage message in the POSIX format.
Also with python3 you might find convenient to use Extended Iterable Unpacking to handle optional positional arguments without additional dependencies:
try:
_, arg1, arg2, arg3, *_ = sys.argv + [None] * 2
except ValueError:
print("Not enough arguments", file=sys.stderr) # unhandled exception traceback is meaningful enough also
exit(-1)
The above argv unpack makes arg2 and arg3 "optional" - if they are not specified in argv, they will be None, while if the first is not specified, ValueError will be thouwn:
Traceback (most recent call last):
File "test.py", line 3, in <module>
_, arg1, arg2, arg3, *_ = sys.argv + [None] * 2
ValueError: not enough values to unpack (expected at least 4, got 3)
My solution is entrypoint2. Example:
from entrypoint2 import entrypoint
#entrypoint
def add(file, quiet=True):
''' This function writes report.
:param file: write report to FILE
:param quiet: don't print status messages to stdout
'''
print file,quiet
help text:
usage: report.py [-h] [-q] [--debug] file
This function writes report.
positional arguments:
file write report to FILE
optional arguments:
-h, --help show this help message and exit
-q, --quiet don't print status messages to stdout
--debug set logging level to DEBUG
import sys
# Command line arguments are stored into sys.argv
# print(sys.argv[1:])
# I used the slice [1:] to print all the elements except the first
# This because the first element of sys.argv is the program name
# So the first argument is sys.argv[1], the second is sys.argv[2] ecc
print("File name: " + sys.argv[0])
print("Arguments:")
for i in sys.argv[1:]:
print(i)
Let's name this file command_line.py and let's run it:
C:\Users\simone> python command_line.py arg1 arg2 arg3 ecc
File name: command_line.py
Arguments:
arg1
arg2
arg3
ecc
Now let's write a simple program, sum.py:
import sys
try:
print(sum(map(float, sys.argv[1:])))
except:
print("An error has occurred")
Result:
C:\Users\simone> python sum.py 10 4 6 3
23
This handles simple switches, value switches with optional alternative flags.
import sys
# [IN] argv - array of args
# [IN] switch - switch to seek
# [IN] val - expecting value
# [IN] alt - switch alternative
# returns value or True if val not expected
def parse_cmd(argv,switch,val=None,alt=None):
for idx, x in enumerate(argv):
if x == switch or x == alt:
if val:
if len(argv) > (idx+1):
if not argv[idx+1].startswith('-'):
return argv[idx+1]
else:
return True
//expecting a value for -i
i = parse_cmd(sys.argv[1:],"-i", True, "--input")
//no value needed for -p
p = parse_cmd(sys.argv[1:],"-p")
Several of our biotechnology clients have posed these two questions recently:
How can we execute a Python script as a command?
How can we pass input values to a Python script when it is executed as a command?
I have included a Python script below which I believe answers both questions. Let's assume the following Python script is saved in the file test.py:
#
#----------------------------------------------------------------------
#
# file name: test.py
#
# input values: data - location of data to be processed
# date - date data were delivered for processing
# study - name of the study where data originated
# logs - location where log files should be written
#
# macOS usage:
#
# python3 test.py "/Users/lawrence/data" "20220518" "XYZ123" "/Users/lawrence/logs"
#
# Windows usage:
#
# python test.py "D:\data" "20220518" "XYZ123" "D:\logs"
#
#----------------------------------------------------------------------
#
# import needed modules...
#
import sys
import datetime
def main(argv):
#
# print message that process is starting...
#
print("test process starting at", datetime.datetime.now().strftime("%Y%m%d %H:%M"))
#
# set local values from input values...
#
data = sys.argv[1]
date = sys.argv[2]
study = sys.argv[3]
logs = sys.argv[4]
#
# print input arguments...
#
print("data value is", data)
print("date value is", date)
print("study value is", study)
print("logs value is", logs)
#
# print message that process is ending...
#
print("test process ending at", datetime.datetime.now().strftime("%Y%m%d %H:%M"))
#
# call main() to begin processing...
#
if __name__ == '__main__':
main(sys.argv)
The script can be executed on a macOS computer in a Terminal shell as shown below and the results will be printed to standard output (be sure the current directory includes the test.py file):
$ python3 test.py "/Users/lawrence/data" "20220518" "XYZ123" "/Users/lawrence/logs"
test process starting at 20220518 16:51
data value is /Users/lawrence/data
date value is 20220518
study value is XYZ123
logs value is /Users/lawrence/logs
test process ending at 20220518 16:51
The script can also be executed on a Windows computer in a Command Prompt as shown below and the results will be printed to standard output (be sure the current directory includes the test.py file):
D:\scripts>python test.py "D:\data" "20220518" "XYZ123" "D:\logs"
test process starting at 20220518 17:20
data value is D:\data
date value is 20220518
study value is XYZ123
logs value is D:\logs
test process ending at 20220518 17:20
This script answers both questions posed above and is a good starting point for developing scripts that will be executed as commands with input values.
Reason for the new answer:
Existing answers specify multiple options.
Standard option is to use argparse, a few answers provided examples from the documentation, and one answer suggested the advantage of it. But all fail to explain the answer adequately/clearly to the actual question by OP, at least for newbies.
An example of argparse:
import argparse
def load_config(conf_file):
pass
if __name__ == '__main__':
parser = argparse.ArgumentParser()
//Specifies one argument from the command line
//You can have any number of arguments like this
parser.add_argument("conf_file", help="configuration file for the application")
args = parser.parse_args()
config = load_config(args.conf_file)
Above program expects a config file as an argument. If you provide it, it will execute happily. If not, it will print the following
usage: test.py [-h] conf_file
test.py: error: the following arguments are required: conf_file
You can have the option to specify if the argument is optional.
You can specify the expected type for the argument using type key
parser.add_argument("age", type=int, help="age of the person")
You can specify default value for the arguments by specifying default key
This document will help you to understand it to an extent.
Using argparse in a Jupyter Notebook throws a TypeError. The same code works fine if I execute the same code as a script. MWE:
import argparse
parser = argparse.ArgumentParser(description='Foo')
parser.add_argument('--name', '-n', default='foo', help='foo')
args = parser.parse_args()
Result:
TypeError: 'level' is an invalid keyword argument for this function
One solution is to parse an empty list of arguments:
import argparse
parser = argparse.ArgumentParser(description='Foo')
parser.add_argument('--name', '-n', default='foo', help='foo')
args = parser.parse_args([])
Another is to use parse_known_args:
args, _ = parser.parse_known_args()
Ipython is running some command-line arguments in the background. This interfers with argparse and optparse.
See this bug for Spyder (Ipython IDE), where -f was being added as a command option and crashing as there was no handler for -f.
You could try checking the arguments currently in play (as they did for the Spyder bug) and putting a dummy handler in place.
Run
import sys
print(sys.argv)
inside Ipython and see what it outputs.
On my system, it gives
['/usr/lib/python3.6/site-packages/ipykernel_launcher.py', '-f', '/run/user/1000/jupyter/kernel-7537e4dd-b5e2-407c-9d4c-7023575cfc7c.json']
Argparse assumes the first entry is the program name (sys.argv[0]). In order to fix this, I had to call
parser = argparse.ArgumentParser(prog='myprogram', description='Foo')
... and now argparse works in my notebook.
When I run your code in a Notebook, I get an argparse usage error message:
usage: ipykernel_launcher.py [-h] [--name NAME]
ipykernel_launcher.py: error: unrecognized arguments: -f /run/user/1000/jupyter/kernel-a6504c0c-bed2-4405-8704-c008f52dcba6.json
With a print(sys.argv) I get
['/home/paul/.local/lib/python3.6/site-packages/ipykernel_launcher.py', '-f', '/run/user/1000/jupyter/kernel-a6504c0c-bed2-4405-8704-c008f52dcba6.json']
sys.argv, which parser parses, contains the values used to launch the Notebook server, which this particular parser is not setup to handle.
parser.parse_known_args() displays:
(Namespace(name='foo'),
['-f',
'/run/user/1000/jupyter/kernel-a6504c0c-bed2-4405-8704-c008f52dcba6.json'])
That extra stuff that your parser can't handle is put in the extras list.
Run with a custom argv list works:
parser.parse_args(['-n', 'foobar'])
Namespace(name='foobar')
It's a good idea to put argparse code (at least the parse_args line) in a __main__ block, so it is not run when the script is imported. It will still run when the script is run as a script.
if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser(description='Foo')
parser.add_argument('--name', '-n', default='foo', help='foo')
args = parser.parse_args()
print(args)
This script also works when using %run stack50763033.py. You can even give it arguments as you would with a script:
%run stack50763033.py -n testing
I have no idea what code is producing the level keyword error. You'll have to give us the traceback if you want help with that.
I am using argparse to parse command line arguments and by default on receiving invalid arguments it prints help message and exit. Is it possible to customize the behavior of argparse when it receives invalid arguments?
Generally I want to catch all invalid arguments and do stuff with them. I am looking for something like:
parser = argparse.ArgumentParser()
# add some arguments here
try:
parser.parse_args()
except InvalidArgvsError, iae:
print "handle this invalid argument '{arg}' my way!".format(arg=iae.get_arg())
So that I can have:
>> python example.py --invalid some_text
handle this invalid argument 'invalid' my way!
You might want to use parse_known_args and then take a look at the second item in the tuple to see what arguments were not understood.
That said, I believe this will only help with extra arguments, not expected arguments that have invalid values.
Some previous questions:
Python argparse and controlling/overriding the exit status code
I want Python argparse to throw an exception rather than usage
and probably more.
The argparse documentation talks about using parse_known_args. This returns a list of arguments that it does not recognize. That's a handy way of dealing with one type of error.
It also talks about writing your own error and exit methods. That error that you don't like passes through those 2 methods. The proper way to change those is to subclass ArgumentParser, though you can monkey-patch an existing parser. The default versions are at the end of the argparse.py file, so you can study what they do.
A third option is to try/except the Systemexit.
try:
parser=argparse.ArgumentParser()
args=parser.parse_args()
except SystemExit:
exc = sys.exc_info()[1]
print(exc)
This way, error/exit still produce the error message (to sys.stderr) but you can block exit and go on and do other things.
1649:~/mypy$ python stack38340252.py -x
usage: stack38340252.py [-h]
stack38340252.py: error: unrecognized arguments: -x
2
One of the earlier question complained that parser.error does not get much information about the error; it just gets a formatted message:
def myerror(message):
print('error message')
print(message)
parser=argparse.ArgumentParser()
parser.error = myerror
args=parser.parse_args()
displays
1705:~/mypy$ python stack38340252.py -x
error message
unrecognized arguments: -x
You could parse that message to find out the -x is the unrecognized string. In an improvement over earlier versions it can list multiple arguments
1705:~/mypy$ python stack38340252.py foo -x abc -b
error message
unrecognized arguments: foo -x abc -b
Look up self.error to see all the cases that can trigger an error message. If you need more ideas, focus on a particular type of error.
===============
The unrecognized arguments error is produced by parse_args, which calls parse_known_args, and raises this error if the extras is not empty. So its special information is the list of strings that parse_known_args could not handle.
parse_known_args for its part calls self.error if it traps an ArgumentError. Generally those are produced when a specific argument (Action) has problems. But _parse_known_args calls self.error if required Action is missing, or if there's a mutually-exclusive-group error. choices can produce a different error, as can type.
You can try subclassing argparse.ArgumentParser() and overriding the error method.
From the argparse source:
def error(self, message):
"""error(message: string)
Prints a usage message incorporating the message to stderr and
exits.
If you override this in a subclass, it should not return -- it
should either exit or raise an exception.
"""
self.print_usage(_sys.stderr)
self.exit(2, _('%s: error: %s\n') % (self.prog, message))
Since the error code 2 is reserved for internal docker usage, I'm using the following to parse arguments in scripts inside docker containers:
ERROR_CODE = 1
class DockerArgumentParser(argparse.ArgumentParser):
def error(self, message):
"""error(message: string)
Prints a usage message incorporating the message to stderr and
exits.
If you override this in a subclass, it should not return -- it
should either exit or raise an exception.
Overrides error method of the parent class to exit with error code 1 since default value is
reserved for internal docker usage
"""
self.print_usage(sys.stderr)
args = {'prog': self.prog, 'message': message}
self.exit(ERROR_CODE, '%(prog)s: error: %(message)s\n' % args)
I am trying to use getoptsfor command line parsing. However, if I set the options to have mandatory arguments via :or = and no argument is given in the command line, the following option is taken as an argument of the first option. I would like this to raise an error instead. How can this be resolved?
Working example:
#!/usr/bin/env python
import sys, getopt, warnings
argv = sys.argv
try:
opts, args = getopt.getopt(argv[1:], "c:", ["config-file=", "sample-list="])
print >> sys.stderr, opts
except getopt.GetoptError as msg:
print msg
Running this script on the command line like this:
python getopt_test.py --config-file --sample-list
results in the following output (opts):
[('--config-file', '--sample-list')]
There is nothing wrong while running :
python getopt_test.py --config-file --sample-list
In your snippet:
opts, args = getopt.getopt(argv[1:], "c:", ["config-file=", "sample-list="])
# config-file requires argument i.e what ever argument come next in sys.argv list
# sample-list requires argument i.e what ever argument come next in sys.argv list
So, when you run as python getopt_test.py --config-file --sample-list,
--sample-list is just an argument to --config-file.
Lets confirm it by printing opts, which is list of tuple element containing first element inside tuple as the option name and second element as the arguments.
tmp/: python get.py --config-file --sample-list
[('--config-file', '--sample-list')] []
tmp/:
# lets put a proper argument for config-file
tmp/: python get.py --config-file 'argument_for_config_file'
[('--config-file', 'argument_for_config_file')] []
tmp/:
# Lets run with proper sample-list
tmp/: python get.py --config-file 'argument_for_config_file' --sample-list 'here is the
list'
[('--config-file', 'argument_for_config_file'), ('--sample-list', 'here is the list')]
[]
tmp/:
So, you need to write your own proper parse to make sure the user is providing the right options and argument. IF you are using optparse.
About the Exception: exception getopt.GetoptError
This is raised when an unrecognized option is found in the argument list or when an option
requiring an argument is given none. But in your case nothing of the rules was violate, thats why it was running silently without any error.
To prevent from all this optparse pitfall: highly recommend to use argparse which has tons of new and good feature to solve all your problem.
I’m able to pass ints and chars as Python command-line args, but can’t pass Strings successfully to the following:
if __name__ == '__main__' :
try :
print(len(sys.argv))
arg1 = ast.literal_eval(sys.argv[1])
arg2 = ast.literal_eval(sys.argv[2])
print(arg1)
print(arg2)
except Exception :
print('error')
The following works:
python test.py 1 2
and prints:
3
1
2
as dos the following:
python test.py ‘1’ ‘2’
which also prints
3
1
2
However, the following does not work:
python test.py ‘one’ ‘two’
It sees the number of args, but throws an Exception trying to grab them:
3
error
Could someone please provide some guidance on how to pass Strings as command-line args?
Thanks.
Your shell will eat those quotes before python ever sees them. Try like this:
python test.py '"one"' '"two"'
As an aside, command-line args are passed as strings anyway. It is odd to literal_eval them like this; a better pattern is to convert the strings to numbers when you need the numbers, and to just leave them as strings otherwise. This will also save you from having to quote-pad the strings twice on the command line.
The argparse module can handle that stuff for you automatically.
optparse.OptionParser may be helpful for you.
You can use it like this:
from optparse import OptionParser
if __name__ == "__main__":
parser = OptionParser(version="%prog 1.0.0")
parser.add_option("-c", "--config", action="store", dest="config_file",
default="test", type="string",
help="let prog load a specified configure file")
parser.add_option("-v", "--verbose", action="store_true", dest="verbose",
default=False, help="let prog run in verbose model")
options, args = parser.parse_args()
if options.verbose:
print "run in verbose model"
print "configure file is %s"%(options.config_file)