We Keep Coding

Passing directory paths as strings to argparse in Python

Scenario: I have a python script that receives as inputs 2 directory paths (input and output folders) and a variable ID. With these, it performs a data gathering procedure from xlsx and xlsm macros, modifies the data and saves to a csv (from the input folder, the inner functions of the code will run loops, to get multiple files and process them, one at a time).
Issue: Since the code was working fine when I was running it from the Spyder console, I decided to step it up and learn about cmd caller, argparse and the main function. I trying to implement that, but I get the following error:
Unrecognized arguments (the output path I pass from cmd)
Question: Any ideas on what I am doing wrong?
Obs: If the full script is required, I can post it here, but since it works when run from Spyder, I believe the error is in my argparse function.
Code (argparse function and __main__):
# This is a function to parse arguments:
def parserfunc():
import argparse
parser = argparse.ArgumentParser(description='Process Files')
parser.add_argument('strings', nargs=3)
args = parser.parse_args()
arguments = args.strings
return arguments
# This is the main caller
def main():
arguments = parserfunc()
# this next function is where I do the processing for the files, based on the paths and id provided):
modifierfunc(arguments[0], arguments[1], arguments[2])
#
if __name__ == "__main__":
main()

If you decided to use argparse, then make use of named arguments, not indexed. Following is an example code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('input')
parser.add_argument('output')
parser.add_argument('id')
args = parser.parse_args()
print(args.input, args.output, args.id) # this is how you use them
In case you miss one of them on program launch, you will get human readable error message like
error: the following arguments are required: id

You could drop the entire parserfunc() function.
sys.argv does indeed contain all arguments (always processed as a string) as mentioned by grapes.
So instead of this:
modifierfunc(arguments[0], arguments[1], arguments[2])
This should suffice:
import sys
modifierfunc(sys.argv[0], sys.argv[1], sys.argv[2])
Perhaps, first do a print, to see if the sys.argv holds the values you expect.
print('Argument 0='+sys.argv[0])
print('Argument 1='+sys.argv[1])
print('Argument 2='+sys.argv[2])

Python: builtins.IndexError: list index out of range [duplicate]

I'm currently teaching myself Python and was just wondering (In reference to my example below) in simplified terms what the sys.argv[1] represents. Is it simply asking for an input?
#!/usr/bin/python3.1
# import modules used here -- sys is a very standard one
import sys
# Gather our code in a main() function
def main():
print ('Hello there', sys.argv[1])
# Command line args are in sys.argv[1], sys.argv[2] ..
# sys.argv[0] is the script name itself and can be ignored
# Standard boilerplate to call the main() function to begin
# the program.
if __name__ == '__main__':
main()
You may have been directed here because you were asking about an IndexError in your code that uses sys.argv. The problem is not in your code; the problem is that you need to run the program in a way that makes sys.argv contain the right values. Please read the answers to understand how sys.argv works.
If you have read and understood the answers, and are still having problems on Windows, check if Python Script does not take sys.argv in Windows fixes the issue. If you are trying to run the program from inside an IDE, you may need IDE-specific help - please search, but first check if you can run the program successfully from the command line.

I would like to note that previous answers made many assumptions about the user's knowledge. This answer attempts to answer the question at a more tutorial level.
For every invocation of Python, sys.argv is automatically a list of strings representing the arguments (as separated by spaces) on the command-line. The name comes from the C programming convention in which argv and argc represent the command line arguments.
You'll want to learn more about lists and strings as you're familiarizing yourself with Python, but in the meantime, here are a few things to know.
You can simply create a script that prints the arguments as they're represented. It also prints the number of arguments, using the len function on the list.
from __future__ import print_function
import sys
print(sys.argv, len(sys.argv))
The script requires Python 2.6 or later. If you call this script print_args.py, you can invoke it with different arguments to see what happens.
> python print_args.py
['print_args.py'] 1
> python print_args.py foo and bar
['print_args.py', 'foo', 'and', 'bar'] 4
> python print_args.py "foo and bar"
['print_args.py', 'foo and bar'] 2
> python print_args.py "foo and bar" and baz
['print_args.py', 'foo and bar', 'and', 'baz'] 4
As you can see, the command-line arguments include the script name but not the interpreter name. In this sense, Python treats the script as the executable. If you need to know the name of the executable (python in this case), you can use sys.executable.
You can see from the examples that it is possible to receive arguments that do contain spaces if the user invoked the script with arguments encapsulated in quotes, so what you get is the list of arguments as supplied by the user.
Now in your Python code, you can use this list of strings as input to your program. Since lists are indexed by zero-based integers, you can get the individual items using the list[0] syntax. For example, to get the script name:
script_name = sys.argv[0] # this will always work.
Although interesting, you rarely need to know your script name. To get the first argument after the script for a filename, you could do the following:
filename = sys.argv[1]
This is a very common usage, but note that it will fail with an IndexError if no argument was supplied.
Also, Python lets you reference a slice of a list, so to get another list of just the user-supplied arguments (but without the script name), you can do
user_args = sys.argv[1:] # get everything after the script name
Additionally, Python allows you to assign a sequence of items (including lists) to variable names. So if you expect the user to always supply two arguments, you can assign those arguments (as strings) to two variables:
user_args = sys.argv[1:]
fun, games = user_args # len(user_args) had better be 2
So, to answer your specific question, sys.argv[1] represents the first command-line argument (as a string) supplied to the script in question. It will not prompt for input, but it will fail with an IndexError if no arguments are supplied on the command-line following the script name.

sys.argv[1] contains the first command line argument passed to your script.
For example, if your script is named hello.py and you issue:
$ python3.1 hello.py foo
or:
$ chmod +x hello.py # make script executable
$ ./hello.py foo
Your script will print:
Hello there foo

sys.argv is a list.
This list is created by your command line, it's a list of your command line arguments.
For example:
in your command line you input something like this,
python3.2 file.py something
sys.argv will become a list ['file.py', 'something']
In this case sys.argv[1] = 'something'

Just adding to Frederic's answer, for example if you call your script as follows:
./myscript.py foo bar
sys.argv[0] would be "./myscript.py"
sys.argv[1] would be "foo" and
sys.argv[2] would be "bar" ... and so forth.
In your example code, if you call the script as follows ./myscript.py foo , the script's output will be "Hello there foo".

Adding a few more points to Jason's Answer :
For taking all user provided arguments: user_args = sys.argv[1:]
Consider the sys.argv as a list of strings as (mentioned by Jason). So all the list manipulations will apply here. This is called "List Slicing". For more info visit here.
The syntax is like this: list[start:end:step]. If you omit start, it will default to 0, and if you omit end, it will default to length of list.
Suppose you only want to take all the arguments after 3rd argument, then:
user_args = sys.argv[3:]
Suppose you only want the first two arguments, then:
user_args = sys.argv[0:2] or user_args = sys.argv[:2]
Suppose you want arguments 2 to 4:
user_args = sys.argv[2:4]
Suppose you want the last argument (last argument is always -1, so what is happening here is we start the count from back. So start is last, no end, no step):
user_args = sys.argv[-1]
Suppose you want the second last argument:
user_args = sys.argv[-2]
Suppose you want the last two arguments:
user_args = sys.argv[-2:]
Suppose you want the last two arguments. Here, start is -2, that is second last item and then to the end (denoted by :):
user_args = sys.argv[-2:]
Suppose you want the everything except last two arguments. Here, start is 0 (by default), and end is second last item:
user_args = sys.argv[:-2]
Suppose you want the arguments in reverse order:
user_args = sys.argv[::-1]

sys.argv is a list containing the script path and command line arguments; i.e. sys.argv[0] is the path of the script you're running and all following members are arguments.

To pass arguments to your python script
while running a script via command line
> python create_thumbnail.py test1.jpg test2.jpg
here,
script name - create_thumbnail.py,
argument 1 - test1.jpg,
argument 2 - test2.jpg
With in the create_thumbnail.py script i use
sys.argv[1:]
which give me the list of arguments i passed in command line as
['test1.jpg', 'test2.jpg']

sys.argv is a attribute of the sys module. It says the arguments passed into the file in the command line. sys.argv[0] catches the directory where the file is located. sys.argv[1] returns the first argument passed in the command line. Think like we have a example.py file.
example.py
import sys # Importing the main sys module to catch the arguments
print(sys.argv[1]) # Printing the first argument
Now here in the command prompt when we do this:
python example.py
It will throw a index error at line 2. Cause there is no argument passed yet. You can see the length of the arguments passed by user using if len(sys.argv) >= 1: # Code.
If we run the example.py with passing a argument
python example.py args
It prints:
args
Because it was the first arguement! Let's say we have made it a executable file using PyInstaller. We would do this:
example argumentpassed
It prints:
argumentpassed
It's really helpful when you are making a command in the terminal. First check the length of the arguments. If no arguments passed, do the help text.

sys.argv will display the command line args passed when running a script or you can say sys.argv will store the command line arguments passed in python while running from terminal.
Just try this:
import sys
print sys.argv
argv stores all the arguments passed in a python list. The above will print all arguments passed will running the script.
Now try this running your filename.py like this:
python filename.py example example1
this will print 3 arguments in a list.
sys.argv[0] #is the first argument passed, which is basically the filename.
Similarly, argv[1] is the first argument passed, in this case 'example'.

Pass arguments from cmd to python script [duplicate]

This question already has answers here:
How to read/process command line arguments?
(22 answers)
Closed 2 years ago.
I write my scripts in python and run them with cmd by typing in:
C:\> python script.py
Some of my scripts contain separate algorithms and methods which are called based on a flag.
Now I would like to pass the flag through cmd directly rather than having to go into the script and change the flag prior to run, I want something similar to:
C:\> python script.py -algorithm=2
I have read that people use sys.argv for almost similar purposes however reading the manuals and forums I couldn't understand how it works.

There are a few modules specialized in parsing command line arguments: getopt, optparse and argparse. optparse is deprecated, and getopt is less powerful than argparse, so I advise you to use the latter, it'll be more helpful in the long run.
Here's a short example:
import argparse
# Define the parser
parser = argparse.ArgumentParser(description='Short sample app')
# Declare an argument (`--algo`), saying that the
# corresponding value should be stored in the `algo`
# field, and using a default value if the argument
# isn't given
parser.add_argument('--algo', action="store", dest='algo', default=0)
# Now, parse the command line arguments and store the
# values in the `args` variable
args = parser.parse_args()
# Individual arguments can be accessed as attributes...
print args.algo
That should get you started. At worst, there's plenty of documentation available on line (say, this one for example)...

It might not answer your question, but some people might find it usefull (I was looking for this here):
How to send 2 args (arg1 + arg2) from cmd to python 3:
----- Send the args in test.cmd:
python "C:\Users\test.pyw" "arg1" "arg2"
----- Retrieve the args in test.py:
print ("This is the name of the script= ", sys.argv[0])
print("Number of arguments= ", len(sys.argv))
print("all args= ", str(sys.argv))
print("arg1= ", sys.argv[1])
print("arg2= ", sys.argv[2])

Try using the getopt module. It can handle both short and long command line options and is implemented in a similar way in other languages (C, shell scripting, etc):
import sys, getopt
def main(argv):
# default algorithm:
algorithm = 1
# parse command line options:
try:
opts, args = getopt.getopt(argv,"a:",["algorithm="])
except getopt.GetoptError:
<print usage>
sys.exit(2)
for opt, arg in opts:
if opt in ("-a", "--algorithm"):
# use alternative algorithm:
algorithm = arg
print "Using algorithm: ", algorithm
# Positional command line arguments (i.e. non optional ones) are
# still available via 'args':
print "Positional args: ", args
if __name__ == "__main__":
main(sys.argv[1:])
You can then pass specify a different algorithm by using the -a or --algorithm= options:
python <scriptname> -a2 # use algorithm 2
python <scriptname> --algorithm=2 # ditto
See: getopt documentation

Parsing empty options in Python

I have an application that allows you to send event data to a custom script. You simply lay out the command line arguments and assign what event data goes with what argument. The problem is that there is no real flexibility here. Every option you map out is going to be used, but not every option will necessarily have data. So when the application builds the string to send to the script, some of the arguments are blank and python's OptionParser errors out with "error: --someargument option requires an argument"
Being that there are over 200 points of data, it's not like I can write separate scripts to handle each combination of possible arguments (it would take 2^200 scripts). Is there a way to handle empty arguments in python's optionparser?

Sorry, misunderstood the question with my first answer. You can accomplish the ability to have optional arguments to command line flags use the callback action type when you define an option. Use the following function as a call back (you will likely wish to tailor to your needs) and configure it for each of the flags that can optionally receive an argument:
import optparse
def optional_arg(arg_default):
def func(option,opt_str,value,parser):
if parser.rargs and not parser.rargs[0].startswith('-'):
val=parser.rargs[0]
parser.rargs.pop(0)
else:
val=arg_default
setattr(parser.values,option.dest,val)
return func
def main(args):
parser=optparse.OptionParser()
parser.add_option('--foo',action='callback',callback=optional_arg('empty'),dest='foo')
parser.add_option('--file',action='store_true',default=False)
return parser.parse_args(args)
if __name__=='__main__':
import sys
print main(sys.argv)
Running from the command line you'll see this:
# python parser.py
(<Values at 0x8e42d8: {'foo': None, 'file': False}>, [])
# python parser.py --foo
(<Values at 0x8e42d8: {'foo': 'empty', 'file': False}>, [])
# python parser.py --foo bar
(<Values at 0x8e42d8: {'foo': 'bar', 'file': False}>, [])

Yes, there is an argument to do so when you add the option:
from optparse import OptionParser
parser = OptionParser()
parser.add_option("--SomeData",action="store", dest="TheData", default='')
Give the default argument the value you want the option to have it is to be specified but optionally have an argument.

I don't think optparse can do this. argparse is a different (non-standard) module that can handle situations like this where the options have optional values.
With optparse you have to either have to specify the option including it's value or leave out both.

Optparse already allows you to pass the empty string as an option argument. So if possible, treat the empty string as "no value". For long options, any of the following work:
my_script --opt= --anotheroption
my_script --opt='' --anotheroption
my_script --opt="" --anotheroption
my_script --opt '' --anotheroption
my_script --opt "" --anotheroption
For short-style options, you can use either of:
my_script -o '' --anotheroption
my_script -o "" --anotheroption
Caveat: this has been tested under Linux and should work the same under other Unixlike systems; Windows handles command line quoting differently and might not accept all of the variants listed above.

Mark Roddy's solution would work, but it requires attribute modification of a parser object during runtime, and has no support for alternative option formattings other than - or --.
A slightly less involved solution is to modify the sys.argv array before running optparse and insert an empty string ("") after a switch which doesn't need to have arguments.
The only constraint of this method is that you have your options default to a predictable value other than the one you are inserting into sys.argv (I chose None for the example below, but it really doesn't matter).
The following code creates an example parser and set of options, extracts an array of allowed switches from the parser (using a little bit of instance variable magic), and then iterates through sys.argv, and every time it finds an
allowed switch, it checks to see if it was given without any arguments following it . If there is no argument after a switch, the empty string will be inserted on the command
line. After altering sys.argv, the parser is invoked, and you can check for options whose values are "", and act accordingly.
#Instantiate the parser, and add some options; set the options' default values to None, or something predictable that
#can be checked later.
PARSER_DEFAULTVAL = None
parser = OptionParser(usage="%prog -[MODE] INPUT [options]")
#This method doesn't work if interspersed switches and arguments are allowed.
parser.allow_interspersed_args = False
parser.add_option("-d", "--delete", action="store", type="string", dest="to_delete", default=PARSER_DEFAULTVAL)
parser.add_option("-a", "--add", action="store", type="string", dest="to_add", default=PARSER_DEFAULTVAL)
#Build a list of allowed switches, in this case ['-d', '--delete', '-a', '--add'] so that you can check if something
#found on sys.argv is indeed a valid switch. This is trivial to make by hand in a short example, but if a program has
#a lot of options, or if you want an idiot-proof way of getting all added options without modifying a list yourself,
#this way is durable. If you are using OptionGroups, simply run the loop below with each group's option_list field.
allowed_switches = []
for opt in parser.option_list:
#Add the short (-a) and long (--add) form of each switch to the list.
allowed_switches.extend(opt._short_opts + opt._long_opts)
#Insert empty-string values into sys.argv whenever a switch without arguments is found.
for a in range(len(sys.argv)):
arg = sys.argv[a]
#Check if the sys.argv value is a switch
if arg in allowed_switches:
#Check if it doesn't have an accompanying argument (i.e. if it is followed by another switch, or if it is last
#on the command line)
if a == len(sys.argv) - 1 or argv[a + 1] in allowed_switches:
sys.argv.insert(a + 1, "")
options, args = parser.parse_args()
#If the option is present (i.e. wasn't set to the default value)
if not (options.to_delete == PARSER_DEFAULTVAL):
if options.droptables_ids_csv == "":
#The switch was not used with any arguments.
...
else:
#The switch had arguments.
...

After checking that the cp command understands e.g. --backup=simple but not --backup simple, I answered the problem like this:
import sys
from optparse import OptionParser
def add_optval_option(pog, *args, **kwargs):
if 'empty' in kwargs:
empty_val = kwargs.pop('empty')
for i in range(1, len(sys.argv)):
a = sys.argv[i]
if a in args:
sys.argv.insert(i+1, empty_val)
break
pog.add_option(*args, **kwargs)
def main(args):
parser = OptionParser()
add_optval_option(parser,
'--foo', '-f',
default='MISSING',
empty='EMPTY',
help='"EMPTY" if given without a value. Note: '
'--foo=VALUE will work; --foo VALUE will *not*!')
o, a = parser.parse_args(args)
print 'Options:'
print ' --foo/-f:', o.foo
if a[1:]:
print 'Positional arguments:'
for arg in a[1:]:
print ' ', arg
else:
print 'No positional arguments'
if __name__=='__main__':
import sys
main(sys.argv)
Self-advertisement: This is part of the opo module of my thebops package ... ;-)

How can I process command line arguments in Python? [duplicate]

This question already has answers here:
How to read/process command line arguments?
(22 answers)
Closed last month.
What would be an easy expression to process command line arguments if I'm expecting anything like 001 or 999 (let's limit expectations to 001...999 range for this time), and few other arguments passed, and would like to ignore any unexpected?
I understand if for example I need to find out if "debug" was passed among parameters it'll be something like that:
if 'debug' in argv[1:]:
print 'Will be running in debug mode.'
How to find out if 009 or 575 was passed?
All those are expected calls:
python script.py
python script.py 011
python script.py 256 debug
python script.py 391 xls
python script.py 999 debug pdf
At this point I don't care about calls like that:
python script.py 001 002 245 568
python script.py some unexpected argument
python script.py 0001
python script.py 02
...first one - because of more than one "numeric" argument; second - because of... well, unexpected arguments; third and fourth - because of non-3-digits arguments.

As others answered, optparse is the best option, but if you just want quick code try something like this:
import sys, re
first_re = re.compile(r'^\d{3}$')
if len(sys.argv) > 1:
if first_re.match(sys.argv[1]):
print "Primary argument is : ", sys.argv[1]
else:
raise ValueError("First argument should be ...")
args = sys.argv[2:]
else:
args = ()
# ... anywhere in code ...
if 'debug' in args:
print 'debug flag'
if 'xls' in args:
print 'xls flag'
EDIT: Here's an optparse example because so many people are answering optparse without really explaining why, or explaining what you have to change to make it work.
The primary reason to use optparse is it gives you more flexibility for expansion later, and gives you more flexibility on the command line. In other words, your options can appear in any order and usage messages are generated automatically. However to make it work with optparse you need to change your specifications to put '-' or '--' in front of the optional arguments and you need to allow all the arguments to be in any order.
So here's an example using optparse:
import sys, re, optparse
first_re = re.compile(r'^\d{3}$')
parser = optparse.OptionParser()
parser.set_defaults(debug=False,xls=False)
parser.add_option('--debug', action='store_true', dest='debug')
parser.add_option('--xls', action='store_true', dest='xls')
(options, args) = parser.parse_args()
if len(args) == 1:
if first_re.match(args[0]):
print "Primary argument is : ", args[0]
else:
raise ValueError("First argument should be ...")
elif len(args) > 1:
raise ValueError("Too many command line arguments")
if options.debug:
print 'debug flag'
if options.xls:
print 'xls flag'
The differences here with optparse and your spec is that now you can have command lines like:
python script.py --debug --xls 001
and you can easily add new options by calling parser.add_option()

Have a look at the optparse module. Dealing with sys.argv yourself is fine for really simple stuff, but it gets out of hand quickly.
Note that you may find optparse easier to use if you can change your argument format a little; e.g. replace debug with --debug and xls with --xls or --output=xls.

optparse is your best friend for parsing the command line. Also look into argparse; it's not in the standard library, though.

If you want to implement actual command line switches, give getopt a look. It's incredibly simple to use, too.

Van Gale is largely correct in using the regular expression against the argument. However, it is NOT absolutely necessary to make everything an option when using optparse, which splits sys.argv into options and arguments, based on whether a "-" or "--" is in front or not. Some example code to go through just the arguments:
import sys
import optparse
claParser = optparse.OptionParser()
claParser.add_option(
(opts, args) = claParser.parse_args()
if (len(args) >= 1):
print "Arguments:"
for arg in args:
print " " + arg
else:
print "No arguments"
sys.exit(0)
Yes, the args array is parsed much the same way as sys.argv would be, but the ability to easily add options if needed has been added. For more about optparse, check out the relevant Python doc.