As described in the accepted answer to Most pythonic way of accepting arguments using optparse, I have a program with a function that performs operations on a string. The program uses argparse to check whether the string is provided as-is or in a file and massages the input as needed to pass it to the function.
Now I want to extend the program with a more advanced version of my function, but still leave the basic version in for comparison, somewhat as in Use argparse to run 1 of 2 functions in my script. Where I believe my situation differs is that regardless of the function that gets called, I want the option of also passing my existing input flags.
Just adding a new argument to the parser and nesting my previous code inside a if/else that checks for that flag doesn't work: it complains about the wrong number of arguments. I am aware of sub-commands, but I am still pretty new with argparse and it just seems like that would be overkill for what I want - but maybe not.
tl;dr: I need to choose one of two functions and one of two input types; both input types apply to both functions. Thanks for any help!
Edited to add code:
p = argparse.ArgumentParser(description="program.py")
p.add_argument("-e", dest='extended') #The new flag causing the trouble
p.add_argument("-s", dest="string")
p.add_argument("-f", dest="infile")
args = p.parse_args()
if args.extended:
if args.infile:
with open(args.infile,'r') as f:
for line in enumerate(f.readlines()):
print 'Input: ', line[1],
output = funcExtended(line[1]) #new and improved function
print 'Extended output: ', output
elif args.string:
output = funcExtended(args.string)
print output
else: #my future default option to grab strings from a database
print 'This will soon work: extended'
else: #I fully realize that I shouldn't have to essentially copy and paste here
if args.infile:
with open(args.infile,'r') as f:
for line in enumerate(f.readlines()):
print 'Input: ', line[1],
output = funcBasic(line[1]) #old and tired function
print 'Basic output: ', output
elif args.string:
output = funcBasic(args.string)
print output
else: #my future default option to grab strings from a database
print 'This will soon work: basic'
This is a command line utility. Issuing
$ python program.py -s 'string'
returns a properly formatted string, as before. But issuing
$ python program.py -s 'string' -e
returns
program.py: error: argument -e: expected one argument
Whew. Thanks again to anybody who can help!
If you change your extended argument to a boolean flag with
p.add_argument("-e", dest='extended', action="store_true")
it will no longer expect an argument. You can then invoke your script with
$ python program.py -e -s 'string'
Finally as a bonus here are some ideas to make your code less redundant:
import argparse
def funcExtended(line):
return " ".join(line)
def funcBasic(line):
return line.upper()
p = argparse.ArgumentParser(description="program.py")
p.add_argument("-e", "--extended", dest="func", action="store_const", const=funcExtended, default=funcBasic)
p.add_argument("-s", "--string")
p.add_argument("-f", "--infile")
args = p.parse_args()
def readlines(args):
if args.infile:
with open(args.infile,'r') as f:
for line in f:
yield line.rstrip("\n")
elif args.string:
yield args.string
else: #my future default option to grab strings from a database
print 'This will soon work: extended'
for line in readlines(args):
print 'Input: ', line
output = args.func(line)
print "Output: ", output
Related
My function convert.py is:
def convert(a,b)
factor = 2194.2
return (a-b)*factor
How do I run it from the command line with input arguments a and b?
I tried:
python convert.py 32 46
But got an error.
I did try to find the answer online, and I found related things but not the answer:
Run function from the command line (Stack Overflow)
How to read/process command line arguments? (Stack Overflow)
http://www.cyberciti.biz/faq/python-command-line-arguments-argv-example/
http://www.saltycrane.com/blog/2007/12/how-to-pass-command-line-arguments-to/
Also, where can I find the answer myself so that I can save this site for more non-trivial questions?
You could do:
import sys
def convert(a,b):
factor = 2194.2
return (a-b)*factor
print(convert(int(sys.argv[1]), int(sys.argv[2])))
If that is all what should do the script, you dont have to define a function:
import sys
factor = 2194.2
print((int(sys.argv[1]), int(sys.argv[2])*factor)
If you want change your file (nonetheless you have to add the colon after the function definiton), you could follow your first linked approach:
python -c 'import convert, sys; print convert.convert(int(sys.argv[1]), int(sys.argv[2])'
There exists a Python module for this sort of thing called argparse, which allows you to do really fancy things around command line flags. You don't really need that - you've just got two numbers on the command line. This can be handled really naively.
Python allows you direct access to the command line arguments via an array called sys.argv - you'll need to import sys first. The first element in this array is always the program name, but the second and third will be the numbers you pass in i.e. sys.argv[1] and sys.argv[2]. For a more complete example:
if len(sys.argv) < 3:
print 'Didnt supply to numbers'
a = int(sys.argv[1])
b = int(sys.argv[2])
Of course you'll need some error checking around making sure they are actuall integers/floats.
A bit of extra reading around sys.argv if you're interested here
To be complete, we can give an argparse example as well:
import argparse
parser = argparse.ArgumentParser(description='')
parser.add_argument('numbers', type=float, nargs=2,
help='Things to perform actions on')
args = parser.parse_args()
a = args.numbers[0]
b = args.numbers[1]
print a, b
So I have a fully functional py script running on Ubuntu 12.04, everything works great. Except I don't like my input methods, it's getting annoying as you'll see below. Before I type out the code, I should say that the code takes two images in a .img format and then does computations on them. Here's what I have:
import os
first = raw_input("Full path to first .img file: ")
second = raw_input("Full path to second .img file: ")
print " "
if os.path.exists(first) == True:
if first.endswith('.img') == False:
print 'You did not give a .img file, try running again'
os.sys.exit()
elif os.path.exists(second) == True:
if second.endswith('.img') == False:
print 'You did not give a .img file, try running again'
os.sys.exit()
else:
print "Your path does not exist, probably a typo. Try again"
os.sys.exit()
Here's what I want; I want to be able to feed python this input straight from the Terminal. In other words, I want to be able to input in the terminal something like
python myscript.py with the two images as input
This way I could make use of the terminal's tab-key shortcut when specifying paths and stuff. Any ideas/suggestions?
EDIT: Ok so I looked into the parsing, and I think I got down how to use it. Here's my code:
import argparse
import nipy
parser = argparse.ArgumentParser()
parser.add_argument("-im", "--image_input", help = "Feed the program an image", type = nipy.core.image.image.Image, nargs = 2)
however now I want to be able to use these files in the script by saying something like first = parser[0] second = parse[1] and do stuff on first and second. Is this achievable?
You want to parse the command line arguments instead of reading input after the program starts.
Use the argparse module for that, or parse sys.argv yourself.
Seeing that the parsing code already exists, all you need to do is accept command-line arguments with Python's sys module:
import sys
first = sys.argv[1]
second = sys.argv[2]
Or, more generally:
import os
import sys
if __name__ == '__main__':
if len(sys.argv) < 2:
print('USAGE: python %s [image-paths]' % sys.argv[0])
sys.exit(1)
image_paths = sys.argv[1:]
for image_path in image_paths:
if not os.path.exists(image_path):
print('Your path does not exist, probably a typo. Try again.')
sys.exit(1)
if image_path.endswith('.img'):
print('You did not give a .img file, try running again.')
sys.exit(1)
NOTES
The first part of the answer gives you what you need to accept command-line arguments. The second part introduces a few useful concepts for dealing with them:
When running a python file as a script, the global variable __name__ is set to '__main__'. If you use the if __name__ == '__main__' clause, you can either run the python file as a script (in which case the clause executes) or import it as a module (in which case it does not). You can read more about it here.
It is customary to print a usage message and exit if the script invocation was wrong.
The variable sys.argv is set to a list of the command-line arguments, and its first item is always the script path, so len(sys.argv) < 2 means no arguments were passed. If you want exactly two arguments, you can use len(sys.argv) != 3 instead.
sys.argv[1:] contains the actual command-line arguments. If you want exactly two arguments, you can reference them via sys.argv[1] and sys.argv[2] instead.
Please don't use if os.path.exists(...)==True and if string.endswith(...)==True syntax. It is much clearer and much more Pythonic to write if os.path.exists and if string.endswith(...) instead.
Using exit() without an argument defaults to exit(0), which means the program terminated successfully. If you are exiting with an error message, you should use exit(1) (or some other non-zero value...) instead.
What you want to do is take in command line parameters, and the best way to do that is using a nifty module called argparse. I have listed below a good resource on how to install and use it.
Here is a great resource for argparse. It is a module used to take command line arguments.
You can probably use sys.argv:
import sys
first = sys.argv[1]
second = sys.argv[2]
Don't forget to check len(sys.argv) before.
I have a file that I've made executable. It has a function in it that I would like to return its results to the command line but, I keep getting NameError messages. To break things down, I'm using LinuxMint Lisa and so far I have:
#! /usr/bin/env python
import mechanize
from BeautifulSoup import BeautifulSoup
import sys
def dictionary(word):
br = mechanize.Browser()
response = br.open('http://www.dictionary.reference.com')
br.select_form(nr=0)
br.form['q'] = word
br.submit()
definition = BeautifulSoup(br.response().read())
trans = definition.findAll('td',{'class':'td3n2'})
fin = [i.text for i in trans]
query = {}
for i in fin:
query[fin.index(i)] = i
return query
print dictionary(sys.argv)
Then I chmod from my terminal:
sudo chmod +x this_file.py
When I call this file from the command-line, I'll enter:
./this_file.py 'pass'(or any other string argument)
Which will return:
TypeError: Must assign a string
So I know I'm obviously not using sys.argv correctly but, I have a feeling like I'm mucking something else up when attempting to return this functions results to the command-line.
Ok, I might as well post this as my answer instead of just a comment:
In
print dictionary(agrv)
argv is misspelled.
It should be
print dictionary(sys.argv)
Also, use sys.argv, argv by itself won't suffice
argv is an attribute of the sys module
Use either
sys.argv
or do
from sys import argv
Shouldn't it have been print dictionary(sys.argv[1]). I guess you want to search the commandline argument in the dictionary.com
The problem with the question as currently posted is that sys.argv is a list, not a string, so when you set the form entry 'q' you are setting it to a list of arguments to the program. You could change the program to pass in the first argument:
print dictionary(sys.argv[1])
Or call the dictionary functions multiple times:
for i in sys.argv[1:]:
print dictionary(i)
Note that we don't want to include the program name itself so omit sys.argv[0].
You don't make it clear what you mean by 'returning' results to the command line. If you just want to print the results, you can do print.
But the error message you're getting has nothing to do with that. It's caused by two things: one, you've put agrv instead of argv. And second, you've imported sys, so you need to reference sys.argv not just argv.
I have an application that allows you to send event data to a custom script. You simply lay out the command line arguments and assign what event data goes with what argument. The problem is that there is no real flexibility here. Every option you map out is going to be used, but not every option will necessarily have data. So when the application builds the string to send to the script, some of the arguments are blank and python's OptionParser errors out with "error: --someargument option requires an argument"
Being that there are over 200 points of data, it's not like I can write separate scripts to handle each combination of possible arguments (it would take 2^200 scripts). Is there a way to handle empty arguments in python's optionparser?
Sorry, misunderstood the question with my first answer. You can accomplish the ability to have optional arguments to command line flags use the callback action type when you define an option. Use the following function as a call back (you will likely wish to tailor to your needs) and configure it for each of the flags that can optionally receive an argument:
import optparse
def optional_arg(arg_default):
def func(option,opt_str,value,parser):
if parser.rargs and not parser.rargs[0].startswith('-'):
val=parser.rargs[0]
parser.rargs.pop(0)
else:
val=arg_default
setattr(parser.values,option.dest,val)
return func
def main(args):
parser=optparse.OptionParser()
parser.add_option('--foo',action='callback',callback=optional_arg('empty'),dest='foo')
parser.add_option('--file',action='store_true',default=False)
return parser.parse_args(args)
if __name__=='__main__':
import sys
print main(sys.argv)
Running from the command line you'll see this:
# python parser.py
(<Values at 0x8e42d8: {'foo': None, 'file': False}>, [])
# python parser.py --foo
(<Values at 0x8e42d8: {'foo': 'empty', 'file': False}>, [])
# python parser.py --foo bar
(<Values at 0x8e42d8: {'foo': 'bar', 'file': False}>, [])
Yes, there is an argument to do so when you add the option:
from optparse import OptionParser
parser = OptionParser()
parser.add_option("--SomeData",action="store", dest="TheData", default='')
Give the default argument the value you want the option to have it is to be specified but optionally have an argument.
I don't think optparse can do this. argparse is a different (non-standard) module that can handle situations like this where the options have optional values.
With optparse you have to either have to specify the option including it's value or leave out both.
Optparse already allows you to pass the empty string as an option argument. So if possible, treat the empty string as "no value". For long options, any of the following work:
my_script --opt= --anotheroption
my_script --opt='' --anotheroption
my_script --opt="" --anotheroption
my_script --opt '' --anotheroption
my_script --opt "" --anotheroption
For short-style options, you can use either of:
my_script -o '' --anotheroption
my_script -o "" --anotheroption
Caveat: this has been tested under Linux and should work the same under other Unixlike systems; Windows handles command line quoting differently and might not accept all of the variants listed above.
Mark Roddy's solution would work, but it requires attribute modification of a parser object during runtime, and has no support for alternative option formattings other than - or --.
A slightly less involved solution is to modify the sys.argv array before running optparse and insert an empty string ("") after a switch which doesn't need to have arguments.
The only constraint of this method is that you have your options default to a predictable value other than the one you are inserting into sys.argv (I chose None for the example below, but it really doesn't matter).
The following code creates an example parser and set of options, extracts an array of allowed switches from the parser (using a little bit of instance variable magic), and then iterates through sys.argv, and every time it finds an
allowed switch, it checks to see if it was given without any arguments following it . If there is no argument after a switch, the empty string will be inserted on the command
line. After altering sys.argv, the parser is invoked, and you can check for options whose values are "", and act accordingly.
#Instantiate the parser, and add some options; set the options' default values to None, or something predictable that
#can be checked later.
PARSER_DEFAULTVAL = None
parser = OptionParser(usage="%prog -[MODE] INPUT [options]")
#This method doesn't work if interspersed switches and arguments are allowed.
parser.allow_interspersed_args = False
parser.add_option("-d", "--delete", action="store", type="string", dest="to_delete", default=PARSER_DEFAULTVAL)
parser.add_option("-a", "--add", action="store", type="string", dest="to_add", default=PARSER_DEFAULTVAL)
#Build a list of allowed switches, in this case ['-d', '--delete', '-a', '--add'] so that you can check if something
#found on sys.argv is indeed a valid switch. This is trivial to make by hand in a short example, but if a program has
#a lot of options, or if you want an idiot-proof way of getting all added options without modifying a list yourself,
#this way is durable. If you are using OptionGroups, simply run the loop below with each group's option_list field.
allowed_switches = []
for opt in parser.option_list:
#Add the short (-a) and long (--add) form of each switch to the list.
allowed_switches.extend(opt._short_opts + opt._long_opts)
#Insert empty-string values into sys.argv whenever a switch without arguments is found.
for a in range(len(sys.argv)):
arg = sys.argv[a]
#Check if the sys.argv value is a switch
if arg in allowed_switches:
#Check if it doesn't have an accompanying argument (i.e. if it is followed by another switch, or if it is last
#on the command line)
if a == len(sys.argv) - 1 or argv[a + 1] in allowed_switches:
sys.argv.insert(a + 1, "")
options, args = parser.parse_args()
#If the option is present (i.e. wasn't set to the default value)
if not (options.to_delete == PARSER_DEFAULTVAL):
if options.droptables_ids_csv == "":
#The switch was not used with any arguments.
...
else:
#The switch had arguments.
...
After checking that the cp command understands e.g. --backup=simple but not --backup simple, I answered the problem like this:
import sys
from optparse import OptionParser
def add_optval_option(pog, *args, **kwargs):
if 'empty' in kwargs:
empty_val = kwargs.pop('empty')
for i in range(1, len(sys.argv)):
a = sys.argv[i]
if a in args:
sys.argv.insert(i+1, empty_val)
break
pog.add_option(*args, **kwargs)
def main(args):
parser = OptionParser()
add_optval_option(parser,
'--foo', '-f',
default='MISSING',
empty='EMPTY',
help='"EMPTY" if given without a value. Note: '
'--foo=VALUE will work; --foo VALUE will *not*!')
o, a = parser.parse_args(args)
print 'Options:'
print ' --foo/-f:', o.foo
if a[1:]:
print 'Positional arguments:'
for arg in a[1:]:
print ' ', arg
else:
print 'No positional arguments'
if __name__=='__main__':
import sys
main(sys.argv)
Self-advertisement: This is part of the opo module of my thebops package ... ;-)
This question already has answers here:
How to read/process command line arguments?
(22 answers)
Closed last month.
What would be an easy expression to process command line arguments if I'm expecting anything like 001 or 999 (let's limit expectations to 001...999 range for this time), and few other arguments passed, and would like to ignore any unexpected?
I understand if for example I need to find out if "debug" was passed among parameters it'll be something like that:
if 'debug' in argv[1:]:
print 'Will be running in debug mode.'
How to find out if 009 or 575 was passed?
All those are expected calls:
python script.py
python script.py 011
python script.py 256 debug
python script.py 391 xls
python script.py 999 debug pdf
At this point I don't care about calls like that:
python script.py 001 002 245 568
python script.py some unexpected argument
python script.py 0001
python script.py 02
...first one - because of more than one "numeric" argument; second - because of... well, unexpected arguments; third and fourth - because of non-3-digits arguments.
As others answered, optparse is the best option, but if you just want quick code try something like this:
import sys, re
first_re = re.compile(r'^\d{3}$')
if len(sys.argv) > 1:
if first_re.match(sys.argv[1]):
print "Primary argument is : ", sys.argv[1]
else:
raise ValueError("First argument should be ...")
args = sys.argv[2:]
else:
args = ()
# ... anywhere in code ...
if 'debug' in args:
print 'debug flag'
if 'xls' in args:
print 'xls flag'
EDIT: Here's an optparse example because so many people are answering optparse without really explaining why, or explaining what you have to change to make it work.
The primary reason to use optparse is it gives you more flexibility for expansion later, and gives you more flexibility on the command line. In other words, your options can appear in any order and usage messages are generated automatically. However to make it work with optparse you need to change your specifications to put '-' or '--' in front of the optional arguments and you need to allow all the arguments to be in any order.
So here's an example using optparse:
import sys, re, optparse
first_re = re.compile(r'^\d{3}$')
parser = optparse.OptionParser()
parser.set_defaults(debug=False,xls=False)
parser.add_option('--debug', action='store_true', dest='debug')
parser.add_option('--xls', action='store_true', dest='xls')
(options, args) = parser.parse_args()
if len(args) == 1:
if first_re.match(args[0]):
print "Primary argument is : ", args[0]
else:
raise ValueError("First argument should be ...")
elif len(args) > 1:
raise ValueError("Too many command line arguments")
if options.debug:
print 'debug flag'
if options.xls:
print 'xls flag'
The differences here with optparse and your spec is that now you can have command lines like:
python script.py --debug --xls 001
and you can easily add new options by calling parser.add_option()
Have a look at the optparse module. Dealing with sys.argv yourself is fine for really simple stuff, but it gets out of hand quickly.
Note that you may find optparse easier to use if you can change your argument format a little; e.g. replace debug with --debug and xls with --xls or --output=xls.
optparse is your best friend for parsing the command line. Also look into argparse; it's not in the standard library, though.
If you want to implement actual command line switches, give getopt a look. It's incredibly simple to use, too.
Van Gale is largely correct in using the regular expression against the argument. However, it is NOT absolutely necessary to make everything an option when using optparse, which splits sys.argv into options and arguments, based on whether a "-" or "--" is in front or not. Some example code to go through just the arguments:
import sys
import optparse
claParser = optparse.OptionParser()
claParser.add_option(
(opts, args) = claParser.parse_args()
if (len(args) >= 1):
print "Arguments:"
for arg in args:
print " " + arg
else:
print "No arguments"
sys.exit(0)
Yes, the args array is parsed much the same way as sys.argv would be, but the ability to easily add options if needed has been added. For more about optparse, check out the relevant Python doc.