We Keep Coding

argparse command line -option after given path

I'm new to python and currently experimenting using argparse to add command line options. However my code is not working, despite looking at various online tutorials and reading up on argparse I still don't fully understand it. My problem is whenever I try to call my -option it gives me a find.py error: argument regex:
Here is my call:
./find.py ../Python -name '[0-9]*\.txt'
../Python is one directory behind my current one and has a list of files/directories. Without the -name option I print out the files with their path (this works fine) but with the -name option I want to print out files matching the regex but it won't work. Here is what I currently have:
#!/usr/bin/python2.7
import os, sys, argparse,re
from stat import *
def regex_type(s, pattern=re.compile(r"[a-f0-9A-F]")):
if not pattern.match(s):
raise argparse.ArgumentTypeError
return s
def main():
direc = sys.argv[1]
for f in os.listdir(direc):
pathname = os.path.join(direc, f)
mode = os.stat(pathname).st_mode
if S_ISREG(mode):
print pathname
parser = argparse.ArgumentParser()
parser.add_argument(
'-name', default=[sys.stdin], nargs="*")
parser.add_argument('regex', type=regex_type)
args = parser.parse_args()
if __name__ == '__main__':
main()

I tweaked your type function to be more informative:
def regex_type(s, pattern=re.compile(r"[a-f0-9A-F]")):
print('regex string', s)
if not pattern.match(s):
raise argparse.ArgumentTypeError('pattern not match')
return s
Called with
2104:~/mypy$ python2 stack50072557.py .
I get:
<director list>
('regex string', '.')
usage: stack50072557.py [-h] [-name [NAME [NAME ...]]] regex
stack50072557.py: error: argument regex: pattern not match
So it tries to pass sys.argv[1], the first string after the script name, to the regex_type function. If it fails it issues the error message and usage.
OK, the problem was the ..; I'll make a directory:
2108:~/mypy$ mkdir foo
2136:~/mypy$ python2 stack50072557.py foo
('regex string', 'foo')
Namespace(name=[<open file '<stdin>', mode 'r' at 0x7f3bea2370c0>], regex='foo')
2138:~/mypy$ python2 stack50072557.py foo -name a b c
('regex string', 'foo')
Namespace(name=['a', 'b', 'c'], regex='foo')
The strings following '-name' are allocated to that attribute. There's nothing in your code that will test them or pass them through the regex_type function. Only the first non-flag string does that.
Reading sys.argv[1] initially does not remove it from the list. It's still there for use by the parser.
I would set up a parser that uses a store_true --name argument, and 2 positionals - one for the dir and the other for regex.
After parsing check args.name. If false print the contents of args.dir. If true, perform your args.regex filter on those contents. glob might be useful.
The parser finds out what your user wants. Your own code acts on it. Especially as a beginner, it is easier and cleaner to separate the two steps.
With:
def parse(argv=None):
parser = argparse.ArgumentParser()
parser.add_argument('-n', '--name', action='store_true')
parser.add_argument('--dir', default='.')
parser.add_argument('--regex', default=r"[a-f0-9A-F]")
args = parser.parse_args(argv)
print(args)
return args
def main(argv=None):
args = parse(argv)
dirls = os.listdir(args.dir)
if args.name:
dirls = [f for f in dirls if re.match(args.regex, f)]
print(dirls)
else:
print(dirls)
I get runs like:
1005:~/mypy$ python stack50072557.py
Namespace(dir='.', name=False, regex='[a-f0-9A-F]')
['test.npz', 'stack49909128.txt', 'stack49969840.txt', 'stack49824248.py', 'test.h5', 'stack50072557.py', 'stack49963862.npy', 'Mcoo.npz', 'test_attribute.h5', 'stack49969861.py', 'stack49969605.py', 'stack49454474.py', 'Mcsr.npz', 'Mdense.npy', 'stack49859957.txt', 'stack49408644.py', 'Mdok', 'test.mat5', 'stack50012754.py', 'foo', 'test']
1007:~/mypy$ python stack50072557.py -n
Namespace(dir='.', name=True, regex='[a-f0-9A-F]')
['foo']
1007:~/mypy$ python stack50072557.py -n --regex='.*\.txt'
Namespace(dir='.', name=True, regex='.*\\.txt')
['stack49909128.txt', 'stack49969840.txt', 'stack49859957.txt']
and help:
1007:~/mypy$ python stack50072557.py -h
usage: stack50072557.py [-h] [-n] [--dir DIR] [--regex REGEX]
optional arguments:
-h, --help show this help message and exit
-n, --name
--dir DIR
--regex REGEX
If I change the dir line to:
parser.add_argument('dir', default='.')
help is now
1553:~/mypy$ python stack50072557.py -h
usage: stack50072557.py [-h] [-n] [--regex REGEX] dir
positional arguments:
dir
optional arguments:
-h, --help show this help message and exit
-n, --name
--regex REGEX
and runs are:
1704:~/mypy$ python stack50072557.py -n
usage: stack50072557.py [-h] [-n] [--regex REGEX] dir
stack50072557.py: error: too few arguments
1705:~/mypy$ python stack50072557.py . -n
Namespace(dir='.', name=True, regex='[a-f0-9A-F]')
['foo']
1705:~/mypy$ python stack50072557.py ../mypy -n --regex='.*\.txt'
Namespace(dir='../mypy', name=True, regex='.*\\.txt')
['stack49909128.txt', 'stack49969840.txt', 'stack49859957.txt']
I get the error because it now requires a directory, even it is '.'.
Note that the script still uses:
if __name__ == '__main__':
main()
My main loads the dir, and applies the regex filter to that list of names. My args.dir replaces your direc.

Python command line arguments - hope to print error when no argument

I have a following simple code:
import sys, getopt
ifile=''
ofile=''
try:
opts, args = getopt.getopt(sys.argv[1:],"h:i:o:")
except getopt.GetoptError as e:
print (str(e))
print("test.py -i input -o output")
sys.exit(2)
for o, a in opts:
if o == '-h':
print 'test.py -i input -o output'
sys.exit()
elif o == '-i':
ifile=a
elif o == '-o':
ofile=a
What should I need to add, if I want to print error (and also help) message 'test.py -i input -o output' when I execute just the script like:
$ python test.py
Thank you

You can write
if len(sys.argv) <= 1:
print('test.py -i input -o output')
exit(1)
after the imports, which basically means that if i don't have any arguments, print the message and quit running the script.

Just as an alternative, if you are interested, the documentation for getopts actually provides a suggestion to use argparse, which significantly reduces the lines of code you have to write to handle arguments.
Look at the bottom of the doc here:
https://docs.python.org/2/library/getopt.html#getopt.GetoptError
And here is the documentation for argparse
https://docs.python.org/2/library/argparse.html#module-argparse
The big bonus with argparse is that has a built in "help" that is nicely formatted. Look at the following example. You can take this code and test on your end too.
from argparse import ArgumentParser, RawTextHelpFormatter
parser = ArgumentParser(
description='This application will perform actions',
formatter_class=RawTextHelpFormatter
)
parser.add_argument(
'-i',
help='Things with i',
)
parser.add_argument(
'-o',
help='Things with o',
)
args = vars(parser.parse_args())
if args.get('i'):
print(args.get('i'))
elif args.get('o'):
print(args.get('o'))
else:
parser.error('Invalid options provided')
Just thought I'd share as an alternative.

how show error if no args was set in console trying execute python script

I need to write python script which need command arguments for executing. I have such code:
try:
opts, args = getopt.getopt(argv,"htt:tf:d:",["from=","to=","on="])
except getopt.GetoptError:
print 'logReader.py -f <from> -t <to> -d <on>'
sys.exit(2)
for opt, arg in opts:
if opt == '-h':
print 'logReader.py -f <from> -t <to> -d <on>'
sys.exit()
# elif opt
elif opt in ("-f", "--from"):
fromTime = arg
elif opt in ("-t", "--to"):
toTime = arg
elif opt in ("-d", "--on"):
onDate = arg
But than I run my script without any arguments it just do nothing. How can I add some check like if no args are specified the error message should be shown in the console (or help)

With no arguments set, opts will be an empty list:
if not opts:
print 'logReader.py -f <from> -t <to> -d <on>'
sys.exit(2)
You should really consider using the argparse module instead, which lets you specify that some command line options are mandatory.

Flexible UNIX command line interface with Python

I was wondering how to create a flexible CLI interface with Python. So far I have come up with the following:
$ cat cat.py
#!/usr/bin/env python
from sys import stdin
from fileinput import input
from argparse import ArgumentParser, FileType
def main(args):
for line in input():
print line.strip()
if __name__ == "__main__":
parser = ArgumentParser()
parser.add_argument('FILE', nargs='?', type=FileType('r'), default=stdin)
main(parser.parse_args())
This handles both stdin and file input:
$ echo 'stdin test' | ./cat.py
stdin test
$ ./cat.py file
file test
The problem is it doesn't handle multiple input or no input the way I would like:
$ ./cat.py file file
usage: cat.py [-h] [FILE]
cat.py: error: unrecognized arguments: file
$ ./cat.py
For multiple inputs it should cat the file multiple times and for no input input should ideally have same the behaviour as -h:
$ ./cat.py -h
usage: cat.py [-h] [FILE]
positional arguments:
FILE
optional arguments:
-h, --help show this help message and exit
Any ideas on creating a flexible CLI interface with Python?

Use nargs='*' to allow for 0 or more arguments:
if __name__ == "__main__":
parser = ArgumentParser()
parser.add_argument('FILE', nargs='*', type=FileType('r'), default=stdin)
main(parser.parse_args())
The help output now is:
$ bin/python cat.py -h
usage: cat.py [-h] [FILE [FILE ...]]
positional arguments:
FILE
optional arguments:
-h, --help show this help message and exit
and when no arguments are given, stdout is used.
If you want to require at least one FILE argument, use nargs='+' instead, but then the default is ignored, so you may as well drop that:
if __name__ == "__main__":
parser = ArgumentParser()
parser.add_argument('FILE', nargs='+', type=FileType('r'))
main(parser.parse_args())
Now not specifying a command-line argument gives:
$ bin/python cat.py
usage: cat.py [-h] FILE [FILE ...]
cat.py: error: too few arguments
You can always specify stdin still by passing in - as an argument:
$ echo 'hello world!' | bin/python cat.py -
hello world!

A pretty good CLI interface the handles file input, standard input, no input, file output and inplace editing:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
def main(args, help):
'''
Simple line numbering program to demonstrate CLI interface
'''
if not (select.select([sys.stdin,],[],[],0.0)[0] or args.files):
help()
return
if args.output and args.output != '-':
sys.stdout = open(args.output, 'w')
try:
for i, line in enumerate(fileinput.input(args.files, inplace=args.inplace)):
print i + 1, line.strip()
except IOError:
sys.stderr.write("%s: No such file %s\n" %
(os.path.basename(__file__), fileinput.filename()))
if __name__ == "__main__":
import os, sys, select, argparse, fileinput
parser = argparse.ArgumentParser()
parser.add_argument('files', nargs='*', help='input files')
group = parser.add_mutually_exclusive_group()
group.add_argument('-i', '--inplace', action='store_true',
help='modify files inplace')
group.add_argument('-o', '--output',
help='output file. The default is stdout')
main(parser.parse_args(), parser.print_help)
The code simply emulates nl and numbers the lines but should serve as a good skeleton for many applications.

Pass bash argument to python script

I am trying to create a bash script which passes its own argument onto a python script. I want it to work like this.
If I run it as this:
script.sh latest
Then within the bash script it runs a python script with the "latest" argument like this:
python script.py latest
Likewise if the bash script is run with the argument 123 then the python script as such:
python script.py 123
Can anyone help me understand how to accomplish this please?

In this case the trick is to pass however many arguments you have, including the case where there are none, and to preserve any grouping that existed on the original command line.
So, you want these three cases to work:
script.sh # no args
script.sh how now # some number
script.sh "how now" "brown cow" # args that need to stay quoted
There isn't really a natural way to do this because the shell is a macro language, so they've added some magic syntax that will just DTRT.
#!/bin/sh
python script.py "$#"

In the pythonscript script.py use getopt.getopt(args, options[, long_options]) to get the arguments.
Example:
import getopt, sys
def main():
try:
opts, args = getopt.getopt(sys.argv[1:], "ho:v", ["help", "output="])
except getopt.GetoptError as err:
# print help information and exit:
print str(err) # will print something like "option -a not recognized"
usage()
sys.exit(2)
output = None
verbose = False
for o, a in opts:
if o == "-v":
verbose = True
elif o in ("-h", "--help"):
usage()
sys.exit()
elif o in ("-o", "--output"):
output = a
else:
assert False, "unhandled option"
# ...
if __name__ == "__main__":
main()

A very goo buit-in parser is argparse. Yo can use it as follows:
import argparse
parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('integers', metavar='N', type=int, nargs='+',
help='an integer for the accumulator')
parser.add_argument('--sum', dest='accumulate', action='store_const',
const=sum, default=max,
help='sum the integers (default: find the max)')
args = parser.parse_args()
print(args.accumulate(args.integers))

In bash, arguments passed to the script are accessed with the $# notation (# being a number. Using $# exactly like that should give you the number of args passed). So if you wanted to pass arguments:
Calling the script:
`#script.sh argument`
Within the script:
python script.py "$1"