I'm new to using the arparse module in python and am hoping someone can help me with the following problem.
I am specifying a variable number of files as inputs using:
parser = argparse.ArgumentParser(description='Get Files')
parser.add_argument('-i','--input', help='Input file(s)',required=True, nargs='+')
args = parser.parse_args()
I would like to specify a variable number of file inputs, each with an associated value of 1 or 2 and am not sure how to do this.
I would like the program to work so that my command line entry should be:
MyProgram.py -i myfile.txt 2 secondfile.txt 1 ...
Once I have this working how do I call each file in the program?
Your code is functional. You could use the grouper recipe to loop through args.input two items at a time:
import argparse
parser = argparse.ArgumentParser(description='Get Files')
parser.add_argument('-i','--input', help='Input file(s)',required=True, nargs='+')
args = parser.parse_args()
for filename, num in zip(*[iter(args.input)]*2):
print(filename, num)
# with open(filename) as f:
# ....
yields
('myfile.txt', '2')
('secondfile.txt', '1')
It would be clearer to have -i once for each pair of inputs, like this:
parser.add_argument("-i", "--input", nargs=2, action='append')
Now, args.input would be a list of lists, like so
[ ['myfile.txt', 2], ['secondfile.txt', 1] ]
It does require slightly more typing for the user, since -i needs to be explicitly typed once per file.
Another option is to specify each argument as a single word, then parse the word using the type argument. I would get rid of the -i argument as well and use positional arguments for required "options".
parser.add_argument('input', nargs='+', type=lambda x: x.rsplit(":", 2))
Usage would be
myscript.py myfile.txt:1 secondfile.txt:2 ...
You might be best off using sys.argv as,
import sys, os
variables = list()
filenames = list()
if( len( sys.argv ) > 1 ):
if( sys.argv[1] == '-i' )or( sys.argv[1] == '--input' ):
N = len( sys.argv[1:] )
for i in range( 2, N, 2 ):
variables.append( int( sys.argv[i] ) )
filenames.append( sys.argv[i+1] )
for file in filenames:
# do something here
os.system( 'less '+file )
I haven't tested this, but that should work.
Try this:
parser = argparse.ArgumentParser(description='Get Files')
parser.add_argument('-i','--input', help='Input file(s)',
required=True, nargs='+', action='append')
args = parser.parse_args()
Related
I have the following files in a python pipeline
#in_para.py
nx = 31
c = 1
#solver.py
import numpy
import os
def simple(output):
ny = nx + 5
c_2 = c + 2
with open(os.path.join(output, 'Output.txt'), 'w') as f:
print("Hello stackoverflow!", file=f)
print("I have a question.", file=f)
if __name__=='__main__':
simple()
#main.py
import os
import numpy
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-C','--Chk',type=str, help='Choose arg')
parser.add_argument('-O','--output',type=str, default="./Output", help=' Provide the output Directory')
args = vars(parser.parse_args())
output = args['output']
if not os.path.exists(output):
os.makedirs(output)
if args['Chk'] == 'compo1':
simple(output)
if __name__=='__main__':
main()
I would like to call the input file in_para.py through command line argument such that
python3 main.py -I in_para -C compo1 -O Ouput_dir
gives me the desired output:
this is a simple test 36
this is a simple test2 3
I know if I do from in_para import * it will solve the problem (in a different manner), but I would like to call the input parameters from the command line as a positional argument and pass it on to solver.py. I haven't been able to find an example in the documentation similar to above task. Besides, the above is just an example, in_para.py and solver.py has several input parameters and several lines of code, so I dont want the user of main.py to go into either file and modify it.
Change the implementation in solver.py
def simple(nx, c, output): # provide parameters as function arguments
ny = nx + 5
c_2 = c + 2
print("this is a simple test", ny)
print("this is a simple test2", c_2)
with open(os.path.join(output, 'Output.txt'), 'w') as f:
print("Hello stackoverflow!", file=f)
print("I have a question.", file=f)
In the main parse the parameters file
... # other imports
import json
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-C','--Chk', type=str, help='Choose arg')
parser.add_argument('-I','--input', type=argparse.FileType("r"), help='Input parameters')
parser.add_argument('-O','--output', type=str, help='output dir')
args = parser.parse_args()
output = args.output
# implement any way of parsing the input file, e.g. a json file
params = json.load(args.input)
... # do something else with the arguments
if args.Chk == 'compo1':
simple(output=output, **params)
... # further code lines
Your inputs file would look like
{"nx": 31, "c": 1}
Comment on edit:
Either you use args = vars(parser.parse_args()) which results in a dictionary or you just use the args = parser.parse_args() to select arguments by attribute. In your original post you used the first approach which I adopted in the first answer. However, I would prefer not using vars.
Note, that the code snippet to create the directory does not add value to the actual question of the post, so I removed it to avoid any distraction and added some placeholder instead.
I'm new to python and currently playing around with argpase. I'm trying to call a function using a directory path given as a command line argument followed by an argparse option(-name) and a regex that goes through all the files in the directory and spits out all the matches to the regex as so:
./find.py ../seek -name '[a-z]*\.txt'
However, I'm getting a error that looks like
usage: find.py [-h] [--path PATH] [-name] [--regex REGEX]
find.py: error: unrecognized arguments: . . / s e e k / p r o g r a m . c
And without the -name its just printing all the files inside the path.
Here is what I have so far:
#!/usr/bin/python2.7
import os, sys, argparse,re
from stat import *
def parse(argv=None):
parser = argparse.ArgumentParser()
parser.add_argument('--path', help='path of directory', action='store')
parser.add_argument('-name', '--name', action='store_true')
parser.add_argument('--regex', default=r"[a-z0-9A-Z]")
args = parser.parse_args(argv)
print(args)
return args
def main(argv=None):
direc = sys.argv[1]
files = []
for f in os.listdir(direc):
pathname = os.path.join(direc, f)
mode = os.stat(pathname).st_mode
if S_ISREG(mode):
args = parse(pathname)
if args.name:
dirls = [re.match(args.regex, pathname)]
print(dirls)
else:
print pathname
if __name__ == '__main__':main()
Any thoughts?
Argument Parser PATH Example : Different type of arguments with custom handlers added. For path here you can pass '-path' followed by path value as argument
import os
import argparse
from datetime import datetime
def parse_arguments():
parser = argparse.ArgumentParser(description='Process command line arguments.')
parser.add_argument('-path', type=dir_path)
parser.add_argument('-e', '--yearly', nargs = '*', help='yearly date', type=date_year)
parser.add_argument('-a', '--monthly', nargs = '*',help='monthly date', type=date_month)
return parser.parse_args()
def dir_path(path):
if os.path.isdir(path):
return path
else:
raise argparse.ArgumentTypeError(f"readable_dir:{path} is not a valid path")
def date_year(date):
if not date:
return
try:
return datetime.strptime(date, '%Y')
except ValueError:
raise argparse.ArgumentTypeError(f"Given Date({date}) not valid")
def date_month(date):
if not date:
return
try:
return datetime.strptime(date, '%Y/%m')
except ValueError:
raise argparse.ArgumentTypeError(f"Given Date({date}) not valid")
def main():
parsed_args = parse_arguments()
if __name__ == "__main__":
main()
In order for your program to operate, you need a path. So, the --path option must take an argument. Modify your parse() function to change the line
parser.add_argument('--path', help='path of directory', action='store')
to
parser.add_argument('--path', help='path of directory', action='store', required=True)
You need to call parse_args() only once. Remove the parse() invocation to the top of the loop.
And you needn't do
direc = sys.argv[1]
if you are using argparse.
re.match() returns a match object, which is probably not what you want to print.
You might want to take a look at match() versus search().
The match() function only checks if the RE matches at the beginning of the string while search() will scan forward through the string for a match.
If you wanted to print the file names matching the regex, you could do
if S_ISREG(mode):
#args = parse(pathname)
if args.name:
#dirls = re.match(args.regex, pathname)
dirls = re.search(args.regex, pathname)
if( dirls ):
print(pathname)
else:
print pathname
So main() should be something like
def main(argv=None):
args = parse(sys.argv[1:])
print(args)
#direc = sys.argv[1]
direc = args.path
files = []
for f in os.listdir(direc):
pathname = os.path.join(direc, f)
mode = os.stat(pathname).st_mode
if S_ISREG(mode):
#args = parse(pathname)
if args.name:
#dirls = re.match(args.regex, pathname)
dirls = re.search(args.regex, pathname)
if( dirls ):
print(pathname)
else:
print pathname
In order to specify the regex matching the file names, you must specify the regex using the --regex option. By default, you've made it to match names having only numbers and (English) letters.
./find.py --path ../seek -name --regex [a-z]\*.txt
or
./find.py --path ../seek -name --regex '[a-z]*.txt'
When I specify the -a switch to add to config file for some reason this calls the read_config file also. For example I use ./script.py -a newfile.txt and the file is added correctly but then returns "Config File not found".
parser = argparse.ArgumentParser(description='Copy multiple Files from a specified data file')
parser.add_argument('-c', '--configfile', default="config.dat", help='file to read the config from')
parser.add_argument('-l', '--location', default="/home/admin/Documents/backup/",help='Choose location to store files')
parser.add_argument('-a', '--add', help='add new line to config data')
def read_config(data):
try:
dest = '/home/admin/Documents/backup/'
# Read in date from config.dat
data = open(data)
# Interate through list of files '\n'
filelist = data.read().split('\n')
# Copy through interated list and strip white spaces and empty lines
for file in filelist:
if file:
shutil.copy(file.strip(), dest)
except FileNotFoundError:
logger.error("Config file not found")
print ("Config File not found")
Not quite sure why this section below is working but read_config is called and finding the except error. Where am I going wrong?
def addtofile(add):
f = open('config.dat', "a")
f.write(add + '\n')
f.close()
args = vars(parser.parse_args())
read = read_config(args['configfile'])
add = addtofile(args['add'])
parser = argparse.ArgumentParser()
parser.add_argument('-c', action='store_true')
a = parser.parse_args()
if a.c:
{
print("get custom config")
}
else:
{
print("using default config file")
}
I think your solution will be:
parser = argparse.ArgumentParser()
parser.add_argument('-c', action='store_true')
a = parser.parse_args()
if a.c:
{
print("get custom config")
}
else:
{
print("using default config file")
}
I am trying to parse command line arguments and I have never done it before. I have tried a few things. The code below is the last thing that I tried, but I received the error: "unrecognized arguments". I want to be able to put something like
copy-column-csv.py cname=Quiz source=source.csv target=target.csv out=out.csv
on the command line, but also have the ability to have it in any order. I am not sure how to go about this. Below is my code:
import argparse
import sys, re
import numpy as np
import smtplib
from random import randint
import csv
import math
import pandas as pd
parser = argparse.ArgumentParser()
parser.add_argument('-cname')
parser.add_argument('-source')
parser.add_argument('-target')
parser.add_argument('-out')
args = parser.parse_args()
#col = sys.argv[1]
#source = sys.argv[2]
#target = sys.argv[3]
#newtarg = sys.argv[4]
sourceFile = pd.read_csv(source)
targetFile = pd.read_csv(target)
del targetFile[cname]
targetFile[col] = pd.Series(sourceFile[col])
targetFile.to_csv(out, index = False)
Assuming well formed arguments, you could split sys.argv up into a dictionary:
args_dict = {}
for arg in sys.argv[1:]:
split = arg.split('=')
args_dict[split[0]] = split[1]
args_dict will look like this:
{'cname': 'Quiz',
'out': 'out.csv',
'source': 'source.csv',
'target': 'target.csv'}
And you can access the elements like so:
print args_dict['cname']
print args_dict['out']
print args_dict['source']
print args_dict['target']
Here is an example of how I'm doing it.
from optparse import OptionParser
from optparse import OptionGroup
parser = OptionParser(usage)
required = OptionGroup(parser, "Required Arguments")
required.add_option("--genome", dest="genome_file", help="File representing genome. FASTA nucleotide format. ")
required.add_option("--anno", dest="anno_file", help="File containing genome annotation information in GTF/GFF3 format. ")
required.add_option("--output", dest="prefix", help="Creates a folder named with the supplied prefix containing output files. ")
parser.add_option_group(required)
if len(args) != 0 or not options.genome_file or not options.anno_file or not options.prefix:
parser.error("Required arguments have not been supplied. \n\t\tUse -h to get more information.")
sys.exit()
You'll pass the arguments when you run it, for example, with --genome=myfile.txt, and in the code that value becomes options.genome_file.
I have been using argparse in a program I am writing however it doesnt seem to create the stated output file.
My code is:
parser.add_argument("-o", "--output", action='store', dest='output', help="Directs the output to a name of your choice")
with open(output, 'w') as output_file:
output_file.write("%s\n" % item)
I have also tried:
parser.add_argument("-o", "--output", action='store', type=argparse.FileType('w'), dest='output', help="Directs the output to a name of your choice")
output_file.write("%s\n" % item)
The error that occurs is :
output_file.write("%s\n" % item)
NameError: name 'output_file' is not defined
Can someone please explain why I am having this error occuring and how I could solve it?
All my code:
from __future__ import print_function
from collections import defaultdict
from itertools import groupby
import argparse #imports the argparse module so it can be used
from itertools import izip
#print = print_function
parser = argparse.ArgumentParser() #simplifys the wording of using argparse as stated in the python tutorial
parser.add_argument("-r1", type=str, action='store', dest='input1', help="input the forward read file") # allows input of the forward read
parser.add_argument("-r2", type=str, action='store', dest='input2', help="input the reverse read file") # allows input of the reverse read
parser.add_argument("-v", "--verbose", action="store_true", help=" Increases the output, only needs to be used to provide feedback to Tom for debugging")
parser.add_argument("-n", action="count", default=0, help="Allows for up to 5 mismatches, however this will reduce accuracy of matching and cause mismatches. Default is 0")
#parser.add_argument("-o", "--output", action='store', type=argparse.FileType('w'), dest='output', help="Directs the output to a name of your choice")
parser.add_argument("-fastq", action="store_true", help=" States your input as fastq format")
parser.add_argument("-fasta", action="store_true", help=" States your input as fasta format")
parser.add_argument("-o", "--output", action='store', dest='output', help="Directs the output to a name of your choice")
args = parser.parse_args()
def class_chars(chrs):
if 'N' in chrs:
return 'unknown'
elif chrs[0] == chrs[1]:
return 'match'
else:
return 'not_match'
with open(output, 'w') as output_file:
s1 = 'aaaaaaaaaaN123bbbbbbbbbbQccc'
s2 = 'aaaaaaaaaaN456bbbbbbbbbbPccc'
n = 0
consec_matches = []
chars = defaultdict(int)
for k, group in groupby(zip(s1, s2), class_chars):
elems = len(list(group))
chars[k] += elems
if k == 'match':
consec_matches.append((n, n+elems-1))
n += elems
print (chars)
print (consec_matches)
print ([x for x in consec_matches if x[1]-x[0] >= 9])
list = [x for x in consec_matches if x[1]-x[0] >= 9]
flatten_list= [x for y in list for x in y]
print (flatten_list)
matching=[y[1] for y in list for x in y if x ==0 ]
print (matching)
magic = lambda matching: int(''.join(str(i) for i in matching)) # Generator exp.
print (magic(matching))
s2_l = s2[magic(matching):]
line3=s1+s2_l
print (line3)
if line3:
output_file.write("%s\n" % item)
You are missing the bit where the arguments are actually parsed:
parser.add_argument("-o", "--output", help="Directs the output to a name of your choice")
args = parser.parse_args()
with open(args.output, 'w') as output_file:
output_file.write("%s\n" % item)
parser.parse_args() will give you an object from which you can access the arguments by name using the long option name bar the dashes.
I think you almost had the most correct answer. The only problem is output_file was not read from the args:
parser.add_argument("-o", "--output", action='store',
type=argparse.FileType('w'), dest='output',
help="Directs the output to a name of your choice")
#output_file is not defined, you want to read args.output to get the output_file
output_file = args.output
#now you can write to it
output_file.write("%s\n" % item)
When I run your script I get:
Traceback (most recent call last):
File "stack23566970.py", line 31, in <module>
with open(output, 'w') as output_file:
NameError: name 'output' is not defined
There's no place in your script that does output = ....
We can correct that with:
with open(args.output, 'w') as output_file:
argparse returns values as attributes of the args object.
Now I get:
Traceback (most recent call last):
File "stack23566970.py", line 62, in <module>
output_file.write("%s\n" % item)
NameError: name 'item' is not defined
Again, there's no item = ... line.
What is item supposed to be?