We Keep Coding

Passing directory paths as strings to argparse in Python

Scenario: I have a python script that receives as inputs 2 directory paths (input and output folders) and a variable ID. With these, it performs a data gathering procedure from xlsx and xlsm macros, modifies the data and saves to a csv (from the input folder, the inner functions of the code will run loops, to get multiple files and process them, one at a time).
Issue: Since the code was working fine when I was running it from the Spyder console, I decided to step it up and learn about cmd caller, argparse and the main function. I trying to implement that, but I get the following error:
Unrecognized arguments (the output path I pass from cmd)
Question: Any ideas on what I am doing wrong?
Obs: If the full script is required, I can post it here, but since it works when run from Spyder, I believe the error is in my argparse function.
Code (argparse function and __main__):
# This is a function to parse arguments:
def parserfunc():
import argparse
parser = argparse.ArgumentParser(description='Process Files')
parser.add_argument('strings', nargs=3)
args = parser.parse_args()
arguments = args.strings
return arguments
# This is the main caller
def main():
arguments = parserfunc()
# this next function is where I do the processing for the files, based on the paths and id provided):
modifierfunc(arguments[0], arguments[1], arguments[2])
#
if __name__ == "__main__":
main()

If you decided to use argparse, then make use of named arguments, not indexed. Following is an example code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('input')
parser.add_argument('output')
parser.add_argument('id')
args = parser.parse_args()
print(args.input, args.output, args.id) # this is how you use them
In case you miss one of them on program launch, you will get human readable error message like
error: the following arguments are required: id

You could drop the entire parserfunc() function.
sys.argv does indeed contain all arguments (always processed as a string) as mentioned by grapes.
So instead of this:
modifierfunc(arguments[0], arguments[1], arguments[2])
This should suffice:
import sys
modifierfunc(sys.argv[0], sys.argv[1], sys.argv[2])
Perhaps, first do a print, to see if the sys.argv holds the values you expect.
print('Argument 0='+sys.argv[0])
print('Argument 1='+sys.argv[1])
print('Argument 2='+sys.argv[2])

Unittest with command-line arguments

From what I understand from another SO post, to unittest a script that takes command line arguments through argparse, I should do something like the code below, giving sys.argv[0] as arg.
import unittest
import match_loc
class test_method_main(unittest.TestCase):
loc = match_loc.main()
self.assertEqual(loc, [4])
if __name__ == '__main__':
sys.argv[1] = 'aaaac'
sys.argv[2] = 'ac'
unittest.main(sys.argv[0])
This returns the error:
usage: test_match_loc.py [-h] text patterns [patterns ...]
test_match_loc.py: error: the following arguments are required: text, patterns
I would like to understand what is going on here deeper. I understand
if __name__ == '__main__':
main()
says that if this is being executed by the 'main', highest level, default interpreter, to just automatically run the 'main' method. I'm assuming
if __name__ == '__main__':
unittest.main()
just happens to be the way you say this for running unittest scripts.
I understand when any script is run, it automatically has an argv object, a vector collecting all the items on the command line.
But I do not understand what unittest.main(sys.arg[0]) would do. What does 'unittest.main' do with arguments? How can I pre-set the values of sys.argv - doesn't it automatically reset every time you run a script? Furthermore, where does this object 'sys.argv' exist, if outside of any script? Finally, what is the correct way to implement tests of command-line arguments?
I am sorry if my questions are vague and misguided. I would like to understand all the components relevant here so I can actually understand what I am doing.
Thank you very much.

Just by playing around with a pair of simple files, I find that modifying sys.argv in the body of the caller module affects the sys.argv that the imported module sees:
import sys
sys.argv[1] = 'aaaac'
sys.argv[2] = 'ac'
class test_method_main(unittest.TestCase):
...
But modifying sys.argv in the main block as you do, does not show up in the imported one. We could dig into the documentation (and code) to see exactly why, but I think it's enough to just identify what works.
Here's what I reconstructed from your previous question of the imported module - with a few diagnostic prints
import argparse
import sys
def main():
print(sys.argv)
parser = argparse.ArgumentParser(
description='Takes a series of patterns as fasta files'
' or strings and a text as fasta file or string and'
' returns the match locations by constructing a trie.')
parser.add_argument('text')
parser.add_argument('patterns', nargs='+')
args = parser.parse_args()
print(args)
return 1
You could also test a parser with your own list of strings, recognising that parse_args uses sys.argv[1:] if its argument is missing or None:
def main(argv=None):
print(argv)
...
args = parser.parse_args(argv)
print(args)
return 1
loc = match_loc.main(['abc','ab']) # and in the caller
Even though I was able to construct a working test case, you really should have given enough information that I didn't need to guess or dig around.

How to make a function accept runtime arguments

I want to define a function such that I want it to accept command line arguments
below is my code
def tc1(resloc):
from uiautomator import Device;
'Do some activity'
with open(resloc, 'a') as text_file:
text_file.write('TC1 PASSED \n')
text_file.close()
else:
with open(resloc, 'a') as text_file:
text_file.write('TC1 FAILED \n')
text_file.close()
if __name__ == '__main__':
tc1("C:\\<pathtoautomation>\\Results\results.txt")
Now when I execute the same using command line from python it continues to refer to the path mentioned here tc1("C:\\<pathtoautomation>\\Results\results.txt") and doesn't consider what I pass in the runtime from the command line
\Scripts>python.exe trail.py C:\\<pathtoautomationresults>\\Results\results.txt

What you are looking is sys.argv
The list of command line arguments passed to a Python script. argv[0]
is the script name (it is operating system dependent whether this is a
full pathname or not). If the command was executed using the -c
command line option to the interpreter, argv[0] is set to the string
'-c'. If no script name was passed to the Python interpreter, argv[0]
is the empty string.
To loop over the standard input, or the list of files given on the
command line, see the fileinput module.
You use it like:
import sys
def main(argv):
# do something with argv.
if __name__ == "__main__":
main(sys.argv[1:]) # the argv[0] is the current filename.
and call it using
python yourfile.py yourargsgohere
Check out a more detailed use here.

You need to use sys.argv to get command line parameters.
in your code it could look like this:
import sys
... # other stuff
if __name__ == '__main__':
tc1(sys.argv[1])
There are many tutorials out there that help you use sys.argv

is there a way to clear python argparse?

Consider the following script:
import argparse
parser1 = argparse.ArgumentParser()
parser1.add_argument('-a')
args1 = parser1.parse_args()
parser2 = argparse.ArgumentParser()
parser2.add_argument('-b')
args2 = parser2.parse_args()
I have several questions:
Is parse_args a one-time method or is there a way to clear the
arguments before adding new ones? (e.g. something like
args1.clear() or parser1.clear())
The result of this script is unusable. Although this script accepts
the -a argument, it does not accept any value for 'a'. Nor does it
accept any -b argument. Is there some way to make any of the arguments really work?
This is my actual scenario: I have 2 scripts. Both import the same
file which has initialization code (load config files, create
loggers, etc.), lets call it init.py This init.py file also parses
the arguments only because it needs one value from it. The problem
is that I need one of the scripts to accept other arguments as well.
Since init.py does something with one argument, I cannot wait with
parse_args. How can I make it work?
Edit:
Here is the output of my script:
[prompt]# python2.7 myscript.py -a
usage: a.py [-h] [-a A]
myscript.py: error: argument -a: expected one argument
[prompt]# python2.7 myscript.py -a 1
Namespace(a='1')
usage: a.py [-h] [-b B]
myscript.py: error: unrecognized arguments: -a 1

Your scenario is quite unclear, but I guess what you're looking for is parse_known_args
Here I guessed that you called init.py from the other files, say caller1.py and caller2.py
Also suppose that init.py only parses -a argument, while the original script will parse the rest.
You can do something like this:
in init.py put this in do_things method:
parser = argparse.ArgumentParser()
parser.add_argument('-a')
parsed = parser.parse_known_args(sys.argv)
print 'From init.py: %s' % parsed['a']
In caller1.py:
init.do_things(sys.argv)
parser = argparse.ArgumentParser()
parser.add_argument('-b')
parsed = parser.parse_known_args(sys.argv)
print 'From caller1.py: %s' % parsed['b']
If you call caller1.py as follows: python caller1.py -a foo -b bar, the result will be:
From init.py: foo
From caller1.py: bar
But if your scenario is not actually like this, I would suggest to use #Michael0x2a answer, which is just to use single ArgumentParser object in caller1.py and pass the value appropriately for init.py

This doesn't really make sense, because for all intents and purposes, the parser object is stateless. There's nothing to clear, since all it does is takes in the console arguments, and returns a Namespace object (a pseudo-dict) without ever modifying anything in the process.
Therefore, you can consider parse_args() to be idempotent. You can repeatedly call it over and over, and the same output will occur. By default, it will read the arguments from sys.argv, which is where the console arguments are stored.
However, note that you can pipe in custom arguments by passing in a list to the parse_args function so that the parser will using something other then sys.argv as input.
I'm not sure what you mean. If you call python myscript.py -a 15, args1 will equal Namespace(a='15'). You can then do args1['a'] to obtain the value of 15. If you want to make the flag act as a toggle, call parser.add_argument('-a', action='store_true'). Here is a list of all available actions.
I would try and confine all the console/interface code into a single module and into a single parser. Basically, remove the code to parse the command line from init.py and the second file into an independent little section. Once you run the parser, which presents a unified interface for everything in your program, pass in the appropriate variables to functions inside init.py. This has the added advantage of keeping the UI separate and more easily interchangeable with the rest of the code.

call program with arguments

i would like to start a python file (.py) with arguments and receive the output of it after it is finished. i have already heard about "popen" and "subprocess.call" but i could not find any tutorials how to use them
does anyone know a good tutorial?

You don't need them ; just launch your file as a program giving argument like
./main.py arg1 arg2 arg3 >some_file
(for that your file must begin with something like #!/usr/bin/env python)
Using sys module you can access them :
arg1 = sys.argv[1]
arg2 = sys.argv[2]
arg3 = sys.argv[3]

i would like to start a python file (.py) with arguments and receive the output of it after it is finished.
Step 1. Don't use subprocess. You're doing it wrong.
Step 2. Read the Python file you want to run. Let's call it runme.py.
Step 3. Read it again. If it's competently written, there is a block of code that starts if __name__ == "__main__":. What follows is the "external interface" to that file. Since you provided no information in the question, I'll assume it looks like this.
if __name__ == "__main__":
main()
Step 4. Read the "main" function invoked by the calling script. Since you provided no information, I'll assume it looks like this.
def main():
options, args = parse_options()
for name in args:
process( options, file )
Keep reading to be sure you see how parse_options and process work. I'll assume parse_options uses optparse.
Step 5. Write your "calling" script.
import runme
import sys
import optparse
options = options= optparse.Values({'this':'that','option':'arg','flag':True})
with open( "theoutput.out", "w" ) as results:
sys.stdout= results
for name in ('some', 'list', 'of', 'arguments' ):
runme.process( options, name )
This is the correct way to run a Python file from within Python.
Actually figure out the interface for the thing you want to run. And run it.

runme.py
print 'catch me'
main.py
import sys
from StringIO import StringIO
new_out = StringIO()
old_out = sys.stdout
sys.stdout = new_out
import runme
sys.stdout = old_out
new_out.seek(0)
print new_out.read()
and...
$ python main.py
catch me

Unless you mean you want to start the Python file from within Python? In which case it's even nicer:
import nameOfPythonFile