I have test which run subprocess on certain executable and test the stdout result.
So I use
#conftest.py
def pytest_addoption(parser):
parser.addoption("--executable", action="store")
#pytest.fixture(scope="session", autouse=True)
def pass_executable(request):
try:
return request.config.getoption("--executable")
except AttributeError:
pass
So that I can use command line arg to set the pass the executable. I wish to use this as a global variable across all my tests. However, I have trouble with the tests which requires #pytest.mark.parametrize decorator. So my solution is to create a test_update_path(pass_executable) to update a global variable PATH, which works.
# test.py
PATH = 'defaultpath/app'
def test_update_path(pass_executable):
global PATH
PATH = pass_executable
print("Gloabl path is update to: ")
print(PATH)
def test_1():
# This will work
print("Now next")
print(PATH)
cmd = [PATH]
stdout, stderr = run_subprocess(cmd)
assert stdout == 'some expected result'
#pytest.mark.parametrize("args", [1, 2, 3])
def test_2(path, args):
print("Now next")
print(PATH)
cmd = paramparser(PATH, args)
stdout, stderr = run_subprocess(cmd)
assert stdout == 'some expected result'
if __name__ == '__main__':
pytest.main()
pytest --executable=newpath/app -s will work fine, but it is an ugly hack. More importantly, it ran a test which was not doing any actual testing. It is also problematic as the argument is not an optional. Without setting --executable. The path will be an NoneType rather than the original default path .
Any suggestion please?
Appreciated.
You don't need global vars, just use the request fixture as test argument to get access to the command line arg, like you already have in pass_executable. This is how I would change both tests:
def test_1(request):
cmd = [request.config.getoption("--executable")]
stdout, stderr = run_subprocess(cmd)
assert stdout == 'some expected result'
#pytest.mark.parametrize("arg", [1, 2, 3])
def test_2(request, arg):
cmd = paramparser(request.config.getoption("--executable"), arg)
stdout, stderr = run_subprocess(cmd)
assert stdout == 'some expected result'
If you don't like the code duplication in both tests, extract it into a fixture and use it as a test argument, just like the built-in request:
#pytest.fixture
def executable(request):
return request.config.getoption("--executable")
def test_1(executable):
cmd = [executable]
stdout, stderr = run_subprocess(cmd)
assert stdout == 'some expected result'
#pytest.mark.parametrize("arg", [1, 2, 3])
def test_2(executable, arg):
cmd = paramparser(executable, arg)
stdout, stderr = run_subprocess(cmd)
assert stdout == 'some expected result'
Related
I've been trying to understand tutorials on how to parse arguments in Python using argparse. Is this how I would pass a command line input so I can run a function?
import argparse
parser = argparse.ArgumentParser(description='A test')
parser.add_argument("--a", default=1, help="Test variable")
args = parser.parse_args()
def foo():
command_line_argument = args.a
bar = 2*args.a
print(bar)
return
if "__name__" == "__main__"
try:
while True:
foo()
except KeyboardInterrupt:
print('User has exited the program')
That while True looks odd to me -- are you asking the reader to keep submitting inputs until they CTRL+C ? Because if so, argparse is the wrong thing to use: see Getting user input
If you intend a single argument then I'd move the parser stuff inside main, which is what gets executed when the script is run as a program as opposed to being imported.
Also, I'd pass a parameter to foo rather than the args block.
Lastly, I guess you're expecting to receive a number so you need type=int or similar.
import argparse
def foo(a):
bar = 2*a
print(bar)
return
if __name__ == "__main__":
try:
# set it up
parser = argparse.ArgumentParser(description='A test')
parser.add_argument("--a", type=int, default=1, help="Test variable")
# get it
args = parser.parse_args()
a = args.a
# use it
foo(a)
except KeyboardInterrupt:
print('User has exited the program')
So:
$ python foo.py --a 1
2
below is in working state
import argparse
parser = argparse.ArgumentParser(description='A test')
parser.add_argument("--a", default=1, help="Test variable", type=int)
args = parser.parse_args()
def foo():
command_line_argument = args.a
bar = 2*args.a
print(bar)
return
if __name__ == "__main__":
try:
while True:
foo()
except KeyboardInterrupt:
print('User has exited the program')
if you run python your-filename.py --a=2 it will print 4 until you stop the execution.
I'd like to pass my own arguments into files that are setup for unittest. So calling it from the command line like this should work:
python Test.py --c keith.ini SomeTests.test_one
Currently I'm running into two issues.
1) Arg parse doesn't allow unknown arguments
usage: Test.py [-h] [--c CONFILE]
Test.py: error: unrecognized arguments: SomeTests.test_one
2) Unit test doesn't allow unknown arguments. So --c fileName is not accepted by unittest and returns:
AttributeError: 'module' object has no attribute 'keith'
So the idea is to collect my arguments and remove them before calling unittest runner.
import unittest
import argparse
myArgs = None
def getArgs( allArgs ):
parser = argparse.ArgumentParser( )
parser.add_argument('--c', dest='conFile', type=str, default=None, help='Config file')
args = parser.parse_args()
if ( args.conFile == None ):
parser.print_help()
return args
class SomeTests(unittest.TestCase):
def test_one(self):
theTest( 'keith' )
def test_two(self):
otherTest( 'keith' )
if __name__ == '__main__':
myArgs = getArgs( sys.argv )
print 'Config File: ' + myArgs.conFile
unittest.main( argv=sys.argv, testRunner = unittest.TextTestRunner(verbosity=2))
Interesting I just found parse_known_args() so I changed the parse line to:
args = parser.parse_known_args(['--c']).
I thought this would solve my issue and give me something to pass to unittest. Unfortunately I get:
Test.py: error: argument --c: expected one argument.
Shouldn't this work?
OK took a bit of effort but figured it out. This is totally possible. The documentation for argparse is not correct. The function parse_known_args() should not include a list of known arguments. Also argparse removes arg[0] which is important to return so other commands see a valid argument list. I'd consider this removal a bug. I have included the final example code.
import unittest
import argparse
import sys
myArgs = None
def getArgs( allArgs ):
parser = argparse.ArgumentParser( )
parser.add_argument('--c', dest='conFile', type=str, default=None, help='Configuration file. (Required)')
args, addArgs = parser.parse_known_args( )
if ( args.conFile == None ):
parser.print_help()
sys.exit(2)
# argparse strips argv[0] so prepend it
return args, [ sys.argv[0]] + addArgs
def verify( expected, actual ):
assert expected == actual, 'Test Failed: '
# Reusable Test
def theTest( exp ):
print 'myargs: ' + str( myArgs )
verify( exp, 'keith' )
def otherTest( exp ):
theTest( exp )
class SomeTests(unittest.TestCase):
def test_one(self):
theTest( 'keith' )
def test_two(self):
otherTest( 'keith2' )
if __name__ == '__main__':
myArgs, addArgs = getArgs( sys.argv )
unittest.main( argv=addArgs, testRunner = unittest.TextTestRunner(verbosity=2))
Once you save this to a file you can call it like the examples below and it will all work.
python Test.py # Requires config file
python Test.py --c keith.ini # Runs all tests
python Test.py --c keith.ini SomeTests # Runs Class
python Test.py --c keith.ini SomeTests.test_one # Runs test
HTH, Enjoy
So I am trying to execute a file and get the returned value back using the python builtin methods available in the subprocess library.
For example, lets say I want to execute this hello_world python file:
def main():
print("in main")
return("hello world!")
if __name__ == '__main__':
main()
I do not care about getting back the statement in main. What I want to get back is the return value hello world!.
I tried numerous things but non of them worked.
Here's a list of what I tried and their outputs:
args is common for all trials:
args = ['python',hello_cmd]
First trial:
p1 = subprocess.Popen(args, stdout=subprocess.PIPE)
print(p1.communicate())
print("returncode is:")
print(p1.returncode)
output is:
(b'in main\n', None)
returncode is:
0
second trial:
p2 = subprocess.check_output(args,stderr=subprocess.STDOUT)
print(p2)
output is:
b'in main\n'
third trial:
output, result = subprocess.Popen(args, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell=False).communicate()
print(output)
print(result)
output is:
b'in main\n'
b''
fourth trial:
p4 = subprocess.run(args, stdout=PIPE, stderr=PIPE)
print(p4)
output is:
CompletedProcess(args=['python', '/path/to/file/hello.py'], returncode=0, stdout=b'in main\n', stderr=b'')
fifth trial:
p5 =subprocess.getstatusoutput(args)
print(p5)
output is:
(0, '')
Can anyone help?
The return value of the main function is not the return code that is passed to the OS. To pass a return code to the OS use sys.exit(), which expects an integer. You can pass it a string, but if you do, Python will pass 1 to the OS.
You cannot return strings as return codes it must be an integer. If you want to act differently depending on the process. Try to map your return code to some function in your main program. For example
def execute_sub_program(): ...
# somewhere else:
return_code = execute_sub_program()
if return_code == 0:
# do something ...
elif ...
You can try with subprocess.run().returncode, it gives 0 if successful execution and 1 if failed execution.
driver.py
import subprocess
args = ['python', './hello_cmd.py']
status_code = subprocess.run(args).returncode
print(["Successful execution", "Failed execution"][status_code])
For happy flow (hello_cmd.py):
def main():
print("in main")
return("hello world!")
if __name__ == '__main__':
main()
For failed flow (hello_cmd.py):
def main():
print("in main")
raise ValueError('Failed')
if __name__ == '__main__':
main()
In following example:
import subprocess
import mock
class MyArgs():
cmd = ''
cmd_args = ''
cmd_path = ''
def __init__(self):
pass
def set_args(self, c, a, p):
self.cmd = c
self.cmd_args = a
self.cmd_path = p
def get_command(self):
return ([self.cmd, self.cmd_args, self.cmd_path])
class Example():
args = MyArgs()
def __init__(self):
pass
def run_ls_command(self):
print 'run_ls_command command:' + str(self.get_command())
p = subprocess.Popen(self.get_command(), stdout=subprocess.PIPE)
out, err = p.communicate()
print out #to verify the mock is working, should output 'output' if the mock is called
return err
def set_args(self, c, a, p):
#this would be more complicated logic in
#future and likely not just one method, this is a MWE
self.args.set_args(c,a,p)
def get_command(self):
return self.args.get_command()
#mock.patch.object(subprocess, 'Popen', autospec=True)
def test_subprocess_popen(mock_popen):
mock_popen.return_value.returncode = 0
mock_popen.return_value.communicate.return_value = ("output", "Error")
e = Example()
e.set_args('ls', '-al', '/bin/foobar')
e.run_ls_command()
#todo: validate arguments called by the popen command for the test
test_subprocess_popen()
The longer term goal is being able to validate more complicated subprocess.Popen commands, which will be constructed by more manipulations on the Example object (though the concept will be the same as this example).
What I would like to do is somehow analyze the arguments sent to the p = subprocess.Popen(self.get_command(), stdout=subprocess.PIPE) command.
However I am not sure how to get those arguments - I know my mock is being called because my output matches expected for the mock.
This is how my code looks and I get an error, while using Popen
test.py
import subprocess
import sys
def test(jobname):
print jobname
p=subprocess.Popen([sys.executable,jobname,parm1='test',parm2='test1'])
if __name__ == "__main__":
test(r'C:\Python27\test1.py')
test1.py
def test1(parm1,parm2):
print 'test1',parm1
if __name__ = '__main__':
test1(parm1='',parm2='')
Error
Syntax error
In test1.py:
You need two equal signs in :
if __name__ = '__main__':
Use instead
if __name__ == '__main__':
since you want to compare the value of __name__ with the string '__main__', not assign a value to __name__.
In test.py:
parm1='test' is a SyntaxError. You can not to assign a value to a variable in the middle of a list:
p=subprocess.Popen([sys.executable,jobname,parm1='test',parm2='test1'])
It appears you want to feed different values for parm1 and parm2 into the function test1.test1. You can not do that by calling python test1.py since there parm1='' and parm2='' are hard-coded there.
When you want to run a non-Python script from Python, use subprocess. But when you want to run Python functions in a subprocess, use multiprocessing:
import multiprocessing as mp
import test1
def test(function, *args, **kwargs):
print(function.__name__)
proc = mp.Process(target = function, args = args, kwargs = kwargs)
proc.start()
proc.join() # wait for proc to end
if __name__ == '__main__':
test(test1.test1, parm1 = 'test', parm2 = 'test1')