A global variable can be easily mocked following these answers. Great. However, this does not work when trying to mock a variable in a script that you call with subprocess.call() in a test with Pytest.
Here is my simplified script in a file called so_script.py:
import argparse
INCREMENTOR = 4
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('input_nr', type=int, help='An int to increment')
args = parser.parse_args()
with open('test.txt', 'w+') as f:
f.write(str(args.input_nr + INCREMENTOR))
Now, say I want to mock the value of INCREMENTOR in my tests to be 1. If I do this:
from subprocess import call
from unittest import mock
def test_increments_with_1():
with mock.patch('so_script.INCREMENTOR', 1):
call(['python', 'so_script.py', '3'])
with open('test.txt', 'r+') as f:
assert f.read() == '4'
The test will fail, because the value of INCREMENTOR remains 4, even though I tried to patch it to 1. So what gets written to the file is 7 instead of 4.
So my question is: how do I mock the INCREMENTOR global variable in my so_script.py file so that, when calling subprocess.call() on it, it remains mocked?
Because the so_script.py script and pytest are executed in different processes, one cannot mock objects in so_script.py while the latter is being called as a different process in tests.
The best solution I found was to put everything from the if __name__ == '__main__: block in a function and test that function with Pytest, mocking whatever I needed to mock. And, to have 100% test coverage (which was my initial intent with calling the script as a subprocess), I applied this solution.
So I dropped using subprocess.call() in my tests and wrote an init() function checking if __name__ == '__main__:, and then mocked __name__ in the tests to test the function, just as the article advises to do. This got me 100% test coverage and full mocking capabilities.
Related
TI am quite new to Python Programming and have a question on testing using Pytest. In a high-level, I have a program that takes 3 pieces of user input and generates a text file in the end. For my tests, I want to basically compare the files my program outputted, with what it should be.
Now, I am not sure how to go about testing. The program itself takes no arguments, but just relies on 3 pieces of user input, which I'll use monkeypatch to simulate. Do I create a new python file called program_test.py and have methods in here that call the original program? I have tried this, but I'm having trouble actually calling the original program and sending in the simulated inputs. Or, do I have tests in the original program (which doesn't make much sense to me).
I want something like this:
import my_program
def test_1():
inputs = iter(['input1', 'input2', 'input3'])
monkeypatch.setattr('builtins.input', lambda x: next(inputs))
my_program
# now do some assertion with some file comparison
# pseudocode
assert filecompare.cmp(expectedfile, actualfile)
This just seems to be running the original program and I think its to do with the import statement i.e. it is never running test_1(), probably because I never call it? Any help would be appreciated!
Without providing your my_program code it's hard to tell what's going on.
Since you are mentioning import problems, I guess your not defining main() and if __name__ == "__main__".
Here's a little example of how you can test that.
First, structure your my_program to have main function which contains the code and then add if __name__ == "__main__" which will allow you to run main function if the my_program is executed directly but also to import my_program as module to other files (without running it, for more information please see: What does if name == "main": do?).
my_program:
def main():
x = input()
y = input()
z = input()
with open("test", "w") as f_out:
f_out.write(f"{x}-{y}-{z}")
if __name__ == "__main__":
main()
Now you can create a test.py file and test the main function of my_program:
import os
import filecmp
import my_program
def test_success(monkeypatch):
inputs = ["input1", "input2", "input3"]
monkeypatch.setattr("builtins.input", lambda: next(iter(inputs)))
my_program.main()
with open("expected", "w") as f_out:
f_out.write("-".join(inputs))
assert filecmp.cmp("expected", "test")
os.remove("test")
os.remove("expected")
def test_fail(monkeypatch):
inputs = ["input1", "input2", "input3"]
monkeypatch.setattr("builtins.input", lambda: next(iter(inputs)))
my_program.main()
with open("expected", "w") as f_out:
f_out.write("something-else-test")
assert not filecmp.cmp("expected", "test")
os.remove("test")
os.remove("expected")
This is an example so I used os.remove to delete the files. Ideally you would define fixtures in your tests to use tempfile and generate random temporary files which will be automatically deleted after your tests.
I'm trying to write a testing program to test many(identical) student assignments. I have a test written using the unittest library. The documentation seems to indicate that each test should be associated with one file. Instead, I'd like to have one test file and use command line arguments to point the test to the file it should test.
I know I can do this by using the argparse module in my unit tests, but is there a better way? It seems like this behavior should be supported in unittest, but I can't find anything in the documentation...
Create the Main test directory and add sub test packages. Have a test runner created for you pointing to the test directory. It could act as a suite. I have attached a piece of code that I have used for my test suite.
import os
import unittest
def main(test_path, test_pattern):
print(('Discovering tests in : {}'.format(test_path)))
suite = unittest.TestLoader().discover(test_path, test_pattern)
unittest.TextTestRunner(verbosity=2).run(suite)
if __name__ == '__main__':
root_path = os.path.abspath('.')
test_path = os.path.join(root_path, 'src/tests/')
test_pattern = 'test_*'
main(test_path, test_pattern)
Generally speaking, unittest is used to test module level python code, not interactions python code has with external programs. AFAIK, writing to stdout (ie. print) means you are either debugging or passing information to another program.
In your case, I don't think unittest is really necessary, unless you want to give assignments that are to "pass this unittest" (which is common in the wild).
Instead I would simply iterate over the directory that contains the assignments, check the stdout using subprocess, then write the results to a csv file:
import subprocess
import os
import csv
ASSIGNMENT_DIR = '/path/to/assignments'
expected_stdout = 'Hello World!'
def _determine_grade(stdout):
if stdout == expected_stdout:
return '100%'
return '0%'
grades = {}
for assignment in os.listdir(ASSIGNMENT_DIR):
filepath = os.path.join(ASSIGNMENT_DIR, assignment)
stdout = subprocesss.check_output(f'python3 {filepath}', shell=True)
grade = _determine_grade(stdout)
grades.append({'assignment':assignment, 'grade':grade})
with open('/path/to/grades.csv', 'w+') as f:
w = csv.DictWriter(f, ('assignment', 'grade'))
w.writeheader()
w.writerows(grades)
Should the function name main() always be empty and have arguments called within the function itself or is it acceptable to have them as inputs to the function e.g main(arg1, arg2, arg3)?
I know it works but I'm wondering if it is poor programming practice. Apologies if this is a duplicate but I couldn't see the question specifically answered for Python.
In most other programming languages, you'd either have zero parameters or two parameters:
int main(char *argv[], int argc)
To denote the arguments passed through to the parameter. However, in Python these are accessed through the sys module:
import sys
def main():
print(sys.argv, len(sys.argv))
But then you could extend this so that you pass through argv and argc into your python function, similar to other languages yes:
import sys
def main(argv, arc):
print(argv, arc)
if __name__ == '__main__':
main(sys.argv, len(sys.argv))
But let's forget about argv/argc for now - why would you want to pass something through to main. You create something outside of main and want to pass it through to main. And this can happen in two instances:
You're calling main multiple times from other functions.
You've created variables outside main that you want to pass through.
Point number 1 is definitely bad practice. main should be unique and called only once at the beginning of your program. If you have the need to call it multiple times, then the code inside main doesn't belong inside main. Split it up.
Point number 2 may seem like it makes sense, but then you do it in practise:
def main(a, b):
print(a, b)
if __name__ == '__main__':
x = 4
y = 5
main(x, y)
But then aren't x and y global variables? And good practice would assume that these are at the top of your file (and multiple other properties - they're constant, etc), and that you wouldn't need to pass these through as arguments.
By following the pattern:
def main():
...stuff...
if __name__ == '__main__':
main()
It allows your script to both to be run directly, and if packaged using setup tools, to have an executable script generated automatically when the package is installed by specifying main as an entry point.
See: https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation
You would add to setup.py something like:
entry_points={
'console_scripts': [
'my_script = my_module:main'
]
}
And then when you build a package, people can install it in their virtual environment, and immediately get a script called my_script on their path.
Automatic script creation like this requires a function that takes no required arguments.
It's a good idea to allow you script to be imported and expose it's functionality both for code reuse, and also for testing. I would recommend something line this pattern:
import argparse
def parse_args():
parser = argparse.ArgumentParser()
#
# ... configure command line arguments ...
#
return parser.parse_args()
def do_stuff(args):
#
# ... main functionality goes in here ...
#
def main():
args = parse_args()
do_stuff(args)
if __name__ == '__main__':
main()
This allows you to run your script directly, have an automatically generated script that behaves the same way, and also import the script and call do_stuff to re-use or test the actual functionality.
This blog post was mentioned in the comments: https://www.artima.com/weblogs/viewpost.jsp?thread=4829 which uses a default argument on main to allow dependency injection for testing, however, this is a very old blog post; the getopt library has been superseded twice since then. This pattern is superior and still allows dependency injection.
I would definitely prefer to see main take arguments rather than accessing sys.argv directly.
This makes the reuse of the main function by other Python modules much easier.
import sys
def main(arg):
...
if __name__ == "__main__":
main(sys.argv[1])
Now if I want to execute this module as a script from another module I can just write (in my other module).
from main_script import main
main("use this argument")
If main uses sys.argv this is tougher.
I know Python unittest. I have some experience using it for testing Python subprograms.
Now I need to add testing my command line application (not just a Python function) written in Python. I want to call it with certain arguments and certain input in stdin and test output in stdout.
How to integrate testing a command line tool with other unittest test cases?
Or what to use instead of unittest?
You can still use the standard unittest format and test the whole application as a standard function. Make a wrapper that makes the script entry point a simple wrapper, like:
if __name__ == "__main__":
sys.exit(main(sys.argv))
As long as you don't abuse global variables for keeping the state, you only need to test the main() function.
If you want to test scripts which you don't have control over, you can still use unittest and subprocess by writing a test like:
def test_process(self):
result = subprocess.run(['your_script', 'your_args', ...], capture_output=True)
self.assertIn('expected out', result.stdout)
def test_process_failure(self):
result = subprocess.run(['your_script', 'your_args', ...], capture_output=True)
self.assertEqual(result.returncode, 1)
I have written some unit tests using unittest in Python. However, they do not simply test objects in the conventional way - rather, they invoke another Python script by calling it using Popen. This is by design - it's a command line utility, so I want to test it as a user would, which includes things such as command-line options, etc.). To be clear, both the unit tests and the script to be tested are written in Python (v3 to be precise).
The script I am testing makes heavy use of datetime.now(), and ideally I would like to mock that value somehow so that I can keep it constant. All the examples I've seen of doing this, though (e.g. this one using mock) assume some form of white-box testing.
Is there a way for me to do this?
Nothing prevents you from testing your CLI without using Popen. You just need to architect your code to make it possible:
Instead of having this:
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
# ... Add args
ns = parser.parse_args()
Do this:
import argparse
def main(argv):
parser = argparse.ArgumentParser()
# ... Add args
parser.parse_args(argv[1:]) # This is the default for argparse
ns = parser.parse_args()
if __name__ == "__main__":
import sys
main(sys.argv)
Then, you can test the main function in isolation (just call main([...]) with a set of args you specify). Note that this should also work (with some adaptation) for other CLI frameworks.
Also, note that if you're indeed using argparse, you'll need to patch ArgumentParser() so that it doesn't call sys.exit when parsing fails.
An easy way to do is to declare a ParsingError exception, and patch ArgumentParser.error(self, message) with:
def error(self, message):
raise ParsingError(message)
You can then use assertRaises in your tests.