I'm trying to test a function that takes input from stdin, which I'm currently testing with something like this:
cat /usr/share/dict/words | ./spellchecker.py
In the name of test automation, is there any way that pyunit can fake input to raw_input()?
The short answer is to monkey patch raw_input().
There are some good examples in the answer to How to display the redirected stdin in Python?
Here is a simple, trivial example using a lambda that throws away the prompt and returns what we want.
System Under Test
cat ./name_getter.py
#!/usr/bin/env python
class NameGetter(object):
def get_name(self):
self.name = raw_input('What is your name? ')
def greet(self):
print 'Hello, ', self.name, '!'
def run(self):
self.get_name()
self.greet()
if __name__ == '__main__':
ng = NameGetter()
ng.run()
$ echo Derek | ./name_getter.py
What is your name? Hello, Derek !
Test case:
$ cat ./t_name_getter.py
#!/usr/bin/env python
import unittest
import name_getter
class TestNameGetter(unittest.TestCase):
def test_get_alice(self):
name_getter.raw_input = lambda _: 'Alice'
ng = name_getter.NameGetter()
ng.get_name()
self.assertEquals(ng.name, 'Alice')
def test_get_bob(self):
name_getter.raw_input = lambda _: 'Bob'
ng = name_getter.NameGetter()
ng.get_name()
self.assertEquals(ng.name, 'Bob')
if __name__ == '__main__':
unittest.main()
$ ./t_name_getter.py -v
test_get_alice (__main__.TestNameGetter) ... ok
test_get_bob (__main__.TestNameGetter) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Update -- using unittest.mock.patch
Since python 3.3 there is new submodule for unittest called mock that does exactly what you need to do. For those using python 2.6 or above there is a backport of mock found here.
import unittest
from unittest.mock import patch
import module_under_test
class MyTestCase(unittest.TestCase):
def setUp(self):
# raw_input is untouched before test
assert module_under_test.raw_input is __builtins__.raw_input
def test_using_with(self):
input_data = "123"
expected = int(input_data)
with patch.object(module_under_test, "raw_input", create=True,
return_value=expected):
# create=True is needed as raw_input is not in the globals of
# module_under_test, but actually found in __builtins__ .
actual = module_under_test.function()
self.assertEqual(expected, actual)
#patch.object(module_under_test, "raw_input", create=True)
def test_using_decorator(self, raw_input):
raw_input.return_value = input_data = "123"
expected = int(input_data)
actual = module_under_test.function()
self.assertEqual(expected, actual)
def tearDown(self):
# raw input is restored after test
assert module_under_test.raw_input is __builtins__.raw_input
if __name__ == "__main__":
unittest.main()
# where module_under_test.function is:
def function():
return int(raw_input("prompt> "))
Previous answer -- replacing sys.stdin
I think the sys module might be what you're looking for.
You can do something like
import sys
# save actual stdin in case we need it again later
stdin = sys.stdin
sys.stdin = open('simulatedInput.txt','r')
# or whatever else you want to provide the input eg. StringIO
raw_input will now read from simulatedInput.txt whenever it is called. If the contents of simulatedInput was
hello
bob
then the first call to raw_input would return "hello", the second "bob" and third would throw an EOFError as there was no more text to read.
You didn't describe what sort of code is in spellchecker.py, which gives me freedom to speculate.
Suppose it's something like this:
import sys
def check_stdin():
# some code that uses sys.stdin
To improve testability of check_stdin function, I propose to refactor it like so:
def check_stdin():
return check(sys.stdin)
def check(input_stream):
# same as original code, but instead of
# sys.stdin it is written it terms of input_stream.
Now most of your logic is in check function, and you can hand-craft whatever input you can imagine in order to test it properly, without any need to deal with stdin.
My 2 cents.
Replace sys.stdin with an instance of StringIO, and load the StringIO instance with the data you want returned via sys.stdin. Also, sys.__stdin__ contains the original sys.stdin object, so restoring sys.stdin after your test is as simple as sys.stdin = sys.__stdin__.
Fudge is a great python mock module, with convenient decorators for doing patching like this for you, with automatic cleanup. You should check it out.
If you are using mock module (written by Michael Foord), in order to mock raw_input function you can use syntax like:
#patch('src.main.raw_input', create=True, new=MagicMock(return_value='y'))
def test_1(self):
method_we_try_to_test(); # method or function that calls **raw_input**
Related
I have a console program written in Python. It asks the user questions using the command:
some_input = input('Answer the question:', ...)
How would I test a function containing a call to input using pytest?
I wouldn't want to force a tester to input text many many times only to finish one test run.
As The Compiler suggested, pytest has a new monkeypatch fixture for this. A monkeypatch object can alter an attribute in a class or a value in a dictionary, and then restore its original value at the end of the test.
In this case, the built-in input function is a value of python's __builtins__ dictionary, so we can alter it like so:
def test_something_that_involves_user_input(monkeypatch):
# monkeypatch the "input" function, so that it returns "Mark".
# This simulates the user entering "Mark" in the terminal:
monkeypatch.setattr('builtins.input', lambda _: "Mark")
# go about using input() like you normally would:
i = input("What is your name?")
assert i == "Mark"
You should probably mock the built-in input function, you can use the teardown functionality provided by pytest to revert back to the original input function after each test.
import module # The module which contains the call to input
class TestClass:
def test_function_1(self):
# Override the Python built-in input method
module.input = lambda: 'some_input'
# Call the function you would like to test (which uses input)
output = module.function()
assert output == 'expected_output'
def test_function_2(self):
module.input = lambda: 'some_other_input'
output = module.function()
assert output == 'another_expected_output'
def teardown_method(self, method):
# This method is being called after each test case, and it will revert input back to original function
module.input = input
A more elegant solution would be to use the mock module together with a with statement. This way you don't need to use teardown and the patched method will only live within the with scope.
import mock
import module
def test_function():
with mock.patch.object(__builtins__, 'input', lambda: 'some_input'):
assert module.function() == 'expected_output'
You can replace sys.stdin with some custom Text IO, like input from a file or an in-memory StringIO buffer:
import sys
class Test:
def test_function(self):
sys.stdin = open("preprogrammed_inputs.txt")
module.call_function()
def setup_method(self):
self.orig_stdin = sys.stdin
def teardown_method(self):
sys.stdin = self.orig_stdin
this is more robust than only patching input(), as that won't be sufficient if the module uses any other methods of consuming text from stdin.
This can also be done quite elegantly with a custom context manager
import sys
from contextlib import contextmanager
#contextmanager
def replace_stdin(target):
orig = sys.stdin
sys.stdin = target
yield
sys.stdin = orig
And then just use it like this for example:
with replace_stdin(StringIO("some preprogrammed input")):
module.call_function()
This can be done with mock.patch and with blocks in python3.
import pytest
import mock
import builtins
"""
The function to test (would usually be loaded
from a module outside this file).
"""
def user_prompt():
ans = input('Enter a number: ')
try:
float(ans)
except:
import sys
sys.exit('NaN')
return 'Your number is {}'.format(ans)
"""
This test will mock input of '19'
"""
def test_user_prompt_ok():
with mock.patch.object(builtins, 'input', lambda _: '19'):
assert user_prompt() == 'Your number is 19'
The line to note is mock.patch.object(builtins, 'input', lambda _: '19'):, which overrides the input with the lambda function. Our lambda function takes in a throw-away variable _ because input takes in an argument.
Here's how you could test the fail case, where user_input calls sys.exit. The trick here is to get pytest to look for that exception with pytest.raises(SystemExit).
"""
This test will mock input of 'nineteen'
"""
def test_user_prompt_exit():
with mock.patch.object(builtins, 'input', lambda _: 'nineteen'):
with pytest.raises(SystemExit):
user_prompt()
You should be able to get this test running by copy and pasting the above code into a file tests/test_.py and running pytest from the parent dir.
Since I need the input() call to pause and check my hardware status LEDs, I had to deal with the situation without mocking. I used the -s flag.
python -m pytest -s test_LEDs.py
The -s flag essentially means: shortcut for --capture=no.
You can do it with mock.patch as follows.
First, in your code, create a dummy function for the calls to input:
def __get_input(text):
return input(text)
In your test functions:
import my_module
from mock import patch
#patch('my_module.__get_input', return_value='y')
def test_what_happens_when_answering_yes(self, mock):
"""
Test what happens when user input is 'y'
"""
# whatever your test function does
For example if you have a loop checking that the only valid answers are in ['y', 'Y', 'n', 'N'] you can test that nothing happens when entering a different value instead.
In this case we assume a SystemExit is raised when answering 'N':
#patch('my_module.__get_input')
def test_invalid_answer_remains_in_loop(self, mock):
"""
Test nothing's broken when answer is not ['Y', 'y', 'N', 'n']
"""
with self.assertRaises(SystemExit):
mock.side_effect = ['k', 'l', 'yeah', 'N']
# call to our function asking for input
I don't have enough points to comment, but this answer: https://stackoverflow.com/a/55033710/10420225
doesn't work if you just copy/pasta.
Part One
For Python3, import mock doesn't work.
You need import unittest.mock and call it as unittest.mock.patch.object(), or from unittest import mock mock.patch.object()...
If using Python3.3+ the above should "just work". If using Python3.3- you need to pip install mock. See this answer for more info: https://stackoverflow.com/a/11501626/10420225
Part Two
Also, if you want to make this example more realistic, i.e. importing the function from outside the file and using it, there's more assembly required.
This is general directory structure we'll use
root/
src/prompt_user.py
tests/test_prompt_user.py
If function in external file
# /root/src/prompt_user.py
def user_prompt():
ans = input("Enter a number: ")
try:
float(ans)
except:
import sys
sys.exit("NaN")
return "Your number is {}".format(ans)
# /root/tests/test_prompt_user.py
import pytest
from unittest import mock
import builtins
from prompt_user import user_prompt
def test_user_prompt_ok():
with mock.patch.object(builtins, "input", lambda _: "19"):
assert user_prompt() == "Your number is 19"
If function in a class in external file
# /root/src/prompt_user.py
class Prompt:
def user_prompt(self):
ans = input("Enter a number: ")
try:
float(ans)
except:
import sys
sys.exit("NaN")
return "Your number is {}".format(ans)
# /root/tests/test_prompt_user.py
import pytest
from unittest import mock
import builtins
from mocking_test import Prompt
def test_user_prompt_ok():
with mock.patch.object(builtins, "input", lambda _: "19"):
assert Prompt.user_prompt(Prompt) == "Your number is 19"
Hopefully this helps people a bit more. I find these very simple examples almost useless because it leaves a lot out for real world use cases.
Edit: If you run into pytest import issues when running from external files, I would recommend looking over this answer: PATH issue with pytest 'ImportError: No module named YadaYadaYada'
A different alternative that does not require using a lambda function and provides more control during the tests is to use the mock decorator from the standard unittest module.
It also has the additional advantage of patching just where the object (i.e. input) is looked up, which is the recommended strategy.
# path/to/test/module.py
def my_func():
some_input = input('Answer the question:')
return some_input
# tests/my_tests.py
from unittest import mock
from path.to.test.module import my_func
#mock.patch("path.to.test.module.input")
def test_something_that_involves_user_input(mock_input):
mock_input.return_value = "This is my answer!"
assert my_func() == "This is my answer!"
mock_input.assert_called_once() # Optionally check one and only one call
The simplest way that works without mocking and easily in doctest for lightweight testing, is just making the input_function a parameter to your function and passing in this FakeInput class with the appropriate list of inputs that you want:
class FakeInput:
def __init__(self, input):
self.input = input
self.index = 0
def __call__(self):
line = self.input[self.index % len(self.input)]
self.index += 1
return line
Here is an example usage to test some functions using the input function:
import doctest
class FakeInput:
def __init__(self, input):
self.input = input
self.index = 0
def __call__(self):
line = self.input[self.index % len(self.input)]
self.index += 1
return line
def add_one_to_input(input_func=input):
"""
>>> add_one_to_input(FakeInput(['1']))
2
"""
return int(input_func()) + 1
def add_inputs(input_func=input):
"""
>>> add_inputs(FakeInput(['1', '5']))
6
"""
return int(input_func()) + int(input_func())
def return_ten_inputs(input_func=input):
"""
>>> return_ten_inputs(FakeInput(['1', '5', '7']))
[1, 5, 7, 1, 5, 7, 1, 5, 7, 1]
"""
return [int(input_func()) for _ in range(10)]
def print_4_inputs(input_func=input):
"""
>>> print_4_inputs(FakeInput(['1', '5', '7']))
1
5
7
1
"""
for i in range(4):
print(input_func())
if __name__ == '__main__':
doctest.testmod()
This also makes your functions more general so you can easily change them to take input from a file rather than the keyboard.
You can also use environment variables in your test code. For example if you want to give path as argument you can read env variable and set default value if it's missing.
import os
...
input = os.getenv('INPUT', default='inputDefault/')
Then start with default argument
pytest ./mytest.py
or with custom argument
INPUT=newInput/ pytest ./mytest.py
Function foo prints to console. I want to test the console print. How can I achieve this in python?
Need to test this function, has NO return statement :
def foo(inStr):
print "hi"+inStr
My test :
def test_foo():
cmdProcess = subprocess.Popen(foo("test"), stdout=subprocess.PIPE)
cmdOut = cmdProcess.communicate()[0]
self.assertEquals("hitest", cmdOut)
You can easily capture standard output by just temporarily redirecting sys.stdout to a StringIO object, as follows:
import StringIO
import sys
def foo(inStr):
print "hi"+inStr
def test_foo():
capturedOutput = StringIO.StringIO() # Create StringIO object
sys.stdout = capturedOutput # and redirect stdout.
foo('test') # Call unchanged function.
sys.stdout = sys.__stdout__ # Reset redirect.
print 'Captured', capturedOutput.getvalue() # Now works as before.
test_foo()
The output of this program is:
Captured hitest
showing that the redirection successfully captured the output and that you were able to restore the output stream to what it was before you began the capture.
Note that the code above in for Python 2.7, as the question indicates. Python 3 is slightly different:
import io
import sys
def foo(inStr):
print ("hi"+inStr)
def test_foo():
capturedOutput = io.StringIO() # Create StringIO object
sys.stdout = capturedOutput # and redirect stdout.
foo('test') # Call function.
sys.stdout = sys.__stdout__ # Reset redirect.
print ('Captured', capturedOutput.getvalue()) # Now works as before.
test_foo()
This Python 3 answer uses unittest.mock. It also uses a reusable helper method assert_stdout, although this helper is specific to the function being tested.
import io
import unittest
import unittest.mock
from .solution import fizzbuzz
class TestFizzBuzz(unittest.TestCase):
#unittest.mock.patch('sys.stdout', new_callable=io.StringIO)
def assert_stdout(self, n, expected_output, mock_stdout):
fizzbuzz(n)
self.assertEqual(mock_stdout.getvalue(), expected_output)
def test_only_numbers(self):
self.assert_stdout(2, '1\n2\n')
Note that the mock_stdout arg is passed automatically by the unittest.mock.patch decorator to the assert_stdout method.
A general-purpose TestStdout class, possibly a mixin, can in principle be derived from the above.
For those using Python ≥3.4, contextlib.redirect_stdout also exists, but it seems to serve no benefit over unittest.mock.patch.
If you happen to use pytest, it has builtin output capturing. Example (pytest-style tests):
def eggs():
print('eggs')
def test_spam(capsys):
eggs()
captured = capsys.readouterr()
assert captured.out == 'eggs\n'
You can also use it with unittest test classes, although you need to passthrough the fixture object into the test class, for example via an autouse fixture:
import unittest
import pytest
class TestSpam(unittest.TestCase):
#pytest.fixture(autouse=True)
def _pass_fixtures(self, capsys):
self.capsys = capsys
def test_eggs(self):
eggs()
captured = self.capsys.readouterr()
self.assertEqual('eggs\n', captured.out)
Check out Accessing captured output from a test function for more info.
You can also use the mock package as shown below, which is an example from
https://realpython.com/lessons/mocking-print-unit-tests.
from mock import patch
def greet(name):
print('Hello ', name)
#patch('builtins.print')
def test_greet(mock_print):
# The actual test
greet('John')
mock_print.assert_called_with('Hello ', 'John')
greet('Eric')
mock_print.assert_called_with('Hello ', 'Eric')
The answer of #Acumenus says:
It also uses a reusable helper method assert_stdout, although this helper is specific to the function being tested.
the bold part seems a big drawback, thus I would do the following instead:
# extend unittest.TestCase with new functionality
class TestCase(unittest.TestCase):
def assertStdout(self, expected_output):
return _AssertStdoutContext(self, expected_output)
# as a bonus, this syntactical sugar becomes possible:
def assertPrints(self, *expected_output):
expected_output = "\n".join(expected_output) + "\n"
return _AssertStdoutContext(self, expected_output)
class _AssertStdoutContext:
def __init__(self, testcase, expected):
self.testcase = testcase
self.expected = expected
self.captured = io.StringIO()
def __enter__(self):
sys.stdout = self.captured
return self
def __exit__(self, exc_type, exc_value, tb):
sys.stdout = sys.__stdout__
captured = self.captured.getvalue()
self.testcase.assertEqual(captured, self.expected)
this allows for the much nicer and much more re-usable:
# in a specific test case, the new method(s) can be used
class TestPrint(TestCase):
def test_print1(self):
with self.assertStdout("test\n"):
print("test")
by using a straight forward context manager. (It might also be desirable to append "\n" to expected_output since print() adds a newline by default. See next example...)
Furthermore, this very nice variant (for an arbitrary number of prints!)
def test_print2(self):
with self.assertPrints("test1", "test2"):
print("test1")
print("test2")
is possible now.
You can also capture the standard output of a method using contextlib.redirect_stdout:
import unittest
from contextlib import redirect_stdout
from io import StringIO
class TestMyStuff(unittest.TestCase):
# ...
def test_stdout(self):
with redirect_stdout(StringIO()) as sout:
my_command_that_prints_to_stdout()
# the stream replacing `stdout` is available outside the `with`
# you may wish to strip the trailing newline
retval = sout.getvalue().rstrip('\n')
# test the string captured from `stdout`
self.assertEqual(retval, "whatever_retval_should_be")
Gives you a locally scoped solution. It is also possible to capture the standard error using contextlib.redirect_stderr().
Another variant is leaning on the logging module rather than print(). This module also has a suggestion of when to use print in the documentation:
Display console output for ordinary usage of a command line script or program
PyTest has built-in support for testing logging messages.
I'm currently creating some unit test. I'm fairly new to them and just trying to get my feet wet. So the current test I'm trying to run is to check for an expected output according to the users input. So I would patch the input with some type of value and then check if I received the stdout message at the end. Sounds kind of confusing, but I hope some one can help. Here is my run code.
def main():
Attack = input("Are we being attacked?!")
if(Attack == "yes"):
print("We are being attacked! Attack Back!")
so in the above example I would test for the print statement since I would be patching the user input with the value of yes. Here is my test suite
import unittest
from unittest.mock import patch
import io
import sys
from RunFile import main
class GetInputTest(unittest.TestCase):
#patch('builtins.input', return_value='yes')
def test_output(self):
saved_stdout = sys.stdout
try:
out = io.StringIO()
sys.stdout = out
main()
output = out.getvalue().strip()
self.assertEqual(output, "We are being attacked! Attack Back!")
finally:
sys.stdout = saved_stdout
if __name__ == "__main__":
unittest.main()
So this obviously doesn't work. So what am I missing? Thank you all in advance!
EDITED: Here is the error message I get when I run the test. I understand the error, just don't know how I would go about fixing it.
Error
Traceback (most recent call last):
File "C:\Python33\lib\unittest\mock.py", line 1087, in patched
return func(*args, **keywargs)
TypeError: test_output() takes 1 positional argument but 2 were given
A function decorated by patch will take the Mock as an additional parameter. You need
#patch('builtins.input', return_value='yes')
def test_output(self, m):
where the second argument m will be a reference to the Mock object that replaces input when test_output is called.
From pydoc unittest.mock under patch:
If patch is used as a decorator and new is
omitted, the created mock is passed in as an extra argument to the
decorated function.
In addition to #chepner's answer, you'll need to use unittest.TestCase's assert methods rather than asserting yourself (pun intended)
class TestStuff(unittest.TestCase):
#patch('builtins.input', return_value='yes')
def test_output(self, new_input):
try:
out = io.StringIO()
sys.stdout = out
main()
output = out.getvalue().strip()
self.assertEqual(output, "We are being attacked! Attack Back!")
finally:
sys.stdout = saved_stdout
However that's probably not the best way to do what you're trying to do. You can patch more than one builtin, you know!
class TestStuff(unittest.TestCase):
#patch('builtins.input', return_value='yes')
#patch('builtins.print')
def test_output(self, new_print, new_input):
# the mocked functions are passed in opposite order
# to where they're decorated
main()
new_print.assert_called_with("We are being attacked! Attack Back!")
If the decorators are scary, you could even do:
class TestStuff(unittest.TestCase):
def test_output(self):
with patch('builtins.input', return_value='yes'), \
patch('builtins.print') as new_print:
main()
new_print.assert_called_with("We are being attacked! Attack Back!")
The following doctest fails:
import logging
logging.basicConfig(level=logging.DEBUG,format='%(message)s')
def say_hello():
'''
>>> say_hello()
Hello!
'''
logging.info('Hello!')
if __name__ == '__main__':
import doctest
doctest.testmod()
These pages
doctest & logging
use doctest and logging in python program
seem to suggest logging.StreamHandler(sys.stdout) and logger.addHandler(handler) but my attempts failed in this respect. (I am new to python, if it wasn't obvious .)
Please help me fix the above code so that the test passes.
Update on Jun 4, 2017: To answer 00prometheus' comments: The accepted answer to use doctest and logging in python program, when I asked this question, seemed unnecessarily complicated. And indeed it is, as the accepted answer here gives a simpler solution. In my highly biased opinion, my question is also clearer than the one I already linked in the original post.
You need to define a "logger" object. This is usually done after import with:
import sys
import logging
log = logging.getLogger(__name__)
When you want to log a message:
log.info('Hello!')
In the code that gets run like a script you set the basicConfig:
if __name__ == '__main__':
import doctest
logging.basicConfig(level=logging.DEBUG, stream=sys.stdout, format='%(message)s')
doctest.testmod()
Edit:
Ok, you were right. It doesn't work, but I got it to work...BUT DO NOT DO THIS! Just use print statements or return what you actually need to check. As your second link says this is just a bad idea. You shouldn't be checking logging output (its for logging). Even the original poster for that second link said they got it to work by switching their logging to using print. But here is the evil code that seems to work:
class MyDocTestRunner(doctest.DocTestRunner):
def run(self, test, compileflags=None, out=None, clear_globs=True):
if out is None:
handler = None
else:
handler = logging.StreamHandler(self._fakeout)
out = sys.stdout.write
logger = logging.getLogger() # root logger (say)
if handler:
logger.addHandler(handler)
try:
doctest.DocTestRunner.run(self, test, compileflags, out, clear_globs)
finally:
if handler:
logger.removeHandler(handler)
handler.close()
if __name__ == '__main__':
logging.basicConfig(level=logging.DEBUG, format='%(message)s')
tests = doctest.DocTestFinder().find(say_hello, __name__)
dt_runner = MyDocTestRunner()
for t in tests:
dt_runner.run(t, out=True)
Edit (continued):
My attempts also failed when trying what your second link suggested. This is because internally doctest reassigns sys.stdout to self._fakeout. That's why nothing short of my hack will work. I actually tell the logger to write to this "fakeout".
Edit (answer to comment):
It's not exactly the code from the link. If it was the code from the link I would say it's not that bad of an option because its not doing anything too complex. My code, however, is using a "private" internal instance attribute that shouldn't be used by a normal user. That is why it is evil.
And yes, logging can be used for testing output, but it does not make much sense to do so in a unittest/doctest and is probably why doctest doesn't include functionality like this out of the box. The TextTest stuff you linked to is all about functional or integration testing. Unittests (and doctests) should be testing small individual components. If you have to capture logged output to make sure your unittest/doctest is correct then you should maybe think about separating things out or not doing these checks in a doctest.
I personally only use doctests for simple examples and verifications. Mostly for usage examples since any user can see an inline doctest.
Edit (ok last one):
Same solution, simpler code. This code doesn't require that you create a custom runner. You still have to create the default runner and stuff because you need to access the "_fakeout" attribute. There is no way to use doctest to check logging output without logging to this attribute as a stream.
if __name__ == '__main__':
dt_runner = doctest.DocTestRunner()
tests = doctest.DocTestFinder().find(sys.modules[__name__])
logging.basicConfig(level=logging.DEBUG, format='%(message)s', stream=dt_runner._fakeout)
for t in tests:
dt_runner.run(t)
One way to do this is by monkey-patching the logging module (my code; docstring contents from import logging are relevant to your question):
#classmethod
def yield_int(cls, field, text):
"""Parse integer values and yield (field, value)
>>> test = lambda text: dict(Monster.yield_int('passive', text))
>>> test(None)
{}
>>> test('42')
{'passive': 42}
>>> import logging
>>> old_warning = logging.warning
>>> warnings = []
>>> logging.warning = lambda msg: warnings.append(msg)
>>> test('seven')
{}
>>> warnings
['yield_int: failed to parse text "seven"']
>>> logging.warning = old_warning
"""
if text == None:
return
try:
yield (field, int(text))
except ValueError:
logging.warning(f'yield_int: failed to parse text "{text}"')
However, a much cleaner approach uses the unittest module:
>>> from unittest import TestCase
>>> with TestCase.assertLogs(_) as cm:
... print(test('seven'))
... print(cm.output)
{}
['WARNING:root:yield_int: failed to parse text "seven"']
Technically you should probably instantiate a TestCase object rather than passing _ to assertLogs as self, since there's no guarantee that this method won't attempt to access the instance properties in the future.
I use the following technique:
Set the logging stream to a StringIO object.
Log away...
Print the contents for the StringIO object and expect the output.
Or: Assert against the contents of the StringIO object.
This should do it.
Here is some example code.
First it just does the whole setup for the logging within the doctest - just to show how it's working.
Then the code shows how the setup can be put into as seperate function setup_doctest_logging that does the setup ànd returns itself a function that prints the log. This keeps the test code more focused and moves the ceremonial part out of the test.
import logging
def func(s):
"""
>>> import io
>>> string_io = io.StringIO()
>>> # Capture the log output to a StringIO object
>>> # Use force=True to make this configuration stick
>>> logging.basicConfig(stream=string_io, format='%(message)s', level=logging.INFO, force=True)
>>> func('hello world')
>>> # print the contents of the StringIO. I prefer that. Better visibility.
>>> print(string_io.getvalue(), end='')
hello world
>>> # The above needs the end='' because print will otherwise add an new line to the
>>> # one that is already in the string from logging itself
>>> # Or you can just expect an extra empty line like this:
>>> print(string_io.getvalue())
hello world
<BLANKLINE>
>>> func('and again')
>>> # Or just assert on the contents.
>>> assert 'and again' in string_io.getvalue()
"""
logging.info(s)
def setup_doctest_logging(format='%(levelname)s %(message)s', level=logging.WARNING):
"""
This could be put into a separate module to make the logging setup easier
"""
import io
string_io = io.StringIO()
logging.basicConfig(stream=string_io, format=format, level=level, force=True)
def log_printer():
s = string_io.getvalue()
print(s, end='')
return log_printer
def other_logging_func(s, e=None):
"""
>>> print_whole_log = setup_doctest_logging(level=logging.INFO)
>>> other_logging_func('no error')
>>> print_whole_log()
WARNING no error
>>> other_logging_func('I try hard', 'but I make mistakes')
>>> print_whole_log()
WARNING no error
WARNING I try hard
ERROR but I make mistakes
"""
logging.warning(s)
if e is not None:
logging.error(e)
if __name__ == '__main__':
import doctest
doctest.testmod()
As mentioned by others, the issue is that doctest modifies sys.stdout after basicConfig created a StreamHandler that holds its own copy. One way to deal with this is to create a stream object that dispatches write and flush to sys.stdout. Another is to bypass the issue altogether by creating your own handler:
class PrintHandler(logging.Handler):
def emit(self, record):
print(self.format(record))
logging.basicConfig(level=logging.DEBUG, format='%(message)s',
handlers=[PrintHandler()])
Say I have a module with the following:
def main():
pass
if __name__ == "__main__":
main()
I want to write a unit test for the bottom half (I'd like to achieve 100% coverage). I discovered the runpy builtin module that performs the import/__name__-setting mechanism, but I can't figure out how to mock or otherwise check that the main() function is called.
This is what I've tried so far:
import runpy
import mock
#mock.patch('foobar.main')
def test_main(self, main):
runpy.run_module('foobar', run_name='__main__')
main.assert_called_once_with()
I will choose another alternative which is to exclude the if __name__ == '__main__' from the coverage report , of course you can do that only if you already have a test case for your main() function in your tests.
As for why I choose to exclude rather than writing a new test case for the whole script is because if as I stated you already have a test case for your main() function the fact that you add an other test case for the script (just for having a 100 % coverage) will be just a duplicated one.
For how to exclude the if __name__ == '__main__' you can write a coverage configuration file and add in the section report:
[report]
exclude_lines =
if __name__ == .__main__.:
More info about the coverage configuration file can be found here.
Hope this can help.
You can do this using the imp module rather than the import statement. The problem with the import statement is that the test for '__main__' runs as part of the import statement before you get a chance to assign to runpy.__name__.
For example, you could use imp.load_source() like so:
import imp
runpy = imp.load_source('__main__', '/path/to/runpy.py')
The first parameter is assigned to __name__ of the imported module.
Whoa, I'm a little late to the party, but I recently ran into this issue and I think I came up with a better solution, so here it is...
I was working on a module that contained a dozen or so scripts all ending with this exact copypasta:
if __name__ == '__main__':
if '--help' in sys.argv or '-h' in sys.argv:
print(__doc__)
else:
sys.exit(main())
Not horrible, sure, but not testable either. My solution was to write a new function in one of my modules:
def run_script(name, doc, main):
"""Act like a script if we were invoked like a script."""
if name == '__main__':
if '--help' in sys.argv or '-h' in sys.argv:
sys.stdout.write(doc)
else:
sys.exit(main())
and then place this gem at the end of each script file:
run_script(__name__, __doc__, main)
Technically, this function will be run unconditionally whether your script was imported as a module or ran as a script. This is ok however because the function doesn't actually do anything unless the script is being ran as a script. So code coverage sees the function runs and says "yes, 100% code coverage!" Meanwhile, I wrote three tests to cover the function itself:
#patch('mymodule.utils.sys')
def test_run_script_as_import(self, sysMock):
"""The run_script() func is a NOP when name != __main__."""
mainMock = Mock()
sysMock.argv = []
run_script('some_module', 'docdocdoc', mainMock)
self.assertEqual(mainMock.mock_calls, [])
self.assertEqual(sysMock.exit.mock_calls, [])
self.assertEqual(sysMock.stdout.write.mock_calls, [])
#patch('mymodule.utils.sys')
def test_run_script_as_script(self, sysMock):
"""Invoke main() when run as a script."""
mainMock = Mock()
sysMock.argv = []
run_script('__main__', 'docdocdoc', mainMock)
mainMock.assert_called_once_with()
sysMock.exit.assert_called_once_with(mainMock())
self.assertEqual(sysMock.stdout.write.mock_calls, [])
#patch('mymodule.utils.sys')
def test_run_script_with_help(self, sysMock):
"""Print help when the user asks for help."""
mainMock = Mock()
for h in ('-h', '--help'):
sysMock.argv = [h]
run_script('__main__', h*5, mainMock)
self.assertEqual(mainMock.mock_calls, [])
self.assertEqual(sysMock.exit.mock_calls, [])
sysMock.stdout.write.assert_called_with(h*5)
Blam! Now you can write a testable main(), invoke it as a script, have 100% test coverage, and not need to ignore any code in your coverage report.
Python 3 solution:
import os
from importlib.machinery import SourceFileLoader
from importlib.util import spec_from_loader, module_from_spec
from importlib import reload
from unittest import TestCase
from unittest.mock import MagicMock, patch
class TestIfNameEqMain(TestCase):
def test_name_eq_main(self):
loader = SourceFileLoader('__main__',
os.path.join(os.path.dirname(os.path.dirname(__file__)),
'__main__.py'))
with self.assertRaises(SystemExit) as e:
loader.exec_module(module_from_spec(spec_from_loader(loader.name, loader)))
Using the alternative solution of defining your own little function:
# module.py
def main():
if __name__ == '__main__':
return 'sweet'
return 'child of mine'
You can test with:
# Override the `__name__` value in your module to '__main__'
with patch('module_name.__name__', '__main__'):
import module_name
self.assertEqual(module_name.main(), 'sweet')
with patch('module_name.__name__', 'anything else'):
reload(module_name)
del module_name
import module_name
self.assertEqual(module_name.main(), 'child of mine')
I did not want to exclude the lines in question, so based on this explanation of a solution, I implemented a simplified version of the alternate answer given here...
I wrapped if __name__ == "__main__": in a function to make it easily testable, and then called that function to retain logic:
# myapp.module.py
def main():
pass
def init():
if __name__ == "__main__":
main()
init()
I mocked the __name__ using unittest.mock to get at the lines in question:
from unittest.mock import patch, MagicMock
from myapp import module
def test_name_equals_main():
# Arrange
with patch.object(module, "main", MagicMock()) as mock_main:
with patch.object(module, "__name__", "__main__"):
# Act
module.init()
# Assert
mock_main.assert_called_once()
If you are sending arguments into the mocked function, like so,
if __name__ == "__main__":
main(main_args)
then you can use assert_called_once_with() for an even better test:
expected_args = ["expected_arg_1", "expected_arg_2"]
mock_main.assert_called_once_with(expected_args)
If desired, you can also add a return_value to the MagicMock() like so:
with patch.object(module, "main", MagicMock(return_value='foo')) as mock_main:
One approach is to run the modules as scripts (e.g. os.system(...)) and compare their stdout and stderr output to expected values.
I found this solution helpful. Works well if you use a function to keep all your script code.
The code will be handled as one code line. It doesn't matter if the entire line was executed for coverage counter (though this is not what you would actually actually expect by 100% coverage)
The trick is also accepted pylint. ;-)
if __name__ == '__main__': \
main()
If it's just to get the 100% and there is nothing "real" to test there, it is easier to ignore that line.
If you are using the regular coverage lib, you can just add a simple comment, and the line will be ignored in the coverage report.
if __name__ == '__main__':
main() # pragma: no cover
https://coverage.readthedocs.io/en/coverage-4.3.3/excluding.html
Another comment by # Taylor Edmiston also mentions it
My solution is to use imp.load_source() and force an exception to be raised early in main() by not providing a required CLI argument, providing a malformed argument, setting paths in such a way that a required file is not found, etc.
import imp
import os
import sys
def mainCond(testObj, srcFilePath, expectedExcType=SystemExit, cliArgsStr=''):
sys.argv = [os.path.basename(srcFilePath)] + (
[] if len(cliArgsStr) == 0 else cliArgsStr.split(' '))
testObj.assertRaises(expectedExcType, imp.load_source, '__main__', srcFilePath)
Then in your test class you can use this function like this:
def testMain(self):
mainCond(self, 'path/to/main.py', cliArgsStr='-d FailingArg')
To import your "main" code in pytest in order to test it you can import main module like other functions thanks to native importlib package :
def test_main():
import importlib
loader = importlib.machinery.SourceFileLoader("__main__", "src/glue_jobs/move_data_with_resource_partitionning.py")
runpy_main = loader.load_module()
assert runpy_main()