I have a python module/script which does a few of these
At various nested levels inside the script I take command line inputs, validate them, apply sensible defaults
I also check if a few directories exist
The above are just two examples. I am trying to find out what is the best "strategy" to test this. What I have done is that I have constructed wrapper functions around raw_input and os.path.exists in my module and then in my test I override these two functions to take input from my array list or do some mocked behaviour. This approach has the following disadvantages
Wrapper functions just exist for the sake of testing and this pollutes the code
I have to remember to use the wrapper function in the code everytime and not just call os.path.exists or raw_input
Any brilliant suggestions?
The short answer is to monkey patch these system calls.
There are some good examples in the answer to How to display the redirected stdin in Python?
Here is a simple example for raw_input() using a lambda that throws away the prompt and returns what we want.
System Under Test
$ cat ./name_getter.py
#!/usr/bin/env python
class NameGetter(object):
def get_name(self):
self.name = raw_input('What is your name? ')
def greet(self):
print 'Hello, ', self.name, '!'
def run(self):
self.get_name()
self.greet()
if __name__ == '__main__':
ng = NameGetter()
ng.run()
$ echo Derek | ./name_getter.py
What is your name? Hello, Derek !
Test case:
$ cat ./t_name_getter.py
#!/usr/bin/env python
import unittest
import name_getter
class TestNameGetter(unittest.TestCase):
def test_get_alice(self):
name_getter.raw_input = lambda _: 'Alice'
ng = name_getter.NameGetter()
ng.get_name()
self.assertEquals(ng.name, 'Alice')
def test_get_bob(self):
name_getter.raw_input = lambda _: 'Bob'
ng = name_getter.NameGetter()
ng.get_name()
self.assertEquals(ng.name, 'Bob')
if __name__ == '__main__':
unittest.main()
$ ./t_name_getter.py -v
test_get_alice (__main__.TestNameGetter) ... ok
test_get_bob (__main__.TestNameGetter) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Solution1: I would do something like this beacuse it works:
def setUp(self):
self._os_path_exists = os.path.exists
os.path.exists = self.myTestExists # mock
def tearDown(self):
os.path.exists = self._os_path_exists
It is not so nice.
Solution2: Restructuring your code was not an option as you said, right?
It would make it worse to understand and unintuitive.
Johnnysweb is spot on with what you need to do, but instead of rolling your own, you can import and use mock. Mock is specifically designed for unit testing, and makes it extremely simple to do what you're trying to do. It's built-in to Python 3.3.
For example, if want to run a unit test that replaces os.path.isfile and always returns True:
try:
from unittest.mock import patch
except ImportError:
from mock import patch
class SomeTest(TestCase):
def test_blah():
with patch("os.path.isfile", lambda x: True):
self.assertTrue(some_function("input"))
This can save you a LOT of boilerplate code, and it's quite readable.
If you need something a bit more complex, for example, replacing supbroccess.check_output, you can create a simple helper function:
def _my_monkeypatch_function(li):
x,y = li[0], li[1]
if x == "Reavers":
return "Gorram"
if x == "Inora":
return "Shiny!"
if x == y:
return "The Ballad of Jayne"
def test_monkey():
with patch("subprocess.check_output", _my_monkeypatch_function):
assertEquals(subprocess.check_output(["Mudder","Mudder"]),
"The Ballad of Jayne")
Related
I would like to programmatically add or mark a function as a test-case in pytest, so instead of writing
def test_my_function():
pass
I would like to do something like (pseudo-api, I know neither pytest.add_test nor pytest.testcase exist by that identifier).
def a_function_specification():
pass
pytest.add_test(a_function_specification)
or
I would like to do something like
#pytest.testcase
def a_function_specification():
pass
Basically I would like to write some test-case-generating decorator that isn't exactly working like pytest.mark/parametrizing which is why I started to dig into the internals but I haven't found an obvious way how this can be done for python code.
The YAML example in the pytest docs seem to use pytest.Item but I have a hard time mapping this to something that would work within python and not as part of a non-Python-file test collection.
Starting from version 2.6, pytest support:
nose-style __test__ attribute on modules, classes and functions, including unittest-style Classes. If set to False, the test will not be collected.
So, you need to add this attribute.
One approach is:
def not_a_test1():
assert 1 + 2 == 3
not_a_test1.__test__ = True
Another is:
def make_test(func):
func.__test__ = True
return func
#make_test
def not_a_test2():
assert 1 + 2 == 3
I was wondering how can I unit test if a recursive function has been called correctly. For example this function:
def test01(number):
if(len(number) == 1):
return 1
else:
return 1+test01(number[1:])
It counts recursvely how many digits a number has (assuming the number type is string)
So, I want to test if the function test01 has been called recursively. It would be ok if it is implemented just like that, but not if it is implemented as:
def test01(number):
return len(number)
EDIT:
The recursive approach is mandatory for educational purposes, so the UnitTest process will automate programming exercises checking. Is there a way to check if the function was called more than once? If that is possible, I can have 2 tests, one asserting the correct output and one to check if the function was called more than once for the same input.
Thank you in advance for your help
Guessing by the tags I assume you want to use unittest to test for the recursive call. Here is an example for such a check:
from unittest import TestCase
import my_module
class RecursionTest(TestCase):
def setUp(self):
self.counter = 0 # counts the number of calls
def checked_fct(self, fct): # wrapper function that increases a counter on each call
def wrapped(*args, **kwargs):
self.counter += 1
return fct(*args, **kwargs)
return wrapped
def test_recursion(self):
# replace your function with the checked version
with mock.patch('my_module.test01',
self.checked_fct(my_module.test01)): # assuming test01 lives in my_module.py
result = my_module.test01('444') # call the function
self.assertEqual(result, 3) # check for the correct result
self.assertGreater(self.counter, 1) # ensure the function has been called more than once
Note: I used import my_module instead of from my_module import test01 so that the first call is also mocked - otherwise the number of calls would be one too low.
Depending on how your setup looks like, you may add further tests manually, or auto-generate the test code for each test, or use parametrization with pytest, or do something else to automate the tests.
Normally a unit test should check at least that your function works and try to test all code paths in it
Your unit test should therefore try to take the main path several times, and then find the exit path, attaining full coverage
You can use the 3rd-party coverage module to see if all your code paths are being taken
pip install coverage
python -m coverage erase # coverage is additive, so clear out old runs
python -m coverage run -m unittest discover tests/unit_tests
python -m coverage report -m # report, showing missed lines
Curtis Schlak taught me this strategy recently.
It utilizes Abstract Syntax Trees and the inspect module.
All my best,
Shawn
import unittest
import ast
import inspect
from so import test01
class Test(unittest.TestCase):
# Check to see if function calls itself recursively
def test_has_recursive_call(self):
# Boolean switch
has_recursive_call = False
# converts function into a string
src = inspect.getsource(test01)
# splits the source code into tokens
# based on the grammar
# transformed into an Abstract Syntax Tree
tree = ast.parse(src)
# walk tree
for node in ast.walk(tree):
# check for function call
# and if the func called was "test01"
if (
type(node) is ast.Call
and node.func.id == "test01"
):
# flip Boolean switch to true
has_recursive_call = True
# assert: has_recursive_call should be true
self.assertTrue(
has_recursive_call,
msg="The function does not "
"make a recursive call",
)
print("\nThe function makes a recursive call")
if __name__ == "__main__":
unittest.main()
Example
def main(p):
if foo_a(p):
return False
return p**2
def foo_a(p):
return p % 11 == 0
Now you can get 100% test coverage by
import unittest
from script import main
class Foobar(unittest.TestCase):
def test_main(self):
self.assertEquals(main(3), 9)
But maybe one wanted foo_a to be p % 2 == 0 instead.
The question
Branch coverage would shed a light on it, but I would also like to know if a function was never called "directly" by a test (such as main is in the example), but only indirectly (such as foo_a in the example).
Is this possible with pytest?
First of all just general line of thought is to unittest foo_a as well
import unittest
from script import main, foo_a
class Foobar(unittest.TestCase):
def test_main(self):
self.assertEquals(main(3), 9)
def test_foo_a(self):
self.assertEquals(foo_a(11), True)
You are probably looking for https://coverage.readthedocs.io/en/coverage-4.5.1/ which can be used with pytest https://pypi.org/project/pytest-cov/, this tool can show you exactly which lines of code had been called during testing
But I think there is another way to check your problem it is called mutation testing, here are some libraries that could help you with it
https://github.com/sixty-north/cosmic-ray
https://github.com/mutpy/mutpy
And also look into property based testing libraries like https://github.com/HypothesisWorks/hypothesis/tree/master/hypothesis-python
I'm learning how to write tests with py.test, but I can't get idea how to mock external calls.
Let say I have a code for testing:
app.py:
import random
def trade_robot(stock, price):
return ' '.join((random.choice(('buy', 'sell', 'keep')), stock))
And I want to be sure I check case for random.choice deciding I need to buy (and I want to check it properly join strings for 'buy' case).
my test code should look like this:
import pytest
from app import trade_robot
import sys
def test_buy_case():
# some mock magic here
assert trade_robot('AAPL', 500) == 'buy AAPL'
if __name__ == "__main__":
pytest.main("-v %s" % sys.argv[0])
What should be in 'some mock magic' to make this test passing every time? Thanks!
That can be done with the library mock, which is included into standard library since Python 3.3.
First, with a context manager:
import random
with mock.patch.object(random, 'choice') as m:
m.return_value = 'buy'
print random.choice(['anything']) # prints 'buy' regardless of the arguments
The same with a decorator:
#mock.patch.object(random, 'choice')
def test_buy(m):
m.return_value = 'buy'
print random.choice(['anything'])
The library also allows making assertions regarding mock calls and irreplaceable for unit testing. Alternatively, some people prefer explicit dependency inversion, which means passing the function random.choice into your code as a function/method/constructor argument and replacing it with a mock in a test.
You could use the 'patch' decorator to mock external calls:
http://www.voidspace.org.uk/python/mock/patch.html
E.g. you could mock the 'random.choice' with
#patch('random.choice')
def test_foo(choice):
choice.return_value = some value
The key is where to patch:
http://www.voidspace.org.uk/python/mock/patch.html#id1
I have a module testing system in Python where individual modules call something like:
class Hello(object):
_DOC_ATTR = { 'greeting': '''
a greeting message.
>>> h = Hello()
>>> h.greeting = 'hi there'
>>> h.greeting
'hi there'
''' }
def __init__(self):
self.greeting = "hello"
class Test(unittest.TestCase):
# tests here
if __name__ == '__main__':
import tester
tester.test(Test)
inside tester, I run the tests in Test along with a doctest on "__main__". This works great and has worked fine for a long time. Our specialized _DOC_ATTR dictionary documents individual attributes on the function when we build into Sphinx. However, doctests within this dictionary are not called. What I would like to do is within tester.test() to run doctests on the values in each class's _DOC_ATTR as well.
The problem that I'm having is trying to find a way within tester.test() to figure out all the variables (specifically classes) defined in __main__. I've tried looking at relevant places in traceback to no avail. I thought that because I was passing in a class from __main__, namely __main__.Test that I'd be able to use the .__module__ from Test to get access to the local variables there, but I can't figure out how to do it.
I would rather not need to alter the call to tester.test(Test) since it's used in hundreds of modules and I've trained all the programmers working on the project to follow this paradigm. Thanks for any help!
I think that I may have found the answer:
import inspect
stacks = inspect.stack()
if len(stacks) > 1:
outerFrame = stacks[1][0]
else:
outerFrame = stacks[0][0]
localVariables = outerFrame.f_locals
for lv in list(localVariables.keys()):
lvk = localVariables[lv]
if (inspect.isclass(lvk)):
docattr = getattr(lvk, '_DOC_ATTR', None)
if docattr is not None:
# ... do something with docattr ...
Another solution: since we are passing the "Test" class in, and in order to run there needs to be a "runTest" function defined, one could also use the func_globals on that function. Note that it cannot be a function inherited from a superclass, such as __init__, so it may have limited functionality for wider use cases.
import inspect
localVariables = Test.runTest.func_globals
for lv in list(localVariables.keys()):
lvk = localVariables[lv]
if (inspect.isclass(lvk)):
#### etc.