We Keep Coding

How to test if a function calls range in python?

I'm a Python instructor and I wanted to give my students a task: write a function that computes the average of a list using a for loop and the range object.
I wanted to run a test on their function to see whether it actually uses the range object. How can I do that?
It should be something like this:
def avg(L):
Pass
def test_range(avg):
...
If avg contains range, then test_range should return True.
I tried solutions that utilize func_code, but apparantly range doesn't have that.

You can use Python's unittest.mock module to wrap around the range function from the builtins module, and then let your test assert that the wrapped range was indeed called.
For example, using Python's unittest framework for writing the test:
import builtins
import unittest
from unittest.mock import patch
# I don't know what L is supposed to be, and I know that there
# are better ways to compute average of a list, but the code
# for calculating the average is not important for this question.
def avg(L):
total = 0
for index in range(len(L)):
total += L[index]
return total / len(L)
class TestAverage(unittest.TestCase):
def test_avg(self):
with patch("builtins.range", wraps=builtins.range) as wrapped_patch:
expected = 47
actual = avg([1,49,91])
self.assertEqual(expected, actual)
wrapped_patch.assert_called()
if __name__ == '__main__':
unittest.main()
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
It uses unittest.mock's patch targetting the builtins.range function. Normally, patch is used in tests to replace the behavior and/or return value of the target, but in this case, you can pass wraps=builtins.range (which gets passed to the underlying Mock object), which means "I just want to spy on the call, but not modify its behavior":
wraps: Item for the mock object to wrap. If wraps is not None then calling the Mock will pass the call through to the wrapped object (returning the real result).
By wrapping it in a Mock object, you can use any of Mock's assert functions to check calls to range, like assert_called which checks whether the target was called at least once. You can be more specific by asserting the number of times range was called:
self.assertTrue(wrapped_patch.call_count == 1)
The assertion would fail if it wasn't called at all:
# Here, `range` wasn't used at all.
def avg(L):
return sum(L) / len(L)
class TestAverage(unittest.TestCase):
# same as the code above
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... FAIL
======================================================================
FAIL: test_avg (main.TestAverage)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/main.py", line 15, in test_avg
wrapped_patch.assert_called()
File "/usr/local/Cellar/python#3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/mock.py", line 888, in assert_called
raise AssertionError(msg)
AssertionError: Expected 'range' to have been called.
The most important thing to note when using patch, is to know exactly where to patch. In this case, you can check the docs or use __module__ to know range's module:
>>> range
<class 'range'>
>>> range.__module__
'builtins'
But the test is a bit naive, because it can still pass even though avg didn't really use range:
def avg(L):
range(len(L)) # Called but really unused. Sneaky!
return sum(L) / len(L)
class TestAverage(unittest.TestCase):
# same as the code above
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
A slightly confusing workaround would be a test that "breaks" range such that, if the function was really using range, then it wouldn't work anymore:
def avg(L):
range(len(L)) # Called but really unused. Sneaky!
return sum(L) / len(L)
class TestAverage(unittest.TestCase):
def test_avg(self):
# same as above
def test_avg_is_really_using_range(self):
L = [10,20,90]
# Is it returning the correct result?
self.assertEqual(avg(L), 40)
# OK, but did it really use `range`?
# Let's try breaking `range` so it always yields 0,
# so we expect the return value to be *different*
with patch("builtins.range", return_value=[0,0,0]):
self.assertNotEqual(avg(L), 40)
So, if avg was sneakily calling but not really using range, then test_avg_is_really_using_range would now fail because avg still yields the correct value even with a broken range:
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... ok
test_avg_really_using_range (main.TestAverage) ... FAIL
======================================================================
FAIL: test_avg_really_using_range (main.TestAverage)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/main.py", line 19, in test_avg_really_using_range
self.assertNotEqual(avg(L), 40)
AssertionError: 40.0 == 40
Lastly, as a side note, I'm using assertEqual here in all the example because the test for the return value is not the focus, but do read up on proper ways to assert possible float values, ex. How to perform unittest for floating point outputs? - python

Remove/overwrite import

I'm trying to set up a grading script for an intro CS class using unittest. Essentially, students submit a python file student.py, which has some number of functions in it that are generally interdependent (meaning that func3() may use func1() in it's computations).
I'm writing unit tests for each method by comparing the output of student.func1 to the output of correct.func1, a method that's known to be the correct implementation (from a file correct.py).
As an example, say that func2 uses func1 in it's computation.
So, either by default or upon student.func1 failing some test, I want to overwrite student.func1 with correct.func1, so student.func2 uses a known correct implementation (and therefore isn't just wrong by default). How could I go about doing that? It seems like the setUp() and tearDown() are similar to what I want, but I don't know how to "unimport" modules in python, and haven't found any resources about it so far.
I'm interested in both the case where student.py contains classes, and func1,func2 are methods of a specific class, and when func1 and func2 are just defined generically within student.py.

The easiest way would be to import student into your module, then catch an AssertionError if a test fails, and replace the bad code inside the student module with your own good code:
import student
import unittest
def safe_f1():
print("Safe f1")
return 1
class TestSomething(unittest.TestCase):
def test_f1(self):
try:
self.assertEqual(student.func1(), 1)
except AssertionError:
student.func1 = safe_f1
raise
def test_f2(self):
self.assertEqual(student.func2(), 2)
Here's a dummy student.py that fails/works:
def func1():
print("Bad f1")
return 2
def func2():
return func1() + 1
return 2
When I run this, I get:
$ python -m unittest test.py
Bad f1
FSafe f1
.
======================================================================
FAIL: test_f1 (test.TestSomething)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/austin/Code/so/test.py", line 13, in test_f1
self.assertEqual(student.func1(), 1)
AssertionError: 2 != 1
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)

Python test to check instance type

I want to use unittest in python to check if a method returns object of the right class.
Every example in the web shows tests for 'type' returned.
For example, to check for <type 'list'> or <type 'type'> , we could use:
self.assertIsInstance(result, list)
self.assertIsInstance(result[0], tuple)
What I am looking for is an example to check for <class'sqlalchemy.orm.query.Query'>
Would appreciate any help. Thankyou.

You could use assertIsInstance(), presumably using isinstance() which is the recommended function for testing types. You could also assertIs() or assertTrue() combined with type() depending on the context:
#assert.py
import unittest
class TestType(unittest.TestCase):
def setUp(self):
self.number = 1
def test_assert_true(self):
self.assertTrue(type(self.number) is int)
def test_assert_is_instance(self):
self.assertIsInstance(self.number, int)
def test_assert_is_with_type(self):
self.assertIs(type(self.number), int)
def test_assert_is(self):
self.assertIs(self.number, int)
if __name__ == '__main__':
unittest.main()
$ python assert.py
test_assert_is (__main__.TestType) ... FAIL
test_assert_is_instance (__main__.TestType) ... ok
test_assert_is_with_type (__main__.TestType) ... ok
test_assert_true (__main__.TestType) ... ok
======================================================================
FAIL: test_assert_is (__main__.TestType)
----------------------------------------------------------------------
Traceback (most recent call last):
File "assert.py", line 19, in test_assert_is
self.assertIs(self.number, int)
AssertionError: 1 is not <type 'int'>
----------------------------------------------------------------------
Ran 4 tests in 0.000s
FAILED (failures=1)
The assertion error of the test_assert_is(self) might lead one to believe the type of 1 is not integer however it's comparing the object represented by 1 and the object that describes the type of an integer. This is most likely why isinstance() is preferred since it's more verbose about what it's checking and less typing is involved, so in general less error prone.

This should work:
self.assertIsInstance(result, sqlalchemy.orm.query.Query)
You need to have import sqlalchemy in the file.

Comparison of multi-line strings in Python unit test

When I compare two Unicode strings in a Python unit test, it gives a nice failure message highlighting which lines and characters are different. However, comparing two 8-bit strings just shows the two strings with no highlighting.
How can I get the highlighting for both Unicode and 8-bit strings?
Here is an example unit test that shows both comparisons:
import unittest
class TestAssertEqual(unittest.TestCase):
def testString(self):
a = 'xax\nzzz'
b = 'xbx\nzzz'
self.assertEqual(a, b)
def testUnicode(self):
a = u'xax\nzzz'
b = u'xbx\nzzz'
self.assertEqual(a, b)
if __name__ == '__main__':
unittest.main()
The results of this test show the difference:
FF
======================================================================
FAIL: testString (__main__.TestAssertEqual)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/mnt/data/don/workspace/scratch/scratch.py", line 7, in testString
self.assertEqual(a, b)
AssertionError: 'xax\nzzz' != 'xbx\nzzz'
======================================================================
FAIL: testUnicode (__main__.TestAssertEqual)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/mnt/data/don/workspace/scratch/scratch.py", line 12, in testUnicode
self.assertEqual(a, b)
AssertionError: u'xax\nzzz' != u'xbx\nzzz'
- xax
? ^
+ xbx
? ^
zzz
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=2)
Update for Python 3
In Python 3, string literals are Unicode by default, so this is mostly irrelevant. assertMultiLineEqual() no longer supports byte strings, so you're pretty much stuck with regular assertEqual() unless you're willing to decode the byte strings to Unicode.

A little digging in the Python source code shows that TestCase registers a bunch of methods to test equality for different types.
self.addTypeEqualityFunc(dict, 'assertDictEqual')
self.addTypeEqualityFunc(list, 'assertListEqual')
self.addTypeEqualityFunc(tuple, 'assertTupleEqual')
self.addTypeEqualityFunc(set, 'assertSetEqual')
self.addTypeEqualityFunc(frozenset, 'assertSetEqual')
try:
self.addTypeEqualityFunc(unicode, 'assertMultiLineEqual')
except NameError:
# No unicode support in this build
pass
You can see that unicode is registered to use assertMultiLineEqual(), but str is not registered for anything special. I have no idea why str is left out, but so far I have been happy with either of the following two methods.
Call Directly
If an 8-bit string isn't registered to use assertMultiLineEqual() by default, you can still call it directly.
def testString(self):
a = 'xax\nzzz'
b = 'xbx\nzzz'
self.assertMultiLineEqual(a, b)
Register String Type
You can also register it yourself. Just add an extra line to your test case's setUp() method. Do it once, and all your test methods will use the right method to test equality. If your project has a common base class for all test cases, that would be a great place to put it.
class TestAssertEqual(unittest.TestCase):
def setUp(self):
super(TestAssertEqual, self).setUp()
self.addTypeEqualityFunc(str, self.assertMultiLineEqual)
def testString(self):
a = 'xax\nzzz'
b = 'xbx\nzzz'
self.assertEqual(a, b)
def testUnicode(self):
a = u'xax\nzzz'
b = u'xbx\nzzz'
self.assertEqual(a, b)
Either of these methods will include highlighting when the string comparison fails.

Python 2.7 mock/patch: understanding assert_called_XYZ()

I'm relatively new to Python and unit testing in Python. From the Java world I know the concept of mocking but it seem to be much different from what I can see in Python.
I found this guide, which I found very helpful: http://www.voidspace.org.uk/python/mock/index.html
But as I wrote my (a bit more complex) tests with mocked out dependencies I noticed a strage behavior.
I decided to create a reduced, simple example which also does not work as I expect it.
Take a look at this, the result and my expectation I have added as comments:
import unittest
from mock import patch, Mock, MagicMock
class BasicTest(unittest.TestCase):
#patch("StringIO.StringIO")
def testSomethingNotWorkingAsExpected(self, StringIOMock):
StringIOMock.assert_called_once() # asserts, but why?
#patch("StringIO.StringIO")
def testSomethingSomehowWorking(self, StringIOMock):
# self.instantiateStringIO() # intentionally commented out
assert StringIOMock.called # does not assert (leading to failure of this test); as expected. If the above line is not commented, this asserts as expected.
def instantiateStringIO(self):
import StringIO
StringIO.StringIO()
Why is assert_called_once() asserting the instantiation of StringIO even it has not been instantiated yet?
And why does assert ClassMock.called bring the expected results?
Using assert not ... to assert a method has not been called I found here: Assert a function/method was not called using Mock. I inverted this pattern in my case by omitting the not.
Somewhere I found the pattern ClassMock.return_value to reference an instance. But I understand this as a way to manupulate the instance of a Mock before it will be called, not as a way to access the instance that might an underliing code have internally created. Or am I wrong?
My environment:
Python 2.7.3
mock 0.8.8
Fedora 19
Probably my understanding of the mock/patch thing is wrong. Could please someone aditionally explain what a class mock does and how it works?
Edit1: Added output
... and added paraphrase in parens to comment in testSomethingSomehowWorking
This is the output:
.F
======================================================================
FAIL: testSomethingSomehowWorking (test_test.BasicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "test_test.py", line 15, in testSomethingSomehowWorking
assert StringIOMock.called # does not assert; as expected
AssertionError
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)

The method assert_called_once does not exist and it does not perform an assertion. It's no different from writing StringIOMock.assert_foo_bar_does_not_exist() or any other method. The mock library doesn't check whether the method called on the mock actually exists.
If you use assert_called_once_with then it fails as expected.
You can use the spec parameter to raise an error when you call a non-existent method:
#patch("StringIO.StringIO", spec=StringIO.StringIO)
def testSomethingNotWorkingAsExpected(self, StringIOMock):
StringIOMock.assert_called_once() # will fail as the method doesn't exist