We Keep Coding

How to dynamically load a function to test unit with exec()?

I want to load the function into the unit test dynamically, passing code as str and use exec() to run them in test.
My current code look like this:
import unittest
class ParametrizedTestCase(unittest.TestCase):
def __init__(self, methodName='runTest', param=None):
super(ParametrizedTestCase, self).__init__(methodName)
exec (param)
self.param = param
#staticmethod
def parametrize(testcase_klass, param=None):
testloader = unittest.TestLoader()
testname = testloader.getTestCaseNames(testcase_klass)
suite = unittest.TestSuite()
for name in testname:
suite.addTest(testcase_klass(name, param=param))
return suite
class TestZero(ParametrizedTestCase):
def test_add(self):
print('param:', self.param)
self.assertEqual(add(1,1), 2)
if __name__ == '__main__':
code = "def add(a, b): return a + b"
suite = ParametrizedTestCase.parametrize(TestZero, param=code)
unittest.TextTestRunner(verbosity=2).run(suite)
and this is current out put:
❯ python test/test0.py
test_add (__main__.TestZero) ... param: def add(a, b): return a + b
ERROR
======================================================================
ERROR: test_add (__main__.TestZero)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/test0.py", line 21, in test_add
self.assertEqual(add(1,1), 2)
NameError: name 'add' is not defined
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
Is it possible to do that and if so, how?

exec with no additional parameters is defining add in the local scope of ParameterizedTestCase.__init__; once that function complete, add goes out of scope.
One solution is to put add in the global scope, so that it's accessible.
exec(param, globals())
However, in general you aren't going to know what name (or names) the execution of param will make available for future tests.

Creating a Unit testcase Class within a function not running testcases

I need to create a unit test class with a function so i can call the function when some event is triggered. I am using the below method but testcases are not executing
gp = r"somefile"
def MyFunc():
if os.path.exists(gp):
print("yes")
class First__Test_Cases(unittest.TestCase):
def test_001(self):
print("1")
def test__002(self):
print("2")
if __name__ == '__main__':
unittest.main()
First__Test_Cases()
else:
print("fail")
MyFunc()
output - Ran 0 tests in 0.000s

Remove MyFunc() and global parts, it should only contain class and main
mytestfile.py
import unittest
class First_Test_Cases(unittest.TestCase):
def test_001(self):
Pass
def test__002(self):
Pass
if __name__ == '__main__':
unittest.main()
Then run
python mytestfile.py
And all tests in the class will be executed:
...
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
You can read more and see more examples in the documentation
If you need to have a function call the test, you should do that in a separate file. Check this post: Run unittests from a different file

According to what I understood from your change in code, this is your use case:
Tests should be run only if a certain file exists. Otherwise, they should be skipped.
For this use case, I would suggest the following solution:
import os
import unittest
gp = "some_file.txt"
msg = "file {} does not exist".format(gp)
class First_Test_Cases(unittest.TestCase):
#unittest.skipUnless(os.path.exists(gp), msg)
def test_001(self):
pass
#unittest.skipUnless(os.path.exists(gp), msg)
def test_002(self):
pass
if __name__ == '__main__':
unittest.main()
The output would be the following if the file does not exist:
ss
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK (skipped=2)
and this one, if it exists:
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
In case if you want your tests to fail, just change the code in this way:
import os
import unittest
gp = "some_file.txt"
msg = "file {} does not exist".format(gp)
class First_Test_Cases(unittest.TestCase):
def test_001(self):
self.assertTrue(os.path.exists(gp), msg) # add this line
# your code
def test_002(self):
self.assertTrue(os.path.exists(gp), msg) # add this line
# your code
Then, the output would be the following:
FF
======================================================================
FAIL: test_001 (__main__.First_Test_Cases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/test.py", line 11, in test_001
self.assertTrue(os.path.exists(gp), msg)
AssertionError: file some_file.txt does not exist
======================================================================
FAIL: test_002 (__main__.First_Test_Cases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/test.py", line 15, in test_002
self.assertTrue(os.path.exists(gp), msg)
AssertionError: file some_file.txt does not exist
----------------------------------------------------------------------
Ran 2 tests in 0.000s
FAILED (failures=2)

Unexpected behavior from unittest.mock.patch

What is wrong with my code below?
I'm expecting assert call_func_once_with("b") to throw an error as call_func was passed 'a'. I confimed that the function was indeed called once and with argument 'a'.
from unittest.mock import Mock, patch
def call_func(x):
pass
#patch("__main__.call_func")
def test_call_func(call_func):
call_func("a")
assert call_func.called_once_with("b")
assert call_func.called == 1
print(call_func.call_args)
test_call_func()
Output:
call('a')

You're not the first person to notice strange things with these types of assertions (see Magic mock assert_called_once vs assert_called_once_with weird behaviour)
For what it's worth, I can only advise that you try to create a test class which inherits from unittest.TestCase and then use the assertEqual method to get more consistent test behaviour:
import unittest
from unittest.mock import patch, call
def call_func(x):
pass
class MyTests(unittest.TestCase):
#patch("__main__.call_func")
def test_call_func(self, call_func_mock):
call_func_mock("a")
# assert call_func_mock.called == 1
# assert call_func_mock.called_once_with("b")
self.assertEqual(call_func_mock.call_count, 1)
self.assertEqual(call_func_mock.call_args_list[0], call("b"))
print(call_func_mock.call_args)
unittest.main()
This gives the following (expected) results:
F
======================================================================
FAIL: test_call_func (__main__.MyTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\unittest\mock.py", line 1179, in patched
return func(*args, **keywargs)
File "C:/scratch.py", line 16, in test_call_func
self.assertEquals(call_func_mock.call_args_list[0], call("b"))
AssertionError: call('a') != call('b')
----------------------------------------------------------------------
Ran 1 test in 0.003s
FAILED (failures=1)
Process finished with exit code 1

Filter tests after discover

I'm currently running my tests like this:
tests = unittest.TestLoader().discover('tests')
unittest.TextTestRunner().run(tests)
Now I want to run a specific test knowing his name (like test_valid_user) but not knowing his class. If there is more than one test with such name than I would like to run all such tests. Is there any way to filter tests after discover?
Or maybe there are other solutions to this problem (please note that it shouldn't be done from command line)?

You can use the unittest.loader.TestLoader.testMethodPrefix instance variable to change the test methods filter according to a different prefix than "test".
Say you have a tests directory with this king of unit tests:
import unittest
class MyTest(unittest.TestCase):
def test_suite_1(self):
self.assertFalse("test_suite_1")
def test_suite_2(self):
self.assertFalse("test_suite_2")
def test_other(self):
self.assertFalse("test_other")
You can write your own discover function to discover only test functions starting with "test_suite_", for instance:
import unittest
def run_suite():
loader = unittest.TestLoader()
loader.testMethodPrefix = "test_suite_"
suite = loader.discover("tests")
result = unittest.TestResult()
suite.run(result)
for test, info in result.failures:
print(info)
if __name__ == '__main__':
run_suite()
remark: the argument "tests" in the discover method is a directory path, so you may need to write a fullpath.
As a result, you'll get:
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 8, in test_suite_1
self.assertFalse("test_suite_1")
AssertionError: 'test_suite_1' is not false
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 11, in test_suite_2
self.assertFalse("test_suite_2")
AssertionError: 'test_suite_2' is not false

Another simpler way, would be to use py.test with the -k option which does a test name keyword scan. It will run any tests whose name matches the keyword expression.
Although that is using the command-line which you didn't want, please not that you can call the command-line from your code using subprocess.call to pass any arguments you want dynamically.
E.g.: Assuming you have the following tests:
def test_user_gets_saved(self): pass
def test_user_gets_deleted(self): pass
def test_user_can_cancel(self): pass
You can call py.test from cli:
$ py.test -k "test_user"
Or from code:
return_code = subprocess.call('py.test -k "test_user"', shell=True)

There are two ways to run a single test method:
Command line:
$ python -m unittest test_module.TestClass.test_method
Using Python script:
import unittest
class TestMyCode(unittest.TestCase):
def setUp(self):
pass
def test_1(self):
self.assertTrue(True)
def test_2(self):
self.assertTrue(True)
if __name__ == '__main__':
testSuite = unittest.TestSuite()
testSuite.addTest(TestMyCode('test_1'))
runner=unittest.TextTestRunner()
runner.run(testSuite)
Output:
------------------------------------------------------------
Ran 1 test in 0.000s
OK

Python Unittest and object initialization

My Python version is 3.5.1
I have a simple code (tests.py):
import unittest
class SimpleObject(object):
array = []
class SimpleTestCase(unittest.TestCase):
def test_first(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
def test_second(self):
simple_object = SimpleObject()
simple_object.array.append(1)
self.assertEqual(len(simple_object.array), 1)
if __name__ == '__main__':
unittest.main()
If I run it with command 'python tests.py' I will get the results:
.F
======================================================================
FAIL: test_second (__main__.SimpleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 105, in test_second
self.assertEqual(len(simple_object.array), 1)
AssertionError: 2 != 1
----------------------------------------------------------------------
Ran 2 tests in 0.003s
FAILED (failures=1)
Why it is happening? And how to fix it. I expect that each tests run will be independent (each test should pass), but it is not as we can see.

The array is shared by all instances of the class. If you want the array to be unique to an instance you need to put it in the class initializer:
class SimpleObject(object):
def __init__(self):
self.array = []
For more information take a look at this question: class variables is shared across all instances in python?

This can also be accomplished directly in unittest if you prefer to use only 1 class. Implement the setUp class. setUp runs before any of the tests are run as the class is instantiated. It is similar to init but conforms to the unittest library. Note the opposite is tearDown which is executed at the end of the test class if you need to build and deprecate objects. Example:
class SimpleObject(unitest.TestCase):
def setUp(self):
# I run first
self.array = []
def some_test(self):
self.assertTrue(self.array == [])
<some test code here>
def tearDown(self):
# I run last
self.array = []. # or whatever teardown code you need
enjoy!