We Keep Coding

Doc-tests run from module and command-line and not from pre-commit hook

I want to run doc-tests of a Python script as part of a pre-commit hook in Python.
In the file set_prefix.py, I have doc-tests in front of functions, which I test before running with:
import doctest
import sys
EXTENSIONS = tuple([".%s" % ending for ending in ["jpg", "heic", "nrw"]])
def is_target_for_renaming(filepath):
"""Returns true if this filepath should be renamed.
>>> is_target_for_renaming("/Users/username/Pictures/document.other_jpg")
True
"""
return filepath.lower().endswith(EXTENSIONS)
def get_failed_tests():
r = doctest.testmod()
return r.failed
def main():
pass
if "__main__" == __name__:
args = sys.argv
test_only = 2 <= len(sys.argv) and "test" == sys.argv[1]
test_failures = get_failed_tests()
print(test_failures)
assert 0 == test_failures
if not test_only:
main()
When I run python3 set_prefix.py test, I get the error I expected.
Yet, when I import the module and call the function:
import set_prefix
if "__main__" == __name__:
test_failures = set_prefix.get_failed_tests()
print(test_failures)
I get 0 failures:
$ python3 temp.py
0
The reason I want to import the module is to run the tests in a pre-commit hook similar to that added by flake8:
#!/usr/local/opt/python/bin/python3.7
import sys
from flake8.main import git
if __name__ == '__main__':
sys.exit(
git.hook(
strict=git.config_for('strict'),
lazy=git.config_for('lazy'),
)
)
Why do the doc-tests run when called from the command-line and the script and not when the script is imported? Would unittest be a better framework, as described in this thread?

doctest.testmod()
runs doctests in __main__ module and it depends on which script you're actually running.
You can fix this with m parameter, but you'll still be forced to add boilerplate code in each module that has doctests. Try this:
doctest.testfile("some_module.py")

Recognizing module level tests

How can I get test_greet to run in the below; note: test_one(when uncommented) is seen and run by the test runner; to be specific, I want the line unittest.main() to correctly pick up the module level test (test_greet).
import unittest
#class MyTests(unittest.TestCase):
# def test_one(self):
# assert 1==2
def test_greet():
assert 1==3
if __name__=="__main__":
unittest.main()

Let's say i have a file called MyTests.py as below:
import unittest
class MyTests(unittest.TestCase):
def test_greet(self):
self.assertEqual(1,3)
Then:
Open a CMD in the folder that MyTests.py exists
Run python -m unittest MyTests
Please note that, all your tests must have the test_ otherwise, it will not be run.

unittest is not running my tests after reorganising file structure

Here is my structure:
directory/
__init__.py (blank)
myClass.py
test/
UnitTest.py
IntegrationTest.py
__init__.py (blank)
Let's use UnitTest as the example (both of them result in 0 tests). Feel free to critique my imports, I was fiddling around with it forever to import the class correctly so I can execute the UnitTest script. Also yes, I did copy a lot of code from other stack exchange questions in my search.
Before I moved to this structure, I had everything in one directory, so I know the test file works (outside of the imports)
import sys
from pathlib import Path
if __name__ == '__main__' and __package__ is None:
file = Path(__file__).resolve()
parent, top = file.parent, file.parents[2]
sys.path.append(str(top))
try:
sys.path.remove(str(parent))
except ValueError: # Already removed
pass
import directory.test
__package__ = 'directory.test'
from myClass import myClass
import unittest
from unittest import mock
print("Running Unit Tests...")
mc = myClass(params)
def mockingResponse(*args, **kwargs):
class MockResponse:
def __init__(self, json_data, status_code):
self.json_data = json_data
self.status_code = status_code
def json(self):
return self.json_data
#if/elses that return value
#mock.patch('myClass.requests.get', side_effect = mockingResponse)
class unitTest(unittest.TestCase):
print("this should run before test call")
def testsomefunc():
response = mc.somefunc()
differenceSet = set(response) ^ set(mockrespose from if/elses)
assert len(differenceSet) == 0
def testotherfunc():
#7 tests in total, same layout, different mockresponse
print("is this actually running")
unittest.main()
Then I get this in terminal:
privacy:~/git/directory$ python3 -m test.UnitTest.py
Running Unit Tests...
this should run before test call
is this actually running
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Checking online, I know it's not the issue of not having my names start with test and it's not because I have it indented wrong. Can't find much else in my search.
My only other thought is that it might be because I ran it as a module, but if I don't I have more problems with importing myClass. Of the many solutions I've tried for importing, this is the only one that has worked thus far.

Python unittest with TestLoader results in ModuleNotFoundError

i try to run a unittest with unittest.TestLoader and unittest.TextTestRunner but get an ModuleNotFoundError everytime i try to run the 'main' test file (here: test_all.py). I have the following file structure:
src.
test_all.py
dir1.
__init__.py
module1.py
submodule1.py
test_module1.py
dir2.
__init__.py
module2.py
submodule2.py
test_module2.py
dir3.
...
The test_all.py file looks like this:
# test_all.py
import os
import unittest
loader = unittest.TestLoader()
suite = loader.discover(os.getcwd())
runner = unittest.TextTestRunner()
runner.run(suite)
And finally the structure of the single testcases look like this:
# test_module1.py
import unittest
from module1 import Module1
class Module1TestCase(unittest.TestCase):
def setUp(self):
# do something
def test_something(self):
# test test
def tearDown(self):
# do something
if __name__ == '__main__':
unittest.main()
So, running the test_all.py always results in an ModuleNotFoundError referencing to the from module1 import Module1 inside the TestCase test_module1.py (and the same in the following TestCases). As far as i can tell there are no circular dependencies. Maybe adding the current Path to the PythonPath would work, but it really makes no sense to me: on the one hand i run the test_all.pyas main in the current directory and on the other the unittest.TestLoader.discover() already takes the current path.
PS: I know that putting all the TestCases in one folder is way better. But first i want to figure out why this is not working. Thanks!

How to create a pattern to filter out python test files

I have a bash script to execute my python tests and I would like to filter all test cases that have NOT_DONE in them
This is what I tried
python3 -m unittest discover -s ${FOLDER} -p 'test_((?!NOT_DONE).)*_ALL.py'
Input example :
test_word_NOT_DONE_more_words_alot_more_words_ALL.py <- This test shouldn't be executed
But this one should :
test_word_more_words_alot_more_words_ALL.py

Path Solution
Directory Structure
unittesting/
launcher.py
tests/
__init__.py
test_finished.py
test_NOT_DONE.py
folder/
__init__.py
test_finished2.py
test_NOT_DONE2.py
Inside each test file is print(__file__), nested under a TestCase method. Therefore, only if the module is imported and the test cases run, will it be executed.
Code:
import importlib
import os
import sys
import unittest
HOME = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, 'tests')
def check_file(file):
'''Check if file is test'''
return all((
'NOT_DONE' not in file,
file.endswith('.py'),
file != '__init__.py'
))
def find_paths(home=HOME):
'''Find all paths'''
os.chdir(HOME)
for root, dirs, files in os.walk('tests'):
for file in files:
if check_file(file):
if root != 'tests':
yield os.path.join(root[len('tests/'):], file)
else:
yield file
def normalize(path):
'''Normalize path to dotted name'''
path = os.path.splitext(path)[0]
unix = path.replace('/', '.')
return unix.replace('\\', '.')
def tests(paths=None):
'''Load and run tests'''
if paths is None:
paths = map(normalize, find_paths())
modules = (importlib.import_module(i) for i in paths)
suite = unittest.TestSuite()
loader = unittest.TestLoader()
for module in modules:
tests = loader.loadTestsFromModule(module)
suite.addTests(tests)
runner = unittest.TextTestRunner()
runner.run(suite)
if __name__ == '__main__':
tests()
As you can see, this gets unwieldly quickly, and very hard to manage. There's a simpler way. It runs, however.
$ python /home/alex/git/unittesting/launcher.py
tests/test_finished.pyc
.tests/folder/test_finished2.pyc
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Pythonic Solution
Inside each of my files that is not complete, I put the variable NOT_DONE = True, and each of my classes has a decorator skipif.
Directory Structure
unittesting/
launcher.py
tests/
__init__.py
test1.py
test2.py
folder/
__init__.py
test3.py
test4.py
In this example, test2 and test4 have NOT_DONE = True, while test1 and test3 have NOT_DONE = False.
An example file is as follows:
import unittest
NOT_DONE = False
# CASES
# -----
#unittest.skipIf(NOT_DONE, 'Reason')
class TestPrint(unittest.TestCase):
def test_print(self):
print(__file__)
if __name__ == '__main__':
unittest.main()
Now, to run I simply do:
$ python -m unittest discover tests
tests/test1.py
tests/folder/test3.py
.
----------------------------------------------------------------------
Ran 4 tests in 0.000s
OK (skipped=2)
Best Approach
Unfinished unittests should have a unittest.skipIf(True, 'Unfinished') line, so you get control not only at the module level, but also at the class or even method level. In the following example, I have one, finished unittest and one unfinished unittest. Running the example skips the first unittest, but runs the rest of the module.
import unittest
# CASES
# -----
#unittest.skipIf(True, 'Not Finished')
class TestPrint(unittest.TestCase):
def test_print(self):
print(__file__)
class TestPrinting(unittest.TestCase):
def test_print(self):
print(__file__)
if __name__ == '__main__':
unittest.main()