I am writing test cases for a module module.py that imports from another module legacy.py. The legacy.py reads os.environ["some_var"] at module level. When I am trying to run test cases for module.py, they are failing with KeyError for some_var in os.environ.
This is how code looks like:
module.py
from legacy import db
def fun():
pass
legacy.py
import os
env = os.environ["some_var"]
class db:
def __init__(self):
pass
When running test cases for module.py, I am getting KeyError: 'some_var'.
I tried patching os (at module level also putting it before importing from module.py in test file) but the import statement is run before it could be patched. I tried looking for similar question on StackOverflow but didn't find the exact problem I am facing here.
How can I mock it? Or any other suggestion to be able to run the test cases. Assume that I cannot modify the legacy.py file.
You could use the mock.patch.dict. There is an example in official doc.
Here is a fully functional example based in your question.
module.py
import os
def get_env_var_value():
env_var = os.environ['PATH']
return env_var
print(get_env_var_value())
test_module.py
from unittest.mock import patch
from unittest import main, TestCase
import module
class MyClassTestCase(TestCase):
#patch.dict('os.environ', {'PATH': '/usr/sbin'})
def test_get_env_var_value(self):
self.assertEqual(module.get_env_var_value(), '/usr/sbin')
if __name__ == '__main__':
main()
Doesn't matter if the environment var is loaded in outside or inside the class/method scope, because you will mock for all test scope.
Related
This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed 3 months ago.
I have a directory and module design that looks like this:
MyProject --- - script.py
|
- helpers --- - __init__.py
|
- class_container.py
|
- helper.py
# class_container.py
class MyClass:
pass
# helper.py
from class_container import MyClass
def func():
# some code using MyClass
# script.py
from helpers.helper import func
When I run script.py:
ModuleNotFoundError: No module named 'class_container'
I tried changing the code in helper.py such as:
# helper.py
from helpers.class_container import MyClass
def func():
# some code using MyClass
Then running script.py started working. But when I explicitly run helper.py:
ModuleNotFoundError: No module named 'helpers'
I want to be able run both script.py and helper.py separately without needing to change the code in any module.
Edit: I figured a solution which is changing helper.py such as:
from pathlib import Path
import sys
sys.path.append(str(Path(__file__).parent))
from class_container import MyClass
def func():
# some code using MyClass
Basically I added the directory of helper.py to sys.path by using sys and pathlib modules and __file__ object. And unlike import statement's behaviour, __file__ will not forget it's roots when imported/used from a different module (i.e it won't become the path of script.py when imported into it. It'll always be the path of helper.py since it was initiated there.).
Though I'd appreciate if someone can show another way that doesn't involve messing with sys.path, it feels like an illegal, 'unpythonic' tool.
You have to create a __init__.py file in the MyProject/helpers directory. Maybe you already have created it. If not, create an empty file.
Then in the MyProject/helpers/helper.py, access the module helpers.class_container like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
You can also use a relative import like this.
from .class_container import MyClass
If you want to run the MyProject/helpers/helper.py independently, add test code in helper.py like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
if __name__ == '__main__':
func()
And run like this in the MyProject directory.(I assume a Linux environment.)
$ python3 -m helpers.helper
The point is to differentiate Python modules from Python scripts and treat them differently.
When I move the import of implementation module within the method in test module, the test works fine.However when I have the import on top , I get an error stating that environment variable is not found.
why is the environment variable not set when I place the import on top of the file and how I can fix it without moving the import inside a function
Error Message
test/test_engine.py:4: in <module>
from reptar_validation_engine import get_client_id
source/engine.py:30: in <module>
ATHENA_DB = os.environ['env']
venv/lib/python3.6/os.py:669: in __getitem__
raise KeyError(key) from None
E KeyError: 'env'
conftest.py
import pytest
#pytest.fixture(autouse=True)
def env_setup(monkeypatch):
monkeypatch.setenv('env', 'dev')
Test Module - This Fails
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../', 'source')))
from engine import get_client_id
def test_get_client_id():
get_client_id()
Test Module - This Works
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../', 'source')))
def test_get_client_id():
from engine import get_client_id
get_client_id()
engine.py
import os
env_val = os.environ['env']
def get_client_id:
pass
The only place you appear to be setting the 'env' environment variable is in the env_setup fixture. Like all fixtures, the code within the fixture only applies while a test is being run. When you try to import engine at the top level of your test module, no test is currently in effect, and so (unless you've set 'env' somewhere else) os.environ['env'] will be unset at that point. Importing engine from within a test function works because, before the test function is run, the fixture gives the environment variable a value.
I don't know what you're trying to accomplish by assigning os.environ['env'] to a top-level module variable, but you're probably going about it the wrong way. In particular, if you set the 'env' envvar beforehand so that module-level import works, then env_val will not be affected by the monkeypatching.
I developed a crawler and it's unit-tests (mainly to validate XPATHs). I want to run specific unit-tests before script execution in order to be sure that HTML structure has not changed and existing XPATHs still working. I don't want the output of unit-test, just a flag: passed or failed.
for example:
tests.py:
import unittest
class CrwTst(unittest.TestCase):
def test_1(self):
[..]
crawler.py
class Crawler(object):
def action_1(self):
[..]
and I want to work like:
if CrwTst.test_1() is True:
Crawler.action_1()
You could potentially do this:
crawler.py
import unittest
from tests import CrwTst
if unittest.TextTestRunner().run(CrwTst('test_1')).wasSuccessful():
Crawler.action_1()
Note however that you may run into an issue with circular imports, because your test presumably already depends on Crawler, and what you are looking to do will make the Crawler depend on the test. This will likely manifest itself as ImportError: cannot import name CrwTst.
To resolve that, you can dynamically import the CrwTst.
crawler.py
import unittest
def function_that_runs_crawler():
from tests import CrwTst # Dynamically import to resolve circular ref
if unittest.TextTestRunner().run(CrwTst('test_1')).wasSuccessful():
Crawler.action_1()
I would like to import a function foo() from module abc.py
However, abc.py contains other functions which rely on modules which are not available for Python (i.e. I cannot import them into python interpreter, because I use ImageJ to run abc.py as Jython)
One solution I found is to put the problematic imports inside the name == "main" check, such as:
# abc.py
def foo():
print("Hello, World!")
def run_main_function():
foo()
...other stuff using IJ...
if __name__ == "__main__":
from ij import IJ
run_main_function()
So when I try to import foo from into another script def.py, e.g.:
# def.py
from abc import foo
def other_func():
foo()
if __name__ == "__main__":
other_func()
This works. But when I put imports in normal fashion, at the top of the script, I get an error: No module named 'ij'. I would like to know if there is a solution to this problem? Specifically, that I put the imports at the top of the script and then within def.py I say to import just the function, without dependencies of abc.py?
I would like to know if there is a solution to this problem? Specifically, that I put the imports at the top of the script and then within def.py I say to import just the function, without dependencies of abc.py?
As far I know, it's the way that python works. You should put that import in the function that uses it if won't be aviable always.
def run_main_function():
from ij import IJ
foo()
Also, don't use abc as a module name, it's a standard library module: Abstract Base Class 2.7, Abstract Base Class 3.6
Edit: don't use trailing .py when importing as Kind Stranger stated.
I typically write both unittests and doctests in my modules. I'd like to automatically run all of my doctests when running the test suite. I think this is possible, but I'm having a hard time with the syntax.
I have the test suite
import unittest
class ts(unittest.TestCase):
def test_null(self): self.assertTrue(True)
if __name__ == '__main__': unittest.main()
I'd like to add to this suite all of the doctests in module module1. How can I do this? I've read the python docs, but I'm not any closer to success, here. Adding the lines
import doctest
import module1
suite = doctest.DocTestSuite(module1)
doesn't work. unittest.main() searches through the current file scope and runs every test case it finds, right? But DocTestSuite produces a test suite. How do I get unittest.main() to run the additional cases in the suite? Or am I just confused and deluded??
Once again, I'd be grateful for any help anyone can offer.
An update to this old question: since Python version 2.7 there is the load_tests protocol and there is no longer a need to write custom code. It allows you to add a function load_tests(), which a test loader will execute to update its collection of unit tests for the current module.
Put a function like this in your code module to package the module's own doctests into a test suite for unittest:
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite())
return tests
Or, put a function like this into your unit test module to add the doctests from another module (for example, package.code_module) into the tests suite which is already there:
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(package.code_module))
return tests
When unittest.TestLoader methods loadTestsFromModule(), loadTestsFromName() or discover() are used unittest uses a test suite including both unit tests and doctests.
In this code i combined unittests and doctests from imported module
import unittest
class ts(unittest.TestCase):
def test_null(self):
self.assertTrue(True)
class ts1(unittest.TestCase):
def test_null(self):
self.assertTrue(True)
testSuite = unittest.TestSuite()
testSuite.addTests(unittest.makeSuite(ts))
testSuite.addTest(unittest.makeSuite(ts1))
import doctest
import my_module_with_doctests
testSuite.addTest(doctest.DocTestSuite(my_module_with_doctests))
unittest.TextTestRunner(verbosity = 2).run(testSuite)
I would recommend to use pytest --doctest-modules without any load_test protocol. You can simply add both the files or directories with your normal pytests and your modules with doctests to that pytest call.
pytest --doctest-modules path/to/pytest/unittests path/to/modules
It discovers and runs all doctests as well.
See https://docs.pytest.org/en/latest/doctest.html
This code will automatically run the doctests for all the modules in a package without needing to manually add a test suite for each module. This can be used with Tox.
import doctest
import glob
import os
import sys
if sys.version_info < (2,7,):
import unittest2 as unittest
else:
import unittest
import mypackage as source_package
def load_module_by_path(path):
"""Load a python module from its path.
Parameters
----------
path : str
Path to the module source file.
Returns
-------
mod : module
Loaded module.
"""
import imp
module_file_basename = os.path.basename(path)
module_name, ext = os.path.splitext(module_file_basename)
mod = imp.load_source(module_name, path)
return mod
def file_contains_doctests(path):
"""Scan a python source file to determine if it contains any doctest examples.
Parameters
----------
path : str
Path to the module source file.
Returns
-------
flag : bool
True if the module source code contains doctest examples.
"""
with open(path) as f:
for line in f:
if ">>>" in line:
return True
return False
def load_tests(loader, tests, pattern):
"""Run doctests for all modules"""
source_dir = os.path.dirname(source_package.__path__[0])
python_source_glob = os.path.join(source_dir, source_package.__name__, "*.py")
python_source_files = glob.glob(python_source_glob)
for python_source_file in python_source_files:
if not file_contains_doctests(python_source_file):
continue
module = load_module_by_path(python_source_file)
tests.addTests(doctest.DocTestSuite(module))
return tests
First I tried accepted answer from Andrey, but at least when running in Python 3.10 and python -m unittest discover it has led to running the test from unittest twice. Then I tried to simplify it and use load_tests and to my surprise it worked very well:
So just write both load_tests and normal unittest tests in a single file and it works!
import doctest
import unittest
import my_module_with_doctests
class ts(unittest.TestCase):
def test_null(self):
self.assertTrue(False)
# No need in any other extra code here
# Load doctests as unittest, see https://docs.python.org/3/library/doctest.html#unittest-api
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(my_module_with_doctests))
return tests
The zope.testing module provide such a functionality.
See
http://www.veit-schiele.de/dienstleistungen/schulungen/testen/doctests
for examples.