I have a module set up roughly as follows:
# foo.py
def generate_things_based_on_other_things():
# some nasty things here
# bar.py
from foo import generate_things_based_on_other_things as generate
def coo():
generate()
# conftest.py
import pytest
#pytest.fixture(autouse=True)
def patch_generate(monkeypatch):
def mock_generate():
print("hello!")
monkeypatch.setattr("app.bar.generate", mock_generate)
# test_bar.py
from bar import coo
def test_coo():
coo()
As per this answer I made sure to monkeypatch the actual imported instance of the function. Any other path throws a "does not exist on module" error.
However when I run the test I hit an error, because the original function generate is being called, despite it being monkeypatched.
I can't figure out why this patch won't stick the way I expect it too.
I would expect this test to print "hello!".
Your paths seem not to match. You do from bar import coo, but use setattr with app.bar. To be sure, you can use the other form of setattr instead, which takes the object and the attribute names separately, e.g.:
import bar # or "from app import bar", whichever is correct for you
#pytest.fixture(autouse=True)
def patch_generate(monkeypatch):
def mock_generate():
print("hello!")
monkeypatch.setattr(bar, "generate", mock_generate)
This way you can be reasonably sure that you are patching the correct object.
Related
what is the easiest way to solve the following problem in extending/altering the functionality of a third party library?
The library offers a class LibraryClass with a function func_to_be_changed. This function has a local variable internal_variable which is the instance of another class SimpleCalculation of that library. I created a new class BetterCalculation in my own module and now want LibraryClass.func_to_be_changed to use an instance of this new class.
# third party library
from third_party_library.utils import SimpleCalculation
class LibraryClass:
def func_to_be_changed(self, x):
# many complicated things go on
internal_variable = SimpleCalculation(x)
# many more complicated things go on
The easiest solution would be, to just copy the code from the third party library, subclass the LibraryClass and overwrite the function func_to_be_changed:
# my module
from third_party_library import LibraryClass
class BetterLibraryClass(LibraryClass):
def func_to_be_changed(self, x):
"""This is an exact copy of LibraryClass.func_to_be_changed."""
# many complicated things go on
internal_variable = BetterCalculation(x) # Attention: this line has been changed!!!
# many more complicated things go on
However, this involves copying of many lines of code. When a new version of the third party class improves on code that was copied without modification, this modifications need to be incorporated manually by another copying step.
I tried to use unittest.mock.patch as I know that the following two snippets work:
# some script
from unittest.mock import patch
import third_party_library
from my_module import BetterCalculation
with patch('third_party_library.utils.SimpleCalculation', BetterCalculation):
local_ = third_party_library.utils.SimpleCalculation(x) # indeed uses `BetterCalculation`
def foo(x):
return third_party_library.utils.SimpleCalculation(x)
with patch('third_party_library.utils.SimpleCalculation', BetterCalculation):
local_ = foo(x) # indeed uses `BetterCalculation`
However, the following does not work:
# some script
from unittest.mock import patch
from third_party_library.utils import SimpleCalculation
from my_module import BetterCalculation
def foo(x):
return SimpleCalculation(x)
with patch('third_party_library.utils.SimpleCalculation', BetterCalculation):
local_ = foo(x) # does not use `BetterCalculation`
# this works again
with patch('__main__.SimpleCalculation', BetterCalculation):
local_ = foo(x) # indeed uses `BetterCalculation`
Therefore, the following won`t work either:
# my module
from unittest.mock import patch
from third_party_library import LibraryClass
from my_module import BetterCalculation
class BetterLibraryClass(LibraryClass):
def func_to_be_changed(self, x):
with patch(
'third_party_library.utils.SimpleCalculation',
BetterCalculation
):
super().func_to_be_changed(x)
Is there an elegant pythonic way to do this? I guess this boils down to the question: What is the equivaltent of __main__ in the last code snippet that needs to replace third_party_library.utils?
Some context
The first string argument in the patch function can have two different meanings depending on the situation. In the first situation the described object has not been imported and is unavailable to the program which would, therefore, result in a NameError without the mocking. However, in the question, an object needs to be overwritten. Therefore, it is available to the program and not using patch would not result in an error.
Disclaimer
I might have used the complete wrong language in here, as for sure there are precise python terms for all the described notions.
Overwriting an object
As shown in the question, the locally imported SimpleCalculation can be overwritten with __main__.SimpleCalculation. Therefore, it is important to remember that you need to tell patch the path to the local object and not how that same object would be imported in the current script.
Let's assume the following module:
# thirdpartymodule/__init__.py
from .utils import foo
def local_foo():
print("Hello local!")
class Bar:
def __init__(self):
foo()
local_foo()
and
# thirdpartymodule/utils.py
def foo():
print("third party module")
We want to override the functions foo and local_foo. But we don't want to override any functions, we want to override the functions foo and local_foo in the context of the file thirdpartymodule/__init__.py. It is unimportant that the function foo enters the context of the file via an import statement, while local_foo is defined locally. So we want to override the functions in the context of thirdpartymodule.foo and thirdpartymodule.local_foo. The context thirdpartymodule.utils.foo is not important here and won't help us. The following snippet illustrates that:
from unittest.mock import patch
from thirdpartymodule import Bar
bar = Bar()
# third party module
# Hello local!
def myfoo():
print("patched function")
with patch("thirdpartymodule.foo", myfoo):
bar = Bar()
# patched function
# Hello local!
# will not work!
with patch("thirdpartymodule.utils.foo", myfoo):
bar = Bar()
# third party module
# Hello local!
with patch("thirdpartymodule.local_foo", myfoo):
bar = Bar()
# third party module
# patched function
In the hypothetical module of the question we first need to assume that the class LibraryClass is defined in the file third_party_library/library_class.py. Then, we want to override third_party_library.library_class.SimpleCalculation and the correct patch would be:
# my module
from unittest.mock import patch
from third_party_library import LibraryClass
from my_module import BetterCalculation
class BetterLibraryClass(LibraryClass):
def func_to_be_changed(self, x):
with patch(
'third_party_library.library_class.SimpleCalculation',
BetterCalculation
):
super().func_to_be_changed(x)
I am trying to implement unittests for my python program. The problem is, that my program is using several imported classes, which I would like to replace by a mocked object/class to verify single functions/methods.
I do not get any errors with my mocked class. But it appears that the mock itself didn't replace the object I wanted to replace.
This is basically my structure:
First the class I want to mock. Might look like that:
class ToMock():
def getSomething(self):
return "something"
The class I want to test looks like this:
from x.y import ToMock
class ClassToTest():
def __init__(self):
self.obj = ToMock()
def returnStuff():
return self.obj.getSomething()
As you can imagine, I want to test the returnStuff method. Therfore I want to mock .getSomething, or better said the whole ToMock object.
The unittest should therefore test the ClassToTest with the mocked ToMock class. I tried several mock.patch variants, but I couldn't get it to run/test properly.
import unittest
from unittest import mock
from a.b import ClassToTest
class TestObject(unittest.TestCase):
def setUp(self):
with mock.patch('x.y.ToMock') as mock_obj:
mock_obj.return_value.getSomething().return_value="mocked return value"
self.test_class = ClassToTest()
result = self.test_class.returnStuff() # This should return now 'mocked return value', I guess?
mock_obj.return_value.getSomething.assert_called_once_with("")
The problem I face is, that the self.test_class.returnStuff() is not "calling" the mocked object, but imports the real class etc. and therefore I am running into timeouts, or similar stuff.
I am sure, that I provide the wrong path for the object which should be mocked. Perhaps someone can hint me into the right direction.
Thanks
-GreNait
The issue is that you are not patching in the correct place. You are patching where the object is defined as opposed to where it is looked up.
a.py
-> Defines ToMock
b.py
-> from a import ToMock
-> some_function/class instantiates ToMock
In your code shown you are patching a.ToMock however you should be patching b.ToMock. That is why it is not running your mock object when testing. You can read more about where to patch here.
I have a function that has a decorator. The decorator accepts arguments and the value of the argument is derived from another function call.
example.py
from cachetools import cached
from cachetools import TTLCache
from other import get_value
#cached(cache=TTLCache(maxsize=1, ttl=get_value('cache_ttl')))
def my_func():
return 'result'
other.py
def get_value(key):
data = {
'cache_ttl': 10,
}
# Let's assume here we launch a shuttle to the space too.
return data[key]
I'd like to mock the call to get_value(). I'm using the following in my test:
example_test.py
import mock
import pytest
from example import my_func
#pytest.fixture
def mock_get_value():
with mock.patch(
"example.get_value",
autospec=True,
) as _mock:
yield _mock
def test_my_func(mock_get_value):
assert my_func() == 'result'
Here I'm injecting mock_get_value to test_my_func. However, since my decorator is called on the first import, get_value() gets called immediately. Any idea if there's a way to mock the call to get_value() before module is imported right away using pytest?
Move the from example import my_func inside your with in your test function. Also patch it where it's really coming from, other.get_value. That may be all it takes.
Python caches modules in sys.modules, so module-level code (like function definitions) only runs on the first import from anywhere. If this isn't the first time, you can force a re-import using either importlib.reload() or by deleting the appropriate key in sys.modules and importing again.
Beware that re-importing a module may have side effects, and you may also want to re-import the module again after running the test to avoid interfering with other tests. If another module was using objects defined in the re-imported module, these don't just disappear, and may not be updated the way it expects. For example, re-importing a module may create a second instance of what was supposed to be a singleton.
One more robust approach would be save the original imported module object somewhere else, delete from sys.modules, re-import with the patched version for the duration of the test, and then put back the original import into sys.modules after the test. You could do this with an import inside of a patch.dict() context on sys.modules.
import mock
import sys
import pytest
#pytest.fixture
def mock_get_value():
with mock.patch(
"other.get_value",
autospec=True,
) as _mock, mock.patch.dict("sys.modules"):
sys.modules.pop("example", None)
yield _mock
def test_my_func(mock_get_value):
from example import my_func
assert my_func() == 'result'
Another possibility is to call the decorator yourself in the test, on the original function. If the decorator used functools.wraps()/functools.update_wrapper(), then original function should be available as a __wrapped__ attribute. This may not be available depending on how the decorator was implemented.
I'm noticing that the class methods are being mocked out properly, but the function is bypassing the mock and running the actual function.
from module1 import myClass
from module2 import my_function
import unittest
from mock import patch
class TestStuff(unittest.TestCase):
#patch('module2.my_function')
#patch('module1.myClass.my_method')
def test_my_function(self, mock_method, mock_function):
test_obj = myClass()
test_obj.my_method()
assert mock_method.called
my_function()
assert mock_function.called
if I print out my_function() and type(my_function) it will not show the mock, but the real function. Does it matter that I'm importing a class then mocking a method, but I'm importing the function directly?
I think I found the issue:
When I'm testing a function, and there's something I want to mock, I should not import it. Importing it causes it to get used, even if I've put the patch decorator.
I think this was general confusion about how mocking/testing works. What I wanted to do was to test a function without actually querying the database. Instead of mocking out the whole function, I could take the line that actually hits the db - query.all() - and make it it's own function, then mock that out.
from module1 import myClass
from module2 import my_function
import unittest
from mock import patch
class TestStuff(unittest.TestCase):
#patch('module2.db_query')
#patch('module1.myClass.my_method')
def test_my_function(self, mock_method, mock_db_query):
test_obj = myClass()
test_obj.my_method()
assert mock_method.called
my_function() # my_function calls db_query(), now mocked out
assert mock_db_query.called
If I had wanted to actually mock out the all of my_function, I could have just not imported it. At least that's how I'm understanding this at the moment.
I ran into a similar issue. Turns out I needed to give it the FULL path to my_function:
#patch('home.myuser.myprojects.mymodule.myfunc')
I want to fake a package in python. I want to define something so that the code can do
from somefakepackage.morefakestuff import somethingfake
And somefakepackage is defined in code and so is everything below it. Is that possible? The reason for doing this is to trick my unittest that I got a package ( or as I said in the title, a module ) in the python path which actually is just something mocked up for this unittest.
Sure. Define a class, put the stuff you need inside that, assign the class to sys.modules["classname"].
class fakemodule(object):
#staticmethod
def method(a, b):
return a+b
import sys
sys.modules["package.module"] = fakemodule
You could also use a separate module (call it fakemodule.py):
import fakemodule, sys
sys.modules["package.module"] = fakemodule
Yes, you can make a fake module:
from types import ModuleType
m = ModuleType("fake_module")
import sys
sys.modules[m.__name__] = m
# some scripts may expect a file
# even though this file doesn't exist,
# it may be used by Python for in error messages or introspection.
m.__file__ = m.__name__ + ".py"
# Add a function
def my_function():
return 10
m.my_function = my_function
Note, in this example its using an actual module (of ModuleType) since some
Python code may expect modules, (instead of a dummy class).
This can be made into a utility function:
def new_module(name, doc=None):
import sys
from types import ModuleType
m = ModuleType(name, doc)
m.__file__ = name + '.py'
sys.modules[name] = m
return m
print(new_module("fake_module", doc="doc string"))
Now other scripts can run:
import fake_module
I took some of the ideas from the other answers and turned them into a Python decorator #modulize which converts a function into a module. This module can then be imported as usual. Here is an example.
#modulize('my_module')
def my_dummy_function(__name__): # the function takes one parameter __name__
# put module code here
def my_function(s):
print(s, 'bar')
# the function must return locals()
return locals()
# import the module as usual
from my_module import my_function
my_function('foo') # foo bar
The code for the decorator is as follows
import sys
from types import ModuleType
class MockModule(ModuleType):
def __init__(self, module_name, module_doc=None):
ModuleType.__init__(self, module_name, module_doc)
if '.' in module_name:
package, module = module_name.rsplit('.', 1)
get_mock_module(package).__path__ = []
setattr(get_mock_module(package), module, self)
def _initialize_(self, module_code):
self.__dict__.update(module_code(self.__name__))
self.__doc__ = module_code.__doc__
def get_mock_module(module_name):
if module_name not in sys.modules:
sys.modules[module_name] = MockModule(module_name)
return sys.modules[module_name]
def modulize(module_name, dependencies=[]):
for d in dependencies: get_mock_module(d)
return get_mock_module(module_name)._initialize_
The project can be found here on GitHub. In particular, I created this for programming contests which only allow the contestant to submit a single .py file. This allows one to develop a project with multiple .py files and then combine them into one .py file at the end.
You could fake it with a class which behaves like somethingfake:
try:
from somefakepackage.morefakestuff import somethingfake
except ImportError:
class somethingfake(object):
# define what you'd expect of somethingfake, e.g.:
#staticmethod
def somefunc():
...
somefield = ...
TL;DR
Patch sys.modules using unittest.mock:
mock.patch.dict(
sys.modules,
{'somefakepackage': mock.Mock()},
)
Explanation
Other answers correctly recommend to fix sys.modules but a proper way to do it is by patching it using mock.patch. Meaning replacing it temporarily (only for when tests are run) with a fake object that optionally imitates the desired behaviour. And restoring it back once tests are finished to not affect other test cases.
The code in TL;DR section will simply make your missing package not raise ImportError. To provide fake package with contents and imitate desired behaviour, initiate mock.Mock(…) with proper arguments (e.g. add attributes via Mock's **kwargs).
Full code example
The code below temporarily patches sys.modules so that it includes somefakepackage and makes it importable from the dependent modules without ImportError.
import sys
import unittest
from unittest import mock
class SomeTestCase(unittest.TestCase):
def test_smth(self):
# implement your testing logic, for example:
self.assertEqual(
123,
somefakepackage_dependent.some_func(),
)
#classmethod
def setUpClass(cls): # called once before all the tests
# define what to patch sys.modules with
cls._modules_patcher = mock.patch.dict(
sys.modules,
{'somefakepackage': mock.Mock()},
)
# actually patch it
cls._modules_patcher.start()
# make the package globally visible and import it,
# just like if you have imported it in a usual way
# placing import statement at the top of the file,
# but relying on a patched dependency
global somefakepackage_dependent
import somefakepackage_dependent
#classmethod # called once after all tests
def tearDownClass(cls):
# restore initial sys.modules state back
cls._modules_patcher.stop()
To read more about setUpClass/tearDownClass methods, see unittest docs.
unittest's built-in mock subpackage is actually a very powerful tool. Better dive deeper into its documentation to get a better understanding.