I have a module written roughyl as follows:
class __Foo:
def __init__(self):
self.__client = CeateClient() # a library function
async def call_database(self):
self.__client.do_thing()
foo = __Foo()
This is designed so that the entire server only has one instance of this client manage, so foo is created once and is then used by various modules.
I am required to now use pytest to, well, test everything, but reading online I cannot find the correct way to path this situation.
The class method call_database sends an API request to a different service, which as part of the test, I want to not actually happen.
The suggestions I see talk about patching the class method itself, but also the class is private so I couldn't really do that.
The point is, when testing a function in a different module, i.e.:
from foo import foo
async def bar():
await foo.call_database()
When testing the function bar, I want the call to the database to not happen.
How would I go about doing that?
Related
I am trying to implement unittests for my python program. The problem is, that my program is using several imported classes, which I would like to replace by a mocked object/class to verify single functions/methods.
I do not get any errors with my mocked class. But it appears that the mock itself didn't replace the object I wanted to replace.
This is basically my structure:
First the class I want to mock. Might look like that:
class ToMock():
def getSomething(self):
return "something"
The class I want to test looks like this:
from x.y import ToMock
class ClassToTest():
def __init__(self):
self.obj = ToMock()
def returnStuff():
return self.obj.getSomething()
As you can imagine, I want to test the returnStuff method. Therfore I want to mock .getSomething, or better said the whole ToMock object.
The unittest should therefore test the ClassToTest with the mocked ToMock class. I tried several mock.patch variants, but I couldn't get it to run/test properly.
import unittest
from unittest import mock
from a.b import ClassToTest
class TestObject(unittest.TestCase):
def setUp(self):
with mock.patch('x.y.ToMock') as mock_obj:
mock_obj.return_value.getSomething().return_value="mocked return value"
self.test_class = ClassToTest()
result = self.test_class.returnStuff() # This should return now 'mocked return value', I guess?
mock_obj.return_value.getSomething.assert_called_once_with("")
The problem I face is, that the self.test_class.returnStuff() is not "calling" the mocked object, but imports the real class etc. and therefore I am running into timeouts, or similar stuff.
I am sure, that I provide the wrong path for the object which should be mocked. Perhaps someone can hint me into the right direction.
Thanks
-GreNait
The issue is that you are not patching in the correct place. You are patching where the object is defined as opposed to where it is looked up.
a.py
-> Defines ToMock
b.py
-> from a import ToMock
-> some_function/class instantiates ToMock
In your code shown you are patching a.ToMock however you should be patching b.ToMock. That is why it is not running your mock object when testing. You can read more about where to patch here.
in the case of unit testing a wrapper library, testing the wrapper without depending/exercising the upstream library is a goal; In a known case, all calls to the upstream library can be mocked and that's what I've done, but I've been frustrated by changes to the wrapper that introduce more calls to the upstream library being missed by the mock tools;
How can I best fail any test that tries to use a given namespace?
My idea currently is to change all the unittest methods to have a monkey patch like
#unittest.mock.patch('wrapper_namespace.upsteam_namespace')
and reply the upstream library with a mock that can be asserted untouched; I'm hoping for an option that works globally, so that I
don't have to add a monkeypatch to every test method, though this level of granularity is acceptable; but also don't have to perform the assertion that the mock was never used in the test methods (or make a decorator to do all that either)
prohibits access to the upstream library from any part of the software
(e.g, Wrapper calls B calls Upstream, B's call to upstream might not be caught)
You don't have to patch every test method. You can easily patch over the class if you're using unittest, or just assign the module to whatever you want to patch over it with. Here's a workable example:
A fake lib in some_lib.py:
def some_lib_func():
raise ValueError("I've been called.")
def some_other_lib_func():
raise ValueError("I've been called.")
class SomeClass:
def __init__(self):
raise ValueError("I've been constructed.")
wrapper.py:
import some_lib
def wrapper1():
some_lib.some_lib_func()
def wrapper2():
some_lib.some_other_lib_func()
def wrapper3():
x = some_lib.SomeClass()
test.py:
from unittest.mock import patch, MagicMock
import unittest
import wrapper
# Alternative:
# wrapper.some_lib = MagicMock()
# Can patch an entire class
#patch('wrapper.some_lib', MagicMock())
class TestWrapper(unittest.TestCase):
def test_wrapper1(self):
wrapper.wrapper1()
def test_wrapper2(self):
wrapper.wrapper2()
def test_wrapper3(self):
wrapper.wrapper3()
if __name__ == "__main__":
unittest.main()
We would explode if the functions/classes in some_lib were called, but they aren't:
Matthews-MacBook-Pro:stackoverflow matt$ python test.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK
Feel free to comment out the patch and comment in wrapper.some_lib = MagicMock(). You'll get the same result in this toy example, but there is a major difference between the two approaches:
When using #patch('wrapper.some_lib', MagicMock()) the patch is only live for that Test Case class.
When using wrapper.some_lib = MagicMock(), however, that patch will stay live for the entire length of your python program, unless you save off the original module and patch it back manually at some point. Everything that is using the wrapper module will get the mocked version.
So you could so something like:
original_lib = wrapper.some_lib
wrapper.some_lib = MagicMock()
...
# call some test suite, every call to the wrapper module will be mocked out
...
wrapper.some_lib = original_lib
...
# call some other test suite that actually needs the real thing
...
HTH.
EDIT: Misread your question slightly, but you can inspect MagicMock objects to see if they've been called, and if so, fail the test. Or just patch over with something that fails when called (instead of MagicMock). I can provide code to do this if requested (just leave a comment), but hopefully the above can get you started. I think the crux of the question was really about the global patching. Cheers!
I have a number of modules that needs to have a database connection instance, and I would prefer they share the same instance and don't create their own. My current way of doing this is to explicitly send each function in all modules the object instance like such:
def func(arg1, arg2, database_connection):
pass
This becomes quite ugly and in a way redundant when there should be a better way to import a separate module containing the instance, but I'm not quite sure how to guarantee that it's actually one single instance, and not multiple instances.
That is, I'm looking for a way to do something like this:
import db_module
def func(arg1, arg2):
database_connection = db_module.get_db_instance()
The solution you've described:
import db_module
def func(arg1, arg2):
database_connection = db_module.get_db_instance()
is perfectly viable because Python imports each module exactly once. If multiple import statements are executed, they each refer to a single instance of the module.
You can read more about modules and importing in the Python Tutorial and the
Python Language Reference.
Is this what you mean? Create a module that imports the defined function?
db_module.py
my_connection = func(x,y)
from db_module import my_connection
The Mock documentation describes a simple and elegant way of applying patches to all of the tests method inside a TestCase:
#patch('foo.bar')
#patch('foo.baz')
#patch('foo.quux')
#patch('foo.narf')
class FooTest(TestCase):
def test_foo(self, bar, baz, quux, narf):
""" foo """
self.assertTrue(False)
However, one issue I've encountered with this method is that if I'd like to call stop() on one of the patches inside one of the test methods, there doesn't appear to be anyway of getting a reference to the patcher object -- the only thing that is passed into the method is the mock objects, in this case bar, baz, quux, narf.
The only way I've found to solve this problem is to move to the pattern described in the Mock docs where the patchers are instantiated and started inside the setUp method of the TestCase and stopped inside the tearDown method. This fits my purpose, but adds a lot of extra boilerplate and isn't as elegant as the class decorator approach.
Is there another way to solve this problem?
1
Say you want to temporarily restore foo.narf in a method. foo.narf is, in the context of the decorated function, a MagicMock object. This object has a _mock_wraps attribute which will be invoked when the mock is called! So at the top of your module, _narf = foo.narf, and in your test case, foo.narf._mock_wraps = _narf.
The catch is that this will only pass through to the real function, not actually swap it back, which means that some test cases will fail (e.g. if they rely on the function object actually being "itself"). And if your mock has other attributes, that could interfere (I haven't tested much) because the passthrough call to _mock_wraps() comes at the bottom of a method that first considers the other properties of the mock.
2
The patch() decorator involves each patcher (separate copies per method) being added to a list called patchings which is a field of the method itself. I.e. you can access this list as self.test_foo.patchings, and go through to find the one you want.
However, start() and stop() are not actually called when you use patch() as a decorator, and the behavior gets tricky once you start reaching in and changing it. So I wrote this context manager.
class unpatch:
def __init__(self, name, method):
compare = patch(name)
self.patcher = next((
p for p in method.patchings
if p.target == compare.getter()
and p.attribute == compare.attribute
), None)
if self.patcher is None:
raise ValueError(name)
def __enter__(self):
self.patcher.__exit__()
def __exit__(self, *exc_info):
self.patcher.__enter__()
Inside your test case, you use it like this:
with unpatch('foo.narf', self.test_foo):
foo.narf()
Disclaimer: this is hacks.
I've decorated a method in Python. And when I import the module that contains the method, the decorator autoruns.
I realize that this is how decorators were made however Is there a way to have decorators NOT do this?
It sounds like what you want to do is to choose what decorator to apply at run time. Something like this might work:
to_decorate = []
def decorate_later(func):
to_decorate.append(func)
return func
#decorate_later
def do_stuff(*args, **kw):
print('I am doing stuff')
#decorate_later
def do_more_stuff(*args, **kw):
print('Even more stuff')
def apply_decorator(decorator):
for func in to_decorate:
globals()[func.func_name] = decorator(func)
Then you can import the module and all the functions will be defined as normal. decorate_later returns the original function unmodified. You can call apply_decorator() to apply a specified decorator to all of the functions in the module that were registered by #decorate_later
This is exactly what the venusian library does; you define your decorators according to their API, but the actual behavior isn't triggered until you do a "scan" of the containing module or package.
You don't even need to have a global app object to use venusian decorators; you can pass in the app object as part of the scan, and it'll get passed along to the decorator implementations. So, for example, the same functions can be shared among multiple owners with only a single decorator, just by doing more than one scan.
This is what the Pyramid web framework uses for e.g. event registration, so that merely importing a module doesn't expect to need an app instance. A good example is their event subscriber.
Use
if __name__ == "__main__":
#code
in the file, where code is all outside a method or class ( that runs when you import it).