I am trying to understand the mock/patch framework, but have a problem. Here are my simplified codes:
file_a.py
class A:
def f(self):
b = B()
b.g()
b.h()
file_b.py
class B:
def g(self):
return network_requests(...)
def h(self):
return "This is should not be mocked."
file_test.py
class SomeTests:
def setUp(self):
with patch('file_b.B', autospec=True) as mock:
mock.g.return_value = "Mocked value"
mock.side_effect = lambda : B()
self.a = A()
def test(self):
self.a.f()
Essentially I want to mock only B.g inside the test, but not B.h. I got some idea from https://docs.python.org/3/library/unittest.mock-examples.html#partial-mocking, but B.g is still not mocked.
Thank you!
In the example that you linked the key problem is
Unfortunately datetime.date is written in C
That is why you need to mock the module and wrap what you don't want to mock (You cannot patch C methods directly).
Is all other cases (patch python objects) you can use just :
with patch('file_b.B.g', autospec=True) as mock_g:
mock_g.return_value = "Mocked value"
Anyway take care that your patch will be active just in the with context, out of it you will find the original reference. To have a better control of the context it you can use also decorators, start() and stop().
I strongly encourage you read carefully patch and where to patch.
Related
As the doc "Where to patch" says, we need to patch where an object is looked up, (not where it's defined); so I understand that it's not possible to - let's say - create a reusable patch for a particular path
Imagine you have several modules importing an object you'd like to mock
# file_a.py
from foo.goo.hoo import settings
# file_b.py
from foo.goo.hoo import settings
# file_c.py
from foo.goo.hoo import settings
I was wondering if there is a way to create a decorator such as:
#mock_settings
def test_whatever(self, settings_mock):
...
instead of this solution:
#patch("some_module.file_a.settings")
def test_whatever(self, settings_mock):
...
#patch("some_module.file_b.settings")
def test_whatever(self, settings_mock):
...
#patch("some_module.file_c.settings")
def test_whatever(self, settings_mock):
...
As mentioned in the question, to patch an object you have to patch its reference in the module to be tested (in case it is imported using from ...import).
To have it patched in several modules, you can patch all of these modules with the same mock, and use that mock. If you know in advance which modules you want to patch, you can just do this. If you don't know them in advance, you have to try to patch the object in all loaded modules -- this may get a bit more complicated.
I will show an example using pytest and a pytest fixture, as this is more compact; you could wrap that in a decorator for usage in unittest, but that will not change the basics. Consider we have a class that needs to be mocked in several modules:
class_to_mock.py
class ClassToMock:
def foo(self, msg):
return msg
module1.py
from class_to_mock import ClassToMock
def do_something():
inst = ClassToMock()
return inst.foo("module1")
module2.py
from class_to_mock import ClassToMock
def do_something_else():
inst = ClassToMock()
return inst.foo("module2")
You can now write a fixture that mocks the class in all of these modules at once (here using pytest-mock for simplicity):
#pytest.fixture
def mocked_class(mocker):
mocked = Mock()
for module in ('module1', 'module2'):
mocker.patch(module + '.ClassToMock', mocked)
yield mocked
This can be used to test both modules:
def test_module1(mocked_class):
mocked_class.return_value.foo.return_value = 'mocked!'
assert module1.do_something() == 'mocked!'
def test_module2(mocked_class):
mocked_class.return_value.foo.return_value = 'mocked!'
assert module2.do_something_else() == 'mocked!'
If you want a generic version that mocks the class in all loaded modules, you can replace the fixture with something like this:
#pytest.fixture
def mocked_class(mocker):
mocked = Mock()
for name, module in list(sys.modules.items()):
if not inspect.ismodule(module):
continue
for cls_name, cls in module.__dict__.items():
try: # need that as inspect may raise for some modules
if inspect.isclass(cls) and cls_name == "ClassToMock":
mocker.patch(name + ".ClassToMock", mocked)
except Exception:
continue
yield mocked
This will work for this specific example - to generalize this, it has to consider more object types, the class shall be configurable, and there may be some more issues - opposed to the more simple version where you just enumerate the modules you want to patch, which will always work.
You could do something similar in unittest.setUp by putting the mock in an instance variable, though that is less elegant, because you are also responsible for stopping the mocking:
class ModulesTest(unittest.TestCase):
def setUp(self):
self.mocked_class = Mock()
self.mocks = []
for module in ('module1', 'module2'):
mocked = mock.patch(module + '.ClassToMock', self.mocked_class)
self.mocks.append(mocked)
mocked.start()
def tearDown(self):
for mocked in self.mocks:
mocked.stop()
def test_module1(self):
self.mocked_class.return_value.foo.return_value = 'mocked!'
assert module1.do_something() == 'mocked!'
And you can also wrap this in a decorator, to answer your original question at least partially:
def mocked_class_to_mock(f):
#wraps(f)
def _mocked_class_to_mock(*args, **kwargs):
mocked_class = Mock()
mocks = []
for module in ('module1', 'module2'):
mocked = mock.patch(module + '.ClassToMock', mocked_class)
mocks.append(mocked)
mocked.start()
kwargs['mocked_class'] = mocked_class # use a keyword arg for simplicity
f(*args, **kwargs)
for mocked in mocks:
mocked.stop()
return _mocked_class_to_mock
...
#mocked_class_to_mock
def test_module3(self, mocked_class):
mocked_class.return_value.foo.return_value = 'mocked!'
assert module3.do_something() == 'mocked!'
Of course, you can do the same with the more generic version, if needed.
Also note that I skipped the simpler case where the object is imported using import .... In this case, you have to patch the original module. In the generic fixture, you probably want to add that case always.
I have a situation where I'm trying to modify the arguments passed to a decorator on one of my class methods. The code looks something like this:
class MyClass(object):
#tryagain(retries=3)
def mymethod(self, arg):
... do stuff ...
My problem is I'd like to alter the "retries" variable to something less than 3 when running my unit tests, but keep it at "3" for the production code. Unfortunately, it doesn't look like I can do something like this:
#tryagain(retries=self.retries)
def mymethod(self, arg):
... do stuff ...
or
#tryagain(retries=MyClass.retries)
def mymethod(self, arg):
... do stuff ...
because the class isn't defined at the point the arguments are passed to the decorator (as near as I can tell).
I also tried to add the variable within the module like so:
retries = 1
def MyClass(object):
#tryagain(retries=retries)
def mymethod(self, arg):
... do stuff ...
but then I can't seem to modify the value of "retries" from within my unit tests. Is there another way to accomplish what I'm trying to do?
I assume you try to reduce the number of retrials to increase test speed.
If so, modifying the number of retries variable doesn't seem to be the best approach. Instead, you could unit test the function mymethod without decorator first, and then create a mock function of mymethod. Let's call it mock_mymethod, decorate it with #tryagain and test if the logic of `tryagain actually works.
Check the mock module to see how to create a mock instance, this article about mock is also worth reading.
You could use an environment variable, set from your calling code (it might be good to put a default in here
import os
# ...
class MyClass(object):
#tryagain(retries=int(os.environ['project_num_retries']))
def mymethod(self, arg):
print("mymethod")
Or use a "globals"-type module, for example: project_settings.py containing:
num_retries = 3
Then
import project_settings
class MyClass(object):
#tryagain(retries=project_settings.num_retries)
def mymethod(self, arg):
print("mymethod")
But I'm not sure decorating your code with test information is how you really should go about it -- what about:
class MyClass(object):
def mymethod(self, arg):
print("mymethod")
Then in something like unittests.py:
DEV_TESTS = True # Change to False for production
num_retries = 3 if not DEV_TESTS else 1
import <your class>
class UnitTests():
def __init__(self):
self.c = <your_class>.MyClass()
#tryagain(retries=num_retries)
def test_mymethod(self):
self.c.mymethod("Foo")
t = UnitTests()
t.test_mymethod()
If you were so inclined, this unittests.py could be used with something like python's unittest package with:
DEV_TESTS = True # Change to False for production
num_retries = 3 if not DEV_TESTS else 1
import unittest
import <your class>
class UnitTests(unittest.TestCase):
def setUp(self):
self.c = <your class>.MyClass()
#tryagain(retries=num_retries)
def test_mymethod(self):
self.c.mymethod("Foo")
Note, I used the following simple example of a #tryagain decorator, yours may be more complicated and require some tuning of the examples:
def tryagain(retries):
def wrap(f):
def wrapped_f(*args,**kwargs):
for _ in xrange(retries):
f(*args,**kwargs)
return wrapped_f
return wrap
I have a base class that defines a class attribute and some child classes that depend on it, e.g.
class Base(object):
assignment = dict(a=1, b=2, c=3)
I want to unittest this class with different assignments, e.g. empty dictionary, single item, etc. This is extremely simplified of course, it's not a matter of refactoring my classes or tests
The (pytest) tests I have come up with, eventually, that work are
from .base import Base
def test_empty(self):
with mock.patch("base.Base.assignment") as a:
a.__get__ = mock.Mock(return_value={})
assert len(Base().assignment.values()) == 0
def test_single(self):
with mock.patch("base.Base.assignment") as a:
a.__get__ = mock.Mock(return_value={'a':1})
assert len(Base().assignment.values()) == 1
This feels rather complicated and hacky - I don't even fully understand why it works (I am familiar with descriptors though). Does mock automagically transform class attributes into descriptors?
A solution that would feel more logical does not work:
def test_single(self):
with mock.patch("base.Base") as a:
a.assignment = mock.PropertyMock(return_value={'a':1})
assert len(Base().assignment.values()) == 1
or just
def test_single(self):
with mock.patch("base.Base") as a:
a.assignment = {'a':1}
assert len(Base().assignment.values()) == 1
Other variants that I've tried don't work either (assignments remains unchanged in the test).
What's the proper way to mock a class attribute? Is there a better / more understandable way than the one above?
base.Base.assignment is simply replaced with a Mock object. You made it a descriptor by adding a __get__ method.
It's a little verbose and a little unnecessary; you could simply set base.Base.assignment directly:
def test_empty(self):
Base.assignment = {}
assert len(Base().assignment.values()) == 0
This isn't too safe when using test concurrency, of course.
To use a PropertyMock, I'd use:
with patch('base.Base.assignment', new_callable=PropertyMock) as a:
a.return_value = {'a': 1}
or even:
with patch('base.Base.assignment', new_callable=PropertyMock,
return_value={'a': 1}):
Perhaps I'm missing something, but isn't this possible without using PropertyMock?
with mock.patch.object(Base, 'assignment', {'bucket': 'head'}):
# do stuff
To improve readability you can use the #patch decorator:
from mock import patch
from unittest import TestCase
from base import Base
class MyTest(TestCase):
#patch('base.Base.assignment')
def test_empty(self, mock_assignment):
# The `mock_assignment` is a MagicMock instance,
# you can do whatever you want to it.
mock_assignment.__get__.return_value = {}
self.assertEqual(len(Base().assignment.values()), 0)
# ... and so on
You can find more details at http://www.voidspace.org.uk/python/mock/patch.html#mock.patch.
If your class (Queue for example) in already imported inside your test - and you want to patch MAX_RETRY attr - you can use #patch.object or simply better #patch.multiple
from mock import patch, PropertyMock, Mock
from somewhere import Queue
#patch.multiple(Queue, MAX_RETRY=1, some_class_method=Mock)
def test_something(self):
do_something()
#patch.object(Queue, 'MAX_RETRY', return_value=1, new_callable=PropertyMock)
def test_something(self, _mocked):
do_something()
Here is an example how to unit-test your Base class:
mocking multiple class attributes of different types (ie: dict and int)
using the #patch decorator and pytest framework with with python 2.7+ or 3+.
# -*- coding: utf-8 -*-
try: #python 3
from unittest.mock import patch, PropertyMock
except ImportError as e: #python 2
from mock import patch, PropertyMock
from base import Base
#patch('base.Base.assign_dict', new_callable=PropertyMock, return_value=dict(a=1, b=2, c=3))
#patch('base.Base.assign_int', new_callable=PropertyMock, return_value=9765)
def test_type(mock_dict, mock_int):
"""Test if mocked class attributes have correct types"""
assert isinstance(Base().assign_dict, dict)
assert isinstance(Base().assign_int , int)
I am doing some unit testing and at some point I need to mock a super call to throw an error, for example:
#classmethod
def myfunc(cls, *args, **kwargs)
try:
super(MyClass, cls).my_function(args, kwargs)
except MyException as e:
#...
I am using the mocker library to mock my objects in general but I haven't found a way to mock this.
Using unittest.mock from the standard library I would do something like this.
In your class definition:
from somelib import ASuperClass
class MyClass(ASuperClass):
def my_cool_method(self):
return super().my_cool_method()
In the module where you are calling MyClass:
from unittest.mock import patch
from mymodule import MyClass
#patch("mypackage.mymodule.ASuperClass.my_cool_method")
def call_with_mock(mocked_super):
myinstance = MyClass()
myinstance.my_cool_method()
# do stuff with `mocked_super`
call_with_mock()
I found a way, sort of hacky but it works, I'll explain with my example, this is based on this response so thanks #kindall:
def my_test(self):
import __builtin__
from mocker import Mocker, KWARGS, ARGS
mymocker = mocker.mock()
mymocker.my_function(ARGS, KWARGS)
mocker.throw(MyException)
def mysuper(*args, **kwargs):
if args and issubclass(MyClass, args[0]):
return mymocker
return original_super(*args, **kwargs)
__builtin__.original_super = super
__builtin__.super = mysuper
with mocker:
MyClass.myfunc()
so essentially what I do is check if the super call is from the class I want to mock, else just do a normal super.
Hope this helps someone :)
In case anyone needs another way to solve this mock:
# some_package/some_module.py
class MyClass(SuperClass):
def some_function(self):
result_super_call = super().function()
# test_file.py
#patch('some_package.some_module.super')
def test_something(self, mock_super):
obj = MyClass()
mock_super().some_function.return_value = None
Using Python 3.6
#Markus is looking in the right place. So long as you're unit testing (i.e. there's only one call to super), you can mock __builtin__.super as in:
with mock.patch('__builtin__.super') as mock_super:
mock_super.side_effect = TypeError
with self.assertRaises(TypeError):
obj.call_with_super()
Python's own Mock class provides a spec argument that should help with that:
with mock.patch('...ParentClass.myfunc') as mocked_fn:
mocked_fn.side_effect = MyException() # Parent's method will raise
instance = mock.Mock(spec=MyClass) # Enables using super()
MyClass.myfunc(instance) # Will enter your `except` block
Well, then you need to mock the my_function method of the superclass of MyClass to blow up.
Say I have a class:
class A():
def f(self):
self._v = 1
Tried:
m=Mocker()
A.f._v = m.mock()
...
but didn't work. Not sure how...
Did you mean Mock library?
from mock import Mock
real = ProductionClass()
real.method = Mock(return_value=3)
real.method(3, 4, 5, key='value')
edit:
You are trying to access A.f._v before mocking which is impossible.
Not sure what are you trying to do, but this will work
>>>A.f = Mock()
>>>a = A()
>>>a.f._v
<Mock name='mock._v' id='42076240'>
The class definition shows an instance variable to set it from outside this class, do something like this:
class A:
def f(self):
self._v = 1
a = A()
a._v = Mock()
If you actually wanted a real class variable, try this:
class A():
_v = None
def f(self):
self.__class__._v = 1
A._v = Mock()
I tried above solutions and it still does not solve my purpose which is exactly what is asked originally. The above approach would update the mocked attribute of my class to have a value of .
My requirement is to set the attribute value from the mocked value I provide in my unit test class.
I could finally resolved this with the help of following approach. Let me know if its not a correct way:
Actual Class:
class ActualClass(object):
name=''
def some_method(self):
name=get_name_from_external_source() #Say, returned name='ActualValue'
print name
Unit Test Class:
from mock import PropertyMock
import unittest
class TestActualClass(unittest.TestCase):
def test_some_method(self):
actual_class=ActualClass()
p=PropertyMock(return_value='Mocked_Name')
type(actual_class).name=p
actual_class.some_method()
When you run some_method in ActualClass through normal execution, the output:
ActualValue
When you run TestActualClass, the output:
Mocked_Name
This implies that class attributes are mocked with a mocked value using PropertyType and you can test the method with mocked value and without worrying about external source method call.