I want to mock a dict RESOURCES imported from another module. The method under test gets an object stored in the dict RESOURCES and calls that object's method query(). I want to test that query() is being called with the proper arguments, so the mocked RESOURCES needs to store mock objects that I can assert on.
How do I achieve this?
When I run the following code, assert_called_with fails because query is never called.
mymodule.py
from othermodule import RESOURCES # just a dict
def get_resource(date):
resource = RESOURCES["Prod"]
return resource.query(date)
test_mymodule.py
from unittest.mock import MagicMock, Mock, patch
from mymodule import get_resource
import pytest
#patch("mymodule.RESOURCES")
def test_get_resources(mock_resources):
mock_prod_resource = Mock()
mock_prod_resource.query = Mock(return_value=[1, 2])
d = {"Prod": mock_prod_resource}
mock_resources.__getitem__.side_effect = d.__getitem__
result = get_resource("2020-11-01")
mock_prod_resource.query.assert_called_with("2020-11-01")
assert result == [1, 2]
Related
How do you make Python's unittest.mock.patch return an object that lets you assign a callable return value?
For example, I have a custom class in myclass.py defined as:
class MyClass:
#property
def someprop(self):
return 'you should never see this in a test'
I want to test a function that acts on data retrieved from someprop. In my real application, someprop actually calls some complicated external database that's not accessible in a unittest, and isn't really necessary for the purposes of the unittest, so I decide to mock a return value using the patch and the faker package.
So my unittest looks like:
import unittest
import unittest.mock
from faker import Faker
from myclass import MyClass
class Tests(unittest.TestCase):
#unittest.mock.patch('myclass.MyClass.someprop')
def test_mock_error(self, mock_myclass_someprop):
class RandomText:
#property
def text(self):
factory = Faker()
return factory.text()
# Make calls to someprop return random text.
mock_myclass_someprop.return_value = RandomText.text
a = MyClass()
actual_text = a.someprop
print('actual text:', actual_text)
self.assertTrue('MagicMock' not in str(actual_text)) # this fails
if __name__ == '__main__':
unittest.main()
Every time the test runs, the patch causes it to access the text property on my RandomText instance instead of someprop, which should return a unique string. However, this fails because the mock is actually returning a value like <MagicMock name='someprop' id='140126618001360'>.
Why is this, and how do I fix it?
I've tried refactoring how I set and call return_value, but no matter what I do, it returns a MagicMock instance instead of a real return value retrieved from my patched callable.
I have a fixture mocking an external library like so, using pytest-mock, which is a wrapper around unittest.mock.
# client.py
import Test as TestLibrary
class LibraryName():
def get_client():
return TestLibrary.Library()
# library_service.py
def using_library():
'''
Edited note: Library().attribute behind the scenes is set to
self.attribute = Attribute()
so this may be affecting the mocking
'''
client = LibraryName.get_client()
return client.attribute.method()
# conftest.py
#pytest.fixture
def library_client_mock(mocker):
import Test as TestLibrary
return mocker.patch.object(TestLibrary, 'Library')
# test_library_service.py
def test_library_method(library_client_mock):
result = using_library()
I can mock a return value like so:
def test_library_method(library_client_mock):
library_client_mock.return_value.attribute.return_value.method.return_value = "test"
result = using_library()
assert result == "test"
but I can't mock throwing an Exception with side_effect
def test_library_method(library_client_mock):
library_client_mock.return_value.attribute.return_value.method.side_effect = TypeError # doesn't work
library_client_mock.return_value.attribute.return_value.method.side_effect = TypeError() # doesn't work
attrs = { 'attribute.method.side_effect': TypeError }
library_client_mock.configure_mock(**attrs) # doesn't work
with pytest.raises(TypeError):
using_library() # fails assertion
what I missing here?
These are the errors in your code:
Change:
library_client_mock.return_value.attribute.return_value.method.return_value = "test"
To:
library_client_mock.return_value.attribute.method.return_value = "test"
Change:
library_client_mock.return_value.attribute.return_value.method.side_effect = TypeError
To:
library_client_mock.return_value.attribute.method.side_effect = TypeError
Explanation
The .return_value must only be used for callable objects e.g. a function as documented:
return_value
Set this to configure the value returned by calling the mock:
>>> mock = Mock()
>>> mock.return_value = 'fish'
>>> mock()
'fish'
Thus, you can use .return_value only for the following:
TestLibrary.Library()
TestLibrary.Library().attribute.method()
But not for:
TestLibrary.Library().attribute
Because .attribute is not a callable e.g. TestLibrary.Library().attribute().
Warning
The way you are patching Library is via its source location at Test.Library (or aliased as TestLibrary.Library). specifically via:
import Test as TestLibrary
return mocker.patch.object(TestLibrary, 'Library')
It works currently because the way you import and use it is via the root path.
# client.py
import Test as TestLibrary
...
return TestLibrary.Library()
...
But if we change the way we imported that library and imported a local version to client.py:
# client.py
from Test import Library # Instead of <import Test as TestLibrary>
...
return Library() # Instead of <TestLibrary.Library()>
...
It will now fail. Ideally, you should patch the specific name that is used by the system under test, which here is client.Library.
import client
return mocker.patch.object(client, 'Library')
Unless you are sure that all files that will use the library will import only the root and not a local version.
#Niel Godfrey Ponciano set me on the right path with this syntax for the side_effect
library_client_mock.return_value.attribute.method.side_effect = TypeError
but it wasn't enough.
In
# conftest.py
#pytest.fixture
def library_client_mock(mocker):
import Test as TestLibrary
return mocker.patch.object(TestLibrary, 'Library')
I had to add an extra mock:
# conftest.py
#pytest.fixture
def library_client_mock(mocker):
import Test as TestLibrary
mock_library_client = mocker.patch.object(TestLibrary, 'Library')
# option 1
mock_attribute = Mock()
# option 2, path to Library.attribute = Attribute()
mock_attribute = mocker.patch.object(TestLibrary.services, 'Attribute', autospec=True)
mock_library_client.attach_mock(mock_attribute, "attribute")
return mock_library_client
and then both of the following statements worked as expected. Although I am not sure why return_value works out of the box without an attached mock, but side_effect does not.
# return_value set correctly
# NOTE return_value needed after each
library_client_mock.return_value.attribute.return_value.method.return_value = "test"
# side_effect set correctly
# NOTE return_value not needed after "attribute"
library_client_mock.return_value.attribute.method.side_effect = TypeError
In my object, I have several "Loader" functions for different types of data that might require loading. They're stored as:
import foo
import bar
class MyClass
LOADERS = {
'foo': {
'context_manager': False,
'func': foo.data.loader,
'kwargs': {'data_orientation': 'columns'}
},
'bar': {
'context_manager': True,
'func': bar.load_data
'kwargs': {}
}
}
def load_data(self, load_type, path):
func = self.LOADERS[load_type]['func']
kwargs = self.LOADERS[load_type]['kwargs']
if self.LOADERS[load_type]['context_manager']:
with open(path, 'rb') as f:
self.data=func(f, **kwargs)
else:
self.data = func(path, **kwargs)
This is working very well in practice, but I'm finding it's murder to test.
When I write my test:
from mock import MagicMock, patch
import sys
sys.modules['foo'] = MagicMock()
sys.modules['bar'] = MagicMock()
from mypackage import MyClass
#patch('foo.data.loader')
def test_load_foo(mock_loader):
my_obj = MyClass()
my_obj.load_data('foo', 'path/to/data')
mock_loader.assert_called_once()
it fails. Called 0 times.
I halfway suspect that it's because it's not being called directly. But that shouldn't really matter.
Can anyone offer any suggestions? I'm using pytest as my testing engine, but I've found it plays nicely with mock in the past.
Thanks!
The issue arises because you are trying to patch a function within a class attribute as opposed to an instance attribute. The behavior is not the same. This does however offer us some flexibility as it is a class attribute, thus we can modify the object directly on the class we instantiate in our test.
from unittest.mock import MagicMock
from mypackage import MyClass
def test_load_foo():
key_to_test = "foo"
mock_loader = MagicMock()
my_obj = MyClass()
my_obj.LOADERS[key_to_test]["func"] = mock_loader
my_obj.load_data(key_to_test, 'path/to/data')
mock_loader.assert_called_once()
Instead of trying to patch the function, we modify our objects instance to use the MagicMock.
Using this approach I receive the following message:
collected 1 item
tests/test_mypackage.py . [100%]
======================================================================= 1 passed in 0.01s =======================================================================
How do I mock a class with this path ablib.Pin? And how do I mock the attributes of an instance off Pin? I would say it should work like this:
mock = MagicMock()
mock.Pin = MagicMock()
mock.Pin.kernel_id = 'N21'
mock.Pin.set_value.return_value = True
mock.Pin.get_value.return_value = 3
modules = {
'ablib': mock,
'ablib.Pin': mock.Pin,
}
patcher = patch.dict('sys.modules', modules)
patcher.start()
When I create a Pin instance I and call get_value or set_value I get a MockInstance instead of True or 3.
>>> p = Pin()
>>> p.set_value(3)
<MagicMock name='mock.Pin().set_value(3)' id='47965968'>
>>> p.kernel_id
<MagicMock name='mock.Pin().kernel_id' id='49231056'>
When I call kernel_id directly on Pin I get the result I need.
>>> Pin.kernel_id
'N21'
How do I mock ablib.Pin in a way that I get the values I want from an instance ablib.Pin()
Here is the working version with minimal changes to your code:
from unittest.mock import patch, MagicMock
mock = MagicMock()
mock.Pin().kernel_id = 'N21'
mock.Pin().set_value.return_value = True
mock.Pin().get_value.return_value = 3
modules = {
'ablib': mock
}
patcher = patch.dict('sys.modules', modules)
patcher.start()
Test:
>>> import ablib
>>> ablib.Pin
<MagicMock name='mock.Pin' id='139917634188240'>
>>> ablib.Pin
<MagicMock name='mock.Pin' id='139917634188240'>
>>> ablib.Pin()
<MagicMock name='mock.Pin()' id='139917634233616'>
>>> ablib.Pin()
<MagicMock name='mock.Pin()' id='139917634233616'>
>>> p = ablib.Pin()
>>> p.set_value(3)
True
>>> p.kernel_id
'N21'
Notice that I've removed mock.Pin = MagicMock() and removed mock.Pin from the dictionary. New MagicMock instance is created by default when you access existing MagicMock attribute or call it. The same instance will be returned next time you do that action. So there is no need to explicitly create MagicMock for mock.Pin (for Pin class), mock.Pin() (for Pin instance) and to register them with patch.dict(). Just registering mock is enough.
Thanks to patch.dict() it works even when module ablib doesn't exist. I believe that it is not possible to achieve with patch() and patch.object(). Nevertheless for real unit-testing I use patch() to mock a class and patch.object() to mock attributes of an instance for some particular test or group of tests. I have not been in a situation when I needed to mock a whole module, but I'm quite new to unit-testing and Python.
I'm trying to get the mock library to return a specific mock object with patch. No matter what I try, if I instantiate a new object inside my 'with' clause, I get a generic Mock object, not my customised one.
Here's my code using logging as an example:
import logging
my_mock = Mock()
my_mock.id = 'i-12341234'
with patch('logging.PlaceHolder', new=my_mock) as mockobj:
print mockobj.id # correctly prints 'i-12341234'
newobj = logging.PlaceHolder(None)
print newobj.id # prints <Mock name='mock().id' id='47607184'>
How can I get mock to return my_mock every time a new instance of logging.Placeholder is instantiated?
As it turns out subclassing Mock and passing it using new_callable works. Since I've spent the time trying to get the version above working I'd still appreciate and answer.
import logging
from mock import Mock, patch
my_mock = Mock()
my_mock.id = 'i-12341234'
with patch('logging.PlaceHolder', new=my_mock) as mockobj:
mockobj.return_value = mockobj # <-- added this line.
print mockobj.id
newobj = logging.PlaceHolder(None)
print newobj.id
Alternative
import logging
from mock import Mock, patch
my_mock = Mock()
my_mock.id = 'i-12341234'
my_mock.return_value = my_mock # <---
with patch('logging.PlaceHolder', new=my_mock) as mockobj:
assert mockobj.id == 'i-12341234'
newobj = logging.PlaceHolder(None)
assert newobj.id == 'i-12341234'