I am trying to monkeypatch a function in an external module I use, but monkeypatch can't seem to access the function because the namespace of the module gets overwritten on import.
Concretely, I use a Bio.PDB.PDBList.PDBList object (biopython module) in my code, and I am trying to patch _urlretrieve in Bio.PDB.PDBList to prevent calls to the internet and instead get files from a local directory, without having to mock the instance methods of PDBList which would be substantially more work. But when I try the naïve:
m.setattr("Bio.PDB.PDBList._urlretrieve", mock_retrieve)
pytest complains:
AttributeError: 'type' object at Bio.PDB.PDBList has no attribute '_urlretrieve'
On further inspection of Bio.PDB, I can see that the module namespace .PDBList seems to be overwritten by the class .PDBList.PDBList:
# Download from the PDB
from .PDBList import PDBList
So that would explain why pytest sees Bio.PDB.PDBList as a type object with no attribute _urlretrieve. My question is, is there any way to get monkeypatch to patch this 'hidden' function?
Concrete example of usage of PDBList class:
from Bio.PDB.PDBList import PDBList
_pdblist = PDBList()
downloaded_file = _pdblist.retrieve_pdb_file('2O8B', pdir='./temp', file_format='pdb')
You are right - since the PDBList class has the same name as the module Bio.PDB.PDBList, after import Bio.PDB.PDBList you won't be able to access the module by its name (shadowing problem). However, you can still grab the imported module object from the loaded modules cache and monkeypatch that:
import sys
from unittest.mock import Mock
import Bio.PDB.PDBList
def test_spam(monkeypatch):
assert isinstance(Bio.PDB.PDBList, type)
with monkeypatch.context() as m:
m.setattr(sys.modules['Bio.PDB.PDBList'], '_urlretrieve', Mock())
...
Related
I am using pytest-mock, but the exception being thrown is from the mock.patch code, I have verified that the same errot occurs if I use the #mock.patch decorator syntax.
MRE (ensure you have pytest and pytest-mock installed, no need to import anything):
def test_image(mocker):
mocker.patch("PIL.Image.save")
Now run pytest on this module.
Error:
E AttributeError: <module 'PIL.Image' from 'c:\\users\\...\\site-packages\\PIL\\Image.py'> does not have the attribute 'save'
I can see clearly that Image.py does contain a function called save, but are functions not considered attributes? I've never heard that word used for the contents of a module.
save is an instance method of PIL.Image.Image class and not PIL.Image module.
You should implement the patch as:
def test_image(mocker):
mocker.patch("PIL.Image.Image.save")
If you need to make assertions that the save method is invoked on the Image instance, you need a name that is bound to the mock.
You can implement that by mocking the Image class and binding a Mock instance to its save method . For example,
def test_image(mocker):
# prepare
klass = mocker.patch("PIL.Image.Image")
instance = klass.return_value
instance.save = mocker.Mock()
# act
# Do operation that invokes save method on `Image` instance
# test
instance.save.assert_called()
I have what should've been a simple task, and it has stumped me for a while. I am trying to patch an object imported into the current module.
Per the answers to Mock patching from/import statement in Python
I should just be able to patch("__main__.imported_obj"). However, this isn't working for me. Please see my below minimal repro (I am running the tests via pytest):
Minimal Repro
This is run using Python 3.8.6.
from random import random
from unittest.mock import patch
import pytest
#pytest.fixture
def foo():
with patch("__main__.random"):
return
def test(foo) -> None:
pass
When I run this code using PyCharm, I get an AttributeError:
AttributeError: <module '__main__' from '/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_pytest_runner.py'> does not have the attribute 'random'
Furthermore, when I enter debugger mode in the line before the with patch, I see the attribute __main__ is not defined. I am not sure if it needs to be defined for patch to work its magic.
NOTE: I know I can use patch.object and it becomes much easier. However, I am trying to figure out how to use patch in this question.
Research
Unable to mock open, even when using the example from the documentation
This question is related because it's both a similar error message and use case. Their solution was to use builtins instead of __main__, but that's because they were trying to patch a built-in function (open).
You are assuming that the module the test is running in is __main__, but that would only be the case if it were called via main. This is usually the case if you are using unittest. With pytest, the tests live in the module they are defined in.
You have to patch the current module, the name of which is accessible via __name__, instead of assuming a specific module name:
from random import random
from unittest.mock import patch
import pytest
#pytest.fixture
def foo():
with patch(__name__ + ".random"):
yield
I am new in Python. I would like to write an unit test for the following function:
from common.ds_factory import DSFactory
class MyClass:
def load(self, parsed_file_key):
ds = DSFactory.getDS()
...
Now I am unable to mock DSFactory using #patch(my_class.DSFactory) as I am using DSFactory inside the function.
You need to patch DSFactory for module that uses it. As mentioned in comments, assuming MyClass is defined in file my_module.py you patch it by using module name: #patch("my_module.DSFactory").
There's a catch - your module has to be importable, so it requires you to create modules.
I have a function that has a decorator. The decorator accepts arguments and the value of the argument is derived from another function call.
example.py
from cachetools import cached
from cachetools import TTLCache
from other import get_value
#cached(cache=TTLCache(maxsize=1, ttl=get_value('cache_ttl')))
def my_func():
return 'result'
other.py
def get_value(key):
data = {
'cache_ttl': 10,
}
# Let's assume here we launch a shuttle to the space too.
return data[key]
I'd like to mock the call to get_value(). I'm using the following in my test:
example_test.py
import mock
import pytest
from example import my_func
#pytest.fixture
def mock_get_value():
with mock.patch(
"example.get_value",
autospec=True,
) as _mock:
yield _mock
def test_my_func(mock_get_value):
assert my_func() == 'result'
Here I'm injecting mock_get_value to test_my_func. However, since my decorator is called on the first import, get_value() gets called immediately. Any idea if there's a way to mock the call to get_value() before module is imported right away using pytest?
Move the from example import my_func inside your with in your test function. Also patch it where it's really coming from, other.get_value. That may be all it takes.
Python caches modules in sys.modules, so module-level code (like function definitions) only runs on the first import from anywhere. If this isn't the first time, you can force a re-import using either importlib.reload() or by deleting the appropriate key in sys.modules and importing again.
Beware that re-importing a module may have side effects, and you may also want to re-import the module again after running the test to avoid interfering with other tests. If another module was using objects defined in the re-imported module, these don't just disappear, and may not be updated the way it expects. For example, re-importing a module may create a second instance of what was supposed to be a singleton.
One more robust approach would be save the original imported module object somewhere else, delete from sys.modules, re-import with the patched version for the duration of the test, and then put back the original import into sys.modules after the test. You could do this with an import inside of a patch.dict() context on sys.modules.
import mock
import sys
import pytest
#pytest.fixture
def mock_get_value():
with mock.patch(
"other.get_value",
autospec=True,
) as _mock, mock.patch.dict("sys.modules"):
sys.modules.pop("example", None)
yield _mock
def test_my_func(mock_get_value):
from example import my_func
assert my_func() == 'result'
Another possibility is to call the decorator yourself in the test, on the original function. If the decorator used functools.wraps()/functools.update_wrapper(), then original function should be available as a __wrapped__ attribute. This may not be available depending on how the decorator was implemented.
I've imported a class from a module, but when I try to patch the class name without it's module as a prefix I get a type error:
TypeError: Need a valid target to patch. You supplied: 'MyClass'
For example, the following code gives me the above error:
import unittest
from mock import Mock, MagicMock, patch
from notification.models import Channel, addChannelWithName, deleteChannelWithName, listAllChannelNames
class TestChannel(unittest.TestCase):
#patch("Channel")
def testAddChannelWithNamePutsChannel(self, *args):
addChannelWithName("channel1")
Channel.put.assert_called_with()
While this second version of the code does not give me the type error:
import unittest
from mock import Mock, MagicMock, patch
from notification.models import Channel, addChannelWithName, deleteChannelWithName, listAllChannelNames
class TestChannel(unittest.TestCase):
#patch("notification.models.Channel")
def testAddChannelWithNamePutsChannel(self, *args):
addChannelWithName("channel1")
Channel.put.assert_called_with()
Why is that? Why can I reference Channel as just "Channel" in other places, yet for the patch I need the module prefix not to get an error? Also, I have a feeling that giving the full module prefix isn't working either because when I call Channel.put.assert_called_with() I get the error that assert_called_with is not an attribute of Channel.put. Can someone explain what's going on? Thank you much!
The patch decorator requires the target to be a full dotted path, as stated in the documentation:
target should be a string in the form ‘package.module.ClassName’. The target is imported and the specified object replaced with the new object, so the target must be importable from the environment you are calling patch from. The target is imported when the decorated function is executed, not at decoration time.
"Channel" is just a string, and patch does not have enough information to find the proper class. This is not the same as the name Channel you use elsewhere, which is imported at the top of the module.
The second test fails because Channel gets imported in the test module then patch replaces Channel in notification.models with a mock object. What patch actually does is change the object the name Channel used inside notification.models point to. The name Channel in the test module has already been defined, so it is not affected. This is actually better explained here: https://docs.python.org/3/library/unittest.mock.html#where-to-patch
To access the patched version of your object, you can either access the module directly:
import unittest
from unittest.mock import patch
from notification.models import Channel, addChannelWithName
from notification import models
class TestChannel1(unittest.TestCase):
#patch("notification.models.Channel")
def testAddChannelWithNamePutsChannel(self, *args):
addChannelWithName("channel1")
models.Channel.put.assert_called_with("channel1")
Or use the patched version passed as an extra argument to the decorated function:
class TestChannel2(unittest.TestCase):
#patch("notification.models.Channel")
def testAddChannelWithNamePutsChannel(self, mock_channel):
addChannelWithName("channel1")
mock_channel.put.assert_called_with("channel1")
If you just want to quickly patch a single method on an object, it's usually easier to use the patch.object decorator:
class TestChannel3(unittest.TestCase):
#patch.object(Channel, 'put')
def testAddChannelWithNamePutsChannel(self, *arg):
addChannelWithName("channel1")
Channel.put.assert_called_with("channel1")