When you patch a function using mock, you have the option to specify autospec as True:
If you set autospec=True then the mock with be created with a spec
from the object being replaced. All attributes of the mock will also
have the spec of the corresponding attribute of the object being
replaced. Methods and functions being mocked will have their arguments
checked and will raise a TypeError if they are called with the wrong
signature.
(http://www.voidspace.org.uk/python/mock/patch.html)
I'm wondering why this isn't the default behaviour? Surely we would almost always want to catch passing incorrect parameters to any function we patch?
The only clear way to explain this, is to actually quote the documentation on the downside of using auto-speccing and why you should be careful when using it:
This isn’t without caveats and limitations however, which is why it is
not the default behaviour. In order to know what attributes are
available on the spec object, autospec has to introspect (access
attributes) the spec. As you traverse attributes on the mock a
corresponding traversal of the original object is happening under the
hood. If any of your specced objects have properties or descriptors
that can trigger code execution then you may not be able to use
autospec. On the other hand it is much better to design your objects
so that introspection is safe [4].
A more serious problem is that it is common for instance attributes to
be created in the init method and not to exist on the class at
all. autospec can’t know about any dynamically created attributes and
restricts the api to visible attributes.
I think the key takeaway here is to note this line: autospec can’t know about any dynamically created attributes and restricts the api to visible attributes
So, to help being more explicit with an example of where autospeccing breaks, this example taken from the documentation shows this:
>>> class Something:
... def __init__(self):
... self.a = 33
...
>>> with patch('__main__.Something', autospec=True):
... thing = Something()
... thing.a
...
Traceback (most recent call last):
...
AttributeError: Mock object has no attribute 'a'
As you can see, auto-speccing has no idea that there is an attribute a being created when creating your Something object.
There is nothing wrong with assigning a value to your instance attribute.
Observe the below functional example:
import unittest
from mock import patch
def some_external_thing():
pass
def something(x):
return x
class MyRealClass:
def __init__(self):
self.a = some_external_thing()
def test_thing(self):
return something(self.a)
class MyTest(unittest.TestCase):
def setUp(self):
self.my_obj = MyRealClass()
#patch('__main__.some_external_thing')
#patch('__main__.something')
def test_my_things(self, mock_something, mock_some_external_thing):
mock_some_external_thing.return_value = "there be dragons"
self.my_obj.a = mock_some_external_thing.return_value
self.my_obj.test_thing()
mock_something.assert_called_once_with("there be dragons")
if __name__ == '__main__':
unittest.main()
So, I'm just saying for my test case I want to make sure that the some_external_thing() method does not affect the behaviour of my unittest, so I'm just assigning my instance attribute the mock per mock_some_external_thing.return_value = "there be dragons".
Answering my own question many years later - another reason is speed.
Depending on how complex your object is, it can be that using autospec can slow your test down significantly. I've found this particularly when patching Django models.
The action of autospeccing itself can execute code, for example via the invocation of descriptors.
>>> class A:
... #property
... def foo(self):
... print("rm -rf /")
...
>>> a = A()
>>> with mock.patch("__main__.a", autospec=False) as m:
... pass
...
>>> with mock.patch("__main__.a", autospec=True) as m:
... pass
...
rm -rf /
Therefore, this is a problematic feature to enable by default and
is opt-in only.
Related
I have a question related to Python unittest.mock.Mock and spec_set functionalities.
My goal is to create a Mock with the following functionalities:
It has a spec of an arbitrary class I decide at creation time.
I must be able to assign on the mock only attributes or methods according to the spec of point 1
The Mock must raise AttributeError in the following situations:
I try to assign an attribute that is not in the spec
I call or retrieve a property that is either missing in the spec_set, or present in the spec_set but assigned according to the above point.
Some examples of the behavior I would like:
class MyClass:
property: int = 5
def func() -> int:
pass
# MySpecialMock is the Mock with the functionalities I am dreaming about :D
mock = MyMySpecialMock(spec_set=MyClass)
mock.not_existing # Raise AttributeError
mock.func() # Raise AttributeError
mock.func = lambda: "it works"
mock.func() # Returns "it works"
I have tried multiple solutions without any luck, or without being explicitly verbose. The following are some examples:
Using Mock(spec_set=...), but it does not raise errors in case I call a specced attribute which I did not explicitly set
Using Mock(spec_set=...) and explicitly override every attribute with a function with an Exception side effect, but it is quite verbose since I must repeat all the attributes...
My goal is to find a way to automatize 2, but I have no clean way to do so. Did you ever encounter such a problem, and solve it?
For the curious ones, the goal is being able to enhance the separation of unit testings; I want to be sure that my mocks are called only on the methods I explicitly set, to avoid weird and unexpected side effects.
Thank you in advance!
spec_set defines a mock object which is the same as the class, but then doesn't allow any changes to be made to it, since it defines special __getattr__ and __setattr__. This means that the first test (calling a non-existent attr) will fail as expected, but then so will trying to set an attr:
from unitest import mock
class X:
pass
m = mock.Mock(spec_set=X)
m.func()
# __getattr__: AttributeError: Mock object has no attribute 'func'
m.func = lambda: "it works"
# __setattr__: AttributeError: Mock object has no attribute 'func'
Instead, you can use create_autospec() which copies an existing function, and adds the mock functions to it, but without affecting __setattr__:
n = mock.create_autospec(X)
n.func()
# __getattr__: AttributeError: Mock object has no attribute 'func'
n.func = lambda: "it works"
n.func()
# 'it works'
I think I found a satisfying answer to my problem, by using the dir method.
To create the Mock with the requirements I listed above, it should be enough to do the following:
def create_mock(spec: Any) -> Mock:
mock = Mock(spec_set=spec)
attributes_to_override = dir(spec)
for attr in filter(lambda name: not name.startswith("__"), attributes_to_override):
setattr(mock, attr, Mock(side_effect=AttributeError(f"{attr} not implemented")))
return mock
When should I use autospec=True when using mock.patch and its variants?
On one hand, this article warns us to always use autospec=True:
... you should always use the create_autospec method and the autospec parameter with the #patch and #patch.object decorators.
On the other hand, autospec has serious drawbacks and limits, as explained in idjaw's answer to this question.
So my question is: when should I use autospec=True or create_autospec, and when should I not use it?
I fear not using autospec may result in tests not breaking when they really should break, as described in the mentioned article. However autospec has its drawbacks. How should I act?
I can understand the motivation to suggest the enforcing of using autospec.
Maybe the following could help provide more clarity on what you get and don't get with autospec.
In short, using autospec ensures that the attributes you use in your mock are in fact part of the class you are mocking.
So, with the example below, I'll illustrate how a test will pass when technically you might not want it to:
Take this simple example we will test:
class Foo:
def __init__(self, x):
self.x = x
class Bar:
def __init__(self):
self.y = 4
self.c = Foo('potato')
And the test code:
class TestAutoSpec(unittest.TestCase):
#patch('some_module.Foo')
def test_autospec(self, mock_foo_class):
mock_foo_obj = mock_foo_class.return_value
bar_obj = some_module.Bar()
self.assertTrue(hasattr(bar_obj.c, 'you_should_fail'))
Now, if you look back at the Foo class, you will clearly see you_should_fail is clearly not an attribute in Foo. However, if you run this test code, it will in fact pass. Which is very misleading.
This is because if an attribute does not exist in a MagicMock, it will still be of type MagicMock. If you print type(bar_obj.c.you_should_fail) in that test, you will end up getting:
<class 'unittest.mock.MagicMock'>
This will certainly cause the hasattr test to pass. If you run the above test again, except change your patch to be: #patch('some_module.Foo', autospec=True), it will fail as it should.
Now, to write a successful test for this and still use autospec=True, you simply create the attribute in your mock testing as needed. Remember, the reason this is needed, is because autospec cannot know about the attributes created dynamically, i.e. in the __init__ when you create an instance.
So, the autospec way to do this, would be:
class TestAutoSpec(unittest.TestCase):
#patch('some_module.Foo', autospec=True)
def test_autospec(self, mock_foo_class):
mock_foo_obj = mock_foo_class.return_value
# create the attribute you need from mocked Foo
mock_foo_obj.x = "potato"
bar_obj = some_module.Bar()
self.assertEqual(bar_obj.c.x, 'potato')
self.assertFalse(hasattr(bar_obj.c, 'poof'))
Now, your test will successfully pass at validating your x attribute, while also validating that you don't have some bogus attribute that does not exist in your real Foo class.
Here is also another explanation by Martijn Pieters, that does not necessarily directly answer your question, but gives a very good example and explanation of using autospec that can help further your understanding:
https://stackoverflow.com/a/31710001/1832539
The Mock documentation describes a simple and elegant way of applying patches to all of the tests method inside a TestCase:
#patch('foo.bar')
#patch('foo.baz')
#patch('foo.quux')
#patch('foo.narf')
class FooTest(TestCase):
def test_foo(self, bar, baz, quux, narf):
""" foo """
self.assertTrue(False)
However, one issue I've encountered with this method is that if I'd like to call stop() on one of the patches inside one of the test methods, there doesn't appear to be anyway of getting a reference to the patcher object -- the only thing that is passed into the method is the mock objects, in this case bar, baz, quux, narf.
The only way I've found to solve this problem is to move to the pattern described in the Mock docs where the patchers are instantiated and started inside the setUp method of the TestCase and stopped inside the tearDown method. This fits my purpose, but adds a lot of extra boilerplate and isn't as elegant as the class decorator approach.
Is there another way to solve this problem?
1
Say you want to temporarily restore foo.narf in a method. foo.narf is, in the context of the decorated function, a MagicMock object. This object has a _mock_wraps attribute which will be invoked when the mock is called! So at the top of your module, _narf = foo.narf, and in your test case, foo.narf._mock_wraps = _narf.
The catch is that this will only pass through to the real function, not actually swap it back, which means that some test cases will fail (e.g. if they rely on the function object actually being "itself"). And if your mock has other attributes, that could interfere (I haven't tested much) because the passthrough call to _mock_wraps() comes at the bottom of a method that first considers the other properties of the mock.
2
The patch() decorator involves each patcher (separate copies per method) being added to a list called patchings which is a field of the method itself. I.e. you can access this list as self.test_foo.patchings, and go through to find the one you want.
However, start() and stop() are not actually called when you use patch() as a decorator, and the behavior gets tricky once you start reaching in and changing it. So I wrote this context manager.
class unpatch:
def __init__(self, name, method):
compare = patch(name)
self.patcher = next((
p for p in method.patchings
if p.target == compare.getter()
and p.attribute == compare.attribute
), None)
if self.patcher is None:
raise ValueError(name)
def __enter__(self):
self.patcher.__exit__()
def __exit__(self, *exc_info):
self.patcher.__enter__()
Inside your test case, you use it like this:
with unpatch('foo.narf', self.test_foo):
foo.narf()
Disclaimer: this is hacks.
Is there any equivalent of strict mocks in python? Some mechanism to report unintended call of mocked methods (action.step2() in this example), just like this in GoogleMock framework.
class Action:
def step1(self, arg):
return False
def step2(self, arg):
return False
def algorithm(action):
action.step1('111')
action.step2('222')
return True
class TestAlgorithm(unittest.TestCase):
def test_algorithm(self):
actionMock = mock.create_autospec(Action)
self.assertTrue(algorithm(actionMock))
actionMock.step1.assert_called_once_with('111')
Looks like it's not supported out of the box. However there are at least two approaches on how to achieve the same result.
Passing list of allowed members
According to mock documentation
spec: This can be either a list of strings or an existing object (a class or instance) that acts as the specification for the mock object. If you pass in an object then a list of strings is formed by calling dir on the object (excluding unsupported magic attributes and methods). Accessing any attribute not in this list will raise an AttributeError.
So, in order to fail your test example just replace
actionMock = mock.create_autospec(Action)
to
actionMock = mock.Mock(spec=['step1'])
Such an approach have certain drawbacks compared to passing class or instance as spec argument, as you have to pass all the allowed methods and than set up expectations on them, effectively registering them twice. Also, if you need to restrict a subset of methods you have to pass list of all methods execept those. This can be achieved as follows:
all_members = dir(Action) # according to docs this is what's happening behind the scenes
all_members.remove('step2') # remove all unwanted methods
actionMock = mock.Mock(spec=all_members)
Setting exceptions on restricted methods
Alternative approach would be to excplicitly set failures on methods you don't want to be called:
def test_algorithm(self):
actionMock = mock.create_autospec(Action)
actionMock.step2.side_effect = AttributeError("Called step2") # <<< like this
self.assertTrue(algorithm(actionMock))
actionMock.step1.assert_called_once_with('111')
This have some limitations as well: you've got to set errors as well as expectations.
As a final note, one radical solution to the problem would be to patch mock to add strict parameter to Mock constructor and send a pull request. Than either it would be accepted or mock maintainers will point out on how to achieve that. :)
Yes, this is possible using the spec= and autospec= arguments. See the mock documentation on Autospeccing for more information. In your example it would become:
action_mock = mock.Mock(spec=Action)
or:
action_mock = mock.Mock('Action', autospec=True)
Another possibility:
Checking call_count individually on restricted methods
Ensure that call_count is zero on methods that should not be called.
class TestAlgorithm(unittest.TestCase):
def test_algorithm(self):
actionMock = mock.create_autospec(Action)
self.assertTrue(algorithm(actionMock))
actionMock.step1.assert_called_once_with('111')
self.assertEqual(actionMock.step2.call_count, 0) # <<< like this
The drawback is that you have to check all unexpected calls one by one.
I'm writing a decorator, and for various annoying reasons[0] it would be expedient to check if the function it is wrapping is being defined stand-alone or as part of a class (and further which classes that new class is subclassing).
For example:
def my_decorator(f):
defined_in_class = ??
print "%r: %s" %(f, defined_in_class)
#my_decorator
def foo(): pass
class Bar(object):
#my_decorator
def bar(self): pass
Should print:
<function foo …>: False
<function bar …>: True
Also, please note:
At the point decorators are applied the function will still be a function, not an unbound method, so testing for instance/unbound method (using typeof or inspect) will not work.
Please only offer suggestions that solve this problem — I'm aware that there are many similar ways to accomplish this end (ex, using a class decorator), but I would like them to happen at decoration time, not later.
[0]: specifically, I'm writing a decorator that will make it easy to do parameterized testing with nose. However, nose will not run test generators on subclasses of unittest.TestCase, so I would like my decorator to be able to determine if it's being used inside a subclass of TestCase and fail with an appropriate error. The obvious solution - using isinstance(self, TestCase) before calling the wrapped function doesn't work, because the wrapped function needs to be a generator, which doesn't get executed at all.
Take a look at the output of inspect.stack() when you wrap a method. When your decorator's execution is underway, the current stack frame is the function call to your decorator; the next stack frame down is the # wrapping action that is being applied to the new method; and the third frame will be the class definition itself, which merits a separate stack frame because the class definition is its own namespace (that is wrapped up to create a class when it is done executing).
I suggest, therefore:
defined_in_class = (len(frames) > 2 and
frames[2][4][0].strip().startswith('class '))
If all of those crazy indexes look unmaintainable, then you can be more explicit by taking the frame apart piece by piece, like this:
import inspect
frames = inspect.stack()
defined_in_class = False
if len(frames) > 2:
maybe_class_frame = frames[2]
statement_list = maybe_class_frame[4]
first_statment = statement_list[0]
if first_statment.strip().startswith('class '):
defined_in_class = True
Note that I do not see any way to ask Python about the class name or inheritance hierarchy at the moment your wrapper runs; that point is "too early" in the processing steps, since the class creation is not yet finished. Either parse the line that begins with class yourself and then look in that frame's globals to find the superclass, or else poke around the frames[1] code object to see what you can learn — it appears that the class name winds up being frames[1][0].f_code.co_name in the above code, but I cannot find any way to learn what superclasses will be attached when the class creation finishes up.
A little late to the party here, but this has proven to be a reliable means of determining if a decorator is being used on a function defined in a class:
frames = inspect.stack()
className = None
for frame in frames[1:]:
if frame[3] == "<module>":
# At module level, go no further
break
elif '__module__' in frame[0].f_code.co_names:
className = frame[0].f_code.co_name
break
The advantage of this method over the accepted answer is that it works with e.g. py2exe.
Some hacky solution that I've got:
import inspect
def my_decorator(f):
args = inspect.getargspec(f).args
defined_in_class = bool(args and args[0] == 'self')
print "%r: %s" %(f, defined_in_class)
But it relays on the presence of self argument in function.
you can use the package wrapt to check for
- instance/class methods
- classes
- freestanding functions/static methods:
See the project page of wrapt: https://pypi.org/project/wrapt/
You could check if the decorator itself is being called at the module level or nested within something else.
defined_in_class = inspect.currentframe().f_back.f_code.co_name != "<module>"
I think the functions in the inspect module will do what you want, particularly isfunction and ismethod:
>>> import inspect
>>> def foo(): pass
...
>>> inspect.isfunction(foo)
True
>>> inspect.ismethod(foo)
False
>>> class C(object):
... def foo(self):
... pass
...
>>> inspect.isfunction(C.foo)
False
>>> inspect.ismethod(C.foo)
True
>>> inspect.isfunction(C().foo)
False
>>> inspect.ismethod(C().foo)
True
You can then follow the Types and Members table to access the function inside the bound or unbound method:
>>> C.foo.im_func
<function foo at 0x1062dfaa0>
>>> inspect.isfunction(C.foo.im_func)
True
>>> inspect.ismethod(C.foo.im_func)
False