I have a class which has an __exit__ and __enter__ function so that I can use it in a with statement, e.g.:
with ClassName() as c:
c.do_something()
I am now trying to write a unit test to test this. Basically, I am trying to test that do_something() has only been called once.
An example (which I called testmocking1):
class temp:
def __init__(self):
pass
def __enter__(self):
pass
def __exit__(self, exc_type, exc_val, exc_tb):
pass
def test_method(self):
return 1
def fun():
with temp() as t:
return t.test_method()
And my test:
import unittest
import test_mocking1
from test_mocking1 import fun
import mock
from mock import patch
class MyTestCase(unittest.TestCase):
#patch('test_mocking1.temp', autospec = True)
def test_fun_enter_called_once(self, mocked_object):
fun()
mocked_object.test_method.assert_called_once()
if __name__ == '__main__':
unittest.main()
So I would expect this to pass, because the test_method has been called exactly once in the function fun(). But the actual result that I get is:
======================================================================
FAIL: test_fun_enter_called_once (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<path_to_virtual_env>\lib\site-packages\mock\mock.py", line 1305, in patched
return func(*args, **keywargs)
File "<File_with_test>", line 11, in test_fun_enter_called_once
mocked_object.test_method.assert_called_once()
File "<path_to_virtual_env>\lib\site-
packages\mock\mock.py", line 915, in assert_called_once
raise AssertionError(msg)
AssertionError: Expected 'test_method' to have been called once. Called 0 times.
How do I test whether a function in a class which is created using a with statement has been called (either once or multiple times), and (related) how do I set the results of those calls (using .side_effect or .return_value)?
The with statement takes whatever __enter__ returns to bind to the name in the as <name> part. You bound it to t:
with temp() as t:
t.test_method()
Note that temp() is called, so the with statement starts with temp.return_value. t is not temp.return_value either, it is whatever temp().__enter__() returns, so you need to use the return value for that call:
entered = mocked_object.return_value.__enter__.return_value
entered.test_method.assert_called_once()
Extending on this, if you want to alter what test_method() returns, do so on the return value of mocked_object.return_value.__enter__.return_value.
You can always print out the mock_calls() attribute of your object to see what has happened to it:
>>> from test_mocking1 import fun
>>> from mock import patch
>>> with patch('test_mocking1.temp', autospec = True) as mocked_object:
... fun()
...
>>> print(mocked_object.mock_calls)
[call(),
call().__enter__(),
call().__enter__().test_method(),
call().__exit__(None, None, None)]
>>> mocked_object.return_value.__enter__.return_value.test_method.called
True
>>> mocked_object.return_value.__enter__.return_value.test_method.call_count
1
Note that your actual implementation of temp.__enter__() returns None, so without mocking your fun() function fails with an attribute error.
Related
If we look at python docs it states:
Most context managers are written in a way that means they can only be used effectively in a with statement once. These single use context managers must be created afresh each time they’re used - attempting to use them a second time will trigger an exception or otherwise not work correctly.
This common limitation means that it is generally advisable to create context managers directly in the header of the with statement where they are used (as shown in all of the usage examples above).
Yet, the example most commonly shared for creating context managers inside classes is:
from contextlib import ContextDecorator
import logging
logging.basicConfig(level=logging.INFO)
class track_entry_and_exit(ContextDecorator):
def __init__(self, name):
self.name = name
def __enter__(self):
logging.info('Entering: %s', self.name)
def __exit__(self, exc_type, exc, exc_tb):
logging.info('Exiting: %s', self.name)
But, when I instantiate this class, I can pass it several times to a with statement:
In [8]: test_context = track_entry_and_exit('test')
In [9]: with test_context:
...: pass
...:
INFO:root:Entering: test
INFO:root:Exiting: test
In [10]: with test_context:
...: pass
...:
INFO:root:Entering: test
INFO:root:Exiting: test
How can I create a class that fails on the second call to the with statement?
Here is a possible solution:
from functools import wraps
class MultipleCallToCM(Exception):
pass
def single_use(cls):
if not ("__enter__" in vars(cls) and "__exit__" in vars(cls)):
raise TypeError(f"{cls} is not a Context Manager.")
org_new = cls.__new__
#wraps(org_new)
def new(clss, *args, **kwargs):
instance = org_new(clss)
instance._called = False
return instance
cls.__new__ = new
org_enter = cls.__enter__
#wraps(org_enter)
def enter(self):
if self._called:
raise MultipleCallToCM("You can't call this CM twice!")
self._called = True
return org_enter(self)
cls.__enter__ = enter
return cls
#single_use
class CM:
def __enter__(self):
print("Enter to the CM")
def __exit__(self, exc_type, exc_value, exc_tb):
print("Exit from the CM")
with CM():
print("Inside.")
print("-----------------------------------")
with CM():
print("Inside.")
print("-----------------------------------")
cm = CM()
with cm:
print("Inside.")
print("-----------------------------------")
with cm:
print("Inside.")
output:
Enter to the CM
Inside.
Exit from the CM
-----------------------------------
Enter to the CM
Inside.
Exit from the CM
-----------------------------------
Enter to the CM
Inside.
Exit from the CM
-----------------------------------
Traceback (most recent call last):
File "...", line 51, in <module>
with cm:
File "...", line 24, in enter
raise MultipleCallToCM("You can't call this CM twice!")
__main__.MultipleCallToCM: You can't call this CM twice!
I used a class decorator for it so that you can apply it to other context manager classes. I dispatched the __new__ method and give every instance a flag called __called, then change the original __enter__ to my enter which checks to see if this object has used in a with-statement or not.
How robust is this? I don't know. Seems like it works, I hope it gave an idea at least.
Arguably the simplest method is mentioned two paragraphs further down in the documentation you have cited:
Context managers created using contextmanager() are also single use context managers, and will complain about the underlying generator failing to yield if an attempt is made to use them a second time
Here is the corresponding invocation for your example:
>>> from contextlib import contextmanager
>>> #contextmanager
... def track_entry_and_exit(name):
... print('Entering', name)
... yield
... print('Exiting', name)
...
>>> c = track_entry_and_exit('test')
>>> with c:
... pass
...
Entering test
Exiting test
>>> with c:
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/contextlib.py", line 115, in __enter__
del self.args, self.kwds, self.func
AttributeError: args
It's even a class although it is written as a function:
>>> type(c)
<class 'contextlib._GeneratorContextManager'>
I suggest to consider iterable class instead of context manager, like this
class Iterable:
"""Iterable that can be iterated only once."""
def __init__(self, name):
self.name = name
self.it = iter([self])
def __iter__(self):
# code to acquire resource
print('enter')
yield next(self.it)
print('exit')
# code to release resource
def __repr__(self):
return f'{self.__class__.__name__}({self.name})'
It can be iterated only one
>>> it = Iterable('iterable')
>>> for item in it:
>>> print('entered', item)
enter
entered Iterable(iterable)
exit
>>> for item in it:
>>> print('entered', item)
RuntimeError: generator raised StopIteration
Context manager can be written in this manner:
class Context:
"""Context manager that can be used only once."""
def __init__(self, name):
self.name = name
self.it = iter([self])
def __enter__(self):
print('enter')
return next(self.it)
def __exit__(self, exc_type, exc, exc_tb):
print('exit')
def __repr__(self):
return f'{self.__class__.__name__}({self.name})'
It works only once
>>> ctx = Context('context')
>>> with ctx as c:
>>> print('entered', c)
enter
entered Context(context)
exit
>>> with ctx as c:
>>> print('entered', c)
enter
StopIteration:
Following is my Class main.py
class First:
def __init__(self):
ret_val = self.do_verify()
print(ret_val['value'])
if ret_val['value'] is False:
raise Exception
print('works in object')
def do_verify(self):
return {'value': True}
if __name__ == '__main__':
First()
When I run python3 main.py it prints
True \n works in object
Following is my test case test_main.py
from unittest.mock import patch
import pytest
from main import First
def fake_do_verify():
print('works')
return {'value': False}
class TestMain(object):
#patch.object(First, 'do_verify', autospec=True)
def test_main(self, fake_do_verify):
with pytest.raises(Exception):
First()
This is failing my test case. I am not sure what exactly I am missing. It should passes the test case as mocking is making value False.
=================================================== FAILURES ====================================================
______________________________________________ TestMain.test_main _______________________________________________
self = <test_main.TestMain object at 0x7f98190a39e8>, fake_do_verify = <function do_verify at 0x7f98190d1158>
#patch.object(First, 'do_verify', autospec=True)
def test_main(self, fake_do_verify):
with pytest.raises(Exception):
> First()
E Failed: DID NOT RAISE <class 'Exception'>
test_main.py:17: Failed
The fake_do_verify() function that you defined is not used at all; what #patch is doing is create a MagicMock to patch First.do_verify. The mock object returns another mock when called, which returns another mock when indexed, which evaluates as False when compared against False with the is operator.
If you want to patch with your function, you can do it as follows (but change fake_do_verify to take a self argument):
#patch.object(First, 'do_verify', fake_do_verify)
def test_main(self):
...
Alternatively, you could do without defining that function, leave the patch as you originally had it, but set the return value of the mock object:
#patch.object(First, 'do_verify', autospec=True)
def test_main(self, fake_do_verify):
fake_do_verify.return_value = {'value': False}
...
I encounter a surprising behaviour of the side_effect parameter in patch.object where the function replacing the original does not receive self
class Animal():
def __init__(self):
self.noise = 'Woof'
def make_noise(self):
return self.noise
def loud(self):
return self.noise.upper() + '!!'
from unittest.mock import patch
dog = Animal()
dog.make_noise()
with patch.object(Animal, 'make_noise', side_effect=loud):
dog.make_noise()
This give:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/lustre/home/production/Applications/python/python-3.4.4/lib/python3.4/unittest/mock.py", line 902, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/lustre/home/production/Applications/python/python-3.4.4/lib/python3.4/unittest/mock.py", line 968, in _mock_call
ret_val = effect(*args, **kwargs)
TypeError: loud() missing 1 required positional argument: 'self'
If I change the loud function to
def loud(*args, **kwargs):
print(args)
print(kwargs)
I get the following output:
()
{}
Is there a way to replace a function from an object and still receive self?
self is only supplied for bound methods (because functions are descriptors). A Mock object is not such a method, and the side_effect function is not bound, so self is indeed not going to be passed in.
If you must have access the instance in a side_effect object, you'll have to patch the function on the class with an actual function:
with patch.object(Animal, 'make_noise', new=loud):
Now make_noise is replaced by the loud function on the Animal class, so it'll be bound:
>>> with patch.object(Animal, 'make_noise', new=loud):
... dog.make_noise()
...
'WOOF!!'
The alternative is to set autospec=True, at which point mock will use a real function to mock out make_noise():
>>> with patch.object(Animal, 'make_noise', autospec=True, side_effect=loud):
... dog.make_noise()
...
'WOOF!!'
Also see the Mocking Unbound Methods section in the mock getting started section.
I am trying to test some code that makes an external call. I want to mock that call out. The call takes keyword args, so I wrote this little helper function in my test:
def mock_function(*args, **kwargs)
io_obj = StringIO()
for k,v in kwargs.iteritems():
io_obj.write("{}: {}\n".format(k, v)
print "\n{}".format(io_obj.getvalue()) # for testing purposes
return io_obj
in my setUp function for the test class, I have this:
#patch('function_to_test')
def setUp(self, mock_dude):
self.mock_client = mock_dude.return_value
self.mock_client.function_to_test.side_effect = mock_function
self.client = ClientClass()
in my test function, I am calling the function that calls the external function.
I get the printout from mock_function, so I know that I am mocking the function correctly. My question is this:
How can I get at the io_obj that is created in mock_function? My external function doesn't return anything.
The Mock object actually captures the arguments it's called with, so you don't need to write your own function to do that. You can access the arguments directly using Mock.call_args, or assert that the mock was called with certain arguments using assert_called_with.
Example:
>>> m = mock.Mock()
>>> m(1,2,3)
<Mock name='mock()' id='139905514719504'>
>>> m.call_args
call(1, 2, 3)
>>> m.assert_called_with(1,2,4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib64/python2.6/site-packages/mock.py", line 835, in assert_called_with
raise AssertionError(msg)
AssertionError: Expected call: mock(1, 2, 4)
Actual call: mock(1, 2, 3)
I wrote a function decorator like this:
def tsfunc(func):
def wrappedFunc():
print '%s() called' % func.__name__
return func()
return wrappedFunc()
#tsfunc
def foo():
pass
foo() # to get it work, use foo instead of foo()
foo()
I got following error message:
foo() called
Traceback (most recent call last):
File "decorator.py", line 11, in <module>
foo()
TypeError: 'NoneType' object is not callable
I get it work by replacing "foo()" with "foo". but I still didn't get the result I expected:
foo() called
seems like the foo function is only called once.
Please help me understand why this is happening.
You should return the wrapper function itself, not its result:
def tsfunc(func):
def wrappedFunc():
print '%s() called' % func.__name__
return func()
return wrappedFunc # Do not call the function, return a reference instead
Decorators replace the decorated item with the return value of the decorator:
#tsfunc
def foo():
# ....
is equivalent to:
def foo():
# ....
foo = tsfunc(foo)
which expands to (in your code):
foo = wrappedFunc()
so you were replacing the function foo with the result of the wrappedFunc() call, not with wrappedFunc itself.
You need to remove the parentheses in
return wrappedFunc
The decorator is supposed to return the wrapper function, not call it.
With this fix, the code produces:
foo() called
foo() called
A little late , but hope this helps, Extending on why the answer provided is accepted
as the error states, 'NoneType' means that the instance/object we are trying to call has no type (it is not a function/int/boolean/class/instance ). Its type is just 'None'
So to summarize, decorators are nothing but extended use of closures with functions being treated as first class citizens (You could get a detail view on closures
It basically means a decorator expects a function to be returned , say a wrapper in most cases, with the original function being unaltered and being called withing the decorator
def tsfunc(func):
def wrappedFunc():
print '%s() called' % func.__name__
return func()
return wrappedFunc() -> Here the function is not returned but called eventually
#tsfunc
def foo():
pass
foo() - > calling this again doesnt make sense to trigger the decorator, since a reference for the method foo is enough. Hence foo works fine but foo() doesn't (method call has completed already but no value is returned) If you try like this, you would see that the variable has 'None' type
def tsfunc(func):
def wrappedFunc():
print '%s() called' % func.__name__
return func()
return wrappedFunc -- Here I made the correction
#tsfunc
def foo():
pass
var1 = foo()
print(var1)
This what happened with the call to foo() when you had the incorrect way of calling the wrapper function rather than returning just the function
So for a decorator to function as per the norms, it should return a wrapper function with the original function being unaltered. And hence it should be rewritten as per the accepted answer
you use this syntax
def tsfunc(func):
def wrappedFunc():
print '%s() called' % func.__name__
return func()
return wrappedFunc # not use wrappedFunc() becaues Because this function runs at the this time
#tsfunc
def foo():
pass
foo() #