I would like to test the batch_write method using unit test.
class DataService:
def __init__(self, table):
dynamodb = boto3.resource('dynamodb', region_name='us-west-2')
self.db = dynamodb.Table(self.table)
def batch_write(self, items):
with self.db.batch_writer() as batch:
for item in items:
batch.put_item(Item=item)
I want to mock the batch_writer(). I have seen usage of mock_open used to mock file open methods. But I don't really understand can i mock this streaming batch_writer() call.
I came across this issue as well this is how i solved it.
from mock import MagicMock
class TestClass():
def __enter__(self, *args):
return self
def __exit__(self, *args):
pass
cls = TestClass()
cls.put_item = MagicMock()
repository.table.batch_writer = MagicMock()
repository.table.batch_writer.return_value = cls
print cls.put_item.call_args_list
Now i can pass my mock test class in place of the batch_writer to mock calls or check params. I know this is an old question but it is one that I struggled with and wished someone would have posted a code example if this is easy to do. If there is better easier way using patch that anyone has please post.
I wrote a slightly different version, based on Codyj110's idea, but it won't require to create the TestClass.
Instead of creating the new class, I'm just setting the __enter__ and __exit__ methods with mocked values.
class TestClassRequiringMockOnDynamoDbBatchWriter:
#patch('boto3.resource')
def test_mocked_batch_writer(self, mock_dynamo_db):
mock_db, mock_batch_writer = self._get_mocked_dynamo_objects(fake_exception)
mock_dynamo_db.return_value = mock_db
assert mock_batch_writer.put_item.call_count == my_expected_calls
#staticmethod
def _get_mocked_dynamo_objects(expected_value):
mock_batch_writer = Mock()
mock_batch_writer.__enter__ = Mock(return_value=mock_batch_writer)
mock_batch_writer.__exit__ = Mock(return_value=None)
# Use side_effect or return_value according to your intention
mock_batch_writer.put_item.side_effect = expected_value
mock_table = Mock()
mock_table.batch_writer.return_value = mock_batch_writer
mock_db = Mock()
mock_db.Table.return_value = mock_table
return mock_db, mock_batch_writer
I hope it helps someone!
Related
I am using pytest to test a method that calls requests.post. Is there a easy way and preferably without third party libraries to do this?
class Dispatcher:
def __init__(self, url):
self.url = url
self.session = None
def dispatch(self):
return self.session.post(self.url).json()
def test_dispatch():
d = Dispatcher(url="")
d.session = # ... here, how can I mock the return value of json()?
result = d.dispatch()
So this is a pretty straightforward example. We want to set session to be a MagicMock object.
from unittest.mock import MagicMock
def test_dispatch():
expected = {"fizz": "buzz"}
mock_session = MagicMock()
mock_session.configure_mock(
**{
"post.return_value": mock_session,
"json.return_value": expected
}
)
d = Dispatcher(url="")
d.session = mock_session
result = d.dispatch()
assert result == expected
Since Mock objects return a brand new mock object when methods are called on them (without being configured), we have to configure the object as such. If we didn't configure post to return the original mock we have, then it would return a brand new mock object and our test would fail. Conversely you can configure another Mock object to be the return value of post and configure that object, but I prefer this approach.
Explanation showing the call is listed below.
self.session is our mock_session object we created
self.session.post(arg, **kwargs) returns mock_session
mock_session.json() returns the dictionary we specified
I have a python method like
import external_object
from external_lib1 import ExternalClass1
from external_lib2 import Hook
class MyClass(self):
def my_method(self):
ExternalClass.get('arg1') #should be mocked and return a specific value with this arg1
ExternalClass.get('arg2') #should be mocked and return a specific value with this arg2
def get_hook(self):
return Hook() # return a mock object with mocked method on it
def my_method(self):
object_1 = external_object.instance_type_1('args') # those are two different object instanciate from the same lib.
object_2 = external_object.instance_type_2('args')
object_1.method_1('arg') # should return what I want when object_1 mocked
object_2.method_2 ('arg') # should return what I want when object_2 mocked
In my test I would like to realise what I put in comments.
I could manage to do it, but every time it gets really messy.
I use to call flexmock for some stuff (by example ExternalClass.get('arg1') would be mock with a flexmock(ExternalClass).should_return('arg').with_args('arg') # etc...) but I'm tired of using different test libs to mock.
I would like to use only the mock library but I struggle to find a consistent way of doing it.
I like to use python's unittest lib. Concretely the unittest.mock which is a great lib to customize side effects and return value in unit tested functions.
They can be used as follows:
class Some(object):
"""
You want to test this class
external_lib is an external component we cannot test
"""
def __init__(self, external_lib):
self.lib = external_lib
def create_index(self, unique_index):
"""
Create an index.
"""
try:
self.lib.create(index=unique_index) # mock this
return True
except MyException as e:
self.logger.error(e.__dict__, color="red")
return False
class MockLib():
pass
class TestSome(unittest.TestCase):
def setUp(self):
self.lib = MockLib()
self.some = Some(self.lib)
def test_create_index(self):
# This will test the method returns True if everything went fine
self.some.create_index = MagicMock(return_value={})
self.assertTrue(self.some.create_index("test-index"))
def test_create_index_fail(self):
# This will test the exception is handled and return False
self.some.create_index = MagicMock(side_effect=MyException("error create"))
self.assertFalse(self.some.create_index("test-index"))
Put the TestSome() class file somewhere like your-codebase-path/tests and run:
python -m unittest -v
I hope it's useful.
I'm trying to make a simple test in python, but I'm not able to figure it out how to accomplish the mocking process.
This is the class and def code:
class FileRemoveOp(...)
#apply_defaults
def __init__(
self,
source_conn_keys,
source_conn_id='conn_default',
*args, **kwargs):
super(v4FileRemoveOperator, self).__init__(*args, **kwargs)
self.source_conn_keys = source_conn_keys
self.source_conn_id = source_conn_id
def execute (self, context)
source_conn = Connection(conn_id)
try:
for source_conn_key in self.source_keys:
if not source_conn.check_for_key(source_conn_key):
logging.info("The source key does not exist")
source_conn.remove_file(source_conn_key,'')
finally:
logging.info("Remove operation successful.")
And this is my test for the execute function:
#mock.patch('main.Connection')
def test_remove_execute(self,MockConn):
mock_coon = MockConn.return_value
mock_coon.value = #I'm not sure what to put here#
remove_operator = FileRemoveOp(...)
remove_operator.execute(self)
Since the execute method try to make a connection, I need to mock that, I don't want to make a real connection, just return something mock. How can I make that? I'm used to do testing in Java but I never did on python..
First it is very important to understand that you always need to Mock where it the thing you are trying to mock out is used as stated in the unittest.mock documentation.
The basic principle is that you patch where an object is looked up,
which is not necessarily the same place as where it is defined.
Next what you would need to do is to return a MagicMock instance as return_value of the patched object. So to summarize this you would need to use the following sequence.
Patch Object
prepare MagicMock to be used
return the MagicMock we've just created as return_value
Here a quick example of a project.
connection.py (Class we would like to Mock)
class Connection(object):
def execute(self):
return "Connection to server made"
file.py (Where the Class is used)
from project.connection import Connection
class FileRemoveOp(object):
def __init__(self, foo):
self.foo = foo
def execute(self):
conn = Connection()
result = conn.execute()
return result
tests/test_file.py
import unittest
from unittest.mock import patch, MagicMock
from project.file import FileRemoveOp
class TestFileRemoveOp(unittest.TestCase):
def setUp(self):
self.fileremoveop = FileRemoveOp('foobar')
#patch('project.file.Connection')
def test_execute(self, connection_mock):
# Create a new MagickMock instance which will be the
# `return_value` of our patched object
connection_instance = MagicMock()
connection_instance.execute.return_value = "testing"
# Return the above created `connection_instance`
connection_mock.return_value = connection_instance
result = self.fileremoveop.execute()
expected = "testing"
self.assertEqual(result, expected)
def test_not_mocked(self):
# No mocking involved will execute the `Connection.execute` method
result = self.fileremoveop.execute()
expected = "Connection to server made"
self.assertEqual(result, expected)
I found that this simple solution works in python3: you can substitute a whole class before it is being imported for the first time. Say I have to mock class 'Manager' from real.manager
class MockManager:
...
import real.manager
real.manager.Manager = MockManager
It is possible to do this substitution in init.py if there is no better place.
It may work in python2 too but I did not check.
I am doing some unit testing and at some point I need to mock a super call to throw an error, for example:
#classmethod
def myfunc(cls, *args, **kwargs)
try:
super(MyClass, cls).my_function(args, kwargs)
except MyException as e:
#...
I am using the mocker library to mock my objects in general but I haven't found a way to mock this.
Using unittest.mock from the standard library I would do something like this.
In your class definition:
from somelib import ASuperClass
class MyClass(ASuperClass):
def my_cool_method(self):
return super().my_cool_method()
In the module where you are calling MyClass:
from unittest.mock import patch
from mymodule import MyClass
#patch("mypackage.mymodule.ASuperClass.my_cool_method")
def call_with_mock(mocked_super):
myinstance = MyClass()
myinstance.my_cool_method()
# do stuff with `mocked_super`
call_with_mock()
I found a way, sort of hacky but it works, I'll explain with my example, this is based on this response so thanks #kindall:
def my_test(self):
import __builtin__
from mocker import Mocker, KWARGS, ARGS
mymocker = mocker.mock()
mymocker.my_function(ARGS, KWARGS)
mocker.throw(MyException)
def mysuper(*args, **kwargs):
if args and issubclass(MyClass, args[0]):
return mymocker
return original_super(*args, **kwargs)
__builtin__.original_super = super
__builtin__.super = mysuper
with mocker:
MyClass.myfunc()
so essentially what I do is check if the super call is from the class I want to mock, else just do a normal super.
Hope this helps someone :)
In case anyone needs another way to solve this mock:
# some_package/some_module.py
class MyClass(SuperClass):
def some_function(self):
result_super_call = super().function()
# test_file.py
#patch('some_package.some_module.super')
def test_something(self, mock_super):
obj = MyClass()
mock_super().some_function.return_value = None
Using Python 3.6
#Markus is looking in the right place. So long as you're unit testing (i.e. there's only one call to super), you can mock __builtin__.super as in:
with mock.patch('__builtin__.super') as mock_super:
mock_super.side_effect = TypeError
with self.assertRaises(TypeError):
obj.call_with_super()
Python's own Mock class provides a spec argument that should help with that:
with mock.patch('...ParentClass.myfunc') as mocked_fn:
mocked_fn.side_effect = MyException() # Parent's method will raise
instance = mock.Mock(spec=MyClass) # Enables using super()
MyClass.myfunc(instance) # Will enter your `except` block
Well, then you need to mock the my_function method of the superclass of MyClass to blow up.
Say I have a class:
class A():
def f(self):
self._v = 1
Tried:
m=Mocker()
A.f._v = m.mock()
...
but didn't work. Not sure how...
Did you mean Mock library?
from mock import Mock
real = ProductionClass()
real.method = Mock(return_value=3)
real.method(3, 4, 5, key='value')
edit:
You are trying to access A.f._v before mocking which is impossible.
Not sure what are you trying to do, but this will work
>>>A.f = Mock()
>>>a = A()
>>>a.f._v
<Mock name='mock._v' id='42076240'>
The class definition shows an instance variable to set it from outside this class, do something like this:
class A:
def f(self):
self._v = 1
a = A()
a._v = Mock()
If you actually wanted a real class variable, try this:
class A():
_v = None
def f(self):
self.__class__._v = 1
A._v = Mock()
I tried above solutions and it still does not solve my purpose which is exactly what is asked originally. The above approach would update the mocked attribute of my class to have a value of .
My requirement is to set the attribute value from the mocked value I provide in my unit test class.
I could finally resolved this with the help of following approach. Let me know if its not a correct way:
Actual Class:
class ActualClass(object):
name=''
def some_method(self):
name=get_name_from_external_source() #Say, returned name='ActualValue'
print name
Unit Test Class:
from mock import PropertyMock
import unittest
class TestActualClass(unittest.TestCase):
def test_some_method(self):
actual_class=ActualClass()
p=PropertyMock(return_value='Mocked_Name')
type(actual_class).name=p
actual_class.some_method()
When you run some_method in ActualClass through normal execution, the output:
ActualValue
When you run TestActualClass, the output:
Mocked_Name
This implies that class attributes are mocked with a mocked value using PropertyType and you can test the method with mocked value and without worrying about external source method call.