I am trying to figure out how to know if a method of class is being called inside a method.
following is the code for the unit test:
# test_unittes.py file
def test_purge_s3_files(mocker):
args = Args()
mock_s3fs = mocker.patch('s3fs.S3FileSystem')
segment_obj = segments.Segmentation()
segment_obj.purge_s3_files('sample')
mock_s3fs.bulk_delete.assert_called()
inside the purge_s3_file method bulk_delete is called but when asserting it says that the method was expected to be called and it is not called!
mocker = <pytest_mock.plugin.MockerFixture object at 0x7fac28d57208>
def test_purge_s3_files(mocker):
args = Args()
mock_s3fs = mocker.patch('s3fs.S3FileSystem')
segment_obj = segments.Segmentation(environment='qa',
verbose=True,
args=args)
segment_obj.purge_s3_files('sample')
> mock_s3fs.bulk_delete.assert_called()
E AssertionError: Expected 'bulk_delete' to have been called.
I don't know how to test this and how to assert if the method is called!
Below you can find the method being testing:
# segments.py file
import s3fs
def purge_s3_files(self, prefix=None):
bucket = 'sample_bucket'
files = []
fs = s3fs.S3FileSystem()
if fs.exists(f'{bucket}/{prefix}'):
files.extend(fs.ls(f'{bucket}/{prefix}'))
else:
print(f'Directory {bucket}/{prefix} does not exist in s3.')
print(f'Purging S3 files from {bucket}/{prefix}.')
print(*files, sep='\n')
fs.bulk_delete(files)
The problem you are facing is that the mock you are setting up is mocking out the class, and you are not using the instance to use and check your mocks. In short, this should fix your problem (there might be another issue explained further below):
m = mocker.patch('s3fs.S3FileSystem')
mock_s3fs = m.return_value # (or mock_s3())
There might be a second problem in how you are not referencing the right path to what you want to mock.
Depending on what your project root is considered (considering your comment here) your mock would need to be referenced accordingly:
mock('app.segments.s3fs.S3FileSystem')
The rule of thumb is that you always want to mock where you are testing.
If you are able to use your debugger (or output to your console) you will (hopefully :)) see that your expected call count will be inside the return_value of your mock object. Here is a snippet from my debugger using your code:
You will see the call_count attribute set to 1. Pointing back to what I mentioned at the beginning of the answer, by making that change, you will now be able to use the intended mock_s3fs.bulk_delete_assert_called().
Putting it together, your working test with modification runs as expected (note, you should also set up the expected behaviour and assert the other fs methods you are calling in there):
def test_purge_s3_files(mocker):
m = mocker.patch("app.segments.s3fs.S3FileSystem")
mock_s3fs = m.return_value # (or m())
segment_obj = segments.Segmentation(environment='qa',
verbose=True,
args=args)
segment_obj.purge_s3_files('sample')
mock_s3fs.bulk_delete.assert_called()
Python mock testing depends on where the mock is being used. So you have the mock the function calls where it is imported.
Eg.
app/r_executor.py
def r_execute(file):
# do something
But the actual function call happens in another namespace ->
analyse/news.py
from app.r_executor import r_execute
def analyse():
r_execute(file)
To mock this I should use
mocker.patch('analyse.news.r_execute')
# not mocker.patch('app.r_executor.r_execute')
Related
I have a method say _select_warehouse_for_order in api/controllers/orders.py file. The method is not part of any class.
Now, I have a new file say api/controllers/dispatchers.py where i need to know which warehouse was selected. I am calling _select_warehouse_for_order from this file to get this information.
Now, in my test cases, I am patching _select_warehouse_for_order like this
from unittest.mock import patch, call
def test_delivery_assignment(self, app_context):
with patch('api.controllers.orders._select_warehouse_for_order') as mock_selected_wh:
mock_selected_wh.return_value = {}
app_context.client.set_url_prefix('/v2')
response = app_context.client.get('/delivery/dispatch')
assert response.status_code == 200
The problem that i am facing is that my patch is not returning empty dictionary. when i started debugging, i noticed that its executed the actual code in _select_warehouse_for_order. Am i missing something here?
Update:
Here is the code in dispatchers.py
from api.controllers.orders import _select_warehouse_for_order
#bp.route("/dispatch")
#login_required
def dispatch():
warehouses = _select_warehouse_for_order(request=request)
if len(warehouses) == 0:
logger.info("No warehouse selected")
return
logger.info("Selected warehouse: %s", warehouses[0].name)
# return response
You must patch where the method is used, not where it is declared. In your case, you are patching 'api.controllers.orders._select_warehouse_for_order' which is where the method is declared. Instead, patch 'dispatchers._select_warehouse_for_order' (possibly prefixed with whatever package contains dispatchers).
The reason for this is because when you do
from api.controllers.orders import _select_warehouse_for_order
you declare a name _select_warehouse_for_order in dispatchers.py that refers to the function which is declared in api/controllers/orders.py. Essentially you have created a second reference to the function. Now when you call
warehouses = _select_warehouse_for_order(request=request)
you are using the reference in dispatchers.py, not the one in api/controllers/orders.py. So in order to replace this function with a patch, you have to use dispatchers._select_warehouse_for_order.
Notice how import is different in python than in Java because we create a new name and assign it to an existing function or class. On the other hand, Java imports tell the compiler where to look for a class when it is mentioned in the code.
I have this python lambda on handler.py
def isolate_endpoints(event=None, context=None):
endpoint_id = event['event']['endpoint_id']
client = get_edr_client()
response = client.isolate_endpoints(endpoint_id=endpoint_id)
return response # E.g {"reply":{"status":"success", "error_msg":null}}
I want to write a unit test for this lambda. However after reading on the unittests and having seen actual implementations of the tests I can comfortably say I have no idea what is being tested exactly. (I know the theory but having hard time understanding the implementation of mocks, magicmocks etc.)
The unittest that I have right now is on test_calls and looks like this:
#mock.patch('project_module.utils.get_edr_client')
def test_isolate_endpoints(get_edr_client: MagicMock):
client = MagicMock()
get_edr_client.return_value = client
mock_event = {"event": {"endpoint_id":"foo"}}
resp = isolate_endpoints(event=mock_event) # Is it right to call this lambda directly from here?
assert resp is not None
expected = [call()] ## what is this ?? # What is supposed to follow this?
assert client.isolate_endpoints.call_args_list == expected
definition & body of get_edr_client in utils.py:
from EDRAPI import EDRClient
def get_edr_client():
return EDRClient(api_key="API_KEY")
I'll try to explain each aspect of the test you have written
#mock.patch('project_module.utils.get_edr_client')
This injects dependencies into your test. You are essentially patching the name 'project_module.utils.get_edr_client' with an auto-created MagicMock object that's passed to you as the test argument get_edr_client. This is useful to bypass external dependencies in your test code. You can pass mock implementations of objects that are used in your code that's under test, but don't need to be tested in this test itself.
def test_isolate_endpoints(get_edr_client: MagicMock):
client = MagicMock()
get_edr_client.return_value = client
Setup: You are setting up the external mock you patched into the test. Making it return more mock objects so that the code under test works as expected.
mock_event = {"event": {"endpoint_id":"foo"}}
resp = isolate_endpoints(event=mock_event)
Invoke: Calling the code that you want to test, with some (possibly mock) argument. Since you want to test the function isolate_endpoints, that's what you should be calling. In other tests you could also try passing invalid arguments like isolate_endpoints(event=None) to see how your code behaves in those scenarios.
assert resp is not None
Verify: Check if the value returned from your function is what you expect it to be. If your function was doing 2+2 this is the part where you check if the result equals 4 or not.
expected = [call()]
assert client.isolate_endpoints.call_args_list == expected
Verify side-effects: Your function has side-effects apart from returning a value. So you should check if those are happening properly or not. You expect the function to call isolate_endpoint on the client mock you injected earlier once. So you're checking if that method was called once with no arguments. The call() is for matching one invocation of the method with no arguments. The [call()] means only one invocation with no arguments. Then you just check if that is indeed the case with the assertion.
A much cleaner way to do the same this is to do this instead:
client.isolate_endpoints.assert_called_once()
Which does the same thing.
I'm currently using nose to perform some tests, and when using generators with nose+xunit output you need to set the current function's __name__ attribute to properly control the name of the test in the xunit output (see here for example).
Since I don't want to hard-code the name of the function each time like this:
def my_function():
for foo in bar:
fn = lambda: some_generated_test(foo)
fn.description = foo.get('name')
my_function.__name__ = foo.get('name')
yield fn
How can I programatically reference the function and set __name__?
I had tried with sys._getframe() which yields various properties about the current function (name etc), which I tried to use with setattr(*something*, "__name__", some_test_name), but that didn't work as I couldn't seem to work out which part of sys._getframe() references the function.
Finally found a solution via SO: https://stackoverflow.com/a/4506081/1808861
A lot more complicated than I expected, but I can now:
def my_function():
for foo in bar:
fn = lambda: some_generated_test(foo)
fn.description = foo.get('name')
setattr(get_func(), "__name__", foo.get('name'))
yield fn
The xunit output then contains the generator's data name entry.
I have a class with some #staticmethod's that are procedures, thus they do not return anything / their return type is None.
If they fail during their execution, they throw an Exception.
I want to unittest this class, but I am struggling with designing positive tests.
For negative tests this task is easy:
assertRaises(ValueError, my_static_method(*args))
assertRaises(MyCustomException, my_static_method(*args))
...but how do I create positive tests? Should I redesign my procedures to always return True after execution, so that I can use assertTrue on them?
Without seeing the actual code it is hard to guess, however I will make some assumptions:
The logic in the static methods is deterministic.
After doing some calculation on the input value there is a result
and some operation is done with this result.
python3.4 (mock has evolved and moved over the last few versions)
In order to test code one has to check that at least in the end it produces the expected results. If there is no return value then the result is usually stored or send somewhere. In this case we can check that the method that stores or sends the result is called with the expected arguments.
This can be done with the tools available in the mock package that has become part of the unittest package.
e.g. the following static method in my_package/my_module.py:
import uuid
class MyClass:
#staticmethod
def my_procedure(value):
if isinstance(value, str):
prefix = 'string'
else:
prefix = 'other'
with open('/tmp/%s_%s' % (prefix, uuid.uuid4()), 'w') as f:
f.write(value)
In the unit test I will check the following:
open has been called.
The expected file name has been calculated.
openhas been called in write mode.
The write() method of the file handle has been called with the expected argument.
Unittest:
import unittest
from unittest.mock import patch
from my_package.my_module import MyClass
class MyClassTest(unittest.TestCase):
#patch('my_package.my_module.open', create=True)
def test_my_procedure(self, open_mock):
write_mock = open_mock.return_value.write
MyClass.my_procedure('test')
self.assertTrue(open_mock.call_count, 1)
file_name, mode = open_mock.call_args[0]
self.assertTrue(file_name.startswith('/tmp/string_'))
self.assertEqual(mode, 'w')
self.assertTrue(write_mock.called_once_with('test'))
If your methods do something, then I'm sure there should be a logic there. Let's consider this dummy example:
cool = None
def my_static_method(something):
try:
cool = int(something)
except ValueError:
# logs here
for negative test we have:
assertRaises(ValueError, my_static_method(*args))
and for possitive test we can check cool:
assertIsNotNone(cool)
So you're checking if invoking my_static_method affects on cool.
I'm new to Python and I'm trying to mock a function only when a specific argument is passed. If other than the desired argument is passed, I'd like to call the original function instead.
In Python 2.7 I tried something like this:
from foo import config
def test_something(self):
original_config = config # config is a Module.
def side_effect(key):
if key == 'expected_argument':
return mocked_result
else:
return original_config.get(key)
config.get = Mock(side_effect=side_effect)
# actualy_test_something...
It won't work 'cause original_config is not a copy of config. It references the same module ending up in an infinite loop. I could try cloning the original config module instead but that seems to be overkill.
Is there something similar to RSpec's mocks I could use? e.g:
obj.stub(:message).with('an_expected_argument').and_return('a_mocked_result')
Any help would be appreciated. Thanks.
You'd need to store a reference to the unpatched function first:
def test_something(self):
original_config_get = config.get
def side_effect(key):
if key == 'expected_argument':
return mocked_result
else:
return original_config_get(key)
config.get = Mock(side_effect=side_effect)
Here original_config_get references the original function before you replaced it with a Mock() object.