We Keep Coding

Mock object with pyTest

I have a function I need to test:
def get_user_by_username(db, username: str) -> Waiter:
"""Get user object based on given username."""
user = db.query(Waiter).filter(Waiter.username == username).first()
return user
Here I try to mock DB call and return correct Waiter object, so my test is:
def test_get_user_by_username(mocker):
waiter = Waiter(id=10, username="string", password="1111")
db_mock = mocker.Mock(return_value=waiter)
result = get_user_by_username(db=db_mock, username="string")
assert result.json()["username"] == "string"
But I get an error TypeError: 'Mock' object is not subscriptable. How I can do it?
According to #Don Kirkby answer, the right solution is:
def test_get_user_by_username(mocker):
waiter = Waiter(id=10, username="string", password="1111")
filter_method = mocker.Mock()
filter_method.first = mocker.Mock(return_value=waiter)
query = mocker.Mock()
query.filter = mocker.Mock(return_value=filter_method)
db_mock = mocker.Mock()
db_mock.query = mocker.Mock(return_value=query)
db_mock.query.filter.first = mocker.Mock(return_value=waiter)
result = get_user_by_username(db=db_mock, username="string")
assert vars(result)["username"] == "string"

You're calling several methods, so you need to mock all of them:
query()
filter()
first()
Here's my best guess at how to do that with your code:
def test_get_user_by_username():
waiter = Waiter(id=10, username="string", password="1111")
filter_method = mocker.Mock()
filter_method.first = mocker.Mock(return_value=waiter)
query = mocker.Mock()
query.filter = mocker.Mock(return_value=filter_method)
db_mock = mocker.Mock()
db_mock.query = mocker.Mock(return_value=query)
db_mock.query.filter.first = mocker.Mock(return_value=waiter)
result = get_user_by_username(db=db_mock, username="string")
assert result.json()["username"] == "string"
I'm not sure what database library you're using, but I recommend looking for a library that provides mock versions of the database methods. For example, I contributed a few features to django-mock-queries that lets you mock out the Django ORM. Either that, or separate the logic you want to test from the database access code.

Not sure why MyMock.env["key1"].search.side_effect=["a", "b"] works but MyMock.env["key1"] = ["a"] with MyMock.env["key2"] = ["b"] does not work

I had created a simple example to illustrate my issue. First is the setup say mydummy.py:
class TstObj:
def __init__(self, name):
self.name = name
def search(self):
return self.name
MyData = {}
MyData["object1"] = TstObj("object1")
MyData["object2"] = TstObj("object2")
MyData["object3"] = TstObj("object3")
def getObject1Data():
return MyData["object1"].search()
def getObject2Data():
return MyData["object2"].search()
def getObject3Data():
return MyData["object3"].search()
def getExample():
res = f"{getObject1Data()}{getObject2Data()}{getObject3Data()}"
return res
Here is the test that failed.
def test_get_dummy1():
dummy.MyData = MagicMock()
mydummy.MyData["object1"].search.side_effect = ["obj1"]
mydummy.MyData["object2"].search.side_effect = ["obj2"]
mydummy.MyData["object3"].search.side_effect = ["obj3"]
assert mydummy.getExample() == "obj1obj2obj3"
The above failed with run time error:
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py:1078: StopIteration
Here is the test that passed:
def test_get_dummy2():
dummy.MyData = MagicMock()
mydummy.MyData["object1"].search.side_effect = ["obj1", "obj2", "obj3"]
assert mydummy.getExample() == "obj1obj2obj3"
Am I missing something? I would have expected test_get_dummy1() to work and test_get_dummy2() to fail and not vice versa. Where and how can I find/learn more information about mocking to explain what is going on...

MyData["object1"] is converted to this function call: MyData.__getitem__("object1"). When you call your getExample method, the __getitem__ method is called 3 times with 3 parameters ("object1", "object2", "object3").
To mock the behavior you could have written your test like so:
def test_get_dummy_alternative():
mydummy.MyData = MagicMock()
mydummy.MyData.__getitem__.return_value.search.side_effect = ["obj1", "obj2", "obj3"]
assert mydummy.getExample() == "obj1obj2obj3"
Note the small change from your version: mydummy.MyData["object1"]... became: mydummy.MyData.__getitem__.return_value.... This is the regular MagicMock syntax - we want to to change the return value of the __getitem__ method.
BONUS:
I often struggle with mock syntax and understanding what's happening under the hood. This is why I wrote a helper library: the pytest-mock-generator. It can show you the actual calls made to the mock object.
To use it in your case you could have added this "exploration test":
def test_get_dummy_explore(mg):
mydummy.MyData = MagicMock()
mydummy.getExample()
mg.generate_asserts(mydummy.MyData, name='mydummy.MyData')
When you execute this test, the following output is printed to the console, which contains all the asserts to the actual calls to the mock:
from mock import call
mydummy.MyData.__getitem__.assert_has_calls(calls=[call('object1'),call('object2'),call('object3'),])
mydummy.MyData.__getitem__.return_value.search.assert_has_calls(calls=[call(),call(),call(),])
mydummy.MyData.__getitem__.return_value.search.return_value.__str__.assert_has_calls(calls=[call(),call(),call(),])
You can easily derive from here what has to be mocked.

How to mock Google Cloud Storage Client which is passed as an argument o method of a class?

I'm having a trouble while mocking Google Cloud Storage Client in Python.
I've tried a couple solutions, but none of them seems to be working in my case. This is what I've tried:
Unit testing Mock GCS
Python unittest.mock google storage - how to achieve exceptions.NotFound as side effect
How to Mock a Google API Library with Python 3.7 for Unit Testing
So basically I have a class for downloading the decoded GCS object.
This is the class that I want to test:
class Extract:
def __init__(self, storage_client: storage.Client) -> None:
self.client = storage_client
#retry.Retry(initial=3.0, maximum=60.0, deadline=240.0)
def _get_storage_object(self, bucket: str, blob: str) -> bytes:
bucket_obj = self.client.get_bucket(bucket)
blob_obj = bucket_obj.blob(blob)
data = blob_obj.download_as_bytes(timeout=60)
return data
def get_decoded_blob(self, bucket: str, blob: str) -> Union[dict, list]:
raw_blob = self._get_storage_object(bucket=bucket, blob=blob)
decoded_blob = decode_data(raw_blob)
return decoded_blob
And this last version of code for unit testing of the class.
def decode_data(data: bytes) -> List[Dict[str, Any]]:
decoded_blob = data.decode('utf-8')
decoded_data = ast.literal_eval(decoded_blob)
return decoded_data
class TestExtract:
#pytest.fixture()
#mock.patch('src.pipeline.extract.storage', spec=True)
def storage_client(self, mock_storage):
mock_storage_client = mock_storage.Client.return_value
mock_bucket = mock.Mock()
mock_bucket.blob.return_value.download_as_bytes.return_value = 'data'.encode('utf-8')
mock_storage_client.bucket.return_value = mock_bucket
return mock_storage_client
def test_storage_client(self, storage_client):
gcs = extract.Extract(storage_client)
output = gcs._get_storage_object('bucket', 'blob')
assert output == b'data'
The error for the above code is that mock does not return return_value which was assigned in the fixture. Instead, it returns the MagicMock object as below:
AssertionError: assert <MagicMock na...806039010368'> == b'data'
Anyone can help me in this case?

Mocking and assert_called_once_with()

I want to assert_called_once_with for the mocked object, here is what I do:
log_service.py
def insert_log(self, log):
session = boto3.Session(profile_name='test')
dynamodb = session.resource('dynamodb')
table = dynanodb.Table('log_info')
table.put_item(Item=log.__dict__)
test_log_service.py
class Test(Unittest.TestCase):
def setUp(self):
self.log_service = LogService()
#patch("boto3.Session")
def test_insert_log(self, mock_session):
log = LogModel()
mock_dynamodb = MagicMock()
mock_table = MagicMock()
mock_session.resource = mock_dynamodb
mock_dynamodb.Table = mock_table
self.log_service.insert_log(log)
mock_table.put_item.assert_called_once_with(Item=log)
I expected that it should true, however, it always return:
AssertionError: Expected 'put_item' to be called once. Called 0 times.
Could someone help me to point out the problem?

How to write pytest for boto3 lambda invoke when it is defined inside a function

I am trying to write pytest to test the following method by mocking the boto3 client. I tried with sample test case. I am not sure if that is right way to do it. Please correct me if I am wrong.
//temp.py
import boto3
import json
def temp_lambda(event):
client_lam = boto3.client('lambda', region_name="eu-west-1") #defined inside the function.
obj = client_lam.invoke(
FunctionName='XYZ',
InvocationType='ABC',
Payload=json.dumps({'payload': event}))
return obj
//test_temp.py
import mock
from unittest.mock import MagicMock, patch
from .temp import temp_lambda
#mock.patch("boto3.client")
def test_temp_lambda(mock_lambda_client):
mocked_response = MagicMock(return_value = 'yes')
mock_lambda_client.invoke.return_value = mocked_response.return_value
event = {}
x = temp_lambda(event)
assert x == 'yes'
I am getting assertion error in output
AssertionError: assert <MagicMock name='client().invoke()' id='2557742644480'> == 'yes'

def test_temp_lambda(context):
with patch('boto3.client') as mocked_response:
mocked= MagicMock()
mocked.invoke.return_value= "ok"
mocked_response.return_value=mocked
event = {}
x = temp_lambda(event)
assert response=='ok'

The golden rule of the mock framework is that you mock where the object is used not where it's defined.
So it would be #mock.patch('temp.boto3.client')