We Keep Coding

Failed to patch object for unit testing

I have a class used to authenticate with google sheet API and retrieve data from some spreadsheets.
Here a part of it:
spreadsheet.py
from typing import Optional
from google.oauth2.credentials import Credentials
from google.auth.transport.requests import Request
import gspread
class GoogleSheet:
def __init__(self, token_file: str):
self.token_file: str = token_file
self.google_client: Optional[gspread.Client] = None
self.gsheet: Optional[gspread.Spreadsheet] = None
def setup_connection(self) -> None:
credentials: Credentials = Credentials.from_authorized_user_file(self.token_file)
if credentials.expired:
credentials.refresh(Request())
self.google_client = gspread.authorize(credentials)
def open_gsheet_by_url(self, gsheet_url: str) -> None:
self.gsheet = self.google_client.open_by_url(gsheet_url)
I wanted to create some tests for the previous code.
Here is what I ended to:
test_spreadsheet.py
import pytest
from spreadsheet import GoogleSheet
from unittest.mock import patch
class TestSpreadSheetData:
#patch('spreadsheet.Credentials')
#patch('spreadsheet.gspread')
def test_setup_connection(self, mocked_creds, mocked_auth):
google_sheet = GoogleSheet('api_token.json')
google_sheet.setup_connection()
assert mocked_creds.from_authorized_user_file.call_count == 1
assert mocked_auth.authorize.call_count == 1
I tried the code above but it didn't work although I had similar approach with different project and packages.
So whenever I run the test, I get the following error:
AssertionError
Assert 0 == 1
Can anyone helps me fix this?

Mock database connection python

I have a file called redis_db.py which has code to connect to redis
import os
import redis
import sys
class Database:
def __init__(self, zset_name):
redis_host = os.environ.get('REDIS_HOST', '127.0.0.1')
redis_port = os.environ.get('REDIS_PORT', 6379)
self.db = redis.StrictRedis(host=redis_host, port=redis_port)
self.zset_name = zset_name
def add(self, key):
try:
self.db.zadd(self.zset_name, {key: 0})
except redis.exceptions.ConnectionError:
print("Unable to connect to redis host.")
sys.exit(0)
I have another file called app.py which is like this
from flask import Flask
from redis_db import Database
app = Flask(__name__)
db = Database('zset')
#app.route('/add_word/word=<word>')
def add_word(word):
db.add(word)
return ("{} added".format(word))
if __name__ == '__main__':
app.run(host='0.0.0.0', port='8080')
Now I am writing unit test for add_word function like this
import unittest
import sys
import os
from unittest import mock
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/../api/")
from api import app # noqa: E402
class Testing(unittest.TestCase):
def test_add_word(self):
with mock.patch('app.Database') as mockdb:
mockdb.return_value.add.return_value = ""
result = app.add_word('shivam')
self.assertEqual(result, 'shivam word added.')
Issue I am facing is that even though I am mocking the db method call it is still calling the actual method in the class instead of returning mocked values and during testing I am getting error with message Unable to connect to redis host..
Can someone please help me in figuring out how can I mock the redis database calls.
I am using unittest module

The issue is that db is defined on module import, so the mock.patch does not affect the db variable. Either you move the instantiation of
db in the add_word(word) function or you patch db instead of Database, e.g.
def test_add_word():
with mock.patch('api.app.db') as mockdb:
mockdb.add = mock.MagicMock(return_value="your desired return value")
result = app.add_word('shivam')
print(result)
Note that the call of add_word has to be in the with block, otherwise the unmocked version is used.

Flask unit tests -- mocking up Aerospike DB

I have the following endpoint,
#developer_blueprint.route("/init_db", methods=["POST"])
def initialize_database():
try:
upload_data(current_app)
logger.debug("Database entries upload.")
return jsonify({"result": "Database entries uploaded."}), 201
except Exception as e:
return jsonify({"error": str(e)})
def upload_data(app):
with open("src/core/data/data.json") as data_file:
data = json.load(data_file)
try:
current_app.db.put(("somenamespace", "test", "default"), data, None)
except Exception as e:
raise e
I'm trying to figure out how to unit test this (we need to get coverage on our code).
Do I just mock up app.db? How can I do that?
Any suggestions would be appreciated.

It is not uncommon to mock database calls for unit testing using something like unittest.mock and then run Aerospike in a container or VM for end-to-end testing.
However, keep in mind that the Aerospike Python client library is written in C for better performance and thus it is not easy to do partial patching (aka "monkey patching"). For example, you will get a TypeError: can't set attributes of built-in/extension type if you try to simply patch out aerospike.Client.put.
One approach is to create a mock client object to replace or sub-class the Aerospike client object. The implementation of this mock object depends on your code and the cases you are testing for.
Take the following example code in which app.db is an instance of the Aerospike client library:
# example.py
import aerospike
import json
class App(object):
db = None
def __init__(self):
config = {'hosts': [('127.0.0.1', 3000)]}
self.db = aerospike.client(config).connect()
def upload_data(app):
with open("data.json") as data_file:
data = json.load(data_file)
try:
app.db.put(("ns1", "test", "default"), data, None)
except Exception as e:
raise e
if __name__ == "__main__":
app = App()
upload_data(app)
In writing unit tests for the upload_data function let's assume you want to test for a success case which is determined to mean that the put method is called and no exceptions are raised:
# test.py
from unittest import TestCase, main
from unittest.mock import PropertyMock, patch
from example import App, upload_data
from aerospike import Client, exception
class MockClient(Client):
def __init__(self, *args, **kwargs):
pass
def put(self, *args, **kwargs):
return 0
class ExampleTestCase(TestCase):
def test_upload_data_success(self):
with patch.object(App, 'db', new_callable=PropertyMock) as db_mock:
db_mock.return_value = client = MockClient()
app = App()
with patch.object(client, 'put') as put_mock:
upload_data(app)
put_mock.assert_called()
if __name__ == '__main__':
main()
In the test_upload_data_success method the App.db property is patched with the MockClient class instead of the aerospike.Client class. The put method of the MockClient instance is also patched so that it can be asserted that the put method gets called after upload_data is called.
To test that an exception raised by the Aerospike client is re-raised from the upload_data function, the MockClient class can be modified to raise an exception explicitly:
# test.py
from unittest import TestCase, main
from unittest.mock import PropertyMock, patch
from example import App, upload_data
from aerospike import Client, exception
class MockClient(Client):
def __init__(self, *args, **kwargs):
self.put_err = None
if 'put_err' in kwargs:
self.put_err = kwargs['put_err']
def put(self, *args, **kwargs):
if self.put_err:
raise self.put_err
else:
return 0
class ExampleTestCase(TestCase):
def test_upload_data_success(self):
with patch.object(App, 'db', new_callable=PropertyMock) as db_mock:
db_mock.return_value = client = MockClient()
app = App()
with patch.object(client, 'put') as put_mock:
upload_data(app)
put_mock.assert_called()
def test_upload_data_error(self):
with patch.object(App, 'db', new_callable=PropertyMock) as db_mock:
db_mock.return_value = MockClient(put_err=exception.AerospikeError)
app = App()
with self.assertRaises(exception.AerospikeError):
upload_data(app)
if __name__ == '__main__':
main()

Python decorator staticmethod object not callable

Hi I'm trying to create a decorator but I'm getting an error of staticmethod object not callable below is my code
from db.persistence import S3Mediator
from sqlalchemy.ext.declarative import declarative_base
import logging
from functools import wraps
Base = declarative_base()
def s3(func):
#wraps(func)
def wrapper(*args, **kwargs):
try:
s3client = S3Mediator.get_s3_connection()
kwargs["s3client"] = s3client
retval = func(*args, **kwargs) #### an error is raised here
except Exception as e:
raise e
return retval
return wrapper
And here is the mediator that instantiate the s3 object
import boto3
import logging
class S3Mediator(object):
s3_client = None
def __init__(self, host, access_key, secret):
self.client = boto3.client(
's3',
aws_access_key_id= access_key,
aws_secret_access_key= secret
)
S3Mediator.s3_client = self.client
#staticmethod
def get_s3_connection():
return S3Mediator.s3_client
Now S3Mediator is already instantiate at the app.py now I'm trying to use this decorator as
#s3
#staticmethod
def s3_connect(s3client):
# code don't reach here. An error is thrown
# do something here
Any idea why its returning a staticmethod object not callable and how to fix this

Ok found the cause of the problem. I put the #staticmethod below my decorator that's why my decorator is thinking that all method of the decorator is static. I just change the
#s3
#staticmethod
def s3_connect(s3client):
# code don't reach here. An error is thrown
# do something here
to this
#staticmethod
#s3
def s3_connect(s3client):
# code don't reach here. An error is thrown
# do something here

mock patch fails for some reason

I'm trying to patch an API for unit testing purposes. I have a class like this:
# project/lib/twilioclient.py
from os import environ
from django.conf import settings
from twilio.rest import TwilioRestClient
def __init__(self):
return super(Client, self).__init__(
settings.TWILIO_ACCOUNT_SID,
settings.TWILIO_AUTH_TOKEN,
)
client = Client()
and this method:
# project/apps/phone/models.py
from project.lib.twilioclient import Client
# snip imports
class ClientCall(models.Model):
'call from Twilio'
# snip model definition
def dequeue(self, url, method='GET'):
'''"dequeue" this call (take off hold)'''
site = Site.objects.get_current()
Client().calls.route(
sid=self.sid,
method=method,
url=urljoin('http://' + site.domain, url)
)
and then the following test:
# project/apps/phone/tests/models_tests.py
class ClientCallTests(BaseTest):
#patch('project.apps.phone.models.Client', autospec=True)
def test_dequeued(self, mock_client):
cc = ClientCall()
cc.dequeue('test', 'TEST')
mock_client.calls.route.assert_called_with(
sid=cc.sid,
method='TEST',
url='http://example.com/test',
)
When I run the test, it fails because the real client is called instead of the mock client.
Looks like it should work to me, and I've been able to make this type of thing work before. I've tried the usual suspects (removing *.pyc) and things don't seem to be changing. Is there a new behavior here that I'm missing somehow?