We Keep Coding

how to write pytest for pymssql?

I am trying to write pytest to test the following method by mocking the database. How to mock the database connection without actually connecting to the real database server.I tried with sample test case. I am not sure if that is right way to do it. Please correct me if I am wrong.
//fetch.py
import pymssql
def cur_fetch(query):
with pymssql.connect('host', 'username', 'password') as conn:
with conn.cursor(as_dict=True) as cursor:
cursor.execute(query)
response = cursor.fetchall()
return response
//test_fetch.py
import mock
from unittest.mock import MagicMock, patch
from .fetch import cur_fetch
def test_cur_fetch():
with patch('fetch.pymssql', autospec=True) as mock_pymssql:
mock_cursor = mock.MagicMock()
test_data = [{'password': 'secret', 'id': 1}]
mock_cursor.fetchall.return_value = MagicMock(return_value=test_data)
mock_pymssql.connect.return_value.cursor.return_value.__enter__.return_value = mock_cursor
x = cur_fetch({})
assert x == None
The result is:
AssertionError: assert <MagicMock name='pymssql.connect().__enter__().cursor().__enter__().fetchall()' id='2250972950288'> == None
Please help.

Trying to mock out a module is difficult. Mocking a method call is simple. Rewrite your test like this:
import unittest.mock as mock
import fetch
def test_cur_tech():
with mock.patch('fetch.pymssql.connect') as mock_connect:
mock_conn = mock.MagicMock()
mock_cursor = mock.MagicMock()
mock_connect.return_value.__enter__.return_value = mock_conn
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_cursor.fetchall.return_value = [{}]
res = fetch.cur_fetch('select * from table')
assert mock_cursor.execute.call_args.args[0] == 'select * from table'
assert res == [{}]
In the above code we're explicitly mocking pymyssql.connect, and
providing appropriate fake context managers to make the code in
cur_fetch happy.
You could simplify things a bit if cur_fetch received the connection
as an argument, rather than calling pymssql.connect itself.

How to mock failed operations with moto, #mock_dynamodb2?

I am currently trying to write unit tests for my python code using Moto & #mock_dynamodb2 . So far it's been working for me to test my "successful operation" test cases. But I'm having trouble getting it to work for my "failure cases".
In my test code I have:
#mock_dynamodb2
class TestClassUnderTestExample(unittest.TestCase):
def setUp(self):
ddb = boto3.resource("dynamodb", "us-east-1")
self.table = ddb.create_table(<the table definition)
self.example_under_test = ClassUnderTestExample(ddb)
def test_some_thing_success(self):
expected_response = {<some value>}
assert expected_response = self.example_under_test.write_entry(<some value>)
def test_some_thing_success(self):
response = self.example_under_test.write_entry(<some value>)
# How to assert exception is thrown by forcing put item to fail?
The TestClassUnderTestExample would look something like this:
class ClassUnderTestExample:
def __init__(self, ddb_resource=None):
if not ddb_resource:
ddb_resource = boto3.resource('dynamodb')
self.table = ddb_resource.Table(.....)
def write_entry(some_value)
ddb_item = <do stuff with some_value to create sanitized item>
response = self.table.put_item(
Item=ddb_item
)
if pydash.get(response, "ResponseMetadata.HTTPStatusCode") != 200:
raise SomeCustomErrorType("Unexpected response from DynamoDB when attempting to PutItem")
return ddb_item
I've been completely stuck when it comes to actually mocking the .put_item operation to return a non-success value so that I can test that the ClassUnderTestExample will handle it as expected and throw the custom error. I've tried things like deleting the table before running the test, but that just throws an exception when getting the table rather than an executed PutItem with an error code.
I've also tried putting a patch for pydash or for the table above the test but I must be doing something wrong. I can't find anything in moto's documentation. Any help would be appreciated!

The goal of Moto is to completely mimick AWS' behaviour, including how to behave when the user supplies erroneous inputs. In other words, a call to put_item() that fails against AWS, would/should also fail against Moto.
There is no build-in way to force an error response on a valid input.
It's difficult to tell from your example how this can be forced, but it looks like it's worth playing around with this line to create an invalid input:
ddb_item = <do stuff with some_value to create sanitized item>

Yes, you can. Use mocking for this. Simple and runnable example:
from unittest import TestCase
from unittest.mock import Mock
from uuid import uuid4
import boto3
from moto import mock_dynamodb2
def create_user_table(table_name: str) -> dict:
return dict(
TableName=table_name,
KeySchema=[
{
'AttributeName': 'id',
'KeyType': 'HASH'
},
],
AttributeDefinitions=[
{
'AttributeName': 'id',
'AttributeType': 'S'
},
],
BillingMode='PAY_PER_REQUEST'
)
class UserRepository:
table_name = 'users'
def __init__(self, ddb_resource):
if not ddb_resource:
ddb_resource = boto3.resource('dynamodb')
self.table = ddb_resource.Table(self.table_name)
def create_user(self, username):
return self.table.put_item(Item={'id': str(uuid4), 'username': username})
#mock_dynamodb2
class TestUserRepository(TestCase):
def setUp(self):
ddb = boto3.resource("dynamodb", "us-east-1")
self.table = ddb.create_table(**create_user_table('users'))
self.test_user_repo = UserRepository(ddb)
def tearDown(self):
self.table.delete()
def test_some_thing_success(self):
user = self.test_user_repo.create_user(username='John')
assert len(self.table.scan()['Items']) == 1
def test_some_thing_failure(self):
self.test_user_repo.table = table = Mock()
table.put_item.side_effect = Exception('Boto3 Exception')
with self.assertRaises(Exception) as exc:
self.test_user_repo.create_user(username='John')
self.assertTrue('Boto3 Exception' in exc.exception)

How to unit test Exception in APIRouter (not FastAPI)

I'm having difficulties doing the unit tests for the function get_data in main.py:
from fastapi import APIRouter
router = APIRouter()
#router.get("/{device_id}")
def get_data(request: Request, device_id: str, query: DataQuery = Depends(DataQuery.depends)):
data_service = DataApiService()
try:
datetime_start_timestamp, datetime_end_timestamp = data_service.validate_dates(query.start, query.end)
df_results = data_service.get_datalake_data(device_id, datetime_start_timestamp, datetime_end_timestamp)
return df_results
except EndDataLowerStartDate as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail='Internal Server Error')
This is the solution I have, test_main.py:
from main import router
client = TestClient(router)
def test_get_data_EndDataLowerStartDate():
mock.patch('main.DataApiService', side_effect = EndDataLowerStartDate)
response = client.get('/xxxxx', params = {'start': '2021-12-31', 'end': '2021-01-01'})
assert response.status_code == 400
assert response.json() == {'detail': 'End date {0} is lower or equal to start date {1}'.format(end, start)}
However, the test only works if I replace in main.py:
router = APIRouter() by app = FastAPI()
and in test_main.py:
client = TestClient(router) by client = TestClient(app).
Otherwise I get the error:
FAILED test_main.py::test_get_data_EndDataLowerStartDate - fastapi.exceptions.HTTPException
If I don't change anything, when I locally make the request, it returns:
AttributeError: 'FastAPI' object has no attribute
'default_response_class'
It is like the exception is raised before it is assert... I cannot change main.py because it is a bigger application inicially built this way. I'm sorry if I was unclear but I would like to know how to make the test pass. Do you have an idea of how I could solve this?

Mocking an API request for downstream module using Python

I am a beginner when it comes to writing tests and mocking.
I have a created two modules. One module object (Site) creates another object from my second module (Item) on init. The Item object makes a call to an API endpoint to get some data using requests.
I want to Mock the API call I am making so I can test things like a bad response and importantly have control over the response data.
I have simplified my code and put below. When I run the test I get back the actual response data and not what I have mocked.
I have a feeling I am not putting my Mock in the right place. Also, I have seen lots of people saying to use #unittest.patch annotation. I am not clear if I should be using that here.
So I am looking for how to get _get_range_detail to actually return a Mocked response from requests and also just general feedback on if it looks like I am approaching this the right way.
# hello_world.py
from mymodule.site import Site
sites = [
dict(
name="site1",
ranges=[
"range1",
"range2"
]
)
]
site_object = Site(sites[0]['name'], sites[0]['ranges'])
for i in site_object.get_ranges_objects():
print(i.range_detail)
# site.py
from mymodule.item import Item
class Site:
def __init__(self, name, ranges):
self.name = name
self.ranges = ranges
self.ranges_objects = []
for my_range in ranges:
self.ranges_objects.append(Item(my_range))
def get_ranges_objects(self):
return self.ranges_objects
# item.py
import requests
class Item:
def __init__(self, range_name):
self.range_name = range_name
self.range_detail = self._get_range_detail(self.range_name)
def _get_range_detail(self, range_name):
uri = "https://postman-echo.com/get?some_cool_value=real_value"
try:
r = requests.get(uri)
if r.status_code == 200:
return r.json()['args']
else:
return None
except Exception as e:
print(e)
exit(1)
# test_site.py
import pytest
from mymodule.site import Site
from unittest import mock
from mymodule.item import requests
def test_get_ranges_objects():
sites = [
dict(
name="site1",
ranges=[
"range1",
"range2"
]
)
]
requests = mock.Mock()
requests.status_code = 200
requests.json.return_value = {
'args': {'some_mock_value': 'mocky'}
}
site_object = Site(sites[0]['name'], sites[0]['ranges'])
assert site_object.name == "site1"
assert isinstance(site_object.ranges_objects, list)
assert site_object.ranges_objects[0].range_detail == dict(some_mock_value='mocky')

You can use pytest-mock. it makes mocking in pytest simple. (pip install pytest-mock)
You should replace requests.get. Simply
requests_get = mock.patch('requests.get').start()
If you use pytest-mock,
requests_get = mocker.patch('requests.get')
Rewrote test case using pytest-mock
# test_site.py
import pytest
from mymodule.site import Site
from unittest import mock
#pytest.fixture
def requests_get(mocker):
requests_get = mocker.patch('requests.get')
yield requests_get
def test_get_ranges_objects(mocker, requests_get):
response = mocker.MagicMock()
response.status_code = 200
response.json.return_value = {'args': {'some_mock_value': 'mocky'}}
requests_get.return_value = response
sites = [
dict(
name="site1",
ranges=[
"range1",
"range2"
]
)
]
site_object = Site(sites[0]['name'], sites[0]['ranges'])
assert site_object.name == "site1"
assert isinstance(site_object.ranges_objects, list)
assert site_object.ranges_objects[0].range_detail == {'some_mock_value': 'mocky'}

Trapping a custom error in python

I'm trying to trap the following error in a try/exception block, but as this is a custom module that is generating the error - not generating a standard error such as ValueError for example. What is the correct way to catch such errors?
Here is my code:
try:
obj = IPWhois(ip_address)
except Exception(IPDefinedError):
results = {}
else:
results = obj.lookup()
The most obvious way:
except IPDefinedError:
gives:
NameError: name 'IPDefinedError' is not defined
The error returned that I want to check for is:
ipwhois.exceptions.IPDefinedError
ipwhois.exceptions.IPDefinedError: IPv4 address '127.0.0.1' is already defined as 'Private-Use Networks' via 'RFC 1918'.

The issue here is the import!
I had the import as
from ipwhois import IPWhois
but I also needed
import ipwhois
So the following works:
try:
obj = IPWhois(ip_address)
except ipwhois.exceptions.IPDefinedError:
results = {}
else:
results = obj.lookup()

Here is a quick recap. Yes, the error in your question did look like it was likely related to an import issue (as per my comment ;) ).
from pprint import pprint as pp
class IPDefinedError(Exception):
"""Mock IPDefinedError implementation
"""
pass
class IPWhois(object):
"""Mock IPWhois implementation
"""
def __init__(self, ip_address):
if ip_address == "127.0.0.1":
raise IPDefinedError(
"IPv4 address '127.0.0.1' is already defined as 'Private-Use Networks' via 'RFC 1918'.")
self._ip_address = ip_address
def lookup(self):
return "RESULT"
def lookup(ip_address):
""" calculates IPWhois lookup result or None if unsuccessful
:param ip_address:
:return: IPWhois lookup result or None if unsuccessful
"""
result = None
try:
obj = IPWhois(ip_address)
result = obj.lookup()
except IPDefinedError as e:
msg = str(e)
print("Error received: {}".format(msg)) # do something with msg
return result
if __name__ == '__main__':
results = map(lookup, ["192.168.1.1", "127.0.0.1"])
pp(list(results)) # ['RESULT', None]