I have a class that I try to mock in tests. The class is located in server/cache.py and looks like:
class Storage(object):
def __init__(self, host, port):
# set up connection to a storage engine
def store_element(self, element, num_of_seconds):
# store something
def remove_element(self, element):
# remove something
This class is used in server/app.py similar to this one:
import cache
STORAGE = cache.Storage('host', 'port')
STORAGE.store_element(1, 5)
Now the problem arise when I try to mock it in the tests:
import unittest, mock
import server.app as application
class SomeTest(unittest.TestCase):
# part1
def setUp(self):
# part2
self.app = application.app.test_client()
This clearly does not work during the test, if I can't connect to a storage. So I have to mock it somehow by writing things in 'part1, part2'.
I tried to achieve it with
#mock.patch('server.app.cache') # part 1
mock.side_effect = ... # hoping to overwriting the init function to do nothing
But it still tries to connect to a real host. So how can I mock a full class here correctly? P.S. I reviewed many many questions which look similar to me, but in vain.
Related
I'm working on unit tests for a service I made that uses confluent-kafka. The goal is to test successful function calls, exception errors, etc. The problem I'm running into is since I'm instantiating the client in the constructor of my service the tests are failing since I'm unsure how to patch a constructor. My question is how do I mock my service in order to properly test its functionality.
Example_Service.py:
from confluent_kafka.schema_registry import SchemaRegistryClient
class ExampleService:
def __init__(self, config):
self.service = SchemaRegistryClient(config)
def get_schema(self):
return self.service.get_schema()
Example_Service_tests.py
from unittest import mock
#mock.patch.object(SchemaRegistryClient, "get_schema")
def test_get_schema_success(mock_client):
schema_Id = ExampleService.get_schema()
mock_service.assert_called()
The problem is that you aren't creating an instance of ExampleService; __init__ never gets called.
You can avoid patching anything by allowing your class to accept a client maker as an argument (which can default to SchemaRegistryClient:
class ExampleService:
def __init__(self, config, *, client_factory=SchemaRegistryClient):
self.service = client_factory(config)
...
Then in your test, you can simply pass an appropriate stub as an argument:
def test_get_schema_success():
mock_client = Mock()
schema_Id = ExampleService(some_config, client_factory=mock_client)
mock_client.assert_called()
Two ways
mock entire class using #mock.patch(SchemaRegistryClient) OR
replace #mock.patch.object(SchemaRegistryClient, "get_schema") with
#mock.patch.object(SchemaRegistryClient, "__init__")
#mock.patch.object(SchemaRegistryClient, "get_schema")
I have a following simplified code example:
from urllib.parse import urlparse, parse_qs
from selenium import webdriver
import pytest
#pytest.fixture(scope="module")
def driver():
options = webdriver.ChromeOptions()
_driver = webdriver.Chrome(options=options)
yield _driver
_driver.close()
#pytest.mark.parametrize("instance_name", ["instance1", "instance2"])
class TestInstance:
#pytest.fixture
def authorization_code(self, instance_name, driver):
driver.get(f"https://{instance_name}.com")
...do some UI actions here
authorization_code = parse_qs(urlparse(redirected_url).query)["code"][0]
#pytest.fixture
def access_token(self, authorization_code):
...obtain access_token here using authorization code
return "access_token"
def test_case_1(self, access_token):
...do some API calls using access_token
def test_case_2(self, access_token):
...do some API calls using access_token
What I would like to do is to execute UI actions in the authorization_code function once and obtain one access_token per instance.
Currently my UI actions are executed for every test case, leading to the fact that UI actions actually execute 2 * 2 = 4 times.
Is it possible to do with pytest?
Or maybe I am missing something in my setup?
In general I would just change the fixture scope: currently it gets recreated every time it is called, hence the reuse of ui actions. This is by design to ensure fixtures are clean. If your fixture didn't depend on the function-level fixture instance you could just put scope="class". (See the docs on scopes).
In this case I'd be tempted to handle the caching myself:
import pytest
from datetime import datetime
#pytest.mark.parametrize("instance", ("instance1", "instance2"))
class TestClass:
#pytest.fixture()
def authcode(self, instance, authcodes={}, which=[0]):
if not instance in authcodes:
authcodes[
instance
] = f"authcode {which[0]} for {instance} at {datetime.now()}"
which[0] += 1
return authcodes[instance]
def test1(self, authcode):
print(authcode)
def test2(self, authcode):
print(authcode)
(which is just used to prove that we don't regenerate the fixture).
This feels inelegant and I'm open to better ways of doing it.
I'm mocking an internal library class (Server) of python that provides connection to HTTP JSON-RPC server. But when running the test the class is not mocking. The class is used calling a project class that is a wrapper for other class that effectively instantiates the Server class.
I extract here the parts of the code that give sense for what I'm talking about.
Unit test:
#patch('jsonrpc_requests.jsonrpc.Server')
def test_get_question_properties(self, mockServer):
lime_survey = Questionnaires()
# ...
Class Questionnaires:
class Questionnaires(ABCSearchEngine):
""" Wrapper class for LimeSurvey API"""
def get_question_properties(self, question_id, language):
return super(Questionnaires, self).get_question_properties(question_id, language)
Class Questionnaires calls the method get_question_properties from class ABCSearchEnginge(ABC). This class initializes the Server class to provide the connection to the external API.
Class ABCSearchEnginge:
class ABCSearchEngine(ABC):
session_key = None
server = None
def __init__(self):
self.get_session_key()
def get_session_key(self):
# HERE the self.server keep getting real Server class instead the mocked one
self.server = Server(
settings.LIMESURVEY['URL_API'] + '/index.php/admin/remotecontrol')
As the test is mocking Server class why it's not mocking? What is the missing parts?
From what i see you didnt add a return value.
Were did you put the mocked value in : #patch('jsonrpc_requests.jsonrpc.Server') ?
If you try to add a MagicMock what happend (Dont forget to add from mock import patch, MagicMock)?
#patch('jsonrpc_requests.Server', MagicMock('RETURN VALUE HERE'))
You also need to Mock the __init__ method (Where Server is this one from jsonrpc_requests import Server):
#patch.object(Server, '__init__', MagicMock(return_value=None))
I extrapolated your problem from my own understanding, maybe you need to fix some path (Mock need the exact path to do the job).
Problem setting up the MVC design with Cherrypy/MySQL. Here is the setup: (assume all the imports are correct)
##controller.py
class User(object):
def __init__(self):
self.model = model.User()
#cherrypy.expose
def index(self):
return 'some HTML to display user home'
## model.py
class Model(object):
_db = None
def __init__(self):
self._db = cherrypy.thread_data.db
class User(Model):
def getuser(self, email):
#get the user with _db and return result
##service.py
class UserService(object):
def __init__(self):
self._model = model.User()
def GET(self, email):
return self._model.getuser(email)
##starting the server
user = controller.User()
user.service = service.UserService()
cherrypy.tree.mount(user, '/user', self.config)
#app.merge(self.config)
cherrypy.engine.subscribe("start_thread", self._onThreadStart)
self._onThreadStart(-1)
def _onThreadStart(self, threadIndex):
cherrypy.thread_data.db = mysql.connect(**self.config["database"])
if __name__ == '__main__':
cherrypy.engine.start()
cherrypy.engine.block()
the above code has error in model.py at the line: cherrypy.thread_data.db.
I got:
AttributeError: '_ThreadData' object has no attribute 'db'
not sure why, could you please point me into the right direction? I can get the connection, and pull info from controller.py at User index, but not in model.py?
Please help.. thanks.
CherryPy doesn't decide for you what tools to use. It is up to you to pick the tools that fit you and your tasks the best. Thus CherryPy doesn't setup any database connection, your cherrypy.thread_data.db, it's your job.
Personally I use the same concept of responsibility separation, kind of MVC, for my CherryPy apps so there follow two possible ways to achieve what you want.
Design note
I would like to note that the simple solution of thread-mapped database connections, at least in case of MySQL, works pretty well in practice. And additional complexity of more old-fashioned connection pools may not be necessary.
There're however points that shouldn't be overlooked. Your database connection may become killed, lost or be in any other state that won't allow you to make queries on it. In this case reconnection must be preformed.
Also pay attention to avoid connection sharing between threads as it will result in hard-to-debug errors and Python crashes. In your example code, it may relate to a service dispatcher and its cache.
Bootstrapping phase
In your code that sets configuration, mounts CherryPy apps, etc.
bootstrap.py
# ...
import MySQLdb as mysql
def _onThreadStart(threadIndex):
cherrypy.thread_data.db = mysql.connect(**config['database'])
cherrypy.engine.subscribe('start_thread', _onThreadStart)
# useful for tests to have db connection on current thread
_onThreadStart(-1)
model.py
import cherrypy
import MySQLdb as mysql
class Model(object):
'''Your abstract model'''
_db = None
def __init__(self):
self._db = cherrypy.thread_data.db
try:
# reconnect if needed
self._db.ping(True)
except mysql.OperationalError:
pass
I wrote a complete CherryPy deployment tutorial, cherrypy-webapp-skeleton, a couple of years ago. You can take a look at the code, as the demo application uses exactly this approach.
Model property
To achieve less code coupling and to avoid import cycles it could be a good idea to move all database related code to model module. It may include, initial connection queries like setting operation timezone, making MySQLdb converters timzeone-aware, etc.
model.py
class Model(object):
def __init__(self):
try:
# reconnect if needed
self._db.ping(True)
except mysql.OperationalError:
pass
#property
def _db(self):
'''Thread-mapped connection accessor'''
if not hasattr(cherrypy.thread_data, 'db'):
cherrypy.thread_data.db = mysql.connect(**config['database'])
return cherrypy.thread_data.db
I am trying to test the following class using unittest and the mock library:
class Connection(object):
def __init__(self, cookie):
self.connect = None
self.session = Session()
self.session.load(cookie)
# do some stuff with self.session
self.some_info = self.session.data['the_info']
How could I test if when I create an instance of Connection, depending on the return of the Session instance, I assert if self.some_info is with the value I am expecting?
I wish to use the mock library. In its documentation I have an example of mocking chained calls (http://www.voidspace.org.uk/python/mock/examples.html#mocking-chained-calls), but it isn't very clear of how I can adapt it to my problem.
The Session.load(cookie) method sets some attributes in the Session instance. I would like to set this values fixed for my tests for every value of cookie.
Assume Connection is located in module package.module.connection
The following code should be how you would test your session:
import mock
class TestConnection(unittest.TestCase):
#mock.patch('package.module.connection.Session')
def test_some_info_on_session_is_set(self, fake_session):
fake_session.data = {'the_info': 'blahblah'}
cookie = Cookie()
connection = Connection(cookie)
self.assertEqual(connection.some_info, 'blahblah')
fake_session.load.assert_called_once_with(cookie)