I've been pulling my hair out trying to figure out how to mock the sqlite3.Cursor class specifically the fetchall method.
Consider the following code sample
import sqlite3
from mock import Mock, patch
from nose.tools import assert_false
class Foo:
def check_name(name):
conn = sqlite3.connect('temp.db')
c = conn.cursor()
c.execute('SELECT * FROM foo where name = ?', name)
if len(c.fetchall()) > 0:
return True
return False
#patch('sqlite3.Cursor.fetchall', Mock(return_value=['John', 'Bob']))
def test_foo():
foo = Foo()
assert_false(foo.check_name('Cane'))
Running nosetests results in no fun error
E
======================================================================
ERROR: temp.test_foo
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/koddsson/.virtualenvs/temp/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/koddsson/.virtualenvs/temp/lib/python2.7/site-packages/mock.py", line 1214, in patched
patching.__exit__(*exc_info)
File "/home/koddsson/.virtualenvs/temp/lib/python2.7/site-packages/mock.py", line 1379, in __exit__
setattr(self.target, self.attribute, self.temp_original)
TypeError: can't set attributes of built-in/extension type 'sqlite3.Cursor'
----------------------------------------------------------------------
Ran 1 test in 0.002s
FAILED (errors=1)
Should I not be able to mock the fetchall method or am I doing something horribly wrong?
I would take the approach of patching out sqlite3 imported in your module and then work from there.
Let's assume your module is named what.py.
I would patch out what.sqlite3 and then mock the return value of .connect().cursor().fetchall.
Here is a more complete example:
from mock import patch
from nose.tools import assert_true, assert_false
from what import Foo
def test_existing_name():
with patch('what.sqlite3') as mocksql:
mocksql.connect().cursor().fetchall.return_value = ['John', 'Bob']
foo = Foo()
assert_true(foo.check_name('John'))
I have found a way to mock sqlite3.Cursor in my tests:
cursor = MagicMock(Cursor)
cursor.fetchall.return_value = [{'column1': 'hello', 'column2': 'world'}]
I am pretty new in python but this is how I do it in Java.
You can't mock everything and databases are particularly tricky. I often find that the right thing to do (esp. with Sqlite since it's so easy) is to load up a test database with mock data and use that in the tests (i.e. fixtures). After all, what you really need to test is whether your code is accessing and querying the database correctly.
The question you are usually trying to answer in a test like this is "If there is X data in the DB and I execute query Y, does that query return Z like I expect", or at a higher level "If I pass parameter X to my method, does it return value Z (based on getting Y from the db)."
In your example, the real question is "Is SELECT * FROM foo where name = ? the right query in this method?" but you don't answer it if you mock the response.
Related
I'm working on code that retrieves information from Twilio's Flow system through their API. That part of the code functions fine, but when I try to mock it for unit testing, it's throwing an error from the mocked api response.
Here is the code being tested:
from twilio.rest import Client
class FlowChecker:
def __init__(self, twilio_sid, twilio_auth_token):
self.twilio_SID = twilio_sid
self.twilio_auth_token = twilio_auth_token
self.client = Client(self.twilio_SID, self.twilio_auth_token)
self.calls = self.client.calls.list()
self.flows = self.client.studio.v2.flows
def get_active_executions(self):
active_executions = []
for flow in self.flows.list():
executions = self.client.studio.v2.flows(flow.sid).executions.list()
for execution in executions:
if execution._properties['status'] != 'ended':
active_executions.append({'flow_sid': flow.sid, 'execution': execution})
And here is my unit test code that's throwing the error:
import unittest
from unittest.mock import Mock, patch
from flows.twilio_flows import FlowChecker
class FlowCheckerTest(unittest.TestCase):
#patch('flows.twilio_flows.Client')
def test_get_active_flows(self, mock_client):
flow_checker = FlowChecker('fake_sid', 'fake_auth_token')
mock_call = Mock()
mock_flow = Mock()
mock_flow.sid = 0
mock_execution = Mock()
mock_execution._properties = {'status': 'ended'}
mock_client.calls.list().return_value = [mock_call]
mock_client.studio.v2.flows = [mock_flow]
mock_client.studio.v2.flows(mock_flow.sid).executions.list().return_value = [mock_execution]
self.assertEqual(flow_checker.get_active_executions(), [])
And here is the error traceback:
Ran 2 tests in 0.045s
FAILED (errors=1)
Error
Traceback (most recent call last):
File "C:\Users\Devon\AppData\Local\Programs\Python\Python310\lib\unittest\mock.py", line 1369, in patched
return func(*newargs, **newkeywargs)
File "C:\Users\Devon\PycharmProjects\Day_35\tests\twilio_flows_test'.py", line 19, in test_get_active_flows_when_empty
mock_client.studio.v2.flows(mock_flow.sid).executions.list().return_value = [mock_execution]
TypeError: 'list' object is not callable
Process finished with exit code 1
As you can see, "mock_client.calls.list().return_value = [mock_call]" doesn't throw any errors during init, and the first code block runs fine. It's only the mocked executions.list() that's throwing the error in the test.
Can anyone clear this up?
Thank you!
I've tried researching this specific issue and was unable to find information addressing it. It's a very specific deeply nested function in a vendor supplied client that I need to test, so I don't know what to try.
The problem isn't with .list(), it's with .flows().
mock_client.studio.v2.flows = [mock_flow]
mock_client.studio.v2.flows(mock_flow.sid).executions.list().return_value = [mock_execution]
You assign .flows to be a list, and then you try to call it like a function, which causes the error.
I think maybe you intended to say .flows[mock_flow.sid] instead of .flows(mock_flow.sid)?
Although even that doesn't make sense. .flows is a one-element list, so you would use .flows[0] to access the first (and only) item.
I'm trying to test a large legacy Django application, and I'm getting confused by Python mocking as I have never worked on large Python application.
Specifically, I have method has a long call chain inside that generates an array:
def update(self): # in some class X
# ...
for z in foo.models.Bar.objects.filter(x=1).select('xyz'):
raise Exception("mocked successfully")
I'd like to mock the foo.models.Bar.objects.filter(x=1).select('xyz').
Attempt 1
I've tried several approaches gleaned from various questions, notably using a decorator:
#mock.patch('foo.models.Bar.objects.filter.select')
def test_update(self, mock_select):
mock_select.return_value = [None]
X().update()
I never hit the inside of the mocked call, however- the test should fail due to the exception being raised.
Attempt 2
#mock.patch('foo.models.Bar')
def test_update(self, mock_Bar):
mock_Bar.objects.filter(x=1).select('xyz').return_value = [None]
X().update()
Attempt 3
#mock.patch('foo.models.Bar')
def test_update(self, mock_Bar):
mock_Bar.objects.filter().select().return_value = [None]
X().update()
Attempt 4
I then tried something more basic, to see if I could get an NPE, which didn't work either.
#mock.patch('foo.models.Bar')
def test_update(self, mock_Bar):
mock_Bar.return_value = None
X().update()
All of my attempts pass the test, instead of the exception firing like I expect it to.
It's late so I assume I must be overlooking something basic in the examples I've seen!?
I was able to have it pass by mocking objects. Attempt #3 is close, you just need to change it to filter.return_value.select.return_value to have it pass. Here's my suggestion though as it seems mocking .objects is the preferred way.
#mock.patch('foo.models.Bar.objects')
def test_update(self, mock_bar_objects):
mock_bar_objects.filter.return_value.select.return_value = [None]
X().update()
EDIT: Test run output:
ERROR: test_update (test_x.TestDjango)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/wholevinski/.virtualenvs/p2/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
return func(*args, **keywargs)
File "/home/wholevinski/so_test/django_mock/test/test_x.py", line 10, in test_update
X().update()
File "/home/wholevinski/so_test/django_mock/foo/x_module.py", line 6, in update
raise Exception("mocked successfully")
Exception: mocked successfully
----------------------------------------------------------------------
Ran 1 test in 0.002s
FAILED (errors=1)
I have a method in Python that looks like this (in comicfile.py):
from zipfile import ZipFile
...
class ComicFile():
...
def page_count(self):
"""Return the number of pages in the file."""
if self.file == None:
raise ComicFile.FileNoneError()
if not os.path.isfile(self.file):
raise ComicFile.FileNotFoundError()
with ZipFile(self.file) as zip:
members = zip.namelist()
pruned = self.prune_dirs(members)
length = len(pruned)
return length
I'm trying to write a unit test for this (I've already tested prune_dirs), and so for this is what I have (test_comicfile.py):
import unittest
import unittest.mock
import comicfile
...
class TestPageCount(unittest.TestCase):
def setUp(self):
self.comic_file = comicfile.ComicFile()
#unittest.mock.patch('comicfile.ZipFile')
def test_page_count(self, mock_zip_file):
# Store as tuples to use as dictionary keys.
members_dict = {('dir/', 'dir/file1', 'dir/file2'):2,
('file1.jpg', 'file2.jpg', 'file3.jpg'):3
}
# Make the file point to something to prevent FileNoneError.
self.comic_file.file = __file__
for file_tuple, count in members_dict.items():
mock_zip_file.return_value.namelist = list(file_tuple)
self.assertEqual(count, self.comic_file.page_count())
When I run this test, I get the following:
F..ss....
======================================================================
FAIL: test_page_count (test_comicfile.TestPageCount)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
return func(*args, **keywargs)
File "/Users/chuck/Dropbox/Projects/chiv/chiv.cbstar/test_comicfile.py", line 86, in test_page_count
self.assertEqual(count, self.comic_file.page_count())
AssertionError: 2 != 0
----------------------------------------------------------------------
Ran 9 tests in 0.010s
FAILED (failures=1, skipped=2)
OK, so self.comic_file.page_count() is returning 0. I tried placing the following line after members = zip.namelist() in page_count.
print('\nmembers -> ' + str(members))
During the test, I get this:
members -> <MagicMock name='ZipFile().__enter__().namelist()' id='4483358280'>
I'm quite new to unit testing and am quite nebulous on using unittest.mock, but my understanding is that mock_zip-file.return_value.namelist = list(file_tuple) should have made it so that the namelist method of the ZipFile class would return each of the file_tuple contents in turn. What it is doing I have no idea.
I think what I'm trying to do here is clear, but I can't seem to figure out how to override the namelist method so that my unit test is only testing this one function instead of having to deal with ZipFile as well.
ZipFile is instantiated as a context manager. to mock it you have to refer to its __enter__ method.
mock_zip_file.return_value.__enter__.return_value.namelist.return_value = list(file_tuple)
What you're trying to do is very clear, but the context manager adds complexity to the mocking.
One trick is that when a mock registers all calls made to it, in this example it is saying it has a call at:
members -> <MagicMock name='ZipFile().__enter__().namelist()' id='4483358280'>
This can guide you in registering your mocked object, replace all () with return_value
I've a class in python that contains a static method. I want to mock.patch it in order to see if it was called. When trying to do it I get an error:
AttributeError: path.to.A does not have the attribute 'foo'
My setup can be simplified to:
class A:
#staticMethod
def foo():
bla bla
Now the test code that fails with error:
def test():
with mock.patch.object("A", "foo") as mock_helper:
mock_helper.return_value = ""
A.some_other_static_function_that_could_call_foo()
assert mock_helper.call_count == 1
You can always use patch as a decorator, my preferred way of patching things:
from mock import patch
#patch('absolute.path.to.class.A.foo')
def test(mock_foo):
mock_foo.return_value = ''
# ... continue with test here
EDIT: Your error seems to hint that you have a problem elsewhere in your code. Possibly some signal or trigger that requires this method that is failing?
I was getting that same error message when trying to patch a method using the #patch decorator.
Here is the full error I got.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/tornado/testing.py", line 136, in __call__
result = self.orig_method(*args, **kwargs)
File "/usr/local/lib/python3.6/unittest/mock.py", line 1171, in patched
arg = patching.__enter__()
File "/usr/local/lib/python3.6/unittest/mock.py", line 1243, in __enter__
original, local = self.get_original()
File "/usr/local/lib/python3.6/unittest/mock.py", line 1217, in get_original
"%s does not have the attribute %r" % (target, name)
AttributeError: <module 'py-repo.models.Device' from
'/usr/share/projects/py-repo/models/Device.py'> does not have the attribute 'get_device_from_db'
What I ended up doing to fix this was changing the patch decorator I used
from
#patch('py-repo.models.Device.get_device_from_db')
to #patch.object(DeviceModel, 'get_device_from_db')
I really wish I could explain further why that was the issue but I'm still pretty new to Python myself. The patch documentation was especially helpful in figuring out what was available to work with. Important: I should note that get_device_from_db uses the #staticmethod decorator which may be changing things. Hope it helps though.
What worked for me:
#patch.object(RedisXComBackend, '_handle_conn')
def test_xcoms(self, mock_method: MagicMock):
mock_method.return_value = fakeredis.FakeStrictRedis()
'_handle_conn' (static function) looks like this:
#staticmethod
def _handle_conn():
redis_hook = RedisHook()
conn: Redis = redis_hook.get_conn()
I'm having difficuly getting my head around python's mock test methodology.
I want to do some mocking on this file.
Since packages xbmc, xbmcaddon and xbmcgui cannot be imported in a normal python environment I've managed to mock them out like this:
class XBMCTestCase(unittest.TestCase):
def setUp(self):
#Mock up any calls to modules that cannot be imported
self.xbmc = Mock()
self.xbmcgui = Mock()
self.xbmcaddon = Mock()
modules = {
'xbmc' : self.xbmc,
'xbmcgui': self.xbmcgui,
'xbmcaddon': self.xbmcaddon
}
self.module_patcher = patch.dict('sys.modules', modules) ##UndefinedVariable
self.module_patcher.start()
See it in action here.
So when I import setlocation.py I get an error like this:
File "/home/paulo/workspace/weather.metoffice/src/metoffice/utils/utilities.py", line 21, in <module>
CACHE_FOLDER = os.path.join(ADDON_DATA_PATH, 'cache')
File "/usr/lib/python2.7/posixpath.py", line 78, in join
path += b
TypeError: unsupported operand type(s) for +=: 'Mock' and 'str'
Even if I mock out 'metoffice.utils' (by adding it to the list of modules in the patch created at setup) I get a similar error in setlocation.py
File "/home/paulo/workspace/weather.metoffice/src/metoffice/setlocation.py", line 32, in <module>
GEOIP_PROVIDER = int(__addon__.getSetting('GeoIPProvider'))
TypeError: int() argument must be a string or a number, not 'Mock'
So I need __addon__.getSetting() to return a string.
Any ideas?
All attempts have failed, but I don't think I have a full grasp of the capabilities of the mock package.
Note I'm on Python 2.7.3 with mock 1.0.1
You need to tell your mocks what to return. The __addon__ value is the result of a xbmcaddon.Addon() call, so you can get access to that mock object with:
addon = self.xbmcaddon.Addon.return_value
because .return_value gives you the actual Mock object that calling Addon() would return.
Now you can tell that Mock object what to return when the getSetting() method is called; there are two values to provide here, so you could use the side_effect to set a sequence of values to return:
addon.getSetting.side_effect = ['some_api_key', '42']
where the first call to __addon__.getSetting() will produce the first value 'some_api_key', the second cal will produce '42'.