I am trying to perform unit tests using Pytest on some of my code. The tests are being run in a separate Conda environment on a Docker. I would like to test certain functionalities of my code but cannot install all the modules of my code, because of the complexity of the installation of some of these modules and the time it would take to run.
How can I import only certain modules from a file, without needing the other modules installed?
If I try running a test, whilst importing a module from a file, my test fails since it cannot import the other modules.
Below is a mock-up of my file system:
test_file.py
from other_file import afunction
def this_test():
assert afunction(2, 2) == 4
other_file.py
import math
import large_program
def afunction(x,y):
return math.pow(x, y)
def anotherfunc():
return large_program()
If I run Pytest, I will get:
E ImportError: No module named 'large_program'
Quite simply: extract the functions that do not depend on large_program into another module and only test this module. Note that you can do this without breaking client code (code depending on your other_file module) by importing the relevant names in other_file:
# utils.py
import math
def afunction(x,y):
return math.pow(x, y)
then
# other_file.py
import large_program
# this will allow client code to access `afunction` from `other_file`
from utils import afunction
def anotherfunc():
return large_program()
and finally:
# test_file.py
# here we import from utils so we don't depend on `large_program`
from utils import afunction
def this_test():
assert afunction(2, 2) == 4
I liked the idea of cristid9 of mocking and combined it with dano's post here. I created an empty file called "nothing.py" that will replace the "large_program" with a unittest mock:
test_file.py
import unittest.mock as mock
import nothing
with mock.patch.dict('sys.modules', large_program=nothing):
from other_file import afunction
def this_test():
assert afunction(2, 2) == 4
The other_file.py looks still like this
import math
import large_program
def afunction(x,y):
return math.pow(x, y)
def anotherfunc():
return large_program()
You can also apply the with statement on multiple modules:
other_file.py
import math
import large_program
import even_larger_program
def afunction(x,y):
return math.pow(x, y)
def anotherfunc():
return large_program()
test_file.py
import unittest.mock as mock
import nothing
with mock.patch.dict('sys.modules', large_program=nothing), mock.patch.dict('sys.modules', even_larger_program=nothing):
from other_file import afunction
def this_test():
assert afunction(2, 2) == 4
Related
I need a cleaner way to import the modules into the test_file.py, I am using pytest for my test. My present pytest setup is like the below which works, but I need a more organised way to import the modules into the test functions without initializing the database client. The global variable is breaking my test when I import using the cleaner way.
conftest.py
import pytest
#pytest.fixture(autouse=True)
def no_request_cassandra_client(mocker):
"""Remove cassandra.cluster.Cluster for all tests."""
mock_cass_cluster = mocker.patch("cassandra.cluster.Cluster")
test_file.py
import pytest
def test_car_class(no_request_cassandra_client):
from A.BMW import car
car_func = car()
assert car_func
This is what I want to have, but the car method has a global variable like the BMW.py, the test is breaking because of the Cluster(["Cluster_ip_address"]) with the cleaner solution.
Cleaner test_file.py
import pytest
from A.BMW import car
def test_car_class():
car_func = car()
assert car_func
BMW.py
from cassandra.cluster import Cluster
cluster = Cluster(["Cluster_ip_address"])
def car():
x = 2 + 3
print(x)
return x
I have a python file the imports certain modules that aren't easily testable on all systems. Just importing the module on macOS for example causes an exception. Is there a way to mock this. Essentially I want to change the source code on one line before actually running the import statement.
Test code
from module_a import func
def test_func():
print("...")
Module code
module_a.py
import os
import numpy as np
# ...
import magicmodulethatbreaksonmac
def func():
# Nothing inside this function needs magicmodulethatbreaksonmac
Would be great if in the test code, I could do some sort of mock on module_a.py to not actually try to import magicmodulethatbreaksonmac but rather just don't import anything as it's not needed to test this function.
Is there a way to put something of the same name in the namespace already that doesn't have anything in it so when it tries to import it again here, nothing happens? (If that is the behavior...)
I would think there's something here, but I'm not seeing it:
https://docs.python.org/3/library/unittest.mock.html
Here's sort of what I'm proposing with the sys.modules solution. You have options as far as if you want to do that mock globally, in each test case:
'''bad_module.py'''
raise RuntimeError("don't import me")
'''my_module.py'''
from bad_module import somestuff
def my_func():
pass
'''test_my_module.py'''
import sys
import unittest
from unittest import mock
def test_stuff():
sys.modules['bad_module'] = mock.Mock()
from my_module import my_func
my_func()
If I don't mock I get this:
...
File "/home/wholevinski/so_test/mock_import/test_my_module.py", line 2, in <module>
from my_module import my_func
File "/home/wholevinski/so_test/mock_import/my_module.py", line 1, in <module>
from bad_module import somestuff
File "/home/wholevinski/so_test/mock_import/bad_module.py", line 1, in <module>
raise RuntimeError("don't import me")
RuntimeError: don't import me
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
With the mock I get this:
(p36) [localhost mock_import]$ nosetests test_my_module.py
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
I've looked around but still don't get how to mock a library used inside a function and assert that its been called properly.
a.py
import win32clipboard
def copy():
win32clipboard.OpenClipboard()
win32clipboard.EmptyClipboard()
win32clipboard.SetClipboardText('dummy')
win32clipboard.CloseClipboard()
test_a.py
import a
import pytest
def test_copy():
# Mock win32clipboard somehow
# Run a.copy()
# assert mock win32clipboard.call_count == 4
There is a mistake in your approach.
win32clipboard is a library, with some classes and methods. You must mock every class from this library you want to use (OpenClipboard, EmptyClipboard, SetClipboardText and CloseClipboard)
import a
import pytest
from unittest.mock import patch
#patch('win32clipboard.OpenClipboard')
#patch('win32clipboard.EmptyClipboard')
#patch('win32clipboard.SetClipboardText')
#patch('win32clipboard.CloseClipboard')
def test_copy(mock_close, mock_set, mock_empty, mock_open):
a.copy()
assert mock_close.called
assert mock_set.called
assert mock_empty.called
assert mock_open.called
Folks,
I have a problem during including file.py to test_file.py namely:
file.py uses Robot library BuiltIn:
from robot.libraries.BuiltIn import BuiltIn
DEFAULT_IPHY_TTI_TRACE_DIR =
os.path.join(BuiltIn().get_variable_value('${OUTPUT_DIR}'), 'iphy_tti_trace')
And when I try to include file.py in my test_file.py
import pytest
#import file.py
I receive:
test_file.py:8: in <module>
/opt/ute/python/lib/python2.7/site-packages/robot/libraries/BuiltIn.py:1331: in get_variable_value
return self._variables[self._get_var_name(name)]
/opt/ute/python/lib/python2.7/site-packages/robot/libraries/BuiltIn.py:75: in _variables
return self._namespace.variables
/opt/ute/python/lib/python2.7/site-packages/robot/libraries/BuiltIn.py:71: in _namespace
return self._get_context().namespace
/opt/ute/python/lib/python2.7/site-packages/robot/libraries/BuiltIn.py:66: in _get_context
raise RobotNotRunningError('Cannot access execution context')
E RobotNotRunningError: Cannot access execution context
How can I mock this? This is posible at all?
Sure, the issue is just that you can't mock the BuiltIn class where it is used (in file.py). You have to mock the class where it is declared (in robot.libraries.BuiltIn).
Using mocks:
from unittest.mock import patch, MagicMock
def _test_default_iphy_tti_trace_dir():
with patch('robot.libraries.BuiltIn.BuiltIn.get_variable_value', return_value='/foo/bar'):
import file
assert file.DEFAULT_IPHY_TTI_TRACE_DIR == '/foo/bar/iphy_tti_trace'
Using monkeypatch fixture:
def test_default_iphy_tti_trace_dir(monkeypatch):
def mocked_get(self, name):
return '/foo/bar'
monkeypatch.setattr('robot.libraries.BuiltIn.BuiltIn.get_variable_value', mocked_get)
import file
assert file.DEFAULT_IPHY_TTI_TRACE_DIR == '/foo/bar/iphy_tti_trace'
Also note that the mocking is done for the scope of a single test only, so you can't import file on top of the test module as the BuiltIn will be unpatched there, raising the context error.
I'm writing unit tests to validate my project functionalities. I need to replace some of the functions with mock function and I thought to use the Python mock library. The implementation I used doesn't seem to work properly though and I don't understand where I'm doing wrong. Here a simplified scenario:
root/connector.py
from ftp_utils.py import *
def main():
config = yaml.safe_load("vendor_sftp.yaml")
downloaded_files = []
downloaded_files = get_files(config)
for f in downloaded_files:
#do something
root/utils/ftp_utils.py
import os
import sys
import pysftp
def get_files(config):
sftp = pysftp.Connection(config['host'], username=config['username'])
sftp.chdir(config['remote_dir'])
down_files = sftp.listdir()
if down_files is not None:
for f in down_files:
sftp.get(f, os.path.join(config['local_dir'], f), preserve_mtime=True)
return down_files
root/tests/connector_tester.py
import unittest
import mock
import ftp_utils
import connector
def get_mock_files():
return ['digital_spend.csv', 'tv_spend.csv']
class ConnectorTester(unittest.TestCase)
#mock.patch('ftp_utils.get_files', side_effect=get_mock_files)
def test_main_process(self, get_mock_files_function):
# I want to use a mock version of the get_files function
connector.main()
When I debug my test I expect that the get_files function called inside the main of connector.py is the get_mock_files(), but instead is the ftp_utils.get_files(). What am I doing wrong here? What should I change in my code to properly call the get_mock_file() mock?
Thanks,
Alessio
I think there are several problems with your scenario:
connector.py cannot import from ftp_utils.py that way
nor can connector_tester.py
as a habit, it is better to have your testing files under the form test_xxx.py
to use unittest with patching, see this example
In general, try to provide working minimal examples so that it is easier for everyone to run your code.
I modified rather heavily your example to make it work, but basically, the problem is that you patch 'ftp_utils.get_files' while it is not the reference that is actually called inside connector.main() but probably rather 'connector.get_files'.
Here is the modified example's directory:
test_connector.py
ftp_utils.py
connector.py
test_connector.py:
import unittest
import sys
import mock
import connector
def get_mock_files(*args, **kwargs):
return ['digital_spend.csv', 'tv_spend.csv']
class ConnectorTester(unittest.TestCase):
def setUp(self):
self.patcher = mock.patch('connector.get_files', side_effect=get_mock_files)
self.patcher.start()
def test_main_process(self):
# I want to use a mock version of the get_files function
connector.main()
suite = unittest.TestLoader().loadTestsFromTestCase(ConnectorTester)
if __name__ == "__main__":
unittest.main()
NB: what is called when running connector.main() is 'connector.get_files'
connector.py:
from ftp_utils import *
def main():
config = None
downloaded_files = []
downloaded_files = get_files(config)
for f in downloaded_files:
print(f)
connector/ftp_utils.py unchanged.