I need a cleaner way to import the modules into the test_file.py, I am using pytest for my test. My present pytest setup is like the below which works, but I need a more organised way to import the modules into the test functions without initializing the database client. The global variable is breaking my test when I import using the cleaner way.
conftest.py
import pytest
#pytest.fixture(autouse=True)
def no_request_cassandra_client(mocker):
"""Remove cassandra.cluster.Cluster for all tests."""
mock_cass_cluster = mocker.patch("cassandra.cluster.Cluster")
test_file.py
import pytest
def test_car_class(no_request_cassandra_client):
from A.BMW import car
car_func = car()
assert car_func
This is what I want to have, but the car method has a global variable like the BMW.py, the test is breaking because of the Cluster(["Cluster_ip_address"]) with the cleaner solution.
Cleaner test_file.py
import pytest
from A.BMW import car
def test_car_class():
car_func = car()
assert car_func
BMW.py
from cassandra.cluster import Cluster
cluster = Cluster(["Cluster_ip_address"])
def car():
x = 2 + 3
print(x)
return x
Related
I'm asked to develop unit tests for a program which is such badly developed that the tests don't run... but the program does. Thus, I need to explain the reason why and I actually don't know!
Here is a piece of code that intends to represent the code I need to test:
from services import myModule1
from services.spec1 import importedFunc
from services.spec2 import getTool
from services.spec3 import getDict
class myClass(object):
def __init__(self, param1, param2):
self.param1 = param1
self.param2 = param2
self.param3 = 0
self.param4 = 0
def myMethod(self):
try:
myVar1 = globalDict['key1']
myVar2 = globalDict['key2']
newVar = importedFunc(par1=myVar1, par2=myVar2, par3=extVar3)
calcParam = myModule1.methodMod1(self.param1)
self.param3 = calcParam["keyParam3"]
self.param4 = newVar.meth1(self.param2)
globTools.send_message(self.param3, self.param4)
except:
globTools.error_message(self.param3, self.param4)
return
class myClass2(object):
def __init__(self, *myclass2_params):
# some piece of code to intialize dedicated attributes
self.add_objects()
def add_objects(self):
# Some piece of code
my_class = myClass(**necessary_params)
# Some piece of code
return
if __name__ == '__main__':
globTools = getTool("my_program")
globalDict = getDict(some_params)
# Some piece of code
my_class2 = myClass2(**any_params)
# Some piece of code
As you can see, the problem is that the class and its methods uses global variables, defined in the main scope. And it's just a quick summary because it's actually a bit more complicated, but I hope it's enough to give you an overview of the context and help me understand why the unit test fail.
I tried to mock the imported modules, but I did not manage to a successful result, so I first tried to make it simple and just initialize all parameters.
I went to this test file:
import unittest
from my_module import myClass
from services import myModule1
from services.spec1 import importedFunc
from services.spec2 import getTool
from services.spec3 import getDict
def test_myClass(unittest.TestCase):
def setUp(self):
globTools = getTool("my_program")
globalDict = getDict(some_params)
def test_myMethod(self):
test_class = myClass(*necessary_parameters)
test_res = test_class.myMethod()
self.assertIsNotNone(test_res)
if __name__ == '__main__':
unittest.main()
But the test fail, telling me 'globTools is not defined' when trying to instantiate myClass
I also tried to initialize variables directly in the test method, but the result is the same
And to be complete about the technical environment, I cannot run python programs directly and need to launch a docker environment via a Jenkins pipeline - I'm not very familiar with this but I imagine it should not have an impact on the result
I guess the problem comes from the variable's scopes, but I'm not able to explain it in this case: why the test fail where as the method itself works (yes, it actually works, or at least the program globally runs without)
It's not as bad as you think. Your setUp method just needs to define the appropriate top-level globals in your module, rather than local variables.
import unittest
import my_module
from my_module import myClass
from services import myModule1
from services.spec1 import importedFunc
from services.spec2 import getTool
from services.spec3 import getDict
class test_myClass(unittest.TestCase):
def setUp(self):
my_module.globTools = getTool("my_program")
my_module.globalDict = getDict(some_params)
def test_myMethod(self):
test_class = myClass(*necessary_parameters)
test_res = test_class.myMethod()
self.assertIsNotNone(test_res)
if __name__ == '__main__':
unittest.main()
Depending on how the code uses the two globals, setUpClass might be a better place to initialize them, but it's probably not worth worrying about. Once you have tests for the code, you are in a better position to remove the dependency on these globals from the code.
Does pytest provides functionality like unittest.mock to check if the mock was actually called once(or once with some parameter)?
Sample Source code:
my_package/my_module.py
from com.abc.validation import Validation
class MyModule:
def __init__(self):
pass
def will_call_other_package(self):
val = Validation()
val.do()
def run(self):
self.will_call_other_package()
Sample test code for the above source code:
test_my_module.py
import pytest
from pytest_mock import mocker
from my_package.my_module import MyModule
#pytest.fixture
def mock_will_call_other_package(mocker):
mocker.patch('my_package.my_module.will_call_other_package')
#pytest.mark.usefixtures("mock_will_call_other_package")
class TestMyModule:
def test_run(self):
MyModule().run()
#check `will_call_other_package` method is called.
#Looking for something similar to what unittest.mock provide
#mock_will_call_other_package.called_once
If you want to use a fixture that does the patching, you can move the patching into a fixture:
import pytest
from unittest import mock
from my_package.my_module import MyModule
#pytest.fixture
def mock_will_call_other_package():
with mock.patch('my_package.my_module.will_call_other_package') as mocked:
yield mocked
# the mocking will be reverted here, e.g. after the test
class TestMyModule:
def test_run(self, mock_will_call_other_package):
MyModule().run()
mock_will_call_other_package.assert_called_once()
Note that you have to use the fixture parameter in the test. Just using #pytest.mark.usefixtures will not give you access to the mock itself. You can still use it to be effective in all tests in the class, if you don't need to access the mock in all tests (or use autouse=True in the fixture).
Also note that you don't need pytest-mock here - but as mentioned by #hoefling, using it makes the fixture better readable, because you don't need the with clause :
#pytest.fixture
def mock_will_call_other_package(mocker):
yield mocker.patch('my_package.my_module.will_call_other_package')
As an aside: you don't need to import mocker. Fixtures are looked up by name, and available automatically if the respective plugin is installed.
You could try this:
import pytest
from my_package.my_module import MyModule
def test_run(mocker):
mocker.patch('my_package.my_module.will_call_other_package')
MyModule().run()
mock_will_call_other_package.assert_called_once()
First of all, you may not need the burden of an external library such as pytest_mock, because pytest already got you covered using the integration with unittest.
You also do not need to use the usefixtures because whenever you need a fixture, you just receive it in your test method.
An ideal scenario based on your own code would look similar to this:
import pytest
from unittest.mock import patch
from com.abc.validation import Validation
class MyModule:
def __init__(self):
pass
def will_call_other_package(self):
val = Validation()
val.do()
def run(self):
self.will_call_other_package()
#pytest.fixture
def call_other_module():
with patch("my_package.my_module.MyModule.will_call_other_package") as _patched:
yield _patched
class TestMyModule:
def test_run_will_call_other_package(self, call_other_module):
call_other_module.assert_not_called()
obj = MyModule()
call_other_module.assert_not_called()
obj.run()
call_other_module.assert_called_once()
And also if you want to make sure that you did infact patch the target MyModule.will_call_other_package, modify your test like this:
class TestMyModule:
def test_run_will_call_other_package(self, call_other_module):
call_other_module.assert_not_called()
obj = MyModule()
call_other_module.assert_not_called()
obj.run()
call_other_module.assert_called_once()
assert False, (MyModule.will_call_other_package, call_other_module)
And you'll see something similar to this:
AssertionError: (<MagicMock name='will_call_other_package' id='140695551841328'>, <MagicMock name='will_call_other_package' id='140695551841328'>)
As you can see the id of both objects are the same, confirming our experiment was successful.
I am using unittest module for writing tests.
I need to test initialization of the object inside a testcase using different inputs.
For this purpose I am importing the class inside setUp(). But when I try to use the class inside test_*() functions, I get this error - NameError: name 'Example' is not defined
Here is my code sample-
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
import Example
def test_sample_function(self):
e = Example(1,2)
I know that I can simply import the class at top of the script. But I do not want to do that. I need to import it only during setup of the testscript.
Looking for some help here.
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
import Example
self.Example = Example
def test_sample_function(self):
e = self.Example(1,2)
There's no reason to import the module in setUp. The module is still available globally in sys.modules, but you've only bound it to a local name that goes away after setUp returns. Just import it globally.
import unittest
import Example
class TestExample(unittest.TestCase):
def test_sample_function(self):
e = Example(1,2)
I am trying to perform unit tests using Pytest on some of my code. The tests are being run in a separate Conda environment on a Docker. I would like to test certain functionalities of my code but cannot install all the modules of my code, because of the complexity of the installation of some of these modules and the time it would take to run.
How can I import only certain modules from a file, without needing the other modules installed?
If I try running a test, whilst importing a module from a file, my test fails since it cannot import the other modules.
Below is a mock-up of my file system:
test_file.py
from other_file import afunction
def this_test():
assert afunction(2, 2) == 4
other_file.py
import math
import large_program
def afunction(x,y):
return math.pow(x, y)
def anotherfunc():
return large_program()
If I run Pytest, I will get:
E ImportError: No module named 'large_program'
Quite simply: extract the functions that do not depend on large_program into another module and only test this module. Note that you can do this without breaking client code (code depending on your other_file module) by importing the relevant names in other_file:
# utils.py
import math
def afunction(x,y):
return math.pow(x, y)
then
# other_file.py
import large_program
# this will allow client code to access `afunction` from `other_file`
from utils import afunction
def anotherfunc():
return large_program()
and finally:
# test_file.py
# here we import from utils so we don't depend on `large_program`
from utils import afunction
def this_test():
assert afunction(2, 2) == 4
I liked the idea of cristid9 of mocking and combined it with dano's post here. I created an empty file called "nothing.py" that will replace the "large_program" with a unittest mock:
test_file.py
import unittest.mock as mock
import nothing
with mock.patch.dict('sys.modules', large_program=nothing):
from other_file import afunction
def this_test():
assert afunction(2, 2) == 4
The other_file.py looks still like this
import math
import large_program
def afunction(x,y):
return math.pow(x, y)
def anotherfunc():
return large_program()
You can also apply the with statement on multiple modules:
other_file.py
import math
import large_program
import even_larger_program
def afunction(x,y):
return math.pow(x, y)
def anotherfunc():
return large_program()
test_file.py
import unittest.mock as mock
import nothing
with mock.patch.dict('sys.modules', large_program=nothing), mock.patch.dict('sys.modules', even_larger_program=nothing):
from other_file import afunction
def this_test():
assert afunction(2, 2) == 4
I'm writing unit tests to validate my project functionalities. I need to replace some of the functions with mock function and I thought to use the Python mock library. The implementation I used doesn't seem to work properly though and I don't understand where I'm doing wrong. Here a simplified scenario:
root/connector.py
from ftp_utils.py import *
def main():
config = yaml.safe_load("vendor_sftp.yaml")
downloaded_files = []
downloaded_files = get_files(config)
for f in downloaded_files:
#do something
root/utils/ftp_utils.py
import os
import sys
import pysftp
def get_files(config):
sftp = pysftp.Connection(config['host'], username=config['username'])
sftp.chdir(config['remote_dir'])
down_files = sftp.listdir()
if down_files is not None:
for f in down_files:
sftp.get(f, os.path.join(config['local_dir'], f), preserve_mtime=True)
return down_files
root/tests/connector_tester.py
import unittest
import mock
import ftp_utils
import connector
def get_mock_files():
return ['digital_spend.csv', 'tv_spend.csv']
class ConnectorTester(unittest.TestCase)
#mock.patch('ftp_utils.get_files', side_effect=get_mock_files)
def test_main_process(self, get_mock_files_function):
# I want to use a mock version of the get_files function
connector.main()
When I debug my test I expect that the get_files function called inside the main of connector.py is the get_mock_files(), but instead is the ftp_utils.get_files(). What am I doing wrong here? What should I change in my code to properly call the get_mock_file() mock?
Thanks,
Alessio
I think there are several problems with your scenario:
connector.py cannot import from ftp_utils.py that way
nor can connector_tester.py
as a habit, it is better to have your testing files under the form test_xxx.py
to use unittest with patching, see this example
In general, try to provide working minimal examples so that it is easier for everyone to run your code.
I modified rather heavily your example to make it work, but basically, the problem is that you patch 'ftp_utils.get_files' while it is not the reference that is actually called inside connector.main() but probably rather 'connector.get_files'.
Here is the modified example's directory:
test_connector.py
ftp_utils.py
connector.py
test_connector.py:
import unittest
import sys
import mock
import connector
def get_mock_files(*args, **kwargs):
return ['digital_spend.csv', 'tv_spend.csv']
class ConnectorTester(unittest.TestCase):
def setUp(self):
self.patcher = mock.patch('connector.get_files', side_effect=get_mock_files)
self.patcher.start()
def test_main_process(self):
# I want to use a mock version of the get_files function
connector.main()
suite = unittest.TestLoader().loadTestsFromTestCase(ConnectorTester)
if __name__ == "__main__":
unittest.main()
NB: what is called when running connector.main() is 'connector.get_files'
connector.py:
from ftp_utils import *
def main():
config = None
downloaded_files = []
downloaded_files = get_files(config)
for f in downloaded_files:
print(f)
connector/ftp_utils.py unchanged.