I'm writing unit tests to validate my project functionalities. I need to replace some of the functions with mock function and I thought to use the Python mock library. The implementation I used doesn't seem to work properly though and I don't understand where I'm doing wrong. Here a simplified scenario:
root/connector.py
from ftp_utils.py import *
def main():
config = yaml.safe_load("vendor_sftp.yaml")
downloaded_files = []
downloaded_files = get_files(config)
for f in downloaded_files:
#do something
root/utils/ftp_utils.py
import os
import sys
import pysftp
def get_files(config):
sftp = pysftp.Connection(config['host'], username=config['username'])
sftp.chdir(config['remote_dir'])
down_files = sftp.listdir()
if down_files is not None:
for f in down_files:
sftp.get(f, os.path.join(config['local_dir'], f), preserve_mtime=True)
return down_files
root/tests/connector_tester.py
import unittest
import mock
import ftp_utils
import connector
def get_mock_files():
return ['digital_spend.csv', 'tv_spend.csv']
class ConnectorTester(unittest.TestCase)
#mock.patch('ftp_utils.get_files', side_effect=get_mock_files)
def test_main_process(self, get_mock_files_function):
# I want to use a mock version of the get_files function
connector.main()
When I debug my test I expect that the get_files function called inside the main of connector.py is the get_mock_files(), but instead is the ftp_utils.get_files(). What am I doing wrong here? What should I change in my code to properly call the get_mock_file() mock?
Thanks,
Alessio
I think there are several problems with your scenario:
connector.py cannot import from ftp_utils.py that way
nor can connector_tester.py
as a habit, it is better to have your testing files under the form test_xxx.py
to use unittest with patching, see this example
In general, try to provide working minimal examples so that it is easier for everyone to run your code.
I modified rather heavily your example to make it work, but basically, the problem is that you patch 'ftp_utils.get_files' while it is not the reference that is actually called inside connector.main() but probably rather 'connector.get_files'.
Here is the modified example's directory:
test_connector.py
ftp_utils.py
connector.py
test_connector.py:
import unittest
import sys
import mock
import connector
def get_mock_files(*args, **kwargs):
return ['digital_spend.csv', 'tv_spend.csv']
class ConnectorTester(unittest.TestCase):
def setUp(self):
self.patcher = mock.patch('connector.get_files', side_effect=get_mock_files)
self.patcher.start()
def test_main_process(self):
# I want to use a mock version of the get_files function
connector.main()
suite = unittest.TestLoader().loadTestsFromTestCase(ConnectorTester)
if __name__ == "__main__":
unittest.main()
NB: what is called when running connector.main() is 'connector.get_files'
connector.py:
from ftp_utils import *
def main():
config = None
downloaded_files = []
downloaded_files = get_files(config)
for f in downloaded_files:
print(f)
connector/ftp_utils.py unchanged.
Related
I'm asked to develop unit tests for a program which is such badly developed that the tests don't run... but the program does. Thus, I need to explain the reason why and I actually don't know!
Here is a piece of code that intends to represent the code I need to test:
from services import myModule1
from services.spec1 import importedFunc
from services.spec2 import getTool
from services.spec3 import getDict
class myClass(object):
def __init__(self, param1, param2):
self.param1 = param1
self.param2 = param2
self.param3 = 0
self.param4 = 0
def myMethod(self):
try:
myVar1 = globalDict['key1']
myVar2 = globalDict['key2']
newVar = importedFunc(par1=myVar1, par2=myVar2, par3=extVar3)
calcParam = myModule1.methodMod1(self.param1)
self.param3 = calcParam["keyParam3"]
self.param4 = newVar.meth1(self.param2)
globTools.send_message(self.param3, self.param4)
except:
globTools.error_message(self.param3, self.param4)
return
class myClass2(object):
def __init__(self, *myclass2_params):
# some piece of code to intialize dedicated attributes
self.add_objects()
def add_objects(self):
# Some piece of code
my_class = myClass(**necessary_params)
# Some piece of code
return
if __name__ == '__main__':
globTools = getTool("my_program")
globalDict = getDict(some_params)
# Some piece of code
my_class2 = myClass2(**any_params)
# Some piece of code
As you can see, the problem is that the class and its methods uses global variables, defined in the main scope. And it's just a quick summary because it's actually a bit more complicated, but I hope it's enough to give you an overview of the context and help me understand why the unit test fail.
I tried to mock the imported modules, but I did not manage to a successful result, so I first tried to make it simple and just initialize all parameters.
I went to this test file:
import unittest
from my_module import myClass
from services import myModule1
from services.spec1 import importedFunc
from services.spec2 import getTool
from services.spec3 import getDict
def test_myClass(unittest.TestCase):
def setUp(self):
globTools = getTool("my_program")
globalDict = getDict(some_params)
def test_myMethod(self):
test_class = myClass(*necessary_parameters)
test_res = test_class.myMethod()
self.assertIsNotNone(test_res)
if __name__ == '__main__':
unittest.main()
But the test fail, telling me 'globTools is not defined' when trying to instantiate myClass
I also tried to initialize variables directly in the test method, but the result is the same
And to be complete about the technical environment, I cannot run python programs directly and need to launch a docker environment via a Jenkins pipeline - I'm not very familiar with this but I imagine it should not have an impact on the result
I guess the problem comes from the variable's scopes, but I'm not able to explain it in this case: why the test fail where as the method itself works (yes, it actually works, or at least the program globally runs without)
It's not as bad as you think. Your setUp method just needs to define the appropriate top-level globals in your module, rather than local variables.
import unittest
import my_module
from my_module import myClass
from services import myModule1
from services.spec1 import importedFunc
from services.spec2 import getTool
from services.spec3 import getDict
class test_myClass(unittest.TestCase):
def setUp(self):
my_module.globTools = getTool("my_program")
my_module.globalDict = getDict(some_params)
def test_myMethod(self):
test_class = myClass(*necessary_parameters)
test_res = test_class.myMethod()
self.assertIsNotNone(test_res)
if __name__ == '__main__':
unittest.main()
Depending on how the code uses the two globals, setUpClass might be a better place to initialize them, but it's probably not worth worrying about. Once you have tests for the code, you are in a better position to remove the dependency on these globals from the code.
I have config data that should be loaded before another code (because another code use it).
So, for now I see only way to do this is to call the function at the top before rest imports:
from Init.Loaders.InitPreLoader import InitPreLoader
# this is my config loader
InitPreLoader.load()
from World.WorldManager import WorldManager
from Init.Loaders.InitLoader import InitLoader
from Init.Registry.InitRegistry import InitRegistry
from Utils.Debug import Logger
# ...
if __name__ == '__main__':
# ...
InitLoader.load()
Does it possible to do this in more elegant way and avoid to violate pep8 ?
P.S. If I need to share more code please let me know
UPD: All my classes declared in separate files
This is PreLoader:
from Typings.Abstract.AbstractLoader import AbstractLoader
from Init.Registry.InitRegistry import InitRegistry
from Config.Init.configs import main_config
class InitPreLoader(AbstractLoader):
#staticmethod
def load(**kwargs):
InitRegistry.main_config = main_config
This is Registry (where I store all my initialized data):
from Typings.Abstract.AbstractRegistry import AbstractRegistry
class InitRegistry(AbstractRegistry):
main_config = None
login_server = None
world_server = None
world_observer = None
identifier_region_map = None
region_octree_map = None
Parent of all classes (except AbstractRegistry) is AbstractBase class (it contains mixin):
from abc import ABC
from Config.Mixins.ConfigurableMixin import ConfigurableMixin
class AbstractBase(ConfigurableMixin, ABC):
pass
This mixin works with main_config from InitRegistry.
Also, after PreLoader's load was called, I load rest data with my InitLoader.load() (see first code snapshot):
from Typings.Abstract.AbstractLoader import AbstractLoader
from Init.Registry.InitRegistry import InitRegistry
from Server.Init.servers import login_server, world_server
from World.Observer.Init.observers import world_observer
from World.Region.Init.regions import identifier_region_map, region_octree_map
class InitLoader(AbstractLoader):
#staticmethod
def load(**kwargs):
InitRegistry.login_server = login_server
InitRegistry.world_server = world_server
InitRegistry.world_observer = world_observer
InitRegistry.identifier_region_map = identifier_region_map
InitRegistry.region_octree_map = region_octree_map
Well, for now I have found solution: I moved from Init.Loaders.InitPreLoader import InitPreLoader to separate file and called InitPreLoader.load() there. But I not like this solution, because my PyCharm IDE highlights it as unused import:
import Init.Init.preloader
from World.WorldManager import WorldManager
# ...
Maybe it is possible to improve this solution ? Or maybe another (more elegant) solition exists ?
I am using unittest module for writing tests.
I need to test initialization of the object inside a testcase using different inputs.
For this purpose I am importing the class inside setUp(). But when I try to use the class inside test_*() functions, I get this error - NameError: name 'Example' is not defined
Here is my code sample-
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
import Example
def test_sample_function(self):
e = Example(1,2)
I know that I can simply import the class at top of the script. But I do not want to do that. I need to import it only during setup of the testscript.
Looking for some help here.
import unittest
class TestExample(unittest.TestCase):
def setUp(self):
import Example
self.Example = Example
def test_sample_function(self):
e = self.Example(1,2)
There's no reason to import the module in setUp. The module is still available globally in sys.modules, but you've only bound it to a local name that goes away after setUp returns. Just import it globally.
import unittest
import Example
class TestExample(unittest.TestCase):
def test_sample_function(self):
e = Example(1,2)
I've a main.py file with a block of code like this:
import urtc
import machine
rtc = urtc.DS3231(machine.I2C(scl=machine.Pin(0), sda=machine.Pin(2)))
from func import * #line 4
Now, func.py file which is imported on line 4 has code something like this:
def current_time():
import urtc
import machine
rtc = urtc.DS3231(machine.I2C(scl=machine.Pin(0), sda=machine.Pin(2)))
return urtc.tuple2seconds(rtc.datetime())
In main.py, I'm already importing urtc and machine and defining rtc. Is it possible to eliminate these 3 lines from function current_time():
import urtc
import machine
rtc = urtc.DS3231(machine.I2C(scl=machine.Pin(0), sda=machine.Pin(2)))
It seems redundant as I already have them in main.py global. How can I use them from main.py global instead of importing them again in function current_time()?
You should pass the urtc.DS3231 instance to the current_time function like so:
def current_time(rtc):
return urtc.tuple2seconds(rtc.datetime())
But you still need to import urtc in func.py so that urtc.tuple2seconds is available.
You should use arguments in your function, this is in fact bad design to do it the way you did.
import urtc
import machine
rtc = urtc.DS3231(machine.I2C(scl=machine.Pin(0), sda=machine.Pin(2)))
from func import *
def current_time(rtc):
return urtc.tuple2seconds(rtc.datetime())
current_time(rtc)
I would suggest that you load the dependencies in func.py (if you are not using them anywhere else in main.py, it is a better practice).
I have a function that creates a temporary directory, switches to that temporary directory, performs some work, and then switches back to the original directory. I am trying to write a unit test that tests this. I don't have a problem verifying that the current directory was changed to the temp dir and changed back again, but I'm having a problem verifying that the important stuff took place in between those calls.
My original idea was to abstract the function into three sub functions so that I could test the call order. I can replace each of the three sub functions with mocks to verify that they are called -- however, I am still presented with the issue of verifying the order. On a mock I can use assert_has_calls, but upon what object do I call that function?
Here is the class I'm trying to test:
import shutil
import os
import subprocess
import tempfile
import pkg_resources
class Converter:
def __init__(self, encoded_file_path):
self.encoded_file_path = encoded_file_path
self.unencoded_file_path = None
self.original_path = None
self.temp_dir = None
def change_to_temp_dir(self):
self.original_path = os.getcwd()
self.temp_dir = tempfile.mkdtemp()
os.chdir(self.temp_dir)
def change_to_original_dir(self):
os.chdir(self.original_path)
shutil.rmtree(self.temp_dir)
def do_stuff(self):
pass
def run(self):
self.change_to_temp_dir()
self.do_stuff()
self.change_to_original_dir()
This is as far as I got writing the test case:
def test_converter(self, pkg_resources, tempfile, subprocess, os, shutil):
encoded_file_path = Mock()
converter = Converter(encoded_file_path)
converter.change_to_temp_dir = Mock()
converter.do_stuff= Mock()
converter.change_to_original_dir = Mock()
assert converter.encoded_file_path == encoded_file_path
assert converter.unencoded_file_path is None
converter.run()
Now that I have each function mocked, I can verify THAT they were called, but not in what ORDER. How do I go about doing this?
One workaround would to be to create a separate mock object, attach methods to it and use assert_has_calls() to check the call order:
converter = Converter(encoded_file_path)
converter.change_to_temp_dir = Mock()
converter.do_stuff = Mock()
converter.change_to_original_dir = Mock()
m = Mock()
m.configure_mock(first=converter.change_to_temp_dir,
second=converter.do_stuff,
third=converter.change_to_original_dir)
converter.run()
m.assert_has_calls([call.first(), call.second(), call.third()])