Question on unit testing
Goal: The goal is to use pyUnit in testCalc.py to unit test the simpleCalc object in calculator.py.
Problem: I cannot successfully import the simpleCalc object from calculator.py into testCalc.py when testCalc is run from a separate directory in the project.
Background: The unit test in testCalc.py runs perfectly fine when it's included in the same directory as calculator.py, but when I move it into a separate folder and try to import the simpleCalc object defined in calculator.py, I get an error. I am trying to learn how to use the pyUnit unit testing framework in a simple project and I'm clearly missing something basic about how to import modules for unit testing in a hierarchical directory structure. The basic calculator_test project described below is a simple project that I created to practice. You can see all the posts I've gone through already at the end of this post.
Ultimate Question: How can I import the simpleCalc object into testCalc.py with the directory hierarchy described below?
Github: https://github.com/jaybird4522/calculator_test/tree/unit_test
Here's my directory structure:
calculator_test/
calculatorAlgo/
__init__.py
calculator.py
test_calculatorAlgo/
__init__.py
testCalc.py
testlib/
__init__.py
testcase.py
Here's the calculator.py file, which describes the simpleCalc object I want to unit test:
# calculator.py
class simpleCalc(object):
def __init__(self):
self.input1 = 0
self.input2 = 0
def set(self, in1, in2):
self.input1 = in1
self.input2 = in2
def subtract(self, in1, in2):
self.set(in1, in2)
result = self.input1 - self.input2
return result
Here's the testCalc.py file, which contains the unit tests:
# testCalc.py
import unittest
from calculatorAlgo.calculator import simpleCalc
class testCalc(unittest.TestCase):
# set up the tests by instantiating a simpleCalc object as calc
def setUp(self):
self.calc = simpleCalc()
def runTest(self):
self.assertEqual(self.calc.subtract(7,3),4)
if __name__ == '__main__':
unittest.main()
I have been running the unit test file with the simple command:
testCalc.py
What I've attempted so far
First Attempt
I tried simply importing the simpleCalc object based on where it's located in the directory structure:
# testCalc.py
import unittest
from .calculatorAlgo.calculator import simpleCalc
class testCalc(unittest....
And got this error:
ValueError: Attempted relative import in non-package
Second Attempt
I tried importing it without relative references:
# testCalc.py
import unittest
import simpleCalc
class testCalc(unittest....
And got this error:
ImportError: No module named simpleCalc
Third Attempt
Based on this post, http://blog.aaronboman.com/programming/testing/2016/02/11/how-to-write-tests-in-python-project-structure/, I tried creating a separate base class called testcase.py which could do the relative imports.
# testcase.py
from unittest import TestCase
from ...calculator import simpleCalc
class BaseTestCase(TestCase):
pass
And changed my imports in testCalc.py
# testCalc.py
import unittest
from testlib.testcase import BaseTestCase
class testCalc(unittest....
And got this error:
ValueError: Attempted relative import beyond toplevel package
Other Resources
Here are some of the posts I've worked through to no avail:
Import a module from a relative path
python packaging for relative imports
How to fix "Attempted relative import in non-package" even with __init__.py
Python importing works from one folder but not another
Relative imports for the billionth time
Ultimately, I feel that I'm just missing something basic, even after a lot of research. This feels like a common setup, and I'm hoping someone can tell me what I'm doing wrong, and that it might help others avoid this problem in the future.
Inside the testCalc.py file, add the following.
import sys
import os
sys.path.append(os.path.abspath('../calculatorAlgo'))
from calculator import simpleCalc
Related
i'm struggling with something that feels like it should be simple.
my current dir looks like this:
root/
└─ __init__.py (tried with it and without)
└─ file_with_class.py
└─ tests_folder/
└─ __init__.py (tried with it and without)
└─ unittest_for_class.py
unittest_for_class.py needs to import the class from file_with_class to test it, i tried to import it in various ways i found online but i just keep getting errors like:
(class name is same as file name lets say its called file_with_class)
File "tests_folder/unittest_for_class.py", line 3, in <module>
from ..file_with_class import file_with_class
ValueError: Attempted relative import in non-package
File "tests_folder/unittest_for_class.py", line 3, in <module>
from file_with_class import file_with_class
ImportError: No module named file_with_class
and others..
what is the correct way to import a class from a .py file that is in the parent folder ?
As a short explanation
import * from ..parent works if your program started at the parent level.
You import submodules which can have cross relations to other submodules or files in the package -> They are only relative inside a package not the os structure.
Option 1
you actually enter via a script in your parent folder and import your mentioned file as a submodule. Nicest, cleanest and intended way, but then your file is no standalone.
Option 2 - Add the parent dictionary to your path
sys.path.append('/path/to/parent')
import parent
This is a little bit dirty as you now have an extra path for your imports but still one of most easiest ones without much trickery.
Further Options and theory
There are quite a few posts here covering this topic relative imports covers quite a few good definitions and concepts in the answers.
Option 3 - Deprecated and not future proof importlib.find_loader
import os
import importlib
current = os.getcwd() # for rollback
os.chdir("..") # change to arbitrary path
loader = importlib.find_loader("parent") # load filename
assert loader
parent = loader.load_module() # now this is your module
assert parent
os.chdir(current) # change back to working dictionary
(Half of an) Option 4
When working with an IDE this might work, Spyder allows the following code. Standard python console does NOT.
import os
current = os.getcwd()
os.chdir("..")
import parent
os.chdir(current)
Following up on #Daraan's answer:
You import submodules which can have cross relations to other submodules or files in the package -> They are only relative inside a package not the os structure.
I've written an experimental, new import library: ultraimport which allows to do just that, relative imports from anywhere in the file system. It will give you more control over your imports.
You could then write in your unittest_for_class.py:
import ultraimport
MyClass = ultraimport("__dir__/../file_with_class.py", "MyClass")
# or to import the whole module
file_with_class = ultraimport("__dir__/../file_with_class.py")
The advantage is that this will always work, independent of sys.path, no matter how you run your script and all the other things that were mentioned.
You can add the parent folder to the search path with sys.path.append() like so:
import sys
sys.path.append('/path/to/parentdir')
from file_with_class import file_with_class
...
See also the tutorial for how Python modules and packages are handled
Just keep from file_with_class import file_with_class. Then run python -m test_folder.unittest_for_class. This supports running the script as if it is a module.
I am writing test cases for a module module.py that imports from another module legacy.py. The legacy.py reads os.environ["some_var"] at module level. When I am trying to run test cases for module.py, they are failing with KeyError for some_var in os.environ.
This is how code looks like:
module.py
from legacy import db
def fun():
pass
legacy.py
import os
env = os.environ["some_var"]
class db:
def __init__(self):
pass
When running test cases for module.py, I am getting KeyError: 'some_var'.
I tried patching os (at module level also putting it before importing from module.py in test file) but the import statement is run before it could be patched. I tried looking for similar question on StackOverflow but didn't find the exact problem I am facing here.
How can I mock it? Or any other suggestion to be able to run the test cases. Assume that I cannot modify the legacy.py file.
You could use the mock.patch.dict. There is an example in official doc.
Here is a fully functional example based in your question.
module.py
import os
def get_env_var_value():
env_var = os.environ['PATH']
return env_var
print(get_env_var_value())
test_module.py
from unittest.mock import patch
from unittest import main, TestCase
import module
class MyClassTestCase(TestCase):
#patch.dict('os.environ', {'PATH': '/usr/sbin'})
def test_get_env_var_value(self):
self.assertEqual(module.get_env_var_value(), '/usr/sbin')
if __name__ == '__main__':
main()
Doesn't matter if the environment var is loaded in outside or inside the class/method scope, because you will mock for all test scope.
I was working on a project with this type of folder tree:
-main.py
-Message_handlers
-__init__.py
-On_message.py
-Other_modules.py
-Exceptions_handler
-__init__.py
-Run_as_mainException.py
Message_handlers and Exceptions_handler are two packages, now my problem is that from inside the module On_message.py I can't import the class Run_as_mainException (inside of the module Run_as_mainException.py) using this code
# This is On_message.py
from ..Exceptions_handler.Run_as_mainException import Run_as_main_Exception
This line gives the error: ImportError: attempted relative import with no known parent package
p.s. Every file has a class inside with the same name of the file, example:
# This is Run_as_mainExample.py
class Run_as_mainExample:
def __init__(self):
# rest of the code
Can anyone help me please?
You have to assume that you're running everything from the level where main.py is located. This means, imagine that you execute python main.py, and you want to import Run_as_main_Exception from main.py, what should you do?
from Exceptions_handler.Run_as_mainException import Run_as_main_Exception
Try using that line in your On_message.py file, and you shouldn't have any problem when running your script from that exact location (remember, the same level where main.py is located)
I developed a crawler and it's unit-tests (mainly to validate XPATHs). I want to run specific unit-tests before script execution in order to be sure that HTML structure has not changed and existing XPATHs still working. I don't want the output of unit-test, just a flag: passed or failed.
for example:
tests.py:
import unittest
class CrwTst(unittest.TestCase):
def test_1(self):
[..]
crawler.py
class Crawler(object):
def action_1(self):
[..]
and I want to work like:
if CrwTst.test_1() is True:
Crawler.action_1()
You could potentially do this:
crawler.py
import unittest
from tests import CrwTst
if unittest.TextTestRunner().run(CrwTst('test_1')).wasSuccessful():
Crawler.action_1()
Note however that you may run into an issue with circular imports, because your test presumably already depends on Crawler, and what you are looking to do will make the Crawler depend on the test. This will likely manifest itself as ImportError: cannot import name CrwTst.
To resolve that, you can dynamically import the CrwTst.
crawler.py
import unittest
def function_that_runs_crawler():
from tests import CrwTst # Dynamically import to resolve circular ref
if unittest.TextTestRunner().run(CrwTst('test_1')).wasSuccessful():
Crawler.action_1()
Consider the following directory structure:
lib/
markov.py
solver.py
test.py
markov.py has the following code: super() throwing an error in Sublime Text, works in PyCharm/Terminal
solver.py only contains a bunch of mathematical functions, here's an example of one:
def mm1_busy_idle(arrival, service):
return round(1 - (arrival / service), 4)
Now, when I try to do import solver in markov.py, PyCharm tells me that there's No module named solver. However, when I do import test it does it just well and I can run the test function from there:
def test():
print("test!")
Also, when I hover over test.test() PyCharm shows me the following tooltip: Cannot find reference 'test' in '__init__.py'.
My question is: Why can I import test and run the function test() but I can't import solver in markov.py?
add one file (__init__.py empty file) in lib,
lib/
__init__.py
markov.py
solver.py
test.py
An ultimate solution:
import sys
sys.path.append("/path/to/lib")