When calling unit test in another directory, it does not import the data I needed.
Directory goes like this:
GenerateValues
--app folder
----process_data.py
----input_files.py
----model_data folder
------data.json
----block_tests
------test_1.py
------test_2.py
--test.py
I need to call unittest in test.py.
In the unit tests under block_tests, I have been calling process_data for assertion. process_data.py calls data.json and use it to do the processing.
input_files.py
folder = "model_data"
data1 = f"{folder}/data.json"
process_data.py
class ProcessData:
def __init__(self):
self.load_data()
def load_data(self):
fn = model.data1
#loads data then to be used in process
test_1.py
class TestData(unittest.TestCase):
#classmethod
def setUpClass(cls) -> None:
cls.data_service = ProcessData()
test.py
loader = unittest.TestLoader()
test_runner = unittest.runner.TextTestRunner()
tests = loader.discover("block_tests")
test_runner.run(tests)
Error is this:
FileNotFoundError: [Errno 2] No such file or directory: 'model_data/data.json'
Pathlib provides some great helpers that will make this a bit easier for you. If you change input_files.py to be:
from pathlib import Path
folder = Path(__file__).parent / "model_data"
data1 = folder / "data.json"
This uses the __file__ attribute to get the path of input_files.py, then uses pathlib.Path to figure out the directory it lives in(.parent). This can then be used to point to the model_data folder, and then the specific json you need to load.
Caveat: I may have the exact paths wrong here because I am reading your directory tree rather than looking at a real one. You may need to tweak the exact path wrangling!
Related
There are already several answered questions to this topic but actually not addressing my problem.
I'm using PyCharm 2021.2 and I want to be able to run unit tests both, individually and as a whole. This is important because if I have many tests and only some of them fail, I usually debug my code and would like to run only my failed tests to see if my debugging was successful. Only then do I want to re-run all the tests.
My setup is as follows:
So, for instance, I want to:
Right-click the folder tests and run all tests in this folder (in my example screenshot, there is only one test_MyClass.py but usually here would be many such tests).
Right-click an individual test, e.g. test_MyClass.py, and run it on its own.
Both possibilities usually work fine. However, when my single tests use some relative paths, for instance, to read some test assets (in my case from the folder containing_folder/tests/testassets), only the option 1) works. The option 2) runs into a FileNotFoundError: No such file or directory.
The code to reproduce this behavior is:
MyClass.py:
class MyClass:
_content = None
def set_content(self, content):
self._content = content
def get_content(self):
return self._content
test_MyClass.py:
import unittest
import io
from ..MyClass import MyClass
class MyClassTests(unittest.TestCase):
myClassInstance = None
#classmethod
def setUpClass(cls):
cls.myClassInstance = MyClass()
def get_file(self, use_case):
path_to_file = "testassets/" + use_case + ".txt"
with io.open(path_to_file, 'r', encoding="utf-8") as file:
file = file.read()
return file
def test_uc_file1(self):
file_content = self.get_file("uc_1")
self.myClassInstance.set_content(file_content)
self.assertEquals("test1", self.myClassInstance.get_content())
def test_uc_file2(self):
file_content = self.get_file("uc_2")
self.myClassInstance.set_content(file_content)
self.assertEquals("test2", self.myClassInstance.get_content())
It seems that path_to_file = "testassets/" + use_case + ".txt" only works as a relative path in the 1) option, but not in the 2) option.
How can I recognize programmatically, which option 1) or 2) I'm starting a test in PyCharm? And which path would then I have to choose for option 2)? I tried ../testassets, ../../testassets, ../../, , ../ but none of them worked for option 2).
Ok, I found how to accomplish what I want.
First of all, I got rid of relative paths when importing. Instead of from ..MyClass import MyClass I use simply from MyClass import MyClass.
Second, my methods setUpClass and get_file now look like this:
#classmethod
def setUpClass(cls):
cls.path = os.path.normpath(os.path.abspath(__file__)) # requires import os
if os.path.isfile(cls.path):
cls.path = os.path.dirname(cls.path)
cls.myClassInstance = MyClass()
def get_file(self, use_case):
path_to_file = self.path + "/testassets/" + use_case + ".txt"
with io.open(path_to_file, 'r', encoding="utf-8") as file:
file = file.read()
return file
The point is that os.path.abspath(__file__) returns a root path of either the directory containing_folder/tests if I choose option 1) to start all tests or the filename containing_folder/tests/test_MyClass.py if I choose option 2) to start a single test. In the if statement
if os.path.isfile(cls.path):
cls.path = os.path.dirname(cls.path)
I generalize both special cases to get the root directory of all the tests and easily find the test assets relative to them.
My Python app contains a subfolder called Tests which I use to run unit tests. All of my files are in the parent folder, which I will call App. The Tests folder contains, say, a test.py file. The App folder contains an app.py file and a file.txt text file.
In my test.py file, I can make my imports like this:
import sys
sys.path.append("PATH_TO_PARENT_DIR")
Say my app.py file contains the following:
class Stuff():
def do_stuff():
with open("file.txt") as f:
pass
Now if I run test.py, I get the following error:
FileNotFoundError: [Errno 2] No such file or directory: 'file.txt'
How can I fix this? Many thanks!
Assuming the file is located in the same folder as your script:
import os
parent_dir = os.path.abspath(os.path.dirname(__file__))
class Stuff():
def do_stuff():
with open(os.path.join(parent_dir, "file.txt")) as f:
pass
Explanation:
__file__ is the path to your script
os.path.dirname get's the directory in which your script sits
os.path.abspath makes that path absolute instead of relative (just in case relative paths mess your script up, it's good practice)
Then all we need to do is combine your parent_dir with the file, we do that using os.path.join.
Read the docs on os.path methods here: https://docs.python.org/3/library/os.path.html
A more explicit version of this code can be written like this, if that helps:
import os
script_path = __file__
parent_dir = os.path.dirname(script_path)
parent_dir_absolute = os.path.abspath(parent_dir)
path_to_txt = os.path.join(parent_dir_absolute, 'file.txt')
The open function looks for the file in the same folder as the script that calls the open function. So, your test.py looks in the tests folder, not the app folder. You need to add the full path to the file.
open('app_folder' + 'text.txt')
or move the test.py file in the same folder as text.txt
I've something like this in my program:
A main script main.py inside a folder named 'OpenFileinaModule'. There's a folder called 'sub' inside it with a script called subScript.py and a file xlFile.xlsx, which is opened by subScript.py.
OpenFileinaModule/
main.py
sub/
__init__.py (empty)
subScript.py
xlFile.xlsx
Here is the code:
sub.Script.py:
import os, openpyxl
class Oop:
def __init__(self):
__file__='xlFile.xlsx'
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
print os.path.join(__location__, __file__)
self.wrkb = openpyxl.load_workbook(os.path.join(__location__,
__file__),read_only=True)
main.py:
import sub.subScript
objt=sub.subScript.Oop()
When I execute main.py, I get the error:
IOError: [Errno 2] No such file or directory: 'C:\\Users\\...\\OpenFileInaModule\\xlFile.xlsx'
It jumps the sub folder...
I've tried
__file__='sub/xlFile.xlsx'
But then the "sub" folder is duplicated:
IOError: [Errno 2] No such file or directory: 'C:\\Users\\...\\OpenFileInaModule\\sub\\sub/xlFile.xlsx'
How to open xlFile.xlsx with subScript.py from main.py?
you're overriding __file__ with __file='xlFile.xlsx', do you mean to do this?
I think you want something like
import os
fname = 'xlFile.xlsx'
this_file = os.path.abspath(__file__)
this_dir = os.path.dirname(this_file)
wanted_file = os.path.join(this_dir, fname)
I'd suggest always using the absolute path for a file, especially if you're on windows when a relative path might not make sense if the file is on a different drive (I actually have no idea what it would do if you asked it for a relative path between devices).
Please avoid using __file__ and __location__ to name your variables, these are more like builtin variables which might cause a confusion.
Note something here:
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
You have not included sub directory and the above joins only the CWD + os.path.dirname(__file__). This doesn't get you to the file. Please read the documentation of os.path.dirname: os.path.dirname(__file__) returns an empty string here.
def __init__(self):
file = 'xlFile.xlsx'
location = os.path.join('sub', file)
location = os.path.abspath(location) # absolute path to file
location = os.path.realpath(location) # rm symbolic links in path
self.wrkb = openpyxl.load_workbook(location)
So I have two main portion of code:
Generates an extensive collection of config files in a directory.
Runs a single config file generated by the previous.
I want to run a test, where first I execute code1 and generate all files and than for each config file run code2 and verify that the results if good.
My attempt so far was:
#pytest.fixture(scope='session')
def pytest_generate_tests(metafunc, tmpdir_factory):
path = os.path.join(tmpdir_factory, "configs")
gc.main(path,
gc.VARIANTS, gc.MODELS,
default_curvature_avg=0.0,
curvature_avg_variation=0.9,
default_gradient_avg=0.0,
gradient_avg_variation=0.9,
default_inversion="approximate",
vary_inversion=False,
vary_projections=True)
params = []
for model in os.listdir(path):
model_path = os.path.join(path, model)
for dataset in os.listdir(model_path):
dataset_path = os.path.join(model_path, dataset)
for file_name in os.listdir(dataset_path):
config_file = os.path.join(dataset_path, file_name)
folder = os.path.join(dataset_path, file_name[:-5])
tmpdir_factory.mktemp(folder)
params.append(dict(config_file=config_file, output_folder=folder))
metafunc.addcall(funcargs=dict(config_file=config_file, output_folder=folder))
def test_compile_and_error(config_file, output_folder):
final_error = main(config_file, output_folder)
assert final_error < 0.9
However, the tmpdir_factory fixture does not work for the pytest_generate_tests method. My questions is how to achieve my goal by generating all of the tests?
First and most importantly,
pytest_generate_tests is meant to be a hook in pytest and not a name for a fixture function. Get rid of the #pytest.fixture before it and have another look in its docs. Hooks should be written in the conftest.py file or a plugin file, and are collected automatically according to the pytest_ prefix.
Now for your matter:
Just use a temporary directory manually using:
import tempfile
import shutil
dirpath = tempfile.mkdtemp()
inside pytest_generate_tests. Save dirpath in a global in the conftest,
and delete in pytest_sessionfinish using
# ... do stuff with dirpath
shutil.rmtree(dirpath)
Source: https://stackoverflow.com/a/3223615/3858507
Remember that if you have more than one test case, pytest_generate_tests will be called for each one. So you better save all your tempdirs in some list and delete all of them in the end. In contrast, if you only need one tempdir than think about using the hook pytest_sesssionstart to create it there and use it later.
i'm having a bit of a path issue when using configobj for python. i'm wondering if there's a way to not use an absolute path in my helper file. For example, instead of:
self.config = ConfigObj('/home/thisuser/project/common/config.cfg')
i want to use something like:
self.config = ConfigObj(smartpath+'/project/common/config.cfg')
Background:
i've placed my config file in a common directory along side a helper class and a utility class:
common/config.cfg
common/helper.py
common/utility.py
the helper class has a method that returns the values within the section of the config. the code is as such:
from configobj import ConfigObj
class myHelper:
def __init__(self):
self.config = ConfigObj('/home/thisuser/project/common/config.cfg')
def send_minion(self, race, weapon):
minion = self.config[race][weapon]
return minion
the utility file imports the helper file and the utility file is called by a bunch of different classes stationed in different folders of my project:
from common import myHelper
class myUtility:
def __init__(self):
self.minion = myHelper.myHelper()
def attack_with_minion(self, race, weapon)
my_minion = self.minion.send_minion(race, weapon)
#... some common code used by all
my_minion.login()
the following files import the utility file and calls the method:
/home/thisuser/project/folder1/forestCastle.py
/home/thisuser/project/folder2/secondLevel/sandCastle.py
/home/thisuser/project/folder3/somewhere/waterCastle.py
self.common.attack_with_minion("ogre", "club")
if i don't use an absolute path and i run forestCastle.py it looks for the config in /home/thisuser/project/folder1/ and i want it to look for it in project/common/ because /home/thisuser will change
You can calculate a new absolute path based on a module filename instead:
import os.path
from configobj import ConfigObj
BASE = os.path.dirname(os.path.abspath(__file__))
class myHelper:
def __init__(self):
self.config = ConfigObj(os.path.join(BASE, 'config.cfg'))
__file__ is the filename of the current module, so for helper.py that would be /home/thisuser/project/common/helper.py; os.path.abspath() makes sure it is a absolute path, and os.path.dirname removes the /helper.py filename to leave you with an absolute path to the 'current' directory.
I'm having a bit of a hard time following what you actually want. However, to expand the path to your home directory in an OS-agnostic fashion, you can use os.path.expanduser:
self.config = ConfigObj(os.path.expanduser('~/project/common/config.cfg'))