i'm having a bit of a path issue when using configobj for python. i'm wondering if there's a way to not use an absolute path in my helper file. For example, instead of:
self.config = ConfigObj('/home/thisuser/project/common/config.cfg')
i want to use something like:
self.config = ConfigObj(smartpath+'/project/common/config.cfg')
Background:
i've placed my config file in a common directory along side a helper class and a utility class:
common/config.cfg
common/helper.py
common/utility.py
the helper class has a method that returns the values within the section of the config. the code is as such:
from configobj import ConfigObj
class myHelper:
def __init__(self):
self.config = ConfigObj('/home/thisuser/project/common/config.cfg')
def send_minion(self, race, weapon):
minion = self.config[race][weapon]
return minion
the utility file imports the helper file and the utility file is called by a bunch of different classes stationed in different folders of my project:
from common import myHelper
class myUtility:
def __init__(self):
self.minion = myHelper.myHelper()
def attack_with_minion(self, race, weapon)
my_minion = self.minion.send_minion(race, weapon)
#... some common code used by all
my_minion.login()
the following files import the utility file and calls the method:
/home/thisuser/project/folder1/forestCastle.py
/home/thisuser/project/folder2/secondLevel/sandCastle.py
/home/thisuser/project/folder3/somewhere/waterCastle.py
self.common.attack_with_minion("ogre", "club")
if i don't use an absolute path and i run forestCastle.py it looks for the config in /home/thisuser/project/folder1/ and i want it to look for it in project/common/ because /home/thisuser will change
You can calculate a new absolute path based on a module filename instead:
import os.path
from configobj import ConfigObj
BASE = os.path.dirname(os.path.abspath(__file__))
class myHelper:
def __init__(self):
self.config = ConfigObj(os.path.join(BASE, 'config.cfg'))
__file__ is the filename of the current module, so for helper.py that would be /home/thisuser/project/common/helper.py; os.path.abspath() makes sure it is a absolute path, and os.path.dirname removes the /helper.py filename to leave you with an absolute path to the 'current' directory.
I'm having a bit of a hard time following what you actually want. However, to expand the path to your home directory in an OS-agnostic fashion, you can use os.path.expanduser:
self.config = ConfigObj(os.path.expanduser('~/project/common/config.cfg'))
Related
When calling unit test in another directory, it does not import the data I needed.
Directory goes like this:
GenerateValues
--app folder
----process_data.py
----input_files.py
----model_data folder
------data.json
----block_tests
------test_1.py
------test_2.py
--test.py
I need to call unittest in test.py.
In the unit tests under block_tests, I have been calling process_data for assertion. process_data.py calls data.json and use it to do the processing.
input_files.py
folder = "model_data"
data1 = f"{folder}/data.json"
process_data.py
class ProcessData:
def __init__(self):
self.load_data()
def load_data(self):
fn = model.data1
#loads data then to be used in process
test_1.py
class TestData(unittest.TestCase):
#classmethod
def setUpClass(cls) -> None:
cls.data_service = ProcessData()
test.py
loader = unittest.TestLoader()
test_runner = unittest.runner.TextTestRunner()
tests = loader.discover("block_tests")
test_runner.run(tests)
Error is this:
FileNotFoundError: [Errno 2] No such file or directory: 'model_data/data.json'
Pathlib provides some great helpers that will make this a bit easier for you. If you change input_files.py to be:
from pathlib import Path
folder = Path(__file__).parent / "model_data"
data1 = folder / "data.json"
This uses the __file__ attribute to get the path of input_files.py, then uses pathlib.Path to figure out the directory it lives in(.parent). This can then be used to point to the model_data folder, and then the specific json you need to load.
Caveat: I may have the exact paths wrong here because I am reading your directory tree rather than looking at a real one. You may need to tweak the exact path wrangling!
Say I have a Directory class which accepts a list of other sub-directories as an argument:
class File:
pass
class Directory:
def __init__(self, directories: List[Directory], files: List[File]):
^^^^^^^^^
self.directories = directories
self.files = files
I am getting an error saying "Directory" is not defined. This code works just fine if I don't pass the types, but how do you handle this use-case in python?
What you need is a recursive typing definition. This can be achieved in Python by providing a string with the name of the type, rather than the type itself.
class File:
pass
class Directory:
def __init__(self, directories: List['Directory'], files: List[File]):
self.directories = directories
self.files = files
An alternative is:
from __future__ import annotations
class File:
pass
class Directory:
def __init__(self, directories: list[Directory], files: list[File]):
self.directories = directories
self.files = files
The import from __future__ causes all types to converted to their string equivalents. And in 3.10, you can use plain old list rather than List
There are already several answered questions to this topic but actually not addressing my problem.
I'm using PyCharm 2021.2 and I want to be able to run unit tests both, individually and as a whole. This is important because if I have many tests and only some of them fail, I usually debug my code and would like to run only my failed tests to see if my debugging was successful. Only then do I want to re-run all the tests.
My setup is as follows:
So, for instance, I want to:
Right-click the folder tests and run all tests in this folder (in my example screenshot, there is only one test_MyClass.py but usually here would be many such tests).
Right-click an individual test, e.g. test_MyClass.py, and run it on its own.
Both possibilities usually work fine. However, when my single tests use some relative paths, for instance, to read some test assets (in my case from the folder containing_folder/tests/testassets), only the option 1) works. The option 2) runs into a FileNotFoundError: No such file or directory.
The code to reproduce this behavior is:
MyClass.py:
class MyClass:
_content = None
def set_content(self, content):
self._content = content
def get_content(self):
return self._content
test_MyClass.py:
import unittest
import io
from ..MyClass import MyClass
class MyClassTests(unittest.TestCase):
myClassInstance = None
#classmethod
def setUpClass(cls):
cls.myClassInstance = MyClass()
def get_file(self, use_case):
path_to_file = "testassets/" + use_case + ".txt"
with io.open(path_to_file, 'r', encoding="utf-8") as file:
file = file.read()
return file
def test_uc_file1(self):
file_content = self.get_file("uc_1")
self.myClassInstance.set_content(file_content)
self.assertEquals("test1", self.myClassInstance.get_content())
def test_uc_file2(self):
file_content = self.get_file("uc_2")
self.myClassInstance.set_content(file_content)
self.assertEquals("test2", self.myClassInstance.get_content())
It seems that path_to_file = "testassets/" + use_case + ".txt" only works as a relative path in the 1) option, but not in the 2) option.
How can I recognize programmatically, which option 1) or 2) I'm starting a test in PyCharm? And which path would then I have to choose for option 2)? I tried ../testassets, ../../testassets, ../../, , ../ but none of them worked for option 2).
Ok, I found how to accomplish what I want.
First of all, I got rid of relative paths when importing. Instead of from ..MyClass import MyClass I use simply from MyClass import MyClass.
Second, my methods setUpClass and get_file now look like this:
#classmethod
def setUpClass(cls):
cls.path = os.path.normpath(os.path.abspath(__file__)) # requires import os
if os.path.isfile(cls.path):
cls.path = os.path.dirname(cls.path)
cls.myClassInstance = MyClass()
def get_file(self, use_case):
path_to_file = self.path + "/testassets/" + use_case + ".txt"
with io.open(path_to_file, 'r', encoding="utf-8") as file:
file = file.read()
return file
The point is that os.path.abspath(__file__) returns a root path of either the directory containing_folder/tests if I choose option 1) to start all tests or the filename containing_folder/tests/test_MyClass.py if I choose option 2) to start a single test. In the if statement
if os.path.isfile(cls.path):
cls.path = os.path.dirname(cls.path)
I generalize both special cases to get the root directory of all the tests and easily find the test assets relative to them.
I have a simple python project with the following directory structure:
sample-project/
main/
config/
config.yaml
docker/
Dockerfile
tests/
__init__.py
sometests.py
config.py
run.py
__init__.py
requirements.txt
README.rst
In the config.py file, I have:
import yaml
class Config:
def __init__(self):
with open("config/config.yaml", 'r') as ymlfile:
cfg = yaml.load(ymlfile)
self.host = cfg["mysql"]["host"]
self.database = cfg["mysql"]["database"]
self.user = cfg["mysql"]["user"]
self.password = cfg["mysql"]["password"]
my_config = Config()
In the run.py file, I have an import statement as follows:
from main.config import my_config
when I run using command line: python3.5 run.py, I get the error:
from main.config import my_config
ImportError: No module named 'main'
But when I add this in run.py imports it works:
import sys
sys.path.append('/home/tomas/sample-project/')
Is there any better way to make it work rather than give absolute path or any other way? Please give some example :)
Generally, never ever touch sys.path from within your program.
Since main/ (not the best name) contains an __init__.py, we'll consider it a package, so let's make it runnable as one. This way imports will be considered correctly.
Add main/__main__.py with e.g.
from main.run import run
if __name__ == "__main__":
run()
Then run your app with
python -m main
instead of
python main/main.py
when mentioning below
from main.config import my_config
keep dir path also
eg:
from sample-project/main.config import my_config
assuming you are executing from /home/tomas
Although I share AKX's sentiment that editing your path is something to avoid I would generally suggest the following method:
import sys
from os.path import dirname
abs_dir = dirname(__file__)
sys.path.append(abs_dir)
The __file__ variable provides access to the location of the file in which it is accessed. This allows you to get an absolute path to any file in your project.
I have a python module I built myself and it's located in /usr/local/lib/python3.4/site-packages/my_module1. In the module I have this:
class Class1(object);
def __init__(self, file_name):
self.file_name = file_name # how can I get the full path of file_name?
How do I get the full of file_name? That is, if it's just a file name without a path, then append the current folder from where the module is being called. Otherwise, treat the file name as a full path.
# /users/me/my_test.py
from module1 import Class1
c = Class1('my_file1.txt') # it should become /users/me/my_file1.txt
c1 = Class1('/some_other_path/my_file1.txt') # it should become /some_other_path/my_file1.txt/users/me/my_file1.txt
Update: Sorry about that. I mis-read your question. All you need to do is pass filename so os.path.abspath().
Example:
import os
filename = os.path.abspath(filename)
To cater for your 2nd case (which I find quite weird):
import os
if os.path.isabs(filenaem):
filename os.path.join(
os.path.join(
filename,
os.getcwd()
),
os.path.basename(filename)
)