I have the following directory structure:
my_program/
foo.py
__init__.py # empty
conf/
config.cfg
__init__.py
In foo.py I have this:
import sys
#sys.path.append('conf/')
import ConfigParser
config = ConfigParser.ConfigParser()
config.read( 'conf/config.cfg' )
In conf/__init__.py I have
__all__ = ["config.cfg"]
I get this error in foo.py that I can fix by giving the full path but not when I just put conf/config.cfg but I want the relative path to work:
ConfigParser.NoSectionError
which actually means that the file can't be loaded (so it can't read the section).
I've tried commenting/un-commenting sys.path.append('conf/') in foo.py but it doesn't do anything.
Any ideas?
Paths are relative to the current working directory, which is usually the directory from which you run your program (but the current directory can be changed by your program [or a module] and it is in general not the directory of your program file).
A solution consists in automatically calculating the path to your file, through the __file__ variable that the Python interpreter creates for you in foo.py:
import os
config.read(os.path.join(os.path.dirname(__file__), 'conf', 'config.cfg'))
Explanation: The __file__ variable of each program (module) contains its path (possibly relative to the current directory when it was loaded, I guess—I could not find anything conclusive in the Python documentation—, which happens for instance when foo.py is imported from its own directory).
This way, the import works correctly whatever the current working directory, and wherever you put your package.
PS: side note: __all__ = ["config.cfg"] is not what you want: it tells Python what symbols (variables, functions) to import when you do from conf import *. It should be deleted.
PPS: if the code changes the current working directory between the time the configuration-reading module is loaded and the time you read the configuration file, then you want to first store the absolute path of your configuration file (with os.path.abspath()) before changing the current directory, so that the configuration is found even after the current directory change.
you can use os package in python for importing an absolute or relative path to the configuration file. Here is a working example with the relative path (we suppose that config.ini in the folder name configuration that is in the subdirectory of your python script folder):
import configparser
import os
path_current_directory = os.path.dirname(__file__)
path_config_file = os.path.join(path_current_directory, 'configuration', config.ini)
config = configparser.ConfigParser()
config.read(path_config_file)
Just load the path from a Variable and why so complex?
from configparser import ConfigParser
prsr=ConfigParser()
configfile="D:\Dummy\DummySubDir\Dummy.ini"
prsr.read(configfile)
print(prsr.sections())
Related
I have a directory like this:
-RootFolder/
- lib/
- file.py
- source/
- book.py
I'm at the book.py, how do I import a class from the file.py?
When you are running book.py, only it's directory is gonna be added to the sys.path automatically. You need to make sure that the path to the RootFolder directory is in sys.path(this is where Python searches for module and packages) then you can use absolute import :
# in book.py
import sys
sys.path.insert(0, r'PATH TO \RootFolder')
from lib.file import ClassA
print(ClassA)
Since you added that directory, Python can find the lib package(namespace package, since it doesn't have __init__.py)
If you start your program from RootFolder (e.g. python -m source.book), the solution by Irfan wani should be perfectly fine. However, if not, then some hacking is required, to let Python know lib also contains relevant code:
import sys
from pathlib import Path
sys.path.append(str(Path(__file__).parent.parent / "lib"))
import file
from lib.file import some_module
I think this should work fine if the rootfolder is added to path.
If not, we need to do that first as shown by #Amadan and #SorousH Bakhtiary (just wanted to add some explanation);
import sys
from pathlib import Path
sys.path.append(str(Path(__file__).parent.parent))
So here, Path(__file__) gives the path to the file where we are writing this code and .parent gives its parent directory path and so on.
Thus, str(Path(__file__).parent.parent) gives us path to the RootFolder and we are just adding that to the path.
I have project that I decided to divide into subfolders. My main module main.py imports other module mod1.py from a subfolder. The imported module mod1.py uses images that are located in another subfolder and refers to them relatively. I can't use absolute path from the drive, but I know the relative path from the beginning of the project structure.
The situation is illustrated below somehow
project
├── main.py
└───subfolder
├───mod1.py
└───pictures
└───pic1.png
So there's a line in mod1.py:
image = Image.open("./pictures/pic1.png")
and when I import mod1.py in main.py and run the program I get an error:
FileNotFoundError: [Errno 2] No such file or directory: './pictures/pic1.png'
How to access those pictures when I run main.py and import a module that relatively refers to them?
I have __init__.py file in the subfolder and all the imports are working.
try this
try:
image = Image.open("pictures/pic1.png")
except FileNotFoundError:
image = Image.open("subfolder/pictures/pic1.png")
so python will try the first path, if it fails, it will try the second, which looks for the main.py file
While Jorge's answer may work in some cases I suggest understanding why yours does not work and look how other people have solved this problem. Lets take a very simple example.
project
dir1
test.py
dir2
test2.py
Lets assume the full path to my project directory is located at /Users/sstacha/tmp/test_python/ and my 2 test files contain the following
test.py
import os
from pathlib import Path
from dir2.test2 import function2
def function1():
path = os.getcwd()
print(f"function1.os.cwd(): {path}")
DIR = Path(__file__).resolve()
print(f"function1.pathlib_path: {DIR}")
function1()
function2()
test2.py
import os
from pathlib import Path
def function2():
path = os.getcwd()
print(f"function2.os.cwd(): {path}")
DIR = Path(__file__).resolve()
print(f"function2.pathlib_path: {DIR}")
if i execute python dir1/test.py from the project directory I get the following output:
$ python dir1/test.py
function1.os.cwd(): /Users/sstacha/tmp/test_python
function1.pathlib_path: /Users/sstacha/tmp/test_python/dir1/test.py
function2.os.cwd(): /Users/sstacha/tmp/test_python
function2.pathlib_path: /Users/sstacha/tmp/test_python/dir1/dir2/test2.py
if I instead cd to dir1 and execute python test.py I get this output:
$ python test.py
function1.os.cwd(): /Users/sstacha/tmp/test_python/dir1
function1.pathlib_path: /Users/sstacha/tmp/test_python/dir1/test.py
function2.os.cwd(): /Users/sstacha/tmp/test_python/dir1
function2.pathlib_path: /Users/sstacha/tmp/test_python/dir1/dir2/test2.py
So your actual problem is that os.cwd() will always be set to whatever directory the os was set to when the python command was run. I also included a line from how the Django settings file handles this issue for python >= 3.4. As you can see, this approach will be relative to whatever file your function is defined in which should be a better / more portable solution regardless of what directory the python executable is called from.
Hopefully that helps you understand your issue better and a possible solution that might work for you.
I realized I never really fully answered the question here would be an example:
DIR = Path(__file__).resolve().parent
image = Image.open(DIR+"/pictures/pic1.png")
By the way if you are using a version earlier than 3.4 this is how the Django settings file approached it:
DIR = os.path.dirname(os.path.abspath(__file__))
image = Image.open(DIR+"/pictures/pic1.png")
I need some help with working with a folder structure in python. I was given an structure like this:
/main-folder
/assets
somefiles.txt
/integrations
/module-folder
__init__.py
ingestion.py
__init__.py
models.py
Inside ingestion.py I have:
import os
from models import MyModel
PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
some_function()
some_processing()
if __name__ == "__main__":
some_function()
Both __init__.py mentioned above are empty.
So I need to process some info and use the models module to store them. when trying to execute intestion.py directly from its dir it says: No module named 'models'. So I'm guessing I have to execute the whole thing as a package. I have no idea how should I import a module located above the package and can't touch the structure.
Any help woud be appreciated.
What you have to do is to add the module's directory to the PYTHONPATH environment variable. If you don't want to do this however, You can modify the sys.path list in your program where the Python interpreter searches for the modules to import, the python documentation says:
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended.
Knowing this, you can do the following in your program:
import sys
# Add the main-folder folder path to the sys.path list
sys.path.append('/path/to/main-folder/')
# Now you can import your module
from main-folder import models
# Or just
import main-folder
I wrote my own module (i.e. my_module.py). This is accessing a database located in the same directory by just opening it with the path 'database.db'. But when I am using my module, the 'database.db' references the directory im using my module in.
For example I have following directory structur:
/main
/my_module
my_module.py
database.db
script.py
So if im now directly using 'database.db' in my_module.py it points towards /main/my_module/database.db.
But if I am using my_module inside script.py it points towards /main/database.db which causes an error.
So my question is: How is it possible to have a path pointing relatively to the modules script my_module.py? (so I can use it in any place) Thanks a lot for answers!
From script.py try using os.path to get the script directory. Then you can create a relative path to the database file:
import os
maindir = os.path.dirname(__file__)
my_module_dir = os.path.join(maindir, 'my_module')
db_path = os.path.join(my_module_dir, 'database.db')
A script at /foo/bar.py tries to run a second script /main.py using the subprocess module. Although main.py runs fine with python main.py in the Windows command prompt, running bar.py which calls main.py causes an error
ConfigParser.NoSectionError: No section: 'user'
Why is there now a problem with the path to settings.ini, and how can we fix it?
~/settings.ini
[user]
id: helloworld
~/foo/bar.py
subprocess.Popen([sys.executable, "../main.py"])
~/main.py
Config = ConfigParser.ConfigParser()
Config.read("settings.ini")
userId = Config.get('user', 'id')
If settings.ini is presumed to be in the same directory as main.py you can deduce its full path from __file__ and read the settings using the full path.
main.py:
import os
ini_path = os.path.join(os.path.dirname(__file__), "settings.ini")
Config = ConfigParser.ConfigParser()
Config.read(ini_path)
Check that the read() method returns a non-empty list otherwise it means that settings.ini is not found. Relative paths are resolved relative to the current working directory (place where you've run python), not script's directory (where bar.py is stored).
You should probably use appdirs.user_config_dir(), to get the directory where to put user's configuration files. Different OSes such as Windows, macOS, Linux distributions may use different defaults. appdirs follows conventions that are appropriate for a given OS.
If you want to get data stored in a file that is packaged with your installed Python modules then you could use pkgutil.get_data() or setuptools' pkg_resources.resource_string() that work even if your package is inside an archive. It is not recommended to construct the paths manually but if you need the directory with the current script then you could put get_script_dir() function into your script—it is more general than os.path.dirname(os.path.abspath(__file__)). If the relative position is always the same then you could use:
#!/usr/bin/env python
import os
import subprocess
import sys
main_script_dir = os.path.join(get_script_dir(), os.pardir)
subprocess.check_call([sys.executable, '-m', 'main'], cwd=main_script_dir)
See Python : How to access file from different directory