Let's say you have a module which contains
myfile = open('test.txt', 'r')
And the 'test.txt' file is in the same folder. If you'll run the module, the file will be opened successfully.
Now, let's say you import that module from another one which is in another folder. The file won't be searched in the same folder as the module where that code is.
So how to make the module search files with relative paths in the same folder first?
There are various solutions by using "__file__" or "os.getcwd()", but I'm hoping there's a cleaner way, like same special character in the string you pass to open() or file().
The solution is to use __file__ and it's pretty clean:
import os
TEST_FILENAME = os.path.join(os.path.dirname(__file__), 'test.txt')
For normal modules loaded from .py files, the __file__ should be present and usable. To join the information from __file__ onto your relative path, there's a newer option than os.path interfaces available since 2014:
from pathlib import Path
here = Path(__file__).parent
fname = here / "test.txt"
with fname.open() as f:
...
pathlib was added to Python in 3.4 - see PEP428. For users still on Python 2.7 wanting to use the same APIs, a backport is available.
Note that when you're working with a Python package, there are better approaches available for reading resources - you could consider moving to importlib-resources. This requires Python 3.7+, for older versions you can use pkgutil. One advantage of packaging the resources correctly, rather than joining data files relative to the source tree, is that the code will still work in cases where it's not extracted on a filesystem (e.g. a package in a zipfile). See How to read a (static) file from inside a Python package? for more details about reading/writing data files in a package.
Related
I know that python mechanism doesn't allow doing relative imports without a known parent package, but I want to know which the reason is.
If relative imports worked without known parent package it would make developers life much easier (I think, correct me if I am wrong)
Example:
From a python script a have to import a global definitions file. Right now I have to do it like this:
DEFS_PATH = '../../../'
sys.path.append(DEFS_PATH)
import DEFINITIONS as defs
If I could import this file just like this without having to specify the -m flag when executing the script or creating a __init__.py file that collects all packages. It would make everything much more easier.
from .... import DEFINITIONS as defs
Of course doing this raises the famous import error:
Obviously this is a toy example, but imagine having to repeat this in hundreds of python scripts...
Is there any workaround for importing relative packages without a known parent package that doesn't involve tha hacky ugly way (sys.path.append(...) or python -m myscript )?
I solved this problem but in a different way. I have a folder where I have lots of global functions that I used in different packages, I could create a Python Package of this folder, however I would have to rebuild it each time I changed something.
The solution that fitted me was to add user-packages.pth file in site-packages directory of my current environment, but it could also be added to global site-packages folder. Inside this user-packages.pth I added the absolute path to my directory where all the global utils are. And now I just have to do from any python script
from utils import data_processing as dp
from utils.database import database_connection as dc
Now I don't need to add in each file sys.path.append("path/to/myutils/")
Note:
The .pth file could have any file name (customName.pth) and paths inside the file should be separated by carriage return ("\n"). Also, paths should be absoulte.
For example:
C:\path\to\utils1
C:\path\to\other\utils2
I'm using Python 2.7. I'm rather new to the python langauge. I have two python modules - "Trailcrest.py" and "Glyph.py", both in the same folder, but not in the Python27 folder.
I need to import "Trailcrest.py" into "Glyph.py", but I am getting the message that "no such module exists".
Additionally, whatever means I use to import the module needs to not be dependent on a solid-state path. This program is cross-platform, and the path can be changed depending on the user's preferences. However, these two modules will always be in the same folder together.
How do I do this?
If you have Trailcrest.py and Glyph.py in the same folder, importing one into the other is as simple as:
import Trailcrest
import Glyph
If this does not work, there seems to be something wrong with your Python setup. You might want to check what's in sys.path.
import sys
print sys.path
To elaborate a bit on Ferdinand Beyer's answer, sys.path is a list of file locations that the default module importer checks. Some, though not all installations of python will add the current directory or the directory of the __main__ module to the path. To make sure that the paths relative to a given module are importable in that module, do something like this:
import os.path, sys
sys.path.append(os.path.dirname(__file__))
But something like that shouldn't ever make it into a "production" product. Instead, use something like distutils to install the module's package into the python site-packages directory.
This can also be achieved using the environment variable PYTHONPATH which also influences Python's search path. This can be done in a shell script so that the Python files do not need to be altered. If you want it to import from the current working directory use the . notation in bash:
export PYTHONPATH=.
python python_prog.py
I have a python app, that I'm developing. There is a need to use another library, that resides in different directory.
The file layout looks like this:
dir X has two project dirs:
current-project
xLibrary
I'd like to use xLibrary in currentProject. I've been trying writting code as if all the sources resided in the same directory and calling my projects main script with:
PYTHONPATH=.:../xLibrary ./current-project.py
but this does not work. I'd like to use its code base without installing the library globaly or copying it to my project's directory. Is it possible? Or if not, how should I deal with this problem.
It's generally a good programming practice to isolate packages into actual packages and treat them as such. If you're sure you'd like to continue with that approach though you can modify the search path from within python via:
import sys
sys.path.append( "<path_containing_the_other_python_files>" )
To avoid embedding absolute paths, you can use os.path.abspath(__file__) to obtain the absolute path to the currently executing .py file and follow up with a few os.path.dirname() calls to construct the proper relative path for inclusion to sys.path
A slightly altered approach that would allow you to get the best of both worlds would be to add an __init__.py file to xLibrary's directory then add the path containing 'xLibrary' to sys.path instead. Subsequent Python code could then import everything "properly" via from xLibrary import my_module rather than just import my_module which could be confusing to people accustomed to the standard package directory layout.
This depends how you use xLibrary from current-project.
If you do something like from xLibrary import module1 inside current-project, the xLibrary needs to be laid out as a Python package:
xLibrary/
xLibrary/__init__.py
xLibrary/module1.py # or whatever other modules the package consists of
In this case, you should include xLibrary's parent directory in PYTHONPATH:
PYTHONPATH=.:.. ./current-project.py
However, if xLibrary is just a collection of Python modules that you import individually (that is, if you do import module1 and import module2 ìn current-project, with xLibrary containing the files module1.py and module2.py), you should include xLibrary in PYTHONPATH just as you did:
PYTHONPATH=.:../xLibrary ./current-project.py
bash$ ln -s ../xLibrary xLibrary
First, it is probably better to use absolute paths in your PYTHONPATH variable.
Second, I don't think you need to add current directory to the path.
Other than that, it would be good to know what it is that doesn't work and what the error message is.
The command line you have there seems to be missing a semicolon
Try these two:
a=12 echo $a
b=12 ;echo $b
...and you'll see the difference.
Apart from the other suggestions, you may consider the virtualenv package. After writing a little setup.py file you can "virtually install" the library and avoid all the PYTHONPATH munging in general.
This is a really good practice and is encouraged by the python community.
Otherwise I prefer the use of sys.path method (by Rakis)
This question already has answers here:
How to read a (static) file from inside a Python package?
(6 answers)
Closed 2 years ago.
I've written a Python package that includes a bsddb database of pre-computed values for one of the more time-consuming computations. For simplicity, my setup script installs the database file in the same directory as the code which accesses the database (on Unix, something like /usr/lib/python2.5/site-packages/mypackage/).
How do I store the final location of the database file so my code can access it? Right now, I'm using a hack based on the __file__ variable in the module which accesses the database:
dbname = os.path.join(os.path.dirname(__file__), "database.dat")
It works, but it seems... hackish. Is there a better way to do this? I'd like to have the setup script just grab the final installation location from the distutils module and stuff it into a "dbconfig.py" file that gets installed alongside the code that accesses the database.
Try using pkg_resources, which is part of setuptools (and available on all of the pythons I have access to right now):
>>> import pkg_resources
>>> pkg_resources.resource_filename(__name__, "foo.config")
'foo.config'
>>> pkg_resources.resource_filename('tempfile', "foo.config")
'/usr/lib/python2.4/foo.config'
There's more discussion about using pkg_resources to get resources on the eggs page and the pkg_resources page.
Also note, where possible it's probably advisable to use pkg_resources.resource_stream or pkg_resources.resource_string because if the package is part of an egg, resource_filename will copy the file to a temporary directory.
Use pkgutil.get_data. It’s the cousin of pkg_resources.resource_stream, but in the standard library, and should work with flat filesystem installs as well as zipped packages and other importers.
That's probably the way to do it, without resorting to something more advanced like using setuptools to install the files where they belong.
Notice there's a problem with that approach, because on OSes with real a security framework (UNIXes, etc.) the user running your script might not have the rights to access the DB in the system directory where it gets installed.
Use the standard Python-3.7 library's importlib.resources module,
which is more efficient than setuptools:pkg_resources
(on previous Python versions, use the backported importlib_resources library).
Attention: For this to work, the folder where the data-file resides must be a regular python-package. That means you must add an __init__.py file into it, if not already there.
Then you can access it like this:
try:
import importlib.resources as importlib_resources
except ImportError:
# In PY<3.7 fall-back to backported `importlib_resources`.
import importlib_resources
## Note that the actual package could have been used,
# not just its (string) name, with something like:
# from XXX import YYY as data_pkg
data_pkg = '.'
fname = 'database.dat'
db_bytes = importlib_resources.read_binary(data_pkg, fname)
# or if a file-like stream is needed:
with importlib_resources.open_binary(data_pkg, fname) as db_file:
...
Is there any way to get Python to use my ActiveTcl installation instead of having to copy the ActiveTcl libraries into the Python/tcl directory?
Not familiar with ActiveTcl, but in general here is how to get a package/module to be loaded when that name already exists in the standard library:
import sys
dir_name="/usr/lib/mydir"
sys.path.insert(0,dir_name)
Substitute the value for dir_name with the path to the directory containing your package/module, and run the above code before anything is imported. This is often done through a 'sitecustomize.py' file so that it will take effect as soon as the interpreter starts up so you won't need to worry about import ordering.