If I set PYTHONPATH to some path /path/to/modules/, then the path is appended to sys.path and I can import modules/packages contained in /path/to/modules/.
However, if I only want access to a single module/package, then adding /path/to/modules/mymod.py or /path/to/modules/mypackage/ to sys.path does not work.
So is there a way to add only a single module/package to the import search path, rather than adding the entire parent directory?
I am asking because I need to import a single package installed under /usr/lib/python3/dist-packages/ from within a virtual environment, and I would prefer not to give the virtual environment access to all of the modules/packages installed under that path. (The package has a complicated build process and cannot be easily installed to a virtual environment.)
I have read https://stackoverflow.com/a/67692/ but I am wondering if it's possible to actually include the package in the import search path, so that the package (and the modules it contains) can be imported normally.
There is no such way. If you want to import a module you have to add its
parent directory to sys.path. But you can remove it later:
sys.path.append('/usr/lib/python3/dist-packages/')
import mypackage
del sys.path[-1]
Related
There is a popular Python question called Importing files from different folder.
But the top answer there mentions adding stuff to "some_file.py", and that will obviously only apply to imports inside that file.
What if I want to specify an additional dir to import from, project-wide?
I don't want to modify PYTHONPATH, as I believe a per-project solution is cleaner.
I don't want to use python packages for this, because I feel they'll probably just complicate stuff. E.g. maybe I'll need to manually recompile .py files into .pyc files every time I make a change to the code in the other folder.
What if I want to specify an additional dir to import from, project-wide?
Solution 1: Package installed in develop mode
Create a regular package with setup.py and install it with -e option:
python -m pip install -e /path/to/dir_with_setup_py/
-e, --editable Install a project in editable mode (i.e. setuptools "develop mode") from a local project
path or a VCS url.
Now, as soon as you update your code, the new version will be used at import
without reinstalling anything.
Solution 2: Dynamically modify sys.path
You can add as many directories dynamically to the search path for Python packages as you want. Make this the very first lines of code you execute:
import sys
sys.path.append('my/path/to/my/file1')
sys.path.append('my/path/to/my/file2')
or to make the first to be found:
sys.path.insert(0, 'my/path/to/my/file1')
sys.path.insert(0, 'my/path/to/my/file2')
Now the files:
my/path/to/my/file1/myscript1.py
my/path/to/my/file2/myscript2.py
can be imported anywhere in your project:
import myscript1
import myscript2
No need to modify sys.path again as long as this Python process is running.
I am a newbie to Python and I have been going through the exercises in LPTHW:
http://learnpythonthehardway.org/book/ex46.html
The explanation of packaging the author gave was very vague, so I did some researching, but I was still left with some questions in the end.
Firstly, what happens if you have a module that imports another module? Do you put the module that is imported into a subdirectory of the directory containing the first module, or in another separate directory in the root?
Secondly, once you have installed the entire package how do you run any scripts that are in the bin folder?
Firstly, what happens if you have a module that imports another
module? Do you put the module that is imported into a subdirectory of
the directory containing the first module, or in another separate
directory in the root?
A module is just a file with Python code - you don't need to put it in a separate directory. The easiest option is to put the file in the same directory that contains the module that will be doing the import.
If you put it in some other directory and that directory is not a package you will not be able to import it at all (unless you manually mess around with PYTHONPATH which you shouldn't).
Secondly, once you have installed the entire package how do you run
any scripts that are in the bin folder?
The way to do this is to specify the scripts for your application in the setup.py file using console_scripts. This way, when your package is installed, your scripts will be built correctly and placed in the right directories on the target platform so that you can directly execute them (without being aware of their location on the filesystem).
As an example, you can browse the setup.py file for the django project.
If you are installing a package, you don't need to know where the actual script is located - simply install the package and execute the command directly from your prompt.
Whenever I try to expirement with Python on Windows, I always run into a wall with the import statements. Python simply can't find anything on Windows -- every import, even for something as core as timezone fails.
I know this has something to do with the PYTHONPATH environment variable. In my case, Python is installed to "C:\Python27". My PYTHONPATH looks like this:
C:\Python27;C:\Python27\DLLs;C:\Python27\Lib
Still, nothing will import. I get errors like this:
File "D:\Code\Django\polls\models.py", line 3, in <module>
from django.utils import timezone
ImportError: cannot import name timezone
What's wrong with my situation?
Take a look at the official docs on using Python on Windows, in particular the section on finding modules.
You have to add the directory where you installed 3rd party modules to your PYTHONPATH if you didn't install them to your Python27\Libs\site-packages directory.
Another option is to get acquainted with pip and virtualenv. These tools make installing 3rd party modules a breeze. Although I don't know how well they are supported on Windows (I mainly do Python development on Linux).
PYTHONPATH = If this variable exists in your environment, Python will add it to the normal search path for modules when you use any import statement; you normally do not modify this as well behaved Python scripts will install themselves in the site-packages directory, and Python searches this by default.
PATH = this is the global file system path. Your operating system will search the directories listed in this variable (from left to right), to find commands when you type something at a command prompt.
In order for Python to work correctly only Windows, the C:\Python27 directory should be listed in PATH. If you ran the installer as an Administrator, the installer will modify the global PATH and add this for you. If you installed it as a normal user, you need to modify the PATH manually.
To add this manually, right click on My Computer and select Properties. Click on Advanced, then Environment Variables. You'll see two boxes - User Variables and System Variables. You can only edit user variables - system variables need administrative access.
Simply add a new variable (or modify the existing PATH) You should also add C:\Python27\Scripts to your PATH as most commands installed by Python scripts (like django-admin.py) are installed here. Directories are separated by ;
Once you have done this; python should work properly for you on Windows.
virtualenv is good option.
else simply add site-packages in path.
In attempting to solve a package management issue here, I learned I may have issues with the way I'm using PYTHONPATH to access packages outside of the site-packages directory.
I receive an import error when attempting to import modules from a package (say wxPython) from its versioned directory folder name (i.e. wx-2.8-msw-unicode) when said directory is not in the python site-packages directory.
To recreate the issue: Create a directory outside of the python directory, say C:\foo. Download wxPython, and place it in that directory (giving C:\foo\wx-2.8-msw-unicode). Add more if you like. Add the directory C:\foo to your PYTHONPATH environment variable. Open a new python interactive shell and run
import sys
for i in sys.paths:
print i
to verify that the path is there (which it is), then
import wx
I receive an Import Error. What am I doing wrong? Is this behavior correct?
As I understand what you're saying, this is expected behaviour. C:\foo is in your Pythonpath, but it does not contain an importable wx module. For import wx to succeed, it has to find one of wx.(py/pyc/pyd/dll/so) or a directory wx containing the file __init__.py, directly in one of the directories on your Pythonpath.
The installer will normally sort out making sure this is in an importable location for you. If you want to do it yourself, you have to ensure that wx ends up in an importable location.
I am new to Python and mostly used my own code. But so now I downloaded a package that I need for some problem I have.
Example structure:
root\
externals\
__init__.py
cowfactory\
__init__.py
cow.py
milk.py
kittens.py
Now the cowfactory's __init__.py does from cowfactory import cow. This gives an import error.
I could fix it and change the import statement to from externals.cowfactory import cow but something tells me that there is an easier way since it's not very practical.
An other fix could be to put the cowfactory package in the root of my project but that's not very tidy either.
I think I have to do something with the __init__.py file in the externals directory but I am not sure what.
Inside the cowfactory package, relative imports should be used such as from . import cow. The __init__.py file in externals is not necessary. Assuming that your project lies in root\ and cowfactory is the external package you downloaded, you can do it in two different ways:
Install the external module
External Python packages usually come with a file "setup.py" that allows you to install it. On Windows, it would be the command "setup.py bdist_wininst" and you get a EXE installer in the "dist" directory (if it builds correctly). Use that installer and the package will be installed in the Python installation directory. Afterwards, you can simply do an import cowfactory just like you would do import os.
If you have pip or easy_install installed: Many external packages can be installed with them (pip even allows easy uninstallation).
Use PYTHONPATH for development
If you want to keep all dependencies together in your project directory, then keep all external packages in the externals\ folder and add the folder to the PYTHONPATH. If you're using the command line, you can create a batch file containing something like
set PYTHONPATH=%PYTHONPATH%:externals
yourprogram.py
I'm actually doing something similar, but using PyDev+Eclipse. There, you can change the "Run configurations" to include the environment variable PYTHONPATH with the value "externals". After the environment variable is set, you can simply import cowfactory in your own modules. Note how that is better than from external import cowfactory because in the latter case, it wouldn't work anymore once you install your project (or you'd have to install all external dependencies as a package called "external" which is a bad idea).
Same solutions of course apply to Linux, as well, but with different commands.
generally, you would use easy_install our pip to install it for you in the appropriate directory. There is a site-packages directory on windows where you can put the package if you can't use easy_install for some reason. On ubuntu, it's /usr/lib/pythonX.Y/dist-packages. Google for your particular system. Or you can put it anywhere on your PYTHONPATH environment variable.
As a general rule, it's good to not put third party libs in your programs directory structure (although there are differing opinions on this vis a vis source control). This keeps your directory structure as minimalist as possible.
The easiest way is to use the enviroment variable $PYTHONPATH. You set it before running your scripts as follows:
export $PYTHONPATH=$PYTHONPATH:/root/externals/
You can add as many folders as you want (provided their separate by :) and python will look in all those folders when importing.