This is what I'm trying to do.
But after that problem resolved, I had this error: ImportError: No module named 'multiply'. What I could figure out was that I had to add the location of my module to PYTHONPATH.
I couldn't find PYTHONPATH in System variables list, so I created one (as mentioned here), added all the paths including the one I needed, but it didn't solve the problem.
So, I re-installed all the features of Python, even the ones I didn't earlier. Everywhere I read solutions, there is only written set this like this or something similar but no-one says where they are defined or how they work.
Hence my question arises why can't I find variables like PYTHONPATH or PYTHONHOME in System variables while Python says they can be set or where are they defined?
Can anyone explain how do environment variables (mentioned in Python's documentation) work?
How can I solve my 'ImportError`?
The installer might not have set those automatically. You can set them yourself in the command prompt if you like:
SET variableName = value
Or, if you use PowerShell:
$env:variableName = value
If you don't want to use system commands at all, it can be done from within python with the os module:
import os
os.environ[variableName] = value
If they are already set, but you want to add something to it, the process is a little different. Let's suppose the path you are trying to add to PYTHONPATH is path.
Using the command prompt:
SET PYTHONPATH=%PYTHONPATH%;path
Python:
import os
os.environ['PYTHONPATH'] = os.environ['PYTHONPATH'] + ";path"
If you are correct and this is happening because the module is not on PYTHONPATH, this should add it onto the path and fir the problem.
If it doesn't fix the ImportError, then the environment variables are probably not the issue.
Links to learn more about environment variables:
http://www.digitalcitizen.life/simple-questions-what-are-environment-variables
https://en.wikipedia.org/wiki/Environment_variable
http://www.computerhope.com/jargon/e/envivari.htm
Hope this helps.
Related
I am new to python so apologies for the newness of the question.
A description of what I want to do:
I have this folder structure:
c:\PythonScripts\SharedFunctions
C:\PythonScripts\Test\RebuildSpatialIndex\RebuildIndexes.py
C:\PythonScripts\Prod\RebuildSpatialIndex\RebuildIndexes.py
I would like the RebuildIndexes.py to be able to use the function in c:\PythonScripts\SharedFunctions.
How do I do that properly in python?
In general, if you want to import a module, the module's parent directory must be in your PYTHONPATH environment variable.
So, in your case, you would add c:\PythonScripts\SharedFunctions to your PYTHONPATH.
I don't use Windows, so I don't know how to set your PYTHONPATH there (which will enable you to have any Python scripts find your local libraries). The documentation for using Python on Windows (Python 2) (Python 3) will help.
To add to the path from within a program, use
import sys
sys.path.append(path_to_local_libraries)
I'm using Python 2.7. I'm rather new to the python langauge. I have two python modules - "Trailcrest.py" and "Glyph.py", both in the same folder, but not in the Python27 folder.
I need to import "Trailcrest.py" into "Glyph.py", but I am getting the message that "no such module exists".
Additionally, whatever means I use to import the module needs to not be dependent on a solid-state path. This program is cross-platform, and the path can be changed depending on the user's preferences. However, these two modules will always be in the same folder together.
How do I do this?
If you have Trailcrest.py and Glyph.py in the same folder, importing one into the other is as simple as:
import Trailcrest
import Glyph
If this does not work, there seems to be something wrong with your Python setup. You might want to check what's in sys.path.
import sys
print sys.path
To elaborate a bit on Ferdinand Beyer's answer, sys.path is a list of file locations that the default module importer checks. Some, though not all installations of python will add the current directory or the directory of the __main__ module to the path. To make sure that the paths relative to a given module are importable in that module, do something like this:
import os.path, sys
sys.path.append(os.path.dirname(__file__))
But something like that shouldn't ever make it into a "production" product. Instead, use something like distutils to install the module's package into the python site-packages directory.
This can also be achieved using the environment variable PYTHONPATH which also influences Python's search path. This can be done in a shell script so that the Python files do not need to be altered. If you want it to import from the current working directory use the . notation in bash:
export PYTHONPATH=.
python python_prog.py
When I run
import sys
print sys.path
on my Mac (Mac OS X 10.6.5, Python 2.6.1), I get the following results.
/Library/Python/2.6/site-packages/ply-3.3-py2.6.egg
...
/Library/Python/2.6/site-packages/ipython-0.10.1-py2.6.egg
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload
/Library/Python/2.6/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode
They are grouped into 5 categories.
/Library/Python/2.6/site-packages/*.egg
/Library/Python/2.6/site-packages
Frameworks/Python.framework/Versions/2.6/lib/python2.6
Frameworks/Python.framework/Versions/2.6/Extras/lib/python
PATH from PYTHONPATH environment variable.
And I can add more paths using the code
sys.path.insert(0, MORE_PATH)
What routines sets up those paths, and when?
Are some of the paths are built in python source code?
Is it possible that the paths inserted with 'sys.path.insert' are ignored? I'm curious about this, as with mod_wsgi, I found the paths are not found with 'sys.path.insert'. I asked another post for this question.
ADDED
Based on Michael's answer, I looked into site.py, and I got the following code.
def addsitepackages(known_paths):
"""Add site-packages (and possibly site-python) to sys.path"""
sitedirs = []
seen = []
for prefix in PREFIXES:
if not prefix or prefix in seen:
continue
seen.append(prefix)
if sys.platform in ('os2emx', 'riscos'):
sitedirs.append(os.path.join(prefix, "Lib", "site-packages"))
elif sys.platform == 'darwin' and prefix == sys.prefix:
sitedirs.append(os.path.join("/Library/Python", sys.version[:3], "site-packages"))
I also think that the directory name that has site.py (/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6 for my Mac) should be built into Python source code.
Most of the stuff is set up in Python's site.py which is automatically imported when starting the interpreter (unless you start it with the -S option). Few paths are set up in the interpreter itself during initialization (you can find out which by starting python with -S).
Additionally, some frameworks (like Django I think) modify sys.path upon startup to meet their requirements.
The site module has a pretty good documentation, a commented source code and prints out some information if you run it via python -m site.
From Learning Python:
sys.path is the module search path.
Python configures it at program
startup, automatically merging the
home directory of the top-level file
(or an empty string to designate the
current working directory), any
PYTHONPATH directories, the contents
of any .pth file paths you've
created, and the standard library
directories. The result is a list of
directory name strings that Python
searches on each import of a new file.
site.py is indeed the answers. I wanted to remove any dependencies on the old Python that is installed by default on my mac. This works pretty good, as 'site.py' is called each time the python interpreter is started.
For Mac, I manually added the following line at the end of main() in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/site.py:
sys.path = filter (lambda a: not a.startswith('/System'), sys.path)
Path has these parts:
OS paths that have your system libraries
current directory python started from
environmental variable $PYTHONPATH
you can add paths at runtime.
Paths are not ignored. But, they may not be found and that will not raise an error.
sys.path should only be added too, not subtracted from. Django would not remove paths.
Adding to the accepted answer, and addressing the comments that say a module shouldn't remove entries from sys.path:
This is broadly true but there are circumstances where you might want to modify sys.path by removing entries. For instance - and this is Mac-specific; *nix/Windows corollaries may exist - if you create a customised Python.framework for inclusion in your own project you may want to ignore the default sys.path entries that point at the system Python.framework.
You have a couple of options:
Hack the site.py, as #damirv indicates, or
Add your own sitecustomize module (or package) to the custom framework that achieves the same end result. As indicated in the site.py comments (for 2.7.6, anyway):
After these path manipulations, an attempt is made to import a module
named sitecustomize, which can perform arbitrary additional
site-specific customizations. If this import fails with an
ImportError exception, it is silently ignored.
Also note: if the PYTHONHOME env var is set, standard libraries will be loaded from this path instead of the default, as documented.
This is not a direct answer to the question, but something I just discovered that was causing the wrong standard libraries to be loaded, and my searches lead me here along the way.
You are using system python /usr/bin/python.
sys.path is set from system files at python startup.
Do not touch those files, in particular site.py, because this may perturb the system.
However, you can change sys.path within python, in particular, at startup :
in ~/.bashrc or ~/.zshrc:
export PYTHONSTARTUP=~/.pythonrc
in ~/.pythonrc:
write your changes to sys.path.
Those changes will be only for you in interactive shells.
For hacking at little risk for the system, install you own and more recent python version.
I have a python package called zypp. It is generated via swig and the rpm package (called python-zypp) puts it in:
rpm -ql python-zypp
/usr/lib64/python2.6/site-packages/_zypp.so
/usr/lib64/python2.6/site-packages/zypp.py
Now, I have a different project which provides an additional sets of APIs. Pure python. Plus some scripts.
The layout is:
bin/script1
python
python/zypp
python/zypp/plugins.py
python/zypp/__init__.py
plugins.py contains a Plugin class. I intended to put this in an rpm, and put it into
/usr/lib64/python2.6/site-packages/zypp/plugins.py
script1 uses this Plugin class. But as I test it running from git, I would like it to find the module from git too if it is not installed. So it has something like:
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), '../python'))
from zypp.plugins import Plugin
However, it seems that if python-zypp is installed on /usr/lib64/python2.6/site-packages/zypp.py, then script1 won't find the plugins submodule anymore. If I uninstall python-zypp, it does.
So my question is if it is possible to extend a module by adding submodules, being the submodules being located in a different load path. Or will they always clash?
An analogy would be, I have a module foo. And I provide foo.extras in a different load path (which may use foo indeed). The script won't find foo.extras if foo is found first in the system load path. If I only use the custom load path, the script may not find foo module if foo.extras uses it.
I have more experience with ruby, but in ruby I could have installed:
/usr/lib64/ruby/gems/1.8/gems/foo-1.0/lib/foo/*
And I could have in my script:
bin/script
lib/foo/extras/*
I could do in script:
$: << File.join(File.dirname(__FILE__), "../lib"
and then my script could
require foo
require foo/extras
No mater if foo/extras is installed on the system or in the custom load path. They don't clash.
The other way around, I found out that with PYTHONPATH the local zypp.plugins is found first. But then the installed zypp module is not found:
import zypp # works, but seems to import the local one
from zypp.plugins import Plugin # works, PYTHONPATH finds it first
repoinfo = zypp.RepoInfo() # does not work
If I understand your question correctly, you want to use the development version of that module instead of the installed module. Therefore, you can use
PYTHONPATH
From the Module Search Path documentation:
When a module named spam is imported, the interpreter searches for a file named spam.py in the current directory, and then in the list of directories specified by the environment variable PYTHONPATH. This has the same syntax as the shell variable PATH, that is, a list of directory names. When PYTHONPATH is not set, or when the file is not found there, the search continues in an installation-dependent default path; on Unix, this is usually .:/usr/local/lib/python.
So, if the GIT tree of the module directory was "/home/username/some/path", you would change the PYTHONPATH to "/home/username/some/path". Or if the PYTHONPATH variable is already in use, you would append ":/home/username/some/path" to it (note the colon separator). In order to make this permanent, add the line "PYTHONPATH=value" to the file "/etc/environment".
sys.path.insert
If you have a start script for your program, you could override the module search path using sys.path.insert(0, "somepath"). This is similar to the sys.path.append call you described but inserts the path into the beginning of the list.
Ok I found the problem, it was an environmental issue, I had the same modules (minus options.py) on the sys.path and it was importing from there instead. Thanks everyone for your help.
I have a series of import statements, the last of which will not work. Any idea why? options.py is sitting in the same directory as everything else.
from snipplets.main import MainHandler
from snipplets.createnew import CreateNewHandler
from snipplets.db import DbSnipplet
from snipplets.highlight import HighLighter
from snipplets.options import Options
ImportError: No module named options
my __init__.py file in the snipplets directory is blank.
I suspect that one of your other imports redefined snipplets with an assignment statement. Or one of your other modules changed sys.path.
Edit
"so the flow goes like this: add snipplets packages to path import..."
No.
Do not modify sys.path -- that way lies problems. Modifying site.path leads to ambiguity about what is -- or is not -- on the path, and what order they are in.
The simplest, most reliable, most obvious, most controllable things to do are the following. Pick exactly one.
Define PYTHONPATH (once, external to your program). A single, simple environment variable that is nearly identical to installation on site-packages.
Install your package in site-packages.
your master branch doesn't have options.py. could it be that you dev and master branches are conflicting?
if this is your actual code then you have option variable at line 21.
Does the following work?
import snipplets.options.Options
If so, one of your other snipplets files probably sets a global variable named options.
Are you on windows? You might want to try defining an __all__ list in your __init__.py file like noted here. It shouldn't make a difference unless you're importing *, but I've seen modules not import unless they were defined there.
Secondly, you might try setting up a virtualenv. Using a lot of site-wide python packages can lead to these kinds of things.
Lastly, make sure the permissions of options are set correctly. I've spent hours trying to figure these things out only to find out it was an issue of me not having permission to import it.