I am not seeing an answer to this out there, so apologies if this is a duplicate. Basically, I am trying to understand how to force my interpreter (2.7) to import a module from site packages if there is a conflict. For example imagine you are running python from a directory (top_level) that has the following structure:
top_level
----cool_mod
----init.py
----sweet_module.py
but you have already installed sweet module to site packages. When in this directory (but no others) if you run:
from cool_mod.sweet_module import *
you will import from the local module, not the global one. Can I change this somehow?
This situation might arise from the case:
top_level
setup.py
----cool_mod
----init.py
----sweet_module.py
You can run cool_mod.sweet_module before installing if you working directory is top_level. But after installing you can import cool_mod.sweet_module from anywhere. However, if you ever import from this directory, even after installation, you still import the local copy
Inserting the site package directory at the begining of sys.path, and then import.
Or, use imp.load_source to load a module from specified path.
Related
Say Ansible was installed by means of "pip install ansible". Right after the install the following import statement succeeds:
from ansible.module_utils.basic import AnsibleModule
Now, a local package named "ansible.module_utils.custom" is created. The directory structure:
ansible/
__init__.py
module_utils/
__init__.py
custom/
__init__.py
utils.py
As soon as this is put in place the aforementioned import statement fails. Claiming "basic" is undefined. The local package does indeed not declare a "basic" subpackage. Only the installed Ansible library does. It seems Python limited its search to the local package only.
I was under the impression Python would consider the complete system path before giving up on finding code. That it would backtrack out of the local package and finally hit the installed Ansible library.
Is this an incorrect assumption ? If so, is it possible at all to make the local package to coexist with the installed package ?
How Import works
import abc
The first thing Python will do is look up the name abc in sys.modules. This is a cache of all modules that have been previously imported.
If the name isn’t found in the module cache, Python will proceed to search through a list of built-in modules. These are modules that come pre-installed with Python and can be found in the Python Standard Library. If the name still isn’t found in the built-in modules, Python then searches for it in a list of directories defined by sys.path. This list usually includes the current directory, which is searched first.
When Python finds the module, it binds it to a name in the local scope. This means that abc is now defined and can be used in the current file without throwing a NameError.
If the name is never found, you’ll get a ModuleNotFoundError. You can find out more about imports in the Python documentation here!
I am trying to understand how import works in jupyter notebook.
My present working directory is "home/username". I have three python modules.
The path names of these modules are as given below.
"/home/username/module1.py"
"/home/username/folder/module2.py"
"/home/username/anaconda3/lib/python3.7/os.py" (which is an inbuilt python module)
Jupyter Notebook:
cell 1:
import module1
Works just fine
cell 2:
import module2 gives
ModuleNotFoundError: No module named 'module2'
cell 3:
import os
Works just fine
It seems like modules in the working directory can be imported without any problem. So, module1.py can be imported. Modules in other directories that are not packages cannot be imported directly. So, module2.py throws an error. But if this is the case how can os.py, which is not the working directory or in another package in the same directory, be imported directly?
This is really more about how python itself works.
You should be able to import module2 with from folder import module2. You should declare /home/username/folder as a package by create a blank init file /home/username/folder/__init__py. I recommend naming the package something more unique, like potrus_folder, that way you don't get naming conflicts down the line.
To explain: Python keeps track of what modules it has available through it's path, it is usually set in your environment variables. To see what folders it looks in for modules you can do import sys then print(sys.path). By default your working directory (/home/username/) will be included, with highest priority (it should thus be either first or last in sys.path, I don't remember). You can add your own folder with sys.path.append('/some/folder'), although it is frowned upon, and you should really add it to your system path, or just keep it as a package in your working directory.
Packages are really just subfolders of paths which have already been added. You access them, as I explained earlier, by using the from X import Y syntax, or if you want to go deeper from X.Z import Y. Remember the __init__.py file.
The path of os library is set in environment*
Whenever you give import it would search all the directories which are added in your environment + the pwd , so you could just add the directory in environment and that would work
By default /home/username/anaconda3/lib/python3.7/ is added by default at the time of installation since there is where most of the module lies, but you can add urs too
I developed a python3 package in Pycharm but am running into some confusing behavior when I try to import modules in my test cases. The problem seems to be with the directory path to the internal package modules. It is a bit difficult to explain the issue, but here is the gist.
So the python package name is pyugend. Now when I try and import a module--inside the package--named Models into a test case, pycharm forces me to reference the path as pyugend.pyugend.Models. So I need to reference pyugend twice.
However, when I build, install, and import this package into a jupyter notebook or some script, then I run into errors about the pyugend package not finding the internal modules. The only way to fix these errors is to change the paths inside of the module to references like pyugend.Models.
So basically, to run tests inside of pycharm I have make sure all of the internal package imports use a directory path like from pyugend.pyugend.Models import ... But when I want to use the package outside of pycharm then I actually have to go into the package, convert all of the import pyugend.pyugend... references to just single import pyugend.Models ... references.
I have included a picture of the directory structure as well as a picture of the __init__.py.
You can add the linesys.path.append(os.path.dirname(os.path.abspath(__file__))) before the imports in __init.py__ and then change the imports to from pyugend.Models import Base_model and so on, to enable consistent behavior wherever you use the package.
My code depends on functions from a module external_module which is in my pythonpath path and which I include as
# global import
import external_module.sub_mod_one as smo
Now I want to share my code but I don't want to force my collaborators to checkout my other git repos and add them to their environment.
So, I thought I can copy the files to the local directory and rewrite the import as
# local import
import sub_mod_one as smo
However, since development goes on, I don't want to do this manually.
Question Is there a python module or vim plugin or something else that does this for me? Namely, copying the the included modules to the current folder and rewriting the import statements?
The "right" solution is to
properly package your "external_module" so it can be installed with pip,
add to your project(s) a pip requirements file referencing your package
then have everybody using virtualenvs
This way the package will be cleanly installed (and at the right version), you don't have to mess with your exports, and you dont have out of sync copies of your package everywhere.
You could use conditional imports:
try:
import external_module.sub_mod_one as smo
except ImportError:
import sub_mod_one as smo
I'm using Python 2.7. I'm rather new to the python langauge. I have two python modules - "Trailcrest.py" and "Glyph.py", both in the same folder, but not in the Python27 folder.
I need to import "Trailcrest.py" into "Glyph.py", but I am getting the message that "no such module exists".
Additionally, whatever means I use to import the module needs to not be dependent on a solid-state path. This program is cross-platform, and the path can be changed depending on the user's preferences. However, these two modules will always be in the same folder together.
How do I do this?
If you have Trailcrest.py and Glyph.py in the same folder, importing one into the other is as simple as:
import Trailcrest
import Glyph
If this does not work, there seems to be something wrong with your Python setup. You might want to check what's in sys.path.
import sys
print sys.path
To elaborate a bit on Ferdinand Beyer's answer, sys.path is a list of file locations that the default module importer checks. Some, though not all installations of python will add the current directory or the directory of the __main__ module to the path. To make sure that the paths relative to a given module are importable in that module, do something like this:
import os.path, sys
sys.path.append(os.path.dirname(__file__))
But something like that shouldn't ever make it into a "production" product. Instead, use something like distutils to install the module's package into the python site-packages directory.
This can also be achieved using the environment variable PYTHONPATH which also influences Python's search path. This can be done in a shell script so that the Python files do not need to be altered. If you want it to import from the current working directory use the . notation in bash:
export PYTHONPATH=.
python python_prog.py