I use a script under virtualenv, that requires bzrlib package which is not available in my virtualenv but is included in my system python packages: /usr/lib/python2.7/dist-packages/bzrlib/
If I want to use it, one option is to extend sys.path, but I would have to include the parent folder /usr/lib/python2.7/dist-packages/ which contains many other packages, that I don't want to make available. Is there any easy way to include just bzrlib package?
What about creating a link in different directory and importing using that? Or even in your own project.
ln -s /package/dir/path /project/dir/path
If you have to load it remotely, here is the link provided by #unutbu showing how to do it:
How to import a module given the full path?
import imp
foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()
Related
I need to make some changes to the way the OSMnx package gets data from the Overpass API.
To do that, I've forked the repo and cloned that fork into a local directory.
I've made a simple change, adding a print functions to start.
I'm trying to figure out the steps for installing and importing the modified version of the package. I've looked at this question
I'm concerned about a namespace conflict, should I change the name of the folder or the name value in the setup.pyfile. I don't want to write over the working version of the package
What file should I point to when installing? setup.py, core.py, _init_.py?
I haven't found any tutorials on how to modify an existing package safely, just stuff on how to make a package from scratch.
import will search the installed packages list first, then it will check the directory it was called from for files that match the package it's looking for.
So either using pip or conda remove the original version of the package being imported.
Then, be sure that the modified version of the package that you want to import is in a sibdirectory of your project directory, and run import package as xx and it should load the modified package files.
If it doesn't, it's likely that the package wasn't removed from the right environment.
I already figured out how one can retrieve the include and site-package paths of a python environment. For instance the following is one of multiple possibilities:
from distutils.sysconfig import get_python_lib, get_python_inc
print(get_python_lib()) # Prints the location of site-packages
print(get_python_inc()) # Prints the location of the include dir
However, I was not able to find a robust method to retrieve the bin folder of a python environment, that is, the folder where python itself and tools like pip, pyinstaller, easy_install, etc., typically reside. Does anyone know how I can get this path from within python?
Some may want to suggest binpath = os.path.dirname(sys.executable). On Mac however, this does not work if python was installed as a Framework (binpath would point at: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS)
What could I use instead that works cross-platform?
The stdlib way to get this location is via sysconfig module. Works on 2.7 and 3.3+. Works whether or not you are using a virtual environment.
>>> from sysconfig import get_path
>>> get_path('scripts')
'/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/bin'
Almost any package with scripts to install will put them there. However, note that with an imperative installer the setup.py code is actually executed, which means it could literally lay down scripts anywhere on the filesystem, given permission.
Running
$ python setup.py build_ext
with the usual Cython extension configuration creates a build directory and places the compiled modules deep within it.
How is the Python interpreter supposed to find them now? Should PYTHONPATH include those sub-sub-directories? It seems kludgy to me. Perhaps this is meant to work differently?
You will find the information here https://docs.python.org/3.5/install/
Build is an intermediary result before python actually install the module. Put in pythonpath the paths to the library that is, for instance:
<dir>/local/lib/python
if you use the "home" installation technique and dir is the directory you have chosen, ex
/home/user2
Presumably, when you write a package containing Cython code, your setup.py will contain something similar to this:
setup(
ext_modules = cythonize("example.pyx")
)
(there are some variations, but that's the general idea). When you run
python setup.py install
or
python setup.py install --user
You will see it creates binary files (with extensions based on your OS - on mine it will be example.so) and copies them to the standard installation directory (also depending on your OS).
These binary files are therefore already in the import path of your Python distribution, and it can import them like regular modules.
Consequently, you do not need to add the build directory to the path. Just install (possibly with --user, or use virtualenv, if you're developing), and let the extensions be imported the regular way.
Previously, I have been using Python 2.7. If I installed a module using pip, it would place the module in /usr/local/lib/python2.7/site-packages. Then, if I were to import that module in a python script, the site-packages directory would be searched to find that module.
However, I want to start using the Anaconda distribution. After downloading this, I notice that there are a number of packages located at /home/karnivaurus/Anaconda/pkgs, for example scikit-learn. Then, in the directory of each packages, there is the directory structure lib/python2.7/site-packages, which contains the modules for that package.
So, it seems that I have now gone from the situation where I only had one site-packages directory, to where I have a number of site-packages directories, one for each package.
My question is: When I create a python script, and want to import a module, how does python know where to look for these modules? Will it look in /usr/local/lib/python2.7/site-packages as well as the site-packages directories that come with Anaconda? What if I want to install another package that does not come with Anaconda -- where should this be installed to?
Thanks!
The paths in which Python will import packages can be seen with the following command:
python -c "import sys; print sys.path"
Please see the documentation, also (as stated in the documentation) if you need to modify the search path, look into PYTHONPATH.
A Python module is just a .py source file. A Python package is simply a collection of modules.
So why do we need programs such as pip to 'install' Python modules? Why not just download the files, put them in our project's folder and import them?
What exactly does it mean to 'install' a module or a package? And what exactly does pip do?
Are things different on Windows and on Linux?
So why do we need programs such as pip to 'install' Python modules? Why not just download the files, put them in our project's folder and import them?
It's just meant to facilitate the installation of softwares without having to bundle all the dependencies nor ask the user to download the files.
You can type pip install mysoftware and that will also install the required dependencies. You can also upgrade a software easily.
What exactly does it mean to 'install' a module or a package? And what exactly does pip do?
It will copy the files in a directory that is in your Python path. This way you will be able to import the package without having to copy the directory in your project.
With your proposal, for each and every project you have to download the required modules as dependencies. You have to download them again and again and add them with your project which is not very suitable though some platform like node.us do it.
What pip do is to keep the modules you installed in /use/lib/python*/site-packages/ so clearly it is included in your Python's path. So, when you try to import a module or package it checks in site-package if it exists. If exists,then this code will be used with your project. If not, you will get an error.