I had to manually build a package and copy it to the site-packages directory. When I type pip list into a console it isn't listed, though I can use it in python scripts. How can I make pip aware of the package?
Installing it via pip is not an option.
You say "Installing it via pip is not an option.", but I'm assuming installing it via pip using a local copy still is. If so, the way to do that is to clone your library into a directory (say /my/lib/dir), where the root of the source for the root package appears below /my/lib/dir (ex: if the package you want to install is imported as import foo, then you should have /my/lib/dir/foo). If there is no file named setup.py in your copy of the code, then you need to create a simple one. Something like
# in a file called setup.py above the `foo` directory
from distutils.core import setup
setup(name='foo',
version='1.0',
packages=['foo'],
)
Finally, run pip install . from /my/lib/dir.
It's definitely a hack, but making pip aware of a package without installing it via pip is asking for a hack :-)
Question
I installed a local package called credentials using
pip install -e c:\users\worker\src\clockwork\lib\credentials
But when I try to import the package from a sibling directory, it fails with an ImporError:
cd c:\users\worker\src\clockwork\bank
python -c "import credentials"
...
ImportError: No module named 'credentials'
Confusingly, the package credentials is listed as successfully installed as shown when I run pip list:
...
credentials (1.0.0, c:\users\worker\src\clockwork\lib\credentials)
...
How can I install my local package so that it can be imported?
Background
I am using Python 3.4 (32-bit). The package contains two files:
credentials\__init__.py
credentials\setup.py
The __init__.py file defines a single function. The setup.py file is short:
from distutils.core import setup
setup(name='credentials', version='1.0.0')
Workaround
I currently add the directory containing the package (c:\users\worker\src\clockwork\lib) to my PATH variable as a workaround. But my question is how to install the package properly so that I do not need to modify the PATH.
Uninstall the python package then install it using:
python -m pip install -e c:\users\worker\src\clockwork\lib\credentials
What is probably happening is that you have multiple python installs and pip is run from one install while you are trying to use the package from another. See also:
What's the difference between pip install and python -m pip install?
https://docs.python.org/3/installing/#basic-usage
The problem centers on setup.py. It needs to declare a package:
from distutils.core import setup
setup(name='credentials', version='1.0.0', packages=['credentials'])
But this setup.py must be in the parent directory of the credentials package, so in the end, the directory structure is:
...\credentials\setup.py
...\credentials\credentials\__init__.py
With this change, the module is found after reinstalling the package.
This could also be caused by two Python installs (but wasn't in my case), and #Mr_and_Mrs_D gives an answer for that case.
I'm new to Python so this may sound silly.
I want to use a Python library I've found on Github, lets say on https://github.com/praw-dev/praw, and I want to be able to do git pull in the future to pull the latest commits.
Question: Should I git clone <git url> in the project directory and delete everything except the praw directory, then in my python script do a import praw?
In iPython,
import praw
gives the error ImportError: No module named praw
Directory Structure
~\myProject\
praw\
myNotebook.ipynb
Actually, if given package is not on PyPI (or you want a specific branch) you can still install it through pip from GitHub with:
pip install git+https://github.com/[repo owner]/[repo]#[branch name]
And for your problem it would be (although #pandita's answer is correct for normal usage case):
pip install git+https://github.com/praw-dev/praw.git
For more information check this answer.
Experimental Python module finder/loader from github, like in golang.
So, in golang we can import like:
import "github.com/parnurzeal/gorequest"
But in python we should install package by our hands:
pip install requests
And import it like:
import requests
But with this magic package and power of PEP-0302 we can do it automatically:
from github_com.kennethreitz import requests
assert requests.get('https://github.com/nvbn/import_from_github_com').status_code == 200
Installation
You should have git, Python 3.2+ and pip:
pip install import_from_github_com
Reference: https://github.com/nvbn/import_from_github_com
Just clone the files in any dir on your python path and then build the lib typically with python setup.py install from the command line.
I typically clone a libray form git in my site_libraries folder ( the folder that holds all of your pip installed packages ). From there you can pull and then build the libraries from git just like any other git repo. Having the files there is nice because all of your libs are in once place on your python path.
You might want to consider using pip instead of git to install and upgrade the package (that is unless you have a pressing reason to use git).
pip install praw
to update the package you can do
pip install --upgrade praw
Also have a look here for further information on how to use pip.
Is it possible to install packages using pip from the local filesystem?
I have run python setup.py sdist for my package, which has created the appropriate tar.gz file. This file is stored on my system at /srv/pkg/mypackage/mypackage-0.1.0.tar.gz.
Now in a virtual environment I would like to install packages either coming from pypi or from the specific local location /srv/pkg.
Is this possible?
PS
I know that I can specify pip install /srv/pkg/mypackage/mypackage-0.1.0.tar.gz. That will work, but I am talking about using the /srv/pkg location as another place for pip to search if I typed pip install mypackage.
What about::
pip install --help
...
-e, --editable <path/url> Install a project in editable mode (i.e. setuptools
"develop mode") from a local project path or a VCS url.
eg, pip install -e /srv/pkg
where /srv/pkg is the top-level directory where 'setup.py' can be found.
I am pretty sure that what you are looking for is called --find-links option.
You can do
pip install mypackage --no-index --find-links file:///srv/pkg/mypackage
From the installing-packages page you can simply run:
pip install /srv/pkg/mypackage
where /srv/pkg/mypackage is the directory, containing setup.py.
Additionally1, you can install it from the archive file:
pip install ./mypackage-1.0.4.tar.gz
1
Although noted in the question, due to its popularity, it is also included.
I am installing pyfuzzybut is is not in PyPI; it returns the message: No matching distribution found for pyfuzzy.
I tried the accepted answer
pip install --no-index --find-links=file:///Users/victor/Downloads/pyfuzzy-0.1.0 pyfuzzy
But it does not work either and returns the following error:
Ignoring indexes: https://pypi.python.org/simple
Collecting pyfuzzy
Could not find a version that satisfies the requirement pyfuzzy (from versions: )
No matching distribution found for pyfuzzy
At last , I have found a simple good way there: https://pip.pypa.io/en/latest/reference/pip_install.html
Install a particular source archive file.
$ pip install ./downloads/SomePackage-1.0.4.tar.gz
$ pip install http://my.package.repo/SomePackage-1.0.4.zip
So the following command worked for me:
pip install ../pyfuzzy-0.1.0.tar.gz.
Hope it can help you.
This is the solution that I ended up using:
import pip
def install(package):
# Debugging
# pip.main(["install", "--pre", "--upgrade", "--no-index",
# "--find-links=.", package, "--log-file", "log.txt", "-vv"])
pip.main(["install", "--upgrade", "--no-index", "--find-links=.", package])
if __name__ == "__main__":
install("mypackagename")
raw_input("Press Enter to Exit...\n")
I pieced this together from pip install examples as well as from Rikard's answer on another question. The "--pre" argument lets you install non-production versions. The "--no-index" argument avoids searching the PyPI indexes. The "--find-links=." argument searches in the local folder (this can be relative or absolute). I used the "--log-file", "log.txt", and "-vv" arguments for debugging. The "--upgrade" argument lets you install newer versions over older ones.
I also found a good way to uninstall them. This is useful when you have several different Python environments. It's the same basic format, just using "uninstall" instead of "install", with a safety measure to prevent unintended uninstalls:
import pip
def uninstall(package):
response = raw_input("Uninstall '%s'? [y/n]:\n" % package)
if "y" in response.lower():
# Debugging
# pip.main(["uninstall", package, "-vv"])
pip.main(["uninstall", package])
pass
if __name__ == "__main__":
uninstall("mypackagename")
raw_input("Press Enter to Exit...\n")
The local folder contains these files: install.py, uninstall.py, mypackagename-1.0.zip
An option --find-links does the job and it works from requirements.txt file!
You can put package archives in some folder and take the latest one without changing the requirements file, for example requirements:
.
└───requirements.txt
└───requirements
├───foo_bar-0.1.5-py2.py3-none-any.whl
├───foo_bar-0.1.6-py2.py3-none-any.whl
├───wiz_bang-0.7-py2.py3-none-any.whl
├───wiz_bang-0.8-py2.py3-none-any.whl
├───base.txt
├───local.txt
└───production.txt
Now in requirements/base.txt put:
--find-links=requirements
foo_bar
wiz_bang>=0.8
A neat way to update proprietary packages, just drop new one in the folder
In this way you can install packages from local folder AND pypi with the same single call: pip install -r requirements/production.txt
PS. See my cookiecutter-djangopackage fork to see how to split requirements and use folder based requirements organization.
Assuming you have virtualenv and a requirements.txt file, then you can define inside this file where to get the packages:
# Published pypi packages
PyJWT==1.6.4
email_validator==1.0.3
# Remote GIT repo package, this will install as django-bootstrap-themes
git+https://github.com/marquicus/django-bootstrap-themes#egg=django-bootstrap-themes
# Local GIT repo package, this will install as django-knowledge
git+file:///soft/SANDBOX/python/django/forks/django-knowledge#egg=django-knowledge
To install only from local you need 2 options:
--find-links: where to look for dependencies. There is no need for the file:// prefix mentioned by others.
--no-index: do not look in pypi indexes for missing dependencies (dependencies not installed and not in the --find-links path).
So you could run from any folder the following:
pip install --no-index --find-links /srv/pkg /path/to/mypackage-0.1.0.tar.gz
If your mypackage is setup properly, it will list all its dependencies, and if you used pip download to download the cascade of dependencies (ie dependencies of depencies etc), everything will work.
If you want to use the pypi index if it is accessible, but fallback to local wheels if not, you can remove --no-index and add --retries 0. You will see pip pause for a bit while it is try to check pypi for a missing dependency (one not installed) and when it finds it cannot reach it, will fall back to local. There does not seem to be a way to tell pip to "look for local ones first, then the index".
Having requirements in requirements.txt and egg_dir as a directory
you can build your local cache:
$ pip download -r requirements.txt -d eggs_dir
then, using that "cache" is simple like:
$ pip install -r requirements.txt --find-links=eggs_dir
What you need is --find-links of pip install.
-f, --find-links If a url or path to an html file, then parse for links to archives. If a local path or
file:// url that's a directory, then look for archives in the directory listing.
In my case, after python -m build, tar.gz package (and whl file) are generated in ./dist directory.
pip install --no-index -f ./dist YOUR_PACKAGE_NAME
Any tar.gz python package in ./dist can be installed by this way.
But if your package has dependencies, this command will prompt error.
To solve this, you can either pip install those deps from official pypi source, then add --no-deps like this
pip install --no-index --no-deps -f ./dist YOUR_PACKAGE_NAME
or copy your deps packages to ./dist directory.
I've been trying to achieve something really simple and failed miserably, probably I'm stupid.
Anyway, if you have a script/Dockerfile which download a python package zip file (e.g. from GitHub) and you then want to install it you can use the file:/// prefix to install it as shown in the following example:
$ wget https://example.com/mypackage.zip
$ echo "${MYPACKAGE_MD5} mypackage.zip" | md5sum --check -
$ pip install file:///.mypackage.zip
NOTE: I know you could install the package straight away using pip install https://example.com/mypackage.zip but in my case I wanted to verify the checksum (never paranoid enough) and I failed miserably when trying to use the various options that pip provides/the #md5 fragment.
It's been surprisingly frustrating to do something so simple directly with pip. I just wanted to pass a checksum and have pip verify that the zip was matching before installing it.
I was probably doing something very stupid but in the end I gave up and opted for this. I hope it helps others trying to do something similar.
In my case, it was because this library depended on another local library, which I had not yet installed. Installing the dependency with pip, and then the dependent library, solved the issue.
If you want to install one local package (package A) to be used inside another local project/package (B) this is quite simple. All you need is to CD to (B) and call:
pip install /path/to/package(A)
Of course you will need to first compile the package (A) with:
sudo python3 ./setup.py install
And, each time you change package A, just run again setup.py in package (A) then pip install ... inside the using project/package (B)
Just add directory on pip command
pip install mypackage file:/location/in/disk/mypackagename.filetype
According to another question, pip offers a facility for uninstalling eggs, which it's help also indicates.
I have a project that, once installed, has a structure in my local site-packages folder that looks like this:
projecta/
projecta-1.0-py2.6.egg-info/
Using an up to date version, pip uninstall projecta asks me the following question:
/path/to/python2.6/site-packages/projecta-1.0-py2.6.egg-info
Proceed (y/n)?
Answering y will remove the .egg-info directory, but not the main projecta directory, without saying there was any sort of error. Why doesn't pip manage or know to remove this directory?
The project itself is installed via a setup.py file using distutils. Are there any special settings I could/should use in that file to help pip with the removal process?
If I recall correctly pip knows how to uninstall packages installed via setuptools/distribute, not raw distutils.
There are some setuptools's features pip is based on - like --record option, which stores package metadata (and it is what allows pip to uninstal package related files).
Try doing:
$ pip install /path/to/projecta
$ pip uninstall projecta