i've installed a python project, and it imports modules(Like almost every project). The problem is when i want to install them(because i haven't got the modules), for example: In the project is imported a module called "a" but when i go and install "a" with pip install a, it says ERROR: Could not find a version that satisfies the requirement a (from versions: none) ERROR: No matching distribution found for a. How could i know the name of the module that is imported in that python project?
Edit:
btw i just found out the module that the project uses comes in the zip where the python project is. How could i install it so it works?
All pip packages are listed here. If you want to import a module called a inside a python script, the command to install it could be sometimes pip install b. Because the name of the stored package can varied from the python import name. To find how to install it the best is to get the pypi url of your package. You can googling the python error ModuleNotFoundError: No module named 'dgvd', it always show you the pypi url in top links.
The good practice in a project is to have a txt file called requirement.txt that you create in bash using this command:
pip freeze > requirement.txt
Then install all packages in once using:
pip install -r requirement.txt
For installing from zip simply use:
pip install *.zip
or specify the path directly:
pip install <path to .zip>
pip install ./my-archive.zip
Same applies for a tarball or any other format. It can be even a folder. However, it has to include a proper setup.py or other mechanism for pip to install it and pip has to support the packaging format (be it archive, networking protocol, version control system (git prefix), etc).
pip install ./my-folder
pip install ./
pip install .
pip install ..
etc
If, however, there is no setup.py present, you'll need to simply copy-paste the files somewhere where your project/module resides (or set PYTHONPATH or sys.path to that folder) to be able to import them. See this or this question for more.
In my requirements.txt I have packages defined in following manner:
Django ~= 2.2.0
It means that when I use pip install -r requirements.txt pip will find the latest available 2.2.x version and install it along with all dependencies.
What I need is requirements-formatted list of all packages with explicit versions that will be installed but without actually installing any packages. So example output would be something like:
Django==2.2.23
package1==0.2.1
package2==1.4.3
...
So in other words I'm looking for something like pip freeze results but without installing anything.
pip-compile is what you need!
Doc: https://github.com/jazzband/pip-tools)
python -m pip install pip-tools
pip-compile requirements.txt --output-file requirements-all.txt
The pip-compile command lets you compile a requirements.txt file from your dependencies, this way you can pip install you dependencies to always have the same environment
TL;DR
Try pipdetree or pip-tree.
Explanation
pip, contrary to most package managers, doesn't have a big dependency graph to look up. What it does is that it lets arbitrary setup code to be executed, which automatically pulls the dependencies. This means that, for example, a package could manage their dependencies in an other way than putting them in requirements.txt (see fastai for an example of a project that handles the dependencies differently).
So, there is, theoretically, no other way to see all the dependencies than to actually run an install on an isolated environment, see what was pulled, then delete the environment (because it could potentially be the same part of the code that does the installation and that brings the dependencies). You could actually do that with venv.
In practice, tools like pipdetree or pip-tree fetch the dependencies based on some standardization of the requirements (most packages separate the dependencies and the installation, and actually let pip handle both).
I deleted a pip package with rm -rf command thinking that the package will be removed. Now the package has been deleted but it still shows up in pip list and I'm unable to remove it with pip uninstall nor can I update with pip install --upgrade.
I'd like to remove it completely. Can anyone please tell me how?
EDIT
The package is psycopg2.
If I try to uninstall :
hammad#hammad-P5QL-E:~$ pip uninstall psycopg2
Can't uninstall 'psycopg2'. No files were found to uninstall.
This is the directory in which psycopg2 was located /usr/lib/python2.7/dist-packagesand I rm -rf 'd it from the same directory.
TIA
packages installed using pip can be uninstalled completely using
pip uninstall <package>
refrence link
pip uninstall is likely to fail if the package is installed using python setup.py install as they do not leave behind metadata to determine what files were installed.
packages still show up in pip list if their paths(.pth file) still exist in your site-packages or dist-packages folder. You'll need to remove them as well in case you're removing using rm -rf
Go to the site-packages directory where pip is installing your packages.
You should see the egg file that corresponds to the package you want to uninstall. Delete the egg file (or, to be on the safe side, move it to a different directory).
Do the same with the package files for the package you want to delete (in this case, the psycopg2 directory).
pip install YOUR-PACKAGE
I'm sure there's a better way to achieve this and I would like to read about it, but a workaround I can think of is this:
Install the package on a different machine.
Copy the rm'ed directory to the original machine (ssh, ftp, whatever).
pip uninstall the package (should work again then).
But, yes, I'd also love to hear about a decent solution for this situation.
I met the same issue while experimenting with my own Python library and what I've found out is that pip freeze will show you the library as installed if your current directory contains lib.egg-info folder. And pip uninstall <lib> will give you the same error message.
Make sure your current directory doesn't have any egg-info folders
Check pip show <lib-name> to see the details about the location of the library, so you can remove files manually.
How can I keep track of the packages when I install them using pip inside a virtualenv?
It seems like a mess now; if I install package A, it automatically install its dependancies; B, C and D. Then I decide to use package N instead which installs its dependancies as well.
Now when I remove package A, its dependancies are not automatically removed.
How I can keep my virtualenv clean? Is there a tool to check for unused packages and remove them?
To remove a package:
pip uninstall package_name
To get list of packages required by any given package (using pip):
pip show package_name
This will show you the packages that are required for it to run, and also the packages that require your package for them to run.
So the best way to uninstall a package with all its dependency packages is to run pip show package_name first to see the list of its dependency packages and then uninstall it along with its dependency packages one by one. For example:
pip show package_name
pip uninstall package_name
pip uninstall dependency_package_1
pip uninstall dependency_package_2
...etc
Making virtualenvs is relatively cheap. You could just create a new virtualenv whenever you get into this situation and run your pip install again.
Not very elegant, but it gets the job done. Of course you need to be maintaining some requirements file for the pip install and it will go faster if you have some local index or cache for pip.
To get a clean environment, create a new one. Some pip hooks can help you on this path:
pip freeze to get list of installed packages and their versions, wich can later be used with
-r <file> to install list of packages, stated in a requirements file
--build <dir> to place builds in a specific directory
--no-clean to not clean up build directories
later you can use those builds with --no-download
--no-deps to not install dependencies
Alternative way is to name each dependency of your project in your "setup.py" or "requirements.txt". Exercise setup.py or pip install cat requirements.txt multiple times with virtualenv in order to run your application successfully. After that, manually add the new dependency to one of the files to keep your dependency in sync.
Is it possible to install packages using pip from the local filesystem?
I have run python setup.py sdist for my package, which has created the appropriate tar.gz file. This file is stored on my system at /srv/pkg/mypackage/mypackage-0.1.0.tar.gz.
Now in a virtual environment I would like to install packages either coming from pypi or from the specific local location /srv/pkg.
Is this possible?
PS
I know that I can specify pip install /srv/pkg/mypackage/mypackage-0.1.0.tar.gz. That will work, but I am talking about using the /srv/pkg location as another place for pip to search if I typed pip install mypackage.
What about::
pip install --help
...
-e, --editable <path/url> Install a project in editable mode (i.e. setuptools
"develop mode") from a local project path or a VCS url.
eg, pip install -e /srv/pkg
where /srv/pkg is the top-level directory where 'setup.py' can be found.
I am pretty sure that what you are looking for is called --find-links option.
You can do
pip install mypackage --no-index --find-links file:///srv/pkg/mypackage
From the installing-packages page you can simply run:
pip install /srv/pkg/mypackage
where /srv/pkg/mypackage is the directory, containing setup.py.
Additionally1, you can install it from the archive file:
pip install ./mypackage-1.0.4.tar.gz
1
Although noted in the question, due to its popularity, it is also included.
I am installing pyfuzzybut is is not in PyPI; it returns the message: No matching distribution found for pyfuzzy.
I tried the accepted answer
pip install --no-index --find-links=file:///Users/victor/Downloads/pyfuzzy-0.1.0 pyfuzzy
But it does not work either and returns the following error:
Ignoring indexes: https://pypi.python.org/simple
Collecting pyfuzzy
Could not find a version that satisfies the requirement pyfuzzy (from versions: )
No matching distribution found for pyfuzzy
At last , I have found a simple good way there: https://pip.pypa.io/en/latest/reference/pip_install.html
Install a particular source archive file.
$ pip install ./downloads/SomePackage-1.0.4.tar.gz
$ pip install http://my.package.repo/SomePackage-1.0.4.zip
So the following command worked for me:
pip install ../pyfuzzy-0.1.0.tar.gz.
Hope it can help you.
This is the solution that I ended up using:
import pip
def install(package):
# Debugging
# pip.main(["install", "--pre", "--upgrade", "--no-index",
# "--find-links=.", package, "--log-file", "log.txt", "-vv"])
pip.main(["install", "--upgrade", "--no-index", "--find-links=.", package])
if __name__ == "__main__":
install("mypackagename")
raw_input("Press Enter to Exit...\n")
I pieced this together from pip install examples as well as from Rikard's answer on another question. The "--pre" argument lets you install non-production versions. The "--no-index" argument avoids searching the PyPI indexes. The "--find-links=." argument searches in the local folder (this can be relative or absolute). I used the "--log-file", "log.txt", and "-vv" arguments for debugging. The "--upgrade" argument lets you install newer versions over older ones.
I also found a good way to uninstall them. This is useful when you have several different Python environments. It's the same basic format, just using "uninstall" instead of "install", with a safety measure to prevent unintended uninstalls:
import pip
def uninstall(package):
response = raw_input("Uninstall '%s'? [y/n]:\n" % package)
if "y" in response.lower():
# Debugging
# pip.main(["uninstall", package, "-vv"])
pip.main(["uninstall", package])
pass
if __name__ == "__main__":
uninstall("mypackagename")
raw_input("Press Enter to Exit...\n")
The local folder contains these files: install.py, uninstall.py, mypackagename-1.0.zip
An option --find-links does the job and it works from requirements.txt file!
You can put package archives in some folder and take the latest one without changing the requirements file, for example requirements:
.
└───requirements.txt
└───requirements
├───foo_bar-0.1.5-py2.py3-none-any.whl
├───foo_bar-0.1.6-py2.py3-none-any.whl
├───wiz_bang-0.7-py2.py3-none-any.whl
├───wiz_bang-0.8-py2.py3-none-any.whl
├───base.txt
├───local.txt
└───production.txt
Now in requirements/base.txt put:
--find-links=requirements
foo_bar
wiz_bang>=0.8
A neat way to update proprietary packages, just drop new one in the folder
In this way you can install packages from local folder AND pypi with the same single call: pip install -r requirements/production.txt
PS. See my cookiecutter-djangopackage fork to see how to split requirements and use folder based requirements organization.
Assuming you have virtualenv and a requirements.txt file, then you can define inside this file where to get the packages:
# Published pypi packages
PyJWT==1.6.4
email_validator==1.0.3
# Remote GIT repo package, this will install as django-bootstrap-themes
git+https://github.com/marquicus/django-bootstrap-themes#egg=django-bootstrap-themes
# Local GIT repo package, this will install as django-knowledge
git+file:///soft/SANDBOX/python/django/forks/django-knowledge#egg=django-knowledge
To install only from local you need 2 options:
--find-links: where to look for dependencies. There is no need for the file:// prefix mentioned by others.
--no-index: do not look in pypi indexes for missing dependencies (dependencies not installed and not in the --find-links path).
So you could run from any folder the following:
pip install --no-index --find-links /srv/pkg /path/to/mypackage-0.1.0.tar.gz
If your mypackage is setup properly, it will list all its dependencies, and if you used pip download to download the cascade of dependencies (ie dependencies of depencies etc), everything will work.
If you want to use the pypi index if it is accessible, but fallback to local wheels if not, you can remove --no-index and add --retries 0. You will see pip pause for a bit while it is try to check pypi for a missing dependency (one not installed) and when it finds it cannot reach it, will fall back to local. There does not seem to be a way to tell pip to "look for local ones first, then the index".
Having requirements in requirements.txt and egg_dir as a directory
you can build your local cache:
$ pip download -r requirements.txt -d eggs_dir
then, using that "cache" is simple like:
$ pip install -r requirements.txt --find-links=eggs_dir
What you need is --find-links of pip install.
-f, --find-links If a url or path to an html file, then parse for links to archives. If a local path or
file:// url that's a directory, then look for archives in the directory listing.
In my case, after python -m build, tar.gz package (and whl file) are generated in ./dist directory.
pip install --no-index -f ./dist YOUR_PACKAGE_NAME
Any tar.gz python package in ./dist can be installed by this way.
But if your package has dependencies, this command will prompt error.
To solve this, you can either pip install those deps from official pypi source, then add --no-deps like this
pip install --no-index --no-deps -f ./dist YOUR_PACKAGE_NAME
or copy your deps packages to ./dist directory.
I've been trying to achieve something really simple and failed miserably, probably I'm stupid.
Anyway, if you have a script/Dockerfile which download a python package zip file (e.g. from GitHub) and you then want to install it you can use the file:/// prefix to install it as shown in the following example:
$ wget https://example.com/mypackage.zip
$ echo "${MYPACKAGE_MD5} mypackage.zip" | md5sum --check -
$ pip install file:///.mypackage.zip
NOTE: I know you could install the package straight away using pip install https://example.com/mypackage.zip but in my case I wanted to verify the checksum (never paranoid enough) and I failed miserably when trying to use the various options that pip provides/the #md5 fragment.
It's been surprisingly frustrating to do something so simple directly with pip. I just wanted to pass a checksum and have pip verify that the zip was matching before installing it.
I was probably doing something very stupid but in the end I gave up and opted for this. I hope it helps others trying to do something similar.
In my case, it was because this library depended on another local library, which I had not yet installed. Installing the dependency with pip, and then the dependent library, solved the issue.
If you want to install one local package (package A) to be used inside another local project/package (B) this is quite simple. All you need is to CD to (B) and call:
pip install /path/to/package(A)
Of course you will need to first compile the package (A) with:
sudo python3 ./setup.py install
And, each time you change package A, just run again setup.py in package (A) then pip install ... inside the using project/package (B)
Just add directory on pip command
pip install mypackage file:/location/in/disk/mypackagename.filetype