I'm trying to install a custom package from a different index other than pypi in my pod.
I created a pip.conf and I run this commands in my dockerfile:
ADD docker_files/pip.conf /etc/pip.conf
ENV PIP_CONFIG_FILE /etc/pip.conf
and when I run RUN echo python3 -m pip config debug I can see in the output that pip does recognise my pip conf and all of the indexes in it. Output:
env_var: PIP_CONFIG_FILE='/etc/pip.conf' env: /etc/pip.conf, exists: True global.index-url: <custom> global.extra-index-url: <more custom> global: /etc/xdg/pip/pip.conf, exists: False /etc/pip.conf, exists: True global.index-url: <custom> global.extra-index-url: <more custom> site: /usr/local/pip.conf, exists: False
but when it searches for the package and it can't find it in pypi, it doesn't search my other indexes:
Searching for <package>
Reading https://pypi.org/simple/<package>/
Couldn't find index page for <package> (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
No local packages or working download links found for <package>
error: Could not find suitable distribution for Requirement.parse(<package>)
The command '/bin/sh -c python setup.py install' returned a non-zero code: 1
I also read that sometimes if the path $HOME/Library/Application Support exists it overwrite everything, but I checked and it does not. I'm using python 3.8.13, I also tried copying the pip conf file to /usr/local/pip.conf and removing the var PIP_CONFIG_FILE from the dockerfile, Nothing worked.
Did I miss something?
Related
I was working on a Python/Flask project, and I am not sure what happened but all of a sudden when I opened my Terminal, it seemed like pip was deleted; when I tried running things like 'pip list' from different directories (including my global one), I got the following
zsh: command not found: pip
Needless to say, when I tried running packages that I had previously installed with pip like Flask, I got the same zsh error as per above.
And when I reinstalled pip and ran pip list, it seems like all of my Python packages got deleted?:
pip 22.1.2
setuptools 58.1.0
wheel 0.37.1
Any ideas what could have caused this? I can reinstall the packages that is no issue. But I fear this will happen again in the future and I have no idea what I did that could have caused this...
Could this have been a $PATH issue?
When I run echo $PATH in my global directory, I get:
/Library/Frameworks/Python.framework/Versions/3.10/bin
/opt/homebrew/bin
/opt/homebrew/sbin
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/Library/Apple/usr/bin
#mast o;
Here is what I get when running find / -name pip:
/Users/patrickchong/PycharmProjects/Live_Server/venv/bin/pip
/Users/patrickchong/PycharmProjects/Live_Server/venv/lib/python2.7/site-packages/pip
/Users/patrickchong/PycharmProjects/JP interview/venv/bin/pip
/Users/patrickchong/PycharmProjects/JP interview/venv/lib/python2.7/site-packages/pip
/Users/patrickchong/opt/anaconda3/bin/pip
/Users/patrickchong/opt/anaconda3/pkgs/pip-21.2.4-py310hecd8cb5_0/bin/pip
/Users/patrickchong/opt/anaconda3/pkgs/pip-21.2.4-py310hecd8cb5_0/lib/python3.10/site-packages/pip
/Users/patrickchong/opt/anaconda3/pkgs/pip-21.2.4-py39hecd8cb5_0/bin/pip
/Users/patrickchong/opt/anaconda3/pkgs/pip-21.2.4-py39hecd8cb5_0/lib/python3.9/site-packages/pip
/Users/patrickchong/opt/anaconda3/lib/python3.9/site-packages/pip
find: /Users/patrickchong/.Trash: Operation not permitted
/Users/patrickchong/.conda/envs/Crypto Trading Bot project/bin/pip
/Users/patrickchong/.conda/envs/Crypto Trading Bot project/lib/python3.10/site-packages/pip
/Users/patrickchong/.conda/envs/AlgoExpert Questions/bin/pip
/Users/patrickchong/.conda/envs/AlgoExpert Questions/lib/python3.10/site-packages/pip
/Applications/iFunia YouTube Downloader.app/Contents/Frameworks/ASVDSDK.framework/Versions/A/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip
/opt/homebrew/lib/python3.9/site-packages/pip
/opt/homebrew/Cellar/python#3.9/3.9.13_1/libexec/bin/pip
/opt/homebrew/Cellar/python#3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/share/doc/python3.9/examples/Tools/msi/pip
find: /dev/fd/3: Not a directory
find: /dev/fd/4: Not a directory
find: /dev/fd/6: Not a directory
(I omitted a lot of the "find ....", because it was a very long list)
Here is what I get when I run pip3:
patrickchong#Patricks-MacBook-Pro ~ % pip3
Usage:
pip3 <command> [options]
Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
config Manage local and global configuration.
search Search PyPI for packages.
cache Inspect and manage pip's wheel cache.
index Inspect information available from package indexes.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
debug Show information useful for debugging.
help Show help for commands.
General Options:
-h, --help Show help.
--debug Let unhandled exceptions propagate outside the
main subroutine, instead of logging them to
stderr.
--isolated Run pip in an isolated mode, ignoring
environment variables and user configuration.
--require-virtualenv Allow pip to only run in a virtual environment;
exit with an error otherwise.
-v, --verbose Give more output. Option is additive, and can be
used up to 3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be
used up to 3 times (corresponding to WARNING,
ERROR, and CRITICAL logging levels).
--log <path> Path to a verbose appending log.
--no-input Disable prompting for input.
--proxy <proxy> Specify a proxy in the form
scheme://[user:passwd#]proxy.server:port.
--retries <retries> Maximum number of retries each connection should
attempt (default 5 times).
--timeout <sec> Set the socket timeout (default 15 seconds).
--exists-action <action> Default action when a path already exists:
(s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
--trusted-host <hostname> Mark this host or host:port pair as trusted,
even though it does not have valid or any HTTPS.
--cert <path> Path to PEM-encoded CA certificate bundle. If
provided, overrides the default. See 'SSL
Certificate Verification' in pip documentation
for more information.
--client-cert <path> Path to SSL client certificate, a single file
containing the private key and the certificate
in PEM format.
--cache-dir <dir> Store the cache data in <dir>.
--no-cache-dir Disable the cache.
--disable-pip-version-check
Don't periodically check PyPI to determine
whether a new version of pip is available for
download. Implied with --no-index.
--no-color Suppress colored output.
--no-python-version-warning
Silence deprecation warnings for upcoming
unsupported Pythons.
--use-feature <feature> Enable new functionality, that may be backward
incompatible.
--use-deprecated <feature> Enable deprecated functionality, that will be
removed in the future.
After running pip freeze I noticed the following warning on top of the list:
WARNING: Could not parse requirement: -atplotlib
So I checked the installed packages using pip list, and indeed the following is considered a package:
Package Version
--------------- -------
-atplotlib 3.0.3
I assume I probably had a typo when installing/upgrading matplotlib, which led to the aforementioned "package" being installed.
But I am not able to remove it as pip uninstall -atplotlib is read as a command and returns the following error:
No such option: -a
I found the following folders:
C:\Users\name\Anaconda3\Lib\site-packages\~atplotlib
C:\Users\name\Anaconda3\Lib\site-packages\~atplotlib-3.0.3-py3.7.egg-info
Is it safe, and sufficient, to remove them?
It is safe to delete the offending folder(s) from your site-packages directory.
More info below (source):
When uninstalling a package, pip will now rename it in place to a name
that cannot be imported, and once it has confirmed that everything
will succeed (including installing new versions if it’s doing an
upgrade), only then will it delete those folders. If something fails,
it renames them back.
Previously, it would copy the entire contents to another directory,
and potentially another drive, and then copy them back if it needed.
So this change is a significant performance improvement, especially
for packages with a lot of files in them.
What you’re seeing here is that the deletion failed for some reason -
perhaps pip crashed? - and so the directories were not removed. I
thought pip ignored them completely, but perhaps something else
changed since I tested that?
The directories are safe to delete.
EDIT: According to this link, provided by Lawrence in his answer
looking for and deleting the incorrectly named folders in your site-package directory should solve the issue.
If this is not sufficient, continue the cleaning as explained below.
Searching for the name of the broken package (without the leading dash) allowed me to find the following two folders:
C:\Users\name\Anaconda3\Lib\site-packages~atplotlib
C:\Users\name\Anaconda3\Lib\site-packages~atplotlib-3.0.3-py3.7.egg-info
Following Hoefling's comment (below)
I checked the SOURCES.txt file in the egg-info directory %dir%/~atplotlib-3.0.3-py3.7.egg-info/SOURCES.txt. Went through the list of paths in this file and made sure all paths listed did not contained ~. Then I renamed the directory ~atplotlib-3.0.3-py3.7.egg-info into atplotlib-3.0.3-py3.7.egg-info (removed the tilde ~).
Finally, I ran pip uninstall atplotlib, which prompted the following:
Uninstalling atplotlib-3.0.3:
Would remove:
C:\Users\name\Anaconda3\Lib\site-packages\atplotlib-3.0.3-py3.7.egg-info
C:\Users\name\Anaconda3\Lib\site-packages\matplotlib
C:\Users\name\Anaconda3\Lib\site-packages\pylab.py
Proceeding with the removal solved the issue (the warning disappeared and the package is not anymore on the package list.
I received the following error myself, after executing:
python -m pip install --upgrade pip --user
WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages)
I went to the file specification within the C:\
Located the -ip (it was located at the top) and I deleted it.
Executed the code again
Results:
Requirement already satisfied: pip in c:\users\GhostFace\appdata\roaming\python\python310\site-packages (21.3.1)
I had to do rm -rf atplotlib after removing ~ character as beginning of the file name.
$ pip freeze | grep atplotlib
$ cd /home/user/.local/lib/python3.8/site-packages
$ mv \~ atplotlib atplotlib
$ pip freeze | grep atplotlib
atplotlib ==3.0.3
$ pip uninstall atplotlib
WARNING: Skipping atplotlib as it is not installed.
$ rm -rf atplotlib
After running pip freeze I noticed the following warning on top of the list:
WARNING: Could not parse requirement: -atplotlib
So I checked the installed packages using pip list, and indeed the following is considered a package:
Package Version
--------------- -------
-atplotlib 3.0.3
I assume I probably had a typo when installing/upgrading matplotlib, which led to the aforementioned "package" being installed.
But I am not able to remove it as pip uninstall -atplotlib is read as a command and returns the following error:
No such option: -a
I found the following folders:
C:\Users\name\Anaconda3\Lib\site-packages\~atplotlib
C:\Users\name\Anaconda3\Lib\site-packages\~atplotlib-3.0.3-py3.7.egg-info
Is it safe, and sufficient, to remove them?
It is safe to delete the offending folder(s) from your site-packages directory.
More info below (source):
When uninstalling a package, pip will now rename it in place to a name
that cannot be imported, and once it has confirmed that everything
will succeed (including installing new versions if it’s doing an
upgrade), only then will it delete those folders. If something fails,
it renames them back.
Previously, it would copy the entire contents to another directory,
and potentially another drive, and then copy them back if it needed.
So this change is a significant performance improvement, especially
for packages with a lot of files in them.
What you’re seeing here is that the deletion failed for some reason -
perhaps pip crashed? - and so the directories were not removed. I
thought pip ignored them completely, but perhaps something else
changed since I tested that?
The directories are safe to delete.
EDIT: According to this link, provided by Lawrence in his answer
looking for and deleting the incorrectly named folders in your site-package directory should solve the issue.
If this is not sufficient, continue the cleaning as explained below.
Searching for the name of the broken package (without the leading dash) allowed me to find the following two folders:
C:\Users\name\Anaconda3\Lib\site-packages~atplotlib
C:\Users\name\Anaconda3\Lib\site-packages~atplotlib-3.0.3-py3.7.egg-info
Following Hoefling's comment (below)
I checked the SOURCES.txt file in the egg-info directory %dir%/~atplotlib-3.0.3-py3.7.egg-info/SOURCES.txt. Went through the list of paths in this file and made sure all paths listed did not contained ~. Then I renamed the directory ~atplotlib-3.0.3-py3.7.egg-info into atplotlib-3.0.3-py3.7.egg-info (removed the tilde ~).
Finally, I ran pip uninstall atplotlib, which prompted the following:
Uninstalling atplotlib-3.0.3:
Would remove:
C:\Users\name\Anaconda3\Lib\site-packages\atplotlib-3.0.3-py3.7.egg-info
C:\Users\name\Anaconda3\Lib\site-packages\matplotlib
C:\Users\name\Anaconda3\Lib\site-packages\pylab.py
Proceeding with the removal solved the issue (the warning disappeared and the package is not anymore on the package list.
I received the following error myself, after executing:
python -m pip install --upgrade pip --user
WARNING: Ignoring invalid distribution -ip (c:\python310\lib\site-packages)
I went to the file specification within the C:\
Located the -ip (it was located at the top) and I deleted it.
Executed the code again
Results:
Requirement already satisfied: pip in c:\users\GhostFace\appdata\roaming\python\python310\site-packages (21.3.1)
I had to do rm -rf atplotlib after removing ~ character as beginning of the file name.
$ pip freeze | grep atplotlib
$ cd /home/user/.local/lib/python3.8/site-packages
$ mv \~ atplotlib atplotlib
$ pip freeze | grep atplotlib
atplotlib ==3.0.3
$ pip uninstall atplotlib
WARNING: Skipping atplotlib as it is not installed.
$ rm -rf atplotlib
I need to debug some failures to install packages using pip and it seems that once I get the error pip is removing all the temporary files, preventing me from looking inside and finding what happened.
The gist below contains the error that I get but I do want to find-out which config file is generating this error and all the files from tmp are removed by the time the script ends.
https://gist.github.com/ssbarnea/b373062dd45de92735c7482b2735c5fb
From my experience with pip 19.0.3, it is sufficient to use the --no-clean option, which is available to both pip install and pip wheel.
It can also be specified in the respective sections of the pip configuration file:
[install]
no-clean = true
[wheel]
no-clean = true
I'm using pip with pip-compile (installed this way: pip install pip-tools)
I got the following error when I run the pip-compile -v command:
pip-compile does not support URLs as packages, unless they are editable. Perhaps add -e option? (constraint was:
aldryn-django==1.8.7.0 from
https://control.aldyn.com/api/v1/apps/serve/aldryn-django/1.8.7.0/592213b1-e515-4447-8ef0-850713571a42/aldryn-django-1.8.7.0.tar.gz#egg=aldryn-django==1.8.7.0
(from -r requirements.in (line 2)))
I have tried with the -e option, but this causes another problem.
pip.exceptions.InstallationError: https://control.aldryn.com/api/v1/apps/serve/aldryn-django/1.8.7.0/592213b1-e515-4447-8ef0-850713571a42/aldryn-django-1.8.7.0.tar.gz#egg=aldryn-django==1.8.7.0 should either be a path to a local project or a VCS url beginning with svn+, git+, hg+, or bzr+
Below is an short extract of my requirements.in file:
\# <INSTALLED_ADDONS> # Warning: text inside the INSTALLED_ADDONS tags is auto-generated. Manual changes will be overwritten.
https://control.aldryn.com/api/v1/apps/serve/aldryn-django/1.8.7.0/592213b1-e515-4447-8ef0-850713571a42/aldryn-django-1.8.7.0.tar.gz#egg=aldryn-django==1.8.7.0
...
\# </INSTALLED_ADDONS>
I'm using Docker container based on the python:2.7-slim image.
The requirements.in work well on one other similar docker container.
I don't know why on mine, pip-compile does not work...
Have you any idea?
run the command from inside the docker container. Divio somehow seem to have fixed this in their install