We Keep Coding

Unable to find installation candidates for nvidia-cudnn-cu11 when trying to install kornia

I'm trying to install kornia using poetry.
System details
Python 3.8.5 (I have to use this version)
poetry 1.1.13 (upgrading to v1.2 is not an option)
I've previously installed kornia (poetry add kornia) but for some unknown reason I'm now getting the following traceback:
RuntimeError
Unable to find installation candidates for nvidia-cudnn-cu11
(8.5.0.96)
at ~.poetry\lib\poetry\installation\chooser.py:72 in choose_for
68│
69│ links.append(link)
70│
71│ if not links:
→ 72│ raise RuntimeError(
73│ "Unable to find installation candidates for {}".format(package)
74│ )
Since this wasn't a very helpful traceback I thought I'd try to pip install to see if I could get kornia installed somehow, then repeat the process using poetry.
First I ran poetry run pip install nvidia-pyindex, then poetry run pip install kornia. kornia was installed, giving me confidence that I might be able to do the same using poetry.
So I ran poetry add nvidia-pyindex which installed, but when I ran poetry add kornia I got the same RuntimeError.
I wondered if having poetry add without pip uninstall might have caused issues. So I started from scratch - I ran:
rm -rf ~/.cache/pypoetry/cache/repositories/pypi/
poetry rm {name of venv}
git checkout master pyproject.toml poetry.lock
poetry add nvidia-pyindex
which created the venv (by the same name as previously - not sure if this is an issue in itself, since the name of the venv was first created in PyCharm and I wonder if there are some files lingering in places I don't know that is the root of my problems?), then poetry add kornia but got the same error.
I also tried to clear poetry's cache as suggested here but that didn't help. I didn't find a solution here. I don't believe this applies since poetry was installed properly and is working fine otherwise. I read through this but didn't see any working solution. I tried a fix mentioned here, which was to add
[[tool.poetry.source]]
name = "nvidia"
url = "https://pypi.ngc.nvidia.com"
secondary = true
to pyproject.toml, hoping that it would avoid the error when poetry tries to install nvidia-cudnn-cu11, but it didn't help either.
I tried poetry add nvidia-cudnn-cu11 and poetry add nvidia-cudnn-cu116 hoping that having added the ndvidia repo as a secondary source would get kornia's dependencies installed before trying to install kornia itself.
{I don't know if secondary sources are only used when poetry adding a package (i.e. kornia) and "ignored" for installation of dependencies (nvidia-cudnn-cu11) of the package to be installed (kornia).}
In any case, that didn't work either.
Running poetry source add nvidia https://pypi.ngc.nvidia.com didn't work - presumably because poetry source add was added to poetry v1.2 and was not part of v1.1.13.
I also tried poetry add --source nvidia https://pypi.ngc.nvidia.com but I got
PermissionError
[Errno 13] Permission denied:
'C:\Users\...\AppData\Local\Temp\tmp2t6am7ei'
at ~.poetry\lib\poetry\utils\helpers.py:101 in download_file
97│
98│ with get(url, stream=True) as response:
99│ response.raise_for_status()
100│
→ 101│ with open(dest, "wb") as f:
102│ for chunk in response.iter_content(chunk_size=chunk_size):
103│ if chunk:
104│ f.write(chunk)
I'm out of ideas and hoped someone might help.

could this be related to this problem?
https://github.com/python-poetry/poetry/issues/6939
https://github.com/pytorch/pytorch/issues/88049
I had a similar issue and was able to fix it by making the torch dependency explicit and install torch version < 1.13:
[tool.poetry.dependencies]
...
torch = "~1.12.1"

Airflow - no module named "airflow.providers" or "airflow.contrib.providers"

We are trying to import both MongoHook and GCSToLocalFilesystemOperator into our Airflow Project:
docs for MongoHook
docs for GCS Operator
...per this post, "Providers package is no longer included with Airflow, but you can separately install them with pip using the specific backport package"
I have had no luck with this. For our DockerFile, we started using this image from the puckel/docker-airflow repo on GitHub.
I've tried adding these two lines to our requirements.txt
apache-airflow-backport-providers-mongo
apache-airflow-backport-providers-google
...and I've tried adding these lines to our DockerFile:
&& pip install apache-airflow-backport-providers-mongo \
&& pip install apache-airflow-backport-providers-google \
...and we're getting the errors mentioned in the title. How can I install these into our docker image/container to use them in our Airflow project?
edit: checking quickly to see if the issue was that I simply wasn't rebuilding the image
edit2: rebuilding the image was needed; however, now we are receiving the error No module named 'airflow.utils.process_utils', which we get when importing GCSToLocalFilesystemOperator. Not sure why Airflow took providers out of the base install, this is very frustrating...

The answer to my original question was that I needed to rebuild the docker image (DOH).
The answer to my second question, I think, was that I was installing these packages twice (once with an explicit pip install in the Dockerfile, and once in the requirements.txt, which is also pip installed in the Dockerfile). When I removed the duplicate install, it seems to be working now.

Python pip install gives “Command ”python setup.py egg_info“ failed with error code 1”

Edit: Yes I know this question already exists, except my question is a bit different and none of the solutions fixed it.
I do most of my Python stuff when I'm at work and not on my personal machine, but I decided to install it on my personal computer as well. I fresh installed python 3.6.1, and created a virtual environment with virtualenv. Then within the virtualenv I tried to pip install urllib (or any module) and I received the error:
(pdbot) C:\Users\user\Documents\pdbot>pip install urllib
Collecting urllib
Using cached urllib-1.21.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\setup.py", line 191
s.connect((base64.b64decode(rip), 017620))
^
SyntaxError: invalid token
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I read elsewhere that this error had something to do with setuptools not being properly installed. So I ran this to attempt to fix the issue:
easy_install -U setuptools
I ended up receiving an even weirder error next:
(pdbot) C:\Users\zeke\Documents\pdbot>easy_install -U setuptools
Searching for setuptools
Reading https://pypi.python.org/simple/setuptools/
Downloading https://pypi.python.org/packages/a9/23/720c7558ba6ad3e0f5ad01e0d6ea2288b486da32f053c73e259f7c392042/setuptools-36.0.1.zip#md5=430eb106788183eefe9f444a300007f0
Best match: setuptools 36.0.1
Processing setuptools-36.0.1.zip
Writing C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\setup.cfg
Running setuptools-36.0.1\setup.py -q bdist_egg --dist-dir C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\egg-dist-tmp-8apak7kn
warning: no files found matching '*' under directory 'setuptools\_vendor'
Copying setuptools-36.0.1-py3.6.egg to c:\users\zeke\documents\pdbot\lib\site-packages
Adding setuptools 36.0.1 to easy-install.pth file
Installing easy_install-script.py script to c:\users\zeke\documents\pdbot\Scripts
Installing easy_install.exe script to c:\users\zeke\documents\pdbot\Scripts
error: [WinError 5] Access is denied: 'c:\\users\\zeke\\documents\\pdbot\\Scripts\\easy_install.exe'
This looks like a permissions error, but I ran these both in an administrator command prompt (Windows 10) and got the same result. I am the only user on this computer and I have all admin permissions. Is this virtualenv causing an issue? How do I remedy it?
EDIT: I was able to fix the permissions issue by leveraging the python executable like so:
python -m easy_install -U setuptools
But it didn't fix the python setup.py egg_info issue. I still get this error message when trying to pip install anything:
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I have tried both python -m pip install urllib and pip install urllib and neither work.

I had the same problem when trying to install urllib, but after doing a pip search urllib, I discovered that the problem was due to the version of urllib. From the search:
$ pip search urllib
...
> urllib5 (5.0.0) - Just increment the number and create a new lib. Never fix the original one.
At the end, a simple
pip install urllib5
within an elevated shell solved it.

Your problem has to do with permissions. The related/similar tools setup_tools, easy_install, and pip all tend to set a default set of permissions on files and folders they try to create in the package installation folder(s), rather than trying to match access permissions of the location they're installing in.
On Linux systems, where files and folders individually have permissions, this is frequently bypassed with the sudo command. On Windows, the equivalent is to run the installer as an Administrator. Since you're in the console, you have to open a console with Administrator privileges to run the pip command in.
Notable under Windows, the modules installed with pip from an Administrator console are still accessible to all users of the system that have the proper path in the PYTHONPATH system environment variable. Under Linux however, the problem is exacerbated by the fact that the files themselves may not be created with read and execute access for other users and may need to have their permissions manually modified after installation.
WARNING: urllib vs urllib2 vs urllibx
Both other answers claim that the problem is you're not specifying the correct "version" of the module in the call to pip. Neither is correct, as the error clearly indicates an installation folder access permissions violation causing the failure, but they also incorrectly recommended VERY unsafe behavior.
pip install urllib != pip install urllib5 these are two completely different packages.
The documentation for pip (https://packaging.python.org/tutorials/installing-packages/#id17) clearly says the way to specify a module version explicitly is pip install 'urllib==5'.
As part of how the package management engine implemented by pip works, running the command pip install urllib will always try to use the latest version of the urllib package, so you shouldn't need to specify the version unless you have some reason that you need a very specific version of the module.

There are two points to make in order to answer your question:
1. You are lucky you did not install that package!
The package you were trying to install was a maliciously created python package that was designed to look like a real package (in this case urllib3). If you had installed it, the package would have operated as normal except it would have sent some basic information about the system on which you installed the package to a URL (you can see more details on this here). You can read more about this fake package at either of the following links:
https://app.threatconnect.com/auth/incident/incident.xhtml?incident=5256822&owner=Common%20Community (you can sign up for a free account to view this one)
http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/index.html
Sending basic information about your systems to an unknown source isn't the worst thing you could do, but is certainly something you want to avoid when possible.
2. To properly install a package...
Specifically urllib:
To install urllib, you need to specify the version of the package you would like to install. For example, pip install urllib3.
Any package in general:
As #Elisabete Coelho suggested, you can use the pip search <package-name> feature to view the available packages. This is not perfect, however, as it may list malicious libraries like the one you were trying to install. A good guideline is that you should follow the installation instructions in a package's documentation closely to avoid any unforeseen issues. This is just an unfortunate necessity of living in a world where people make pretend python packages.

Conda 'ImportError: No module named ruamel.yaml.comments'

Conda gives error when I run any command with it.
Traceback (most recent call last):
File "/usr/local/bin/conda", line 7, in <module>
from conda.cli.main import main
File "/usr/local/lib/python2.7/dist-packages/conda/cli/__init__.py", line 8, in <module>
from .main import main # NOQA
File "/usr/local/lib/python2.7/dist-packages/conda/cli/main.py", line 46, in <module>
from ..base.context import context
File "/usr/local/lib/python2.7/dist-packages/conda/base/context.py", line 18, in <module>
from ..common.configuration import (Configuration, MapParameter, PrimitiveParameter,
File "/usr/local/lib/python2.7/dist-packages/conda/common/configuration.py", line 40, in <module>
from ruamel.yaml.comments import CommentedSeq, CommentedMap # pragma: no cover
ImportError: No module named ruamel.yaml.comments

The module ruamel.yaml.comments will normally be loaded from site-packages/ruamel/yaml/comments.py, and not from site-packages/ruamel_yaml/comments.py
Conda seems to have problems with properly supporting namespaces (ruamel.) which I can only attribute to not (yet) being fully pip compatible. That although "namespaces are a honking good idea", and package namespaces have been around for many years.
Assuming you can extend "conda" installations with pip you could try to do a normal install of ruamel.yaml with:
pip install ruamel_yaml==0.11.14
I would not normally recommend such an old version, but that is more likely to work in combination with the version conda uses itself internally.
The alternative would be to switch to using python and pip without conda, that way you can just use the latest version of software from PyPI.

Try pip install ruamel.yaml
It works for me.

Try conda install ruamel.yaml ... pip didnt work for me

Try sudo pip install ruamel_yaml

I went into this file:
/anaconda2/lib/python2.7/site-packages/dateparser/utils/__init__.py
edited this line:
import ruamel.yaml as yaml
to read
import ruamel_yaml as yaml
Changing the dot to an underscore worked for me.... I hope it works for you.

this worked for me:
pip install --upgrade ruamel.yaml --ignore-installed ruamel.yaml
from an answer in matsci.org
https://matsci.org/t/modulenotfounderror-no-module-named-ruamel/36183

The above answer didn't work for me. I had to do a fresh install of the core conda components as described in the conda docs here. Copy and pasted below:
Issue: My conda is broken and I want to fix it without blowing away the current installation
I am getting a conda error and want to reinstall Miniconda to fix it but when I try, it gives me the error that Miniconda (or Anaconda) is already installed and will not let me continue. I want to force the installation.
Resolution: Install Miniconda using the -f (force) option
Download and install the appropriate Miniconda for your computer operating system from the Miniconda download page using the force or -f option as shown:
bash Miniconda3-latest-MacOSX-x86_64.sh -f
NOTE: Substitute the appropriate filename and version for your
operating system.
NOTE: Be sure that you install to same install location as your
existing install so it overwrites the core conda files and does not
install a duplicate in a new folder.

Go to anaconda3\lib\site-packages\rpcq_base.py
and change line #22 :
from ruamel import yaml
to
from ruamel_yaml as yaml

This might not be a popular answer, but it finally helped me after many hours of troubleshooting:
Uninstall conda (I used this stack overflow solution) and also rm -rf miniconda3 in my home directory, fwiw.
Reinstalled conda using data camp's tutorial.
No other solutions worked for me after lots of head banging.

For python3 use
pip3 install ruamel_yaml
if pip3 not installed try at first
sudo apt install python3-pip

For me this was a conda/pip error. I'd tried to install (cwltool in my case) through pip.
It completed successfully, but then running any command gave me the error like above.
ImportError: No module named ruamel.yaml.
It turned out that the pip binary wasn't part of my conda env and was installing cwltool into a completely separate location.
To resolve the issue I completed the following:
conda activate <env I want to install cwltool into>
conda install -y pip
# Run 'rehash' now if you're using zsh to ensure you're using the right pip
pip install cwltool
cwltool -h

To add to what #user612161 has said, go to the directory of parent module (dateparser in this case) requiring ruamel.yaml:
cd anaconda2/lib/python2.7/site-packages/dateparser
and change all occurrences of ruamel.yaml into ruamel_yaml by the following command (Linux):
find . -name '*.py' | xargs sed -i 's/ruamel.yaml/ruamel_yaml/g'

The quick and easy is to ignore the previously installed version in an upgrade
pip install --ignore-installed ruamel_yaml==0.17.4

I was trying to link Bloomberg to Python
pip install --index-url=https://bcms.bloomberg.com/pip/simple blpapi
pip install xbbg
so far, so good.... then I tried to import a module from the package xbbg:
from xbbg import blp
and I was faced with an error, it couldn't find "ruamel.yaml" within the "param.py" within the xbbg module
When I dug into the folder C:/Anaconda3/Lib/site-packages I could see that there was a folder there called ruamel_yaml so I went back to the param.py file and edited ruamel.yaml to be ruamel_yaml as suggested in other posts.
"from xbbg import blp" now worked and I'm able to take data directly from Bloomberg into Python now. Problem solved.
I have a feeling that this issue is being caused by downloading different versions at different times as I've found the learning curve to get setup on Python difficult with many false starts. I was tearing my hair out a bit because I just got Python up and running linked to Bloomberg on my work pc but when I tried to link Bloomberg up to Python on my laptop it kept getting stuck with the "ruamel" issue. The version of Python on my laptop is much older than the version on my work pc. What makes me think that its a version issue is that I did not have to edit ruamel.yaml to be ruamel_yaml in order for me to link Python and BB.
These are just ideas, I'm too inexperienced at this stage to offer much more than to share what happened.

Cannot import sqlite3 in Python3

I am unable to import the sqlite3 module in Python, version 3.5.0. Here's what I get:
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/sqlite3/__init__.py", line 23, in <module>
from sqlite3.dbapi2 import *
File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named '_sqlite3'
I know, I know, there are PLENTY of StackOverflow posts and support forums across the web where people complain about this problem, but none of the posted solutions have worked for me so far. Here's where I've been:
I also have Python 2.6.6 installed on this server, which is running CentOS 6.8 x86_64. I can open up the Python REPL and import sqlite3 just fine when using Python 2.6.6. I can also use sqlite3 from straight from bash and nothing seems awry.
This helpful question looked promising. I tried to re-configure and re-compile Python3.5 with the --enable-loadable-sqlite-extensions option, as user jammyWolf suggested. Nope, same error still occurs.
I've been using virtual environments like a good boy, but I have root access to this server. So, I was a bad boy and ran python3 as root without any virtualenvs activated. Still no luck. So I don't think it has anything to do with permissions.
I noticed that in the error message, it says No module named '_sqlite3'. This thread suggests that the underscore before the module name means that the module is an implementation detail, and isn't exposed in the API. ... I'm not sure what to make of this information, but there may be a hint somewhere in there.
Any ideas?

Falsetru is correct, I am going to go into a bit more detail for those not familiar (linux instructions). If you are getting this error, chances are you are using a version of python that was compiled without the correct headers. Here's a step by step guide to get it sorted. (Python 3.X.X instructions)
Install the required sqlite libraries
sudo apt-get install libsqlite3-dev
Uninstall python (I'm using python 3.6.5 as an example in this guide)
sudo apt-get remove python3.6
Download python from source
cd /tmp && wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
Unpack the archive
tar -xvf Python-3.6.5.tgz
Configure
cd Python-3.6.5 && ./configure
Make and install (and go make coffee while you're at it)
make && sudo make install
If you did everything correctly running "python3.6 -V" should give you your python version. Note you will have to rebuild any virtual environments you have as well.
One final caveat you may encounter.
zipimport.ZipImportError: can't decompress data; zlib not available
This happens if you don't have the following zlib library installed:
sudo apt-get install zlib1g-dev

Install sqlite-devel package which includes header, library that is required to build sqlite3 extension.
yum install sqlite-devel
NOTE: Python does not include sqlite3 library itself, but an extension module (wrapper).

I copied sqlite3.dll to this folder:
C:\Users\*****\Anaconda3\DLLs
It worked after two weeks trying everything else.