I have a simple .gitlab-ci.yml script that builds my python project.
image: python:3.9.6-slim-buster
variables:
PIP_DEFAULT_TIMEOUT: 300
before_script:
- pip install poetry==1.1.7
- poetry config virtualenvs.create false
- poetry install
When I run the CI pipeline, I periodically get such errors and the job is interrupted with a failure.
First type of error:
...
• Installing toml (0.10.2)
• Installing uvloop (0.16.0)
• Installing watchgod (0.8.2)
• Installing websockets (10.3)
ConnectionError
HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/flake8-eradicate/1.2.1/json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa5c5625dc0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
at /usr/local/lib/python3.9/site-packages/requests/adapters.py:565 in send
561│ if isinstance(e.reason, _SSLError):
562│ # This branch is for urllib3 v1.22 and later.
563│ raise SSLError(e, request=request)
564│
→ 565│ raise ConnectionError(e, request=request)
566│
567│ except ClosedPoolError as e:
568│ raise ConnectionError(e, request=request)
569│
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
Second type of error:
...
• Installing gitpython (3.1.27)
OSError
Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.9/site-packages/certifi/cacert.pem
at /usr/local/lib/python3.9/site-packages/requests/adapters.py:263 in cert_verify
259│ if not cert_loc:
260│ cert_loc = extract_zipped_paths(DEFAULT_CA_BUNDLE_PATH)
261│
262│ if not cert_loc or not os.path.exists(cert_loc):
→ 263│ raise OSError(
264│ f"Could not find a suitable TLS CA certificate bundle, "
265│ f"invalid path: {cert_loc}"
266│ )
267│
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1
What is most interesting is that these errors are triggered on completely different libraries at different times. I have to do a retry many times so that it installs everything stably. What could be the problem and how to solve it?
For info: I use dockerized gitlab runner with docker executor on CentOS 7
I think this has to do more with poetry.
Could be more likely due to parallel installations.
You can either check restrict multiple installers but this will slow down the installations.
poetry config installer.max-workers=1
Had seen a discussion in poetry where they said this is less likely to happen in 1.2 but since 1.2 isn't release you can use 1.2.0b1 and check
Check this out for more details https://github.com/python-poetry/poetry/issues/3336.
Regarding the
Could not find a suitable TLS CA certificate bundle, invalid path: .../site-packages/certifi/cacert.pem
This is probably caused because poetry install is acting in the very environment where poetry is installed, causing it to uninstall its own dependencies because they aren't listed in the lockfile. This is explicitly warned in CI recommendations in the installation guide:
https://python-poetry.org/docs/#ci-recommendations
If you install Poetry via pip, ensure you have Poetry installed into an isolated environment that is not the same as the target environment managed by Poetry. If Poetry and your project are installed into the same environment, Poetry is likely to upgrade or uninstall its own dependencies (causing hard-to-debug and understand errors).
You would notice because the logs from the install instruction start not creating a virtualenv and follow removing packages that are about to break the installation:
Installing dependencies from lock file
Package operations: 58 installs, 0 updates, 2 removals
• Removing certifi (2022.6.15)
• Removing setuptools (65.3.0)
Solution:
poetry config virtualenvs.create true
and be careful using poetry inside tox (tox creates venv, tox installs poetry, poetry removes certifi)
Regarding the
Max retries exceeded with url: ... Failed to establish a new connection: [Errno -2] Name or service not known
I think this deserves a totally different Q&A, but this might simply be some PyPI downtime you unluckily observed.
Related
I have set up a new environment on conda.
Once it was created, I disabled threading so that the performance of the software I would be using will be maximised using:
conda env config vars set OMP_NUM_THREADS=1
conda env config vars set NUMEXPR_MAX_THREADS=1
I then tried to install the software and got the error
CondaSSLError: OpenSSL appears to be unavailable on this machine. OpenSSL is required to
download and install packages.
Exception: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /conda-forge/win-64/current_repodata.json (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))
I though this could be specific to the software.
To test this, I tried installing python with
conda install -c conda-forge python
I got the same error here
CondaSSLError: OpenSSL appears to be unavailable on this machine. OpenSSL is required to
download and install packages.
Exception: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /conda-forge/win-64/current_repodata.json (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))
It seems to be an issue with the environment, but I have set up other environments on this machine before and never had this issue.
Does anyone know what could be causing this issue?
I have now resolved this!
If anyone else has the same issue, going back into the base environment and then using "conda install -n name_of_environment python" worked.
for my job purposes, I need a MacOS, but there is no possibility to use a real mac. So I decided to set up virtual machine, using Virtualbox. After the machine was successfully setup, I've opened the Terminal app and run the following commands:
1)/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" (Homebrew install)
2)brew install python (Python 3 install)
3)pip3 install numpy
As an output of the last command, I got the next error message:
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
ERROR: Could not find a version that satisfies the requirement numpy (from versions: none)
ERROR: No matching distribution found for numpy
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
Of course, I've tried to find a solution online. Reinstalling python or openssl via Homebrew didn't help. Linking openssl or installing other CA also didn't solve the problem. How can I actually fix this?
P.C. I also know that Apple is a sh*t company and tries to have minimum hardware and software of other companies. That's why they use LibreSSL instead of OpenSSL. But the last one is one that Python requests in order to have the ability to install packages via pip. After python was installed, the following message was in terminal:
This formula is keg-only, which means it was not symlinked into /usr/local.
If you need to have openssl#1.1 first in your PATH run:
echo 'export PATH="/usr/local/opt/openssl#1.1/bin:$PATH"' >> ~/.bash_profile
Running proposed command also doesn't solve the problem. Any thoughts?
I was trying to install Tensorflow by creating an environment but I got an error because I my Self-Signed Certificate. I fixed this once with pip but I don't know how to do that with Anaconda.
The error is:
Collecting package metadata (current_repodata.json): failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/main/win-64/current_repodata.json>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
If your current network has https://www.anaconda.com blocked, please file
a support request with your network engineering team.
SSLError(MaxRetryError('HTTPSConnectionPool(host=\'repo.anaconda.com\', port=443): Max retries exceeded with url: /pkgs/main/win-64/current_repodata.json (Caused by SSLError(SSLError("bad handshake: Error([(\'SSL routines\', \'tls_process_server_certificate\', \'certificate verify failed\')])")))'))
I tried to find a command to fix that but could not find one. I am supposed to be able to create the environment without any issue.
I have searched on Internet and found something related to Conda issue.
Before going to 1) and 2) steps, just type
conda config --set ssl_verify no
Then do following:
Create new environment by typing(optional)
conda create -n tensorflow pip python=3.7
activate tensorflow
Install Tensorflow 1.9
pip install --ignore-installed --upgrade tensorflow==1.9
I didn't get any error with this. If you have errors, comment down below.
no matter what I do I get this error
C:\Users\MyPc>conda update --all
Solving environment: failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://repo.anaconda.com/pkgs/free/win-64/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
If your current network has https://www.anaconda.com blocked, please file
a support request with your network engineering team.
SSLError(MaxRetryError('HTTPSConnectionPool(host=\'repo.anaconda.com\', port=443): Max retries exceeded with url: /pkgs/free/win-64/repodata.json.bz2 (Caused by SSLError("Can\'t connect to HTTPS URL because the SSL module is not available."))'))
I have searched all the internet , re-installed anaconda and did whatever is there on suggestion but this issue is still there.
Windows 10
C:\Users\MyPc>anaconda --version
anaconda Command line client (version 1.7.2)
C:\Users\MyPc>conda --version
conda 4.5.12
In my case I got this kind of error message when I try to run this command
conda install tensorflow
here is the Error message
CondaSSLError: OpenSSL appears to be unavailable on this machine. OpenSSL is required to download and install packages.
Exception: HTTPSConnectionPool(host='repo.anaconda.com', port=443): Max retries exceeded with url: /pkgs/main/win-64/current_repodata.json (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))
This is the solution for that
step 01
go to your installation path of anaconda3
step 02
now go to the this file path
anaconda3\Library\bin
step 03
now select this DLL file and copy that
libcrypto-1_1-x64.dll
libssl-1_1-x64.dll
step 04
after that go to this file path and paste that to inside of that folder
anaconda3\DLLs
I had this same issue, and the thing that fixed it for me was to install an earlier 32-bit version of Conda. For some reason the newer 64-bit version seemed prone to this error. You can find an earlier version of Conda here:
https://repo.continuum.io/archive/
You should search for an Anaconda3 version that only has x86, not x86_64.
I had the same problem, the simple solution is:
open anaconda navigator from start menu, then run CMD.exe Prompt, and install from there, that's it.
I have been trying to install pymc for some time on a Windows PC behind a very complicated proxy; effectively making this an installation on a computer not connected to the internet. I have tried - unsuccessfully - to set a proxy in the condarc file but I still get error messages
conda install -c https://conda.binstar.org/pymc pymc
Fetching package metadata: SSL verification error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
To solve this, you need to download the tar file (even if using Windows) that the installer is trying to fetch and then use offline mode:
Use this command (which will throw an error), to determine the file to be downloaded:
>conda install -c https://conda.binstar.org/pymc pymc
Fetching package metadata: SSL verification error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
....
Solving package specifications: ......................
The following packages will be downloaded:
package | build
---------------------------|-----------------
pymc-2.3.5 | np110py35_0 402 KB defaults
The following NEW packages will be INSTALLED:
pymc: 2.3.5-np110py35_0 defaults
Proceed ([y]/n)? y
Fetching packages ...
Could not connect to https://repo.continuum.io/pkgs/free/win-64/pymc-2.3.5-np110py35_0.tar.bz2
... [error message continues]...
Now download the tar file mentioned in the error message:
https://repo.continuum.io/pkgs/free/win-64/pymc-2.3.5-np110py35_0.tar.bz2
And then run this command with the path to the tar file:
>conda install --offline C:\pymc-2.3.5-np110py35_0.tar.bz2
Just of notes: "-c conda-forge" might be handy for some other packages. For example:
conda install -c conda-forge python-levenshtein
Adding to the solution above. Anyone met "SSL verification error" can just turn the verification step off temporarily in ~/.condarc
channels:
- defaults
ssl_verify: true
In this way, you could install from your local hub.
I was not able to run offline installation in Anaconda (on Win10) because Anaconda always tried to connect to remote channels or collecting metadata process failed.
To solve this I had to:
Download requested module as bz2 file (or convert tar.gz into tar.bz2)
Run Anaconda prompt (ev. navigate to folder with bz2 file)
Run offline installation with conda install path-to-bz2, i.e conda install zeep-4.0.0.tar.bz2