python-request with OpenSSL on macOS Sierra - python

Hi allI met a problem when I tried to use python request package, you may find the error as follows:
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 512, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)
Then I updated the openssl library to the latest version with brew in terminal.
>brew update
>brew install openssl
However while linking the latest openssl with
>brew link openssl -force
it came up error from macOS:
Warning: Refusing to link: openssl
Linking keg-only openssl means you may end up linking against the insecure,
deprecated system OpenSSL while using the headers from Homebrew's openssl.
Instead, pass the full include/library paths to your compiler e.g.:
-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib
I'm not sure what to do in this case and there's barely no practical solution other than gaining root access which I just want to avoid.
Can anyone give me some hint? I really appreciate that.
BTW., my macOS version is Sierra 10.12.3.
I don't have ms dotnet installed on my mac. Please don't address that related answer.

Upgrading pyOpenSSL worked for me.
sudo pip install --upgrade pyOpenSSL
If you are prompted with the following warning:
Cannot uninstall 'pyOpenSSL'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
You can do the ff:
sudo pip install --upgrade pyOpenSSL --ignore-installed pyOpenSSL

Related

SSL error on macOS Monterey when using python urllib via RDFLib

So I have been getting the following error when trying to request an HTTPS resource using the python RDFLib module:
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
What I have tried thus far:
Installed Python 3.10.4 from python.org
Executed the Install Certificates.command successfully tried the request again and I get the same error
Created a virtual environment using python3 -m venv venv which is based on Python 3.10.4, activated it, updated certifi just in case using pip3 install certifi --upgrade as it was suggested in other posts, and tried the request again and I still got the error.
Out of desperation, I did a clean install of Monterey and still get the same error
So it appears that the issue is that the certificates are not actually getting installed even though the command executes successfully:
/Applications/Python\ 3.10/Install\ Certificates.command ; exit;
meh#iMac ~ % /Applications/Python\ 3.10/Install\ Certificates.command ; exit;
-- pip install --upgrade certifi
Requirement already satisfied: certifi in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages (2021.10.8)
-- removing any existing file or link
-- creating symlink to certifi certificate bundle
-- setting permissions
-- update complete
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
I have actually been trying to solve this for many months, coming back and trying again and again but I can't seem to find a solution.
This seems to be a common issue that is usually resolved by executing the Install Certificates.command but that simply doesn't work for me. It's worth noting that I have the same issue on a brand new MacBook Pro with Monterey.
I would appreciate any suggestions on what I can try next.
It turns out that the target resource has a misconfigured certificate. Since there is no way to disable certificate verification in RDFLib - as far as I can tell - I have worked around the solution by requesting the resource via the python requests module with verify=False instead. Then I load the resource into RDFLib as text.

MacOS 10.13 High Sierra "pip3 install" command doesn't work (SSLError)

for my job purposes, I need a MacOS, but there is no possibility to use a real mac. So I decided to set up virtual machine, using Virtualbox. After the machine was successfully setup, I've opened the Terminal app and run the following commands:
1)/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" (Homebrew install)
2)brew install python (Python 3 install)
3)pip3 install numpy
As an output of the last command, I got the next error message:
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
ERROR: Could not find a version that satisfies the requirement numpy (from versions: none)
ERROR: No matching distribution found for numpy
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
Of course, I've tried to find a solution online. Reinstalling python or openssl via Homebrew didn't help. Linking openssl or installing other CA also didn't solve the problem. How can I actually fix this?
P.C. I also know that Apple is a sh*t company and tries to have minimum hardware and software of other companies. That's why they use LibreSSL instead of OpenSSL. But the last one is one that Python requests in order to have the ability to install packages via pip. After python was installed, the following message was in terminal:
This formula is keg-only, which means it was not symlinked into /usr/local.
If you need to have openssl#1.1 first in your PATH run:
echo 'export PATH="/usr/local/opt/openssl#1.1/bin:$PATH"' >> ~/.bash_profile
Running proposed command also doesn't solve the problem. Any thoughts?

Can't install tornado

I am setting up tensorflow and i got this output:
ERROR: matplotlib 1.3.1 requires nose, which is not installed.
so i am doing
sudo easy_install tornado
and i got as output:
earching for tornado
Reading https://pypi.python.org/simple/tornado/
Download error on https://pypi.python.org/simple/tornado/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) -- Some packages may not be found!
Couldn't find index page for 'tornado' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) -- Some packages may not be found!
No local packages or download links found for tornado
error: Could not find suitable distribution for Requirement.parse('tornado')
what should i do?
thanks
francesco
ERROR: matplotlib 1.3.1 requires nose, which is not installed.
It seems that you are running the command behind a proxy or a firewall. If this is the case, then you need to set the corresponding environment variables:
set http_proxy="username:password#host:port"
set https_proxy="username:password#host:port"
Otherwise, you can install tornado using pip by running (if you are not behind a proxy, then ignore --proxy):
pip --proxy http://user:password#proxy-host:port install tornado
EDIT: I would suggest to run
sudo -H pip install --ignore-installed matplotlib
in order to install the required packages and ignore the current version.

Python PIP and InsecurePlatformWarning

I'm trying to install requests library for Python (2.6.6 on CentOS6) as
sudo pip install requests[security]
I'm getting these error messages:
/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Collecting requests[security]
/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
I ensured all system libraries are in place
sudo yum install openssl-devel python-devel libffi-devel
and all is good there
so I tried
sudo pip install pyopenssl ndg-httpsclient pyasn1
and
sudo pip install --trusted-host pypi.python.org requests
and these fail as well.
Due to some old code I need to stick to Python 2.6
Any idea what is going wrong here and how to address it?
PS
I have decided to downgrade pip and it came up that errors came up with pip version 6.1+. Thoughts on that?
Indeed, as of 2019, quite a few people are still facing this issue, as they cannot upgrade their system Python (and system-supplied openssl) due to various limitations.
If you'd like to fix the InsecurePlatformWarning .. (Caused by SSLError .. SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version'), the following solution can help someone using an old Linux distro:
Unable to install Python packages using pip in Ubuntu Linux: InsecurePlatformWarning, SSLError, tlsv1 alert protocol version
Python 2.6.6 on CentOS6
In your specific case, the last cryptography version to officially support Python 2.6 was cryptography-2.1.1, although a newer version might work.
The first manylinux1 cryptography package to include its own OpenSSL (1.1.0f) shared lib binding for Linux, regardless of what's on your platform, has appeared in cryptography-2.0 version. The last cryptography that still officially supported Centos 6.4 was version 1.8.2, while newer versions officially supported CentOS 7.x only: https://pypi.org/project/cryptography/#files
Note, cryptography versions<2.0 have no manylinux1 and didn't ship the openssl/ssllib with them, relying on the system-supplied one only (back then, they did so only for other OSes that had/have even bigger SSL problems). While the cryptography-2.0 (manylinux1) is still worth trying, you will probably have to compile the newer openssl version (1.0.1 - 1.1.0) locally on Centos 6.x, before compiling cryptography 1.8.2 to link against it:
https://cryptography.io/en/latest/installation/#using-your-own-openssl-on-linux
https://cryptography.io/en/latest/installation/#rhel-centos
There is also an option to build your own statically-linked wheels that will work on your own systems:
https://cryptography.io/en/latest/installation/#static-wheels
That said,
if all you needed was just to install the requests lib (without solving the notorious SSLError), specifically in Python 2.6.6 on CentOS6, and you are not going to need TLS 1.2 protocol support from requests, you could simply use your Centos repo for that:
$ yum search requests
$ sudo yum install python26-requests
Or, you could get and install requests in a similar way as in the above-mentioned solution, from wheels or from tar.gz downloaded from:
https://pypi.org/project/requests/
Note, the latest requests on PyPI depends on the following Python packages:
urllib3, idna, chardet, certifi
Luckily, these four have no other dependencies and can be obtained from PyPI directly, in the same way - the "Download Files" section on PyPI:
https://pypi.org/project/urllib3/
https://pypi.org/project/idna/
https://pypi.org/project/chardet/
https://pypi.org/project/certifi/
You could download and try to install their latest versions, or use 'Release History' section to locate older versions, depending on your use-case and system-specific requirements.
Note, many web servers (including PyPI) nowadays require TLS 1.2 support, both in the client and server backend, to establish an HTTPS connection, so if you will need this type of connections, requests will not work without fixing the SSLError the way discussed above (or by compiling a newer Python 2.7.9+ against OpenSSL version 1.0.2+ that would have to be compiled as well in that case, before compiling new Python, to link ssl module with it).
Try this:
sudo pip install requests
[secure] is not necessary

Python requests throwing SSL errors

This is a followup to SSLError using requests for python:
I have just installed requests on a Mac OSX 10.8.5. My first attempt at doing requests.get failed on missing certificate:
SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
The thread above says to look for /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/re‌​quests/cacert.pem but actually I don't even have a .../site-packages/requests directory. It's not clear to me if this should have been added by the installation (I used pip)
Further threads and the requests docs say to install certifi, so I did. But now I get a different error:
python -c 'import requests; requests.get("https://api.github.com/events")' /usr/lib/anaconda/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Traceback (most recent call last):
...
File "/usr/lib/anaconda/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm
Thanks!
Notice that you're using HTTPS. As mentioned in the Requests manual
To check a host’s SSL certificate, you can use the verify argument [...] By default, verify is set to True
Here are few ways to fix that:
Update OpenSSL (probably will solve your problem)
Taken from here:
If you encounter one of the following errors:
error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm
error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm
The software you are using might be compiled with a version too old of OpenSSL that does not take certificates signed with sha256WithRSAEncryption into account.
It requires at least OpenSSL 0.9.8o for a total management of SHA256. OpenSSl 0.9.7m only assures a partial management, for server
mode only.
Check your openssl version by
openssl version
OpenSSL 1.0.1k-fips 8 Jan 2015
If you have a smaller version than OpenSSL0.9.8o, you have to update its version (OS X):
brew update
brew install openssl
brew link --force openssl
If that doesn't work, try this way:
brew uninstall openssl
rm -rf /usr/local/openssl
brew install openssl
there's an issue with openssl installed before OS X 10.10.3 and reinstalling it fixes it
these command lines will uninstall openssl, remove its folder from your hard-disk and install it again (the updated version)
Install certifi
Taken from here
By default Requests bundles a set of root CAs that it trusts, sourced
from the Mozilla trust store. However, these are only updated once for
each Requests version. This means that if you pin a Requests version
your certificates can become extremely out of date.
From Requests version 2.4.0 onwards, Requests will attempt to use
certificates from certifi if it is present on the system. This allows
for users to update their trusted certificates without having to
change the code that runs on their system.
For the sake of security we recommend upgrading certifi frequently!
In other word, try to install certifi, if you have Request 2.4.0 or newer:
pip install certifi
Hopefully, this will fix the problem.
Use different version of OpenSSL and Requests
Looking into it using Google, I have found that there is a problem with OpenSSL in Python 2:
https://github.com/docker/docker-py/issues/465#issuecomment-76520363
https://github.com/Homebrew/homebrew/issues/38226
https://github.com/docker/compose/issues/1484
However, I am using Python 2.7.6, Requests 2.2.1 and OpenSSL 1.0.1f 6 Jan 2014 and everything runs correctly.
Pass the certificate
In other cases, you may need to tell requests.get the path to the certificate file, if the host's certificate was signed by you.
requests.get("https://api.github.com/events", verify=True, cert=['/path/to/my/ca.crt'])
Set the verify argument to False (NOT RECOMMENDED!)
In case you want to avoid the certificate verification, you have to pass verify=False to the request.get method.
python -c 'import requests; requests.get("https://api.github.com/events", verify=False)'
or from script.py file:
import requests
res = requests.get("https://api.github.com/events", verify=False)
print res
terminal:
$ python script.py
<Response [200]>
Important: Very bad idea; You can be MITM attacked, which is a critical security vulnerability.

Categories