I am using the python stack on Heroku and am trying to use the requests library to access an https API that uses TLS 1.2
I have got thing working on my local environment by upgrading openssl to 1.0.1 and I need to do the same thing in my Heroku environment.
Here is the error in Heroku and I've confirmed that the openssl version is OpenSSL 0.9.8k 25 Mar 2009
requests.exceptions.SSLError: [Errno bad handshake] [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'sslv3 alert handshake failure')]
Is there a python buildpack that can support this? It looks like there is a ruby buildpack, but I haven't seen it verified to work OpenSSL 1.0.1 on Heroku
I tried to mimic that behavior, but don't really know what I'm doing with a fork of the standard python buildpack.
Thanks!
Cedar-14, has OpenSSL 1.0.1f. Heroku-16, the most recent Heroku stack, has OpenSSL 1.0.2g. Stack package details
The upgrade guide is here:
https://devcenter.heroku.com/articles/cedar-14-migration
Related
So I have been getting the following error when trying to request an HTTPS resource using the python RDFLib module:
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
What I have tried thus far:
Installed Python 3.10.4 from python.org
Executed the Install Certificates.command successfully tried the request again and I get the same error
Created a virtual environment using python3 -m venv venv which is based on Python 3.10.4, activated it, updated certifi just in case using pip3 install certifi --upgrade as it was suggested in other posts, and tried the request again and I still got the error.
Out of desperation, I did a clean install of Monterey and still get the same error
So it appears that the issue is that the certificates are not actually getting installed even though the command executes successfully:
/Applications/Python\ 3.10/Install\ Certificates.command ; exit;
meh#iMac ~ % /Applications/Python\ 3.10/Install\ Certificates.command ; exit;
-- pip install --upgrade certifi
Requirement already satisfied: certifi in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages (2021.10.8)
-- removing any existing file or link
-- creating symlink to certifi certificate bundle
-- setting permissions
-- update complete
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
I have actually been trying to solve this for many months, coming back and trying again and again but I can't seem to find a solution.
This seems to be a common issue that is usually resolved by executing the Install Certificates.command but that simply doesn't work for me. It's worth noting that I have the same issue on a brand new MacBook Pro with Monterey.
I would appreciate any suggestions on what I can try next.
It turns out that the target resource has a misconfigured certificate. Since there is no way to disable certificate verification in RDFLib - as far as I can tell - I have worked around the solution by requesting the resource via the python requests module with verify=False instead. Then I load the resource into RDFLib as text.
I'm getting below error while installing Jupiter, does anyone came across similar issues.
pip3.5 install jupyter
Collecting jupyter
/grid/common/pkgs/python/v3.5.2/lib/python3.5/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
Could not fetch URL https://pypi.python.org/simple/jupyter/: There was a problem confirming the ssl certificate: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645) - skipping
Could not find a version that satisfies the requirement jupyter (from versions: )
No matching distribution found for jupyter
Here are the env details
OS: RHEL6
Python - 3.5.2
Pip: 9.0.1
Curl: 7.47.1
Openssl 1.0.1e
You can:
Go to Python root folder and run the "Install Certificates.command" file.
You may follow https://www.youtube.com/watch?v=BVS3U1OQzO4
Our devops recently turned on SSL on our in-house ElasticSearch servers, while our Ubuntu dev boxes are connecting to it fine, but it's causing SSLError on Mac dev boxes (running Django).
SSLError at /search
ConnectionError(EOF occurred in violation of protocol (_ssl.c:590)) caused by: SSLError(EOF occurred in violation of protocol (_ssl.c:590))
What I know so far:
Not limited to El Capitan, also breaks on earlier version
We can connect to other ES service over https, our devops told me our ES service has higher https requirements
Openssl v0.9.8 has handshake problem on the ES service
Openssl v1.0.1 works fine on the ES service
There are many posts online around this problem but none helped.
I have tried:
brew link --force openssl (but EL Capitan is stopping it), none of the solutions worked from Homebrew refusing to link OpenSSL
adding /usr/local/opt/openssl/lib to DYLD_LIBRARY_PATH
upgrading python from 2.7.10 to 2.7.12
rebuilding virtualenv
None of the above worked...
Questions:
Is it due to OpenSSL version?
Is it due to Python version due to _ssl.c?
How do I fix this?
Finally nailed it, basically you will have to install Python that has updated openssl
check in Python cmd
import ssl
print ssl.OPENSSL_VERSION
It should print 0.9.x
Now to update:
brew install openssl, don't worry about the linking if it rejects, make sure it's >=1.x.x
'brew install python --with-brewed-openssl`
recreate virtualenv by specifying the brewed version of Python, eg. mkvirtualenv --python=/usr/.../Cellar/python/bin/python <project>
This is a followup to SSLError using requests for python:
I have just installed requests on a Mac OSX 10.8.5. My first attempt at doing requests.get failed on missing certificate:
SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
The thread above says to look for /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/cacert.pem but actually I don't even have a .../site-packages/requests directory. It's not clear to me if this should have been added by the installation (I used pip)
Further threads and the requests docs say to install certifi, so I did. But now I get a different error:
python -c 'import requests; requests.get("https://api.github.com/events")' /usr/lib/anaconda/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Traceback (most recent call last):
...
File "/usr/lib/anaconda/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm
Thanks!
Notice that you're using HTTPS. As mentioned in the Requests manual
To check a host’s SSL certificate, you can use the verify argument [...] By default, verify is set to True
Here are few ways to fix that:
Update OpenSSL (probably will solve your problem)
Taken from here:
If you encounter one of the following errors:
error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm
error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm
The software you are using might be compiled with a version too old of OpenSSL that does not take certificates signed with sha256WithRSAEncryption into account.
It requires at least OpenSSL 0.9.8o for a total management of SHA256. OpenSSl 0.9.7m only assures a partial management, for server
mode only.
Check your openssl version by
openssl version
OpenSSL 1.0.1k-fips 8 Jan 2015
If you have a smaller version than OpenSSL0.9.8o, you have to update its version (OS X):
brew update
brew install openssl
brew link --force openssl
If that doesn't work, try this way:
brew uninstall openssl
rm -rf /usr/local/openssl
brew install openssl
there's an issue with openssl installed before OS X 10.10.3 and reinstalling it fixes it
these command lines will uninstall openssl, remove its folder from your hard-disk and install it again (the updated version)
Install certifi
Taken from here
By default Requests bundles a set of root CAs that it trusts, sourced
from the Mozilla trust store. However, these are only updated once for
each Requests version. This means that if you pin a Requests version
your certificates can become extremely out of date.
From Requests version 2.4.0 onwards, Requests will attempt to use
certificates from certifi if it is present on the system. This allows
for users to update their trusted certificates without having to
change the code that runs on their system.
For the sake of security we recommend upgrading certifi frequently!
In other word, try to install certifi, if you have Request 2.4.0 or newer:
pip install certifi
Hopefully, this will fix the problem.
Use different version of OpenSSL and Requests
Looking into it using Google, I have found that there is a problem with OpenSSL in Python 2:
https://github.com/docker/docker-py/issues/465#issuecomment-76520363
https://github.com/Homebrew/homebrew/issues/38226
https://github.com/docker/compose/issues/1484
However, I am using Python 2.7.6, Requests 2.2.1 and OpenSSL 1.0.1f 6 Jan 2014 and everything runs correctly.
Pass the certificate
In other cases, you may need to tell requests.get the path to the certificate file, if the host's certificate was signed by you.
requests.get("https://api.github.com/events", verify=True, cert=['/path/to/my/ca.crt'])
Set the verify argument to False (NOT RECOMMENDED!)
In case you want to avoid the certificate verification, you have to pass verify=False to the request.get method.
python -c 'import requests; requests.get("https://api.github.com/events", verify=False)'
or from script.py file:
import requests
res = requests.get("https://api.github.com/events", verify=False)
print res
terminal:
$ python script.py
<Response [200]>
Important: Very bad idea; You can be MITM attacked, which is a critical security vulnerability.
I am trying to upload a new python project from my mac to PyPI via:
python setup.py sdist upload -r pypi
When I try to upload a package, I get the following error on OSX:
Submitting dist/PyTreasuryDirect-0.1.0.tar.gz to https://pypi.python.org/pypi
error: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
This question is slightly related to these two questions, ssl with pip and when importing the ssl library. However I need setup.py to work in my case, how can I get past this ssl error?
I also get the same error on Python 2.7.9 and Python 3.5 using OSX 10.10.2 (14C109)
The solution for OS X 10.10 is to use its default Python version 2.7.6.
This problem is related to the fact that starting from Python version 2.7.9 certificate validation for stdlib http clients is enabled by default (PEP 476). This issue is described in Python bug tracker.