Trying to connect to Azure CosmosDB mongo server results into an SSL handshake error.
I am using Python3 and Pymongo to connect to my Azure CosmosDB. The connection works fine if I run the code with Python27 but causes the below error when using Python3:
import pymongo
from pymongo import MongoClient
import json
import sys
def check_server_status(client, data):
'''check the server status of the connected endpoint'''
db = client.result_DB
server_status = db.command('serverStatus')
print('Database server status:')
print(json.dumps(server_status, sort_keys=False, indent=2, separators=(',', ': ')))
coll = db.file_result
print (coll)
coll.insert_one(data)
def main():
uri = "mongodb://KEY123#backend.documents.azure.com:10255/?ssl=true&replicaSet=globaldb"
client = pymongo.MongoClient(uri)
emp_rec1 = {
"name":"Mr.Geek",
"eid":24,
"location":"delhi"
}
check_server_status(client, emp_rec1)
if __name__ == "__main__":
main()
Running this on Python3 results into below error:
pymongo.errors.ServerSelectionTimeoutError: SSL handshake failed:
backendstore.documents.azure.com:10255: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
Here is my successful output when I run the same code with Python27:
Database server status: { "_t": "OKMongoResponse", "ok": 1 }
Collection(Database(MongoClient(host=['backend.documents.azure.com:10255'],
document_class=dict, tz_aware=False, connect=True, ssl=True,
replicaset='globaldb'), u'result_DB'), u'file_result')
On Windows you can do like this
pip install certifi
Then use it in code:
import certifi
ca = certifi.where()
client = pymongo.MongoClient(
"mongodb+srv://username:password#cluster0.xxxxx.mongodb.net/xyzdb?retryWrites=true&w=majority", tlsCAFile=ca)
Solved the problem with this change:
client = pymongo.MongoClient(uri, ssl_cert_reqs=ssl.CERT_NONE)
The section Troubleshooting TLS Errors of the PyMongo offical document `TLS/SSL and PyMongo introduces the issue as below.
TLS errors often fall into two categories, certificate verification failure or protocol version mismatch. An error message similar to the following means that OpenSSL was not able to verify the server’s certificate:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
This often occurs because OpenSSL does not have access to the system’s root certificates or the certificates are out of date. Linux users should ensure that they have the latest root certificate updates installed from their Linux vendor. macOS users using Python 3.6.0 or newer downloaded from python.org may have to run a script included with python to install root certificates:
open "/Applications/Python <YOUR PYTHON VERSION>/Install Certificates.command"
Users of older PyPy and PyPy3 portable versions may have to set an environment variable to tell OpenSSL where to find root certificates. This is easily done using the certifi module from pypi:
$ pypy -m pip install certifi
$ export SSL_CERT_FILE=$(pypy -c "import certifi; print(certifi.where())")
You can try to follow the description above to fix your issue, which seems to be for Linux and Mac Users. On Windows, I can not reproduce your issue in Python 3.7 and 3.6. If you have any concern, please feel free to let me know.
Faced the same issue when trying to connect mongodb from Digital Ocean,
Solved by using this function with params in MongoClient:
def get_client(host,port,username,password,db):
return MongoClient('mongodb://{}:{}/'.format(host,port),
username=username,
password=password,
authSource=db,
ssl=True,ssl_cert_reqs=ssl.CERT_NONE)
client = get_client("host-ip","port","username","password","db-name")
On Mac Mojave 10.14.6 , I used (PyMongo 3.10 and python 3.7), to solve:
flask pymongo pymongo.errors.ServerSelectionTimeoutError [SSL: CERTIFICATE_VERIFY_FAILED]
Execute in terminal:
sudo /Applications/Python\ 3.7/Install\ Certificates.command
If you use other python version, only change versión number (In my case, i have Python 3.7)
cluster = MongoClient(
"url",
ssl=True,
ssl_cert_reqs=ssl.CERT_NONE,
)
By default pymongo relies on the operating system’s root certificates.
It could be that Atlas itself updated its certificates or it could be that something on your OS changed. “certificate verify failed” often occurs because OpenSSL does not have access to the system’s root certificates or the certificates are out of date. For how to troubleshoot see TLS/SSL and PyMongo — PyMongo 3.12.0 documentation 107.
pls Try :
client = pymongo.MongoClient(connection, tlsCAFile=certifi.where())
and dont forget to install certifi
On mac Monterey, I used pymongo 3.12.1 and virtual environment
To solve, use
ssl_cert_reqs=CERT_NONE
with mongodb url
Related
I am able to connect to a certain URL with cURL, after I installed the corresponding SSL certificates:
$ export MY_URL=https://www.infosubvenciones.es/bdnstrans/GE/es/convocatoria/616783
$ curl -vvvv $MY_URL # Fails
$ sudo openssl x509 -inform pem -outform pem -in /tmp/custom-cert.pem -out /usr/local/share/ca-certificates/custom-cert.crt
$ sudo update-ca-certificates
$ curl -vvvv $MY_URL # OK
However, requests (or httpx, or any other library I use) refuses to do so:
In [1]: import os
...: import requests
...: requests.get(os.environ["MY_URL"])
---------------------------------------------------------------------------
SSLCertVerificationError Traceback (most recent call last)
...
SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)
My understanding is that requests uses certifi and as such these custom certificates are not available here:
In [1]: import certifi
In [2]: certifi.where()
Out[2]: '/tmp/test_ca/.venv/lib/python3.10/site-packages/certifi/cacert.pem'
I have already tried a number of things, like trying to use the system CA bundle:
export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt same error
requests.get(..., verify="/etc/ssl/certs/ca-certificates.crt") same error
switched to httpx + a custom SSL context as explained in the docs, same error
attempted truststore as discussed in this httpx issue, same error
How can I make Python (requests, httpx, raw ssl, anything) use the same certificates that cURL is successfully using?
The only thing that worked so far, inspired by this hackish SO answer, is to do verify=False. But I don't want to do that.
In [9]: requests.get(
...: my_url,
...: verify=False,
...: )
/tmp/test_ca/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py:1043: InsecureRequestWarning: Unverified HTTPS request is being made to host 'xxx'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
i tried your stuff on my system (Manjaro Linux, python 3.10) i can make a connection. I downloaded the complete certificate chain from the website (with my browser). After that i can use it with:
r = requests.get(url=URL, verify=<path to pem file>)
and with
export REQUESTS_CA_BUNDLE=<path to pem>
r = requests.get(url=URL)
I tried the export within pyCharm.
So the python stuff is working and you may have a problem in your certificates. Without this stuff i get the ssl error (of course), because python does not use the system certs as you mentioned correct. In my pem-file i have 3 certificates. Maybe you have only 1 and the others are in the global store, so that curl does not need the complete chain, instead of python. You should try to download the complete chain with your browser and try again.
Hey I got a couple of errors and I need help to fix them because I can't find a solution for that.
Why am I using an old af version of ubuntu?
Because I need to build android 4 and the easiest way to get the right versions of the needed packages is to use an older version of ubuntu which is confirmed to work
So please don't tell me to just upgrade to the latest ubuntu
Some commands that don't work because of a failure with ssl
when trying to access any website with firefox it returns ssl_error_no_cypher_overlap
BUT google searches are working fine
$ repo init --depth=1 -u http://github.com/CyanogenMod/android.git -b ics
Downloading Repo source from http://gerrit.googlesource.com/git-repo
fatal: Cannot get http://gerrit.googlesource.com/git-repo/clone.bundle
fatal: error unknown url type: https
fatal: cloning the git-repo repository failed, will remove '.repo/repo'
a small information how I got repo to work but didn't break apt (apt uses python2 and repo needs python3)
I aliased python as python3 so when I run python as user it refers to python3 but when I run python as root it refers to the python symlink (/usr/bin/python) which is python2
$ sudo apt-add-repository ppa:relan/exfat
Traceback (most recent call last):
File "/usr/bin/apt-add-repository", line 88, in <module>
ppa_info = get_ppa_info_from_lp(user, ppa_name)
File "/usr/lib/python2.7/dist-packages/softwareproperties/ppa.py", line 83, in get_ppa_info_from_lp
curl.perform()
pycurl.error: (60, 'server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none')
$ wget https://www.python.org/ftp/python/3.10.1/Python-3.10.1.tgz
--2021-12-25 21:43:11-- https://www.python.org/ftp/python/3.10.1/Python-3.10.1.tgz
Resolving www.python.org... 2a04:4e42:3::223, 151.101.12.223
Connecting to www.python.org|2a04:4e42:3::223|:443... connected.
OpenSSL: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
Unable to establish SSL connection.
$ curl -v https://www.python.org/ftp/python/3.10.1/Python-3.10.1.tgz
* About to connect() to www.python.org port 443 (#0)
* Trying 2a04:4e42:3::223... connected
* Connected to www.python.org (2a04:4e42:3::223) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS alert, Server hello (2):
* error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
* Closing connection #0
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
Solutions I found online and tried but didn't work
checked if time is correct
update-ca-certificates -f
Tell me if you need further information/logs or anything else
Every help is appreciated!
The problem is not (yet) the certificates, it fails before validating these. Instead the versions of the SSL libraries used a simply too old. This means your software stack is way to old for today's requirements. There is no easy way to fix this.
In detail:
The openssl version in 11.10 is 0.9.8 which has no support for modern protocols like TLS 1.2 or even TLS 1.3. Similar the version of Firefox at the time of Ubuntu 11.10 did not support TLS 1.2 either (even though NSS and not openssl was used as SSL library).
I'm setting up a new server using Pytoh, tornado.
I'm going to set HTTPS.
But when server loads server certificate files, load_cert_chain function generates an error.
I purchased those certificate files from GoDaddy and it was okay on Apache Server before.
So certificate file and key file match and ca file is also okay.
python version: 3.7.4
tornado version : 4.5.2
openssl version: OpenSSL 1.1.1c 28 May 2019
ssl_ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ssl_ctx.load_cert_chain("../server.crt", "../server.key")
ssl_ctx.load_verify_locations("../ca.crt")
ssl_ctx.verify_mode = ssl.CERT_REQUIRED
https_server = tornado.httpserver.HTTPServer(app, ssl_options=ssl_ctx)
This is error message.
ssl_ctx.load_cert_chain("../server.crt", "../server.key")
ssl.SSLError: [SSL] PEM lib (_ssl.c:3854)
I solved this problem. I use NginX server to host python server with digital signed certificates.
I am trying to use Sqlalchemy package to be able to connect to Redshift datawarehouse.
from sqlalchemy import create_engine
import psycopg2
red_engine=create_engine('redshift+psycopg2://admin:XXXXXXXXX#IP:PORT/DBNAME')
test = pd.read_sql_query(query,red_engine)
Error message I get is :
OperationalError: (psycopg2.OperationalError) SSL error: certificate verify failed
I am unable to see why this SSL veirifcation is falling all, used to go smoothly earlier.
I encountered the same error. I updated the sqlalchemy-redshift package to the latest, 0.7.0, which updates the certificates used to connect to Redshift.
To update, use pip install -U sqlalchemy-redshift.
I am using Python Requests library to connect to a REST server by using .pem certificates to authenticate who I am to establish a connection so I can start collecting data, parsing it, etc. When I run my program via Eclipse or the terminal, I get this error:
[('system library', 'fopen', 'Permission denied'), ('BIO routines', 'FILE_CTRL', 'system lib'), ('SSL routines', 'SSL_CTX_use_certificate_file', 'system lib')]
However, if I run as 'sudo' - the requests library works as intended and I am able to retrieve the data. Unfortunately, running as 'sudo' has side effects that where the default Python interpreter is the root interpreter, which is Python2. However, there are a lot of library dependencies that are needed from Anaconda.
For context, here is the function I am using to establish a connection:
PEM_FILE = os.path.expanduser("path/to/pem/file.pem") #Path is to a folder in root level
def set_up_connection(self):
#URL's to request a connection with
rest_auth = 'https://www.restwebsite.com/get/data'
ip_address = self.get_ip_address()
body = json.dumps({'userid':'user', 'password':'pass', 'ip_address':ip_address})
try:
resp = self.session.post(rest_auth, data=body, cert=PEM_FILE, verify=False)
values = resp.json()
token = values['token']
except Exception as e:
print(e)
token = None
return token, ip_address
TLDR; Using 'python rest_connector.py' renders an error. Running that command as sudo works.
Context for the certificates: The .pem cert permissions is set to 600 (rw-------).
To try and solve my problem running as sudo, I have started a terminal and run 'sudo -Es' which sets the terminal to run as root and uses Anaconda as my default interpreter, BUT I end up with a handshake error:
[('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca'), ('SSL routines', 'ssl3_read_bytes', 'ssl handshake failure')]
If someone can help me solve it this way, it would be a nice temporary fix, but I still need to be able to run this without sudo.
Thanks, in advance.
The username needs to be able to read the file. You can verify by running ls path_to_file.pem.
If you have changed ownership of the file, you might still be missing executable permissions on the directories containing the file.
You can potentially fix that with chmod -R +x ~/path/to_directory_containing_perm
Ok, so I managed to solve this and I'll post what I did in case anyone else stumbles upon this with a similar problem.
Permissions being set to 600 for the certs and pem file, ownership being set to root, and performing the openssl hashing function, the only problem was where the certs were placed in the sub directories.
While I placed the certs into 'etc/pki/tls/certs', they actually belonged in 'etc/ssl/certs'. The same goes for the .pem file except that .pem would be placed in a restricted folder, 'private'. After moving the files to the correct folder. Then I was able to set the verify param for the request to the cert_path and everything worked like I needed it to.
resp = self.session.post(rest_auth, data=body, cert=PEM_FILE, verify=cert_path)
'etc/pki/tls/certs' is the directory for Fedora distribution of Linux. 'etc/ssl/certs' is the directory for the Ubuntu distribution of Linux.
I hope this helps someone.