(psycopg2.OperationalError) SSL error: certificate verify failed - python

I am trying to use Sqlalchemy package to be able to connect to Redshift datawarehouse.
from sqlalchemy import create_engine
import psycopg2
red_engine=create_engine('redshift+psycopg2://admin:XXXXXXXXX#IP:PORT/DBNAME')
test = pd.read_sql_query(query,red_engine)
Error message I get is :
OperationalError: (psycopg2.OperationalError) SSL error: certificate verify failed
I am unable to see why this SSL veirifcation is falling all, used to go smoothly earlier.

I encountered the same error. I updated the sqlalchemy-redshift package to the latest, 0.7.0, which updates the certificates used to connect to Redshift.
To update, use pip install -U sqlalchemy-redshift.

Related

SSL certificate verify failed while configuring mongodb with python

I am trying to add a json file to mongo db database as a collection. I have installed pymongo module from pycharm terminal. I can connect using mongoDB compass as well. But is is giving this error while trying to connect through code.
"pymongo.errors.ServerSelectionTimeoutError: cluster0-shard-00-01.m49ca.mongodb.net:27017: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate...."
My code is here:
from pymongo import MongoClient
client = MongoClient('first part of conn string' + urllib.parse.quote('password') + 'second part of conn string')"
Can anybody help me with this issue?

PyMongo - can't connect to localhost?

I have tried running the following:
import pymongo
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
db = client.test_database
collection = db.test_collection
collection.find_one()
but I get
ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused
Any suggestions on how to fix this? I am running this behind a corporate proxy, but have already set environment variables for that in .bashrc.
EDIT
If I run mongo from the terminal, I get
$ mongo
MongoDB shell version v4.0.13
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
2019-10-16T10:30:23.269+0100 E QUERY [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:344:17
#(connect):2:6
exception: connect failed
Its look like you didn't run any mongo instance on your machine
run the command mongod -f <config_file> and then test it again (this may needs a sudo permissions)

SSL Handshake issue with Pymongo on Python3

Trying to connect to Azure CosmosDB mongo server results into an SSL handshake error.
I am using Python3 and Pymongo to connect to my Azure CosmosDB. The connection works fine if I run the code with Python27 but causes the below error when using Python3:
import pymongo
from pymongo import MongoClient
import json
import sys
def check_server_status(client, data):
'''check the server status of the connected endpoint'''
db = client.result_DB
server_status = db.command('serverStatus')
print('Database server status:')
print(json.dumps(server_status, sort_keys=False, indent=2, separators=(',', ': ')))
coll = db.file_result
print (coll)
coll.insert_one(data)
def main():
uri = "mongodb://KEY123#backend.documents.azure.com:10255/?ssl=true&replicaSet=globaldb"
client = pymongo.MongoClient(uri)
emp_rec1 = {
"name":"Mr.Geek",
"eid":24,
"location":"delhi"
}
check_server_status(client, emp_rec1)
if __name__ == "__main__":
main()
Running this on Python3 results into below error:
pymongo.errors.ServerSelectionTimeoutError: SSL handshake failed:
backendstore.documents.azure.com:10255: [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)
Here is my successful output when I run the same code with Python27:
Database server status: { "_t": "OKMongoResponse", "ok": 1 }
Collection(Database(MongoClient(host=['backend.documents.azure.com:10255'],
document_class=dict, tz_aware=False, connect=True, ssl=True,
replicaset='globaldb'), u'result_DB'), u'file_result')
On Windows you can do like this
pip install certifi
Then use it in code:
import certifi
ca = certifi.where()
client = pymongo.MongoClient(
"mongodb+srv://username:password#cluster0.xxxxx.mongodb.net/xyzdb?retryWrites=true&w=majority", tlsCAFile=ca)
Solved the problem with this change:
client = pymongo.MongoClient(uri, ssl_cert_reqs=ssl.CERT_NONE)
The section Troubleshooting TLS Errors of the PyMongo offical document `TLS/SSL and PyMongo introduces the issue as below.
TLS errors often fall into two categories, certificate verification failure or protocol version mismatch. An error message similar to the following means that OpenSSL was not able to verify the server’s certificate:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
This often occurs because OpenSSL does not have access to the system’s root certificates or the certificates are out of date. Linux users should ensure that they have the latest root certificate updates installed from their Linux vendor. macOS users using Python 3.6.0 or newer downloaded from python.org may have to run a script included with python to install root certificates:
open "/Applications/Python <YOUR PYTHON VERSION>/Install Certificates.command"
Users of older PyPy and PyPy3 portable versions may have to set an environment variable to tell OpenSSL where to find root certificates. This is easily done using the certifi module from pypi:
$ pypy -m pip install certifi
$ export SSL_CERT_FILE=$(pypy -c "import certifi; print(certifi.where())")
You can try to follow the description above to fix your issue, which seems to be for Linux and Mac Users. On Windows, I can not reproduce your issue in Python 3.7 and 3.6. If you have any concern, please feel free to let me know.
Faced the same issue when trying to connect mongodb from Digital Ocean,
Solved by using this function with params in MongoClient:
def get_client(host,port,username,password,db):
return MongoClient('mongodb://{}:{}/'.format(host,port),
username=username,
password=password,
authSource=db,
ssl=True,ssl_cert_reqs=ssl.CERT_NONE)
client = get_client("host-ip","port","username","password","db-name")
On Mac Mojave 10.14.6 , I used (PyMongo 3.10 and python 3.7), to solve:
flask pymongo pymongo.errors.ServerSelectionTimeoutError [SSL: CERTIFICATE_VERIFY_FAILED]
Execute in terminal:
sudo /Applications/Python\ 3.7/Install\ Certificates.command
If you use other python version, only change versión number (In my case, i have Python 3.7)
cluster = MongoClient(
"url",
ssl=True,
ssl_cert_reqs=ssl.CERT_NONE,
)
By default pymongo relies on the operating system’s root certificates.
It could be that Atlas itself updated its certificates or it could be that something on your OS changed. “certificate verify failed” often occurs because OpenSSL does not have access to the system’s root certificates or the certificates are out of date. For how to troubleshoot see TLS/SSL and PyMongo — PyMongo 3.12.0 documentation 107.
pls Try :
client = pymongo.MongoClient(connection, tlsCAFile=certifi.where())
and dont forget to install certifi
On mac Monterey, I used pymongo 3.12.1 and virtual environment
To solve, use
ssl_cert_reqs=CERT_NONE
with mongodb url

ServerSelectionTimeoutError Pymongo

I'm trying out pymongo for the first time and I keep getting a ServerSelectionTimeoutError. When using mongo commandline to login I run a command as follows
$ mongo-3.0 --ssl test.net:27080/db_qa --sslAllowInvalidCertificates -u content -p
MongoDB shell version: 3.0.12
Enter password:
and I'm able to connect fine but with pymongo I get the error
pymongo.errors.ServerSelectionTimeoutError: test.net:27080: [Errno 60] Operation timed out
My code is as follows
from pymongo import MongoClient
client = MongoClient('mongodb://content:<password>#test.net:27080/db_qa')
client.server_info()
Your connection string is missing the options that your shell command line provides, namely ssl and option to allow invalid certificate.
You could add ?ssl=true&ssl_cert_reqs=CERT_NONE after the database name in the string you are passing to MongoClient or see other options for certificate handling on MongoClient page (scroll to "SSL configuration" section)
So what worked for me was my refreshing my current IP which changed under the "setup connection security" tab

subunit2sql configuration for Azure database

I'm trying to setup a Azure powered database to be used by subunit2sql.
In the subunit2sql db setup steps, it is mentioned to create the schema as:
subunit2sql-db-manage --database-connection mysql://subunit:pass#127.0.0.1/subunit upgrade head
Replacing the mysql tab with mssql, and after creating a odbc SDN connection to the azure DB - I'm able to see the database.
However, subunit2sql connection always fails with the below trace.
My understanding is that I might not be using a proper config for alchemy/python, but the messages are confusing me, as if I use isql for eg with the SDN DB info I can connect to it fine.
Command I'm running is:
subunit2sql-db-manage --verbose --database-connection mssql://user:'pwd'#remote_host/DB upgrade head
I do have the odbc config under /etc/odbcinst.ini and /etc/odbc.ini but the above fails with:
oslo_db.exception.DBError: (pyodbc.Error) ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')
(sorry I cannot add subuni2sql tag here as I don't have points)
Sharing my testing. Hope it helps.
According to the requirements.txt of project subunit2sql on GitHub, the tool required Python package SQLAlchemy, so I refer to the SQLAlchemy document support for the Microsoft SQL Server database via the pymssql driver to set the database connection with the Azure SQL Database connection string, please see below.
subunit2sql-db-manage --database-connection mssql+pymssql://<username>#<hostname>:<password>#<hostname>.database.windows.net:1433/<database> upgrade head
Note: You can find the variables username, password, hostname & database in the SQL Database connection string on Azure portal.
For connecting SQL database successfully, you need to do the steps below before running the above command.
As reference, My environment is Ubuntu 14.04 LTS.
Refering to the document Configure development environment for Python development to install the required package for Python.
Configuring the /etc/odbcinst.ini file, for example as below.
[FreeTDS]
Driver=/usr/local/lib/libtdsodbc.so
Setup=/usr/local/lib/libtdsodbc.so
Server={hostname}.database.windows.net
UsageCount=1
Port=1433
Database={database}
User={username}#{hostname}
Password={password}
TDS_Version=7.2
client_charset=utf-8
Configuring the firewall settings to add the client ip, please refer to https://azure.microsoft.com/en-us/documentation/articles/sql-database-configure-firewall-settings/.
Althought I got the error information as below, successfully connected the SQL Database.
oslo_db.exception.DBError: (pymssql.ProgrammingError) (102, "Incorrect syntax near ''.DB-Lib error message 102, severity 15:\nGeneral SQL Server error: Check messages from the SQL Server\nDB-Lib error message 156, severity 15:\nGeneral SQL Server error: Check messages from the SQL Server\n") [SQL: 'INSERT INTO test_metadata_new (id,key`, value, test_id, new_test_id) SELECT tm.id, tm.key, tm.value, tm.test_id, tn.new_id FROM test_metadata tm INNER JOIN tests_new tn ON tn.id = tm.test_id']

Categories