How can I connect to my db2 database with sqlalchemy when the authentication is using kerberos?
When using pyodbc the connection string contains AuthenticationMethod=4, which lets kerberos handle the authentication and I don't need to provide username and password.
Is there a way to either pass a pyodbc.connect object directly into sqlalchemy or can I alternatively tell sqlalchemy to use kerberos?
My odbc connection string looks like this:
connstr = 'ApplicationUsingThreads=0;' \
...: 'FloatingPointParameters=0;' \
...: 'DoubleToStringPrecision=16;DB=NYRMPDI1;' \
...: 'AuthenticationMethod=4;' \
...: f'IpAddress={ip_address};' \
...: f'TcpPort={port};' \
...: f'DRIVER={driver_location}'
I can't find any way to pass this into sqlalchemy create_engine.
ibm_db_sa with an IBM Db2 driver supports kerberos connections with pyodbc, both DSN-LESS and DSN connection-strings, and it works with all three types of IBM Db2-driver (fat client, run-time-client, and ODBC and CLI driver). Different configurations are necessary for the fat-client+runtime-client, versus the ODBC and CLI client.
By default, unless you tell it otherwise, the installation of ibm_db_sa or ibm_db modules will install the IBM 'ODBC and CLI client'.
Your odbcinst.ini needs to define a driver-name (in my example I call it DB2CLI but you give it any name you prefer), and specify the library to load (example libdb2.so) from the correct path.
Here is an example of a DSN-LESS connection string, which you must urlencode before passing to create_engine():
CONNECTION_STRING=("DRIVER={DB2CLI};HOSTNAME=192.168.1.178;PORT=60000;KRBPLUGIN=IBMkrb5;AUTHENTICATION=KERBEROS;DATABASE=SAMPLE;")
quoted_connection_string=urllib.parse.quote_plus(CONNECTION_STRING)
engine = create_engine('ibm_db_sa+pyodbc:///?odbc_connect={}'.format(quoted_connection_string))
If you prefer a DSN connection, you must define all the details in the db2dsdriver.cfg and have a stanza for the dsn in the active odbc.ini that references the driver you configured in your odbcinst.ini, and you must specify only the DSN in the connection-string like this:
CONNECTION_STRING=("DSN=SAMPLE;")
engine = create_engine('ibm_db_sa+pyodbc:///?odbc_connect={}'.format(CONNECTION_STRING))
For DSN connections, it helps if you first get the kerberos connection working with isql defore you get it working with sqlalchemy because the troubleshooting seems easier.
I tested with these component versions:
ubuntu 16.04 LTS x64
python 3.6.8 in a virtualenv
ibm_db 3.0.1
ibm_db_sa 0.3.5
unixODBC 2.3.4
pyodbc 4.0.30
IBM Db2 data server driver 11.1.4.4a (optional)
IBM Db2 ODBC and CLI driver (default)
local and remote Db2-LUW servers whose Db2-instances are kerberized already.
Steps to try:
For DSN connections, configure your active db2dsdriver.cfg with dsn and database with parameter Authentication, parameter value Kerberos.
For the fat-client and runtime-client, configure your IBM Data Server Client CLNT_KRB_PLUGIN parameter to IBMkrb5 via db2 update dbm cfg using CLNT_KRB_PLUGIN IBMkrb5. (You don't need this step when using the ODBC and CLI driver).
Configure your active odbcinst.ini for Db2 to use the correct libdb2.so library as supplied by your Db2 client, and reference this driver-name either in your DSN-LESS python code, or in your odbc.ini for DSN-connections.
For DSN connections only, configure your active odbc.ini to use the Db2 driver specified in odbcinst.ini and mention Authentication = kerberos in your DSN stanza in odbc.ini.
For DSN connections, Omit any userid/password from the active odbc.ini file. For DSN-LESS connectiond you don't need any reference to the database in the odbc.ini or db2dsdriver.cfg.
For DSN connections only, Verify db2cli validate -dsn $YOURDSN -connect for a remote database completes successfully without a userid or password. This proves that the CLI layer is using kerberos.
(Optional) For Db2 fat client, or runtime client, verify you can connect to a catalogued remote database at the shell command line db2 connect to $YOUR_REMOTE_DATABASE (without needing to enter a userid/password). This proves that regular shell scripts can connect to the database with kerberos authentication.
If you are using either the Db2 fat client, or the Db2 runtime client then you need to dot in / source the correct db2profile before running either isql or your python script.
Related
Is there a way to use the oracle python driver package cx_Oracle and specify Kerberos authentication?
I've seen this done using oracle jdbc drivers, but not with cx_Oracle specifically. Below is my current connection code:
dsn = cx_Oracle.makedsn(host="some host", port="some port", service_name="some service")
con = cx_Oracle.connect("user", "password", dsn, threaded=True)
Maybe these will help:
Similar setup?
Connect to Database using oracle client and kerberos with Python
CX-Oracle or Python OracleDB with External Auth:
https://github.com/oracle/python-cx_Oracle/issues/61#issuecomment-320121457
Can you please let me know how to connect to DB2 on IBM Cloud using python?
I have tried the below steps.
installed ibm_db using pip install ibm_db
Created a free tier Db2 service on IBM cloud
Generated Service credentials key
Trying to establish a connection with the database with the credential details (Database, host, port, user id, and password) extracted from the Service credentials key
import ibm_db
print("Creating connection.......")
conn_string = "DATABASE=bludb;HOSTNAME=54a2f15b-5c0f-46df-8954-7e38e612c2bd.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud;PORT=32733;PROTOCOL=TCPIP;UID=<userId>;PWD=<password>;"
conn = ibm_db.connect(conn_string,"","")
if conn:
print("Connection ...... [SUCCESS]")
else:
print("Connection ...... [FAILURE]")
I am getting below error message:
SQLCODE=-30082n: [IBM][CLI Driver] SQL30082N Security processing failed with reason "17" ("UNSUPPORTED FUNCTION"). SQLSTATE=08001
It seems like you are on the new Db2 on Cloud lite plan with non-standard ports and SSL enforced. When you connect to Db2 using the Python driver and use SSL, you have to add the SECURITY=SSL property, e.g.:
conn_string = "DATABASE=bludb;HOSTNAME=yourhostname;PORT=<port>;PROTOCOL=TCPIP;UID=<userId>;PWD=<password>;SECURITY=SSL"
I think the easy way to do this is with SQL Magic. This way, you can just type a SQL statement just by adding %sql before your query
First install the packages
!pip install sqlalchemy==1.3.9
!pip install ibm_db_sa
Then,
%load_ext sql
Finally, run the following code by replacing your username, password, hostname and SSL. You can find these in your IBM DB2 under credentials.
%sql ibm_db_sa://my-username:my-password#hostname:port/BLUDB?security=SSL
Now you can run any SQL by using %sql before or %%sql if the whole cell is going to be SQL. Example:
%sql SELECT * FROM TABLENAME;
or
%%sql
SELECT *
FROM TABLENAME;
There's a python package called ibm_db, does this link or (cited in the first one) this one help?
I'm using Python on Centos 7 and I have installed GSK8Kit with DB2 11.3 client.
So I set:
IBM_DB_HOME=/path/to/my/db2client/sqllib - ODBC and clidriver
Also I set:
LD_LIBRARY_PATH = $IBM_DB_HOME/lib:$LD_LIBRARY_PATH
Then I installed ibm_db:
pip install ibm_db
I added my db2servercert.arm into mykeydb.kdb file, located /opt/IBM/db2/GSK8KitStore and I'm using the same version of GSK8Kit on client and server.
gsk8capicmd_64 -cert -add -db mykeydb.kdb -stashed -label "DB2 Server
self-signed certificate" -file db2servercert.arm -format ascii -trust enable
According to this IBM docs: https://www.ibm.com/support/knowledgecenter/SSEPGG_11.1.0/com.ibm.db2.luw.admin.sec.doc/doc/t0053518.html
From Db2 V10.5 FP5 onwards, the SSLClientKeystoredb and SSLClientKeystash keywords are not needed in the connection string, db2cli.ini file, FileDSN, or db2dsdriver.cfg file. If you have not set or passed values for the SSLClientKeystoreddb and SSLClientKeystash keywords, the CLI/ODBC client driver will create a default key database internally during the first SSL connection. The Client driver will call GSKit API's to create a key database populated with the default root certificates.
Now I'm trying to create ibm_db connection string for db2 SSL connection using various scenarios:
Security=ssl and SSLServerCertificate=/path/to/my/db2servercert.arm "Database=sampledb;Protocol=tcpip;Hostname=myhost;Servicename=50001;Security=ssl;SSLServerCertificate=/path/to/my/db2servercert.arm;"
SECURITY=SSL and SSLClientKeystoredb=/opt/IBM/db2/GSK8KitStore/mykeydb.kdb and SSLClientKeystash=/opt/IBM/db2/GSK8KitStore/mystashfile.sth
"Database=sampledb;Protocol=tcpip;Hostname=myhost;Servicename=50001;Security=ssl;SSLClientKeystoredb=/opt/IBM/db2/GSK8KitStore/mykeydb.kdb;SSLClientKeystash=/opt/IBM/db2/GSK8KitStore/mystashfile.sth;"
Security=ssl
"Database=sampledb;Protocol=tcpip;Hostname=myhost;Servicename=50001;Security=ssl;"
In 1) and 2) I was able to connect without any SSL error connections, but in 3) I'm getting Socket 414 error:
[IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "SSL".
Communication API being used: "SOCKETS". Location where the error was detected: "".
Communication function detecting the error: "sqlccSSLSocketSetup". Protocol specific error code(s): "414", "", "". SQLSTATE=08001
That means:
https://www.ibm.com/support/knowledgecenter/en/SSAL2T_7.1.0/com.ibm.cics.tx.doc/reference/r_gskit_error_codes.html,
414 error: GSK_ERROR_BAD_CERT - Incorrectly formatted certificate received from partner.
Note: on another machine with the same config and ibm_db installed this connection string works (I'm sure I missed smth)
"Database=sampledb;Protocol=tcpip;Hostname=myhost;Servicename=50001;Security=ssl;"
My questions are:
Which env variables or db2 client parameters I have to configure to connect only with Security=ssl property?
How does ibm_db work under the hood, when trying to connect to db2 remote server and where I can find this root certificate based on which it automatically generate its own keydb.kdb file as mentioned in IBM docs?
Thx for any idea ;)
If you're using a self-signed SSL certificate, you can't connect without using options 1 or 2.
In option 1 you're supplying the certificate's public key directly, to allow the Db2 client to validate the Db2 server. This is already using the "in memory keystore" that you're asking about in question #2.
In option 2, you would have imported the same public key into your keystore to allow the Db2 client to validate the server.
If you want to connect using only Security=SSL, your Db2 server's SSL certificate needs to come from one of the CAs already in the system keystore.
I believe that when the Db2-documentation writes "The Client driver will call GSKit API's to create a key database populated with the default root certificates", it means that the dynamically created kdb will contain the certs for some common commercial CAs, and (if specified) will also contain the cert specified by SSLServerCertificate.
As you are using a self-signed certificate, the CA certs will be ignored in this case.
If you are connecting to a Db2-server that runs on Linux/Unix/Windows, using IBM's drivers, and want an encrypted connection that uses the target Db2-instance public-key as part of the encryption, then you must tell the Db2-client the location of that certificate (which contains the Db2-instance public key) in one way or another.
For a linux client, thay cert will either be in a statically created kdb (via GSKit commands), or in a dynamically created kdb as specified by using the SSLServerCertificate property. For a Db2-client running on Microsoft Windows the certificate can additionally be fetched from the MS keystore if Db2-client is configured to use that.
The source code for ibm_db module is available on github. However, the client-side SSL work happens not in ibm_db module but instead happens in the (closed source) Db2-driver along with (closed source) libraries for GSKit. To see some of what's happening under the covers you can trace the CLI driver. Refer to the Db2-documentation online for details of CLI tracing.
I am trying to connect to the Oracle instance which is running on Windows 10 through python using cx_Oracle package from a mac machine.
Now while connecting it throw below error.
'ORA-21561: OID generation failed\n'
My Sample code:
import cx_Oracle
DSN = cx_Oracle.makedsn(host=server, port=port, service_name=database)
# Below is the DNS
# (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.1.9)(PORT=50244))(CONNECT_DATA=(SERVICE_NAME=devXDB)))
con = cx_Oracle.Connection(user, password, DSN)
However I am able to connect from same machine (mac) using SQL developer and PyCharm's database browser. I searched across and did not find any solution related to remote instance. The solutions suggested for seems to be working only for the local instances in which one has to edit/update etc/hosts or related file on windows 10.
Thanks in advance.
This was indeed the problem of /etc/hosts file issue.
One thing to note here even if the oracle instance is running on a remote machine you client machine's (from where you are connecting to the oracle instance) /etc/hosts file should have the entry like this.
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting. Do not change this entry.
##
127.0.0.1 localhost localhost.localdomain Amits-iMac.local
Replace 'Amits-iMac.local' to your client's hostname.
How can i connect to remote MS Access DB using pyodbc, before i can able to connect to MS SQL server using pyodbc via freetds. But if i use the same freetds configuration for remote access, 'i'm not getiing the success.
Any suggestions?
freeTDS is ODBC driver for using the TDS communication protocol. MS SQL Server uses the TDS protocol.
Access does not, so it is not a surprise that it doesn't work. What you need is an ODBC driver for the Jet engine used by Access. And it exists. It is called libmdb. You can install the packages to get it set up.
Read MDB Tools Installation Guide for more information.