I am trying to establish a connection to the impala database through a Python script using a keytab instead the normal user/password combination, but am unable to find any tutorials online whatsoever, the code I am currently using is:
conn = connect(host=impala_host, port=impala_port, use_ssl=True, auth_mechanism="PLAIN", user=username, password=pwd, database=impala_db)
cursor = conn.cursor()
However, I want to connect using keytab instead of my password.
It appears that you are trying to use that library: https://github.com/cloudera/impyla
Please have a look at Usage section in README.md there:
from impala.dbapi import connect
conn = connect(host='my.host.com', port=21050)
cursor = conn.cursor()
cursor.execute('SELECT * FROM mytable LIMIT 100')
print cursor.description # prints the result set's schema
results = cursor.fetchall()
...
...
Related
i've been trying to get some data from my db by using below code, but the code is not working. is there any mistake that i made in the code, if so how can i fix it.
NOTE: i took the below code from just a script not a django or flesk web app.
def db():
conn = psycopg2.connect(
"dbname=mydb user=postgres password=****** host=*.*.*.*")
cur = conn.cursor()
cur.execute("""SELECT * FROM MddPublisher""")
query_results = cur.fetchall()
print(query_results)
db()
ERROR: psycopg2.errors.UndefinedTable: relation "mddpublisher" does not exist LINE 1: SELECT * FROM MddPublisher
additionally,i want to show below code to prove that connection is ok. the problem is that i can't receive data from my db whenever i try to execute select command through python.
def print_tables():
conn = psycopg2.connect(
"dbname=mydb user=postgres password=***** host=*.*.*.*.*")
cur = conn.cursor()
cur.execute("""SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'""")
for table in cur.fetchall():
print(table)
print_tables()
OUTPUT:
('MddPublisher',)
This is probably an issue with case sensitivity. Postgresql names are usually normalized to lower case. However, when used inside double quotes, they keep their case. So, to access a table named MddPublisher you must write it like "MddPublisher".
All the gory details are in Section 4.1.1, Identifiers and Key Words in the Postgresql 14 docs.
I have a postgres db running on docker. I am able to access this db via my sql client Dbeaver and when I run select statements I see the expected results.
I would like to be able to query this db via a Python script and after some searching found psycopg2 package.
When I run the code below it 'looks' like it's successful, the conn and cursor objects appear as a variables.
import pandas as pd
import psycopg2
# connect to db
conn = psycopg2.connect(
host="localhost",
database="postgres",
user="postgres",
password="example")
# create a cursor
cur = conn.cursor()
However, when trying to query the db using cur.connect(), , variable ex_data is None. This exact same query via my sql client returns a table of data.
ex_data = cur.execute('select * from myschema.blah limit 10;')
How can I query my db via Python using psycopg2? Desired result wold be a data frame with the result set from the query string above.
I am trying to create pyspark jdbc dataframe using LDAP connections from Oracle.
Below code is working for normal connection string based JDBC dataframe creation which worked.
creds = {"user":"USER_NAME",
"password":"PASSWORD",
"driver": "oracle.jdbc.OracleDriver"}
connection_string = "jdbc:oracle:thin:#//hostname.com:1521/myint.domain.com"
df = spark_session.read.jdbc(url=connection_string, table=query, properties=creds)
Now, our database configurations changed and we are supposed to use LDAP based authentication only.
Hence I tried changing connection_string as below.
connection_string = "myint"
But, it is throwing below issue.
py4j.protocol.Py4JJavaError: An error occurred while calling o51.jdbc.
: java.lang.NullPointerException
Without Spark, I tried connecting using cx_Oracle module(python module for connecting to Oracle) for testing and it worked.
Before:
host = 'myint.domain.com'
ip = 'jdbc:oracle:thin:#//hostname.com'
port = 1521
conn = cx_Oracle.makedsn(ip, port, service_name=host)
db = cx_Oracle.connect('USER_NAME', 'PASSWORD', conn) # giving the conn object
cursor = db.cursor()
cursor.execute("""select * from myschema.mytable fetch first 5 rows only""")
for row in cursor:
print(row)
After:
db = cx_Oracle.connect('USER_NAME', 'PASSWORD', "myint") # Now, just giving only the database
cursor = db.cursor()
cursor.execute("""select * from myschema.mytable fetch first 5 rows only""")
for row in cursor:
print(row)
I need to achieve the same in Pyspark JDBC dataframe by just passing only the database name. Please suggest what needs to be done.
I hope this question can be applicable for scala spark as well.
First of all thank you for your help.
I was trying to retrieve some data from a sybase IQ database using python, but I can not make it.
I've tried with the following code( from https://github.com/sqlanywhere/sqlanydb):
import sqlanydb
conn = sqlanydb.connect(uid='dba', pwd='sql', eng='demo', dbn='demo' )
curs = conn.cursor()
curs.execute("select 'Hello, world!'")
print( "SQL Anywhere says: %s" % curs.fetchone() )
curs.close()
conn.close()
Unfotunately it gives me the following error:
InterfaceError: ('Could not load dbcapi. Tried: None,dbcapi.dll,libdbcapi_r.so,libdbcapi_r.dylib', 0)
Does anyone know how to fix it?
Thanks in advance
Jessica
On windows, first you need to add the data source name (DSN).
You do this by searching for 'odbc data source administrator' on windows and creating a DSN for 'SQL Anywhere 12'. Fill in the necessary information like username,password,host,port,server name and database name. Finally test the connection as shown.
Once finished you can call the code as follows:
import sqlanydb
conn = sqlanydb.connect(dsn='SYBASE_IQ')
curs = conn.cursor()
curs.execute("select 'Hello, world!'")
print( "SQL Anywhere says: %s" % curs.fetchone())
curs.close()
conn.close()
Get and install the SYBASE ODBC DRIVER.
Configure the DSN on your PC.
On Windows, search for the Microsoft ODBC Administrator. Then create a DSN.
Python code:
using SQLAchemy
import sqlalchemy as sa
from sqlalchemy import create_engine, event
from sqlalchemy.engine.url import URL
import urllib
params = urllib.parse.quote_plus('DSN=dsn_name;PWD=user_pwd')
engine = sa.create_engine("sybase+pyodbc:///?odbc_connect={}".format(params))
with engine.connect() as cursor:
cursor.execute(""" SELECT * FROM database """)
Using PyODBC
import pyodbc
conn = pyodbc.connect('DSN=dsn_name;PWD=user_pwd')
with conn:
cursor = conn.cursor()
cursor.execute(""" SELECT * FROM database """)
I'm using Visual Studio 2017 with a Python Console environment. I have a MySQL database set up which I can connect to successfully. I can also Insert data into the DB. Now I'm trying to display/fetch data from it.
I connect fine, and it seems I'm fetching data from my database, but nothing is actually printing to the console. I want to be able to fetch and display data, but nothing is displaying at all.
How do I actually display the data I select?
#importing module Like Namespace in .Net
import pypyodbc
#creating connection Object which will contain SQL Server Connection
connection = pypyodbc.connect('Driver={SQL Server};Server=DESKTOP-NJR6F8V\SQLEXPRESS;Data Source=DESKTOP-NJR6F8V\SQLEXPRESS;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False')
cursor = connection.cursor()
SQLCommand = ("SELECT ID FROM MyAI_DB.dbo.WordDefinitions WHERE ID > 117000")
#Processing Query
cursor.execute(SQLCommand)
#Commiting any pending transaction to the database.
connection.commit()
#closing connection
#connection.close()
I figured it out. I failed to include the right Print statement. Which was:
print(cursor.fetchone())
I also had the connection.commit statement in the wrong place (it was inserted even executing the Print statement). The final code that worked was this:
#importing module Like Namespace in .Net
import pypyodbc
#creating connection Object which will contain SQL Server Connection
connection = pypyodbc.connect('Driver={SQL Server};Server=DESKTOP-NJR6F8V\SQLEXPRESS;Data Source=DESKTOP-NJR6F8V\SQLEXPRESS;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=True;ApplicationIntent=ReadWrite;MultiSubnetFailover=False')
cursor = connection.cursor()
SQLCommand = ("SELECT * FROM MyAI_DB.dbo.WordDefinitions")
#Processing Query
cursor.execute(SQLCommand)
#Commiting any pending transaction to the database.
print(cursor.fetchone())
connection.commit()
#closing connection
#connection.close()