I have the following python code
cred_dict = load_credentials()
user = cred_dict["user"]
pwd = cred_dict["pwd"]
host = cred_dict["host"]
port = cred_dict["port"]
db = cred_dict["db"]
schema = cred_dict["schema"]
con_string = f'Driver={{PostgreSQL Unicode}};Server={host};Database={db};Port={port};UID={user};PWD={pwd};'
params = urllib.parse.quote_plus(con_string)
con = sqlalchemy.create_engine("mssql:///?odbc_connect=%s" % params,fast_executemany=True)
con = con.connect()
but I keep getting ERROR: function schema_name() does not exist.
I have tested different drivers but im using the PostgreSQL Unicode for my DSN setting in ODBC, and it is working w/o any issues. I have also tried mssql+pyodbc but it does not do any difference.
Note, I cannot use pyodbc.connect("DSN="+DSN) since I have to use an sqlalchemy-connection for our production-environment
I managed to work around it by installing psycopg2 and then do
con = create_engine(f'postgresql://{self.user}:{self.pwd}#{self.host}:{self.port}/{self.db}')
which then is passed to
df = pd.read_sql(query=query, con=con)
Related
Consider below code:-
Now the below works
connection = cx_Oracle.connect(dsn = 'DSNAME')
But when I use below format for SqlAlchemy it doesn't work, I get TypeError: Invalid arguments dsn passed:
connection = create_engine('oracle+cx_oracle://' , dsn = 'DSNAME')
SQLAlchemy requires a database connection URI, there is an article about it on their documentation. It requires the format
oracle+cx_oracle://user:pass#host:port/dbname[?key=value&key=value...]
Have you tried the following?
connection = create_engine('oracle+cx_oracle://' + 'DSNAME')
it seems pretty old post, I tried with below code, it works . hope this helps. providing empty username/passwd, they are read from wallet, location is mentioned in sqlnet.ora
tnsnames.ora:
t1 = (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ip)(PORT=1521)(KEY=dbpdb1))(CONNECT_DATA=(SERVICE_NAME=dbsvc1.oracle.com))).
sqlnet.ora:
WALLET_LOCATION =
(SOURCE =
(METHOD = FILE)
(METHOD_DATA =
(DIRECTORY = $walletdir)
)
)
SQLNET.WALLET_OVERRIDE = TRUE
from sqlalchemy import create_engine
cstr='oracle://:#t1'
print(cstr)
engine = create_engine(
cstr,
convert_unicode=False,
echo=True
)
s='select * from emp'
conn = engine.connect()
result = conn.execute(s)
for row in result:
print (row)
I've been stuck for a few days trying to run some code in MySQL to fill a database that I have already created. Initially upon running I got the error 1251 :
"Client does not support authentication protocol requested by server; consider upgrading MySQL client". In the MySQL documentation and stackoverflow answers I found, I was led to change the default insecureAuth setting from the default false to true. Here is the code I am currently using...
import datetime
import MySQLdb as mdb
from math import ceil
def obtain_btc():
now = datetime.datetime.utcnow()
symbols = ['BTC', 'Crypto', 'Bitcoin', 'No Sector', 'USD', now, now]
return symbols
def insert_btc_symbols(symbols, insecureAuth):
db_host = 'localhost'
db_user = 'natrob'
db_pass = '**********'
db_name = 'securities_master'
con = mdb.connect(host=db_host,user=db_user,passwd=db_pass,db=db_name,{insecureAuth:true})
column_str = "ticker, instrument, name, sector, currency, created_date, last_updated_date"
insert_str = (("%s, ")*7)[:2]
final_str = ("INSERT INTO symbols (%s) VALUES (%s)" % (column_str,insert_str))
print (final_str,len(symbols))
with con:
cur = con.cursor()
for i in range(0,int(ceil(len(symbols)/100.0))):
cur.executemany(final_str,symbols[i*100:(i+1)*100-1])
if __name__ == "__main__":
symbols = obtain_btc()
insert_btc_symbols(symbols)
I recently have gotten the error: "non-keyword arg after keyword arg". I've tried to switch the order to no avail, which leads me to believe that I may not be changing the default setting correctly. Any help or advice is appreciated. Thank you.
The issue looks like is coming from {insecureAuth:true} where it is not a keyword argument. ie var=value. I'm not familiar with the library but if that is a keyword then you should be able to set it as a keyword or pass it with **
con = mdb.connect(host=db_host,user=db_user,passwd=db_pass,db=db_name,insecureAuth=True)
or
con = mdb.connect(host=db_host,user=db_user,passwd=db_pass,db=db_name,**{insecureAuth:true})
I managed to get the section of code working by getting the public key for the password and using that in place of the normal password. This was in lieu of using the insecureAuth parameters.
I have pyodbc code that I use to connect to a DSN, however for some reason it is no longer working and I cannot figure out why (the drivers are empty even though they are there).
So I want to try and convert everything to use SQLAlchemy instead.
My current code for connecting to the database is:
conn = pyodbc.connect('DSN=QueryBuilder')
cursor = conn.cursor()
stringA = "SELECT GrantInformation.Call FROM GrantInformation"
cursor.execute(stringA)
rows = cursor.fetchall()
How would I get this to do the same in SQLAlchemy, I have checked the documentation and I am still confused.
Many thanks
I used:
from sqlalchemy import create_engine
engine = create_engine("""{}://{}:{}#{}/{}"""
.format(SQL Server,nick,mypassword,myservername,querybuilder))
df = pd.read_sql_query("SELECT GrantInformation.Call FROM GrantInformation")
and I got:
File "<ipython-input-5-f7837462519f>", line 4
.format(SQL Server,nick,mypassword,myservername,querybuilder))
^
SyntaxError: invalid syntax
Also declared the variables before, and I now get:
ArgumentError: Could not parse rfc1738 URL from string 'SQL Server://nick:mypassword#myhost/querybuilder'
from sqlalchemy import create_engine
engine = create_engine("""{}://{}:{}#{}/{}"""
.format(driver,user,password,host,database))
df = pd.read_sql_query("SELECT GrantInformation.Call FROM GrantInformation", engine)
Use one of the below code format to create engine
from sqlalchemy import create_engine
# default
engine = create_engine('mysql://scott:tiger#localhost/foo')
# mysql-python
engine = create_engine('mysql+mysqldb://scott:tiger#localhost/foo')
# MySQL-connector-python
engine = create_engine('mysql+mysqlconnector://scott:tiger#localhost/foo')
# OurSQL
engine = create_engine('mysql+oursql://scott:tiger#localhost/foo')
# query
connection = engine.connect()
result = connection.execute("select username from users")
database name = foo, username = scott, password = tiger, host = localhost
Reference: http://docs.sqlalchemy.org/en/latest/dialects/mysql.html
I am trying to do a sanity testing of newly installed Oracle client 12.2 in RHEL 7 linux from a Python program, but it fails with the above error, not sure what I am missing on there. Please help with this case :
cx_Oracle.DatabaseError: ORA-12514: TNS:listener does not currently know of service
requested in connect descriptor
my tnsnames.ora file under /home directory
FRDLD2D1 =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(Host = frdld2d1.de.db.com)(Port = 1825))
)
(CONNECT_DATA =
(SID = FRDLD2D1)
)
)
and my python program goes below
#!/usr/bin/python
import cx_Oracle
#connection = cx_Oracle.connect('PNTH_LOGGINGB_OWNER/password')
connection = cx_Oracle.connect('PNTH_LOGGINGB_OWNER/password#10.245.63.34:1825/orcl')
cursor = connection.cursor()
querystring = "select * from BDR_JOB_MASTER_LOG where ROWNUM <= 1;"
cursor.execute(querystring)
frdld2d1.de.db.com - IP address : 10.245.63.34
Appreciate if any points the glitch on here.
tnsping utility is not there to test since it is an instaclient version
oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm.
But with SQLPlus, I am able to connect the database without any issues.
Please use this as your connection string :
connection = cx_Oracle.connect('PNTH_LOGGINGB_OWNER', 'hdgf_76trf',
cx_Oracle.makedsn('10.245.63.34',1825,'FRDLD2D1') );
Changing SID = FRDLD2D1 to SERVICE_NAME = FRDLD2D1 in your TNSNAMES.ORA file may be an alternative.
You just make dsn in python without config file (tnsnames.ora)
dsn = cx_Oracle.makedsn(host='10.245.63.34', port=1825, sid='FRDLD2D1')
con = cx_Oracle.connect(user='PNTH_LOGGINGB_OWNER', password='password', dsn=dsn)
I am trying to use sqlalchemy to connect to an oracle DB. I was expecting the following to work given that it appears the exact syntax is shown in the sqlalchemy documentation.
oracle_db = sqlalchemy.create_engine('oracle://user:pass#server:1521/dev')
but this results in the error:
dsn = self.dbapi.makedsn(url.host, port, **makedsn_kwargs)
TypeError: makedsn() takes no keyword arguments
The following call initially works without the service name
oracle_db = sqlalchemy.create_engine('oracle://user:pass#server:1521')
But when trying to connect it fails with an error complaining that the SERVICE_NAME was not provided.
ORA-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA
Oddly this works with cx_Oracle directly:
con = cx_Oracle.connect('user/pass#server:1521/dev')
How am I supposed to connect to the specific service?
Attempts
I have tried to use cx_Oracle.makedsn() explicitly from this question with no luck as well.
Trying to use ? options in the connection string
oracle_db = sqlalchemy.create_engine('oracle://user:pass#server:1521/?sid=dev')
works initially but when I try oracle_db.connect() I get the same ORA-12504 error shown above.
Based on the documentation at Sqlalchemy Documentation, you should probably use the cx_oracle engine. The connect string is:
oracle+cx_oracle://user:pass#host:port/dbname[?key=value&key=value...]
with an option of service_name or sid as follows:
oracle+cx_oracle://user:pass#host:1521/?service_name=hr
oracle+cx_oracle://user:pass#host:1521/?sid=hr
Try using this connection string:
engine = create_engine("oracle+cx_oracle://<username>:<password>#(DESCRIPTION = (LOAD_BALANCE=on) (FAILOVER=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = <host>)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = devdb)))")
It worked for me.