PyODBC takes ~7 seconds to establish a connection with Azure SQL Server, is there a way to minimize this?
import os
import sys
import logging, logging.handlers
import getopt
import pyodbc
from database import *
# set up logging
logging.getLogger().setLevel(logging.INFO)
console = logging.StreamHandler()
console.setFormatter(logging.Formatter('%(asctime)s %(name)-12s %(levelname)s %(message)s'))
console.setLevel(logging.INFO)
logging.getLogger().addHandler(console)
logger = logging.getLogger("testapp")
def connect():
return pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
def purgeStoreData(conn, div_id, purge_days, lookback_days, store_start, store_end):
store_list = get_store_list(conn, div_id, store_start, store_end)
cursor = conn.cursor()
for store in store_list:
logger.info("Store %s ...", store)
cursor.execute("some query")
if __name__ == "__main__":
try:
conn = connect()
purgeStoreData(conn, DIV_ID, PURGE_DAYS, LOOKBACK_DAYS, STORE_START, STORE_END)
logger.info("*** Completed succesfully")
finally:
conn.close()
Is there a way to display the network latency ?
Could you please try to connect using the ODBC Driver 17 for SQL Server? You may find this way faster.
import pyodbc
server = '<server>.database.windows.net'
database = '<database>'
username = '<username>'
password = '{<password>}'
driver= '{ODBC Driver 17 for SQL Server}'
with pyodbc.connect('DRIVER='+driver+';SERVER=tcp:'+server+';PORT=1433;DATABASE='+database+';UID='+username+';PWD='+ password) as conn:
with conn.cursor() as cursor:
cursor.execute("SELECT TOP 3 name, collation_name FROM sys.databases")
row = cursor.fetchone()
while row:
print (str(row[0]) + " " + str(row[1]))
row = cursor.fetchone()
Microsoft drivers for Python are here.
Please add the following to the connection string also: "Encrypt=yes;TrustServerCertificate=no;". When a client first attempts a connection to SQL Azure, it sends an initial connection request. Consider this a "pre-pre-connection" request. At this point the client does not know if SSL/Encryption is required and waits an answer from SQL Azure to determine if SSL is indeed required throughout the session (not just the login sequence, the entire connection session). A bit is set on the response indicating so. Then the client library disconnects and reconnects armed with this information.
When you set "Encrypt connection" to "yes" you avoid the "pre-pre-connection" and the connection is faster.
In addition, please verify latency on your internet connection.
Related
Created API's in python Azure function and Got "Login Timeout error" when Connecting to physical Database. I am using Pyodbc to Make connections with DB.
connection_string = f'DRIVER={{ODBC Driver 17 for SQL Server}};SERVER=tcp:{MSSQL_SERVER_FQDN};DATABASE={MSSQL_DATABASE_NAME};UID={MSSQL_DATABASE_USER};PWD={MSSQL_DATABASE_PASSWD}'
logger.debug(f'Connection string: "{connection_string}"')
conn = pyodbc.connect(connection_string)
cursor = conn.cursor()
{"code": 400, "message": "[HYT00] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0) (SQLDriverConnect)"}
One of the workaround I tried to connect with Azure SQL Server using the Azure Function (Python Stack).
Make Sure you're using the correct connection string for ODBC and use it in the below code:
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
error = ''
try:
with pyodbc.connect('Driver={ODBC Driver 17 for SQL Server};Server=tcp:<yourservername>.database.windows.net,1433;Database=mySampleDatabase;Uid=azurekrish;Pwd=<YourPasswordHere>;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;')as conn:
with conn.cursor() as cursor:
cursor.execute("INSERT INTO [ApplicationLogs] ([LogMessage],[CreateDate]) VALUES('This is a test log message 1.', GETDATE())")
cursor.execute("SELECT * FROM [ApplicationLogs]")
row = cursor.fetchall()
except Exception as e:
error = str(e)
return func.HttpResponse(f"row fetch: {row}")
Browser response for connecting Azure SQL and getting values:
"Login Timeout error" when Connecting to physical Database.
The "Timeout expired" error commonly occurs when an instance of the SQL Server Database Engine is not running, when the server name was typed incorrectly, or when there are network problems or firewalls. Here are some causes of it: Troubleshooting: Timeout Expired.
I am trying to connect Python to our remote SQL Server but I am not getting it. Following is a code that I used.
server = 'server,1433'
database = 'db'
username = 'username'
password = 'pw'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
cursor.execute('SELECT top 1 * FROM db.dbo.t_location')
for row in cursor:
print(row)
We have 2 servers. One is database server but I use application server for SQL which connects to database server. This is the error I'm getting. I am trying for a week but I'm not sure what am I missing here.
Any help would be appreciated
OperationalError: ('08001', '[08001] [Microsoft][ODBC Driver 17 for SQL Server]TCP Provider: No such host is known.\r\n (11001) (SQLDriverConnect); [08001] [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired (0); [08001] [Microsoft][ODBC Driver 17 for SQL Server]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online. (11001)')
ADDED:
connection_str = ("Driver={SQL Server Native Client 11.0};"
"Server= 10.174.124.12,1433;"
#"Port= 1433;"
"Database=AAD;"
"UID=dom\user;"
"PWD=password;"
)
connection = pyodbc.connect(connection_str)
data = pd.read_sql("select top 1 * from dbo.t_location with (nolock);",connection)
I used the above code and now I see this error. Seems like it worked but failed to login. Usually I have to use Windows authentication in SSMS once I put my credentials to login in remote desktop.
('28000', "[28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user 'dom\user'. (18456) (SQLDriverConnect); [28000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Login failed for user 'dom\user'. (18456)")
Answer:
I am excited that I finally found a solution using pymssql. I don't know pyodbc wasn't working but I am sure I must have had done something wrong. I used below code to get the data from remote SQL server using Python.
import pymssql
conn = pymssql.connect(
host=r'10.174.124.12',
user=r'dom\user',
password=r'password',
database='db'
)
cursor = conn.cursor(as_dict=True)
cursor.execute('Select top 4 location_id, description from t_location with (nolock)')
data = cursor.fetchall()
data_df = pd.DataFrame(data)
cursor.close()
Ignore my code at this moment. I still have to do some cleaning but this code will work.
Finally to answer my question, I had to use pymssql which worked. I did not have to put the port number which was making me confused. Thanks everyone for taking out time to answer.
import pymssql
conn = pymssql.connect(
host=r'10.174.124.12',
user=r'dom\user',
password=r'password',
database='db'
)
cursor = conn.cursor(as_dict=True)
cursor.execute('Select top 4 location_id, description from t_location with (nolock)')
data = cursor.fetchall()
data_df = pd.DataFrame(data)
cursor.close()
you can use this function :
def connectSqlServer(Server , Database , Port , User , Password):
try:
conn = pyodbc.connect('Driver={SQL Server}; Server='+Server+';
Database='+Database+'; Port='+Port+'; UID='+User+'; PWD='+Password+';')
cursor = conn.cursor()
except Exception as e:
print("An error occurred when connecting to DB, error details: {}".format(e))
return False, None
else:
return True, cursor
I am having problems connecting to a remote Oracle DB using cx_Oracle in my python application.
I have tried a lot of different ways of configuring/formulating my connection string based on a lot of google searching etc but I seem to get the same error message almost each time no matter what I try.
My attempts looks like this:
import cx_Oracle
ip = '[IP ADDRESS]'
port = [PORT]
service_name = '[SERVICE NAME]'
dsn = cx_Oracle.makedsn(ip, port, service_name=service_name)
db = cx_Oracle.connect('[USERNAME]', '[PASSWORD], dsn)
Result: DatabaseError: ORA-12170: TNS:Connect timeout occurred
import cx_Oracle
conn_str = '[USERNAME]/[PASSWORD]#[HOST IP]/[SERVICE NAME]'
conn = cx_Oracle.connect(conn_str)
Result: DatabaseError: ORA-12170: TNS:Connect timeout occurred
import cx_Oracle
user= '[USERNAME]'
pwd = '[PASSWORD]'
host = '[HOST IP]'
service_name = '[SERVICE NAME]'
portno = '[PORT]'
conn = cx_Oracle.connect(user, pwd, '{}:{}/{}'.format(host,portno,service_name))
Result: DatabaseError: ORA-12170: TNS:Connect timeout occurred
import cx_Oracle
connstr = '[USERNAME]/[PASSWORD]#[SERVICE NAME]'
conn = cx_Oracle.connect(connstr)
Result: DatabaseError: ORA-12154: TNS:could not resolve the connect identifier specified
I have Toad installed on my PC and have no problems what so ever connecting to the DB with that.
Any ideas what could be the problem ?
Thanks in advance
I have this standard way to connect
import cx_Oracle
host="myserver"
port=myport
sid='myservicename'
user='myuser'
password='mypassword'
sid = cx_Oracle.makedsn(host, port, service_name=sid)
connection = cx_Oracle.connect(user, password, sid, encoding="UTF-8")
cursor = connection.cursor()
cursor.execute('select 1 from dual')
And it works without any issue. In your case, it is quite strange that you got a timeout, which normally indicates a network problem rather than a connection issue. If you are using cx_Oracle version 8, you don't need to specify the encoding as UTF-8 as it is the default one.
See how it works.
C:\python>type testconn.py
#from __future__ import print_function # needed for Python 2.7
import cx_Oracle
import os
host="myserver"
port=myport
sid='database_service_name'
user='myuser'
password='mypassword'
sid = cx_Oracle.makedsn(host, port, service_name=sid)
connection = cx_Oracle.connect(user, password, sid, encoding="UTF-8")
cursor = connection.cursor()
cursor.execute('select 1 from dual')
for row in cursor:
print(row)
C:\python>python testconn.py
(1,)
C:\python>
I am using mysql.connector to connect to a mysql DB, while i can connect manually to the db using sql server management, if i try connecting via code, it returns this error after awhile :
mysql.connector.errors.OperationalError: 2055: Lost connection to MySQL server at 'host:1234', system error: 10054 An existing connection was forcibly closed by the remote host
These are the connection details :
connection = mysql.connector.connect(host='host',
port = '1234',
database='DBname',
user='Usr',
password='pwd')
If I create a local mysql DB, the connection works just fine.
I assume some security stuff is going on, anyone else had encountered this situation ? Anything that I'm doing wrong? Should I add anything to the connection.connect input ?
Full code for reference :
import mysql.connector
from mysql.connector import Error
connection = mysql.connector.connect(host='host',
port = '1234',
database='DBname',
user='Usr',
password='pwd')
sql_select_Query = "select * from TableName"
cursor = connection.cursor()
cursor.execute(sql_select_Query)
records = cursor.fetchall()
print("Total numb of rows selected is : ", cursor.rowcount)
print("\nPrinting each row")
for row in records:
print(row)
connection.close()
I am not able to connect to MySQL sever using python it gives and error which says
MySQLdb._exceptions.OperationalError: (1130, "Host 'LAPTOP-0HDEGFV9' is not allowed to connect to this MySQL server")
The code I'm using:
import MySQLdb
db = MySQLdb.connect(host="LAPTOP-0HDEGFV9", # your host, usually localhost
user="root", # your username
passwd="abcd13de",
db="testing") # name of the data base
cur = db.cursor()
cur.execute("SELECT * Employee")
for row in cur.fetchall():
print(row[0])
db.close()
This is an authorization problem not a connectivity problem. Is the db running locally? If not, confirm with the admin where it is hosted. If so, try changing the host parameter to 127.0.0.1?
As described here the admin can get the hostname by running:
select ##hostname;
show variables where Variable_name like '%host%';
If the connection was timing out you could try setting the connect_timeout kwarg but that's already None by default.