I am using pyodbc to connect to an azure sql database. My source code looks like this:
import pyodbc
server = 'sqlserver.database.windows.net'
database = 'database'
username = 'username'
password = 'password'
conn= pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server}'+';SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+';PWD='+ password ';Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;')
cursor = conn.cursor()
cursor.execute("query")
I am able to connect to a sql database. The only thing which is not working properly is that pyodbc does not connect to the database I have specified in the database variable. It always connects to the master database.
What I have tried so far is to print the name of the databases on the target sql server using SELECT * FROM sys.databases while being connected to the master database. I was able to see the database I am trying to connect to. Anyone got an idea what goes wrong in my source code?
In general, I would assume that the connection string needs to be different in your case.
As per pyodbc docs:
[...]the most important thing to remember is that pyodbc does not even look at the connection string. It is passed directly to the database driver unmodified (through SQLDriverConnect). Connection strings are therefore driver-specific and all ODBC connection string documentation should be valid.
https://github.com/mkleehammer/pyodbc/wiki/Connecting-to-databases
However, since you are connecting to db OK and your connection string seems to be ignored, I would say that if you are using Windows then the connection parameters seem likely to be defined in the ODBC DSN, which can be changed in Control panel. If that is the case, and you have ODBC parameters defined in DSN, most likely your connection string is ignored, except for the choice of the DSN.
Related
I am trying to switch a pyodbc connection to sqlalchemy engine. My working pyodbc connection is:
con = pyodbc.connect('DSN=SageLine50v23;UID=#####;PWD=#####;')
This is what I've tried.
con = create_engine('pyodbc://'+username+':'+password+'#'+url+'/'+db_name+'?driver=SageLine50v23')
I am trying to connect to my Sage 50 accounting data but just can't work out how to build the connection string. This is where I downloaded the odbc driver https://my.sage.co.uk/public/help/askarticle.aspx?articleid=19136.
I got some orginal help for the pyodbc connection using this website (which is working) https://www.cdata.com/kb/tech/sageuk-odbc-python-linux.rst but would like to use SQLAlchemy for it connection with pandas. Any ideas? Assume the issue is with this part pyodbc://
According to this thread Sage 50 uses MySQL to store its data. However, Sage also provides its own ODBC driver which may or may not use the same SQL dialect as MySQL itself.
SQLAlchemy needs to know which SQL dialect to use, so you could try using the mysql+pyodbc://... prefix for your connection URI. If that doesn't work (presumably because "Sage SQL" is too different from "MySQL SQL") then you may want to ask Sage support if they know of a SQLAlchemy dialect for their product.
I can currently connect to my SQL Server and query any database I want to directly.
The problem is when I want to query a linked server. I cannot directly reference the linked servers name in the connect() method and I have to connect to a local database first and then run an OPENQUERY() against the linked server.
This seams like a odd work around. Is there a way to query the linked server directly (from my research you cannot connect directly to a linked server) or at least connect to the server without specifying a database where I can then run the OPENQUERY() for anything without having to first connect to a database?
Example of what I have to do currently:
import pyodbc
ex_value = "SELECT * FROM OPENQUERY(LinkedServerName,'SELECT * FROM LinkedServerName.SomeTable')"
# I have to connect to some local database on the server and cannot connect to linked server initially.
odbc_driver, server, db = '{ODBC Driver 17 for SQL Server}', 'MyServerName', 'LocalDatabase'
with pyodbc.connect(driver=odbc_driver, host=server, database=db, trusted_connection='yes') as conn:
conn.autocommit = False
cursor = conn.cursor()
cursor.execute(ex_value)
tables = cursor.fetchall()
for row in tables:
print('Row: {}'.format(row))
cursor.close()
As Sean mentioned, a linked server is just a reference to another server that's stored within the local server.
You do not need to manage 100+ user credentials though. If you have the users using Windows auth, and you have Kerberos working between the servers, the linked server can just impersonate you when it connects to the other server via the linked server definition.
Then you can use either 4 part names to refer to objects on the other server, or use OPENQUERY when you want more control over what gets executed where.
Finally, if they're both SQL Servers and both use the same collation, make sure you set the linked server option to say they are collation compatible. That can make a major difference to your linked server performance. I regularly see systems where that isn't set and it should be.
I am using sqlalchemy to connect to MySQL database and found a strange behavior.
If I query
LOAD DATA LOCAL INFILE
'C:\\\\Temp\\\\JaydenW\\\\iata_processing\\\\icer\\\\rename\\\\ICER_2017-10-
12T09033
7Z023870.csv
It pops an error:
sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1148, u'The used
command is not allowed with this MySQL versi
on') [SQL: u"LOAD DATA LOCAL INFILE
'C:\\\\Temp\\\\JaydenW\\\\iata_processing\\\\icer\\\\rename\\\\ICER_2017-10-
12T090337Z023870.csv' INTO TABLE genie_etl.iata_icer_etl LINES TERMINATED BY
'\\n'
IGNORE 1 Lines (rtxt);"] (Background on this error at:
http://sqlalche.me/e/2j85)
And I find the reason is that:
I need to set the parameter as
args = "mysql+pymysql://"+username+":"+password+"#"+hostname+"/"+database+"?
local_infile=1"
If I use MySQL official connection library. I do not need to do so.
myConnection = MySQLdb.connect(host=hostname, user=username, passwd=password, db=database)
Can anyone help me to understand the difference between the two mechanisms?
The reason is that the mechanisms use different drivers.
In SQLAlchemy you appear to be using the pymysql engine, which uses the PyMySQL Connection class to create the DB connection. That one requires the user to explicitly pass the local_infile parameter if they want to use the LOAD DATA LOCAL command.
The other example uses MySQLdb, which is basically a wrapper around the MySQL C API (and to my knowledge not the official connection library; that would be MySQL Connector Python, which is also available on SQLAlchemy as mysqlconnector). This one apparently creates the connection in a way that the LOAD DATA LOCAL is enabled by default.
I'm having some trouble with trying to create a read-only connection and not sure if it's a bug or my error.
Pyodbc's documentation indicates that it is possible to create a readonly connection. See https://mkleehammer.github.io/pyodbc/api-module.html
When running the following though I receive no errors and it runs as if the READONLY keyword was not present at all (the update goes through).
import pyodbc
readonly_conn_str = "DRIVER={SQL Server Native Client 10.0};SERVER=...;DATABASE=...;UID=...;PWD=...;READONLY=True;"
conn = pyodbc.connect(readonly_conn_str)
cursor = conn.cursor()
result = cursor.execute(update_query)
cursor.commit()
print(result.rowcount)
If I try using function keywords I get the same thing.
...
conn = pyodbc.connect(conn_str, readonly=True)
...
When we invoke pyodbc.connect with readonly=True, pyodbc dutifully calls the ODBC function
ret = SQLSetConnectAttr(cnxn->hdbc, SQL_ATTR_ACCESS_MODE, (SQLPOINTER)SQL_MODE_READ_ONLY, 0);
SQL_MODE_READ_ONLY is one of the standard ODBC SQL_ATTR_ACCESS_MODE attributes. How that actually affects the behaviour of the ODBC connection is up to the ODBC driver. As stated in the ODBC documentation:
SQL_MODE_READ_ONLY is used by the driver or data source as an indicator that the connection is not required to support SQL statements that cause updates to occur. This mode can be used to optimize locking strategies, transaction management, or other areas as appropriate to the driver or data source. The driver is not required to prevent such statements from being submitted to the data source. The behavior of the driver and data source when asked to process SQL statements that are not read-only during a read-only connection is implementation-defined.
In other words, pyodbc has passed the "read only" attribute to the ODBC driver. It is up to the driver to decide whether it should be interpreted as a hint, a hard limitation, or something to be simply ignored.
I can connect via pyODBC to an unsupported database over ODBC. Queries appear to execute correctly. If I try to connect using mssql+pyodbc, I can't connect properly (image not found).
I've tried "base:///", "base+pyodbc:///", or "pyodbc:///".
Do I need to write my own "dialect" that doesn't make any changes to base, and are there any useful (up to date) guides on how to do this?
EDIT:
import pyodbc
conn = pyodbc.connect(DSN = "ODBCCONNECTIONNAME", UID = "ODBCUSER", PWD="PASSWORD")
cursor = conn.cursor()
Also works with this line replacing the connection above:
conn = pyodbc.connect("DSN=fmp_production;UID=odbc_user;PWD=Pwd222")
I can then run selects, and modifications fine, using standard SQL.
EDIT:
Okay, so I'm getting an error:
"Abort trap: 6" basically my python process is crashing out. I've tried testing with a functioning ODBC connection to a MySQL database, using:
engine = create_engine('mysql+pyodbc://root:rootpwd#testenvironment')
If I include the name of the database, then I get an image not found error instead. But I think the actual problem is whatever is crashing my python.