I am operating a python 3 notebook in Azure Data Studio (similar to jupyter notebook) and trying to access an existing global temp table (##TEMP1) via pyodbc.
Here is my code:
import pandas as pd
import pyodbc
con = pyodbc.connect(
Trusted_Connection="yes",
driver="{ODBC Driver 17 for SQL Server}",
host="SERVERNAME",
database=db,
trustServerCertificate="yes",
multisubnetfailover="yes",
applicationIntent="ReadOnly",
)
sql = """
select * from ##TEMP1
"""
data = pd.read_sql(sql, con)
con.close()
data.head()
In Azure Data Studio, when I switch the kernel to sql and simply query select * from ##TEMP1, it returns results, however when I try to run via the python code above via pyodbc, it's returning the following error.
DatabaseError: Execution failed on sql ' select * from ##TEMP1 ':
('####', "[####] [Microsoft][ODBC Driver 17 for SQL Server][SQL
Server]Invalid object name '##TEMP1'. (###) (SQLExecDirectW)")
Please help, those much smarter than I! :)
I think you want something like this:
I use pyodbc to make the connection and query the database. As you can see I create a curser with pyodbc and execute the curser with the SQL query string as an argument. Then use fetchall() to return the result for the curser
import pandas as pd
import pyodbc
con = pyodbc.connect(Trusted_Connection='yes', driver = '{ODBC Driver 17 for SQL Server}',host = 'SERVERNAME', database = db,trustServerCertificate='yes',multisubnetfailover='yes',applicationIntent='ReadOnly')
crsr = conn.cursor()
sql = '''
select * from ##TEMP1
'''
crsr.execute(sql)
results = crsr.fetchall()
con.close()
data.head()
Related
I am attempting to write a Python script that can take Excel sheets and import them into my SQL Server Express (with Windows Authentication) database as tables. To do this, I am using pandas to read the Excel files into a pandas DataFrame, I then hope to use pandas.to_sql() to import the data into my database. To use this function, however, I need to use sqlalchemy.create_engine().
I am able to connect to my database using pyodbc alone, and run test queries. This conection is done with the followng code:
def create_connection(server_name, database_name):
config = dict(server=server_name, database= database_name)
conn_str = ('SERVER={server};DATABASE={database};TRUSTED_CONNECTION=yes')
return pyodbc.connect(r'DRIVER={ODBC Driver 13 for SQL Server};' + conn_str.format(**config))
...
server = '<MY_SERVER_NAME>\SQLEXPRESS'
db = '<MY_DATABASE_NAME>
connection = create_connection(server, db)
cursor = connection.cursor()
cursor.execute('CREATE VIEW test_view AS SELECT * FROM existing_table')
cursor.commit()
However, this isn't much use as I can't use pandas.to_sql() - to do so I need an engine from sqlalchemy.create_engine(), but I am struggling to figure out how to use my same details in my create_connection() function above to successfully create an engine and connect to the database.
I have tried many, many combinations along the lines of:
engine = create_engine("mssql+pyodbc://#C<MY_SERVER_NAME>\SQLEXPRESS/<MY_DATABASE_NAME>?driver={ODBC Driver 13 for SQL Server}?trusted_connection=yes")
conn = engine.connect().connection
or
engine = create_engine("mssql+pyodbc://#C<MY_SERVER_NAME>\SQLEXPRESS/<MY_DATABASE_NAME>?trusted_connection=yes")
conn = engine.connect().connection
A Pass through exact Pyodbc string works for me:
import pandas as pd
from sqlalchemy import create_engine
from sqlalchemy.engine import URL
connection_string = (
r"Driver=ODBC Driver 17 for SQL Server;"
r"Server=(local)\SQLEXPRESS;"
r"Database=myDb;"
r"Trusted_Connection=yes;"
)
connection_url = URL.create(
"mssql+pyodbc",
query={"odbc_connect": connection_string}
)
engine = create_engine(connection_url)
df = pd.DataFrame([(1, "foo")], columns=["id", "txt"])
pd.to_sql("test_table", engine, if_exists="replace", index=False)
I'm using 32-bit Python along with 32-bit Microsoft Access.
I connect to the database and create a cursor.
conn = pyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=P:\path\database.accdb;')
c = conn.cursor()
I then read in a table for future use:
c.execute('select * from tbl')
tbl = c.fetchall()
But when I try to make a copy of the table using:
query = 'CREATE TABLE tbl2 LIKE tbl'
c.execute(query)
I get a programming error:
ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in CREATE TABLE statement. (-3551) (SQLExecDirectW)')
I'm new to PYODBC but this is a pretty basic SQL line. Any tips on what might be wrong?
Due to a lack of documentation, the best way I found to figure this out is to use Microsoft Access to create a template for the query...example below worked.
query = 'SELECT tbl.* INTO tbl2 FROM tbl'
c.execute(query)
I am attempting to write a Python script that can take Excel sheets and import them into my SQL Server Express (with Windows Authentication) database as tables. To do this, I am using pandas to read the Excel files into a pandas DataFrame, I then hope to use pandas.to_sql() to import the data into my database. To use this function, however, I need to use sqlalchemy.create_engine().
I am able to connect to my database using pyodbc alone, and run test queries. This conection is done with the followng code:
def create_connection(server_name, database_name):
config = dict(server=server_name, database= database_name)
conn_str = ('SERVER={server};DATABASE={database};TRUSTED_CONNECTION=yes')
return pyodbc.connect(r'DRIVER={ODBC Driver 13 for SQL Server};' + conn_str.format(**config))
...
server = '<MY_SERVER_NAME>\SQLEXPRESS'
db = '<MY_DATABASE_NAME>
connection = create_connection(server, db)
cursor = connection.cursor()
cursor.execute('CREATE VIEW test_view AS SELECT * FROM existing_table')
cursor.commit()
However, this isn't much use as I can't use pandas.to_sql() - to do so I need an engine from sqlalchemy.create_engine(), but I am struggling to figure out how to use my same details in my create_connection() function above to successfully create an engine and connect to the database.
I have tried many, many combinations along the lines of:
engine = create_engine("mssql+pyodbc://#C<MY_SERVER_NAME>\SQLEXPRESS/<MY_DATABASE_NAME>?driver={ODBC Driver 13 for SQL Server}?trusted_connection=yes")
conn = engine.connect().connection
or
engine = create_engine("mssql+pyodbc://#C<MY_SERVER_NAME>\SQLEXPRESS/<MY_DATABASE_NAME>?trusted_connection=yes")
conn = engine.connect().connection
A Pass through exact Pyodbc string works for me:
import pandas as pd
from sqlalchemy import create_engine
from sqlalchemy.engine import URL
connection_string = (
r"Driver=ODBC Driver 17 for SQL Server;"
r"Server=(local)\SQLEXPRESS;"
r"Database=myDb;"
r"Trusted_Connection=yes;"
)
connection_url = URL.create(
"mssql+pyodbc",
query={"odbc_connect": connection_string}
)
engine = create_engine(connection_url)
df = pd.DataFrame([(1, "foo")], columns=["id", "txt"])
pd.to_sql("test_table", engine, if_exists="replace", index=False)
There are a lot of tips online on how to use pyodbc to run a query in MS Access 2007, but all those queries are coded in the Python script itself. I would like to use pyodbc to call the queries already saved in MS Access. How can I do that?
If the saved query in Access is a simple SELECT query with no parameters then the Access ODBC driver exposes it as a View so all you need to do is use the query name just like it was a table:
import pyodbc
connStr = (
r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=C:\Users\Public\Database1.accdb;"
)
cnxn = pyodbc.connect(connStr)
sql = """\
SELECT * FROM mySavedSelectQueryInAccess
"""
crsr = cnxn.execute(sql)
for row in crsr:
print(row)
crsr.close()
cnxn.close()
If the query is some other type of query (e.g., SELECT with parameters, INSERT, UPDATE, ...) then the Access ODBC driver exposes them as Stored Procedures so you need to use the ODBC {CALL ...} syntax, as in
import pyodbc
connStr = (
r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=C:\Users\Public\Database1.accdb;"
)
cnxn = pyodbc.connect(connStr)
sql = """\
{CALL mySavedUpdateQueryInAccess}
"""
crsr = cnxn.execute(sql)
cnxn.commit()
crsr.close()
cnxn.close()
I am trying to move data from a database I own to a database I only have read/write permissions on (as a temp table). However, I get an error. Please see the below code:
import pandas as pd
import pyodbc
import sqlalchemy
con = pyodbc.connect(r'Driver={SQL Server};Server=MYServer;Database=DB_People;Trusted_Connection=yes;)
sSQL = "SELECT * FROM tbl_People;"
df = pd.read_sql(sSQL, con)
con.close()
con = pyodbc.connect(r'Driver={SQL Server};Server=NOT_MyServer;Database=DB_People;Trusted_Connection=yes;)
sSQL = "CREATE TABLE [##People Table] (p_Name varchar(25), p_DOB char(10));"
con.execute(sSQL)
con.committ()
engine = sqlalchemy.create_engine('mssql://NOT_MyServer/DB_People?Driver=SQL+Server+Native+Client+11.0?trusted_connection=yes')
df.to_sql("##People Table")
I get the error:
DBAPIError: (pyodbc.Error) ('IM002', '[IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified (0) (SQLDriverConnect)')
Any clues as to what the issue is? I specified a Driver in my connection and I know the Temp table exists because when I connect via SSMS I can query the table (although there are no records at this point.
I
It seems as if I have found the problem(s):
The "Create Table" function is unneeded as the df.to_sql creates the table
Instead of the "+" in the Driver name, I was able to replace them with %20
This worked for me.