I'm trying to execute a SQL SELECT statment from Microsoft SQL Server and write that data to an excel spreadsheet.
However, whenever I execute the Python script, I am getting this error:
Traceback (most recent call last):
File "C:\Users\rrw\AppData\Roaming\Python\Python310\site-packages\sqlalchemy\engine\cursor.py", line 955, in fetchone
row = dbapi_cursor.fetchone()
pyodbc.Error: ('HY010', '[HY010] [Microsoft][ODBC Driver 17 for SQL Server]Function sequence error (0) (SQLFetch)')
Traceback (most recent call last):
File "F:\Astro\Python\AstroPy\WriteSQLData.py", line 91, in <module>
for row in rs:
I can see from the error, that it doesn't like the line: for row in rs at the very end of the script. But I can't figure out why.
Is there anything I am missing?
Here is my script:
from sqlalchemy import create_engine
from sqlalchemy.engine import URL
import pyodbc
import pandas as pd
import csv
import configparser
# Get data from configuration ini file
config = configparser.ConfigParser()
config.read('databaseConfig.ini')
destinationFile = config['destination']['fileName']
# Database Connection Code
connection_string = "DRIVER={ODBC Driver 17 for SQL Server};SERVER=ASTROLAB;DATABASE=AstronomyMaps;UID=xyz;PWD=xyz"
connection_url = URL.create(
"mssql+pyodbc", query={"odbc_connect": connection_string})
engine = create_engine(connection_url)
# Simple test query
qry = "SELECT TOP (1000) * FROM [AstronomyMaps].[dbo].[starMapA]"
with engine.connect() as con:
rs = con.execute(qry)
# Write query data to Excel spreadsheet
with open(destinationFile, 'w', newline='') as f:
a = csv.writer(f, delimiter=',')
a.writerow([x[0] for x in cursor.description])
a.writerows(row)
for row in rs:
print(row)
When you exit the context manager (with block) the statement has been executed but the rows haven't been retrieved yet. However, exiting the context manager also "terminates" the connection so for row in rs: throws an error when the ODBC driver tries to call SQLFetch.
You can avoid the error by using .all() to retrieve the rows while still in the with block:
qry = "SELECT TOP (1000) * FROM [AstronomyMaps].[dbo].[starMapA]"
with engine.connect() as con:
rs = con.exec_driver_sql(qry).all()
Related
I am trying to transfer data from postgresql db table to sqlite table using python. Here is my code:
import sqlite3
import csv
connect = sqlite3.connect("server.db")
cursor = connect.cursor()
sql = """CREATE TABLE users (
name TEXT,
id INT,
xp INT
)"""
cursor.execute(sql)
connect.commit()
with open('users.csv', 'r', encoding="utf-8") as f:
no_records = 0
for row in f:
cursor.execute(f"INSERT INTO users (?,?,?)", row.split(','))
connect.commit()
no_records += 1
connect.close()
But when running this script I get sqlite3.OperationalError: near "?": Syntax error:
Traceback (most recent call last):
File "C:\Users\belog\sort_hat\cr.py", line 19, in <module>
cursor.execute(f"INSERT INTO users (?,?,?)", row.split(','))
sqlite3.OperationalError: near "?": syntax error
How to fix it and is it possible to import data is easier without using python?
Your syntax is missing VALUES:
INSERT INTO users VALUES(?,?,?)
I have a python program that makes an SQL query to an Oracle DB using a query saved in a text file *.sql as shown below. I'm getting a "type error" as TypeError: expecting string or bytes object pointing to line with sql = query. I have not been able to solve it. Here is the program:
import pandas as pd
import cx_Oracle
from sys import exit
conn= cx_Oracle.connect('DOMINA/S#U#ex021-orc.corp.mycompany.com:1540/domp_domi_bi')
# READ SQL QUERY FROM FILE
with open(r'C:\Users\Au321103\.spyder-py3\Validation\DOMINA_query.sql') as f:
query = f.readlines()
# IMPORT INTO PANDAS DATA FRAME
try:
df = pd.read_sql(con = conn,
sql = query) # QUERY READ FROM .sql FILE
finally:
conn.close()
df.head()
exit()
Thank you in advance as I'm trying to learn these db queries.
I am writing a python script to get image data (blob) from Informix/Oracle database and upload images to AWS S3. Part of my code is as below:
try:
cur = conn.cursor()
cur.execute(sql)
for row in cur:
client = trim(row[0])
date = trim(row[1])
filename = trim(row[2])
imageblob = row[3].read()
write_file(filename, imageblob)
I got the following error (Informix case):
Error: <class '_informixdb.InterfaceError'>
Traceback (most recent call last):
File "UploadImagesToS3.py", line 57, in getImageFromDB
imageblob = row[3].read()
InterfaceError: Sblob is not open
Could anyone give a help? The code needs to be compatible with both Informix and Oracle DB. Thanks
So I am trying to retrieve data from database with MySQLdb in pandas dataframe.
import MySQLdb as mysqldb
import MySQLdb.cursors
import pandas as pd
def connection():
db = mysqldb.connect(
host="123.456.7.890",
user="user",
passwd="password",
db="database",
port=12345,
cursorclass=MySQLdb.cursors.DictCursor
)
return db
mysql = connection()
def testing():
cur = mysql.cursor()
query = cur.execute("select * from table1")
result = cur.fetchall()
cur.close()
result_df = pd.DataFrame(result)
return result_df
When I print the 'testing' function, I get an error:
Traceback (most recent call last):
File "C:/Users/xx/PycharmProjects/practice/python.py", line 97, in <module>
print(testing())
File "C:/Users/xx/PycharmProjects/practice/python.py", line 94, in testing
result_df = pd.DataFrame(result)
File "C:\Users\xx\PycharmProjects\practice\venv\lib\site-packages\pandas\core\frame.py", line 422, in __init__
raise ValueError('DataFrame constructor not properly called!')
ValueError: DataFrame constructor not properly called!
I put cursorclass to MySQLdb.cursors.DictCursor to get the data in dictionary form but instead it seems like I'm getting them in tuple. Currently using python 3.7.
It works when I use pymysql but seems quite slow.
I'm working on a Python script that writes records from a stored procedure to a text file. I'm having issues executing the stored procedure with parameters.
I'm not sure what I could do differently to execute this stored procedure with both parameters. You can see the error I'm getting below.
Any insight would be appreciated.
Here's my code
# Import Python ODBC module
import pyodbc
# Create connection
cnxn = pyodbc.connect(driver="{SQL Server}",server="<server>",database="<database>",uid="<username>",pwd="<password>")
cursor = cnxn.cursor()
# Execute stored procedure
storedProc = "exec database..stored_procedure('param1', 'param2')"
# Loop through records
for irow in cursor.execute(storedProc):
# Create a new text file for each ID
myfile = open('c:/Path/file_' + str(irow[0]) + '_' + irow[1] + '.txt', 'w')
# Write retrieved records to text file
myfile.write(irow[2])
# Close the file
myfile.close()
Here's the error
Traceback (most recent call lst):
File "C:\Path\script.py", line 12, in <module>
for irow in cursor.execute(storedProc):
pyodbc.ProgrammingError: ('42000', "[4200] [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near 'param1'. <102> <SQLExecDirectW>">
I was able to fix the syntax error by removing the parenthesis from the query string.
# Execute stored procedure
storedProc = "exec database..stored_procedure('param1', 'param2')"
should be
# Execute stored procedure
storedProc = "exec database..stored_procedure 'param1','param2'"
This worked for me
query = "EXEC [store_proc_name] #param1='param1', #param2= 'param2'"
cursor.execute(query)
For SQL Server:
cursor.execute('{call your_sp (?)}',var_name)