Using SQL Server stored procedures from Python (pyodbc) - python

I'm have a stored procedure, code:
DECLARE #RC int
DECLARE #id varchar(13)
DECLARE #pw varchar(13)
DECLARE #depart varchar(32)
DECLARE #class varchar(12)
DECLARE #name varchar(12)
DECLARE #birthday varchar(10)
DECLARE #grade int
DECLARE #subgrade int
SELECT #id = 'test'
SELECT #pw = '12345'
SELECT #depart = 'none'
SELECT #class = 'GM'
SELECT #name = 'name'
SELECT #birthday = 'None'
SELECT #grade = 3
SELECT #subgrade = 2
EXEC #RC = [my_database].[dbo].[my_table] #id, #pw, #depart, #class, #name, #birthday, #grade, #subgrade
DECLARE #PrnLine nvarchar(4000)
PRINT 'Stored Procedure: my_database.dbo.my_table'
SELECT #PrnLine = ' Return Code = ' + CONVERT(nvarchar, #RC)
How i can make a raw sql query to create account using this procedure?
I'm using flask and pyodbc.


From the pyodbc documentation
To call a stored procedure right now, pass the call to the execute method using either a format your database recognizes or using the ODBC call escape format. (The ODBC driver will then reformat the call for you to match the given database.)
For SQL Server you would use something like this:
# SQL Server format
cursor.execute("exec sp_dosomething(123, 'abc')")
# ODBC format
cursor.execute("{call sp_dosomething(123, 'abc')}")
So to call your procedure
id_ = 'test'
pw = '12345'
depart = 'none'
class_ = 'GM'
name = 'name'
birthday = 'None'
grade = 3
subgrade = 2
sql = 'exec [my_database].[dbo].[my_table](?, ?, ?, ?, ?, ?, ?, ?)'
values = (id_, pw, depart, class_, name, birthday, grade, subgrade)
cursor.execute(sql, (values))


The accepted answer does not address the issue of capturing the return value from the stored procedure, which can be done like this:
id_ = 'test'
pw = '12345'
depart = 'none'
class_ = 'GM'
name = 'name'
birthday = 'None'
grade = 3
subgrade = 2
sql = """\
SET NOCOUNT ON;
DECLARE #RC int;
EXEC #RC = [my_database].[dbo].[my_sp] ?, ?, ?, ?, ?, ?, ?, ?;
SELECT #RC AS rc;
"""
values = (id_, pw, depart, class_, name, birthday, grade, subgrade)
cursor.execute(sql, values)
rc = cursor.fetchval() # pyodbc convenience method similar to cursor.fetchone()[0]


Don't forget SET NOCOUNT ON in your stored procedure.


Another flavour of Gord's answer is using OUTPUT and named parameters (to be defined within the Stored procedure) for clarity.
id_ = 'test'
pw = '12345'
depart = 'none'
class_ = 'GM'
name = 'name'
birthday = 'None'
grade = 3
subgrade = 2
sql = """\
DECLARE #RC int;
EXEC [my_database].[dbo].[my_sp] #RC OUTPUT, #id_=?, #pw=?, #depart=?, #class_=?, #name=?, #birthday=?, #grade=?, #subgrade=?;
SELECT #RC AS rc;
"""
values = (id_, pw, depart, class_, name, birthday, grade, subgrade)
cursor.execute(sql, values)
rc = cursor.fetchval()


With a cursor initialized by your connection, the sp can be called directly as follow
sql = " exec your_SP #codemp = ?, #fecha = ? "
prm = (dict['param1'], dict['param2'])
cursor.execute(qry, params)


After searching everywhere for this solution, i couldnt find a simplified version. All results seem to overcomplicate this that should be so easy to do. Heres my solution.
import pyodbc
import pandas as pd
import datetime as d
conn = pyodbc.connect('Driver=;'
'Server=;'
'Database=;'
'UID=;'
'PWD=;')
# define parameters to be passed in and out
quarter_date = d.date(year=2020, month=10, day=1)
SQL = r'exec TERRITORIES_SP #quarterStart = ' + "'" + str(quarter_date) + "'"
print(SQL)
try:
cursor = conn.cursor()
cursor.execute(SQL)
cursor.close()
conn.commit()
finally:
conn.close()


For MSSQL the correct format is this:
SQL = 'exec sp_UpdateUserGoogleAuthenticated ''?'', ''?'''
Try running the Stored Procedure in MSSQL in the SQL Query window and it will fail every time with () surrounding the ? marks. If you escape the single quotes it will allow for variables with spaces in them.

Related

Insert variable Data into SQL server using python

def LiraRateApiCall():
R = requests.get(url)
timestamp = R.json()['buy'][-1][0]/1000
format_date = '%d/%m/%y'
date = datetime.fromtimestamp(timestamp)
buyRate = R.json()['buy'][-1][1]
print(date.strftime(format_date))
print(buyRate)
#ADDDING TO SQL SERVER
conn = odbc.connect("Driver={ODBC Driver 17 for SQL Server};"
'Server=LAPTOP-36NUUO53\SQLEXPRESS;'
'Database=test;'
'Trusted_connection=yes;')
cursor = conn.cursor()
cursor.execute('''
INSERT INTO Data_table (Time1,Price)
VALUES
('date',140),
('Date2' , 142)
''')
conn.commit()
cursor.execute('SELECT * FROM Data_table')
for i in cursor:
print(i)
How do i pass the variables date and buy rate to the table instead of putting in values liek i did (i put in'date' , 140 for example but i want to pass variables not specific values)

You'll need to check the driver version that you're using, but what you're looking for is the concept of bind variables. I'd suggest you look into the concept of fast_executemany as well - that should help speed things up. I've edited your code to show how bind variables typically work (using the (?, ?) SQL syntax), but there are other formats out there.
def LiraRateApiCall():
R = requests.get(url)
timestamp = R.json()['buy'][-1][0]/1000
format_date = '%d/%m/%y'
date = datetime.fromtimestamp(timestamp)
buyRate = R.json()['buy'][-1][1]
print(date.strftime(format_date))
print(buyRate)
#ADDDING TO SQL SERVER
conn = odbc.connect("Driver={ODBC Driver 17 for SQL Server};"
'Server=LAPTOP-36NUUO53\SQLEXPRESS;'
'Database=test;'
'Trusted_connection=yes;')
cursor = conn.cursor()
#Setup data
data = [('date',140), ('Date2' , 142)]
#Use executemany since we have a list
cursor.executemany('''
INSERT INTO Data_table (Time1,Price)
VALUES (?, ?)
''', data)
conn.commit()
cursor.execute('SELECT * FROM Data_table')
for i in cursor:
print(i)

I dont understand at all your question
If you want to pass the variables:
insert_sql = 'INSERT INTO Data_table (Time1,Price) VALUES (' + date + ',' + str(buyRate) + ')'
cursor.execute(insert_sql)
If you want to do dynamic Insert:
You can only insert knowing the values ​​or by inserting with a select
INSERT INTO table
SELECT * FROM tableAux
WHERE condition;
That or you could iterate through the fields you have in a table, extract them and compare it to your variables to do a dynamic insert.
With this select you can extract the columns.
SELECT *
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'table1'

Python sqlite operation,SQL statement 'where field in (1,2)' syntax error

Python sqlite operation,SQL statement 'where field in (1,2)' syntax error
The error is:sqlite3.OperationalError: near ":id": syntax error
My search of the Official Python documentation and Google failed to find the answer:
https://docs.python.org/3/library/sqlite3.html
How should arguments be passed?
'''first create test.db:table and field
CREATE TABLE test_tab (
id INTEGER PRIMARY KEY ASC,
test_num INT,
test_field TEXT
);
'''
import sqlite3
con = sqlite3.connect('test.db')
con.set_trace_callback(print) # start the debug
d = [
(111,'aaa'),
(111,'bbb'),
(111,'ccc'),
(444,'ddd')
]
sql = "insert into `test_tab` (`test_num`, `test_field`) values (?,?)"
cursor = con.executemany(sql, d)
con.commit() # Execute successfully
#####################
# wrong code begin,why sql 'in ()' is wrong?
sql = "SELECT * from `test_tab` where `test_num`=:num AND `id` in :id"
par = {'num': 111, 'id': (1,2)} # The number of 'id' parameters is uncertain
x = con.execute(sql, par)
print(x.fetchall())

In the second query, you would need actually separate placeholders for every value in the IN clause. In addition, I would use ? here:
num = 111
ids = (1, 2)
par = (num,) + ids
sql = "select * from test_tab where test_num = ? AND id in "
in_clause = '(?' + ', ?'*(len(ids) - 1) + ')'
sql = sql + in_clause
x = con.execute(sql, par)
print(x.fetchall())
The SQL query generated by the above script is:
select * from test_tab where test_num = ? AND in (?, ?)
and we bind (111, 1, 2) to the three ? placeholders.

Transfering Data in MS Access Using Python

I have an ever growing and changing database that reflects a permits passed by the State and EPA.
As the database changes and updates I need to transfer the relevant information.
The script does two things; first it checks which fields are the same and creates a list of fields and data that will be inserted into the new database. Second to insert the data into the new database.
Problem is I cannot get it to insert. I have matched everything like it says online in various ways but i get error ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in INSERT INTO statement. (-3502) (SQLExecDirectW)').
I cannot figure out how to prevent it.
Code:
import pyodbc
importDatabase = r"J:\ENVIRO FIELD\AccessDatabases\MS4\MS4 Town Databases\~Template\MS4_Apocalypse Import DEV 1.accdb"
"Create the Import Database Connection"
connectionImport = pyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=%s;' %(importDatabase))
cursorImport = connectionImport.cursor()
"####---Outfall Section---####"
"Import the outfall names into the new database"
tbl = "tbl_Outfall_1_Profile"
exportList = []
importList = []
for row in cursorImport.columns(table = "tblExportMigration_Outfall_1_Profile"):
field = row.column_name
exportList.append(field)
for row in cursorImport.columns(table = "tbl_Outfall_1_Profile"):
field = row.column_name
importList.append(field)
matchingList = []
for field in exportList:
if field != "outfallID":
if field in importList:
matchingList.append(field)
else:
continue
sqlValue = ""
for field in matchingList:
sqlValue += "[%s], " %(field)
sqlValue = sqlValue[:-2]
sql = "SELECT %s from %s" %(sqlValue, "tblExportMigration_Outfall_1_Profile")
for rowA in cursorImport.execute(sql):
tupleList = list(rowA)
tupleList = ["" if i == None else i for i in tupleList]
tupleValues = tuple(tupleList)
sqlUpdate = """INSERT INTO tbl_Outfall_1_Profile (%s) Values %s;""" %(sqlValue, tupleValues)
cursorImport.execute(sqlUpdate)
cursorImport.close()
This is the sql string I create
"INSERT INTO tbl_Outfall_1_Profile ([profile_OutfallName], [profile_HistoricalName1], [profile_HistoricalName2], [profile_HistoricalName3], [profile_HistoricalName4]) Values ('756', '', '', '', '');"

Taking what #Gord Thompson said I was actually able to create a dynamic parameter flow
First created a module to create the ?
def Defining_Paramters(length):
parameterString = ""
for x in range(1,length):
parameterString += "?, "
parameterString += "?"
return parameterString
Then stuck it into the string for the sql update
sqlUpdate = sqlUpdate = "INSERT INTO %s (%s) Values (%s);" %(table, sqlFrameworkSubStr, parameters)
Run the cursor and commit it
cursorTo.execute(sqlUpdate, (dataTuple))
connectionTo.commit()
It would seem that you have to create the query in its entirety then have your data in tuple format for entry

This is the sql string [I think] I create
Try this:
sqlUpdate = """INSERT INTO tbl_Outfall_1_Profile (%s) Values (%s);""" %(sqlValue, tupleValues)
or perhaps:
sqlUpdate = "INSERT INTO tbl_Outfall_1_Profile (%s) Values (%s);" %(sqlValue, tupleValues)

Incorrect Number of Bindings Error with SQLite

A little background I am doing this in python 2.7 for the reason of Alexa Rank through SeoLib but am completely open to updating if that would help this issue or possibly solve future issues.
Now this program sorts through sites that I have in a predetermined csv that looks like the following:
site
00rbt.com
I am specifically getting the following error:
File "igorPanda.py", line 84, in <module>
update_site(site,cur_ip,cur_rank,cur_hash)
File "igorPanda.py", line 30, in update_site
(site))
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 9 supplied.
Where this error is occuring is:
def update_site(site,cur_ip,cur_rank,cur_hash):
conn = sqlite3.connect('/root/Database/Sites.db')
cursor = conn.cursor()
with conn:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)",
(site))
record = cursor.fetchone()
if record[0] == 1:
cursor.execute("UPDATE sites SET cur_ip = ?, cur_rank = ?, cur_hash = ? WHERE site = ?",
(cur_ip,cur_rank,cur_hash,site))
else:
cursor.execute("INSERT into sites values (?,?,?,?,?,?,?,?)",
(site,cur_ip,None,cur_rank,None,None,cur_hash,None))
My entire code except imports is:
#Updates the DB
def update_site(site,cur_ip,cur_rank,cur_hash):
conn = sqlite3.connect('/root/Database/Sites.db')
cursor = conn.cursor()
with conn:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)",
(site))
record = cursor.fetchone()
if record[0] == 1:
cursor.execute("UPDATE sites SET cur_ip = ?, cur_rank = ?, cur_hash = ? WHERE site = ?",
(cur_ip,cur_rank,cur_hash,site))
else:
cursor.execute("INSERT into sites values (?,?,?,?,?,?,?,?)",
(site,cur_ip,None,cur_rank,None,None,cur_hash,None))
#Moves CSV for Historical Savings
bashmove = "mv top.csv /root/Desktop/scripts/results-igor/top-$(date +%m-%d-%Y).csv"
#Sets file with all sites to variable
filename='/root/Desktop/scripts/sorted.csv'
#Creates hash algorithm for later use
hasher = hashlib.sha256()
sess = requests.Session()
x = datetime.datetime.now()
date = x.strftime('%Y-%m-%d')
regex = r"^(?:https?:)?(?:\/\/)?(?:[^#\n]+#)?(?!www\.)?([^:\/\n]+)\w*\.\b(com|org|co|be|de|br|(\w+\b))" #regex to get stripepd website no https://www. or anything afte$
df = pd.DataFrame()
df = df.append(pd.read_csv(filename), ignore_index=True)
ip = "NOT FOUND"
for i in df.index:
#print(i)
site = df['site'][i]
try :
ip = socket.gethostbyname(site)
page = requests.get('http://' + df['site'][i], timeout=5)
hasher.update((page.text).encode('utf-8'))
except: #ignore errors if the site is bad
pass
try :
alexa_rank = seo.get_alexa('http://{}'.format(site)) #seolib gets the alexa ranking
#alexa_rank = None
except:
pass
site = site
cur_ip = ip
cur_rank = alexa_rank
cur_hash = hasher.hexdigest()
update_site(site,cur_ip,cur_rank,cur_hash)
rd = call(["/root/Desktop/scripts/./rDNSlookup.sh", site, ip]) #call bash script to get reverse DNS of ip
wi = call(["/root/Desktop/scripts/./whois.sh", site]) #call bash script to print host info
with open('/root/Desktop/scripts/IGOR_His/'+ date + 'sites.csv', 'a') as f:
print >> f, 'site: ',site,', ip: ',ip,', rank: ',alexa_rank, ', hash', cur_hash
shutil.copy("/root/Database/Sites.db", "/var/www/html/sites/Sites.db")
The table has the following columns:
site,cur_ip,prev_ip,cur_rank,prev_rank,play,cur_hash,prev_hash

TL;DR: use (site,)
In detail:
cursor.execute expects the second paramter to be an iterable.
When your code says:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", (site))
Then (site) is not a tuple - you probably meant to convert it to a tuple by wrapping it in ().
So what you meant/want is:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", (site,))
Notice the extra ,!
To avoid this tuple confusion, you can also use a list:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", [site])
The issue in even more detail:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", (site))
is the same as:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", site)
is the same (in your case) as:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", "00rbt.com")
The string 00rbt.com is used as iterable, and since it has 9 characters, you get the error:
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 9 supplied.

You get this error, because here:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", (site))
you should pass, as the 2nd argumnet of execute() a tuple and not just a string.
This is easily fixed by adding a comma like this:
cursor.execute("SELECT EXISTS(SELECT * from sites WHERE site = ?)", (site,))
But, if your version of SQLite is 3.24.0+, you should consider taking a different approach by choosing UPSERT as the method to do what you want.
You want to insert a new row if the site value does not exist in the table or update the existing row if it does exist.
So, assuming there is already a unique constraint for the column site, you can do it more efficiently with:
sql = """
INSERT into sites values (?,?,?,?,?,?,?,?)
ON CONFLICT(site) DO UPDATE SET
cur_ip = EXCLUDED.cur_ip, cur_rank = EXCLUDED.cur_rank, cur_hash = EXCLUDED.cur_hash
"""
cursor.execute(sql, (site,cur_ip,None,cur_rank,None,None,cur_hash,None))

How can I get input variables to be stored in a sqlite3 table?

I am trying to get the user input the information so it can be stored in a database using SQLite3 but the error: "sqlite3.OperationalError: no such column: first_name" keeps popping up and I don't know why. Any suggestions?
import sqlite3
connection = sqlite3.connect('database.db')
cursor = connection.cursor()
create_table = '''
CREATE TABLE example (
fname VARCHAR(20),
lname VARCHAR(30),
gender CHAR(1));'''
cursor.execute(create_table)
first_name = input('First name : ')
surname = input('surname: ')
gend = input('Gender: ')
add_staff = '''INSERT INTO example (fname, lname, gender)
VALUES (first_name, surname, gend);'''
cursor.execute(add_staff)
cursor.execute('SELECT * FROM example')
result = cursor.fetchall()
for r in result:
print(r)
cursor.execute('SELECT * FROM example')
I would like to stick to sqlite3 rather than using another database library within python. Thanks.

You need to pass the values of your variables to your SQL command, using placeholders.
add_staff = '''INSERT INTO example (fname, lname, gender)
VALUES (?, ?, ?);'''
cursor.execute(add_staff, (first_name, surname, gend))

Categories