It does not seem like SqlAlchemy supports calling stored procedures. Did anybody find a workaround for this that works with SQL Server?
Sample procedure:
CREATE PROCEDURE list_lock_set #name varchar (5), #requester varchar(30)
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO list_lock (name, requester, acquired) values (#name, #requester, GETDATE())
RETURN 0
END
GO
This works:
import pyodbc
dbh = pyodbc.connect(driver=''{SQL Server}'', server=srv, database=db, uid=uid, pwd=pwd)
dbc = dbh.cursor()
dbc.execute("list_lock_set ?, ?", ['bbc', 'pyodbc'])
dbc.commit()
This does not produce an error but also but does not work:
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQL Server', echo=True)
engine.execute("list_lock_set ?, ?", ['bbc', 'sqlalchemy'])
Thank you.
EDIT: It appears the best solution is to fish out pyodbc cursor from the engine:
cursor = engine.raw_connection().cursor()
cursor.execute("list_lock_set ?, ?", ['bbc', 'using cursor'])
cursor.commit()
I can also obtain pyodbc Connection:
engine.raw_connection().connection
and set autocommit=True, but that might interfere with engine logic. Many thanks to #Batman.
To have it working in sqlalchemy I managed to do it this way:
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQL Server', echo=True)
with engine.begin() as conn:
conn.execute("exec dbo.your_proc")
I remember this giving me grief before too. From memory either session.execute() or connection.execute() worked for me. There's also a callproc() method, which is probably the right way to go. http://docs.sqlalchemy.org/en/latest/core/connections.html
Also, I've had issues in the past with MSSQL which seemed to be due to something asynchronous happening where the method was returning before the procedure was finished, which resulted in unpredictable results on the database. I found that putting a time.sleep(1) (or whatever the appropriate number is) right after the call fixed this.
Yes, Even I was facing the same issue. SQLAlchemy was executing the procedure but no actions were happening.
So I tweaked it by adding connection.execute("exec <Procedure_name>")
Following on from #MATEITA LUCIAN:
Add input parameters to the stored procedure
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQLServer', echo=True)
with engine.begin() as conn:
conn.execute('EXEC dbo.your_proc #textinput_param = "sometext", #intinput_param = 4')
For those that aren't sure about how or what to refer to with the '....//usr:passw#srv/db?driver=SQL server', echo=True... part of creating the engine I would suggest reading this and choosing Option 1 - Provide a DSN.
Retrieve output from the stored procedure
from sqlalchemy import create_engine
engine = create_engine('mssql+pyodbc://usr:passw#srv/db?driver=SQLServer', echo=True)
with engine.begin() as conn:
result = conn.execute('EXEC dbo.your_proc #textinput_param = "sometext", #intinput_param = 4')
for row in result:
print('text_output: %s \t int_output: %d' % (row['text_field'], row['int_field']))
Related
In the website I'm making with Flask, I'm trying to "upgrade" my search bar by using the pg_trgm PostgreSQL extension to match words even after being spelled wrong. However, I'm having trouble figuring out how to use the extension's custom functions alongside SQLAlchemy.
Will I have to use raw SQL to properly perform the queries, or is there a way to do this more cleanly?
Assuming the extension is configured correctly, these functions should be accessible like any other database function. For example, here's how the similarity function might be called using SQLAlchemy core:
import sqlalchemy as sa
engine = sa.create_engine('postgresql+psycopg2:///test', echo=True, future=True)
metadata = sa.MetaData()
tbl = sa.Table('t70546926', metadata,
sa.Column('c1', sa.String, primary_key=True),
sa.Column('c2', sa.String))
tbl.drop(engine, checkfirst=True)
tbl.create(engine)
ins = sa.insert(tbl).values(c1='Alice', c2='Alison')
with engine.begin() as conn:
conn.execute(ins)
query = sa.select(sa.func.similarity(tbl.c.c1, tbl.c.c2).label('similarity_result'))
with engine.connect() as conn:
rows = conn.execute(query)
for row in rows:
print(row.similarity_result)
For Flask-SQLAlchemy, you might do something like this:
result = dbsession.query(sa.func.similarity(val1, val2)).all()
I am trying to add a new auto incremented row in an MSSQL database and read the id value created. When I do the SQL statement runs okay but as soon as I try to query the row from the returned ResultProxy object I get a 'Function sequence error'
I am using SQL Alchemy (pyodbc) to connect to the database and using execute() to run text queries. Running the SQL command via SQL Management Studio works fine.
This is what I am trying;
from sqlalchemy import create_engine
from sqlalchemy.sql import text
dbstring = "mssql+pyodbc://<SNIPPED>"
engine = create_engine(dbstring)
con = engine.connect()
res = con.execute(text("""INSERT INTO test(v) OUTPUT INSERTED.id VALUES( :v )"""), {"v": 10})
print(res) # This prints <sqlalchemy.engine.result.ResultProxy object>
print(res.first()) # This is the line where things go wrong
The final line of code causes the following error;
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('HY010', '[HY010] [Microsoft][SQL Server Native Client 11.0]Function sequence error (0) (SQLFetch)')
Any ideas what I am doing wrong?
Thanks, Andrew
Use engine.begin() instead engine.connect() like so:
with engine.begin() as con:
new_id = con.execute(text("""INSERT INTO test(v) OUTPUT INSERTED.id VALUES( :v )"""), {"v": 10}).scalar()
Related question: Get MSSQL autogenerated id from INSERT Statement using SQLAlchemy
I am trying to create a database with pg8000 driver of postgressql , but unable to create. Creating a db manually and then connecting to it works fine with me, but i need to create db with my code. I am getting error "sqlalchemy.exc.ProgrammingError: (pg8000.ProgrammingError)". I have tried below code for creating a db.
from sqlalchemy import create_engine
dburl = "postgresql+pg8000://user:pswd#myip:5432/postgres/"
engine = create_engine(dburl)
conn = engine.connect()
conn.execute("COMMIT")
conn.execute("CREATE DATABASE qux")
I also tried with below -
from sqlalchemy import create_engine
from sqlalchemy.engine import url
settings ={"drivername" : "postgresql+pg8000", "host" : "myip","port" : 5432,"username" : "user","password" : "pswd","database" : "MyTestDB"}
db=create_engine(url.URL(**settings))
db.execute("commit")
This is the exact Error i am getting """sqlalchemy.exc.ProgrammingError: (pg8000.ProgrammingError) ('ERROR', '25001', 'CREATE DATABASE cannot run inside a transaction block') [SQL: 'create database workDB']""""
Please suggest as to how i can create this db...
Here's a solution:
from sqlalchemy import create_engine
dburl = "postgresql+pg8000://user:pswd#myip:5432/postgres/"
engine = create_engine(dburl)
conn = engine.connect()
con.rollback() # Make sure we're not in a transaction
con.autocommit = True # Turn on autocommit
conn.execute("CREATE DATABASE qux")
con.autocommit = False # Turn autocommit back off again
The docs talk about this problem of executing commands that can't be run in a transaction. The thing is that pg8000 automatically executes a begin transaction before any execute() if there isn't already a transaction in progress. This is fine until you come to execute a command that can't be executed inside a transaction. In that case you have to enter autocommit mode, which implicitly starts a transaction before the statement and commits it afterwards, but automatically avoids doing this if (like CREATE DATABASE) the statement can't be executed within a transaction.
I am trying to just create a temporary table in my SQL database, where I then want to insert data (from a Pandas DataFrame), and via this temporary table insert the data into a 'permanent' table within the database.
So far I have something like
""" Database specific... """
import sqlalchemy
from sqlalchemy.sql import text
dsn = 'dsn-sql-acc'
database = "MY_DATABASE"
connection_str = """
Driver={SQL Server Native Client 11.0};
Server=%s;
Database=%s;
Trusted_Connection=yes;
""" % (dsn,database)
connection_str_url = urllib.quote_plus(connection_str)
engine = sqlalchemy.create_engine("mssql+pyodbc:///?odbc_connect=%s" % connection_str_url, encoding='utf8', echo=True)
# Open connection
db_connection = engine.connect()
sql_create_table = text("""
IF OBJECT_ID('[MY_DATABASE].[SCHEMA_1].[TEMP_TABLE]', 'U') IS NOT NULL
DROP TABLE [MY_DATABASE].[SCHEMA_1].[TEMP_TABLE];
CREATE TABLE [MY_DATABASE].[SCHEMA_1].[TEMP_TABLE] (
[Date] Date,
[TYPE_ID] nvarchar(50),
[VALUE] nvarchar(50)
);
""")
db_connection.execute("commit")
db_connection.execute(sql_create_table)
db_connection.close()
The "raw" SQL-snippet within sql_create_table works fine when executed in SQL Server, but when running the above in Python, nothing happens in my database...
What seems to be the issue here?
Later on I would of course want to execute
BULK INSERT [MY_DATABASE].[SCHEMA_1].[TEMP_TABLE]
FROM '//temp_files/temp_file_data.csv'
WITH (FIRSTROW = 2, FIELDTERMINATOR = ',', ROWTERMINATOR='\n');
in Python as well...
Thanks
These statements are out of order:
db_connection.execute("commit")
db_connection.execute(sql_create_table)
Commit after creating your table and your table will persist.
This python code should run statements on the database, but the sql statements are not executed:
from sqlalchemy import *
sql_file = open("test.sql","r")
sql_query = sql_file.read()
sql_file.close()
engine = create_engine(
'postgresql+psycopg2://user:password#localhost/test', echo=False)
conn = engine.connect()
print sql_query
result = conn.execute(sql_query)
conn.close()
The test.sql file contains SQL statements which create 89 tables.
The tables are not created if I specify 89 tables, but if I reduce the number of tables to 2 to it works.
Is there a limit on the number of queries that can be executed within the conn.execute? How do a run any number of raw queries like this?
Perhaps, forcing the autocommit:
conn.execute(RAW_SQL).execution_options(autocommit=True))
Other approach is using transactions and doing the commit:
t = conn.begin()
try:
conn.execute(RAW_SQL)
t.commit()
except:
t.rollback()
PD: You can put the execution_options in the create_engine parameters too.
Why do you use raw SQL with SQLAlchemy? If you have no good reason for that, you should use other methods:
http://docs.sqlalchemy.org/en/rel_0_7/orm/tutorial.html
http://docs.sqlalchemy.org/en/rel_0_7/core/schema.html#metadata-describing