I want to call a stored procedure and receive the output parameter in python. I am using sqlAlchemy and can use parameters but do not know how to have the output be read into a variable. I understand that there is a outParam() attribute in sqlAlchemy, but I have not found a useful example.
Here is a simple SQL code for testing:
CREATE PROCEDURE [dbo].[Test]
#numOne int,
#numTwo int,
#numOut int OUTPUT
AS
BEGIN
SET NOCOUNT ON;
SET #numOut = #numOne + #numTwo
END
And simple python:
engine = sqlalchemy.create_engine("mssql+pyodbc:///?odbc_connect=%s" % params)
outParam = 0
result = engine.execute('Test ? ,?, ? OUTPUT', [1, 2, outParam])
outParam is still 0. I have tried modifying it with:
outParam = sqlalchemy.sql.outparam("ret_%d", type_=int)
But this produces a "Programming Error." What am I missing?
SQLAlchemy returns a ResultProxy object. Try it like this:
engine = sqlalchemy.create_engine(...)
rproxy = engine.execute(...)
result = rproxy.fetchall()
result should be a list of RowProxy objects that you can treat like dictionaries, keyed on the column names from the query.
If you are looking for a true OUT param, your approach is almost correct. There is just a small error in the first parameter in the call to sqlalchemy.sql.outparam (but it is valid Python syntax). It should be like this:
outParam = sqlalchemy.sql.outparam("ret_%d" % i, type_=int)
Note the change to the first parameter: it just needed a value to substitute into the format string. The first parameter is the key value, and most likely %d is to be replaced with the column index number (which should be in i).
Related
This question is probably me not understanding architecture of (new) sqlalchemy, typically I use code like this:
query = select(models.Organization).where(
models.Organization.organization_id == organization_id
)
result = await self.session.execute(query)
return result.scalars().all()
Works fine, I get a list of models (if any).
With a query with specific columns only:
query = (
select(
models.Payment.organization_id,
models.Payment.id,
models.Payment.payment_type,
)
.where(
models.Payment.is_cleared.is_(True),
)
.limit(10)
)
result = await self.session.execute(query)
return result.scalars().all()
I am getting first row, first column only. Same it seems to: https://docs.sqlalchemy.org/en/14/core/connections.html?highlight=scalar#sqlalchemy.engine.Result.scalar
My understanding so far was that in new sqlalchemy we should always call scalars() on the query, as described here: https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#migration-orm-usage
But with specific columns, it seems we cannot use scalars() at all. What is even more confusing is that result.scalars() returns sqlalchemy.engine.result.ScalarResult that has fetchmany(), fechall() among other methods that I am unable to iterate in any meaningful way.
My question is, what do I not understand?
My understanding so far was that in new sqlalchemy we should always call scalars() on the query
That is mostly true, but only for queries that return whole ORM objects. Just a regular .execute()
query = select(Payment)
results = sess.execute(query).all()
print(results) # [(Payment(id=1),), (Payment(id=2),)]
print(type(results[0])) # <class 'sqlalchemy.engine.row.Row'>
returns a list of Row objects, each containing a single ORM object. Users found that awkward since they needed to unpack the ORM object from the Row object. So .scalars() is now recommended
results = sess.scalars(query).all()
print(results) # [Payment(id=1), Payment(id=2)]
print(type(results[0])) # <class '__main__.Payment'>
However, for queries that return individual attributes (columns) we don't want to use .scalars() because that will just give us one column from each row, normally the first column
query = select(
Payment.id,
Payment.organization_id,
Payment.payment_type,
)
results = sess.scalars(query).all()
print(results) # [1, 2]
Instead, we want to use a regular .execute() so we can see all the columns
results = sess.execute(query).all()
print(results) # [(1, 123, None), (2, 234, None)]
Notes:
.scalars() is doing the same thing in both cases: return a list containing a single (scalar) value from each row (default is index=0).
sess.scalars() is the preferred construct. It is simply shorthand for sess.execute().scalars().
I'm trying to use SQLAlchemy's insert and execute methods with a list of dictionaries as the values, and to get back the inserted ids. The database is Postgres, which should support RETURNING.
The code looks more or less like this:
table = MyThing.__table__
insert_stmt = table.insert().values({"name": bindparam("name_val")}).returning(table.c.id)
values = [{"name_val": "a"}, {"name_val": "b"}]
result = session.execute(insert_stmt, values).fetchall()
(trimmed down for readability and some obfuscation)
When the value list is bigger than one (as in the example above) I get:
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) no results to fetch
but when the list only have one value the new id is returned properly.
I understand that I probably missed something in the way I'm using returning but I can't figure out what.
The same bind parameter is repeated in the list of values you're passing in the second argument to session.execute.
values ought to be a list when inserting multiple values and sqlalchemy by default binds the parameters unless literal_binds option is set to True so that it is safe to do without explicitly binding the parameters
values = [{"name_val": "a"}, {"name_val": "b"}]
insert_stmt = table.insert().values(values).returning(table.c.id)
result = session.execute(insert_stmt).fetchall()
Say you wanted to keep the existing explicit binding, values passed for the insert statement must be a keyword argument. Your existing query will be updated as follow:
bind_params = [{"name_val": "a"}, {"name_val": "b"}]
insert_stmt = table.insert().values(name_val=bindparam("name_val")).returning(table.c.id)
result = session.execute(insert_stmt, bind_params).fetchall()
I have the following table in mysql(5.7.12):
class Story(db.Model):
sections_ids = Column(JSON, nullable=False, default=[])
sections_ids is basicly a list of integers [1, 2, ...,n].
I need to get all rows where sections_ids contains X.
I tried the following:
stories = session.query(Story).filter(
X in Story.sections_ids
).all()
but it throws:
NotImplementedError: Operator 'contains' is not supported on this expression
Use JSON_CONTAINS(json_doc, val[, path]):
from sqlalchemy import func
# JSON_CONTAINS returns 0 or 1, not found or found. Not sure if MySQL
# likes integer values in WHERE, added == 1 just to be safe
session.query(Story).filter(func.json_contains(Story.section_ids, X) == 1).all()
As you're searching an array at the top level, you do not need to give path. Alternatively beginning from 8.0.17 you can use value MEMBER OF(json_array), but using it in SQLAlchemy is a little less ergonomic in my opinion:
from sqlalchemy import literal
# self_group forces generation of parenthesis that the syntax requires
session.query(Story).filter(literal(X).bool_op('MEMBER OF')(Story.section_ids.self_group())).all()
For whoever get here, but is using PostgreSQL instead:
your fields should be of the type sqlalchemy.dialects.postgresql.JSONB (and not sqlalchemy_utils.JSONType) -
Then you can use the Comparator object that is associated with the field with its contains (and others) operators.
Example:
Query(Mymodel).filter(MyModel.managers.comparator.contains(["user#gmail.com"]))
(note that the contained part must be a JSON fragment, not just a string)
If mysql has no output...
if record[0][0]:
will return an error
IndexError: tuple index out of range
the only solution i know to fix this issue is:
try:
if record[0][0]:
# Do stuff
except IndexError:
pass
but this looks like a very heavy wrapper script
only to find out if
record[0][0]
has no data. ( no value )
is there something lighter that can be done such as..
if record[0][0] = ""
?
UPDATE:
This is my MYSQL code:
a = _mysql.escape_string(a)
db=b()
db.query("select * from b where a='" + a + "' limit 1")
result = db.store_result()
record = result.fetch_row()
UPDATE:
turns out what worked is:
if record:
rather than
if record[0]:
or
if record[0][0]:
In the general case, if you want to check if an item exists in a list, just check that it exists. Exceptions are considered Pythonic code. Using another construct for access checking is likely to be less readable and suffer from performance problems.
However, if you're really interested in something else.. how about this?
>>> if record[0]:
... field = record[0][0]
This works because an empty list ([]) evaluates as False in an if statement.
>>> record = [[]]
>>> if record[0]: # returns an empty list, e.g. []
... field = record[0][0] # is not executed
A simpler alternative:
import MySQLdb
conn = MySQLdb.connect(passwd="sekret",db="foo")
cur = conn.cursor()
cur.execute("select * from b where a=%s limit 1", (a,))
for result in cur:
print(result)
Note the changes:
Use MySQLdb, not the underlying _mysql* API
Don't concatenate variables into SQL query strings, this will lead to SQL injection.
Iterate over the cursor to get the results
In Python, there is a way to get a default value from a dict but not from a list. E.g. in a dict:
x = mydict.get('key') # x will be None if there is no 'key'
(you can also provide a different default as a 2nd arg to get() method)
Now, it would be convenient to have something like that for lists. Getting an item from a list
is in some ways very similar to getting an item from a dict, but not exactly the same. Python
made a design decision to not have a similar method for lists.
Of course, you can make your own very easily. I sometimes use a function in my own library
called 'getitem', which returns a default arg; however it only looks up one level of a list,
because I feel multiple levels is too much of a corner case and it's probably worth using
an exception with multiple levels. But, for one level you can do:
def getitem(seq, index, default=None):
"""Get item from a `seq` at `index`, return default if index out of range."""
try : return seq[index]
except IndexError : return default
Note that there's a gotcha: you can't assume that getting None back means there is no
item, if your list may itself contain None values. Seems obvious but that's something
you have to remember.
You can easily extend this function to accept multiple indexes and handle multiple
levels of lists, but then you may ask: how do I know at which level there was an
IndexError?
I have a sql query as follows
select cloumn1,column2,count(column1) as c
from Table1 where user_id='xxxxx' and timestamp > xxxxxx
group by cloumn1,column2
order by c desc limit 1;
And I successed in write the sqlalchemy equvalent
result = session.query(Table1.field1,Table1.field2,func.count(Table1.field1)).filter(
Table1.user_id == self.user_id).filter(Table1.timestamp > self.from_ts).group_by(
Table1.field1,Travelog.field2).order_by(desc(func.count(Table1.field1))).first()
But I want to avoid using func.count(Table1.field1) in the order_by clause.
How can I use alias in sqlalchemy? Can any one show any example?
Aliases are for tables; columns in a query are given a label instead. This trips me up from time to time too.
You can go about this two ways. It is sufficient to store the func.count() result is a local variable first and reuse that:
field1_count = func.count(Table1.field1)
result = session.query(Table1.field1, Table1.field2, field1_count).filter(
Table1.user_id == self.user_id).filter(Table1.timestamp > self.from_ts).group_by(
Table1.field1, Travelog.field2).order_by(desc(field1_count)).first()
The SQL produced would still be the same as your own code would generate, but at least you don't have to type out the func.count() call twice.
To give this column an explicit label, call the .label() method on it:
field1_count = func.count(Table1.field1).label('c')
and you can then use that same label string in the order_by clause:
result = session.query(Table1.field1, Table1.field2, field1_count).filter(
Table1.user_id == self.user_id).filter(Table1.timestamp > self.from_ts).group_by(
Table1.field1, Travelog.field2).order_by(desc('c')).first()
or you could use the field1_count.name attribute:
result = session.query(Table1.field1, Table1.field2, field1_count).filter(
Table1.user_id == self.user_id).filter(Table1.timestamp > self.from_ts).group_by(
Table1.field1, Travelog.field2).order_by(desc(field1_count.name)).first()
Can also using the c which is an alias of the column attribute but in this case a label will work fine as stated.
Will also point out that the filter doesn't need to be used multiple times can pass comma separated criterion.
result = (session.query(Table1.field1, Table1.field2,
func.count(Table1.field1).label('total'))
.filter(Table1.c.user_id == self.user_id, Table1.timestamp > self.from_ts)
.group_by(Table1.field1,Table1.field2)
.order_by(desc('total')).first())