We Keep Coding

How to union two counts queries in SQLAlchemy?

I have two queries and the only difference between then is that one is counting the success status and the other failure status. Is there a way to get this result in just one query? I'm using SQLALchemy to do the queries.
success_status_query = (
db_session.query(Playbook.operator, func.count(Playbook.operator).label("success"))
.filter(Playbook.opname != "failed")
.join(AccountInfo, AccountInfo.hardware_id == Playbook.hardware_id)
.group_by(Playbook.operator)
)
failure_status_query = (
db_session.query(Playbook.operator, func.count(Playbook.operator).label("failure"))
.filter(Playbook.opname == "failed")
.join(AccountInfo, AccountInfo.hardware_id == Playbook.hardware_id)
.group_by(Playbook.operator)
)

You can use conditions on Count, your query will look like:
query = (
db_session.query(
Playbook.operator,
func.count(
case(
[((Playbook.opname != "failed"), Playbook.operator)],
else_=literal_column("NULL"),
)
).label("success"),
func.count(
case(
[((Playbook.opname == "failed"), Playbook.operator)],
else_=literal_column("NULL"),
)
).label("failure"),
)
.join(AccountInfo, AccountInfo.hardware_id == Playbook.hardware_id)
.group_by(Playbook.operator)
)

you can use the or_() operator like the sample bellow:
from sqlalchemy import or_
stmt = select(users_table).where(
or_(
users_table.c.name == 'wendy',
users_table.c.name == 'jack'
)
)
It won't be a straight swap, but you will be able to work it out.
You can find more information in the SQLAlchemy documentation:
https://docs.sqlalchemy.org/en/14/core/sqlelement.html#sqlalchemy.sql.expression.or_

Erratic sort order for hybrid_property sum expression in Flask-Admin using SQLAlchemy

Given the following models, I would like to add a sortable column to Flask Admin that shows all upcoming payments for an Organisation:
organisation_payments = db.Table(
'organisation_payments',
db.Column('payment_id', MyUUID(), db.ForeignKey('payment.id')),
db.Column('organisation_id', MyUUID(), db.ForeignKey('organisation.id'))
)
class Organisation(db.Model):
id = Column(ModelUUID(), primary_key=True)
#hybrid_property
def upcoming_payments(self):
payments = Payment.query.filter(
Payment.organisations.any(id=self.id),
Payment.status == 'active'
).all()
return sum([payment.amount for payment in payments])
#upcoming_payments.expression
def upcoming_payments(cls):
return select([
func.sum(Payment.amount)
]).where(and_(
organisation_payments.c.organisation_id == cls.id,
Payment.status == 'active'
)).label('upcoming_payments')
class Payment(db.Model):
id = Column(ModelUUID(), primary_key=True)
amount = db.Column(db.Integer())
status = db.Column(db.Unicode(255), default='active')
organisations = db.relationship(
'Organisation',
secondary=organisation_payments,
backref=db.backref('payments', lazy='dynamic')
)
Note that a Payment could theoretically be mapped to multiple Organisations, hence the many-to-many relationship.
I have added upcoming_payments to column_sortable_list in a Flask-Admin ModelView, but when sorting on that column the results are erratic: https://i.stack.imgur.com/VxrAE.png
This is the most minimal version of the code, but I've also tried:
using coalesce to force 0 for rows with no upcoming payments
using as_scalar() in place of .label()
Payment.organisations.any(id=cls.id) in place of organisation_payments.c.organisation_id == cls.id (this produced even more confusing and inconsistent results, and sorting asc/desc made no difference)
The values returned from the regular hybrid_property are correct, as is the result if I run this in the shell:
stmt = select([
func.sum(Payment.amount)
]).where(and_(
Payment.organisations.any(id=org.id),
Payment.status == 'active',
))
res = db.engine.execute(stmt)
res.fetchone()
(4036200L,)
However the result is wildly inaccurate if I run this:
stmt = select([
func.sum(Payment.amount)
]).where(and_(
organisation_payments.c.organisation_id == org.id,
Payment.status == 'active',
))
res = db.engine.execute(stmt)
res.fetchone()
(1204440000L,)
But neither the any() or association table method returns the correct sort order in Flask-Admin. What am I doing wrong here? I feel like I must be missing a distinct or a subquery, but I'm stumped.
UPDATE: All thanks to Ilja Everilä for guiding me to the answer:
#upcoming_payments.expression
def upcoming_payments(cls):
q = select([coalesce(func.sum(Payment.amount), 0)])
q = q.where(and_(
organisation_payments.c.charity_id == cls.id,
Payment.status == 'active',
Payment.deleted.isnot(True)
))
j = Payment.__table__.join(organisation_payments)
q = q.select_from(j)
return q.label('upcoming_donations')

UPDATE: All thanks to Ilja Everilä for guiding me to the answer:
#upcoming_payments.expression
def upcoming_payments(cls):
q = select([coalesce(func.sum(Payment.amount), 0)])
q = q.where(and_(
organisation_payments.c.charity_id == cls.id,
Payment.status == 'active',
Payment.deleted.isnot(True)
))
j = Payment.__table__.join(organisation_payments)
q = q.select_from(j)
return q.label('upcoming_donations')

How to build a sqlalchemy query that has a filter with the #> operator?

I have this flask/sqlalchemy query that works below:
paged_data = House.query\
.with_entities(
House.name,
db.func.earth_distance(
db.func.ll_to_earth(current_latitude, current_longitude),
House.earth_location
).label('house_distance_from_current_location')
)\
.filter(db.text('earth_box(ll_to_earth(:latitude, :longitude), :radius)'
' #> ll_to_earth(houses.latitude, houses.longitude)'
)
)\
.params(latitude=latitude, longitude=longitude, radius=radius)\
.all()
but I prefer to clean it up to look more like the code below...obviously it won't work quite like that...but I can't figure out how write it...in particular, the use of the #> operator which is not valid Python the way a > operator is.
paged_data = House.query\
.with_entities(
House.name,
db.func.earth_distance(
db.func.ll_to_earth(current_latitude, current_longitude),
House.earth_location
).label('house_distance_from_current_location')
)\
.filter(
db.func.earth_box(db.func.ll_to_earth(current_latitude, current_longitude), radius) #> House.earth_location
)\
.params(latitude=latitude, longitude=longitude, radius=radius)\
.all()
Is there a way to do this, or do I just need to settle with using text()?
For reference here is the House model (it uses a create_materialized_view() function which works fine):
class House(db.Model):
__table__ = create_materialized_view(
'houses',
db.select([
House.id.label('id'),
House.name.label('name'),
House.latitude.label('latitude'),
House.longitude.label('longitude'),
db.func.ll_to_earth(House.latitude, House.longitude).label('earth_location'),
]).where(
db.and_(
House.reviewed == True,
House.deleted == False
)
)
)

You need op method
.filter(
db.func.earth_box(db.func.ll_to_earth(current_latitude, current_longitude), radius)
.op('#>')(House.earth_location)
)

SQLAlchemy search using varying keywords

Im struggling at a very specific problem here:
I need to make a LIKE search in SQLAlchemy, but the amount of keywords are varying.
Heres the code for one keyword:
search_query = request.form["searchinput"]
if selectet_wg and not lagernd:
query = db_session.query(
Artikel.Artnr,
Artikel.Benennung,
Artikel.Bestand,
Artikel.Vkpreisbr1
).filter(
and_(
Artikel.Benennung.like("%"+search_query+"%"),
Artikel.Wg == selectet_wg
)
).order_by(Artikel.Vkpreisbr1.asc())
"searchinput" looks like this : "property1,property2,property3", but also can be just 1,2,5 or more propertys.
I want to split the searchinput at "," (yes i know how to do that :) ) and insert another LIKE search for every property.
So for the above example the search should be looking like this:
search_query = request.form["searchinput"]
if selectet_wg and not lagernd:
query = db_session.query(
Artikel.Artnr,
Artikel.Benennung,
Artikel.Bestand,
Artikel.Vkpreisbr1
).filter(
and_(
Artikel.Benennung.like("%"+search_query+"%"), #property1
Artikel.Benennung.like("%"+search_query+"%"), #property2
Artikel.Benennung.like("%"+search_query+"%"), #property3
Artikel.Wg == selectet_wg
)
).order_by(Artikel.Vkpreisbr1.asc())
I dont think its a smart idea just to make an if statement for the amount of propertys and write down the query serveral times...
Im using the newest version of sqlalchemy and python 3.4

It should be possible to create a list of you like filters and pass them all to and_.
First create a list of the like queries:
search_queries = search_query.split(',')
filter = [Artikel.Benennung.like("%"+sq"%") for sq in search_queries]
Then pass them to and_, unpacking the list:
and_(
Artikel.Wg == selectet_wg,
*filter
)
*filter has to be the last argument to and_ otherwise python will give you an error.

You can call filter multiple times:
search_query = request.form["searchinput"]
if selectet_wg and not lagernd:
query = db_session.query(
Artikel.Artnr,
Artikel.Benennung,
Artikel.Bestand,
Artikel.Vkpreisbr1
).filter(Artikel.Wg == selectet_wg)
for prop in search_query.split(','):
query = query.filter(Artikel.Benennung.like("%"+prop+"%"))
query = query.order_by(Artikel.Vkpreisbr1.asc())

sqlalchemy add entity from a subquery

I want to use outerjoin operation on a subquery and also include values from the subquery also.
My code
q_responses = session.query(Candidate, CandidateProfile)
.join(CandidateProfile, CandidateProfile.candidate_id == Candidate.id)
subq = (session.query(AppAction.candidate_id, Activity.archived)\
.join(Activity, and_(AppAction.candidate_id == Activity.candidate_id,
Activity.archived == 1)))\
.subquery("subq")
responses = q_responses.outerjoin(subq, Candidate.id == subq.c.candidate_id).all()
So I get the result in this format
(Candidate, CandidateProfile)
But I also want to include the archived value from subquery in the result.
By reading many relevant posts from the internet, I have tried
add_entity(subq.c.archived)
with_entities
add_column
select_from
But all those have resulted in some error.
Please help me out.

Please share your error for when you try add_column. The code below should work just fine (assuming that it does work without like which contains add_column):
responses = (
q_responses
.add_column(subq.c.archived) # #new
.outerjoin(subq, Candidate.id == subq.c.candidate_id)
).all()
Also you could have created a query straight away with this column included:
subq = (
session.query(AppAction.candidate_id, Activity.archived)
.join(Activity, and_(AppAction.candidate_id == Activity.candidate_id,
Activity.archived == 1))
).subquery("subq")
q_responses = (
session.query(Candidate, CandidateProfile, subq.c.archived)
.join(CandidateProfile, CandidateProfile.candidate_id == Candidate.id)
.outerjoin(subq, Candidate.id == subq.c.candidate_id)
).all()