So I have a query...
qry = Query(models.UserList).filter(...)
models.UserList contains a mixin method that will return a count of a related table
So what I would like to do is something like ...
qry = qry.order_by(models.UserList.contactCount)
However this produces invalid sql of course.
So how can I do something like
qry.add_column((this.contactCount).label("contactCount"))
this.contactCount is just bad psuedocode, but hopefully you get the point.
I've been looking at how to do this for a while, but sqlalchemy doc isnt the easiest thing to find stuff in.
any help appreciated
You can use either a column_property or a hybrid_property for that. For example:
class UserList(Base):
...
contactCount = column_property(
select(
[func.count()],
contact_id == Contact.id,
).label('contactCount'),
deferred=True,
)
Related
I have currently worked abit with ORM using Peewee and I have been trying to understand how I am able to get the field url from the table. The condition is that column visible needs to be true as well. Meaning that if visible is True and the store_id is 4 then return all the url as set.
I have currently done something like this
from peewee import (
Model,
TextField,
BooleanField
)
from playhouse.pool import PooledPostgresqlDatabase
# -------------------------------------------------------------------------
# Connection to Postgresql
# -------------------------------------------------------------------------
postgres_pool = PooledPostgresqlDatabase(
'xxxxxxx',
host='xxxxxxxx',
user='xxxxxxxx',
password='xxxxxx',
max_connections=20,
stale_timeout=30,
)
# ------------------------------------------------------------------------------- #
class Products(Model):
store_id = TextField(column_name='store_id')
url = TextField(column_name='url')
visible = BooleanField(column_name='visible')
class Meta:
database = postgres_pool
db_table = "develop"
#classmethod
def get_urls(cls):
try:
return set([i.url for i in cls.select().where((cls.store_id == 4) & (cls.visible))])
except Products.IntegrityError:
return None
However using the method takes around 0.13s which feels abit too long for me than what it supposed to do which I believe is due to the for loop and needing to put it as a set() and I wonder if there is a possibility that peewee can do something like cls.select(cls.url).where((cls.store_id == 4) & (cls.visible) and return as set()?
How many products do you have? How big is this set? Why not use distinct() so that the database de-duplicates them for you? What indexes do you have? All of these questions are much more pertinent than "how do I make this python loop faster".
I'd suggest that you need an index on store_id, visible or store_id where visible.
create index "product_urls" on "products" ("store_id") where "visible"
You could even use a covering index but this may take up a lot of disk space:
create index "product_urls" on "products" ("store_id", "url") where visible
Once you've got the actual query sped up with an index, you can also use distinct() to make the db de-dupe the URLs before sending them to Python. Additionally, since you only need the URL, just select that column and use the tuples() method to avoid creating a class:
#classmethod
def get_urls(cls):
query = cls.select(cls.url).where((cls.store_id == 4) & cls.visible)
return set(url for url, in query.distinct().tuples())
Lastly please read the docs: http://docs.peewee-orm.com/en/latest/peewee/querying.html#iterating-over-large-result-sets
I am using the Peewee library in Python and I want to check if a query exists. I do not want to create a record if it doesn't exist, so I don't want to use get_or_create. There must be a better solution than to use try/except with get but I don't see anything. Please let me know if there is a better way. Thanks.
You can use .exists():
query = User.select().where(User.username == 'charlie')
if query.exists():
# A user named "charlie" exists.
cool()
http://docs.peewee-orm.com/en/latest/peewee/api.html?highlight=exists#SelectBase.exists
If you just need to check existence use the accepted answer.
If you are going to use the record if it exists you can make use of Model.get_or_none() as this removes the need to use a try/catch and will not create a record if the record doesn't exist.
class User(peewee.Model):
username = peewee.CharField(unique=True)
user = User.get_or_none(username='charlie')
if user is not None:
# found user, do something with it
pass
Alternatively, if you want to check if e.g. some other table refers this record, you can use WHERE EXISTS (subquery) clause. It is not supported natively by PeeWee, but it can be easily constructed:
subquery = Child.select(Param('1')).where(Child.parent == Parent.id)
parents_with_children = Parent.select().where(
Clause(SQL('EXISTS'), subquery))
It is equivalent to the following SQL:
SELECT * FROM parent
WHERE EXISTS (SELECT 1 FROM child
WHERE child.parent_id = parent.id);
Here I used SELECT 1 for subquery to avoid fetching unneeded information (like child.id). Not sure if such optimization is actually required.
UPD (Feb 2022)
After more than 5 years of peewee evolution, it looks like the Clause class is gone.
The following code may work (I didn't have a chance to test it though):
subquery = Child.select(Param('1')).where(Child.parent == Parent.id)
parents_with_children = Parent.select().where(
NodeList((SQL('EXISTS'), subquery)))
I want to pass a class to a function, and don't like to pass the name again.
class TableClass(Base):
__table__ = Table('t1', metadata, autoload=True)
def get_name(TableClass):
print TableClass.GetTableName() # print 't1'
get_name(TableClass)
So, I search it with google, and there is no answer.
According To:
How to discover table properties from SQLAlchemy mapped object
I can use this:
print TableClass.__table__.name
Independent on whether you use declarative extension or not, you can use the Runtime Inspection API:
def get_name(TableClass):
from sqlalchemy import inspect
mapper = inspect(TableClass)
print mapper.tables[0].name
Please note that a class can have multiple tables mapped to it, for example when using Inheritance.
print TableClass.__tablename__
works for me
In SQLAlchemy you can fetch table information with tableclass attributes.
In your example
print TableClass.__tablename__ # Prints 't1'
According to #Aylwyn Lake 's Finding
print TableClass.__table__.name
I just hit this issue myself, passing the object around and didn't want to pass the string of the tablename as well..
Turns out, it's just a property of the Table object, eg:
table = new Table("somename",meta)
...
print("My table is called: " + table.name)
None of the answers worked for me.
This did:
For Class Object:
TableClass.sa_class_manager.mapper.mapped_table.name
For Instance Object:
tableObj.sa_instance_state.mapper.mapped_table.name
model.py
class Tdzien(models.Model):
dziens = models.SmallIntegerField(primary_key=True, db_column='DZIENS')
dzienrok = models.SmallIntegerField(unique=True, db_column='ROK')
class Tnogahist(models.Model):
id_noga = models.ForeignKey(Tenerg, primary_key=True, db_column='ID_ENERG')
dziens = models.SmallIntegerField(db_column='DZIENS')
What I want is to get id_noga where dzienrok=1234. I know that dziens should be
dziens = models.ForeignKey(Tdzien)
but it isn't and I can't change that. Normally I would use something like
Tnogahist.objects.filter(dziens__dzienrok=1234)
but I don't know how to join and filter those tables without foreignkey.
No joins without a foreign key as far as I know, but you could use two queries:
Tnogahist.objects.filter(dziens__in=Tdzien.objects.filter(dzienrok=1234))
It's possible to join two tables by performing a raw sql query. But for this case it's quite nasty, so I recommend you to rewrite your models.py.
You can check how to do this here
It would be something like this:
from django.db import connection
def my_custom_sql(self):
cursor = connection.cursor()
cursor.execute("select id_noga
from myapp_Tnogahist a
inner join myapp_Tdzien b on a.dziens=b.dziens
where b.dzienrok = 1234")
row = cursor.fetchone()
return row
Could you do this with .extra? From https://docs.djangoproject.com/en/dev/ref/models/querysets/#extra:
where / tables
You can define explicit SQL WHERE clauses — perhaps to perform
non-explicit joins — by using where. You can manually add tables to
the SQL FROM clause by using tables.
To provide a little more context around #paul-tomblin's answer,
It's worth mentioning that for the vast majority of django users; the best course of action is to implement a conventional foreign key. Django strongly recommends avoiding the use of extra() saying "use this method as a last resort". However, extra() is still preferable to raw queries using Manager.raw() or executing custom SQL directly using django.db.connection
Here's an example of how you would achieve this using django's .extra() method:
Tnogahist.objects.extra(
tables = ['myapp_tdzien'],
where = [
'myapp_tnogahist.dziens=myapp_tdzien.dziens',
'myapp_tdzien.dzienrok=%s',
],
params = [1234],
)
The primary appeal for using extra() over other approaches is that it plays nicely with the rest of django's queryset stack, like filter, exclude, defer, values, and slicing. So you can probably plug it in alongside traditional django query logic. For example: Tnogahist.objects.filter(...).extra(...).values('id_noga')[:10]
Let's say I have a class like this:
class Foo(declarativeBase):
bars1 = relationship(Bar.Bar, secondary=foos_to_bars1, collection_class=set())
bars2 = relationship(Bar.Bar, secondary=foos_to_bars2, collection_class=list())
(Each of the relationships gives me "Bar"s with a certain conditions). At a certain point, I want to get instances of "Foo"s that have a "bar" (instance of Bar.Bar) in any of the relationships.
If I try to do:
def inAnyBar(bar)
query(Foo).filter(or_(Foo.bars1.contains(bar), Foo.bars2.contains(bar)).all()
I get an empty result.
It looks (to me) like I'm doing something like:
query(Foo).join(Foo.bars1).filter(Foo.bars1.contains(bar)).\
join(Foo.bars2).filter(Foo.bars1.contains(bar))
Since Foo.bars1 doesn't contain bar, the second filter gives empty results.
I've been able to find a workaround with subqueries (each join+filter in a subquery, then or_ all the subqueries) but I'd like to know if there's a better way to do it...
I found this:
http://techspot.zzzeek.org/2008/09/09/selecting-booleans/
That does what I want to do, but it's for SqlAlchemy 0.5 and I'm (almost) certain that there's a "cleaner" way to do it with SqlAlchemy 0.6.6
Thank you!
You are right, session.query(Foo).filter(Foo.bars1.contains(bar)|Foo.bars2.contains(bar)) produces the following SQL:
SELECT "Foo".id AS "Foo_id"
FROM "Foo", foos_to_bars1 AS foos_to_bars1_1, foos_to_bars2 AS foos_to_bars2_1
WHERE "Foo".id = foos_to_bars1_1.foo AND ? = foos_to_bars1_1.bar OR
"Foo".id = foos_to_bars2_1.foo AND ? = foos_to_bars2_1.bar
which returns incorrect result when one of the secondary tables is empty. Seems like a bug in SQLAlchemy. However replacing contains() with any() fixed the problem (it uses EXISTS subqueries):
session.query(Foo).filter(Foo.bars1.any(id=bar.id)|Foo.bars2.any(id=bar.id))
Also you can specify OUTER JOIN explicitly:
Bar1 = aliased(Bar)
Bar2 = aliased(Bar)
session.query(Foo).outerjoin((Bar1, Foo.bars1)).outerjoin((Bar2, Foo.bars2))\
.filter((Bar1.id==bar.id)|(Bar2.id==bar.id))