Is it possible to add to a SQLAlchemy relationship using ids rather than objects?
For example, consider two declarative SQLAlchemy classes, Review and Artist, with a relationship between them:
class Review(Base):
artist_id = Column(Integer, ForeignKey('artist.id'))
artist = relationship(Artist, backref=backref('reviews', order_by=id))
# etc.
class Artist(Base):
# etc.
With a list of review ids to add to an artist, I seem to need to look up the artist from the id, then add the artist object to the review, like this:
for review_id in review_ids:
review = session.query(Review).filter(Review.id==review_id).first()
artist.reviews.append(review)
I'm sure it would be more efficient to skip the lookup and just add the ids, but is this possible?
Your best bet is probably to compose an update expression against the backing tables. Otherwise, you can't really modify a Review without actually querying for it (and that's what your doing; you aren't actually modifying the Artist at all).
Assuming Review.id is the primary key, That would roughly be:
conn = session.connection()
conn.execute(Review.__table__
.update()
.values(artist_id=artist_id)
.where(Review.id.in_(review_ids))
)
I think if you want to stick with the pure ORM solution, you'll have to live with the somewhat inefficient querying pattern.
#TokenMacGuy has provided a good alternative. You could integrate that approach by adding add_reviews() to the Artist() class. This would augment the 'standard' orm api, so you could use either depending on the situation.
from sqlalchemy.orm.session import object_session
class Artist(Base):
def add_reviews(self, review_ids):
sess = object_session(self)
if isinstance(review_ids, int): review_ids = [review_ids]
sess.execute(
Review.__table__
.update()
.values(artist_id=self.artist_id)
.where(Review.id.in_(review_ids))
)
sess.refresh(self)
review_ids = [123, 556, 998, 667, 111]
artist.add_reviews(review_ids)
review_id = 333
artist.add_reviews(review_id)
I'm not sure if I understood. I guess you want something like this:
reviews = session.query(Review).filter(Review.id.in_(review_ids)).all()
artist.reviews.extend(reviews)
Related
I am having trouble with the following setup of a sqlalchemy ORM connected to a postgresql db.
class Map(Base):
__tablename__ = "map"
id = Column(BigInteger, Sequence(name="myseq"), primary_key=True)
cmp_1_id = Column(BigInteger, ForeignKey("component.id"))
cmp_2_id = Column(BigInteger, ForeignKey("component.id"))
class Component(Base):
__tablename__ = "component"
id = Column(BigInteger, Sequence(name="myseq"), primary_key=True)
map_1 = relationship("Map", back_populates="cmp_1", foreign_keys=Map.cmp_1_id, uselist=False)
map_2 = relationship("Map", back_populates="cmp_2", foreign_keys=Map.cmp_2_id, uselist=False)
Map.cmp_1 = relationship(
"Component", back_populates="map_1", primaryjoin=Map.cmp_1_id == Component.id
)
Map.cmp_2 = relationship(
"Component", back_populates="map_2", primaryjoin=Map.cmp_2_id == Component.id
)
Now I want to query a specific Map object, whose cmp_1 object has a certain "known_value" of other_attribute. I tried various statements, using Query API and Select API and with a colleague finally found this solution to be working:
(session.query(Map.id)
.join(Map.cmp_1)
.where(Component.other_attribute=="known_value")
.one()[0]
)
During my research on the topic I read through some other SO articles, which raised further questions. So here they come:
My main question: why can't I do the query like this:
(session.query(Alias_Map_Expanded.id)
.where(Map.cmp_1.other_attribute=="known_value")
).one()[0]
This raises the exception AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Map.cmp_1 has an attribute 'other_attribute'
More generally: how would I design the model better (as in: more robust and easier to jump between relations) to possibly be able to do the above? The relationships need to be One (Component) To Many (Map), i.e a Map object points to one to two (cmp_2 is optional) components. In turn a component can be pointed to from multiple Map rows/objects.
Based on this: Should I always define a foreign key along with a relationship to not break the relationship inside the db? Update: I removed this question because I now find it rather misleading and not really worth having it answered.
Based on that: I guess I also need to use the post_update to not have a circular dependency? Or do I misinterpret the use of post_update?
Thanks in advance!
After some thorough consulting of the extensive sqlalchemy docs I found some answers:
To my first question and the related query: in my ORM classes I did not specify the loading type of the data, leaving it at the default type "lazy". Therefore the other_attribute attribute's value is not loaded with the first query but rather it would take a second call to query1_result.other_attribute upon which the related content would be queried separately. Alternatively I'd need to specify an eager loading type for the proposed query to be working.
I just figured, even if I use eager-loaded queries, I still cannot filter related objects, using class-level ORM syntax, because at that point the ORM instance has not yet mapped its relative. The filtering ("where" clause) needs to be formulated on SQL level, i.e like the first example I gave above...
There is most likely no meaningful answer to that, especially without deeper knowledge of my database structure...
Third question, based on link: I think my question is somewhat strange and maybe even misleading. I will remove it from the original post.
Last question, based on 2nd link: I haven't investigated so much more on this question, being it the least important to me, but I think I got the concept of post_update wrong and will not need it for my purpose.
I got all of it from sqlalchemy docs, so in case you hit that question and have a similar problem, work your way through the extensive documentation conglomeration. The answer is most likely there.
I have a concrete base model, from which other models inherit (all models in this question have been trimmed for brevity):
class Order(models.Model):
state = models.ForeignKey('OrderState')
Here are a few examples of the "child" models:
class BorrowOrder(Order):
parts = models.ManyToManyField('Part', through='BorrowOrderPart')
class ReturnOrder(Order):
parts = models.ManyToManyField('Part', through='ReturnOrderPart')
As you can see from these examples, each child model has a many-to-many relationship of Parts through a custom table. Those custom through-tables look something like this:
class BorrowOrderPart(models.Model):
borrow_order = models.ForeignKey('BorrowOrder', related_name='borrowed_parts')
part = models.ForeignKey('Part')
qty_borrowed = models.PositiveIntegerField()
class ReturnOrderPart(models.Model):
return_order = models.ForeignKey('ReturnOrder', related_name='returned_parts')
part = models.ForeignKey('Part')
qty_returned = models.PositiveIntegerField()
Note that the "quantity" field in each through table has a custom name (unfortunately): qty_borrowed or qty_returned. I'd like to be able to query the base table (so that I'm searching across all order types), and include an annotated field for each that sums these quantity fields:
# Not sure what I specify in the Sum() call here, given that the fields
# I'm interested in are different depending on the child's type.
qs = models.Order.objects.annotate(total_qty=Sum(???))
# For a single model, I would do something like:
qs = models.BorrowOrder.objects.annotate(
total_qty=Sum('borrowed_parts__qty_borrowed'))
So I guess I have two related questions:
Can I annotate a child-model's data through a query on the parent model?
If so, can I conditionally specify the field to be annotated, given that the actual field name changes depending on the model in question?
This feels to me like a place where using When() and Case() might be helpful, but I'm not sure how I'd build the necessary logic.
The problem is that, when you are querying the base model (in multi-table inheritance), it's hard to find out which subclass the object actually is. See How to know which is the child class of a model.
The query might be achievable in theory, with something like
SELECT
CASE
WHEN child1.base_ptr_id IS NOT NULL THEN ...
WHEN child2.base_ptr_id IS NOT NULL THEN ...
END AS ...
FROM base
LEFT JOIN child1 ON child1.base_ptr_id = base.id
LEFT JOIN child2 ON child2.base_ptr_id = base.id
...
but I don't know how to translate that in Django and I think it would be too much trouble to do it. It could be done, if not anything else using raw queries.
Another solution would be to add to the base class a field that specifies which actual subclass each object is; in that case, you'd need to make as many queries as there are subclasses and join them. I don't like this solution either. Update: After I slept on this I conclude that the most Django-like solution would be not to query the parent model in the first place; simply query the submodels and join the results. I would explore the third option below only if there were performance or other practical problems.
Another idea is to create a database view (with CREATE VIEW) based on the above SQL query and translate it into a Django model with managed = False, and query that one. Maybe this is somewhat cleaner than the other solutions, but it is a bit non-standard.
I have three models
class A(Model):
...
class B(Model):
id = IntegerField()
a = ForeignKey(A)
class C(Model):
id = IntegerField()
a = ForeignKey(A)
I want get the pairs of (B.id, C.id), for which B.a==C.a. How do I make that join using the django orm?
Django allows you to reverse the lookup in much the same way that you can use do a forward lookup using __:
It works backwards, too. To refer to a “reverse” relationship, just use the lowercase name of the model.
This example retrieves all Blog objects which have at least one Entry whose headline contains 'Lennon':
Blog.objects.filter(entry__headline__contains='Lennon')
I think you can do something like this, with #Daniel Roseman's caveat about the type of result set that you will get back.
ids = B.objects.prefetch_related('a', 'a__c').values_list('id', 'a__c__id')
The prefetch related will help with performance in older versions of django if memory serves.
I'm using an declarative SQLAlchemy class to perform computations. Part of the computations require me to perform the computations for all configurations provided by a different table which doesn't have any foreign key relationships between the two tables.
This analogy is nothing like my real application, but hopefully will help to comprehend what I want to happen.
I have a set of cars and a list of paint colors.
The car object has a factory which provides a car in all possible colors
from sqlalchemy import *
from sqlachemy.orm import *
def PaintACar(car, color):
pass
Base = declarative_base()
class Colors(Base):
__table__ = u'colors'
id = Column('id', Integer)
color= Column('color', Unicode)
class Car(Base):
__table__ = u'car'
id = Column('id', Integer)
model = Column('model', Unicode)
# is this somehow possible?
all_color_objects = collection(...)
# I know this is possible, but would like to know if there's another way
#property
def all_colors(self):
s = Session.object_session(self)
return s.query(A).all()
def CarColorFactory(self):
for color in self.all_color_objects:
yield PaintACar(self, color)
My question: Is it possible to produce all_color_objects somehow? Without having to resort to finding the session and manually issuing a query as in the all_colors property?
It's been a while, so I'm providing the best answer I saw (as a comment by zzzeek). Basically, I was looking for one-off syntactic sugar. My original 'ugly' implementation works just fine.
what better way would there be here besides getting a Session and producing the query you
want? Are you looking for being able to add to the collection and that automatically
flushes things? (just add the objects to the Session?) Do you not like using
object_session(self) >(you can build some mixin class or something that hides that for
you?) It's not really clear >what the problem is. The objects here have no relationship to
the parent class so there's no particular intelligence SQLAlchemy would be able to add.
– zzzeek Jun 17 at 5:03
I'm a beginner in SQLAlchemy and found query can be done in 2 method:
Approach 1:
DBSession = scoped_session(sessionmaker())
class _Base(object):
query = DBSession.query_property()
Base = declarative_base(cls=_Base)
class SomeModel(Base):
key = Column(Unicode, primary_key=True)
value = Column(Unicode)
# When querying
result = SomeModel.query.filter(...)
Approach 2
DBSession = scoped_session(sessionmaker())
Base = declarative_base()
class SomeModel(Base):
key = Column(Unicode, primary_key=True)
value = Column(Unicode)
# When querying
session = DBSession()
result = session.query(SomeModel).filter(...)
Is there any difference between them?
In the code above, there is no difference. This is because, in line 3 of the first example:
the query property is explicitly bound to DBSession
there is no custom Query object passed to query_property
As #petr-viktorin points out in the answer here, there must be a session available before you define your model in the first example, which might be problematic depending on the structure of your application.
If, however, you need a custom query that adds additional query parameters automatically to all queries, then only the first example will allow that. A custom query class that inherits from sqlalchemy.orm.query.Query can be passed as an argument to query_property. This question shows an example of that pattern.
Even if a model object has a custom query property defined on it, that property is not used when querying with session.query, as in the last line in the second example. This means something like the first example the only option if you need a custom query class.
I see these downsides to query_property:
You cannot use it on a different Session than the one you've configured (though you could always use session.query then).
You need a session object available before you define your schema.
These could bite you when you want to write tests, for example.
Also, session.query fits better with how SQLAlchemy works; query_property looks like it's just added on top for convenience (or similarity with other systems?).
I'd recommend you stick to session.query.
An answer (here) to a different SQLAlchemy question might help. That answer starts with:
You can use Model.query, because the Model (or usually its base class, especially in cases where declarative extension is used) is assigned Session.query_property. In this case the Model.query is equivalent to Session.query(Model).