I wish to create a mapped attribute of an object which is populated from another table.
Using the SQLAlchemy documentation example, I wish to make a user_name field exist on the Address class such that it can be both easily queried and easily accessed (without a second round trip to the database)
For example, I wish to be able to query and filter by user_name Address.query.filter(Address.user_name == 'wcdolphin').first()
And also access the user_name attribute of all Address objects, without performance penalty, and have it properly persist writes as would be expected of an attribute in the __tablename__
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email = Column(String(50))
user_name = Column(Integer, ForeignKey('users.name'))#This line is wrong
How do I do this?
I found the documentation relatively difficult to understand, as it did not seem to conform to most examples, especially the Flask-SQLAlchemy examples.
You can do this with a join on the query object, no need to specify this attribute directly. So your model would look like:
from sqlalchemy import create_engine, Column, Integer, String, ForeignKey
from sqlalchemy.orm import sessionmaker, relation
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
engine = create_engine('sqlite:///')
Session = sessionmaker(bind=engine)
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email = Column(String(50))
user_id = Column(Integer, ForeignKey("users.id"))
Base.metadata.create_all(engine)
A query after addresses with filtering the username looks like:
>>> session = Session()
>>> session.add(Address(user=User(name='test')))
>>> session.query(Address).join(User).filter(User.name == 'test').first()
<__main__.Address object at 0x02DB3730>
Edit: As you can directly access the user from an address object, there is no need for directly referencing an attribute to the Address class:
>>> a = session.query(Address).join(User).filter(User.name == 'test').first()
>>> a.user.name
'test'
If you truly want Address to have a SQL enabled version of "User.name" without the need to join explicitly, you need to use a correlated subquery. This will work in all cases but tends to be inefficient on the database side (particularly with MySQL), so there is possibly a performance penalty on the SQL side versus using a regular JOIN. Running some EXPLAIN tests may help to analyze how much of an effect there may be.
Another example of a correlated column_property() is at http://docs.sqlalchemy.org/en/latest/orm/mapped_sql_expr.html#using-column-property.
For the "set" event, a correlated subquery represents a read-only attribute, but an event can be used to intercept changes and apply them to the parent User row. Two approaches to this are presented below, one using regular identity map mechanics, which will incur a load of the User row if not already present, the other which emits a direct UPDATE to the row:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base= declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('users.id'))
email = Column(String(50))
Address.user_name = column_property(select([User.name]).where(User.id==Address.id))
from sqlalchemy import event
#event.listens_for(Address.user_name, "set")
def _set_address_user_name(target, value, oldvalue, initiator):
# use ORM identity map + flush
target.user.name = value
# use direct UPDATE
#object_session(target).query(User).with_parent(target).update({'name':value})
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
s.add_all([
User(name='u1', addresses=[Address(email='e1'), Address(email='e2')])
])
s.commit()
a1 = s.query(Address).filter(Address.user_name=="u1").first()
assert a1.user_name == "u1"
a1.user_name = 'u2'
s.commit()
a1 = s.query(Address).filter(Address.user_name=="u2").first()
assert a1.user_name == "u2"
Related
I am new in Sqlalchemy and trying to achieve the following goal with relationship():
There is an User table which stores user data.
Every user is able to invite other user with an invite_code.
Every user keeps a list of invitation, every invitation includes the invite_code and the invitee User
I think the relationship between User and Invitation is one-to-many. Since Invitation contains User, then I think it is probably better to use self-referential relationship to represent the inviter-to-invitaions(invitees) relationship and use an association object to store the invite_code.
I checked the sqlalchemy documentation and the question, tried to implement the classed like this:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Invitation(Base):
__tablename__ = 'invitation'
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee = relationship('User') #Need HELP here
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
inviters = relationship('User',
secondary='invitation',
primaryjoin=id==Invitation.invitee_id,
secondaryjoin=id==Invitation.inviter_id,
backref='invitees')
invitations = relationship('Invitation')# Need HELP here
def __repr__(self):
return f'User: {self.name}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
inviter1.invitees = [invitee1, invitee2]
inviter2.invitees = [invitee1]
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' Inviter: ', user.inviters)
print(' Invitee: ', user.invitees)
print()
If the lines with comment #Need HELP here are deleted, I can get the corresponding inviters and invitees, but cannot get the invite_code. If the #Need HELP here code are added, the error is:
Exception has occurred: AmbiguousForeignKeysError
Could not determine join condition between parent/child tables on relationship Invitation.invitee - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
Is there a way to add extra data column in association object like association object for many-to-many relationship for self referential table?
Sorry for the too much text, I didn't find any reference document on the web.
Finally, I figured it out with the help of foreign_keys:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
sent_invitations = relationship('Invitation', foreign_keys='Invitation.inviter_id', back_populates='inviter', cascade='all, delete')
received_invitations=relationship('Invitation', foreign_keys='Invitation.invitee_id', back_populates='invitee', cascade='all, delete')
def __repr__(self):
return f'User: {self.name}'
class Invitation(Base):
__tablename__ = 'invitation'
id = Column(Integer, primary_key=True)
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'))
invitee_id = Column(Integer, ForeignKey('user.id'))
inviter=relationship('User', foreign_keys=[inviter_id], back_populates='sent_invitations')
invitee=relationship('User', foreign_keys=[invitee_id], back_populates='received_invitations')
def __repr__(self):
return f'Invitation: {self.inviter} invited {self.invitee} with {self.invite_code}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
invitation1 = Invitation(invite_code=50, inviter=inviter1, invitee=invitee1)
invitation2 = Invitation(invite_code=20, inviter=inviter2, invitee=invitee2)
invitation3 = Invitation(invite_code=22, inviter=inviter1, invitee=inviter2)
invitation4 = Invitation(invite_code=44, inviter=invitee1, invitee=inviter2)
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' sent_invitation: ', user.sent_invitations)
print(' received_invitation: ', user.received_invitations)
print()
invitations = db.query(Invitation).all()
for invitation in invitations:
print(invitation)
db.delete(inviter1)
db.delete(invitee2)
db.commit()
I have a many to many relationship that has a specific set of characteristics. I thought I could implement this in sqlalchemy with an association table as below:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy import Column, Integer, ForeignKey, Unicode, Enum
import enum
Base = declarative_base()
class Person(Base):
__tablename__ = 'person'
id = Column(Integer, primary_key=True)
name = Column(Unicode)
worksAt = relationship('Address', secondary='parelationship')
manages = relationship('Address', secondary='parelationship')
resides = relationship('Address', secondary='parelationship')
## How do I specify the additional constraint of
## parelationship.relation = Relationships.resident?
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
name = Column(Unicode)
class Relationships(enum.Enum):
resident = 1
worker = 2
manager = 3
class PersonAddressRelationship(Base):
__tablename__ = 'parelationship'
personId = Column(Integer, ForeignKey('person.id'), primary_key=True)
adressID = Column(Integer, ForeignKey('address.id'), primary_key=True)
relation = Column(Enum(Relationships), primary_key=True)
Is there a neat way of specifying the worksAt, manages, resides relationships (and equally worksHere, isManagedBy etc in the Address table)?
Either define the primaryjoin or the secondaryjoin with the additional predicate, or use a derived table as secondary.
Using a derived table:
worksAt = relationship(
'Address',
secondary=lambda:
PersonAddressRelationship.__table__.select().
where(PersonAddressRelationship.relation == Relationships.worker).
alias(),
viewonly=True)
Using primaryjoin:
manages = relationship(
'Address', secondary='parelationship',
primaryjoin=lambda:
and_(Person.id == PersonAddressRelationship.personId,
PersonAddressRelationship.relation == Relationships.manager),
viewonly=True)
Using secondaryjoin:
resides = relationship(
'Address', secondary='parelationship',
secondaryjoin=lambda:
and_(Address.id == PersonAddressRelationship.adressID,
PersonAddressRelationship.relation == Relationships.manager),
viewonly=True)
Note that in all the examples the expression is passed as a callable (a lambda), so that it can be lazily evaluated during mapper configuration.
I am using Eve-SQLAlchemy==0.5.0
I would like to perform a nested query using Postman on my users such that I find all users that are within a specified organization.
Using SQL I would write my query such that:
select * from app_user
left join user_organization on user_organization.user_id = app_user.id
left join organization on organization.id = user_organization.organization_id
where organization.id = 2
I have a user model, an organization model, and a relational model linking the two user_organization.
from sqlalchemy import Column, DateTime, func, String, Integer
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class BaseModel(Base):
id = Column(Integer, primary_key=True, autoincrement=True)
__abstract__ = True
_created = Column(DateTime, default=func.now())
_updated = Column(DateTime, default=func.now(), onupdate=func.now())
_etag = Column(String(40))
class User(BaseModel):
__tablename__ = 'app_user'
organizations = relationship("Organization", secondary=UserOrganization.__tablename__)
class Organization(BaseModel):
__tablename__ = 'organization'
name = Column(String)
class UserOrganization(BaseModel):
__tablename__ = 'user_organization'
user_id = Column(Integer,
ForeignKey('app_user.id', ondelete='CASCADE'))
organization_id = Column(Integer,
ForeignKey('organization.id', ondelete='CASCADE'))
In my settings.py I have the resources registered:
# Resource Registration
DOMAIN = DomainConfig({
'organization': ResourceConfig(Organization),
'user': ResourceConfig(User)
}).render()
I have a series of postman collections setup, and using a GET request I can easily query any attribute... GET localhost:5000/user?where={"id":1}
I have tried (amongst many other things):
GET user?where={"organizations": {"organization_id" :2 }}
GET user?where={"organizations": 2}
It seems it's not possible at the moment due to a bug. I will try to fix it within the next week.
The code in https://github.com/pyeve/eve-sqlalchemy/blob/master/eve_sqlalchemy/parser.py#L73 is causing a GET ?where={"organizations": 2} to result in a SQL expression like user_id = 42 AND organization_id = 42 is generated. Which rarely makes any sense.
I try to test the code example on SQLAlchemy documentation about handling multiple join paths. However after I create a customer object, both relationship attributes are None. I wonder how to properly handle multiple join paths? Do I need to create a relationship in Address class too? When do I need to use back_populates?
Handling Multiple Join Paths
from sqlalchemy import create_engine
from sqlalchemy import Integer, ForeignKey, String, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class Customer(Base):
__tablename__ = 'customer'
id = Column(Integer, primary_key=True)
name = Column(String)
billing_address_id = Column(Integer, ForeignKey("address.id"))
shipping_address_id = Column(Integer, ForeignKey("address.id"))
billing_address = relationship("Address", foreign_keys=[billing_address_id])
shipping_address = relationship("Address", foreign_keys=[shipping_address_id])
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
street = Column(String)
city = Column(String)
state = Column(String)
zip = Column(String)
engine = create_engine('sqlite:///Testing.db')
Base.metadata.create_all(engine)
a1 = Address(street="a street", city="a city", state="A", zip="12345")
a2 = Address(street="b street", city="b city", state="B", zip="1233")
c1 = Customer(name="Jack")
print(c1.billing_address)
The relationships in Address are not required, if you don't need them. back_populates= is for explicitly linking 2 relationships together, or as the docs put it, "establish “bidirectional” behavior between each other".
The reason why your customer and addresses are not linked is that you never link them. Pass the addresses to the customer during construction or set them afterwards:
c1 = Customer(name="Jack", billing_address=a1)
c1.shipping_address = a2
Now when you add c1 to a session and commit, SQLAlchemy will handle inserting a1, a2, and c1 in the correct order so that it can fill in the foreign key attributes of c1. This happens because by default a relationship() has the save-update cascade enabled, which places associated objects to the session as well.
In a Pyramid application I'm working on, I have the following scenario:
class Widget(Base):
__tablename__ = 'widgets'
id = Column(Integer, primary_key=True)
name = Column(String(50))
sidebar = Column(mysql.TINYINT(2))
def __init__(self, name, sidebar):
self.name = name
self.sidebar = sidebar
class Dashboard(Base):
__tablename__ = 'dashboard'
user_id = Column(Integer, ForeignKey('users.id'), primary_key=True)
widget_id = Column(Integer, ForeignKey('widgets.id'), primary_key=True)
delta = Column(mysql.TINYINT)
widget = relationship('Widget')
def __init__(self, user_id, widget_id, delta):
self.user_id = user_id
self.widget_id = widget_id
self.delta = delta
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
login = Column(Unicode(255), unique=True)
password = Column(Unicode(60))
fullname = Column(Unicode(100))
dashboard = relationship('Dashboard', order_by='Dashboard.widget.sidebar, Dashboard.delta')
def __init__(self, login, password, fullname):
self.login = login
self.password = crypt.encode(password)
self.fullname = fullname
So, I want the User 'dashboard' relationship to have the dashboard records for the user but ordered by 'sidebar' (which is a relationship property of Dashboard). Currently I am getting this error:
sqlalchemy.exc.InvalidRequestError: Property 'widget' is not an instance of ColumnProperty (i.e. does not correspond directly to a Column).
Is this ordering possible in a relationship declaration?
Thanks!
With this, try to think what SQL SQLAlchemy should emit when it tries to load User.dashboard. Like SELECT * FROM dashboard JOIN widget ... ORDER BY widget.sidebar ? Or SELECT * FROM dashboard ORDER BY (SELECT sidebar FROM widget... ? ordering the results by a different table is too open-ended of a job for relationship() to decide on it's own. The way this can be done is by providing a column expression in terms of Dashboard that can provide this ordering, when the ORM emits a simple SELECT against dashboard's table, as well as when it refers to it in a not-so-simple SELECT where it might be joining across User, Dashboard tables at once (e.g. eager loading).
We provide custom SQL expressions, particularly those that involve other tables, using column_property(), or alternatively with deferred() when we don't want that expression to be loaded by default (as is likely the case here). Example:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Widget(Base):
__tablename__ = 'widgets'
id = Column(Integer, primary_key=True)
name = Column(String(50))
sidebar = Column(Integer)
class Dashboard(Base):
__tablename__ = 'dashboard'
user_id = Column(Integer, ForeignKey('users.id'), primary_key=True)
widget_id = Column(Integer, ForeignKey('widgets.id'), primary_key=True)
delta = Column(Integer)
widget = relationship('Widget')
widget_sidebar = deferred(select([Widget.sidebar]).where(Widget.id == widget_id))
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
login = Column(Unicode(255), unique=True)
dashboard = relationship('Dashboard', order_by='Dashboard.widget_sidebar, Dashboard.delta')
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
w1, w2 = Widget(name='w1', sidebar=1), Widget(name='w2', sidebar=2)
s.add_all([
User(login='u1', dashboard=[
Dashboard(
delta=1, widget=w1
),
Dashboard(
delta=2, widget=w2
)
]),
])
s.commit()
print s.query(User).first().dashboard
the final SQL emitted by the load of ".dashboard" is:
SELECT dashboard.user_id AS dashboard_user_id, dashboard.widget_id AS dashboard_widget_id, dashboard.delta AS dashboard_delta
FROM dashboard
WHERE ? = dashboard.user_id ORDER BY (SELECT widgets.sidebar
FROM widgets
WHERE widgets.id = dashboard.widget_id), dashboard.delta
Keep in mind that MySQL does a terrible job optimizing for subqueries like the one above. If you need high performance here, you might consider copying the value of "sidebar" into "dashboard", even though that makes consistency more difficult to maintain.