We Keep Coding

SQLAlchemy listen for attribute change in many-to-many relationship and change other attributes of initiator

I'm new to the SQLAlchemy and I wrote a simple CRUD database layout like this: I have three tables, Customer, Product, and Template. The idea is this: Each customer can have a template of the products he usually orders. When I fetch a particular customer from the database, his template along with all the products should be fetched as well. So I have a one-to-one relationship (customer-template) and one many-to-many relationship (template-product). Now, a template should contain fields such as quantity of a particular product, along with its net, gross and tax values. I'd like to have a listener on the quantity column, such that when the quantity of a particular product is changed, other attributes I mentioned will be changed too. So the code I wrote is as follows (please, if you can, also verify whether all the relationships are written appropriately)
from sqlalchemy import event
from sqlalchemy.orm import relationship, exc, column_property, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm.query import Query as _Query
from sqlalchemy import Table, Column, Integer, String, Boolean, ForeignKey, UniqueConstraint, create_engine, Numeric
from decimal import *
# decimal operations settings
getcontext().prec = 6
getcontext().rounding = ROUND_HALF_UP
engine = create_engine('sqlite:///test.db')
Base = declarative_base()
# Initalize the database if it is not already.
Base.metadata.create_all(engine)
# Create a session to handle updates.
Session = sessionmaker(bind=engine)
# customer has a one-to-one relationship with template, where customer is the parent
class Customer(Base):
__tablename__ = "customers"
id = Column(Integer, primary_key=True)
alias = Column(String)
firm_name = Column(String)
last_name = Column(String)
first_name = Column(String)
tax_id = Column(String, nullable=False)
address = Column(String)
postal_code = Column(String)
city = Column(String)
payment = Column(Boolean)
template = relationship("Template", uselist=False, back_populates="customer")
# product has many-to-many relationship with template, where template is the parent
class Product(Base):
__tablename__ = "products"
id = Column(Integer, primary_key=True)
product_name = Column(String, nullable=False)
symbol = Column(String)
unit = Column(String, nullable=False)
unit_net_price = Column(Numeric, nullable=False)
vat_rate = Column(Numeric, nullable=False)
per_month = Column(Boolean, nullable=False)
# association table for the products-template many-to-many relationship
association_table = Table('association', Base.metadata,
Column('product_id', Integer, ForeignKey('product.id')),
Column('template_id', Integer, ForeignKey('template.id'))
)
# template implements one-to-one relationship with customer and many-to-many relationship with product
class Template(Base):
id = Column(Integer, primary_key=True)
customer_id = Column(Integer, ForeignKey("customer.id"))
customer = relationship("Customer", back_populates="template")
products = relationship("Product", secondary=association_table)
quantity = Column(Numeric)
net_val = Column(Numeric)
tax_val = Column(Numeric)
gross_val = Column(Numeric)
# TODO: implement constructor
def __init__(self, **kwargs):
self.quantity = Decimal(0.0)
self.net_val = Decimal(0.0)
self.tax_val = Decimal(0.0)
self.gross_val = Decimal(0.0)
#event.listens_for(Template.quantity, "set")
def quantity_listener(target, value, oldvalue, initiator):
print(target)
print(initiator)
# target.net_val =
# target.tax_val =
# target.gross_val =
Now, I'm unsure how should I get a particular initiator and set its values, since products in Template table is a list (?)

I'd probably do it like this. It's hard to use Sqlalchemy orm in such cases as you can't access the Session object we all got used to use.
#event.listens_for(Template, "after_update")
def quantity_listener(mapper, connection, target):
field = 'quantity'
added, _, deleted = get_history(target, field)
# added is a new values of the specified field
# deleted is the old values
# so we basically check if 'quantity' has changed
# its a tuple btw
if added and deleted and added[0] != deleted[0]:
stmt = Template.update(). \
values(Template.something=Template.something * Template.other_something). \
where(Template.id == target.id)
conn.execute(stmt)
# here goes another updates for products or you can have another
# listener for them
There might be a better way to accomplish this. I can't debug this right now and I can't manage to get your examples working. Hope it helps.

Additional primary key constraints on association object when specifying relationship in sqlalchemy

I have a many to many relationship that has a specific set of characteristics. I thought I could implement this in sqlalchemy with an association table as below:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy import Column, Integer, ForeignKey, Unicode, Enum
import enum
Base = declarative_base()
class Person(Base):
__tablename__ = 'person'
id = Column(Integer, primary_key=True)
name = Column(Unicode)
worksAt = relationship('Address', secondary='parelationship')
manages = relationship('Address', secondary='parelationship')
resides = relationship('Address', secondary='parelationship')
## How do I specify the additional constraint of
## parelationship.relation = Relationships.resident?
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
name = Column(Unicode)
class Relationships(enum.Enum):
resident = 1
worker = 2
manager = 3
class PersonAddressRelationship(Base):
__tablename__ = 'parelationship'
personId = Column(Integer, ForeignKey('person.id'), primary_key=True)
adressID = Column(Integer, ForeignKey('address.id'), primary_key=True)
relation = Column(Enum(Relationships), primary_key=True)
Is there a neat way of specifying the worksAt, manages, resides relationships (and equally worksHere, isManagedBy etc in the Address table)?

Either define the primaryjoin or the secondaryjoin with the additional predicate, or use a derived table as secondary.
Using a derived table:
worksAt = relationship(
'Address',
secondary=lambda:
PersonAddressRelationship.__table__.select().
where(PersonAddressRelationship.relation == Relationships.worker).
alias(),
viewonly=True)
Using primaryjoin:
manages = relationship(
'Address', secondary='parelationship',
primaryjoin=lambda:
and_(Person.id == PersonAddressRelationship.personId,
PersonAddressRelationship.relation == Relationships.manager),
viewonly=True)
Using secondaryjoin:
resides = relationship(
'Address', secondary='parelationship',
secondaryjoin=lambda:
and_(Address.id == PersonAddressRelationship.adressID,
PersonAddressRelationship.relation == Relationships.manager),
viewonly=True)
Note that in all the examples the expression is passed as a callable (a lambda), so that it can be lazily evaluated during mapper configuration.

How do I query resources in the nested collection in eve-sqlalchemy?

I am using Eve-SQLAlchemy==0.5.0
I would like to perform a nested query using Postman on my users such that I find all users that are within a specified organization.
Using SQL I would write my query such that:
select * from app_user
left join user_organization on user_organization.user_id = app_user.id
left join organization on organization.id = user_organization.organization_id
where organization.id = 2
I have a user model, an organization model, and a relational model linking the two user_organization.
from sqlalchemy import Column, DateTime, func, String, Integer
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class BaseModel(Base):
id = Column(Integer, primary_key=True, autoincrement=True)
__abstract__ = True
_created = Column(DateTime, default=func.now())
_updated = Column(DateTime, default=func.now(), onupdate=func.now())
_etag = Column(String(40))
class User(BaseModel):
__tablename__ = 'app_user'
organizations = relationship("Organization", secondary=UserOrganization.__tablename__)
class Organization(BaseModel):
__tablename__ = 'organization'
name = Column(String)
class UserOrganization(BaseModel):
__tablename__ = 'user_organization'
user_id = Column(Integer,
ForeignKey('app_user.id', ondelete='CASCADE'))
organization_id = Column(Integer,
ForeignKey('organization.id', ondelete='CASCADE'))
In my settings.py I have the resources registered:
# Resource Registration
DOMAIN = DomainConfig({
'organization': ResourceConfig(Organization),
'user': ResourceConfig(User)
}).render()
I have a series of postman collections setup, and using a GET request I can easily query any attribute... GET localhost:5000/user?where={"id":1}
I have tried (amongst many other things):
GET user?where={"organizations": {"organization_id" :2 }}
GET user?where={"organizations": 2}

It seems it's not possible at the moment due to a bug. I will try to fix it within the next week.
The code in https://github.com/pyeve/eve-sqlalchemy/blob/master/eve_sqlalchemy/parser.py#L73 is causing a GET ?where={"organizations": 2} to result in a SQL expression like user_id = 42 AND organization_id = 42 is generated. Which rarely makes any sense.

flask-sqlalchemy multiple relationship types between two tables

I am having trouble setting up multiple relationships between two models. These are my two models as I have them now:
class Product(db.Model):
tablename='product'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(50))
image_id = db.Column(db.Integer, db.ForeignKey('image.id'))
image = db.relationship('Image',uselist=False,backref=db.backref('product'))
class Image(db.Model):
__tablename__='address'
id = db.Column(db.Integer, primary_key=True)
normal = db.Column(db.String(200))
product_id = db.Column(db.Integer, db.ForeignKey('product.id'))
product = db.relationship('Product', backref='product_images')
Product should have a one-to-one with a cover image, and a one to many with a gallery of other images. However, there is a circular dependency with the foreign keys.
I would like to do this in only two tables. Is there another way to implement these two relationships?
At this point code above throws:
sqlalchemy.exc.AmbiguousForeignKeysError

There are two circular dependencies here:
The foreign keys are mutually dependent on the existence of each table. One of the fks must be created after the dependent table already exists. Set use_alter=True and name='some_name on one to resolve this.
The relationships both need to resolve the primary_key of their target after insert, but are mutually dependent on both having already been commited. Set post_update=True on one to resolve this.
See the following documentation:
CircularDependencyError
Dependent foreign keys
Dependent relationships
Here is a working example demonstrating the solution.
from sqlalchemy import create_engine, Column, Integer, String, ForeignKey, Table
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, relationship
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
session = Session()
Base = declarative_base(bind=engine)
class Product(Base):
__tablename__ = 'product'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False)
# cover image foreign key
# use_alter=True along with name='' adds this foreign key after Image has been created to avoid circular dependency
cover_id = Column(Integer, ForeignKey('image.id', use_alter=True, name='fk_product_cover_id'))
# cover image one-to-one relationship
# set post_update=True to avoid circular dependency during
cover = relationship('Image', foreign_keys=cover_id, post_update=True)
class Image(Base):
__tablename__ = 'image'
id = Column(Integer, primary_key=True)
path = Column(String, nullable=False)
product_id = Column(Integer, ForeignKey(Product.id))
# product gallery many-to-one
product = relationship(Product, foreign_keys=product_id, backref='images')
# nothing special was need in Image, all circular dependencies were solved in Product
Base.metadata.create_all()
# create some images
i1 = Image(path='img1')
i2 = Image(path='img2')
i3 = Image(path='img3')
i4 = Image(path='img4')
# create a product with those images, one of which will also be the cover
p1 = Product(name='sample', images=[i1, i2, i3, i4], cover=i2)
session.add(p1)
session.commit()
print 'cover:', p1.cover.path # prints one cover image path
print 'images:', [i.path for i in p1.images] # prints 4 gallery image paths
print 'image product:', p1.images[0].product.name # prints product name from image perspective

SQLAlchemy declarative property from join (single attribute, not whole object)

I wish to create a mapped attribute of an object which is populated from another table.
Using the SQLAlchemy documentation example, I wish to make a user_name field exist on the Address class such that it can be both easily queried and easily accessed (without a second round trip to the database)
For example, I wish to be able to query and filter by user_name Address.query.filter(Address.user_name == 'wcdolphin').first()
And also access the user_name attribute of all Address objects, without performance penalty, and have it properly persist writes as would be expected of an attribute in the __tablename__
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email = Column(String(50))
user_name = Column(Integer, ForeignKey('users.name'))#This line is wrong
How do I do this?
I found the documentation relatively difficult to understand, as it did not seem to conform to most examples, especially the Flask-SQLAlchemy examples.

You can do this with a join on the query object, no need to specify this attribute directly. So your model would look like:
from sqlalchemy import create_engine, Column, Integer, String, ForeignKey
from sqlalchemy.orm import sessionmaker, relation
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
engine = create_engine('sqlite:///')
Session = sessionmaker(bind=engine)
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email = Column(String(50))
user_id = Column(Integer, ForeignKey("users.id"))
Base.metadata.create_all(engine)
A query after addresses with filtering the username looks like:
>>> session = Session()
>>> session.add(Address(user=User(name='test')))
>>> session.query(Address).join(User).filter(User.name == 'test').first()
<__main__.Address object at 0x02DB3730>
Edit: As you can directly access the user from an address object, there is no need for directly referencing an attribute to the Address class:
>>> a = session.query(Address).join(User).filter(User.name == 'test').first()
>>> a.user.name
'test'

If you truly want Address to have a SQL enabled version of "User.name" without the need to join explicitly, you need to use a correlated subquery. This will work in all cases but tends to be inefficient on the database side (particularly with MySQL), so there is possibly a performance penalty on the SQL side versus using a regular JOIN. Running some EXPLAIN tests may help to analyze how much of an effect there may be.
Another example of a correlated column_property() is at http://docs.sqlalchemy.org/en/latest/orm/mapped_sql_expr.html#using-column-property.
For the "set" event, a correlated subquery represents a read-only attribute, but an event can be used to intercept changes and apply them to the parent User row. Two approaches to this are presented below, one using regular identity map mechanics, which will incur a load of the User row if not already present, the other which emits a direct UPDATE to the row:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base= declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('users.id'))
email = Column(String(50))
Address.user_name = column_property(select([User.name]).where(User.id==Address.id))
from sqlalchemy import event
#event.listens_for(Address.user_name, "set")
def _set_address_user_name(target, value, oldvalue, initiator):
# use ORM identity map + flush
target.user.name = value
# use direct UPDATE
#object_session(target).query(User).with_parent(target).update({'name':value})
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
s.add_all([
User(name='u1', addresses=[Address(email='e1'), Address(email='e2')])
])
s.commit()
a1 = s.query(Address).filter(Address.user_name=="u1").first()
assert a1.user_name == "u1"
a1.user_name = 'u2'
s.commit()
a1 = s.query(Address).filter(Address.user_name=="u2").first()
assert a1.user_name == "u2"