Goal: Create an SQLAlchemy attribute which tracks/follows changes in another object's SQLAlchemy attribute.
Given:
class ClazzA():
attributeA = Column(JSONDict)
class ClazzB():
attributeB = Column(?)
objectA = ClazzA()
objectA.attributeA = {'foo': 1}
objectB = ClazzB()
objectB.attributeB = objectA.attributeA
objectA.attributeA['foo'] = 2
JSONDict is associated with MutableDict as described here: http://docs.sqlalchemy.org/en/latest/orm/extensions/mutable.html#module-sqlalchemy.ext.mutable , i.e. the JSONDict type allows for mutation tracking.
So we have this dictionary on objectA whose changes are being recorded by SQLAlchemy. I would like for attributeB to track attributeA such that even if the application is restarted (i.e. the attributes are reloaded from the DB), then attributeB will continue to reflect changes made to attributeA's dictionary.
Of course, this is closely related to the fact that Python doesn't have an idea of pointers. I was wondering if SQLAlchemy has a solution for this particular problem.
TL;DR
You want a one-to-many relationship.
from sqlalchemy import ForeignKey, Integer, Column
from sqlalchemy.orm import relationship
class Widget(Base):
__tablename__ = 'widget'
widget_id = Column(Integer, primary_key=True)
# name columns, type columns, ...
json = Column(JSONDict)
class ClazzB(Base):
__tablename__ = 'clazzb'
clazzb_id = Column(Integer, primary_key=True)
# Your "attributeB"
widget_id = Column(Integer,
ForeignKey('widget.widget_id',
onupdate='cascade',
ondelete='cascade'),
nullable=False)
widget = relationship('Widget')
# possible association_proxy
#widget_json = association_proxy('widget', 'json')
Using a Relationship
Define a relationship between models ClazzA and ClazzB. Now since we don't have the whole picture, the below definitions are just examples.
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
class ClazzA(Base): # replace Base with the base class of your models
__tablename__ = 'clazza' # replace with the real tablename
# T is the type of your primary key, the column name is just an example
clazza_id = Column(T, primary_key=True)
class ClazzB(Base):
# The column that will relate this model to ClazzA
clazza_id = Column(T, ForeignKey('clazza.clazza_id',
onupdate='cascade',
ondelete='cascade'),
nullable=False)
# A handy accessor for relationship between mapped classes,
# not strictly required. Configurable to be either very lazy
# (loaded if accessed by issuing a SELECT) or eager (JOINed
# when loading objectB for example)
objectA = relationship('ClazzA')
Now instead of adding a reference to attributeA of ClazzA to ClazzB add a reference to related objectA to objectB on initialization.
objectB = ClazzB(..., objectA=objectA)
The two are now related and to access attributeA of related objectA through objectB do
objectB.objectA.attributeA
No need to track changes to attributeA, since it is the attributeA of the instance.
Now if you must have an attribute attributeB on ClazzB (to avoid refactoring existing code or some such), you could add a property
class ClazzB:
#property
def attributeB(self):
return self.objectA.attributeA
which will return the attributeA of the related objectA with
objectB.attributeB
objectB.attributeB['something'] = 'else'
and so on.
There is also an SQLAlchemy method for accessing attributes across relationships: association proxy. It supports simple querying, but is not for example subscriptable.
class ClazzB(Base):
attributeB = association_proxy('objectA', 'attributeA')
If you wish for ClazzB.attributeB to access values from the JSONDict under certain key, you can for example use something like this
class ClazzB(Base):
key = Column(Unicode)
#property
def attributeB(self):
return self.objectA.attributeA[self.key]
You can also make attributeB work as an SQL expression on class level using hybrid properties, if you need such a thing. You would have to write your class level expressions yourself though.
Related
When I have two objects, linked with a "relation" in SQLAlchemy, I realised that simply assigning to that relation is not enough to propagate the values to the other object. For example (see below), if I have a "user" table and a "contact" table (both are highly contrived, but demonstrate the issue well), and a "user" can have multiple "contacts". In that case I will have a foreign key between the users and contacts. If I create both an instance of User and Contact and later assign the user to the contact, I would expect the attributes of the FK to be updated (even without a DB flush) but they are not. Why? And how can I tell SA to do this automatically?
This would be something I would expect to work, but as you can see in the full example below, it does not:
user = User(name='a', lname='b')
contact(type='email', value='foo#bar.com')
contact.user = user
assert contact.username == 'a' # <-- Fails because the attribute is still `None`
Full runnable example:
"""
This code example shows two tables related to each other by a composite key,
using an SQLAlchemy "relation" to allow easy access to related items.
However, as the last few lines show, simply assigning an object A to the
relation of object B does not update the attributes of object B until at least
a "flush" is called.
"""
from sqlalchemy import Column, ForeignKeyConstraint, Unicode, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relation, sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = "user"
name = Column(Unicode, primary_key=True)
lname = Column(Unicode, primary_key=True)
class Contact(Base):
__tablename__ = "contact"
__table_args__ = (
ForeignKeyConstraint(
['username', 'userlname'],
['user.name', 'user.lname']
),
)
username = Column(Unicode, primary_key=True)
userlname = Column(Unicode, primary_key=True)
type = Column(Unicode)
value = Column(Unicode)
user = relation(User)
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
user = User(name="John", lname="Doe")
contact = Contact(type='email', value='john.doe#example.com')
contact.user = user # <-- How can I tell SA to set the FKs on *contact* here?
session.add(contact)
print('Before flush: contact.username user=%r' % contact.username)
session.flush()
print('After flush : contact.username user=%r' % contact.username)
According to this answer - https://stackoverflow.com/a/52911047/4981223 it is not possible:
The FK of the child object isn't updated until you issue a flush() either explicitly or through a commit(). I think the reason for this is that if the parent object of a relationship is also a new instance with an auto-increment PK, SQLAlchemy needs to get the PK from the database before it can update the FK on the child object (but I stand to be corrected!).
I'm stuck on this issue and I don't know how to fix it. This is my models.py file:
models.py
class TripDetail(db.Model):
"""
Base class for every table that contains info about a trip.
"""
__abstract__ = True
__bind_key__ = 'instructions'
id = db.Column(db.Integer, primary_key=True)
# list of fields
class Overview(TripDetail):
"""
Class that contains general information about a trip.
"""
__tablename__ = 'overviews'
__table_args__ = (
db.ForeignKeyConstraint(['user_id', 'calendar_id'], ['calendars.user_id', 'calendars.id'], ondelete='CASCADE'),
) # constraints on other tables, omitted here
user_id = db.Column(db.Integer, primary_key=True)
calendar_id = db.Column(db.Integer, primary_key=True)
calendar = db.relationship('Calendar', backref=db.backref('overviews', cascade='delete'), passive_deletes=True)
# other fields
class Step(TripDetail):
__tablename__ = 'steps'
overview_id = db.Column(db.Integer, db.ForeignKey('overviews.id', ondelete='CASCADE'))
overview = db.relationship('Overview', backref=db.backref('steps', cascade='delete'), passive_deletes=True)
# also other fields
And this is how I add items to the DB (the response parameter contains a dict that matches the classes, in such a way that it can be unpacked directly):
def add_instruction(response):
"""
Adds a travel instruction to the database.
"""
steps = response.pop('steps')
overview = Overview(**response)
for step in steps:
Step(overview=overview, **step)
db.session.add(overview)
db.session.commit()
logger.info(f"Stored instruction with PK {(overview.id, overview.user_id, overview.calendar_id, overview.event_id)}")
Now, the overviews table is filled up correctly, but steps stays empty. Inspecting the logs, I receive this warning:
SAWarning: Object of type not in session, add operation along 'Overview.steps' will not proceed
(orm_util.state_class_str(state), operation, prop))
What am I doing wrong?
Normally, when add()ing objects to a session, their related objects will get auto-added like you wanted. That behavior is controlled by the relationship's cascade.
Setting cascade to 'delete' in Steps.overview removes the default 'save-update', which is what turns on the auto-adding. You could just add it back with cascade='save-update, delete', but take a look at the possible traits and see what else you might need. A common set is 'all, delete-orphan'.
And remember these are strictly ORM behaviors; setting a 'delete' in your cascade won't set the column's ON [event] CASCADE.
Well, I've solved this by expliciting adding the created step to the session. Still have no idea what the warning means though, so I'll just leave this here. My fix:
for step in steps:
step = Step(overview=overview, **step) # explicitly add
db.session.add(step)
Newish to SQLAlchemy (so my terminology may be a bit off). I want to create a database object inside the constructor of another, but the problem is I can't add said object to the session, so I get an error.
My schema looks a bit like the following:
class Tag(Base):
__tablename__ = 'tag'
id = Column(Integer, Sequence('tag_id_seq'), primary_key=True, nullable=False)
type = Column(String(1), nullable=False)
name = Column(String(255), unique=True, nullable=False)
def __init__(self, type, name):
self.type=type
self.name=name
def __repr__(self):
return "<Tag('%s')>" % (self.id)
class Venue:
__tablename__ = 'venue'
tag_id = Column(Integer)
tag_type = Column(String(1), nullable=False)
location = Column(String(255), nullable=False)
tag = ForeignKeyConstraint(
(tag_id, tag_type),
(Tag.id, Tag.type),
onupdate='RESTRICT',
ondelete='RESTRICT',
)
def __init__(self,name,location):
self.tag = Tag('V',name)
self.location = location
When I do the following:
session.add(Venue("Santa's Cafe","South Pole"))
I get an error:
UnmappedInstanceError: Class '__builtin__.instance' is not mapped
I assume this is because the the Tag object created in Venue's constructor is not added to the session. My question is how/when do I do this. (I'd really prefer to create that Tag object in the constructor if possible). I think I could do this with a scoped_session but that seems like a really bad solution.
Thanks
Inherit Venue from Base. Otherwise Venue won't be mapped.
Move ForeignKeyConstraint to __table_args__.
Replace currently meaningless tag property with relationship to Tag. The default value of cascade parameter to relationship contains 'save-update' rule that will add new referred object to the same session as parent.
From documentation:
save-update - cascade the Session.add() operation. This cascade
applies both to future and past calls to add(), meaning new items
added to a collection or scalar relationship get placed into the same
session as that of the parent, and also applies to items which have
been removed from this relationship but are still part of unflushed
history.
Let's have a classes X and Y and relations between them x2y and y2x.
From class_mapper(Class).iterate_properties iterator we can get all class's properties.
So x2y and y2x are RelationshipProperty and what I hope to get from is a class or a class name of objects on remote side of relation.
I've already tried to make a solution.
I've found x2y.remote_side[0].table.name, made a tables_map which maps a table name to a class and it works fine for one-to-many and one-to-one. If I use it for many-to-many the table name is an association table.
Any hints on how can I get the remote side class?
X.x2y.property.mapper.class_
relatonshipproperty will eventually get class-level attribute documentation the same as mapper does now.
edit. Here is a test which illustrates the above returning "Y" from "X", and no reflection doesn't create relationships so should have no effect:
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class X(Base):
__tablename__ = 'x'
id = Column(Integer, primary_key=True)
x2y = relationship("Y")
class Y(Base):
__tablename__ = 'y'
id = Column(Integer, primary_key=True)
x_id = Column(Integer, ForeignKey("x.id"))
assert X.x2y.property.mapper.class_ is Y
I've found that a method argument() on relationshipproperty returns remote class.
for prop in class_mapper(X).iterate_properties:
if isinstance(prop, RelationshipProperty):
relation = prop
relation.argument()
I am currently trying to create the following database schema with SQLAlchemy (using ext.declarative):
I have a base class MyBaseClass which provides some common functionality for all of my publicly accessible classes, a mixin class MetadataMixin that provides functionality to query metadata from imdb and store it.
Every class that subclasses MetadataMixin has a field persons which provides a M:N relationship to instances of the Person class, and a field persons_roles which provides a 1:N relationship to an object (one for each subclass) which stores the role a concrete Person plays in the instance of the subclass.
This is an abbreviated version of what my code looks like at the moment:
from sqlalchemy import Column, Integer, Enum, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyBaseClass(object):
"""Base class for all publicly accessible classes"""
id = Column(Integer, primary_key=True)
class Person(MyBaseClass):
"""A Person"""
name = Column(Unicode)
movies = association_proxy('movie_roles', 'movie',
creator=lambda m: _PersonMovieRole(movie=m))
shows = association_proxy('show_roles', 'show',
creator=lambda s: _PersonShowRole(show=s=))
class _PersonMovieRole(Base):
"""Role for a Person in a Movie"""
__tablename__ = 'persons_movies'
id = Column(Integer, primary_key=True)
role = Column(Enum('none', 'actor', 'writer', 'director', 'producer'),
default='none')
person_id = Column(Integer, ForeignKey('persons.id'))
person = relationship('Person', backref='movie_roles')
movie_id = Column(Integer, ForeignKey('movies.id'))
movie = relationship('Movie', backref='persons_roles')
class _PersonShowRole(Base):
"""Role for a Person in a Show"""
__tablename__ = 'persons_shows'
id = Column(Integer, primary_key=True)
role = Column(Enum('none', 'actor', 'writer', 'director', 'producer'),
default='none')
person_id = Column(Integer, ForeignKey('persons.id'))
person = relationship('Person', backref='show_roles')
show_id = Column(Integer, ForeignKey('shows.id'))
show = relationship('Episode', backref='persons_roles')
class MetadataMixin(object):
"""Mixin class that provides metadata-fields and methods"""
# ...
persons = association_proxy('persons_roles', 'person',
creator= #...???...#)
class Movie(Base, MyBaseClass, MetadataMixin):
#....
pass
What I'm trying to do is to create a generic creator function for association_proxy that creates either a PersonMovieRole or a PersonShowRole object, depending on the class of the concrete instance that a Person is added to. What I'm stuck on at the moment is that I don't know how to pass the calling class to the creator function.
Is this possible, or is there maybe even an easier way for what I'm trying to accomplish?
By the time your persons field is defined, you cannot really know what class it will end up in. Python takes up ready dictionaries of class members and creates classes out of them (via type.__new__), but when it happens, those members are already fully defined.
So you need to provide the required information directly to the mixin, and tolerate the small duplication it will create in your code. I'd opt for interface similar to this one:
class Movie(Base, MyBaseClass, MetadataMixin('Movie')):
pass
(You cannot have MetadataMixin(Movie) either, for the exact same reasons: Movie requires its base classes to be completely defined by the time the class is created).
To implement such "parametrized class", simply use a function:
def MetadataMixin(cls_name):
"""Mixin class that provides metadata-fields and methods"""
person_role_cls_name = 'Person%sRole' % cls_name
person_role_cls = Base._decl_class_registry[person_role_cls_name]
class Mixin(object):
# ...
persons = association_proxy('persons_roles', 'person',
creator=person_role_cls)
return Mixin
This works because what we're looking up in Base._decl_class_registry - the registry of all classes descending from your declarative base - is not the final class (e.g. Movie), but the association object (e.g. PersonMovieRole).