I am currently trying to create the following database schema with SQLAlchemy (using ext.declarative):
I have a base class MyBaseClass which provides some common functionality for all of my publicly accessible classes, a mixin class MetadataMixin that provides functionality to query metadata from imdb and store it.
Every class that subclasses MetadataMixin has a field persons which provides a M:N relationship to instances of the Person class, and a field persons_roles which provides a 1:N relationship to an object (one for each subclass) which stores the role a concrete Person plays in the instance of the subclass.
This is an abbreviated version of what my code looks like at the moment:
from sqlalchemy import Column, Integer, Enum, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyBaseClass(object):
"""Base class for all publicly accessible classes"""
id = Column(Integer, primary_key=True)
class Person(MyBaseClass):
"""A Person"""
name = Column(Unicode)
movies = association_proxy('movie_roles', 'movie',
creator=lambda m: _PersonMovieRole(movie=m))
shows = association_proxy('show_roles', 'show',
creator=lambda s: _PersonShowRole(show=s=))
class _PersonMovieRole(Base):
"""Role for a Person in a Movie"""
__tablename__ = 'persons_movies'
id = Column(Integer, primary_key=True)
role = Column(Enum('none', 'actor', 'writer', 'director', 'producer'),
default='none')
person_id = Column(Integer, ForeignKey('persons.id'))
person = relationship('Person', backref='movie_roles')
movie_id = Column(Integer, ForeignKey('movies.id'))
movie = relationship('Movie', backref='persons_roles')
class _PersonShowRole(Base):
"""Role for a Person in a Show"""
__tablename__ = 'persons_shows'
id = Column(Integer, primary_key=True)
role = Column(Enum('none', 'actor', 'writer', 'director', 'producer'),
default='none')
person_id = Column(Integer, ForeignKey('persons.id'))
person = relationship('Person', backref='show_roles')
show_id = Column(Integer, ForeignKey('shows.id'))
show = relationship('Episode', backref='persons_roles')
class MetadataMixin(object):
"""Mixin class that provides metadata-fields and methods"""
# ...
persons = association_proxy('persons_roles', 'person',
creator= #...???...#)
class Movie(Base, MyBaseClass, MetadataMixin):
#....
pass
What I'm trying to do is to create a generic creator function for association_proxy that creates either a PersonMovieRole or a PersonShowRole object, depending on the class of the concrete instance that a Person is added to. What I'm stuck on at the moment is that I don't know how to pass the calling class to the creator function.
Is this possible, or is there maybe even an easier way for what I'm trying to accomplish?
By the time your persons field is defined, you cannot really know what class it will end up in. Python takes up ready dictionaries of class members and creates classes out of them (via type.__new__), but when it happens, those members are already fully defined.
So you need to provide the required information directly to the mixin, and tolerate the small duplication it will create in your code. I'd opt for interface similar to this one:
class Movie(Base, MyBaseClass, MetadataMixin('Movie')):
pass
(You cannot have MetadataMixin(Movie) either, for the exact same reasons: Movie requires its base classes to be completely defined by the time the class is created).
To implement such "parametrized class", simply use a function:
def MetadataMixin(cls_name):
"""Mixin class that provides metadata-fields and methods"""
person_role_cls_name = 'Person%sRole' % cls_name
person_role_cls = Base._decl_class_registry[person_role_cls_name]
class Mixin(object):
# ...
persons = association_proxy('persons_roles', 'person',
creator=person_role_cls)
return Mixin
This works because what we're looking up in Base._decl_class_registry - the registry of all classes descending from your declarative base - is not the final class (e.g. Movie), but the association object (e.g. PersonMovieRole).
Related
I am using SQLAlchemy in Python and am declaring my classes inheriting from a declarative base as follows:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class SomeClass(Base):
__tablename__ = 'some_table'
id = Column(Integer, primary_key=True)
name = Column(String(50))
As a user I would like to define the __tablename__ as a parameter, and not a hard-coded value , something like this:
class SomeClass(Base):
__tablename__ = f'{environment}_some_table'
id = Column(Integer, primary_key=True)
name = Column(String(50))
It is my understanding that f'{environment}_some_table' will be be evaluated when I import this package, and therefore I won't be able to modify it at a later stage (i.e. in an interactive notebook). I have a broken piece of code that tries to solve this through nested classes and encapsulation, but I do not manage to reference the instance variable of an outer class.
class Outer:
def __init__(self, env):
self.environment = env
class SomeClass(Base):
__tablename__ = f'{self.environment}_some_table'
id = Column(Integer, primary_key=True)
name = Column(String(50))
I have read in a couple of SO questions that it is better not to use nested classes since no special relationship is established between these classes.
So what kind of design should I use to solve this problem?
Thanks in advance!
you can make all your model definitions inside a function scope so the will be depended on outer arguments:
def create_models(environment):
class SomeClass(Base):
__tablename__ = f'{environment}_some_table'
id = Column(Integer, primary_key=True)
name = Column(String(50))
...
globals().update(locals()) # update outer scope if needed
... # some time later
create_models('cheese')
... # now create the db session/engine/etc ...
another choice to look at is the builtin reload method. check it out :)
I struggled writing this question. It gets into what may be a complicated and uncommon use case.
I have defined several ORM classes in one project which is responsible for maintaining a common database schema and core functionality. For example, let's say this is the model.email module.
from sqlalchemy import Column, Index, ForeignKey
from sqlalchemy import Boolean, Integer, Text
from . import Base
class CampaignDB(Base):
"""
Note: this basic database mapper class is expected to be extended.
When sub-classing, be mindful to override mappings to other extended classes.
"""
__tablename__ = 'campaigns'
audience_id = Column(Integer, ForeignKey("audiences.id"))
active = Column(Boolean)
name = Column(Text)
These ORM classes are imported into several other projects as a package. In some cases, these ORM classes are subclassed to provide additional functionality. For example, here the CampaignDB class is subclassed to provide support for sending email in a specific project context.
from model.email import CampaignDB
class Campaign(CampaignDB):
"""
Provides sending capability to the email campaign ORM class.
"""
def __init__(self, audience_id=None, active=None, name=None):
self.audience_id = audience_id
self.active = active
self.name = name
def send(self):
print("send emails to the audience")
Now I would like to refactor the CampaignDB and subclassed Campaign classes to be polymorphic bases using SQLAlchemy's Class Inheritance Hierarchies. For example, I'd like make CampaignDB a base class for EmailCampaignDB and PushCampaignDB. I would then like to extend EmailCampaignDB and PushCampaignDB separately, as say, EmailCampaign and PushCampaign in the importing project. However, I would like to still be able to query for Campaign and be returned instances of EmailCampaign and PushCampaign.
I have made several attempts to solve this but run into problems. In particular, session.query(Campaign).all() returns no results because SQLAlchemy doesn't seem to consider it as a base class. The generated SQL has the following WHERE clause: WHERE email.campaigns.type IN (NULL)
Here's the gist of what I am attempting.
class CampaignDB(Base):
"""
Note: this basic database mapper class is expected to be extended.
When sub-classing, be mindful to override mappings to other extended classes.
"""
__tablename__ = 'campaigns'
audience_id = Column(Integer, ForeignKey("audiences.id"))
active = Column(Boolean)
name = Column(Text)
type = Column(String(16))
__mapper_args__ = {
'polymorphic_on': type
}
class EmailCampaignDB(CampaignBaseDB):
__mapper_args__ = {
'polymorphic_identity': 'email'
}
class PushCampaignDB(CampaignBaseDB):
__mapper_args__ = {
'polymorphic_identity': 'push'
}
def send(self):
print("send push notifications to the audience")
class Campaign(CampaignDB):
pass
class EmailCampaign(EmailCampaignDB):
def send(self):
print("send emails to the audience")
class PushCampaign(PushCampaignDB):
def send(self):
print("send push notifications to the audience")
Is this possible? Is there a better way to achieve this in this "packaged ORM" context?
I managed to find a way to make this work by changing the way I think about the problem. I gave up on trying to create and query a Campaign subclass of CampaignDB. I also used the Declarative API which seems to facilitate subclassing EmailCampaignDB and PushCampaignDB.
The model.email module in the core project:
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy import Column, Index, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy import Boolean, Integer, Text
from . import Base
class CampaignBaseDB(Base):
"""
Note: this basic database mapper class is expected to be extended.
When sub-classing, be mindful to override mappings to other extended classes.
"""
__tablename__ = 'campaign_bases'
#declared_attr
def __mapper_args__(cls):
return {
'polymorphic_on': cls.type,
}
audience_id = Column(Integer, ForeignKey("audiences.id"))
active = Column(Boolean)
name = Column(Text)
type = Column(String(16))
class EmailCampaignDB(CampaignBaseDB):
#declared_attr
def __mapper_args__(cls):
return {
'polymorphic_identity': 'email'
}
class PushCampaignDB(CampaignBaseDB):
#declared_attr
def __mapper_args__(cls):
return {
'polymorphic_identity': 'push'
}
Subclassing the campaign classes in the importing project:
from model.email import EmailCampaignDB, PushCampaignDB
class EmailCampaign(EmailCampaignDB):
def send(self):
print("send emails to the audience")
class PushCampaign(PushCampaignDB):
def send(self):
print("send push notifications to the audience")
Polymorphic query:
for campaign in db.query(CampaignBaseDB).all():
campaign.send()
#> send emails to the audience
#> send push notifications to the audience
This approach does result in "SAWarning: Reassigning polymorphic association for identity", so I still feel like there would be a better way.
I'm using Flask and Peewee to create models for Users and Appointments, and each of them have a ForeignKeyField referencing each other. The problem is, if I define one above the other, Flask will give me x is not defined.
For example:
class User(Model):
appointments = ForeignKeyField(Appointment, related_name='appointments')
class Appointment(Model):
with_doctor = ForeignKeyField(User, related_name='doctor')
This would return 'User' is not defined. How am I able to fix this problem?
You generally don't define the foreign key on both sides of the relationship, especially when the relationship isn't a one-to-one.
You should also set the related_name to something that makes sense on the related model. user.appointments is probably a better name for accessing a User's Appointments than user.doctor.
class User(Model):
pass
class Appointment(Model):
with_doctor = ForeignKeyField(User, related_name='appointments')
You can define it as a string as far as i know:
class User(Model):
appointments = ForeignKeyField("Appointment", related_name='appointments')
class Appointment(Model):
with_doctor = ForeignKeyField(User, related_name='doctor')
if the declaration is before the creation, you have the option to write it as a string.
I'm not sure, i think that the bind is at runtime.
try some changes like this, an example for Names & Addresses :
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
addresses = db.relationship('Address', backref='person', lazy='dynamic')
class Address(db.Model):
id = db.Column(db.Integer, primary_key=True)
person_id = db.Column(db.Integer, db.ForeignKey('person.id'))
as far as i know, you should use db.ForeignKey() for a relationship like this ... ForeignKeyField() defined in django
Goal: Create an SQLAlchemy attribute which tracks/follows changes in another object's SQLAlchemy attribute.
Given:
class ClazzA():
attributeA = Column(JSONDict)
class ClazzB():
attributeB = Column(?)
objectA = ClazzA()
objectA.attributeA = {'foo': 1}
objectB = ClazzB()
objectB.attributeB = objectA.attributeA
objectA.attributeA['foo'] = 2
JSONDict is associated with MutableDict as described here: http://docs.sqlalchemy.org/en/latest/orm/extensions/mutable.html#module-sqlalchemy.ext.mutable , i.e. the JSONDict type allows for mutation tracking.
So we have this dictionary on objectA whose changes are being recorded by SQLAlchemy. I would like for attributeB to track attributeA such that even if the application is restarted (i.e. the attributes are reloaded from the DB), then attributeB will continue to reflect changes made to attributeA's dictionary.
Of course, this is closely related to the fact that Python doesn't have an idea of pointers. I was wondering if SQLAlchemy has a solution for this particular problem.
TL;DR
You want a one-to-many relationship.
from sqlalchemy import ForeignKey, Integer, Column
from sqlalchemy.orm import relationship
class Widget(Base):
__tablename__ = 'widget'
widget_id = Column(Integer, primary_key=True)
# name columns, type columns, ...
json = Column(JSONDict)
class ClazzB(Base):
__tablename__ = 'clazzb'
clazzb_id = Column(Integer, primary_key=True)
# Your "attributeB"
widget_id = Column(Integer,
ForeignKey('widget.widget_id',
onupdate='cascade',
ondelete='cascade'),
nullable=False)
widget = relationship('Widget')
# possible association_proxy
#widget_json = association_proxy('widget', 'json')
Using a Relationship
Define a relationship between models ClazzA and ClazzB. Now since we don't have the whole picture, the below definitions are just examples.
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
class ClazzA(Base): # replace Base with the base class of your models
__tablename__ = 'clazza' # replace with the real tablename
# T is the type of your primary key, the column name is just an example
clazza_id = Column(T, primary_key=True)
class ClazzB(Base):
# The column that will relate this model to ClazzA
clazza_id = Column(T, ForeignKey('clazza.clazza_id',
onupdate='cascade',
ondelete='cascade'),
nullable=False)
# A handy accessor for relationship between mapped classes,
# not strictly required. Configurable to be either very lazy
# (loaded if accessed by issuing a SELECT) or eager (JOINed
# when loading objectB for example)
objectA = relationship('ClazzA')
Now instead of adding a reference to attributeA of ClazzA to ClazzB add a reference to related objectA to objectB on initialization.
objectB = ClazzB(..., objectA=objectA)
The two are now related and to access attributeA of related objectA through objectB do
objectB.objectA.attributeA
No need to track changes to attributeA, since it is the attributeA of the instance.
Now if you must have an attribute attributeB on ClazzB (to avoid refactoring existing code or some such), you could add a property
class ClazzB:
#property
def attributeB(self):
return self.objectA.attributeA
which will return the attributeA of the related objectA with
objectB.attributeB
objectB.attributeB['something'] = 'else'
and so on.
There is also an SQLAlchemy method for accessing attributes across relationships: association proxy. It supports simple querying, but is not for example subscriptable.
class ClazzB(Base):
attributeB = association_proxy('objectA', 'attributeA')
If you wish for ClazzB.attributeB to access values from the JSONDict under certain key, you can for example use something like this
class ClazzB(Base):
key = Column(Unicode)
#property
def attributeB(self):
return self.objectA.attributeA[self.key]
You can also make attributeB work as an SQL expression on class level using hybrid properties, if you need such a thing. You would have to write your class level expressions yourself though.
It will be simplest to explain with code example, in Python I can do so to achieve model inheritance:
"""Image model"""
from sqlalchemy import Column, ForeignKey
from sqlalchemy.types import Integer, String, Text
from miasto_3d.model.meta import Base
class Image(Base):
__tablename__ = "image"
image_id = Column(Integer, primary_key=True)
path = Column(String(200))
def get_mime(self):
#function to get mime type from file
pass
"""WorkImage model"""
class WorkImage(Image, Base):
__tablename__ = "work_images"
image_id = Column(Integer, ForeignKey("image.image_id"), primary_key=True)
work_id = Column(Integer, ForeignKey("work.id"))
work = relation("Work", backref=backref('images',order_by='WorkImage.work_id'))
"""UserAvatar model"""
class UserAvatar(Image, Base):
__tablename__ = "user_avatars"
image_id = Column(Integer, ForeignKey("image.image_id"), primary_key=True)
user_id = Column(Integer, ForeignKey("user.id"))
user = relation("User", backref=backref('images',order_by='UserAvatar.user_id'))
How I do similar things in Rails? Or maybe there is another, better way to do it?
I know paperclip, but I don't like it's conception to use shared table to store photo and model data.
It looks like you're wanting either a polymorphic association or perhaps single table inheritance.
Since you don't define database fields in the model, you cannot inherit database schema in this way - all your fields will need to be specified per table in a migration. You probably should use paperclip, if only because reinventing the wheel is a pain. It works really well, and abstracts away from the actual database structure for you.
In Rails, rather than model inheritance, shared functionality tends to be implemented in modules, like so:
http://handyrailstips.com/tips/14-drying-up-your-ruby-code-with-modules