We Keep Coding

python Class from two sources

I am trying to sync data between our accounting software and our internal software.
The accounting software runs on MSSQL and our software runs on PGSQL
Is it possible to create a single class with two sources of data and possibly two different structure and a mapping method inside? or is it easier to create two distinct class with a mapping method?
One class:
class test(Base):
__tablename__ = 'test'
def __init__(self, type='MSSQL'):
self.type = type
self.__tablename__ = __tablename__
#classmethod
def DBMSSQL(cls):
return cls('MSSQL')
#classmethod
def DBPGSQL(cls):
return cls('PGSQL')
How to map fields between the two type and how to change the engine used?
Or with two classes:
class testMSSQL(Base):
__tablename__ = 'test'
__table_args__ = {"schema": "dbo"}
to_testPGSQL():
pass
class testPGSQL(Base):
__tablename__ = 'testPG'
to_testMSSQL():
pass
Thanks

Subclassing Class Inheritance Hierarchies from SQLAlchemy

I struggled writing this question. It gets into what may be a complicated and uncommon use case.
I have defined several ORM classes in one project which is responsible for maintaining a common database schema and core functionality. For example, let's say this is the model.email module.
from sqlalchemy import Column, Index, ForeignKey
from sqlalchemy import Boolean, Integer, Text
from . import Base
class CampaignDB(Base):
"""
Note: this basic database mapper class is expected to be extended.
When sub-classing, be mindful to override mappings to other extended classes.
"""
__tablename__ = 'campaigns'
audience_id = Column(Integer, ForeignKey("audiences.id"))
active = Column(Boolean)
name = Column(Text)
These ORM classes are imported into several other projects as a package. In some cases, these ORM classes are subclassed to provide additional functionality. For example, here the CampaignDB class is subclassed to provide support for sending email in a specific project context.
from model.email import CampaignDB
class Campaign(CampaignDB):
"""
Provides sending capability to the email campaign ORM class.
"""
def __init__(self, audience_id=None, active=None, name=None):
self.audience_id = audience_id
self.active = active
self.name = name
def send(self):
print("send emails to the audience")
Now I would like to refactor the CampaignDB and subclassed Campaign classes to be polymorphic bases using SQLAlchemy's Class Inheritance Hierarchies. For example, I'd like make CampaignDB a base class for EmailCampaignDB and PushCampaignDB. I would then like to extend EmailCampaignDB and PushCampaignDB separately, as say, EmailCampaign and PushCampaign in the importing project. However, I would like to still be able to query for Campaign and be returned instances of EmailCampaign and PushCampaign.
I have made several attempts to solve this but run into problems. In particular, session.query(Campaign).all() returns no results because SQLAlchemy doesn't seem to consider it as a base class. The generated SQL has the following WHERE clause: WHERE email.campaigns.type IN (NULL)
Here's the gist of what I am attempting.
class CampaignDB(Base):
"""
Note: this basic database mapper class is expected to be extended.
When sub-classing, be mindful to override mappings to other extended classes.
"""
__tablename__ = 'campaigns'
audience_id = Column(Integer, ForeignKey("audiences.id"))
active = Column(Boolean)
name = Column(Text)
type = Column(String(16))
__mapper_args__ = {
'polymorphic_on': type
}
class EmailCampaignDB(CampaignBaseDB):
__mapper_args__ = {
'polymorphic_identity': 'email'
}
class PushCampaignDB(CampaignBaseDB):
__mapper_args__ = {
'polymorphic_identity': 'push'
}
def send(self):
print("send push notifications to the audience")
class Campaign(CampaignDB):
pass
class EmailCampaign(EmailCampaignDB):
def send(self):
print("send emails to the audience")
class PushCampaign(PushCampaignDB):
def send(self):
print("send push notifications to the audience")
Is this possible? Is there a better way to achieve this in this "packaged ORM" context?

I managed to find a way to make this work by changing the way I think about the problem. I gave up on trying to create and query a Campaign subclass of CampaignDB. I also used the Declarative API which seems to facilitate subclassing EmailCampaignDB and PushCampaignDB.
The model.email module in the core project:
from sqlalchemy.ext.declarative import declared_attr
from sqlalchemy import Column, Index, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy import Boolean, Integer, Text
from . import Base
class CampaignBaseDB(Base):
"""
Note: this basic database mapper class is expected to be extended.
When sub-classing, be mindful to override mappings to other extended classes.
"""
__tablename__ = 'campaign_bases'
#declared_attr
def __mapper_args__(cls):
return {
'polymorphic_on': cls.type,
}
audience_id = Column(Integer, ForeignKey("audiences.id"))
active = Column(Boolean)
name = Column(Text)
type = Column(String(16))
class EmailCampaignDB(CampaignBaseDB):
#declared_attr
def __mapper_args__(cls):
return {
'polymorphic_identity': 'email'
}
class PushCampaignDB(CampaignBaseDB):
#declared_attr
def __mapper_args__(cls):
return {
'polymorphic_identity': 'push'
}
Subclassing the campaign classes in the importing project:
from model.email import EmailCampaignDB, PushCampaignDB
class EmailCampaign(EmailCampaignDB):
def send(self):
print("send emails to the audience")
class PushCampaign(PushCampaignDB):
def send(self):
print("send push notifications to the audience")
Polymorphic query:
for campaign in db.query(CampaignBaseDB).all():
campaign.send()
#> send emails to the audience
#> send push notifications to the audience
This approach does result in "SAWarning: Reassigning polymorphic association for identity", so I still feel like there would be a better way.

SQLAlchemy Metadata Relationship with Multiple Foreign Keys

I'm new to SQLAlchemy and trying to set up an ORM for an existing database. I'm setting up the tables using metadata and specifying the foreign keys myself. The table setup looks like this:
class User(Base):
__table__ = Table('users', metadata,
Column('user_id', Integer, primary_key=True),
autoload=True)
class Transaction(Base):
__table__ = Table('transaction', metadata,
Column('transaction_id', Integer, primary_key=True),
Column('seller_id', Integer, ForeignKey('users.user_id')),
Column('buyer_id', Integer, ForeignKey('users.user_id')),
autoload=True)
seller = relationship('User', foreign_keys=[seller_id])
buyer = relationship('User', foreign_keys=[buyer_id])
This doesn't run, with the error:
NameError: name 'seller_id' is not defined
Any idea what's wrong?

To understand why you get the error you should refresh your understanding on class construction in Python:
When a class definition is entered, a new namespace is created, and used as the local scope — thus, all assignments to local variables go into this new namespace. In particular, function definitions bind the name of the new function here.
In your example you have no assignments to seller_id that would introduce the name, and so an attempt to use that name during class construction raises the NameError. What you do have available in the current namespace during class construction is the __table__ that you assigned. In fact this exact use case is documented under "Using a Hybrid Approach with __table__":
Note that when the __table__ approach is used, the object is immediately usable as a plain Table within the class declaration body itself, as a Python class is only another syntactical block.
In other words access the columns through the Table object bound to the name __table__:
class Transaction(Base):
__table__ = Table('transaction', metadata,
Column('transaction_id', Integer, primary_key=True),
Column('seller_id', Integer, ForeignKey('users.user_id')),
Column('buyer_id', Integer, ForeignKey('users.user_id')),
autoload=True)
seller = relationship('User', foreign_keys=[__table__.c.seller_id])
buyer = relationship('User', foreign_keys=[__table__.c.buyer_id])

SqlAlchemy attribute that tracks an assigned attribute

Goal: Create an SQLAlchemy attribute which tracks/follows changes in another object's SQLAlchemy attribute.
Given:
class ClazzA():
attributeA = Column(JSONDict)
class ClazzB():
attributeB = Column(?)
objectA = ClazzA()
objectA.attributeA = {'foo': 1}
objectB = ClazzB()
objectB.attributeB = objectA.attributeA
objectA.attributeA['foo'] = 2
JSONDict is associated with MutableDict as described here: http://docs.sqlalchemy.org/en/latest/orm/extensions/mutable.html#module-sqlalchemy.ext.mutable , i.e. the JSONDict type allows for mutation tracking.
So we have this dictionary on objectA whose changes are being recorded by SQLAlchemy. I would like for attributeB to track attributeA such that even if the application is restarted (i.e. the attributes are reloaded from the DB), then attributeB will continue to reflect changes made to attributeA's dictionary.
Of course, this is closely related to the fact that Python doesn't have an idea of pointers. I was wondering if SQLAlchemy has a solution for this particular problem.

TL;DR
You want a one-to-many relationship.
from sqlalchemy import ForeignKey, Integer, Column
from sqlalchemy.orm import relationship
class Widget(Base):
__tablename__ = 'widget'
widget_id = Column(Integer, primary_key=True)
# name columns, type columns, ...
json = Column(JSONDict)
class ClazzB(Base):
__tablename__ = 'clazzb'
clazzb_id = Column(Integer, primary_key=True)
# Your "attributeB"
widget_id = Column(Integer,
ForeignKey('widget.widget_id',
onupdate='cascade',
ondelete='cascade'),
nullable=False)
widget = relationship('Widget')
# possible association_proxy
#widget_json = association_proxy('widget', 'json')
Using a Relationship
Define a relationship between models ClazzA and ClazzB. Now since we don't have the whole picture, the below definitions are just examples.
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
class ClazzA(Base): # replace Base with the base class of your models
__tablename__ = 'clazza' # replace with the real tablename
# T is the type of your primary key, the column name is just an example
clazza_id = Column(T, primary_key=True)
class ClazzB(Base):
# The column that will relate this model to ClazzA
clazza_id = Column(T, ForeignKey('clazza.clazza_id',
onupdate='cascade',
ondelete='cascade'),
nullable=False)
# A handy accessor for relationship between mapped classes,
# not strictly required. Configurable to be either very lazy
# (loaded if accessed by issuing a SELECT) or eager (JOINed
# when loading objectB for example)
objectA = relationship('ClazzA')
Now instead of adding a reference to attributeA of ClazzA to ClazzB add a reference to related objectA to objectB on initialization.
objectB = ClazzB(..., objectA=objectA)
The two are now related and to access attributeA of related objectA through objectB do
objectB.objectA.attributeA
No need to track changes to attributeA, since it is the attributeA of the instance.
Now if you must have an attribute attributeB on ClazzB (to avoid refactoring existing code or some such), you could add a property
class ClazzB:
#property
def attributeB(self):
return self.objectA.attributeA
which will return the attributeA of the related objectA with
objectB.attributeB
objectB.attributeB['something'] = 'else'
and so on.
There is also an SQLAlchemy method for accessing attributes across relationships: association proxy. It supports simple querying, but is not for example subscriptable.
class ClazzB(Base):
attributeB = association_proxy('objectA', 'attributeA')
If you wish for ClazzB.attributeB to access values from the JSONDict under certain key, you can for example use something like this
class ClazzB(Base):
key = Column(Unicode)
#property
def attributeB(self):
return self.objectA.attributeA[self.key]
You can also make attributeB work as an SQL expression on class level using hybrid properties, if you need such a thing. You would have to write your class level expressions yourself though.

SQLAlchemy: Multiple Inheritance with dynamic 'association_proxy' creator function

I am currently trying to create the following database schema with SQLAlchemy (using ext.declarative):
I have a base class MyBaseClass which provides some common functionality for all of my publicly accessible classes, a mixin class MetadataMixin that provides functionality to query metadata from imdb and store it.
Every class that subclasses MetadataMixin has a field persons which provides a M:N relationship to instances of the Person class, and a field persons_roles which provides a 1:N relationship to an object (one for each subclass) which stores the role a concrete Person plays in the instance of the subclass.
This is an abbreviated version of what my code looks like at the moment:
from sqlalchemy import Column, Integer, Enum, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyBaseClass(object):
"""Base class for all publicly accessible classes"""
id = Column(Integer, primary_key=True)
class Person(MyBaseClass):
"""A Person"""
name = Column(Unicode)
movies = association_proxy('movie_roles', 'movie',
creator=lambda m: _PersonMovieRole(movie=m))
shows = association_proxy('show_roles', 'show',
creator=lambda s: _PersonShowRole(show=s=))
class _PersonMovieRole(Base):
"""Role for a Person in a Movie"""
__tablename__ = 'persons_movies'
id = Column(Integer, primary_key=True)
role = Column(Enum('none', 'actor', 'writer', 'director', 'producer'),
default='none')
person_id = Column(Integer, ForeignKey('persons.id'))
person = relationship('Person', backref='movie_roles')
movie_id = Column(Integer, ForeignKey('movies.id'))
movie = relationship('Movie', backref='persons_roles')
class _PersonShowRole(Base):
"""Role for a Person in a Show"""
__tablename__ = 'persons_shows'
id = Column(Integer, primary_key=True)
role = Column(Enum('none', 'actor', 'writer', 'director', 'producer'),
default='none')
person_id = Column(Integer, ForeignKey('persons.id'))
person = relationship('Person', backref='show_roles')
show_id = Column(Integer, ForeignKey('shows.id'))
show = relationship('Episode', backref='persons_roles')
class MetadataMixin(object):
"""Mixin class that provides metadata-fields and methods"""
# ...
persons = association_proxy('persons_roles', 'person',
creator= #...???...#)
class Movie(Base, MyBaseClass, MetadataMixin):
#....
pass
What I'm trying to do is to create a generic creator function for association_proxy that creates either a PersonMovieRole or a PersonShowRole object, depending on the class of the concrete instance that a Person is added to. What I'm stuck on at the moment is that I don't know how to pass the calling class to the creator function.
Is this possible, or is there maybe even an easier way for what I'm trying to accomplish?

By the time your persons field is defined, you cannot really know what class it will end up in. Python takes up ready dictionaries of class members and creates classes out of them (via type.__new__), but when it happens, those members are already fully defined.
So you need to provide the required information directly to the mixin, and tolerate the small duplication it will create in your code. I'd opt for interface similar to this one:
class Movie(Base, MyBaseClass, MetadataMixin('Movie')):
pass
(You cannot have MetadataMixin(Movie) either, for the exact same reasons: Movie requires its base classes to be completely defined by the time the class is created).
To implement such "parametrized class", simply use a function:
def MetadataMixin(cls_name):
"""Mixin class that provides metadata-fields and methods"""
person_role_cls_name = 'Person%sRole' % cls_name
person_role_cls = Base._decl_class_registry[person_role_cls_name]
class Mixin(object):
# ...
persons = association_proxy('persons_roles', 'person',
creator=person_role_cls)
return Mixin
This works because what we're looking up in Base._decl_class_registry - the registry of all classes descending from your declarative base - is not the final class (e.g. Movie), but the association object (e.g. PersonMovieRole).