We Keep Coding

How to set the default value of a column in sqlalchemy to the value of a column from a relationship?

I am setting up a Sqlalchemy mapper for a sqlite database. My User class has a non-nullable relationship with my Team class. The code I already have is as follows:
class Team(Base):
__tablename__ = 'teams'
team_id = Column(Integer, primary_key=True)
# Using Integer as holder for boolean
is_local = Column(Integer, default=0)
class User(Base):
__tablename__ = 'users'
user_id = Column(Integer, primary_key=True)
team_id = Column(Integer, ForeignKey(Team.team_id), default=1, nullable=False)
team = relationship('Team')
is_local = Column(Integer, default=0)
I would like to establish that the value of User.is_local is by default the value of Team.is_local for the User's linked team.
However, after the creation of the User, I would still like the ability to modify the user's is_local value without changing the values of the team or any other user on the team.
So if I were to execute
faraway = Team(is_local=1)
session.add(faraway)
session.commit()
u = User(team=faraway)
session.add(u)
session.commit()
print(bool(u.is_local))
The result should be True
So far, I have tried context-sensitive default functions as suggested by https://stackoverflow.com/a/36579924, but I have not been able to find the syntax allowing me to reference Team.is_local
Is there a simple way to do this?

The first suggestion from SuperShoot, using a sql expression as the default appears to work. Specifically,
is_local = Column(Integer, default=select([Team.is_local]).where(Team.team_id==team_id))
gives me the logic I require.

SQLAlchemy generic relationship mixin

I'm working on a SQLAlchemy defining a bunch of mixin classes that applications should be able to import and extend their model.
When looking at the documentation, mixin classes are create knowing the final table name however, in the case of a generic library, the final table name that will be used by the application is not known.
Take the following mixin classes:
import sqlalchemy as sa
class UserMixin(object):
id = sa.Column(sa.Integer(), primary_key=True)
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
class ItemMixin(object):
id = sa.Column(sa.Integer(), primary_key=True)
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
class OrdersMixin(object):
id = sa.Column(sa.Integer(), primary_key=True)
user_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
Then an application defining its models:
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyUser(UserMixin, Base):
__tablename__ = 'myuser'
class MyItem(ItemMixin, Base):
__tablename__ = 'myitem'
total = sa.Column(sa.Integer())
class MyOrders(OrdersMixin, Base):
__tablename__ = 'myorders'
I have two issues with this model:
Except from redefining the relationship columns in the extending models, how can the mixin class build the relationship on its own.
Type of the foreign key is assumed by the mixin class, but the id of the table may come from the application itself or from another mixin class.
Is the model I'm trying to implement correct? What would be the right way to tackle this problem?

A Mixin is a class that copies its data into a table, but for SQL it matters if you're the owner of that data (the table) vs being a reference (the foreign key).
It looks like you're attempting to create Mixins that are both sources of truth and references. Which isn't possible in SQL.
Taking your example one step further and defining OrdersMixin like this will make the issues more obvious I think.
class OrdersMixin(UserMixin, ItemMixin):
id = sa.Column(sa.Integer(), primary_key=True)
For example MyOrders would end up like this once things are resolved.
class MyOrders(Base):
__tablename__ = 'myorders'
# This is from UserMixin
id = sa.Column(sa.Integer(), primary_key=True)
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
# This is from ItemMixin
id = sa.Column(sa.Integer(), primary_key=True)
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
# From OrdersMixin
id = sa.Column(sa.Integer(), primary_key=True) # This is defined last so it overrides all others with the same name.
user_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
With how you have the Mixin defined any table that used that Mixin would have primary_keys for the id column, which would conflict. Additionally you are duplicating every column in the Mixin, which in general you want to avoid in SQL (see Database normal form).
The final result would be something like this. Which is a whole bunch of columns meaning you wouldn't need to refer to any other tables and all of the references id you had were overwritten, meaning you wouldn't be able to join them anyway.
class MyOrders(Base):
__tablename__ = 'myorders'
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
id = sa.Column(sa.Integer(), primary_key=True) # This is defined last so it overrides all others with the same name.
user_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
To avoid that I keep my Mixins separate from initial table definition. I.e. I use a Mixin for when I want another table to refer to that table.
The following is close to what I think you were hoping to achieve.
import sqlalchemy as sa
from sqlalchemy import orm
class UserMixin(object):
user_id = sa.Column(sa.Integer(), ForeignKey("myuser.id"), index=True)
user = orm.relationship("MyUser")
class ItemMixin(object):
item_id = sa.Column(sa.Integer(), ForeignKey("myitem.id"), index=True)
item = orm.relationship("MyItem")
class OrdersMixin(UserMixin, ItemMixin):
order_id = sa.Column(sa.Integer(), sa.ForeignKey('myorders.id'))
user_id = sa.Column(sa.Integer(), sa.ForeignKey('myorders.user_id'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('myorders.item_id'))
Note in the Mixins I gave every column a unique name so that there aren't conflicts and in OrdersMixin even though I'm using UserMixin and ItemMixin I'm overriding the user_id and item_id columns because otherwise anything using the OrdersMixin would have foreign keys pointing to three different tables which would confuse the automatic query builder. But it will still add the user and item relations (and since they are defined as foreign keys to the original tables in MyOrders table I think the relationship will just work).
Then I would change your tables to look like this.
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyUser(Base):
__tablename__ = "myuser"
id = sa.Column(sa.Integer(),primary_key=True)
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
class MyItem(Base):
__tablename__ = "myitem"
id = sa.Column(sa.Integer(),primary_key=True)
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
class MyOrders(Base, UserMixin, OrdersMixin):
__tablename__ = "myorders"
id = sa.Column(sa.Integer(),primary_key=True)
The original table definition owns the columns (source of truth) defining them individually and Mixins (of this kind) are good to define references so subsequent references don't need define each of them individually. A Mixin can't be defined to be both a reference and a source of truth. In light of that instead of overriding the column each time like OrdersMixin it's better to just define it once canonically (the table) and once as a reference (the Mixin).

SQLAlchemy: #property mapping?

I have following models:
class Details(db.Model):
details_id = db.Column(db.Integer, primary_key=True)
details_main = db.Column(db.String(50))
details_desc = db.Column(db.String(50))
class Data(db.Model):
data_id = db.Column(db.Integer, primary_key=True)
data_date = db.Column(db.Date)
details_main = db.Column(db.String(50))
#property
def details_desc(self):
result = object_session(self).\
scalar(
select([Details.details_desc]).
where(Details.details_main == self.details_main)
)
return result
Now, I would like to run query using filter which depends on defined property. I get empty results (of course proper data is in DB). It doesn't work because, probably, I have to map this property. The question is how to do this? (One limitation: FK are not allowed).
Data.query\
.filter(Data.details_desc == unicode('test'))\
.all()

You can implement this with a regular relationship and an association proxy:
class Data(db.Model):
data_id = db.Column(db.Integer, primary_key=True)
data_date = db.Column(db.Date)
details_main = db.Column(db.String(50))
details = relationship(
Details,
primaryjoin=remote(Details.details_main) == foreign(details_main))
details_desc = association_proxy('details', 'details_desc')
Since there are no foreign keys in the schema, you need to tell SQLAlchemy yourself what the join condition for the relationship should be. This is what the remote() and foreign() annotations do.
With that in place, you can use an association_proxy "across" the relationship to create a property on Data which will work the way you want.

How to avoid inserting duplicate entries when adding values via a sqlalchemy relationship?

Let's assume we have two tables in a many to many relationship as shown below:
class User(db.Model):
__tablename__ = 'user'
uid = db.Column(db.String(80), primary_key=True)
languages = db.relationship('Language', lazy='dynamic',
secondary='user_language')
class UserLanguage(db.Model):
__tablename__ = 'user_language'
__tableargs__ = (db.UniqueConstraint('uid', 'lid', name='user_language_ff'),)
id = db.Column(db.Integer, primary_key=True)
uid = db.Column(db.String(80), db.ForeignKey('user.uid'))
lid = db.Column(db.String(80), db.ForeignKey('language.lid'))
class Language(db.Model):
lid = db.Column(db.String(80), primary_key=True)
language_name = db.Column(db.String(30))
Now in the python shell:
In [4]: user = User.query.all()[0]
In [11]: user.languages = [Language('1', 'English')]
In [12]: db.session.commit()
In [13]: user2 = User.query.all()[1]
In [14]: user2.languages = [Language('1', 'English')]
In [15]: db.session.commit()
IntegrityError: (IntegrityError) column lid is not unique u'INSERT INTO language (lid, language_name) VALUES (?, ?)' ('1', 'English')
How can I let the relationship know that it should ignore duplicates and not break the unique constraint for the Language table? Of course, I could insert each language separately and check if the entry already exists in the table beforehand, but then much of the benefit offered by sqlalchemy relationships is gone.

The SQLAlchemy wiki has a collection of examples, one of which is how you might check uniqueness of instances.
The examples are a bit convoluted though. Basically, create a classmethod get_unique as an alternate constructor, which will first check a session cache, then try a query for existing instances, then finally create a new instance. Then call Language.get_unique(id, name) instead of Language(id, name).
I've written a more detailed answer in response to OP's bounty on another question.

I would suggest to read Association Proxy: Simplifying Association Objects. In this case your code would translate into something like below:
# NEW: need this function to auto-generate the PK for newly created Language
# here using uuid, but could be any generator
def _newid():
import uuid
return str(uuid.uuid4())
def _language_find_or_create(language_name):
language = Language.query.filter_by(language_name=language_name).first()
return language or Language(language_name=language_name)
class User(Base):
__tablename__ = 'user'
uid = Column(String(80), primary_key=True)
languages = relationship('Language', lazy='dynamic',
secondary='user_language')
# proxy the 'language_name' attribute from the 'languages' relationship
langs = association_proxy('languages', 'language_name',
creator=_language_find_or_create,
)
class UserLanguage(Base):
__tablename__ = 'user_language'
__tableargs__ = (UniqueConstraint('uid', 'lid', name='user_language_ff'),)
id = Column(Integer, primary_key=True)
uid = Column(String(80), ForeignKey('user.uid'))
lid = Column(String(80), ForeignKey('language.lid'))
class Language(Base):
__tablename__ = 'language'
# NEW: added a *default* here; replace with your implementation
lid = Column(String(80), primary_key=True, default=_newid)
language_name = Column(String(30))
# test code
user = User(uid="user-1")
# NEW: add languages using association_proxy property
user.langs.append("English")
user.langs.append("Spanish")
session.add(user)
session.commit()
user2 = User(uid="user-2")
user2.langs.append("English") # this will not create a new Language row...
user2.langs.append("German")
session.add(user2)
session.commit()

How can I make property comparison able to be compiled to SQL expression in SQLAlchemy?

There are two tables that one column of table A is pointing another table B's primary key.
But they are placed in different database, so I cannot configure them with foreign key.
Configuring via relationship() is unavailable, so I implemented property attribute manually.
class User(Base):
__tablename__ = 'users'
id = Column(BigInteger, id_seq, primary=True)
name = Column(Unicode(256))
class Article(Base):
__tablename__ = 'articles'
__bind_key__ = 'another_engine'
# I am using custom session configures bind
# each mappers to multiple database engines via this attribute.
id = Column(BigInteger, id_seq, primary=True)
author_id = Column(BigInteger, nullable=False, index=True)
body = Column(UnicodeText, nullable=False)
#property
def author(self):
_session = object_session(self)
return _session.query(User).get(self.author_id)
#author.setter
def author(self, user):
if not isinstance(user, User):
raise TypeError('user must be a instance of User')
self.author_id = user.id
This code works well for simple operations. But it causes dirty queries making SQLAlchemy's features meaningless.
Code would be simple if it was configured via relationship() (e.g. query.filter(author=me)) got messed up(e.g. query.filter(author_id=me.id)).
Relationship(e.g. join) related features are never able to be used in query building.
Can I use property attribute, at least, in building query criterion(filter()/filter_by())?

you can still use relationship here. If you stick to "lazy loading", it will query for the related item in database B after loading the lead item in database A. You can place a ForeignKey() directive in the Column, even if there isn't a real one in the database. Or you can use primaryjoin directly:
class User(Base):
__tablename__ = 'users'
id = Column(BigInteger, id_seq, primary=True)
name = Column(Unicode(256))
class Article(Base):
__tablename__ = 'articles'
__bind_key__ = 'another_engine'
id = Column(BigInteger, id_seq, primary=True)
author_id = Column(BigInteger, nullable=False, index=True)
body = Column(UnicodeText, nullable=False)
author = relationship("User",
primaryjoin="foreign(Article.author_id) == User.id")