We Keep Coding

Python SQLalchemy Join lookup by User

Wrapping my head around a way to get a list of Jobs associated to a User. My DB Model goes a little something like this.
class Job(db.Model):
id = db.Column(db.Integer, primary_key=True)
# Relationship Rows
actions = db.relationship('JobAction', backref='job')
class JobAction(db.Model):
id = db.Column(db.Integer, primary_key=True)
# Linked Rows
user_id = db.Column(db.Integer, db.ForeignKey('users.id'))
# Relationship Rows
user = db.relationship('User', foreign_keys=[user_id], backref='jobactions')
I need to get a list of Jobs that are associated to a User. I can use either the User already matching a logged in users details. Or the user.id.
I was looking at something like the below, but no dice. I can see it's overly optimistic a query, but can't see what's up. Potentially a missing Join.
# Get User first.
user = User.query.filter_by(id=1).first()
# Get their Jobs
jobs = Job.query.filter_by(actions.user=user).all()
Any ideas would be greatly appreciated.
Cheers,

I'm guessing you are missing a foreign key. If your database model looked like this:
class User(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
jobactions = db.relationship("JobAction", back_populates="user")
class Job(db.Model):
__tablename__ = 'jobs'
id = db.Column(db.Integer, primary_key=True)
jobactions = db.relationship('JobAction', backref='job')
class JobAction(db.Model):
__tablename__ = 'jobactions'
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('users.id'))
job_id = db.Column(db.Integer, db.ForeignKey('jobs.id'))
user = db.relationship(User, back_populates="jobactions")
job = db.relationship(Job, back_populates="jobactions")
Then you could use:
jobs = [ jobaction.job for jobaction in user.jobactions ]

SQLAlchemy: #property mapping?

I have following models:
class Details(db.Model):
details_id = db.Column(db.Integer, primary_key=True)
details_main = db.Column(db.String(50))
details_desc = db.Column(db.String(50))
class Data(db.Model):
data_id = db.Column(db.Integer, primary_key=True)
data_date = db.Column(db.Date)
details_main = db.Column(db.String(50))
#property
def details_desc(self):
result = object_session(self).\
scalar(
select([Details.details_desc]).
where(Details.details_main == self.details_main)
)
return result
Now, I would like to run query using filter which depends on defined property. I get empty results (of course proper data is in DB). It doesn't work because, probably, I have to map this property. The question is how to do this? (One limitation: FK are not allowed).
Data.query\
.filter(Data.details_desc == unicode('test'))\
.all()

You can implement this with a regular relationship and an association proxy:
class Data(db.Model):
data_id = db.Column(db.Integer, primary_key=True)
data_date = db.Column(db.Date)
details_main = db.Column(db.String(50))
details = relationship(
Details,
primaryjoin=remote(Details.details_main) == foreign(details_main))
details_desc = association_proxy('details', 'details_desc')
Since there are no foreign keys in the schema, you need to tell SQLAlchemy yourself what the join condition for the relationship should be. This is what the remote() and foreign() annotations do.
With that in place, you can use an association_proxy "across" the relationship to create a property on Data which will work the way you want.

How to avoid inserting duplicate entries when adding values via a sqlalchemy relationship?

Let's assume we have two tables in a many to many relationship as shown below:
class User(db.Model):
__tablename__ = 'user'
uid = db.Column(db.String(80), primary_key=True)
languages = db.relationship('Language', lazy='dynamic',
secondary='user_language')
class UserLanguage(db.Model):
__tablename__ = 'user_language'
__tableargs__ = (db.UniqueConstraint('uid', 'lid', name='user_language_ff'),)
id = db.Column(db.Integer, primary_key=True)
uid = db.Column(db.String(80), db.ForeignKey('user.uid'))
lid = db.Column(db.String(80), db.ForeignKey('language.lid'))
class Language(db.Model):
lid = db.Column(db.String(80), primary_key=True)
language_name = db.Column(db.String(30))
Now in the python shell:
In [4]: user = User.query.all()[0]
In [11]: user.languages = [Language('1', 'English')]
In [12]: db.session.commit()
In [13]: user2 = User.query.all()[1]
In [14]: user2.languages = [Language('1', 'English')]
In [15]: db.session.commit()
IntegrityError: (IntegrityError) column lid is not unique u'INSERT INTO language (lid, language_name) VALUES (?, ?)' ('1', 'English')
How can I let the relationship know that it should ignore duplicates and not break the unique constraint for the Language table? Of course, I could insert each language separately and check if the entry already exists in the table beforehand, but then much of the benefit offered by sqlalchemy relationships is gone.

The SQLAlchemy wiki has a collection of examples, one of which is how you might check uniqueness of instances.
The examples are a bit convoluted though. Basically, create a classmethod get_unique as an alternate constructor, which will first check a session cache, then try a query for existing instances, then finally create a new instance. Then call Language.get_unique(id, name) instead of Language(id, name).
I've written a more detailed answer in response to OP's bounty on another question.

I would suggest to read Association Proxy: Simplifying Association Objects. In this case your code would translate into something like below:
# NEW: need this function to auto-generate the PK for newly created Language
# here using uuid, but could be any generator
def _newid():
import uuid
return str(uuid.uuid4())
def _language_find_or_create(language_name):
language = Language.query.filter_by(language_name=language_name).first()
return language or Language(language_name=language_name)
class User(Base):
__tablename__ = 'user'
uid = Column(String(80), primary_key=True)
languages = relationship('Language', lazy='dynamic',
secondary='user_language')
# proxy the 'language_name' attribute from the 'languages' relationship
langs = association_proxy('languages', 'language_name',
creator=_language_find_or_create,
)
class UserLanguage(Base):
__tablename__ = 'user_language'
__tableargs__ = (UniqueConstraint('uid', 'lid', name='user_language_ff'),)
id = Column(Integer, primary_key=True)
uid = Column(String(80), ForeignKey('user.uid'))
lid = Column(String(80), ForeignKey('language.lid'))
class Language(Base):
__tablename__ = 'language'
# NEW: added a *default* here; replace with your implementation
lid = Column(String(80), primary_key=True, default=_newid)
language_name = Column(String(30))
# test code
user = User(uid="user-1")
# NEW: add languages using association_proxy property
user.langs.append("English")
user.langs.append("Spanish")
session.add(user)
session.commit()
user2 = User(uid="user-2")
user2.langs.append("English") # this will not create a new Language row...
user2.langs.append("German")
session.add(user2)
session.commit()

python sqlalchemy - map a table to a certain data structure

I have a table defined with relationships and I noticed that even though I don't use joins in my query, the information is still retrieved:
class Employee(Base):
__tablename__ = "t_employee"
id = Column(Identifier(20), Sequence('%s_id_seq' % __tablename__), primary_key=True, nullable=False)
jobs = relationship("EmployeeJob")
roles = relationship("EmployeeRole")
class EmployeeJob(Base):
__tablename__ = "t_employee_job"
id = Column(Integer(20), Sequence('%s_id_seq' % __tablename__), primary_key=True, nullable=False)
employee_id = Column(Integer(20), ForeignKey('t_employee.id', ondelete="CASCADE"), primary_key=True)
job_id = Column(Integer(20), ForeignKey('t_job.id', ondelete="CASCADE"), primary_key=True)
class EmployeeRole(Base):
__tablename__ = "t_employee_role"
id = Column(Integer(20), Sequence('%s_id_seq' % __tablename__), primary_key=True, nullable=False)
employee_id = Column(Integer(20), ForeignKey('t_employee.id', ondelete="CASCADE"), nullable=False)
location_id = Column(Identifier(20), ForeignKey('t_location.id', ondelete="CASCADE"))
role_id = Column(Integer(20), ForeignKey('t_role.id', ondelete="CASCADE"), nullable=False)
session.query(Employee).all() retrieves also the roles and jobs but does so by querying the db for each row.
I have 2 questions about this situation:
1. In terms of performance I guess I should do the join by myself. Am I correct?
2. How do I map a table to a certain data structure? For example, I want to get the list of employees with their roles where each role should be represented by an Array of location ID and role ID e.g. {id:1, jobs:[1,2,3], roles:[[1,1],[1,2],[2,3]]}

1) Please read Eager Loading from the SA documentation.
By default, relationships are loaded lazy on first access to it. In your case, you could use, for example, Joined Load, so that the related rows would be loaded in the same query:
qry = (session.query(Employee).
options(joinedload(Employee.jobs)).
options(joinedload(Employee.roles))
).all()
If you want those relationships to be always loaded when an Employee is loaded, you can configure the relationship to automatically be loaded:
class Employee(Base):
# ...
jobs = relationship("EmployeeJob", lazy="joined")
roles = relationship("EmployeeRole", lazy="subquery")
2) Just create a method to extract the data structure from your query. Something like below should do it (using qry from first part of the answer):
def get_structure(qry):
res = [{"id": e.id,
"jobs": [j.id for j in e.jobs],
"roles": [[r.location_id, r.role_id] for r in e.roles],
}
for e in qry
]
return res
Also note: your EmployeeJob table has funny primary_key, which includes both the id column as well as two ForeignKey columns. I think you should choose either one or the other.

I have finally found a way to accomplish my second issue and decided to answer my own question for the benefit of others:
from sqlalchemy.ext.hybrid import hybrid_property
class Employee(Base):
__tablename__ = "t_employee"
id = Column(Identifier(20), Sequence('%s_id_seq' % __tablename__), primary_key=True, nullable=False)
_jobs = relationship("EmployeeJob", lazy="joined", cascade="all, delete, delete-orphan")
_roles = relationship("EmployeeRole", lazy="joined", cascade="all, delete, delete-orphan")
#hybrid_property
def jobs(self):
return [item.employee_id for item in self._jobs]
#jobs.setter
def jobs(self, value):
self._jobs = [EmployeeJob(job_id=id) for id in value]
#hybrid_property
def roles(self):
return [[item.location_id, item.role_id] for item in self._roles]
#roles.setter
def roles(self, value):
self._roles = [EmployeeRole(location_id=l_id, role_id=r_id) for l_id, r_id in value]
The cascade in the relationship is to ensure that the orphans are deleted once the list is updated, and the decorators define the getter and setter of each complex property
Thank you van for pointing me to the right direction!

What is the most elegant way to assign a new set of many-to-many related tags via association_proxy in SQLAlchemy?

This is my Flask-SQLAlchemy Declarative code:
from sqlalchemy.ext.associationproxy import association_proxy
from my_flask_project import db
tagging = db.Table('tagging',
db.Column('tag_id', db.Integer, db.ForeignKey('tag.id', ondelete='cascade'),
primary_key=True),
db.Column('role_id', db.Integer, db.ForeignKey('role.id', ondelete='cascade'),
primary_key=True)
)
class Tag(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), unique=True, nullable=False)
def __init__(self, name=None):
self.name = name
class Role(db.Model):
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('user.id', ondelete='cascade'))
user = db.relationship('User', backref=db.backref('roles', cascade='all',
lazy='dynamic'))
...
tags = db.relationship('Tag', secondary=tagging, cascade='all',
backref=db.backref('roles', cascade='all'))
tag_names = association_proxy('tags', 'name')
__table_args__ = (
db.UniqueConstraint('user_id', 'check_id'),
)
I think it's pretty standard many-to-many tagging solution. Now, I'd like to get all tags for a role and set new set of tags to a role.
The first one is pretty easy:
print role.tags
print role.tag_names
However, the second one made me stumbling upon my Python code all day long :-( I thought I could do this:
role.tag_names[:] = ['red', 'blue', 'white']
...or at least something similar using role.tags[:] = ..., but everything I invented raised many integrity errors, as SQLAlchemy didn't check if there are any existing tags and tried to insert all of them as completely new entities.
My final solution is:
# cleanup input
tag_names = set(filter(None, tag_names))
# existings tags to be updated
to_update = [t for t in role.tags if t.name in tag_names]
# existing tags to be added
to_add = list(
Tag.query.filter(Tag.name.in_(tag_names - set(role.tag_names)))
)
# tags to be created
existing_tags = to_update + to_add
to_create = [Tag(name) for name in tag_names - set([t.name for t in existing_tags])]
# assign new tags
role.tags[:] = existing_tags + to_create
# omitted bonus: find a way how to get rid of orphan tags
The question is: Is this really the right solution? Is there any more elegant way how to solve this trivial problem? I thik the whole matter is related to this question. Maybe I'm just silly, maybe I'm making things overcomplicated... anyway, thank you for any suggestions!

Actually SQLAlchemy does check if the object exists by calling Session.merge(). But it does it by identity — its primary key. The simplest solution is to make name primary key and everything will work. Sure, the three tables chain will become redundant in this case unless you are going to add some additional fields into Tag (e.g. counter).