So I would like to have users add an item and an arbitrary category. Right now I use if statements to make sure that if the category has been created already, not to add it again. Is there a better way to make use of SQLAlchemy relationships so that I could skip some of the logic I had to write to ensure that the categories are unique?
Here are the model's I used:
class Category(Base):
__tablename__ = 'category'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
class Item(Base):
__tablename__ = 'item'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
description = Column(String)
category_id = Column(Integer, ForeignKey('category.id'))
category = relationship(Category)
date_created = Column(DateTime)
date_updated = Column(DateTime)
user_id = Column(Integer, ForeignKey('user.id'))
user = relationship(User)
Here is an example of how I would edit an item:
if new_category_name != category.name:
if db_session.query(Category).\
filter_by(name=new_category_name).count() == 0:
new_category = Category(name=new_category_name)
else:
new_category = db_session.query(Category)\
.filter_by(name=new_category_name).one()
is_last_of_category = db_session.query(Item)\
.filter_by(category_id=item.category_id).count() == 1
if is_last_of_category:
db_session.delete(category)
item.category = new_category
db_session.commit()
Any other suggestions you are willing to make I am happy to listen to.
Use the unique constraint,
Quoting from sqlalchemy's docs
unique – When True, indicates that this column contains a unique
constraint, or if index is True as well, indicates that the Index
should be created with the unique flag. To specify multiple columns in
the constraint/index or to specify an explicit name, use the
UniqueConstraint or Index constructs explicitly.
Example from sqlalchemy documentation:
from sqlalchemy import UniqueConstraint
meta = MetaData()
mytable = Table('mytable', meta,
# per-column anonymous unique constraint
Column('col1', Integer, unique=True),
Column('col2', Integer),
Column('col3', Integer),
# explicit/composite unique constraint. 'name' is optional.
UniqueConstraint('col2', 'col3', name='uix_1')
)
Related
I have UniqueConstraint on field, but it wont allow me to add multiple entries (two is max!)
from sqlalchemy import Column, Integer, String, Boolean, UniqueConstraint
class Cart(SqlAlchemyBase):
__tablename__ = 'cart'
__table_args__ = (UniqueConstraint('is_latest'), {})
sid = Column(Integer, primary_key=True)
is_latest = Column(Boolean, index=True, nullable=False)
name = Column(String)
I would like to support more entries, so that one name can have two variants:
name=foo, is_latest=True
name=foo, is_latest=False
name=bar, is_latest=True
name=bar, is_latest=False
but then reject any subsequent attempt to write name=foo (or bar) and is_latest=True
What you are trying to achieve here is a type 2 slowly changing dimension, this is a topic that has been discussed extensively and I encourage you to look it up.
When I look at your table you seem to use sid as a surrogate key, but I fail to see what is the natural key and what will be updated as time goes.
Anyway, there are several ways to achieve SCD type 2 result without the need to worry about your check, but the the simplest in my mind is to keep on adding records with your natural key and when querying, select only the one with highest surrogate key (autoincrementing integer), no need for current uniqueness here as only the latest value is fetched.
There are examples for versioning rows in SQLAlchemy docs, but since website come and go, I'll put a simplified draft of the above approach here.
class VersionedItem(Versioned, Base):
id = Column(Integer, primary_key=True) # surrogate key
sku = Column(String, index=True) # natural key
price = Column(Integer) # the value that changes with time
#event.listens_for(Session, "before_flush")
def before_flush(session, flush_context, instances):
for instance in session.dirty:
if not (
isinstance(instance, VersionedItem)
and session.is_modified(instance)
and attributes.instance_state(instance).has_identity
):
continue
make_transient(instance) # remove db identity from instance
instance.id = None # remove surrogate key
session.add(instance) # insert instance as new record
Looks like a Partial Unique Index can be used:
class Cart(SqlAlchemyBase):
__tablename__ = 'cart'
id = Column(Integer, primary_key=True)
cart_id = Column(Integer)
is_latest = Column(Boolean, default=False)
name = Column(String)
__table_args__ = (
Index('only_one_latest_cart', name, is_latest,
unique=True,
postgresql_where=(is_latest)),
)
name=foo, is_latest = True
name=foo, is_latest = False
name=bar, is_latest = False
name=bar, is_latest = False
And when adding another name=foo, is_latest = True
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "only_one_latest_cart"
DETAIL: Key (name, is_latest)=(foo, t) already exists.
I've three tables User, Device and Role. I have created a many-to-many relation b/w User and Device like this;
#Many-to-Many relation between User and Devices
userDevices = db.Table("user_devices",
db.Column("id", db.Integer, primary_key=True),
db.Column("user_id", db.Integer, db.ForeignKey("user.id")),
db.Column("device_id", db.Integer, db.ForeignKey("device.id"))))
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(60), index=True, unique=True)
devices = db.relationship("Device", secondary=userDevices, backref=db.backref('users'), lazy="dynamic")
class Device(db.Model):
__tablename__ = 'device'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), unique=True)
This works quiet well. I can assign a device d1 to user u1 > d1.users.append(u1), and user to device > u1.devices.append(d1) and db.session.commit().
What I want more is to extend the table user_devices with one more column as role_id which will be ForeignKey for Role table. So that this table user_devices will clearly describe a Role for specific User on specific Device. after adding a column role_id in table user_devices I described Role table as;
class Role(db.Model):
__tablename__ = 'role'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), unique=True)
device = db.relationship("Device", secondary=userDevices, backref=db.backref('roles'), lazy="dynamic")
In this way, how can I assign a Role r1 to User u1 on Device d1 ?
here is what I tried:
# First get the device, user and role
deviceRow = db.session.query(Device).filter(Device.name=="d1").first()
userRow = db.session.query(User).filter(User.username=="u1").first()
roleRow = db.session.query(Role).filter(Role.name == "r1").first()
# Then add the user on that device
deviceRow.users.append(userRow)
deviceRow.roles.append(roleRow)
This creates two rows in the table user_devices
Is there any way that we could add two attributes into the table like this ?;
deviceRow.users.append(userRow).roles.append(roleRow)
so that it creates only one row after commit() ?
An association of 3 entities is no more a simple many to many relationship. What you need is the association object pattern. In order to make handling the association a bit easier map it as a model class instead of a simple Table:
class UserDevice(db.Model):
__tablename__ = "user_devices"
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey("user.id"), nullable=False)
device_id = db.Column(db.Integer, db.ForeignKey("device.id"), nullable=False)
role_id = db.Column(db.Integer, db.ForeignKey("role.id"), nullable=False)
__table_args__ = (db.UniqueConstraint(user_id, device_id, role_id),)
user = db.relationship("User", back_populates="user_devices")
device = db.relationship("Device")
role = db.relationship("Role", back_populates="user_devices")
class User(db.Model):
__tablename__ = "user"
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(60), index=True, unique=True)
user_devices = db.relationship("UserDevice", back_populates="user")
class Role(db.Model):
__tablename__ = "role"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(60), unique=True)
user_devices = db.relationship("UserDevice", back_populates="role")
To associate a user with a device and a role create a new UserDevice object:
device = db.session.query(Device).filter(Device.name == "d1").first()
user = db.session.query(User).filter(User.username == "u1").first()
role = db.session.query(Role).filter(Role.name == "r1").first()
assoc = UserDevice(user=user, device=device, role=role)
db.session.add(assoc)
db.session.commit()
Note that the ORM relationships are no longer simple collections of Device etc., but UserDevice objects. This is a good thing: when you iterate over user.user_devices for example, you get information on both the device and the role the user has on it. If you do wish to provide the simpler collections as well for situations where you for example don't need the role information, you can use an associationproxy.
There is a way to have 3-way many-to-many that is not a composition of two many-to-many relationships. You need an association object because the syntax for using just a Table doesn't allow 3-way many-to-many (because secondary explicitly refers to a 2-way many-to-many).
Here is a minimum example of how to do that in general:
from sqlalchemy import ForeignKey, Column, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base, declared_attr
Base = declarative_base()
# Helper classes to simplify the other classes:
# 1. Adds ch column
# 2. Defines how to print it
class Ch:
ch = Column(String, nullable=False)
def __str__(self):
return self.ch
# 3. Automatically determines table name (for foreign key)
class AutoNamed:
#declared_attr
def __tablename__(cls):
return cls.__name__
class ABC(AutoNamed, Base):
a_id = Column(Integer, ForeignKey('A.a_id'), primary_key=True)
b_id = Column(Integer, ForeignKey('B.b_id'), primary_key=True)
c_id = Column(Integer, ForeignKey('C.c_id'), primary_key=True)
a = relationship('A', back_populates='abcs')
b = relationship('B', back_populates='abcs')
c = relationship('C', back_populates='abcs')
def __repr__(self):
return f'{self.a} {self.b} {self.c}'
class A(Ch, AutoNamed, Base):
a_id = Column(Integer, primary_key=True)
abcs = relationship('ABC', back_populates='a')
class B(Ch, AutoNamed, Base):
b_id = Column(Integer, primary_key=True)
abcs = relationship('ABC', back_populates='b')
class C(Ch, AutoNamed, Base):
c_id = Column(Integer, primary_key=True)
abcs = relationship('ABC', back_populates='c')
Ok, now a little explanation:
ABC is an association table that needs a single instance of each of the tables in the 3-way many-to-many.
Each of A, B, C will have references to all ABC objects that involve them added automatically when you instantiate an ABC instance.
There is a gotchya: when you use relationship.secondary, the property on the object is a list of the other type (in their case, parent.children is a list of Children objects). However, in the docs for "association objects", when translating this to Association objects, although they still name the property on the parent object children, it is actually a list of Association objects. Here, I make this explicit by calling the property abcs.
You can instantiate these like normal:
anA = A(ch='x')
anB = B(ch='y')
anC = C(ch='z')
anABC = ABC(a=anA, b=anB, c=anC)
sess.add(anABC)
As a sanity check, here's the SQL that gets generated from this for SQLite. Exactly what we expect.
CREATE TABLE "A" (
ch VARCHAR NOT NULL,
a_id INTEGER NOT NULL,
PRIMARY KEY (a_id)
);
CREATE TABLE "B" (
ch VARCHAR NOT NULL,
b_id INTEGER NOT NULL,
PRIMARY KEY (b_id)
);
CREATE TABLE "C" (
ch VARCHAR NOT NULL,
c_id INTEGER NOT NULL,
PRIMARY KEY (c_id)
);
CREATE TABLE "ABC" (
a_id INTEGER NOT NULL,
b_id INTEGER NOT NULL,
c_id INTEGER NOT NULL,
PRIMARY KEY (a_id, b_id, c_id),
FOREIGN KEY(a_id) REFERENCES "A" (a_id),
FOREIGN KEY(b_id) REFERENCES "B" (b_id),
FOREIGN KEY(c_id) REFERENCES "C" (c_id)
);
Still wrapping my head around SqlAlchemy and have run into a few issues. Not sure if it is because I am creating the relationships incorrectly, querying incorrect, or both.
The general idea is...
one-to-many from location to user (a location can have many users but users can only have one location).
many-to-many between group and user (a user can be a member of many groups and a group can have many members).
Same as #2 above for desc and user.
My tables are created as follows:
Base = declarative_base()
class Location(Base):
__tablename__ = 'location'
id = Column(Integer, primary_key=True)
name = Column(String)
group_user_association_table = Table('group_user_association_table', Base.metadata,
Column('group_id', Integer, ForeignKey('group.id')),
Column('user_id', Integer, ForeignKey('user.id')))
class Group(Base):
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
name = Column(String)
users = relationship('User', secondary=group_user_association_table, backref='group')
desc_user_association_table = Table('desc_user_association', Base.metadata,
Column('desc_id', Integer, ForeignKey('desc.id')),
Column('user_id', Integer, ForeignKey('user.id')))
class Desc(Base):
__tablename__ = 'desc'
id = Column(Integer, primary_key=True)
name = Column(String)
users = relationship('User', secondary=desc_user_association_table, backref='desc')
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
user_name = Column(String)
location_id = Column(Integer, ForeignKey('location.id'))
groups = Column(String, ForeignKey('group.id'))
descs = Column(String, ForeignKey('desc.id'))
location = relationship('Location', backref='user')
Here are some examples as to how I am creating the data (all being scraped from the web):
location = Location(id=city[1], name=city[0]) #city = ('name', id)
profile = User()
profile.id = int(str(span2class[0].a['href'][7:]))
profile.user_name = str(span2class[0].a.img['alt'])
profile.location_id = location.id
g = Group(id=gid, name=str(group.contents[0])) # add the group to the Group table
self.db_session.add(g)
# Now add the gid to a list that will be added to the profile that eventually gets added to the user table
profile.groups.append(str(gid)) # stick the gid into the list
profile.groups = ','.join(profile.groups) # convert list to csv string
# Repeat basically same thing above for desc
self.db_session.add(profile)
self.db_session.commit()
As far as queries go, I've got some of the basic ones working such as:
for instance in db_session.query(User).all():
print instance.id, instance.user_name
But when it comes to performing a join to get (for example) group.id and group.name for a specific user.id... nothing I've tried has worked. I am guessing that the form would be something like the following:
db_session.query(User, Group).join('users').filter(User.id==42)
but that didn't work.
Joins works from left to right, so you should join on the relationship from User to Group:
db_session.query(User, Group).join(User.group).filter(User.id == 42)
But this return you a list of tuples (<User>, <Group>), so if the user belongs to 2 or more groups, you will receive 2 or more rows.
If you really want to load both the user and its groups in one (SQL) query, a better way would be to load a user, but configure query to preload groups:
u = (session.query(User)
.options(joinedload(User.group))
.get(42)
)
print("User = {}".format(u))
for g in u.group:
print(" Group = {}".format(g))
I'm new with sqlalchemy and I want to do this as simply as possible, yet correctly. I want to track domain use across multiple companies on a monthly basis, so I set up the following tables:
class Company(Base):
__tablename__ = 'company'
id = Column(Integer, primary_key = True)
name = Column('name', String)
class Domains(Base):
__tablename__ = 'domains'
id = Column(Integer, primary_key=True)
name = Column('name', String, unique=True)
class MonthlyUsage(Base):
'''
Track domain usage across all
companies on a monthly basis.
'''
__tablename__ = 'monthlyusage'
month = Column(DateTime)
company_id = Column(Integer, ForeignKey('company.id'))
domain_id = Column(Integer, ForeignKey('domains.id'))
# <...other columns snipped out...>
company = relationship('Company', backref='company_assoc')
domain = relationship('Domains', backref='domain_assoc')
This works fine, until I add usage details for the second month. Then I get duplicate key value errors:
*sqlalchemy.exc.IntegrityError: (IntegrityError) duplicate key value violates unique constraint "monthlyusage_pkey"*
Does this mean I have to split out the "monthlyusage" into a third table? That seems unnecessarily complicated, since all that needs to be unique is the month, company_id, and domain_id fields.
Any suggestions for my layout here, to keep it as simple as possible, yet still correct?
TIA!
Ok, I needed to add a primary key column to MonthlyUsage. The code below now works...
class MonthlyUsage(Base):
'''
Track domain usage across all
companies on a monthly basis.
'''
__tablename__ = 'monthlyusage'
month = Column(DateTime)
month_id = Column(Integer, primary_key=True)
company_id = Column(Integer, ForeignKey('company.id'), primary_key=True)
domain_id = Column(Integer, ForeignKey('domains.id'), primary_key=True)
# <...other columns snipped out...>
company = relationship('Company', backref='company_assoc')
domain = relationship('Domains', backref='domain_assoc')
I created a many to many relationship with sqlalchemy like this:
subject_books = Table('subject_books', Base.metadata,
Column('subject_id', Integer, ForeignKey('subjects.id')),
Column('book_id', Integer, ForeignKey('books.id')),
Column('group', Integer)
)
class Subject(Base):
__tablename__ = 'subjects'
id = Column(Integer, primary_key=True)
value = Column(Unicode(255), unique=True)
class Book(Base):
__tablename__ = 'books'
id = Column(Integer, primary_key=True)
title = Column(Unicode(255))
isbn = Column(Unicode(24))
subjects = relationship('Subject', secondary=subject_books, collection_class=attribute_mapped_collection('group'), backref='books')
after that I created a test like following:
book = Book(title='first book',isbn='test')
book.subjects[0] = Subject(value='first subject')
book.subjects[1] = Subject(value='second subject')
session.add(book)
transaction.commit()
and it works fine. But what I really want is to store more than one subject with the same group value, so I tried the following test:
book = Book(title='first book',isbn='test')
book.subjects[0] = [Subject(value='first subject'),Subject(value='second subject')]
book.subjects[1] = [Subject(value='third subject'),Subject(value='forth subject')]
session.add(book)
transaction.commit()
but it does not work.
Can this be done using sqlalchemy?
Thanks in Advance
Razi
I think you are constructing wrong relation ship.
Your relation ship must be
book M2M subject
subject M2M group
So you have to create one more model for group and that must be assign as m2m in Subject
Your models will be like.
subject_books = Table('subject_books', Base.metadata,
Column('subject_id', Integer, ForeignKey('subjects.id')),
Column('book_id', Integer, ForeignKey('books.id')),
)
subject_group = Table('subject_groups', Base.metadata,
Column('group_id', Integer, ForeignKey('groups.id')),
Column('subject_id', Integer, ForeignKey('subjects.id')),
)
class Subject(Base):
__tablename__ = 'subjects'
id = Column(Integer, primary_key=True)
value = Column(Unicode(255), unique=True)
groups = relationship('Groups', secondary=subject_groups, backref='subjects')
class Groups(Base):
__tablename__ = 'groups'
id = Column(Integer, primary_key=True)
name = Column(Unicode(255), unique=True)
class Book(Base):
__tablename__ = 'books'
id = Column(Integer, primary_key=True)
title = Column(Unicode(255))
isbn = Column(Unicode(24))
subjects = relationship('Subject', secondary=subject_books, backref='books')
I also check the docs for attribute_mapped_collection. But each time I found that each key is associated with only one object not more then one. If you read anywhere then please provide the link so I can check that how it will be fit in your code.
I think this will be help you.