Still wrapping my head around SqlAlchemy and have run into a few issues. Not sure if it is because I am creating the relationships incorrectly, querying incorrect, or both.
The general idea is...
one-to-many from location to user (a location can have many users but users can only have one location).
many-to-many between group and user (a user can be a member of many groups and a group can have many members).
Same as #2 above for desc and user.
My tables are created as follows:
Base = declarative_base()
class Location(Base):
__tablename__ = 'location'
id = Column(Integer, primary_key=True)
name = Column(String)
group_user_association_table = Table('group_user_association_table', Base.metadata,
Column('group_id', Integer, ForeignKey('group.id')),
Column('user_id', Integer, ForeignKey('user.id')))
class Group(Base):
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
name = Column(String)
users = relationship('User', secondary=group_user_association_table, backref='group')
desc_user_association_table = Table('desc_user_association', Base.metadata,
Column('desc_id', Integer, ForeignKey('desc.id')),
Column('user_id', Integer, ForeignKey('user.id')))
class Desc(Base):
__tablename__ = 'desc'
id = Column(Integer, primary_key=True)
name = Column(String)
users = relationship('User', secondary=desc_user_association_table, backref='desc')
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
user_name = Column(String)
location_id = Column(Integer, ForeignKey('location.id'))
groups = Column(String, ForeignKey('group.id'))
descs = Column(String, ForeignKey('desc.id'))
location = relationship('Location', backref='user')
Here are some examples as to how I am creating the data (all being scraped from the web):
location = Location(id=city[1], name=city[0]) #city = ('name', id)
profile = User()
profile.id = int(str(span2class[0].a['href'][7:]))
profile.user_name = str(span2class[0].a.img['alt'])
profile.location_id = location.id
g = Group(id=gid, name=str(group.contents[0])) # add the group to the Group table
self.db_session.add(g)
# Now add the gid to a list that will be added to the profile that eventually gets added to the user table
profile.groups.append(str(gid)) # stick the gid into the list
profile.groups = ','.join(profile.groups) # convert list to csv string
# Repeat basically same thing above for desc
self.db_session.add(profile)
self.db_session.commit()
As far as queries go, I've got some of the basic ones working such as:
for instance in db_session.query(User).all():
print instance.id, instance.user_name
But when it comes to performing a join to get (for example) group.id and group.name for a specific user.id... nothing I've tried has worked. I am guessing that the form would be something like the following:
db_session.query(User, Group).join('users').filter(User.id==42)
but that didn't work.
Joins works from left to right, so you should join on the relationship from User to Group:
db_session.query(User, Group).join(User.group).filter(User.id == 42)
But this return you a list of tuples (<User>, <Group>), so if the user belongs to 2 or more groups, you will receive 2 or more rows.
If you really want to load both the user and its groups in one (SQL) query, a better way would be to load a user, but configure query to preload groups:
u = (session.query(User)
.options(joinedload(User.group))
.get(42)
)
print("User = {}".format(u))
for g in u.group:
print(" Group = {}".format(g))
Related
I'm new to the SQLAlchemy and I wrote a simple CRUD database layout like this: I have three tables, Customer, Product, and Template. The idea is this: Each customer can have a template of the products he usually orders. When I fetch a particular customer from the database, his template along with all the products should be fetched as well. So I have a one-to-one relationship (customer-template) and one many-to-many relationship (template-product). Now, a template should contain fields such as quantity of a particular product, along with its net, gross and tax values. I'd like to have a listener on the quantity column, such that when the quantity of a particular product is changed, other attributes I mentioned will be changed too. So the code I wrote is as follows (please, if you can, also verify whether all the relationships are written appropriately)
from sqlalchemy import event
from sqlalchemy.orm import relationship, exc, column_property, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm.query import Query as _Query
from sqlalchemy import Table, Column, Integer, String, Boolean, ForeignKey, UniqueConstraint, create_engine, Numeric
from decimal import *
# decimal operations settings
getcontext().prec = 6
getcontext().rounding = ROUND_HALF_UP
engine = create_engine('sqlite:///test.db')
Base = declarative_base()
# Initalize the database if it is not already.
Base.metadata.create_all(engine)
# Create a session to handle updates.
Session = sessionmaker(bind=engine)
# customer has a one-to-one relationship with template, where customer is the parent
class Customer(Base):
__tablename__ = "customers"
id = Column(Integer, primary_key=True)
alias = Column(String)
firm_name = Column(String)
last_name = Column(String)
first_name = Column(String)
tax_id = Column(String, nullable=False)
address = Column(String)
postal_code = Column(String)
city = Column(String)
payment = Column(Boolean)
template = relationship("Template", uselist=False, back_populates="customer")
# product has many-to-many relationship with template, where template is the parent
class Product(Base):
__tablename__ = "products"
id = Column(Integer, primary_key=True)
product_name = Column(String, nullable=False)
symbol = Column(String)
unit = Column(String, nullable=False)
unit_net_price = Column(Numeric, nullable=False)
vat_rate = Column(Numeric, nullable=False)
per_month = Column(Boolean, nullable=False)
# association table for the products-template many-to-many relationship
association_table = Table('association', Base.metadata,
Column('product_id', Integer, ForeignKey('product.id')),
Column('template_id', Integer, ForeignKey('template.id'))
)
# template implements one-to-one relationship with customer and many-to-many relationship with product
class Template(Base):
id = Column(Integer, primary_key=True)
customer_id = Column(Integer, ForeignKey("customer.id"))
customer = relationship("Customer", back_populates="template")
products = relationship("Product", secondary=association_table)
quantity = Column(Numeric)
net_val = Column(Numeric)
tax_val = Column(Numeric)
gross_val = Column(Numeric)
# TODO: implement constructor
def __init__(self, **kwargs):
self.quantity = Decimal(0.0)
self.net_val = Decimal(0.0)
self.tax_val = Decimal(0.0)
self.gross_val = Decimal(0.0)
#event.listens_for(Template.quantity, "set")
def quantity_listener(target, value, oldvalue, initiator):
print(target)
print(initiator)
# target.net_val =
# target.tax_val =
# target.gross_val =
Now, I'm unsure how should I get a particular initiator and set its values, since products in Template table is a list (?)
I'd probably do it like this. It's hard to use Sqlalchemy orm in such cases as you can't access the Session object we all got used to use.
#event.listens_for(Template, "after_update")
def quantity_listener(mapper, connection, target):
field = 'quantity'
added, _, deleted = get_history(target, field)
# added is a new values of the specified field
# deleted is the old values
# so we basically check if 'quantity' has changed
# its a tuple btw
if added and deleted and added[0] != deleted[0]:
stmt = Template.update(). \
values(Template.something=Template.something * Template.other_something). \
where(Template.id == target.id)
conn.execute(stmt)
# here goes another updates for products or you can have another
# listener for them
There might be a better way to accomplish this. I can't debug this right now and I can't manage to get your examples working. Hope it helps.
I am having trouble constructing a query using SQLalchemy. Here is a simplified representation of the models I have defined:
Models
Project
class Project(Base):
__tablename__ = 'project'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False, unique=True)
# User associations
users = relationship(
'User',
secondary='user_project_association'
)
User
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False, unique=True)
# Project associations
projects = relationship(
'Project',
secondary='user_project_association'
)
User <-> Project (association)
class UserProjectAssociation(Base):
__tablename__ = 'user_project_association'
# User association.
user_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
user = relationship('User', backref='project_associations')
# Project association.
project_id = Column(Integer, ForeignKey('project.id'), primary_key=True)
project = relationship('Project', backref='user_associations')
Query
I want to perform a query on the projects table such that the result contains information about the projects as well as information about the associated users - if there are any. I am including a filter based on the user name. I am eventually going to send the result as JSON via a REST API so I would prefer the results as python {dict} objects rather than SQLAlchemy objects. The query I am performing looks like:
# Add return fields
query = session.query(
Project.id,
Project.name,
User.id.label('users.id'),
User.name.label('users.name')
)
# Add join statements
query = query.outerjoin(User, Project.users)
# Add filters
query = query.filter(
Project.name == 'proj1',
User.name != 'jane.doe' # <--- I think this is causing the issue.
)
# Execute
results = query.all()
data = [result._asdict() for result in results]
print(data)
Results
The database contains a project called proj1 which doesn't have any associated users. In this particular scenario, I am filtering on a user column and the user association does not exist. However, I am still expecting to get a row for the project in my results but the query returns an empty list. The result I am expecting would look something like this:
[{'id': 1, 'name': 'proj1', 'users.id': None, 'users.name': None}]
Can someone explain where I am going wrong?
You have to account for the NULL values that result from the left join, since != compares values and NULL is the absence of value, so the result of NULL != 'jane.doe' is NULL, not true:
query = query.filter(
Project.name == 'proj1',
or_(User.name == None, User.name != 'jane.doe')
)
Note that SQLAlchemy handles equality with None in a special way and produces IS NULL. If you want to be less ambiguous you could also use User.name.is_(None).
So I would like to have users add an item and an arbitrary category. Right now I use if statements to make sure that if the category has been created already, not to add it again. Is there a better way to make use of SQLAlchemy relationships so that I could skip some of the logic I had to write to ensure that the categories are unique?
Here are the model's I used:
class Category(Base):
__tablename__ = 'category'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
class Item(Base):
__tablename__ = 'item'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
description = Column(String)
category_id = Column(Integer, ForeignKey('category.id'))
category = relationship(Category)
date_created = Column(DateTime)
date_updated = Column(DateTime)
user_id = Column(Integer, ForeignKey('user.id'))
user = relationship(User)
Here is an example of how I would edit an item:
if new_category_name != category.name:
if db_session.query(Category).\
filter_by(name=new_category_name).count() == 0:
new_category = Category(name=new_category_name)
else:
new_category = db_session.query(Category)\
.filter_by(name=new_category_name).one()
is_last_of_category = db_session.query(Item)\
.filter_by(category_id=item.category_id).count() == 1
if is_last_of_category:
db_session.delete(category)
item.category = new_category
db_session.commit()
Any other suggestions you are willing to make I am happy to listen to.
Use the unique constraint,
Quoting from sqlalchemy's docs
unique – When True, indicates that this column contains a unique
constraint, or if index is True as well, indicates that the Index
should be created with the unique flag. To specify multiple columns in
the constraint/index or to specify an explicit name, use the
UniqueConstraint or Index constructs explicitly.
Example from sqlalchemy documentation:
from sqlalchemy import UniqueConstraint
meta = MetaData()
mytable = Table('mytable', meta,
# per-column anonymous unique constraint
Column('col1', Integer, unique=True),
Column('col2', Integer),
Column('col3', Integer),
# explicit/composite unique constraint. 'name' is optional.
UniqueConstraint('col2', 'col3', name='uix_1')
)
I have a problem with SQL Alchemy, while trying to think about an SQL schema I encountered the following problem.
My schema is based on 2 classes, Flight and Trip.
A Trip includes 2 fields: flights_to and flights_from.
Any of the fields is basically a list of flights, it could be made of one flight, or many flights (Connection flights).
class Trip(Base):
__tablename__ = "Trip"
__table_args__ = {'sqlite_autoincrement': True}
id = Column(Integer, primary_key = True)
flights_to = relationship("Flight", backref="Trip")
flights_from = relationship("Flight", backref="Trip")
class Flight(Base):
__tablename__ = "Flight"
__table_args__ = {'sqlite_autoincrement': True}
id = Column(Integer, primary_key = True)
arrival_airport = Column(String(20))
departure_airport = Column(String(20))
flight_number = Column(Integer)
trip_id = Column(Integer, ForeignKey('Trip.id'))
The problem happens when I create 2 fields in the same type:
sqlalchemy.exc.ArgumentError: Error creating backref 'Trip' on relationship 'Trip.flights_from': property of that name exists on mapper 'Mapper|Flight|Flight'
I have thought about using 2 inheriting classes of types FlightTo and FlightFrom and saving them at two different tables, but what if I want to use a FlightFrom as a FlightTo? will the flight be duplicated in 2 tables?
I would appreciate your help.
backref is used to define a new property on the other class you are using relationship with. So you can't have two property which have the same name
You should rename your backref for the flights_from to any other name than Trip.
It will work then.
For Example:
class Person(Model):
id = Column(Integer, primary_key=True)
name = Column(String)
address = relationship("Address",backref="address")
class Address(Model):
id = Column(Integer, primary_key=True)
house_no = Column(Integer)
person_id = Column(Integer, ForeignKey('person.id'))
So you can access the person name with house_no 100 by:
query_address = Address.query.filter_by(house_no=100).first()
person = query_address.address
This returns you the person object.
Thus if you have multiple such names , it will give you an error
I have two tables, users and contacts. I query the contacts table and get a list of a user's contacts. I would then like to be able to write Contact.first_name (where first_name is a row from the users table) and print out that contact's first name.
Currently, my Contact object does not recognize any attributes of the user table.
Here is some code:
class User(Base):
""" Basic User definition """
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
first_name = Column(Unicode(255))
last_name = Column(Unicode(255))
contacts = relationship('Contact', backref='users')
class Contact(Base):
__tablename__ = 'contacts'
id = Column(Integer, primary_key=True)
user_id = Column(Integer)
contact_id = Column(Integer)
__table_args__ = (ForeignKeyConstraint([id], [User.id]), {})
Here is my query:
Contact.query.filter(Contact.user_id == self.user_id).filter(Contact.state == True).all()
To be honest, I'm unsure of how to properly map my two foreign keys Contact.user_id and Contact.contact_id to the User.id row. Maybe this is the source of my problem?
I'm very new to using SQLAlchemy, so this is a learning experience here. Thanks for your help.
What you have here is class User which essentially refers to itself. In other words, it's a self-referential many-to-many relationship. Your model definitions should look like this:
# This is so called association table, which links two tables in many-to-many
# relationship. In this case it links same table's ('users') different rows.
user_contacts = Table(
'user_contacts', Base.metadata,
Column('user_id', Integer, ForeignKey('users.id'), primary_key=True),
Column('contact_id', Integer, ForeignKey('users.id'), primary_key=True),
)
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
contacts = relationship(
'User',
secondary=user_contacts,
primaryjoin=id==user_contacts.c.user_id,
secondaryjoin=id==user_contacts.c.contact_id
)
Then you can do things like the following:
u1 = User(first_name='Foo', last_name='Foo')
u2 = User(first_name='Bar', last_name='Bar')
u3 = User(first_name='Baz', last_name='Baz')
u1.contacts = [u2, u3]
session.add(u1)
session.commit()
# ... and in some other place in your code ...
u = User.query.get(1)
print u.contacts[0].first_name