EDIT:
Please excuse me, as I have just realised I've made an error with the example below. Here's what I want to achieve:
Say I have the three tables as described below. When a user enters a query, it will search all three tables for results where name is LIKE %query%, but only return unique results.
Here's some example data and output:
Data:
**Grandchild:**
id: 1
name: John
child_id: 1
**Grandchild:**
id: 2
name: Jesse
child_id: 2
**Child:**
id: 1
name: Joshua
parent_id: 1
**Child:**
id: 2
name: Jackson
parent_id: 1
**Parent:**
id: 1
name: Josie
If a user searches for "j" it will return the two Grandchild entries: John and Jesse.
If a user searches for "j, Joshua", it will return only the Grandchildren who's child is Joshua - in this case, only John.
Essentially, I want to search for all the Grandchild entries, and then if the user types in more key words, it will filter those Grandchildren down based on their related Child entry's name. "j" will return all grandchildren starting with "j", "j, Josh" will return all grandchildren starting with "j" and whom have their Child LIKE %Josh%.
So, I have a setup like this:
Grandchild{
id
name
child_id
}
Child{
id
name
parent_id
}
Parent{
id
name
}
Grandchild is linked/mapped to child. Child is mapped to Parent.
What I want to do, is something like below, where I search all three databases at once:
return Grandchild.query.filter(or_(
Grandchild.name.like("%" + query + "%"),
Grandchild.child.name.like("%" + query + "%"),
Grandchild.child.parent.name.like("%" + query + "%")
)).all()
Obviously the query above is incorrect, and returns an error:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object has an attribute 'name'
What would the correct way to go about what I'm attempting be?
I am running MySQL, Flask-SQLAlchemy (which I believe extends SQLAlchemy), Flask.
As for me, it is better to modify your data model (if it is possible).
You can create a self-referenced table 'People' like that:
People
{
id,
name,
parent_id,
grandparent_id,
}
class People(Base):
__tablename__ = "people"
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(Unicode(255), nullable=False)
parent_id = Column(Integer, ForeignKey('people.id')) # parent in hierarchy
grandparent_id = Column(Integer, ForeignKey('people.id')) # grandparent in hierarchy
# relationships
parent = relationship("People", primaryjoin="People.parent_id==People.id",
remote_side=[id])
grandparent = relationship("People", primaryjoin="People.grandparent_id==People.id",
remote_side=[id])
Then the things get more obvious:
session.query(People).filter(People.name.like("%" + query + "%"))
Related
I have a table with hierarchical data where each row contains an id, parent_id, level which is the depth, and also a path enumeration hierarchy_hint which is a plain string with JSON-like formatting. For example:
id
parent_id
level
hierarchy_hint
1
null
'level1'
'{"level1": "root"}'
2
1
'level2'
'{"level1": "root", "level2: "shoes"}'
3
2
'level3'
'{"level1": "root", "level2: "shoes", "level3": "athletic"}'
(For the sake of example I used integers, but the ids are UUIDs)
I'd like to use hierarchy_hint to get all descendants of a given Node at a particular level. The model already has a self referential relationship established to get direct descendants (children). The model looks like:
class Node(db.Model):
id = db.Column(UUID(as_uuid=True), primary_key=True)
parent_id = db.Column(UUID(as_uuid=True), db.ForeignKey('node.id'))
children = db.relationship('Node')
level = db.Column(db.String(length=4000), primary_key=False)
hierarchy_hint = db.Column(db.String(length=200), primary_key=False)
I've tried adding to the model:
level_threes = relationship(
"Node", primaryjoin=and_(
foreign(hierarchy_hint).like(func.regexp_replace(hierarchy_hint, '}', ',%%')),
level == 'level3'), viewonly=True)
But I get the error:
sqlalchemy.exc.ArgumentError: Relationship Node.level_threes could not determine any unambiguous local/remote column pairs based on join condition and remote_side arguments. Consider using the remote() annotation to accurately mark those elements of the join condition that are on the remote side of the relationship.
I suspect that because I'm modifying the hierarchy_hint using regexp_replace I should be using a secondary table somehow but I'm not experienced enough with SQLAlchemy to know what exactly to do.
I'm open also to alternative suggestions for how to do this using recursive queries or otherwise but I would like to be able to access the property in the same way as I access children to avoid refactoring elsewhere.
I'm using Flask, SQLAlchemy, and PostgreSQL.
In plain SQL what I'm trying to do might look like this:
with ancestor as (select hierarchy_hint from node where id='2')
select node.id from node, ancestor
where node.hierarchy_hint and node.level='level4';
Here we would expect to get all the level4 items within the shoes tree but no level4 items from any other trees nor any level5 etc. items from shoes.
I actually think I figured out what you are trying to do. I think you can do something like below. You are probably aware but you might want to double check that your path in hierarchy_hint is deterministic because I'm not sure regular JSON puts the keys in the same order all the time. It seemed to work with dumps when I tested it though. I tried to add some data to make sure it wasn't returning everything in the same level.
Aliased relationship
class Node(Base):
__tablename__ = 'nodes'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('nodes.id'))
parent = relationship('Node', backref='children', remote_side='nodes.c.id')
level = Column(String(length=200), primary_key=False)
name = Column(String(length=200), primary_key=False)
hierarchy_hint = Column(String(length=4000), primary_key=False)
# After class is constructed.
aliased_node = aliased(Node)
# Attach the relationship after class is constructed so we can use alias.
Node.level_threes = relationship(
aliased_node, primaryjoin=and_(
aliased_node.hierarchy_hint.like(func.regexp_replace(Node.hierarchy_hint, '}', ',%%')),
aliased_node.level == 'level3'),
viewonly=True,
uselist=True,
remote_side=[aliased_node.level, aliased_node.hierarchy_hint],
foreign_keys=Node.hierarchy_hint)
Base.metadata.create_all(engine)
node_values = [
('level1', 'root', {"level1": "root"}),
('level2', 'shoes', {"level1": "root", "level2": "shoes"}),
('level3', 'athletic shoes', {"level1": "root", "level2": "shoes", "level3": "athletic"}),
]
node_values2 = [
('level2', 'shirts', {"level1": "root", "level2": "shirts"}),
('level3', 'athletic shirts', {"level1": "root", "level2": "shirts", "level3": "athletic"}),
]
def build_branch(session, parent, node_values):
nodes = []
for level, name, value in node_values:
parent = Node(level=level, name=name, parent=parent, hierarchy_hint=dumps(value))
session.add(parent)
nodes.append(parent)
return nodes
with Session(engine) as session:
nodes = []
# Build the original branch.
nodes.extend(build_branch(session, None, node_values))
# Now attach another branch below root.
nodes.extend(build_branch(session, nodes[0], node_values2))
print(len(nodes))
session.commit()
# Print out to check structure
nodes = session.query(Node).order_by(Node.id).all()
for node in nodes:
print(node.id, node.name, node.parent_id, node.level, node.hierarchy_hint)
shoes = session.query(Node).filter(Node.name == 'shoes').first()
athletic_shoes = session.query(Node).filter(Node.name == 'athletic shoes').first()
athletic_shirts = session.query(Node).filter(Node.name == 'athletic shirts').first()
# Check relationship...
assert athletic_shoes in shoes.level_threes, "Athletic shoes should be below shoes"
assert athletic_shirts not in shoes.level_threes, "Athletic shirts should not be below shoes"
Output
1 None level1 {"level1": "root"}
2 1 level2 {"level1": "root", "level2": "shoes"}
3 2 level3 {"level1": "root", "level2": "shoes", "level3": "athletic"}
4 3 level2 {"level1": "root", "level2": "shirts"}
5 4 level3 {"level1": "root", "level2": "shirts", "level3": "athletic"}
I have two classes:
Trade
TradeItem
One Trade has multiple TradeItems. It's a simple 1-to-many relationship.
Trade
class Trade(Base):
__tablename__ = 'trade'
id = Column(Integer, primary_key=True)
name= Column(String)
trade_items = relationship('TradeItem',
back_populates='trade')
TradeItem
class TradeItem(Base):
__tablename__ = 'trade_item'
id = Column(Integer, primary_key=True)
location = Column(String)
trade_id = Column(Integer, ForeignKey('trade.id'))
trade = relationship('Trade',
back_populates='trade_items')
Filtering
session.query(Trade).filter(Trade.name == trade_name and TradeItem.location == location)
I want to get the Trade object by name and only those Trade Items where the location matches the given location name.
So if there is a trade object: Trade(name='Trade1') and it has 3 items:
TradeItem(location='loc1'),TradeItem(location='loc2'),TradeItem(location='loc2')
Then the query should return only my target trade items for the trade:
session.query(Trade).filter(Trade.name == 'Trade1' and TradeItem.location == 'loc1').first()
Then I expect the query to return Trade with name Trade1 but only one Trade item populated with location as loc1
But when I get the trade object, it has all of the Trade Items and it completely ignores the filter condition.
You can use the contains_eager option to to load only the child rows selected by the filter.
Note that to create a WHERE ... AND ... filter you can use sqlalchemy.and_, or chain filter or filter_by methods, but using Python's logical and will not work as expected.
from sqlalchemy import orm, and_
trade = (session.query(Trade)
.filter(and_(Trade.name == 'Trade1', TradeItem.location == 'loc1'))
.options(orm.contains_eager(Trade.trade_items))
.first())
The query generates this SQL :
SELECT trade_item.id AS trade_item_id, trade_item.location AS trade_item_location, trade_item.trade_id AS trade_item_trade_id, trade.id AS trade_id, trade.name AS trade_name
FROM trade
JOIN trade_item ON trade.id = trade_item.trade_id
WHERE trade.name = 'trade1' AND trade_item.location = 'loc1'
LIMIT 1 OFFSET 0
In a single model Person there is an address information. But since we are not separating this yet to another table. I would like to only query the address information out of Person table. Would it be possible using hybrid_property If not what else do I need to achieve this stuff?
I wanna avoid this one:
db.session.query(Person.id, Person.address_secret_id, Person.address_name).get(pk)
The model
class Person(db.Model):
# some lengthy information
# address
address_secret_id = db.Column(db.Unicode, nullable=True)
address_name = db.Column(db.Unicode, nullable=True)
#hybrid_property
def address(self):
# I never tested this but i know this is wrong.
return self.id + self.address_secret_id + self.address_name
Usage:
db.session.query(Person.address).get(pk)
Expected Output:
{id: 1, address_secret_id: xxxx, address_name: 'forgetmeland'}
How can I achieve an output that is only retrieving the desired field? It doesn't need to be dict or tuple as long as Im getting what is needed.
If you are trying to avoid having to type db.session.query(Person.id, Person.address_secret_id, Person.address_name), just add an address_details property on the person model.
class Person(db.Model):
# some lengthy information
# address
address_secret_id = db.Column(db.Unicode, nullable=True)
address_name = db.Column(db.Unicode, nullable=True)
#property
def address_details(self):
keys = ('id', 'address_secret_id', 'address_name')
return {k: getattr(self, k) for k in in keys}
Probably less lines of code than trying to use some sort of hybrid query, and still just the one trip to the database.
Query would be:
Person.query.get(1).address_details
I've been using SQLAlchemy 0.9.2 with Python Version 2.7.3 and have run into a bit of an odd problem that I can't quite seem to explain. Here is my relevant code:
Base = declarative_base()
class Parent(Base):
__tablename__ = 'parents'
__table_args__ = (UniqueConstraint('first_name', 'last_name', name='_name_constraint'),)
id = Column(Integer, primary_key=True)
first_name = Column(String(32), nullable=False)
last_name = Column(String(32), nullable=False)
children = relationship(Child, cascade='all,delete', backref='parent')
## Constructors and other methods ##
class Child(Base):
__tablename__ = 'children'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('parents.id'))
foo = Column(String(32), nullable=False)
## Constructors and other methods ##
So a pretty basic set of models. The problem I'm experiencing is that I want to add a child to a parent that is saved to the database. The kicker is that the child is currently related to a parent that is not in the database. Consider the following example:
database_engine = create_engine("mysql://user:password#localhost/db", echo=False)
session = scoped_session(sessionmaker(autoflush=True,autocommit=False))
p1 = Parent("Foo", "Bar") # Create a parent and append a child
c1 = Child("foo")
p1.children.append(c1)
session.add(p1)
session.commit() # This works without a problem
db_parent = session.query(Parent).first()
db_parent.children.append(Child("bar"))
session.commit() # This also works without a problem
p2 = Parent("Foo", "Bar")
c3 = Child("baz")
p2.children.append(c3)
db_parent = session.query(Parent).first()
db_parent.children.append(p2.children[0])
session.commit() # ERROR: This blows up
The error I'm receiving is that I'm breaking an integrity Constraint, namely '_name_constraint'. SQLAlchemy is telling me that is trying to insert a Parent with the same information. My question is, why in the world is it trying to add a secondary parent?
These are the steps I've taken so far and don't have a good answer for:
I've inspected db_parent.children[2] It points to the same memory address as p1 once I have appended it to the list
I've inspected p2.children after the append. Oddly, p2 has no children once I have appended its child to db_parent I think this has something to do with what is going on, I just don't understand why its happening
Any help would be much appreciated, as I simply don't understand what's going on here. If you need me to post more please let me know. Thanks in advance.
Okay, after some more digging I think I have found a solution to my problem, but I don't yet have the answer as to why its happening the way it is, but I think I may have a guess. The solution I discovered was to use session.expunge(p2) before session.commit()
I started exploring SQLAlchemy's Internals, particularly, the instance state. I found that once you add the child to the parent, the original parent's state becomes pending. Here is an example:
from sqlalchemy import inspect
p2 = Parent("Foo", "Bar")
p2_inst = inspect(p2)
c3 = Child("Baz")
c3_inst = inspect(c3)
db_parent = session.query(Parent).first()
db_parent_inst = inspect(db_parent)
print("Pending State before append:")
print("p2_inst : {}".format(p2_inst.pending))
print("c3_inst : {}".format(c3_inst.pending))
print("db_parent_inst : {}".format(db_parent_inst.pending))
db_parent.children.append(p2.children[0])
print("Pending State after append:")
print("p2_inst : {}".format(p2_inst.pending))
print("c3_inst : {}".format(c3_inst.pending))
print("db_parent_inst : {}".format(db_parent_inst.pending))
session.expunge(p2)
print("Pending State after expunge:")
print("p2_inst : {}".format(p2_inst.pending))
print("c3_inst : {}".format(c3_inst.pending))
print("db_parent_inst : {}".format(db_parent_inst.pending))
session.commit()
The result of running this will be:
Pending State before append:
p2_inst : False
c3_inst : False
db_parent_inst : False
Pending State after append:
p2_inst : True
c3_inst : True
db_parent_inst : False
Pending State after expunge:
p2_inst : False
c3_inst : True
db_parent_inst : False
And there you have it. Once I thought about it a bit, I suppose it makes sense. There is no reason for the db_parent to ever enter a "pending" state because, you're not actually doing anything to the record in MySQL. My guess on why p2 becomes pending is due to an order of operations? In order for c3 to become pending, then all of its relationships must exist (to include p2) and so even when you change the child's parent, the session still think that it needs to add the parent.
I'd love for someone more knowledgeable on SQLAlchemy to correct me, but to the best of my knowledge, that's my best explanation :)
This question already has an answer here:
Sqlalchemy - Can we use date comparison in relation definition?
(1 answer)
Closed 2 years ago.
I am working on defining my mapping with SQLAlchemy and I am pretty much done except one thing.
I have a 'resource' object and an association table 'relation' with several properties and a relationship between 2 resources.
What I have been trying to do almost successfully so far, is to provide on the resource object 2 properties: parent and children to traverse the tree stored by the association table.
A relation between 2 properties only last for a while, so there is a start and end date. Only one resource can be the parent of another resource at a time.
My problem is that if I expire one relation and create a new one, the parent property is not refreshed. I am thinking maybe there an issue with the primaryjoin for the parent property of resource.
Here is some code:
resource_table = model.tables['resource']
relation_table = model.tables['resource_relation']
mapper(Resource, resource_table,
properties = {
'type' : relation(ResourceType,lazy = False),
'groups' : relation(Group,
secondary = model.tables['resource_group'],
backref = 'resources'),
'parent' : relation(Relation, uselist=False,
primaryjoin = and_(
relation_table.c.res_id == resource_table.c.res_id,
relation_table.c.end_date > func.now())),
'children' : relation(Relation,
primaryjoin = and_(
relation_table.c.parent_id == resource_table.c.res_id,
relation_table.c.end_date > func.now()))
}
)
mapper(Relation, relation_table,
properties = {
'resource' : relation(Resource,
primaryjoin = (relation_table.c.res_id == resource_table.c.res_id)),
'parent' : relation(Resource,
primaryjoin = (relation_table.c.parent_id == resource_table.c.res_id))
}
)
oldrelation = resource.parent
oldrelation.end_date = datetime.today()
relation = self.createRelation(parent, resource)
# Here the relation object has not replaced oldrelation in the resource object
Any idea ?
Thanks,
Richard Lopes
Consider using >= instead of > in date comparison.