Entities:
class Car(db.Entity):
image = Set("Image")
name = Required(str, 40,)
class Image(db.Entity):
name = Required(str, unique=True)
car = Required(Car)
And I want to get information on cars-list page: one car - one image of that car.
I use:
cars = select((car, car.image) for car in Car).distinct().show()
I have
Car[1]|Image[1]
Car[1]|Image[2]
...
Car[5]Image[1]
Car[5]Image[2]
...
But I need only first image of each car, not all images.
How to do this?
Thanks!
Code to get only first image for each car (image with lowest id):
#!/usr/bin/env python
from pony.orm import *
db = Database()
db.bind('sqlite', ':memory:', create_db=True)
class Car(db.Entity):
image = Set("Image", lazy=True)
name = Required(str, 40)
class Image(db.Entity):
name = Required(str, unique=True)
car = Required(Car)
db.generate_mapping(create_tables=True)
sql_debug(True)
with db_session:
car1 = Car(name='car1')
car2 = Car(name='car2')
image1 = Image(name='image1', car = car1)
image2 = Image(name='image2', car = car1)
image3 = Image(name='image3', car = car2)
cars = select((car, car.name, image, image.name) for car in Car for image in car.image if image.id == min(car.image.id)).show()
Result:
GET CONNECTION FROM THE LOCAL POOL
SWITCH TO AUTOCOMMIT MODE
BEGIN IMMEDIATE TRANSACTION
INSERT INTO "Car" ("name") VALUES (?)
[u'car1']
INSERT INTO "Car" ("name") VALUES (?)
[u'car2']
INSERT INTO "Image" ("name", "car") VALUES (?, ?)
[u'image1', 1]
INSERT INTO "Image" ("name", "car") VALUES (?, ?)
[u'image2', 1]
INSERT INTO "Image" ("name", "car") VALUES (?, ?)
[u'image3', 2]
SELECT "car"."id", "car"."name", "image"."id", "image"."name"
FROM "Car" "car"
LEFT JOIN "Image" "image-1"
ON "car"."id" = "image-1"."car"
LEFT JOIN "Image" "image"
ON "car"."id" = "image"."car"
GROUP BY "car"."id", "car"."name", "image"."id", "image"."name"
HAVING "image"."id" = MIN("image-1"."id")
car |car.name|image |image.name
------+--------+--------+----------
Car[1]|car1 |Image[1]|image1
Car[2]|car2 |Image[3]|image3
COMMIT
RELEASE CONNECTION
By the way, the generated sql is rather strange, I don't think that executing two identical left joins makes any sense.
Edit: OK, it makes sense (see comments).
Related
I'm using SQLAlchemy 1.4 to build my database models (posgresql).
I've stablished relationships between my models, which I follow using the different SQLAlchemy capabilities. When doing so, the fields of the related models get aliases which don't work for me.
Here's an example of one of my models:
from sqlalchemy import Column, DateTime, ForeignKey, Integer, func
from sqlalchemy.orm import relationship
class Process(declarative_model()):
"""Process database table class.
Process model. It contains all the information about one process
iteration. This is the proces of capturing an image with all the
provided cameras, preprocess the images and make a prediction for
them as well as computing the results.
"""
id: int = Column(Integer, primary_key=True, index=True, autoincrement=True)
"""Model primary key."""
petition_id: int = Column(Integer, ForeignKey("petition.id", ondelete="CASCADE"))
"""Foreign key to the related petition."""
petition: "Petition" = relationship("Petition", backref="processes", lazy="joined")
"""Related petition object."""
camera_id: int = Column(Integer, ForeignKey("camera.id", ondelete="CASCADE"))
"""Foreign key to the related camera."""
camera: "Camera" = relationship("Camera", backref="processes", lazy="joined")
"""Related camera object."""
n: int = Column(Integer, comment="Iteration number for the given petition.")
"""Iteration number for the given petition."""
image: "Image" = relationship(
"Image", back_populates="process", uselist=False, lazy="joined"
)
"""Related image object."""
datetime_init: datetime = Column(DateTime(timezone=True), server_default=func.now())
"""Datetime when the process started."""
datetime_end: datetime = Column(DateTime(timezone=True), nullable=True)
"""Datetime when the process finished if so."""
The model works perfectly and joins the data by default as expected, so far so good.
My problem comes when I make a query and I extract the results through query.all() or through pd.read_sql(query.statement, db).
Reading the documentation, I should get aliases for my fields like "{table_name}.{field}" but instead of that I'm getting like "{field}_{counter}". Here's an example of a query.statement for my model:
SELECT process.id, process.petition_id, process.camera_id, process.n, process.datetime_init, process.datetime_end, asset_quality_1.id AS id_2, asset_quality_1.code AS code_1, asset_quality_1.name AS name_1, asset_quality_1.active AS active_1, asset_quality_1.stock_quality_id, pit_door_1.id AS id_3, pit_door_1.code AS code_2, petition_1.id AS id_4, petition_1.user_id, petition_1.user_code, petition_1.load_code, petition_1.provider_code, petition_1.origin_code, petition_1.asset_quality_initial_id, petition_1.pit_door_id, petition_1.datetime_init AS datetime_init_1, petition_1.datetime_end AS datetime_end_1, mask_1.id AS id_5, mask_1.camera_id AS camera_id_1, mask_1.prefix_path, mask_1.position, mask_1.format, camera_1.id AS id_6, camera_1.code AS code_3, camera_1.pit_door_id AS pit_door_id_1, camera_1.position AS position_1, image_1.id AS id_7, image_1.prefix_path AS prefix_path_1, image_1.format AS format_1, image_1.process_id
FROM process LEFT OUTER JOIN petition AS petition_1 ON petition_1.id = process.petition_id LEFT OUTER JOIN asset_quality AS asset_quality_1 ON asset_quality_1.id = petition_1.asset_quality_initial_id LEFT OUTER JOIN stock_quality AS stock_quality_1 ON stock_quality_1.id = asset_quality_1.stock_quality_id LEFT OUTER JOIN pit_door AS pit_door_1 ON pit_door_1.id = petition_1.pit_door_id LEFT OUTER JOIN camera AS camera_1 ON camera_1.id = process.camera_id LEFT OUTER JOIN mask AS mask_1 ON camera_1.id = mask_1.camera_id LEFT OUTER JOIN image AS image_1 ON process.id = image_1.process_id
Does anybody know how can I change this behavior and make it alias the fields like “{table_name}_{field}"?
SQLAlchemy uses label styles to configure how columns are labelled in SQL statements. The default in 1.4.x is LABEL_STYLE_DISAMBIGUATE_ONLY, which will add a "counter" for columns with the same name in a query. LABEL_STYLE_TABLENAME_PLUS_COL is closer to what you want.
Default:
q = session.query(Table1, Table2).join(Table2)
q = q.set_label_style(LABEL_STYLE_DISAMBIGUATE_ONLY)
print(q)
gives
SELECT t1.id, t1.child_id, t2.id AS id_1
FROM t1 JOIN t2 ON t2.id = t1.child_id
whereas
q = session.query(Table1, Table2).join(Table2)
q = q.set_label_style(LABEL_STYLE_TABLENAME_PLUS_COL)
print(q)
generates
SELECT t1.id AS t1_id, t1.child_id AS t1_child_id, t2.id AS t2_id
FROM t1 JOIN t2 ON t2.id = t1.child_id
If you want to enforce a style for all orm queries you could sublcass Session:
class MySession(orm.Session):
_label_style = LABEL_STYLE_TABLENAME_PLUS_COL
and use this class for your sessions, or pass it it a sessionmaker, if you use one:
Session = orm.sessionmaker(engine, class_=MySession)
You can use the label argument of the Column or the relationship method to specify the custom name for a field.
For example, to give a custom label for the process.petition_id field, you can use:
petition_id = Column(Integer, ForeignKey("petition.id", ondelete="CASCADE"), label='process_petition_id')
And for the petition relationship, you can use:
petition = relationship("Petition", backref="processes", lazy="joined", lazyload=True, innerjoin=True, viewonly=False, foreign_keys=[petition_id], post_update=False, cascade='all, delete-orphan', passive_deletes=True, primaryjoin='Process.petition_id == Petition.id', single_parent=False, uselist=False, query_class=None, foreignkey=None, remote_side=None, remote_side_use_alter=False, order_by=None, secondary=None, secondaryjoin=None, back_populates=None, collection_class=None, doc=None, extend_existing=False, associationproxy=None, comparator_factory=None, proxy_property=None, impl=None, _create_eager_joins=None, dynamic=False, active_history=False, passive_updates=False, enable_typechecks=None, info=None, join_depth=None, innerjoin=None, outerjoin=None, selectin=None, selectinload=None, with_polymorphic=None, join_specified=None, viewonly=None, comparison_enabled=None, useexisting=None, label='process_petition')
With this, the fields should be aliased to process_petition_id and process_petition respectively.
I have next data structure:
from enum import IntEnum, unique
from pathlib import Path
from datetime import datetime
from peewee import *
#unique
class Status(IntEnum):
CREATED = 0
FAIL = -1
SUCCESS = 1
db_path = Path(__file__).parent / "test.sqlite"
database = SqliteDatabase(db_path)
class BaseModel(Model):
class Meta:
database = database
class Unit(BaseModel):
name = TextField(unique=True)
some_field = TextField(null=True)
created_at = DateTimeField(default=datetime.now)
class Campaign(BaseModel):
id_ = AutoField()
created_at = DateTimeField(default=datetime.now)
class Task(BaseModel):
id_ = AutoField()
status = IntegerField(default=Status.CREATED)
unit = ForeignKeyField(Unit, backref="tasks")
campaign = ForeignKeyField(Campaign, backref="tasks")
Next code create units, campaign and tasks:
def fill_units(count):
units = []
with database.atomic():
for i in range(count):
units.append(Unit.create(name=f"unit{i}"))
return units
def init_campaign(count):
units = Unit.select().limit(count)
with database.atomic():
campaign = Campaign.create()
for unit in units:
Task.create(unit=unit, campaign=campaign)
return campaign
The problem appears when I'm trying to add more units into existing campaign. I need to select units which haven't been used in this campaign. In SQL I can do this using next query:
SELECT * FROM unit WHERE id NOT IN (SELECT unit_id FROM task WHERE campaign_id = 1) LIMIT 10
But how to do this using peewee?
The only way I've found yet is:
def get_new_units_for_campaign(campaign, count):
unit_names = [task.unit.name for task in campaign.tasks]
units = Unit.select().where(Unit.name.not_in(unit_names)).limit(count)
return units
It's somehow works but I'm 100% sure that it's the dumbest way to implement this. Could you show me the proper way to implement this?
Finally I found this:
Unit.select().where(Unit.id.not_in(campaign.tasks.select(Task.unit))).limit(10)
Which produces
SELECT "t1"."id", "t1"."name", "t1"."some_field", "t1"."created_at" FROM "unit" AS "t1" WHERE ("t1"."id" NOT IN (SELECT "t2"."unit_id" FROM "task" AS "t2" WHERE ("t2"."campaign_id" = 1))) LIMIT 10
Which matches with SQL query I've provided in my question.
P.S. I've done some research and it seems to be a proper implementation, but I'd appreciate if somebody correct me and show the better way (if exist).
I'm trying to set here two fields which interact with each other. In this case, what I'm looking for is to use the variable "code", which will be written within the code_entry and then, I will use this code to bring the description from a sql database. The problem is that it doesn't do anything when I write any number, what's the issue?
def frame_basic_info_widgets():
pm_master_var = tk.StringVar(frame_basic_info,"0")
size = tk.StringVar(frame_basic_info,"")
series = tk.StringVar(frame_basic_info,"")
colour = tk.StringVar(frame_basic_info,"")
code = tk.IntVar(frame_basic_info,0)
global product_description
product_description = tk.StringVar(frame_basic_info,"No Product Selected")
n = 0
product_master_fields = {"Size" :"Product Size",
"Series" :"Product Series",
"Colour" :"Product Colour"}
code_entry = tk.Entry(frame_basic_info,textvariable = code,bg="light grey").grid(row=0,column=0,pady=20,padx=20)
product_label = tk.Label(frame_basic_info,textvariable = product_description,bg="light grey").grid(row=0,column=1,pady=20,padx=20)
def change_product_description():
print(code.get())
product_description = pd.read_sql_query('SELECT product_description FROM Product_Master WHERE id = '+code.get(), engine)
product_label.update()
print(product_description)
product_description.trace("w",change_product_description)
'''
I need help with querying model to get deeper relatonship.
Animals is a main table. It should easly load imgs and stories and its pretty simple. But AnimalsStories has own imgs in same AnimalsImgs table. AnimalsImgs has imgs for both classes Animals and AnimalsStories and both classes have relationships with it.
So i should be able to load all Animals and their stories in AnimalsStories and then from this class i should be able to use .img attrubute which reffers to AnimalsImgs and holds imgs for stories. SqlAlchemy say that it is possible with subqueryload. And it is. But only two levels down. AnimalsStories.img never load.
class Animals(db.Model):
__tablename__ = 'animals'
id = db.Column(db.BigInteger, primary_key=True)
imgs = db.relationship("AnimalsImgs", backref=db.backref('animals',lazy=False))
stories= db.relationship("AnimalsStories",lazy='joined')
class AnimalsImgs(db.Model):
__tablename__ = 'animals_imgs'
id = db.Column(db.BigInteger, primary_key=True,autoincrement='auto')
id_animal = db.Column(db.BigInteger, db.ForeignKey('animals.id'),nullable=False)
class AnimalsStories(db.Model):
__tablename__='animals_stories'
id = db.Column(db.BigInteger, primary_key=True)
id_animal= db.Column(db.BigInteger,db.ForeignKey('animals.id'), nullable=False)
id_animal_img = db.Column(db.BigInteger,db.ForeignKey('animals_imgs.id'))
img=db.relationship("AnimalsImgs", uselist=False)
I've tried something like this :
query = Animals.query.options(subqueryload(Animals.stories).subqueryload(AnimalsStories.img))
result = query.all()
print(result)
for res in result:
print(res.rescue.img)
And ended with " AttributeError: 'InstrumentedList' object has no attribute 'img' "
It should be pretty simple to query deeper objects. I think the problem is somewhere in models structures.
Edit #1
I've ended up with solution. It was not that hard.
Animals reffers to stories as one to many. Stories reffers to Imgs as one to one. So with query i posted (with subqueryload) it can be done by:
query = Animals.query.options(subqueryload(Animals.stories).subqueryload(AnimalsStories.img))
result = query.all()
print(result)
for res in result:
print(res.stories)
for r in res.stories: # stories appears as array so they are iterable
print(r.img)
And it prints all the levels pretty clear.
It can be done as well, maybe even better, with join. We are sure that result doesnt have any missing or empty arrays.
query = Animals.query.join(AnimalsStories).join(AnimalsImgs)
I was not sure which types of relationship you want to model. I made the following assumptions:
A image can have a relationship to one or no animal, as well to one or no story
A story has a relationship to one animal, as well as to none or several images
A animal can have relationships to none or several images, as well to none or several stories
This would lead to a model like this:
class Animals(db.Model):
__tablename__ = 'animals'
id = db.Column(db.Integer, primary_key=True)
imgs = db.relationship("AnimalsImgs", backref=db.backref('animals'),lazy=True)
stories= db.relationship("AnimalsStories", backref=db.backref('animals'),
lazy=True)
class AnimalsImgs(db.Model):
__tablename__ = 'animals_imgs'
id = db.Column(db.Integer, primary_key=True)
id_animal = db.Column(db.Integer, db.ForeignKey('animals.id'),nullable=True)
id_animal_story =
db.Column(db.Integer,db.ForeignKey('animals_stories.id'),nullable=True)
class AnimalsStories(db.Model):
__tablename__='animals_stories'
id = db.Column(db.Integer, primary_key=True)
id_animal= db.Column(db.Integer,db.ForeignKey('animals.id'), nullable=False)
imgs=db.relationship("AnimalsImgs", backref=db.backref('stories'),lazy=True)
Now I have added the following scenrios:
An animal a1 that has a relationship to an image ai1 and to an story ast1. (but not relationship between the story and the image.
An animal a2 that has a relationship to story ast2. Ast2 has a relationship to a image ai2. But no relationship betwenn ast2 and a2.
An animal a3 that has a relationship to a story ast3 and a image ai3. ai3 has also a relationship to ast3.
Code sample like this:
a1 = Animals()
a2 = Animals()
a3 = Animals ()
a4 = Animals ()
db.session.add(a1)
db.session.add(a2)
db.session.add(a3)
db.session.add(a4)
ai1 = AnimalsImgs(animals = a1)
db.session.add(ai1)
ast1 = AnimalsStories(animals = a1)
db.session.add(ast1)
ast2 = AnimalsStories(animals = a2)
db.session.add(ast2)
ai2 = AnimalsImgs(stories = ast2)
db.session.add(ai2)
ast3 = AnimalsStories(animals = a3)
db.session.add(ast3)
ai3 = AnimalsImgs(animals = a3, stories = ast3)
db.session.add(ai3)
If you want to query all images that are connected to stories, you can use:
animalimages_ofallstories_ofalanimals =
AnimalsImgs.query.join(AnimalsStories).join(Animals).all()
You get image ai2 and ai3 as result. (ai1 is not connected to a story but only to an animal)
I have an ObjectListView that displays information retrieved from an SQLite DB with SQLAlchemy.
def setupOLV(self):
self.loanResultsOlv.SetEmptyListMsg("No Loan Records Found")
self.loanResultsOlv.SetColumns([
ColumnDefn("Date Issued", "left", 100, "date_issued",
stringConverter="%d-%m-%y"),
ColumnDefn("Card Number", "left", 100, "card_id"),
ColumnDefn("Student Number", "left", 100, "person_id"),
ColumnDefn("Forename", "left", 150, "person_fname"),
ColumnDefn("Surname", "left", 150, "person_sname"),
ColumnDefn("Reason", "left", 150, "issue_reason"),
ColumnDefn("Date Due", "left", 100, "date_due",
stringConverter="%d-%m-%y"),
ColumnDefn("Date Returned", "left", 100, "date_returned",
stringConverter="%d-%m-%y")
])
I also have three models, Loan:
class Loan(DeclarativeBase):
"""
Loan model
"""
__tablename__ = "loans"
id = Column(Integer, primary_key=True)
card_id = Column(Unicode, ForeignKey("cards.id"))
person_id = Column(Unicode, ForeignKey("people.id"))
date_issued = Column(Date)
date_due = Column(Date)
date_returned = Column(Date)
issue_reason = Column(Unicode(50))
person = relation("Person", backref="loans", cascade_backrefs=False)
card = relation("Card", backref="loans", cascade_backrefs=False)
Person:
class Person(DeclarativeBase):
"""
Person model
"""
__tablename__ = "people"
id = Column(Unicode(50), primary_key=True)
fname = Column(Unicode(50))
sname = Column(Unicode(50))
and Card:
class Card(DeclarativeBase):
"""
Card model
"""
__tablename__ = "cards"
id = Column(Unicode(50), primary_key=True)
active = Column(Boolean)
I am trying to join the tables (loans and people) in order to retrieve and display the information in my ObjectListView. Here is my query method:
def getQueriedRecords(session, filterChoice, keyword):
"""
Searches the database based on the filter chosen and the keyword
given by the user
"""
qry = session.query(Loan)
if filterChoice == "person":
result = qry.join(Person).filter(Loan.person_id=='%s' % keyword).all()
elif filterChoice == "card":
result = qry.join(Person).filter(Loan.card_id=='%s' % keyword).all()
return result
I can retrieve and display every field stored in the loans table but forename and surname (should be drawn from people table and joined on person.id) are blank in my ObjectListView. I have SQL output on so I can see the query and it is not selecting at all from the people table.
How can I modify my query/ObjectListView to retrieve and display this information. ?
UPDATE: I have created an example script that is runnable here.
You're only querying for a Loan (qry = session.query(Loan)). Why do you expect something else to be in the results besides what's in the SELECT statement?
I admit that I am pretty new to SQLAlchemy myself, but I thought I would share what I use to display results from my queries. I have a program that uses a SQLite DB with 4+ tables and I pull data from 2-3 of them in a single query and display this information in an ObjectListView. I owe Mike Driscoll for his in depth tutorials, particularly wxPython and SqlAlchemy: An Intro to MVC and CRUD.
Here is what I would possibly add/change in your code.
In your model section add a "display" class such as:
def OlvDisplay(object):
def __init__(self, date_issued, card_id, person_id, fname, sname,
issue_reason, date_due, date_returned):
self.date_issued = date_issued
self.card_id = card_id
self.person_id = person_id
self.person_fname = fname
self.person_sname = sname
self.issue_reason = issue_reason
self.date_due = date_due
self.date_returned = date_returned
This display class is used in the convertResults definition below and assists with making sure the data is formatted properly for the ObjectListView.
The adjustment to your existing query function:
def getQueriedRecords(session, filterChoice, keyword):
"""
Searches the database based on the filter chosen and the keyword
given by the user
"""
qry = session.query(Loan)
if filterChoice == "person":
result = qry.join(Person).filter(Loan.person_id=='%s' % keyword).all()
elif filterChoice == "card":
result = qry.join(Person).filter(Loan.card_id=='%s' % keyword).all()
convertedResults = convertResults(result)
return convertedResults
What we're doing here is creating a local variable that is essentially running the conversion definition and storing the results for the next line, which returns those results.
And the "Convertor" function:
def convertResults(results):
finalResults = []
for record in results:
result = OlvDisplay(
record.date_issued,
record.card_id,
record.person_id,
record.person.fname,
record.person.sname,
record.issue_reason,
record.date_due,
record.date_returned
)
finalResults.append(result)
return finalResults
The important part here are the 2 lines:
record.person.fname
record.person.sname
Since we are wanting to pull information from another table using the established relationship it is important to refer to that relationship to actually see the data.
And to populate the ObjectListView Widget:
theOutput = getQueriedRecords(session, filterChoice, keyword)
self.setupOLV.SetObjects(theOutput)
Hope this helps you out.
-MikeS