In a single model Person there is an address information. But since we are not separating this yet to another table. I would like to only query the address information out of Person table. Would it be possible using hybrid_property If not what else do I need to achieve this stuff?
I wanna avoid this one:
db.session.query(Person.id, Person.address_secret_id, Person.address_name).get(pk)
The model
class Person(db.Model):
# some lengthy information
# address
address_secret_id = db.Column(db.Unicode, nullable=True)
address_name = db.Column(db.Unicode, nullable=True)
#hybrid_property
def address(self):
# I never tested this but i know this is wrong.
return self.id + self.address_secret_id + self.address_name
Usage:
db.session.query(Person.address).get(pk)
Expected Output:
{id: 1, address_secret_id: xxxx, address_name: 'forgetmeland'}
How can I achieve an output that is only retrieving the desired field? It doesn't need to be dict or tuple as long as Im getting what is needed.
If you are trying to avoid having to type db.session.query(Person.id, Person.address_secret_id, Person.address_name), just add an address_details property on the person model.
class Person(db.Model):
# some lengthy information
# address
address_secret_id = db.Column(db.Unicode, nullable=True)
address_name = db.Column(db.Unicode, nullable=True)
#property
def address_details(self):
keys = ('id', 'address_secret_id', 'address_name')
return {k: getattr(self, k) for k in in keys}
Probably less lines of code than trying to use some sort of hybrid query, and still just the one trip to the database.
Query would be:
Person.query.get(1).address_details
Related
I've got file objects of different types, which inherit from a BaseFile, and add custom attributes, methods and maybe fields. The BaseFile also stores the File Type ID, so that the corresponding subclass model can be retrieved from any BaseFile object:
class BaseFile(models.Model):
name = models.CharField(max_length=80, db_index=True)
size= models.PositiveIntegerField()
time_created = models.DateTimeField(default=datetime.now)
file_type = models.ForeignKey(ContentType, on_delete=models.PROTECT)
class FileType1(BaseFile):
storage_path = '/path/for/filetype1/'
def custom_method(self):
<some custom behaviour>
class FileType2(BaseFile):
storage_path = '/path/for/filetype2/'
extra_field = models.CharField(max_length=12)
I also have different types of events which are associated with files:
class FileEvent(models.Model):
file = models.ForeignKey(BaseFile, on_delete=models.PROTECT)
time = models.DateTimeField(default=datetime.now)
I want to be able to efficiently get all files of a particular type which have not been involved in a particular event, such as:
unprocessed_files_type1 = FileType1.objects.filter(fileevent__isnull=True)
However, looking at the SQL executed for this query:
SELECT "app_basefile"."id", "app_basefile"."name", "app_basefile"."size", "app_basefile"."time_created", "app_basefile"."file_type_id", "app_filetype1"."basefile_ptr_id"
FROM "app_filetype1"
INNER JOIN "app_basefile"
ON("app_filetype1"."basefile_ptr_id" = "app_basefile"."id")
LEFT OUTER JOIN "app_fileevent" ON ("app_basefile"."id" = "app_fileevent"."file_id")
WHERE "app_fileevent"."id" IS NULL
It looks like this might not be very efficient because it joins on BaseFile.id instead of FileType1.basefile_ptr_id, so it will check ALL BaseFile ids to see whether they are present in FileEvent.file_id, when I only need to check the BaseFile ids corresponding to FileType1, or FileType1.basefile_ptr_ids.
This could result in a significant performance difference if there are a very large number of BaseFiles, but FileType1 is only a small subset of that, because it will be doing a large amount of unnecessary lookups.
Is there a way to force Django to join on "app_filetype1"."basefile_ptr_id" or otherwise achieve this functionality more efficiently?
Thanks for the help
UPDATE:
Using annotations and Exists subquery seems to do what I'm after, however the resulting SQL still appears strange:
unprocessed_files_type1 = FileType1.objects.annotate(file_event=Exists(FileEvent.objects.filter(file=OuterRef('pk')))).filter(file_event=False)
SELECT "app_basefile"."id", "app_basefile"."name", "app_basefile"."size", "app_basefile"."time_created", "app_basefile"."file_type_id", "app_filetype1"."basefile_ptr_id",
EXISTS(
SELECT U0."id", U0."file_id", U0."time"
FROM "app_fileevent" U0
WHERE U0."file_id" = ("app_filetype1"."basefile_ptr_id"))
AS "file_event"
FROM "app_filetype1"
INNER JOIN "app_basefile" ON ("app_filetype1"."basefile_ptr_id" = "app_basefile"."id")
WHERE EXISTS(
SELECT U0."id", U0."file_id", U0."time"
FROM "app_fileevent" U0
WHERE U0."file_id" = ("app_filetype1"."basefile_ptr_id")) = 0
It appears to be doing the WHERE EXISTS subquery twice instead of just using the annotated 'file_event' label... Maybe this is just a Django/SQLite driver bug?
I am trying to create a custom primary_key within my helpdesk/models.py that I will use to track our help desk tickets. I am in the process of writing a small ticking system for our office.
Maybe there is a better way? Right now I have:
id = models.AutoField(primary_key=True)
This increments in the datebase as; 1, 2, 3, 4....50...
I want to take this id assignment and then use it within a function to combine it with some additional information like the date, and the name, 'HELPDESK'.
The code I was using is as follows:
id = models.AutoField(primary_key=True)
def build_id(self, id):
join_dates = str(datetime.now().strftime('%Y%m%d'))
return (('HELPDESK-' + join_dates) + '-' + str(id))
ticket_id = models.CharField(max_length=15, default=(build_id(None, id)))
The idea being is that the entries in the database would be:
HELPDESK-20170813-1
HELPDESK-20170813-2
HELPDESK-20170814-3
...
HELPDESK-20170901-4
...
HELPDESK-20180101-50
...
I want to then use this as the ForeignKey to link the help desk ticket to some other models in the database.
Right now what's coming back is:
HELPDESK-20170813-<django.db.models.fields.AutoField>
This post works - Custom Auto Increment Field Django Curious if there is a better way. If not, this will suffice.
This works for me. It's a slightly modified version from Custom Auto Increment Field Django from above.
models.py
def increment_helpdesk_number():
last_helpdesk = helpdesk.objects.all().order_by('id').last()
if not last_helpdesk:
return 'HEL-' + str(datetime.now().strftime('%Y%m%d-')) + '0000'
help_id = last_helpdesk.help_num
help_int = help_id[13:17]
new_help_int = int(help_int) + 1
new_help_id = 'HEL-' + str(datetime.now().strftime('%Y%m%d-')) + str(new_help_int).zfill(4)
return new_help_id
It's called like this:
help_num = models.CharField(max_length=17, unique=True, default=increment_helpdesk_number, editable=False)
If gives you the following:
HEL-20170815-0000
HEL-20170815-0001
HEL-20170815-0002
...
The numbering doesn't start over after each day, which is something I may look at doing. The more I think about it; however, I am not sure if I even need the date there as I have a creation date field in the model already. So I may just change it to:
HEL-000000000
HEL-000000001
HEL-000000002
...
In my models.py, I have 2 models :
class Well(models.Model):
id = models.AutoField(db_column='ID', primary_key=True, verbose_name='DNS')
block = models.ForeignKey(Block, db_column='Block_ID', verbose_name='Block')
uwi = models.CharField(db_column='UWI', max_length=20, blank=True, verbose_name='UWI')
welllocation = models.ForeignKey('Welllocation', db_column='WellLocation_ID', verbose_name='Location')
# Plus a number of other columns
class Welldrillingdetails(models.Model):
id = models.AutoField(db_column='ID', primary_key=True, verbose_name='DNS')
well = models.ForeignKey(Well, db_column='Well_ID', verbose_name='Well')
casingdetails = models.ForeignKey(Casingdetails, db_column='CasingDetails_ID', verbose_name='Casing Name')
holesize = models.CharField(db_column='holeSize', max_length=15, blank=True, verbose_name='Hole Size (inch)')
# Plus a number of other columns
The table Welldrillingdetails uses pk of Well table as foreign key.
I have a python dictionary that has an element list having list of primary keys of well table which I have converted from JSON.
raw_data = {"wells": [1,2], "seams": ["2"], "tables": [{"name": "Well", "fields": []}, {"name": "Stratigraphy", "fields": []}]}
When I am fetching the objects of Welldrillingdetails model using :
ObjList.append(Welldrillingdetails.objects.all().filter(well__in=raw_data['wells']))
It is working fine. But when I am doing the same for the Well table using :
ObjList.append(Well.objects.in_bulk(raw_data['wells']))
or
ObjList.append(Well.objects.all().filter(id__in=raw_data['wells']))
or
ObjList.append(Well.objects.all().filter(pk__in=raw_data['wells']))
it is not working and giving the error:
FieldError at /cbm/ajaxp/display_data/
Cannot resolve keyword 'well' into field. Choices are: ai, artificiallifts, block, category, coalseamdisp, coalseamseval, coredata, desorbedgascomp, dewateringdetails, drillcompletedate, drilleddepth, electrologs, gc, gl, hermtestingdate, hydrofracdata, id, kb, latitude, loggerdepth, longitude, maceral, minifractestresults, normalisedgc, objective, observations, pmrockprop, presentstatus, profile, projecteddepth, proximate, ptobjectinterval, releaseorderdate, releaseorderno, reserviorwelltestdata, rigname, rigreleasedate, spuddate, stratigraphy, toc, toposheet, triaxialstrength, ultimate, uwi, wcrfile, welldrillingdetails, welllocation, welltype
Is the syntax different while fetching using primary key?
First of all use related_name in relationships it helps you better determine and recognize reverse relations.
class Well(models.Model):
id = models.AutoField(db_column='ID', primary_key=True, verbose_name='DNS')
block = models.ForeignKey(Block, db_column='Block_ID', verbose_name='Block', related_name="well")
uwi = models.CharField(db_column='UWI', max_length=20, blank=True, verbose_name='UWI')
welllocation = models.ForeignKey('Welllocation', db_column='WellLocation_ID', verbose_name='Location', related_name="well")
# Plus a number of other columns
class Welldrillingdetails(models.Model):
id = models.AutoField(db_column='ID', primary_key=True, verbose_name='DNS')
well = models.ForeignKey(Well, db_column='Well_ID', verbose_name='Well', related_name='drillingdetails')
casingdetails = models.ForeignKey(Casingdetails, db_column='CasingDetails_ID', verbose_name='Casing Name', related_name="drillingdetails")
holesize = models.CharField(db_column='holeSize', max_length=15, blank=True, verbose_name='Hole Size (inch)')
# Plus a number of other columns
Bulk does not work because your list is not actually passed:
ObjList.append(Well.objects.in_bulk(list(raw_data['wells'])))
In case the above does not evaluate again, force the list before the call:
well_ids = list(raw_data["wells"])
ObjList.append(Well.objects.in_bulk(well_ids))
With the reverse_relation above you can also:
ObjList.append(Welldrillingdetails.objects.select_related('well').filter(well__in=raw_data['wells']))
Actually, your queries are correct, this one will work for sure:
ObjList.append(Well.objects.all().filter(pk__in=raw_data['wells']))
I think a problem is in ObjList, I suspect it's not a normal list. From your bug report
FieldError at /cbm/ajaxp/display_data/
Cannot resolve keyword 'well' into field. Choices are: ai, artificiallifts, block, category, coalseamdisp, coalseamseval, coredata, desorbedgascomp, dewateringdetails, drillcompletedate, drilleddepth, electrologs, gc, gl, hermtestingdate, hydrofracdata, id, kb, latitude, loggerdepth, longitude, maceral, minifractestresults, normalisedgc, objective, observations, pmrockprop, presentstatus, profile, projecteddepth, proximate, ptobjectinterval, releaseorderdate, releaseorderno, reserviorwelltestdata, rigname, rigreleasedate, spuddate, stratigraphy, toc, toposheet, triaxialstrength, ultimate, uwi, wcrfile, welldrillingdetails, welllocation, welltype
it looks like ObjList has welldrillingdetails attribute but is missing well. My guess is that ObjList somehow translates class name of model you are appending into attribute name and well attribute is missing.
I have an ObjectListView that displays information retrieved from an SQLite DB with SQLAlchemy.
def setupOLV(self):
self.loanResultsOlv.SetEmptyListMsg("No Loan Records Found")
self.loanResultsOlv.SetColumns([
ColumnDefn("Date Issued", "left", 100, "date_issued",
stringConverter="%d-%m-%y"),
ColumnDefn("Card Number", "left", 100, "card_id"),
ColumnDefn("Student Number", "left", 100, "person_id"),
ColumnDefn("Forename", "left", 150, "person_fname"),
ColumnDefn("Surname", "left", 150, "person_sname"),
ColumnDefn("Reason", "left", 150, "issue_reason"),
ColumnDefn("Date Due", "left", 100, "date_due",
stringConverter="%d-%m-%y"),
ColumnDefn("Date Returned", "left", 100, "date_returned",
stringConverter="%d-%m-%y")
])
I also have three models, Loan:
class Loan(DeclarativeBase):
"""
Loan model
"""
__tablename__ = "loans"
id = Column(Integer, primary_key=True)
card_id = Column(Unicode, ForeignKey("cards.id"))
person_id = Column(Unicode, ForeignKey("people.id"))
date_issued = Column(Date)
date_due = Column(Date)
date_returned = Column(Date)
issue_reason = Column(Unicode(50))
person = relation("Person", backref="loans", cascade_backrefs=False)
card = relation("Card", backref="loans", cascade_backrefs=False)
Person:
class Person(DeclarativeBase):
"""
Person model
"""
__tablename__ = "people"
id = Column(Unicode(50), primary_key=True)
fname = Column(Unicode(50))
sname = Column(Unicode(50))
and Card:
class Card(DeclarativeBase):
"""
Card model
"""
__tablename__ = "cards"
id = Column(Unicode(50), primary_key=True)
active = Column(Boolean)
I am trying to join the tables (loans and people) in order to retrieve and display the information in my ObjectListView. Here is my query method:
def getQueriedRecords(session, filterChoice, keyword):
"""
Searches the database based on the filter chosen and the keyword
given by the user
"""
qry = session.query(Loan)
if filterChoice == "person":
result = qry.join(Person).filter(Loan.person_id=='%s' % keyword).all()
elif filterChoice == "card":
result = qry.join(Person).filter(Loan.card_id=='%s' % keyword).all()
return result
I can retrieve and display every field stored in the loans table but forename and surname (should be drawn from people table and joined on person.id) are blank in my ObjectListView. I have SQL output on so I can see the query and it is not selecting at all from the people table.
How can I modify my query/ObjectListView to retrieve and display this information. ?
UPDATE: I have created an example script that is runnable here.
You're only querying for a Loan (qry = session.query(Loan)). Why do you expect something else to be in the results besides what's in the SELECT statement?
I admit that I am pretty new to SQLAlchemy myself, but I thought I would share what I use to display results from my queries. I have a program that uses a SQLite DB with 4+ tables and I pull data from 2-3 of them in a single query and display this information in an ObjectListView. I owe Mike Driscoll for his in depth tutorials, particularly wxPython and SqlAlchemy: An Intro to MVC and CRUD.
Here is what I would possibly add/change in your code.
In your model section add a "display" class such as:
def OlvDisplay(object):
def __init__(self, date_issued, card_id, person_id, fname, sname,
issue_reason, date_due, date_returned):
self.date_issued = date_issued
self.card_id = card_id
self.person_id = person_id
self.person_fname = fname
self.person_sname = sname
self.issue_reason = issue_reason
self.date_due = date_due
self.date_returned = date_returned
This display class is used in the convertResults definition below and assists with making sure the data is formatted properly for the ObjectListView.
The adjustment to your existing query function:
def getQueriedRecords(session, filterChoice, keyword):
"""
Searches the database based on the filter chosen and the keyword
given by the user
"""
qry = session.query(Loan)
if filterChoice == "person":
result = qry.join(Person).filter(Loan.person_id=='%s' % keyword).all()
elif filterChoice == "card":
result = qry.join(Person).filter(Loan.card_id=='%s' % keyword).all()
convertedResults = convertResults(result)
return convertedResults
What we're doing here is creating a local variable that is essentially running the conversion definition and storing the results for the next line, which returns those results.
And the "Convertor" function:
def convertResults(results):
finalResults = []
for record in results:
result = OlvDisplay(
record.date_issued,
record.card_id,
record.person_id,
record.person.fname,
record.person.sname,
record.issue_reason,
record.date_due,
record.date_returned
)
finalResults.append(result)
return finalResults
The important part here are the 2 lines:
record.person.fname
record.person.sname
Since we are wanting to pull information from another table using the established relationship it is important to refer to that relationship to actually see the data.
And to populate the ObjectListView Widget:
theOutput = getQueriedRecords(session, filterChoice, keyword)
self.setupOLV.SetObjects(theOutput)
Hope this helps you out.
-MikeS
I'm trying to make some generic apps using Sql Alchemy, such as tags or rating for any model. But I couldn't find any help in the docs. I really liked what I could do with the django contenttypes framework ? Is there any similar functionality in Sql Alchemy ?
I once wrote some example code about something similar to this (see http://taketwoprogramming.blogspot.com/2009/08/reusable-sqlalchemy-models.html).
The basic idea is that you can create a model like this:
#commentable
class Post(Base):
__tablename__ = 'posts'
id = sa.Column(sa.Integer, primary_key=True)
text = sa.Column(sa.String)
...where commentable is defined like this...
class BaseComment(object):
pass
def build_comment_model(clazz):
class_table_name = str(class_mapper(clazz).local_table)
metadata = clazz.metadata
comment_class_name = clazz.__name__ + 'Comment'
comment_class = type(comment_class_name, (BaseComment,), {})
comment_table_name = class_table_name + '_comments'
comment_table = sa.Table(comment_table_name, metadata,
sa.Column('id', sa.Integer, primary_key=True),
sa.Column(class_table_name + '_id',
sa.Integer,
sa.ForeignKey(class_table_name + '.id')),
sa.Column('text', sa.String),
sa.Column('name', sa.String(100)),
sa.Column('url', sa.String(255)),
)
mapper(comment_class, comment_table)
return comment_class, comment_table
def commentable(clazz):
comment_class, comment_table = build_comment_model(clazz)
clazz.Comment = comment_class
setattr(clazz, 'comments', relation(comment_class))
def add_comment(self, comment):
self.comments.append(comment)
setattr(clazz, 'add_comment', add_comment)
return clazz
Basically, the commentable decorator dynamically creates a new type and table, along with some helper methods to the decorated class. This is the test I used to test that the code works, which shows some example of how it would work...
class TestModels(SATestCase):
def test_make_comment(self):
p = Post()
p.text = 'SQLAlchemy is amazing!'
text = 'I agree!'
name = 'Mark'
url = 'http://www.sqlalchemy.org/'
c = Post.Comment()
c.text = text
c.name = name
c.url = url
p.add_comment(c)
Session.add(p)
# This is a method I use to force the reload of the objects from
# the database to make sure that when I test them, I'm actually
# pulling from the database rather than just getting the data
# of the object still in the session.
p = self.reload(p)
self.assertEquals(len(p.comments), 1)
c = p.comments[0]
self.assertEquals(c.text, text)
self.assertEquals(c.name, name)
self.assertEquals(c.url, url)
I wrote this awhile ago, but I don't think there's anything in SQLA that will do this kind of thing for you, but you can create something similar without too much trouble. In my example, I created new mapped classes and methods to use it on the fly in a class decorator.
I never really made much use out of it, but it might give you some ideas.