I have a table that stores tasks submitted by users, with timestamps. I would like to write a query that returns certain rows based on when they were submitted (was it this day/week/month..).
To check if it was submitted on this week, I wanted to use date.isocalendar()[1] function. The problem is, that my timestamps are datetimes, so I would need to transform those to dates.
Using func:
filter(func.date(Task.timestamp) == datetime.date(datetime.utcnow()))
works properly.
But I need the date object's isocalendar() method, so I try
filter(func.date(Task.timestamp).isocalendar()[1]==datetime.date(datetime.utcnow()).isocalendar()[1])
and it's no good, I get AttributeError: Neither 'Function' object nor 'Comparator' object has an attribute 'isocalendar'
If I make a simple query and try datetime.date(task.timestamp).isocalendar()[1] it works properly.
How do I get it to work in the query's filter?
Rule of thumb when understanding and debugging sqlalchemy queries is to always think – "How will it look in SQL?"
isocalendar() is a python function, and sqlalchemy query filters get compiled to SQL. Moreover, isocalendar() returns a tuple – and while rendering tuple comparison as SQL is probably possible, it's more trouble then it's worth. You should compare scalars and find sql date functions that suit you.
It seems you're looking to compare week number, so something like this should do the trick:
filter(func.week(Task.timestamp)==datetime.utcnow().isocalendar()[1])
Can you try sqlalchemy.extract(func.date('year', Task.timestamp)) == ... ?
You cannot mix pure python functions with those which are executed on the SQL backend. From your code it looks like you are trying to filter on the iso week. One way to do it would be to load everything from the database into memory and perform the filtering there. Obviously, it most cases it is far from efficient.
An alternative would be to use respective SQL functions, which sqlalchemy will call for you. On MySQL it looks like the function you need is weekofyear, so your filter might look similar to below:
_utcnow = datetime.utcnow().date()
_isoweek = _utcnow.isocalendar()[1]
q = db.session.query(...)
# ...
q = q.filter(db.func.weekofyear(Task.timestamp) == _isoweek)
Related
I think I have misunderstood something critical and obvious, I am attempting to use the SqlAlchemy ORM to select a row from a table in a database and return the values. The first step is to get the row and access it, from there I think I'm good however, when I select the row and try and investigate the object that is returned, all I find is an address for the object in memory: <main.UserMap object at 0x000001F65A54B490>,
Attempting to use objprint to investigate the object gives no further information, I am confused as per my understanding the resulting row object should behave like a tuple so at least objprint should find a variety of entries within it even if it can't print them due to them being differen't data types.
The statement in question:
select(UserMap).where(UserMap.google_auth == '***********************')
a more basic select(UserMap) also seems to give a similar result.
The table contains some fields as strings, some as integers and some in date and time formats but obviously only one type per column.
I am using session.execute, I would like to avoid query as I understand that it's functionality is being deprecated in the 2.x API, if I have misunderstood this then I am happy to attempt that but would still like to understand what I am getting wrong about the result and row objects.
As I said, I think I have missed something important, but don't realise what, as far as I can see I am executing the statements in much the same way as shown in the ORM documentation e.g:
>>> stmt = select(User).where(User.name == 'spongebob')
>>> with Session(engine) as session:
... for row in session.execute(stmt):
... print(row)
<main.UserMap object at 0x000001F65A54B490> is the default string representation of an ORM object. You can retrieve the attributes (properties) of the object using the standard syntax:
me = UserMap(user_name="Gord")
print(me) # <main.UserMap object at 0x000001F65A54B490>
print(me.user_name) # Gord
I am new to this and trying to learn python. I wrote a select statement in python where I used a parameter
Select """cln.customer_uid = """[(num_cuid_number)])
TypeError: string indices must be integers
Agree with the others, this doesn't look really like Python by itself.
I will see even without seeing the rest of that code I'll guess the [(num_cuid_number)] value(s) being returned is a string, so you'll want to convert it to integer for the select statement to process.
num_cuid_number is most likely a string in your code; the string indices are the ones in the square brackets. So please first check your data variable to see what you received there. Also, I think that num_cuid_number is a string, while it should be in an integer value.
Let me give you an example for the python code to execute: (Just for the reference: I have used SQLAlchemy with flask)
#app.route('/get_data/')
def get_data():
base_sql="""
SELECT cln.customer_uid='%s' from cln
""" % (num_cuid_number)
data = db.session.execute(base_sql).fetchall()
Pretty sure you are trying to create a select statement with a "where" clause here. There are many ways to do this, for example using raw sql, the query should look similar to this:
query = "SELECT * FROM cln WHERE customer_uid = %s"
parameters = (num_cuid_number,)
separating the parameters from the query is secure. You can then take these 2 variables and execute them with your db engine like
results = db.execute(query, parameters)
This will work, however, especially in Python, it is more common to use a package like SQLAlchemy to make queries more "flexible" (in other words, without manually constructing an actual string as a query string). You can do the same thing using SQLAlchemy core functionality
query = cln.select()
query = query.where(cln.customer_uid == num_cuid_number)
results = db.execute(query)
Note: I simplified "db" in both examples, you'd actually use a cursor, session, engine or similar to execute your queries, but that wasn't your question.
I've read the documentation on how to do this but it is not working. I am trying to get my query to sort by date newest first, but this is not working:
wrQuery = WorkRequest.query()
wrQuery.order('-date')
wrResult = wrQuery.fetch()
I have also tried ('-WorkRequest.date') but both results in similar errors:
TypeError: order() expects a Property or query Order; received '-date'
The query objects are immutable, your 3rd statement references the object from your 1st statement, not the ordered one created in the 2nd statement. See also NDB Query builder doesn't work as expected.
This should work instead:
wrQuery = WorkRequest.query()
wrOrderedQuery = wrQuery.order('-WorkRequest.date')
wrResult = wrOrderedQuery.fetch()
This looks like you are using NDB rather than DataStore, in which case the docs show you unequivocally how to do it. This should work:
wrQuery = wrQuery.order(-WorkRequest.date)
I have a stored procedure in a postgresql database.
I'm trying to use the function within a python flask app with sqlalchemy. That query looks like this:
from sqlalchemy import func
appts = db.session.execute(func.getopenappointments(current_user.id))
for appt in appts:
# work with each appt
The result from this query is an object of type sqlalchemy.engine.result.ResultProxy. Each iteration of that object looks like this:
('(2,"2017-09-15 10:00:00",6,cleaning,available,5)',)
The problem is I am used to referring to the columns with something like:
for appt in appts:
print(appt.id)
But this fails due to id not existing. What I have realized is the output is pretty much a string that I have to parse with python split() just to get the values I need. How I can keep this a stored procedure but be able to refer to the output by columns, or at least as a tuple and not a regular string?
Take a look at this question. There is a construct called from_statement that can be used to interpret the results of a SQL statement as an SQLAlchemy ORM model.
So I'm assuming that you have an Appointment class that is an ORM mapper, either because you used declarative_base or because you used the mapper function directly.
Then you can do something like
appts = db.session.query(Appointment).from_statement(func.getopenappointments(current_user.id))
That will run your SQL stored procedure and interpret the result if it is a set of Appointment objects.
Consider this query:
query = Novel.objects.< ...some filtering... >.annotate(
latest_chapter_id=Max("volume__chapter__id")
)
Actually what I need is to annotate each Novel with its latest Chapter object, so after this query, I have to execute another query to select actual objects by annotated IDs. IMO this is ugly. Is there a way to combine them into a single query?
Yes, it's possible.
To get a queryset containing all Chapters which are the last in their Novels, simply do:
from django.db.models.expressions import F
from django.db.models.aggregates import Max
Chapters.objects.annotate(last_chapter_pk=Max('novel__chapter__pk')
).filter(pk=F('last_chapter_pk'))
Tested on Django 1.7.
Possible with Django 3.2+
Make use of django.db.models.functions.JSONObject (added in Django 3.2) to combine multiple fields (in this example, I'm fetching the latest object, however it is possible to fetch any arbitrary object provided that you can get LIMIT 1) to yield your object):
MainModel.objects.annotate(
last_object=RelatedModel.objects.filter(mainmodel=OuterRef("pk"))
.order_by("-date_created")
.values(
data=JSONObject(
id="id", body="body", date_created="date_created"
)
)[:1]
)
Yes, using Subqueries, docs: https://docs.djangoproject.com/en/3.0/ref/models/expressions/#subquery-expressions
latest_chapters = Chapter.objects.filter(novel = OuterRef("pk"))\
.order_by("chapter_order")
novels_with_chapter = Novel.objects.annotate(
latest_chapter = Subquery(latest_chapters.values("chapter")[:1]))
Tested on Django 3.0
The subquery creates a select statement inside the select statement for the novels, then adds this as an annotation. This means you only hit the database once.
I also prefer this to Rune's answer as it actually annotates a Novel object.
Hope this helps, anyone who came looking like much later like I did.
No, it's not possible to combine them into a single query.
You can read the following blog post to find two workarounds.