We Keep Coding

SQLAlchemy distinct ignoring order_by

I'm trying to run a query from my flask application using SQLAlchemy where I order the results by the timestamp in descending order. From the returned order I want to pull the distinct sender_id's. Unfortunately distinct ignores my requested order_by and pulls the data from the standard table layout.
messagesReceived = Message.query.filter_by(recipient_id=user_id).order_by(Message.timestamp.desc()).group_by(Message.sender_id).distinct()
I'm still new to SQLAlchemy and through the tutorials and lessons I have done I haven't encountered this, I tried googling but I don't think I am phrasing it correctly to get the answer. I'm currently trying to wrap my head around sub-queries as I think that might be a way to make it work, but asking here in the meantime.

Your SQL query is illogical. You select the entire message, but group by person_id. Unless it is unique, it is indeterminate which row the values are selected from for the group row. ORDER BY is logically performed after GROUP BY, and since timestamp is now indeterminate, so is the resulting order. Some SQL DBMS do not even allow such a query to run, as it is not allowed by the SQL standard.
To fetch distinct sender_ids ordered by their latest timestamp per sender_id do
messagesReceived = db.session.query(Message.sender_id).\
filter_by(recipient_id=user_id).\
group_by(Message.sender_id).\
order_by(db.func.max(Message.timestamp))

If you're using PostgreSQL,
messagesReceived = Message.query.filter_by(recipient_id=user_id).order_by(Message.sender_id.asc(), Message.timestamp.desc()).distinct(Message.sender_id).all()
There may not be a need to group by sender_id. Distinct can also be applied at the Query level (affects the entire query, not just the column), as described here, so the order_by needs the sender_id, post which, the distinct can be applied on a specific column.
This may be specific to PostgreSQL, however, so if you're using another DB, I would recommend the distinct expression as outlined here.
from sqlalchemy import distinct, func
stmt = select([func.count(distinct(users_table.c.name))])

Firestore query takes a too long time to get the value of only one field

. Hi, community.
I have a question/issue about firestore query from Firebase.
I have a collection of around 18000 documents. I would like to get the value of a single same field of some of these documents. I use the python firestore_v1 library from google-cloud-python client. So, for example with list_edges.length = 250:
[db_firestore.document(f"edges/{edge['id']}").get({"distance"}).to_dict()["distance"] for edge in list_edges]
it takes like 30+ seconds to be evaluated, meanwhile with the equal collection on MongoDB it takes not more than 3 seconds doing this and loading the whole object, not only a one field:
list(db_mongo["edges"].find({"city_id":{"$eq":city_id},"id": {"$in": [edge_id for edge in list_edges]}}))
...having said that, I thought the solution could be separate the large collection by city_id, so I create a new collection and copy the corresponded documents inside, so now the query looks like:
[db_firestore.document(f"edges/7/edges/{edge['id']}").get({"distance"}).to_dict()["distance"] for edge in list_edges]
where 7 is a city_id.
However, it takes the same time. So, maybe the issue is around the .get() method, but I could not find any optimized solution for my case.
Could you help me with this? Thanks!
EDITED
I've got the answer from firestore support. The problem is that I make 250 requests doing .get() for each document separately. The idea is to get all the data I want in only one request, so I need to modify the query.
Let's assume I have the next DB:
edges collection with multiples edge_id documents. For each new request, I use a new generated list of edges I need to catch.
In MongoDB, I can do it with the $in operator (having edge_id inside the document), but in firestore, the 'in' operator only accepts up to 10 equality.
So, I need to find out another way to do this.
Any ideas? Thanks!

Firebase recently added support for a limited in operation. See:
The blog post announcing the feature.
The documentation on in and array-contains-any queries.
From the latter:
cities_ref = db.collection(u'cities')
query = cities_ref.where(u'country', u'in', [u'USA', u'Japan'])
A few caveats though:
You can have at most 10 values in the in clause, and you can have only on in (or array-contains-any) clause in query.
I am not sure if you can use this operator to select by ID.

Flask and SQLAlchemy sort in display without new query?

I'm displaying the results from an SQLAlchemy (Flask-SQLAlchemy) query on a particular view. However the sorting/order is only set by what I originally passed into the query ( order_by(desc(SelectedTable.date_changed)) ). I'm trying to now add functionality that each column that is displayed can be selected to order the presentation.
Is there a way to alter the way a returned query object is sorted once it's returned to create this behavior? Or will I need to build custom queries for each possible column that could be sorted by and ascending/descending?
Is there a recipe for implementing something like this? I've tried google, here, the Flask, Flask-SQLAlchemy, and SQLAlchemy docs for something along these lines but haven't seen anything that touches on the subject and beginning to think that I'm going to need to use custom queries or without new queries try some JavaScript in the Jinja Template to achieve this.
Thanks!

Escaping special characters in django-haystack query

I'm trying to use Solr specific syntax with some of my django-haytack queries. For example I'd like to search: "state:Georgia", but haystack sends it to Solr as "state\:Georgia", breaking the query. A Raw query can be used but it seems to need to know the field and query beforehand like so: sqs = SearchQuerySet().filter(author=Raw('state:Georgia')), but I'm not always sure beforehand what the exact field should be. It could be state, collector, material, category or a number of others. Does anyone know a way around this or how I can access the user's actual query?
Or perhaps I'm going down the wrong path and a custom Clean method would be in order?
Thanks

Parsing SQL Query into a DOM-like tree to enable automatic permutation?

I have a large and complicated sql view that I am attempting to debug. There is a record not showing in the view and I need to determine which clause or join is causing the record to now show up. At the moment I am doing this in a very manual way, removing clauses one at a time and running the query to see if the required row shows up.
I think that it would be great if I could do this programmatically because I end up diving into queries like this about once a fortnight.
Does anybody know if there is a way to parse an SQL query into a tree of objects (for example those in sqlalchemy.sql.expression) so I am able to permuate the tree and execute the results?

If you don't already have the view defined in SQLAlchemy, I don't think it can help you.
You could try something like sqlparse which might get you some of the way there.
You could permute it's output and execute the permutations as raw sql using SQLA.