I need to get the last X rows from a table, but in order of the ID. How could I achieve this?
query = users.select().order_by(users.c.id.desc()).limit(5)
print reversed(conn.execute(query).fetchall() )
something like that anyway
This worked for me...
c=session.query(a).order_by(a.id.desc()).limit(2)
c=c[::-1]
This solution is 10 times faster than the python slicing solution proposed by BrendanSimon.
I believe you can prefix the order_by parameter with a '-' to get reverse order.
query = users.select().order_by(-users.c.id.desc()).limit(5)
Also, I believe you can use python slices as an alternative to limit.
query = users.select().order_by(users.c.id.desc())[-5:]
query = users.select().order_by(-users.c.id.desc())[:5]
Related
I have a database which I need to return any value that matches the query or looks similar.
Using flask-sqlalchemy I can filter manually, but I'm having trouble getting a list of objects using list comprehensions or any other more pythonic method.
I have already tried to create a dict with all model columns (which I have a list ) and the search value passed to the filter_by query
column_dict_and_value = {column_name:'value' for column_name in columns}
a = model.query.filter_by(**column_dict_and_value).all()
...but doesn't even return an existing object, although
a = model.query.filter_by(column_name='value').all()
actually returns the object.
I tried filter(.like('%')) and it sort of works
a = model.query.filter(model.column_name.like('valu%')).all()
returns me a list of all the objects that matches that pattern but for that column, and I want to iterate over all columns. Basically a full blown search since I'm not sure what the user is going to look for and I want to show the object if the query exist on any column or a list of objects if the query is incomplete, like the * in any search.
I tried to use list comprehensions to iterate through the model attributes but I'm not allowed to do that apparently:
a = [model.query.filter(model.column.like('valu%')) for column in columns]
...this complains that column is not an attribute of the model, which makes sense due to the syntax, but I had to try anyway, innit?
I'm a newbie regarding databases and class objects so please be gentle. I tried to look for similar answers but I can't find the one that suits me. the filter_by(one_column, second_column, etc) doesn't look to me very pythonic and if for any reason I change the model i need to change the query too where as creating a dict comprehension of the model seems to me more foolproof.
SOLUTION
Based on calestini proposed answer I wrote this code that seems to do the trick. It also cleans the list because if all() doesn't have any result (since it looks in every field) it returns None and adds up to the list. I'd rather prefer it clean.
Note: All my fields are text. Please check the third proposed solution from calestini if yours differ. Not tested it though.
columns = [
"foo",
"bar",
"foobar"
]
def list_everything(search):
d = {column: search for column in columns}
raw = [
model.query.filter(getattr(model, col).ilike(f"{val}%")).all()
for col, val in d.items()
]
return [item for item in raw if item]
I'll keep optimizing and update this code if I come with a better solution. Thanks a lot
Probably the reason why you are not getting any result from
a = model.query.filter_by(**column_dict_and_value).all()
is because filter_by is testing for direct equality. In your case you are looking for a pattern, so you will need to loop and use filter as opposed to filter_by.
Assuming all your columns are String or Text types, you can try the following:
a = model.query
for col, val in column_dict_and_value.items():
a = a.filter(getattr(model, col).ilike(f'{val}%')) #ilike for case insensitive
a = a.all()
Or vs And
The issue can also be that in your case you could be testing for intersection but expecting union. In other words, you are returning something only if the pattern matches in all columns. If instead you want a result in case any column matches, then you need to tweak the code a bit:
condition = or_(*[getattr(model, col).ilike(f'{val}%') for col, val in column_dict_and_value.items()])
a = model.query.filter(codition).all()
Now if you actually have other data types among the columns, then you can try to first verify if the column type and then pass the same logic:
for col, val in column_dict_and_value.items():
## check if field python equivalent is string
if isinstance(class_.__table__.c[col].type.python_type, str):
...
How can I get the following using Pandas query.
SELECT site_id, count(issue) FROM [Randall]
where site_id >3
group by site_id
LIMIT 10
My query could be found below; However, when executed it have 2 'issue' columns, one for the actual issue and another the 'count' and I have repetitive issues. what I want is to sum the issues by site.
w_alarms.groupby(['site_id', 'issue']).size()
Somethink like
w_alarms[w_alarms.site_id > 3].groupby('site_id')['issue'].count()
Try
w_alarms.siteid[w_alarms.siteid>3].value_counts().head(10)
You don't provide example of output you desired.
You might want this:
w_alarms.issue.groupby[w_alarms.siteid[w_alarms.siteid > 3]].count()
Using a queryset in Django (in a view) I only want to get said rows 51-100. i.e. I only want it to return these rows.
is this possible and how within .
objectQuerySet = Recipient.objects.filter(incentiveid=incentive).order_by('fullname')
I don't want to use any paging system etc this is just a one time thing?
Thank you
You can use slicing to execute a LIMIT OFFSET statement:
objectQuerySet = Recipient.objects.filter(incentiveid=incentive).order_by('fullname')[51:100]
MyModel:
name = models.CharField(max_length=255)
I try to sort the queryset. I just think about this:
obj = MyModel.objects.all().sort_by(-len(name)) #???
Any idea?
The new hotness (as of Django 1.8 or so) is Length()
from django.db.models.functions import Length
obj = MyModel.objects.all().order_by(Length('name').asc())
you might have to sort that in python..
sorted(MyModel.objects.all(),key=lambda o:len(o.name),reverse=True)
or I lied ( A quick google search found the following)
MyModel.objects.extra(select={'length':'Length(name)'}).order_by('length')
You can of course sort the results using Python's sorted, but that's not ideal. Instead, you could try this:
MyModel.objects.extra(select={'length':'Length(name)'}).order_by('length')
You'll need to use the extra argument to pass an SQL function:
obj = MyModel.objects.all().extra(order_by=['LENGTH(`name`)'])
Note that this is db-specific: MySQL uses LENGTH, others might use LEN.
I have a large set of values V, some of which are likely to exist in a table T. I would like to insert into the table those which are not yet inserted. So far I have the code:
for value in values:
s = self.conn.execute(mytable.__table__.select(mytable.value == value)).first()
if not s:
to_insert.append(value)
I feel like this is running slower than it should. I have a few related questions:
Is there a way to construct a select statement such that you provide a list (in this case, 'values') to which sqlalchemy responds with records which match that list?
Is this code overly expensive in constructing select objects? Is there a way to construct a single select statement, then parameterize at execution time?
For the first question, something like this if I understand your question correctly
mytable.__table__.select(mytable.value.in_(values)
For the second question, querying this by 1 row at a time is overly expensive indeed, although you might not have a choice in the matter. As far as I know there is no tuple select support in SQLAlchemy so if there are multiple variables (think polymorhpic keys) than SQLAlchemy can't help you.
Either way, if you select all matching rows and insert the difference you should be done :)
Something like this should work:
results = self.conn.execute(mytable.__table__.select(mytable.value.in_(values))
available_values = set(row.value for row in results)
to_insert = set(values) - available_values