We Keep Coding

How to clear ordering when using .first() in django

I have a query in database as follows. Whenever I try to do a .first() on the query, The equivalent query that gets run in the database is as follows
SELECT * FROM “user" WHERE UPPER("user"."email"::text) = UPPER(%s) **ORDER BY** "registration_user"."id" ASC LIMIT 1
I want to get rid of the order by clause as it interferes with indexes being applied correctly in the db and is also a costly operation. How can I refactor the code below?
users = User.objects.filter(email__iexact=email)
users.query.**clear_ordering**(True)
if users.count() > 0 :
return users.first()

If no ordering is specified, that would mean that two calls with .first() can return a different element, and non-determinism often results in a lot of problems.
The ORDER BY pk is added by the .first() [Django-doc] call, so it is not part of your query at all. If the queryset has no ordering, then .first() will add an ordering by pk (primary key), as is described in the documentation:
first()
Returns the first object matched by the queryset, or None if there
is no matching object. If the QuerySet has no ordering defined, then
the queryset is automatically ordered by the primary key. This can
affect aggregation results as described in Interaction with default
ordering or order_by().
If you really do not want an ordering, you can subscript the queryset:
users = User.objects.filter(email__iexact=email)
if users.exists():
return users[0]
But that does not look like a very good idea. If no order is specified, then the database can return any record that matches the filtering condition, and it can thus return a different record each query.

How to retrieve value from queryset in django?

I am trying to retrieve different .values() from query sets but am having an issue with it returning the proper values. How to write my model so that I can retrieve attributes using the .values() method?
I have tried to change the model's __str__ method to return a dictionary but that does not work or I am doing it wrong.
class Settings(models.Model):
bb_bonus_qualify = models.CharField(max_length=16, default=38.00)
service_breakpoint = models.CharField(max_length=16, default=1700.00)
def __str__(self):
return '%s: %s, %s: %s' % (
'bb_bonus', self.bb_bonus_qualify, 'service_breakpoint', self.service_breakpoint)
I would like to say Settings.objects.last().values('bb_bonus') and be returned the value which is self.bb_bonus_qualify. The common error I seem to get is: AttributeError: 'Settings' object has no attribute 'values'

The problem here is that your .last() will retrieve the last Settings object. You thus will call .values('bb_bonus') on the Settings object. Since a model has no .values(..) method, it will thus not return anything.
You can however retrieve the value of a certain column from a queryset, with:
Settings.objects.values_list('bb_bonus_qualify', flat=True).last()
We here thus use .values_list(..) [Django-doc], this accepts the names of the columns as parameters. It will then usually return a QuerySet of lists with these values. But if you specify one column; then, as the documentation says:
If you only pass in a single field, you can also pass in the flat parameter. If True, this will mean the returned results are single values, rather than one-tuples.
So that means we create a QuerySet of singular values, and we then will retrieve the last entry of that queryset. Note that we do not fetch all the elements from the queryset, the .last() is "injected" in the query we perform on the database, so the result is the scalar value of that column for the last record.
The .values_list(..) thus needs to be performed before the .last(), since otherwise you are talking to a Settings object, not to a QuerySet.

AFAIK __str__ has nothing to do with .values() - the problem here is that you need to specify the values before getting a specific item, rather than the other way round:
Settings.objects.values('bb_bonus').last()

Order a django queryset by ManyToManyField = Value

If have some models like:
class Tag(models.Model):
name = models.CharField()
class Thing(models.Model):
title = models.CharField()
tags = models.ManyToManyField(Tag)
I can do a filter:
Thing.objects.filter(tags__name='foo')
Thing.objects.filter(tags__name__in=['foo', 'bar'])
But is it possible to order a queryset on the tags value?
Thing.objects.order_by(tags__name='foo')
Thing.objects.order_by(tags__name__in=['foo','bar'])
What I would expect (or like) back in this example, would be ALL Thing models, but ordered where they have a Tag/Tags that I know. I don't want to filter them out, but bring them to the top.
I gather this is possible using the FIELD operator, but seemingly I can only make it work on columns in that models table, e.g. title, but not on linked tables.
Thanks!
EDIT: After having accepted the below solution, I realised a bug/limitation with it.
If a particular Thing has multiple Tags, then (due to the left join done behind the scenes in the SQL) it will produce one entry for that Thing, for each Tag that it has. With a True or False for each Tag that matches or not.
Adding .distinct() to the queryset helps only slightly, limiting to a max of 2 rows per Thing (i.e. one tagged=True, and one tagged=False).
I know what I need to do in the SQL, which is to MAX() the CASE(), and then GROUP BY Thing's primary key, which means I will get one row per Thing, and if there has been any tag matches, tagged will be True (and False otherwise).
I see the way that people typically achieve this kind of thing is to use .values() like this:
Thing.objects.values('pk').annotate(tagged=Max(Case(...)))
But the result is only pk and tagged, I need the whole Thing model as the result. So I've managed to achieve what I want, thusly:
from django.db.models import Case, When, Max, BooleanField
tags = ['music'] # for example
queryset = Thing.objects.all().annotate(tagged=Max(Case(
When(tags__name__in=tags, then=True),
default=False,
output_field=BooleanField()
)))
queryset.query.group_by = ['pk']
queryset.order_by('-tagged')
This seems to work, but the group by mechanism feels weird/hacky. Is it acceptable/reliable to group in this way?
Sorry for the epic updated :(

I'd try annotate the query with the conditional value that turns true when the tag is in the list you provide
from django.db.models import Case, When, IntegerField
Thing.objects.annotate(tag_is_known=Case(
When(tags__name__in=['foo', 'bar'], then=1),
default=0,
output_field=IntegerField()
))
Next we use that annotation we called tag_is_known to sort with order_by():
Thing.objects.annotate(tag_is_known=...).order_by('tag_is_known')
Boolean version
Thing.objects.annotate(tag_is_known=Case(
When(tags__name__in=['foo', 'bar'], then=True),
default=False,
output_field=BooleanField()
))

Move an element inside a Queryset

I have a queryset in my django view.
I would like to change the position of one of the item in order it to be the first element of the queryset.
Any idea on how to do that?
EDIT:
I have a queryset:
qs = UserProfile.objects.filter(myfilter=whatever)
I know that inside this queryset I have:
specific_user.userprofile
What I want to do is to put the user.userprofile in first position into my queryset because I use this queryset in a loop in my template:
{% for i in qs %}
<div> i.name </div>
And I want to be sure that the first name of the list is the name of the specific_user.

QuerySet objects in Django are abstract representations of the result of database queries. Generally speaking, it represents SQL that might later be executed when specific objects from the queryset are requested, but until that happens, no queries are executed. So you can't really think of a queryset like a list, which might be arbitrarily reordered. You can request the objects in your queryset to be sorted using its order_by method, so if you want a particular object to come first, it must have some sortable attribute on the DB side of things by which it can be ordered. This could be a column or a value calculated from its columns using F expressions.
But, for your needs, it seems like you could just make a list of your queryset in your view:
profiles = [specific_user.userprofile] + [profile for profile in UserProfile.objects.filter(myfilter=whatever).exclude(id=specific_user.userprofile.id)]
Add that to your context and iterate over it instead of the original queryset in your template.

Since Django 1.8 you have Conditional Expressions which you can use to define a specific order.
You would set a special case when the UserProfile is specific_user.userprofile:
qs = UserProfile.objects.filter(myfilter=whatever)\
.order_by(Case(When(pk=specific_user.userprofile.pk, then=0), default=1)))
We set the value of 0 in the special case, and 1 as default.
This way the specified user would always show first.

Django Query using .order_by() and .latest()

I have a model:
class MyModel(models.Model):
creation_date = models.DateTimeField(auto_now_add = True, editable=False)
class Meta:
get_latest_by = 'creation_date'
I had a query in my view that did the following:
instances = MyModel.objects.all().order_by('creation_date')
And then later I wanted instances.latest(), but it would not give me the correct instance, in fact it gave me the first instance. Only when I set order_by to -creation_date or actually removed the order_by from the query did .latest() give me the correct instance. This also happens when I test this manually using python manage.py shell instead of in the view.
So what I've done now is in the Model's Meta I've listed order_by = ['creation_date'] and not used that in the query, and that works.
I would have expected .latest() to always return the most recent instance based on a (date)(time) field. Could anyone tell me whether it's correct that .latest() behaves strangely when you use order_by in the query?

I would have expected .latest() to always return the most recent instance based on a (date)(time) field.
The documentation says that
If your model's Meta specifies get_latest_by, you can leave off the field_name argument to latest(). Django will use the field specified in get_latest_by by default.
All this means is that when you fire MyModel.objects.latest() you will get the latest instance based on the date/time field. And when I tested your code using sample data, it indeed did.
And then later I wanted instances.latest(), but it would not give me the correct instance, in fact it gave me the first instance.
You have misunderstood the way latest() works. When called on MyModel.objects it returns
the latest instance in the table. When called on a queryset, latest will return the first object in the queryset. Your queryset consisted of all instances of MyModel ordered by creation_date in ascending order. It is only natural then that latest on this queryset should return the first row of the queryset. This incidentally happens to be the oldest row in the table.
One way to get a better understanding is to view the query fired for latest.
Case 1:
from django.db import connection
MyModel.objects.latest()
print connection.queries[-1]['sql']
This prints:
SELECT "app_mymodel"."id", "app_mymodel"."creation_date" FROM
"app_mymodel" ORDER BY "app_mymodel"."creation_date" DESC LIMIT 1
Note the ordering by creation_date DESC and the LIMIT clause. The former is thanks to get_latest_by whereas the latter is the contribution of latest.
Now, case 2:
MyModel.objects.order_by('creation_date').latest()
print connection.queries[-1]['sql']
prints
SELECT "app_mymodel"."id", "app_mymodel"."creation_date" FROM
"app_mymodel" ORDER BY "app_mymodel"."creation_date" ASC LIMIT 1
Note that the ordering has changed to creation_date ASC. This is the result of the explicit order_by. The LIMIT is tacked on er, later courtesy latest.
Let us also see Case 3: where you explicitly specify the field_name for objects.latest().
MyModel.objects.latest('id')
print connection.queries[-1]['sql']
shows
SELECT "app_mymodel"."id", "app_mymodel"."creation_date" FROM "app_mymodel"
ORDER BY "app_mymodel"."id" DESC LIMIT 1

I guess this is a known bug in Django that was fixed after 1.3 was released.

This worked for me
latestsetuplist = SetupTemplate.objects.order_by('-creationTime')[:10][::1]

if we have the value of id or date
post_id = BlogPost.objects.get(id=id)
try:
previous_post = BlogPost.objects.all().order_by('id')[post_id.id-2:post_id.id-1]
except:
previous_post = None
try:
next_post = BlogPost.objects.all().order_by('id')[post_id.id:post_id.id+1]
except:
next_post = None
it worked for me, even if an id is missing it picks next or previous value to that