We Keep Coding

How to clear ordering when using .first() in django

I have a query in database as follows. Whenever I try to do a .first() on the query, The equivalent query that gets run in the database is as follows
SELECT * FROM “user" WHERE UPPER("user"."email"::text) = UPPER(%s) **ORDER BY** "registration_user"."id" ASC LIMIT 1
I want to get rid of the order by clause as it interferes with indexes being applied correctly in the db and is also a costly operation. How can I refactor the code below?
users = User.objects.filter(email__iexact=email)
users.query.**clear_ordering**(True)
if users.count() > 0 :
return users.first()

If no ordering is specified, that would mean that two calls with .first() can return a different element, and non-determinism often results in a lot of problems.
The ORDER BY pk is added by the .first() [Django-doc] call, so it is not part of your query at all. If the queryset has no ordering, then .first() will add an ordering by pk (primary key), as is described in the documentation:
first()
Returns the first object matched by the queryset, or None if there
is no matching object. If the QuerySet has no ordering defined, then
the queryset is automatically ordered by the primary key. This can
affect aggregation results as described in Interaction with default
ordering or order_by().
If you really do not want an ordering, you can subscript the queryset:
users = User.objects.filter(email__iexact=email)
if users.exists():
return users[0]
But that does not look like a very good idea. If no order is specified, then the database can return any record that matches the filtering condition, and it can thus return a different record each query.

Django how to query objects one by one

I have to query an object from a model, this object is called "exercise" and has many fields(title, body, answers, etc.) I need to get "exercise objects" one by one with all their fields, do some rendering on them and give back a pdf of the exercise as result.
The problem is, if I do:
exercises = ExerciseModel.objects.all()
I get all of them. If I do:
some_exercises = ExerciseModel.objects.filter(something=something)
I get "some of them" depending on filter. And if i do:
exercise = ExerciseModel.objects.get()
I get
error get() returned more than one exercise
How can I get them one by one? I must be able to use them as input for another function.

If you need to perform the task on only 1 exercise, use get() instead. In such case, you need a criteria to make sure get() will return 1 and only 1 result.
Example:
ex = ExerciseModel.objects.get(pk=123)
From the doc:
Returns the object matching the given lookup parameters, which should be in the format described in Field lookups.
get() raises MultipleObjectsReturned if more than one object was found. The MultipleObjectsReturned exception is an attribute of the model class.
get() raises a DoesNotExist exception if an object wasn’t found for the given parameters. This exception is an attribute of the model class.
When you have a QuerySet with filters that ensure the underlying SQL request will return only 1 row, you can call get() without argument on the QuerySet. This return the row as model instance, instead of returning it as a list containing 1 elemnt.
Original answer:
Both filter() and all() methods return a QuerySet instance. You can iterate on it to perform a task for each "exercise" returned by your request
for exercise in ExerciseModel.objects.filter(something=something):
# ... do what you need on your exercise
From the documentation:
A QuerySet is iterable, and it executes its database query the first time you iterate over it. For example, this will print the headline of all entries in the database

What you need is .iterator() which will turn a queryset to an iterator:
exercises = ExerciseModel.objects.all().iterator()
then you get iterate over it in a for loop:
for exercise in exercises:
...
This will improve performance when you have large number of items. However, it has some downsides as well as stated in the docs
Hope it helps!

.get() must return a single object, not a QuerySet instance. If you want to get a single exercise then you must pass a parameter to the .get instance. For example, retrieving an object by name would require the following code:
exercise = ExerciseModel.objects.get(name="an_exercise")
If you want to iterate through all the objects without actually retrieving a QuerySet containing the objects, you could use the following code:
for i in range(Exercise.objects.all().count()):
exercise = Exercise.objects.get(pk=i)
... operate on object ...

Get full label name of ChoiceField in QuerySet values list

I have a model with a ChoiceField, and I'm doing getting a QuerySet as values, i.e.:
list = Model.objects.filter(a='1').values('id','a','choicefield')
In this case the value of choicefield would be returned as whatever is stored in the DB. If I were getting the model returned I could do list.0.get_choicefield_display to get the full label value. There doesn't seem to be any way to get the value though when I use a values list. I've tried changing out the variable name on the values list with a get_x_display version but that doesn't work either.
Any ideas?

I think the easiest way would be to use only() instead of values(). Like values, only retrieves a subset of the database columns. The difference is that it creates model instance objects, which means that you can use get_X_display() as usual.
instances = MyModel.objects.filter(a='1').only('id', 'a', 'choicefield')
instances[0].get_choicefield_display() # will work fine
Of course, you should check out the caveats in the documentation on only() and defer(), but for the purposes you've described I don't see a problem.
If you want to use values() there are many ways to solve this problem. One approach is to define the mapping explicitly in your model:
from collections import OrderedDict
class MyModel(models.Model):
CHOICES_DICT = OrderedDict((
(1, 'Choice 1'),
(2, 'Choice 2'),
))
choicefield = models.IntegerField(choices=CHOICES_DICT.items())
...
Now you can explicitly look up up the string value in your view:
values = MyModel.objects.filter(a='1').values('id','a','choicefield')
name = MyModel.CHOICES_DICT[values[0]['choicefield']]
You could also create a custom template filter to do the same thing at the template level.

How to combine two querysets when defining choices in a ModelMultipleChoiceField?

I am using a piece of code in two separate places in order to dynamically generate some form fields. In both cases, dynamic_fields is a dictionary where the keys are objects and the values are lists of objects (in the event of an empty list, the value is False instead):
class ExampleForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
dynamic_fields = kwargs.pop('dynamic_fields')
super(ExampleForm, self).__init__(*args, **kwargs)
for key in dynamic_fields:
if dynamic_fields[key]:
self.fields[key.description] = forms.ModelMultipleChoiceField(widget=forms.CheckboxSelectMultiple, queryset=dynamic_fields[key], required=False)
class Meta:
model = Foo
fields = ()
In one view, for any key the value is a list of objects returned with a single DB query - a single, normal queryset. This view works just fine.
In the other view, it takes multiple queries to get everything I need to construct a given value. I am first instantiating the dictionary with the values set equal to blank lists, then adding the querysets I get from these multiple queries to the appropriate lists one at a time with basic list comprehension (dict[key] += queryset). This makes each value a 2-D list, which I then flatten (and remove duplicates) by doing:
for key in dict:
dict[key] = list(set(dict[key]))
I have tried this several different ways - directly appending the queries in each queryset to the values/lists, leaving it as a list of lists, using append instead of += - but I get the same error every time: 'list' object has no attribute 'none'.
Looking through the traceback, the error is coming up in the form's clean method. This is the relevant section from the code in django.forms.models:
def clean(self, value):
if self.required and not value:
raise ValidationError(self.error_messages['required'], code='required')
elif not self.required and not value:
return self.queryset.none() # the offending line
My thought process so far: in my first view, I'm generating the list that serves as the value for each key via a single query, but I'm combining multiple queries into a list in my second view. That list doesn't have a none method like I would normally have with a single queryset.
How do I combine multiple querysets without losing access to this method?
I found this post, but I'm still running into the same issue using itertools.chain as suggested there. The only thing I've been able to accomplish with that is changing the error to say 'chain' or 'set' object has no attribute 'none'.
Edit: here's some additional information about how the querysets are generated. I have the following models (only relevant fields are shown):
class Profile(models.Model):
user = models.OneToOneField(User)
preferred_genres = models.ManyToManyField(Genre, blank=True)
class Genre(models.Model):
description = models.CharField(max_length=200, unique=True)
parent = models.ForeignKey("Genre", null=True, blank=True)
class Trope(models.Model):
description = models.CharField(max_length=200, unique=True)
genre_relation = models.ManyToManyField(Genre)
In (the working) view #1, the dictionary I use to generate my fields has keys equal to a certain Genre, and values equal to a list of Genres for whom the key is a parent. In other words, for every key, the queryset is Genre.objects.filter(parent=key, **kwargs).
In the non-functional view #2, we need to start with the profile's preferred_genres field. For every preferred_genre I need to pull the associated Tropes and combine them into a single queryset. Right now, I am looping through preferred_genres and doing something like this:
for g in preferred_genres:
tropeset = g.trope_set.all()
This gets me a bunch of individual querysets containing the information I need, but I can't find a way to combine the multiple tropesets into one big queryset (as opposed to a list without the none attribute). (As an aside, this also hammers my database with a bunch of queries. I am also trying to wrap my head around how I can maybe use prefetch_related to reduce the number of queries, but one thing at a time.)
If I can't combine these querysets into one but CAN somehow accomplish these lookups with a single query, I am all ears! I am now reading the documentation regarding complex queries with the Q object. It is tantalizing - I can conceptualize how this would accomplish what I'm looking for, but only if I can call all of the queries at one time. Since I have to call them iteratively one at a time, I am not sure how to use the Q object to | or & them together.

You can combine querysets by using the | and & operators.
from functools import reduce
from operator import and_, or_
querysets = [q1, q2, q3, ...] # List of querysets you want to combine.
# Objects that are present in *at least one* of the queries
combined_or_querysets = reduce(or_, querysets[1:], querysets[0])
# Objects that are present in *all* of the queries
combined_and_querysets = reduce(and_, querysets[1:], querysets[0])
From Django 1.11+ you can also use the union and intersection methods.

I've found a "solution" to this problem. If I structure the query like so, I can get everything I need in one swoop without having to combine querysets after the fact:
desired_value = Trope.objects.filter(genre_relation__in=preferred_genres).distinct()
I still do not know how to combine multiple querysets into one without losing the inherent "queryset-ness" that seems to be necessary for the form to render properly. However, for my specific use case, restructuring the query as noted renders the issue moot.

Django: accessing model attributes

Apologies for the noobish question, I am completely new to both Python and Django and trying to make my first app.
I have a simple class
class About(models.Model):
title = models.CharField(max_length=200)
description = models.TextField()
date = models.DateTimeField('date added')
to which I've added a single record. I can access this with
about = About.objects.filter(id=1)
however, if I try to use dot syntax to access its attributes I get the following error
>>> about.title
Traceback (most recent call last):
File "<console>", line 1, in <module>
AttributeError: 'QuerySet' object has no attribute 'title'
I know how to use unicode in the model to specify a nicer return value such as
def __unicode__(self):
return self.title
should I be using this to format the model data into a dictionary/list? Or am I just completely missing some default behaviour?

In your case, about is a QuerySet object, not an instance of your model. Try
print about[0].title
Alternatively, use get() to retrieve a single instance of the model:
about = About.objects.get(id=1)
print about.title

Filter returns a QuerySet and not the single object you are looking for. Use get instead of filter.
Methods that return new QuerySets
filter
...
Methods that do not return QuerySets
get
...
http://docs.djangoproject.com/en/dev/ref/models/querysets/

As the documentation explains, filter always returns a QuerySet, which is a list-like collection of items, even if only one element matches the filter condition. So you can slice the list to access your element - about[0] - or, better, use get() instead:
about = About.objects.get(id=1)
print about.title

If you want get just one row
about = About.objects.get(pk=1)
now about is an object(one row)
filter returns list, so for accessing items in list you must use index(about[0]) or for loop.but get return exactly one row.