I am learning class based views in Django. I was reading the Django documentation and read about queryset attribute and the get_queryset() method. When googled them I came across this answer.
I tried to replicate the result using my code:
class IndexView(generic.ListView):
template_name = 'polls/index.html'
context_object_name = 'latest_question_list'
def get_queryset(self):
return Question.objects.order_by('-pub_date')[:2]
class IndexView2(generic.ListView):
template_name = 'polls/index2.html'
context_object_name = 'latest_question_list2'
queryset = Question.objects.all()
In answer it is mentioned that when you set queryset, the queryset is created only once, when you start your server. On the other hand, the get_queryset method is called for every request.
But I was able to insert questions in database and they were available in the page index2.html without restarting, I was able to change the database and changes were reflected on the page index2.html after refreshing the page.
I further googled and found this link. In the DRF website, it is mentioned that queryset will get evaluated once, and those results will be cached for all subsequent requests.
Can you point where I am going wrong ? What link I am missing ?
A QuerySet is evaluated once, but the default implementation of get_queryset, will use queryset.all(), thus each time constructing a new queryset that will force reevaluation.
Indeed, the implementation of the .get_queryset(…) method [GitHub] works with:
def get_queryset(self):
if self.queryset is not None:
queryset = self.queryset
if isinstance(queryset, QuerySet):
queryset = queryset.all()
elif self.model is not None:
queryset = self.model._default_manager.all()
else:
raise ImproperlyConfigured(
"%(cls)s is missing a QuerySet. Define "
"%(cls)s.model, %(cls)s.queryset, or override "
"%(cls)s.get_queryset()." % {
'cls': self.__class__.__name__
}
)
ordering = self.get_ordering()
if ordering:
if isinstance(ordering, str):
ordering = (ordering,)
queryset = queryset.order_by(*ordering)
return queryset
THis thus means that we each time make a new "copy" of the QuerySet that will be evaluated. In case the queryset is not specified, it will look for the model attribute, and work with the _default_manager for that model.
If you specified an ordering attribute, than that means it will also order the queryset.
Related
Good day to all, I have been trying to make this query be consulted every time the REST service is used in the API, but only the first time it obtains the data from the DB and when the data changes the service only brings the cache stored data
My Code:
urls.py
router.register(r'cron-log',views.CronLogViewSet, base_name='cron-log')
Views.py - my viewset class
class CronLogViewSet(viewsets.ModelViewSet):
queryset = Cron_log.objects.all().order_by('-id').values()[:5:1]
serializer_class = CronLogSerializer
Models.py my model class from Cron_log
class Cron_log(models.Model):
log = models.CharField(max_length=40)
time = models.CharField(max_length=40)
def as_dict(self):
return {'log':self.log,'time':self.time}
Serializer.py serializer class
class CronLogSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Cron_log
fields = ('log','time')
I tried with a for before the queryset, with list(query_set) but the issue still there thanks! for your help
The QuerySet is not updating for a couple reasons. 1. It is being evaluated with a slice that uses the step parameter (the step parameter is the 1 in [:5:1] (which is superfluous as 1 is the default - unless you want to trigger an evaluation)). 2. It is evaluated only once in the class body definition which does not update every time an instance of the viewset is initialized. See the Django documentation's section on when querysets are evaluated for more information regarding slicing.
If you want the viewset to always return an updated queryset of objects limited to the first 5 ordered by descending id, one option is to place the limit slice in a get_queryset instance method that returns the queryset limited to the first 5 results.
class CronLogViewSet(viewsets.ModelViewSet):
queryset = Cron_log.objects.all().order_by('-id')
serializer_class = CronLogSerializer
def get_queryset(self):
queryset = super(CronLogViewSet, self).get_queryset()
return queryset.values()[:5]
See the Django Rest Framework documentation's section on filtering and, more specifically, the implementation of views.GenericAPIView.get_queryset for more information.
I am using Class-based Generic views Listview for listing all objects.
My views.py:
class PostsList(ListView):
model = Post
template_name = "index.html"
My Urls.py:
urlpatterns = [
url(r'^$',PostsList.as_view(), name = "home"),
]
This gives me a list of all the posts. Now I want to filter/sort posts based on certain fields of Post Model, say price. Do I need to write this myself? If yes Which method of PostsLists class do I override ? def get, def get_context ?
I see the get method for Listview defined as below. In it can I pass URL query-parameters as **kwargs directly or I have to overwrite the below method in my class.
def get(self, request, *args, **kwargs):
....
You can override the get_queryset method:
Keep a mapping of all the parameters that you can get in the url kwargs.
def get_queryset(self):
queryset = Post.objects.all()
if self.request.GET.get('price'):
queryset = queryset.filter(price=self.request.GET.get('price'))
return queryset
When using Django's class based views, avoid overriding get() or post() if possible. These methods do a lot, and if you override them, you may have to duplicate a lot of the built in functionality. There are normally more specific methods that you can override.
In your case, you can filter the queryset dynamically with the get_queryset method. You can access GET parameters with self.request.GET. For example:
class PostsList(ListView):
model = Post
def get_queryset(self):
"""Filter by price if it is provided in GET parameters"""
queryset = super(PostsList, self).get_queryset()
if 'price' in self.request.GET:
queryset = queryset.filter(price=self.request.GET['price'])
return queryset
If your url captures arguments, you can access them with self.args (positional) and self.kwargs (name based).
See the docs on dynamic filtering for more info.
I have a Mixin that allow me update the objects that I have created already, thing is that I have too many models and each one with different fields, this Mixin when not found the object return a 404, I need when the object is not found return the form for create the object associated to predio_id object , I have tried with get_object_or_create, but with this method I have to pass each field. How can achieve that when the object is not found, return his corresponding empty form for create it?
class UpdateModelMixin(object):
def get_object(self):
return get_object_or_404(self.model,predio_id=self.kwargs['predio_id'])
and it's called to view like this one:
class ManejoGeneralUpdateView(UpdateModelMixin, UpdateView):
model = ManejoGeneral
form_class = FormManejoGeneral
success_url = '/'
template_name = 'manejo_finca/edit/manejo_general.html'
Note that the UpdateView that I wrote here is just one of almost 30 o 40 UpdateViews that I have because each UpdateView call a different form and template
It was pretty simple, since get_or_create return a tuple, just add [0] at the end of the query:
class UpdateModelMixin(object):
def get_object(self):
return Persona.objects.get_or_create(predio_id=self.kwargs['predio_id'])[0]
I have the following models:
class User(models.Model):
name = models.Charfield()
email = models.EmailField()
class Friendship(models.Model):
from_friend = models.ForeignKey(User)
to_friend = models.ForeignKey(User)
And those models are used in the following view and serializer:
class GetAllUsers(generics.ListAPIView):
authentication_classes = (SessionAuthentication, TokenAuthentication)
permission_classes = (permissions.IsAuthenticated,)
serializer_class = GetAllUsersSerializer
model = User
def get_queryset(self):
return User.objects.all()
class GetAllUsersSerializer(serializers.ModelSerializer):
is_friend_already = serializers.SerializerMethodField('get_is_friend_already')
class Meta:
model = User
fields = ('id', 'name', 'email', 'is_friend_already',)
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
if request.user != obj and Friendship.objects.filter(from_friend = user):
return True
else:
return False
So basically, for each user returned by the GetAllUsers view, I want to print out whether the user is a friend with the requester (actually I should check both from_ and to_friend, but does not matter for the question in point)
What I see is that for N users in database, there is 1 query for getting all N users, and then 1xN queries in the serializer's get_is_friend_already
Is there a way to avoid this in the rest-framework way? Maybe something like passing a select_related included query to the serializer that has the relevant Friendship rows?
Django REST Framework cannot automatically optimize queries for you, in the same way that Django itself won't. There are places you can look at for tips, including the Django documentation. It has been mentioned that Django REST Framework should automatically, though there are some challenges associated with that.
This question is very specific to your case, where you are using a custom SerializerMethodField that makes a request for each object that is returned. Because you are making a new request (using the Friends.objects manager), it is very difficult to optimize the query.
You can make the problem better though, by not creating a new queryset and instead getting the friend count from other places. This will require a backwards relation to be created on the Friendship model, most likely through the related_name parameter on the field, so you can prefetch all of the Friendship objects. But this is only useful if you need the full objects, and not just a count of the objects.
This would result in a view and serializer similar to the following:
class Friendship(models.Model):
from_friend = models.ForeignKey(User, related_name="friends")
to_friend = models.ForeignKey(User)
class GetAllUsers(generics.ListAPIView):
...
def get_queryset(self):
return User.objects.all().prefetch_related("friends")
class GetAllUsersSerializer(serializers.ModelSerializer):
...
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
friends = set(friend.from_friend_id for friend in obj.friends)
if request.user != obj and request.user.id in friends:
return True
else:
return False
If you just need a count of the objects (similar to using queryset.count() or queryset.exists()), you can include annotate the rows in the queryset with the counts of reverse relationships. This would be done in your get_queryset method, by adding .annotate(friends_count=Count("friends")) to the end (if the related_name was friends), which will set the friends_count attribute on each object to the number of friends.
This would result in a view and serializer similar to the following:
class Friendship(models.Model):
from_friend = models.ForeignKey(User, related_name="friends")
to_friend = models.ForeignKey(User)
class GetAllUsers(generics.ListAPIView):
...
def get_queryset(self):
from django.db.models import Count
return User.objects.all().annotate(friends_count=Count("friends"))
class GetAllUsersSerializer(serializers.ModelSerializer):
...
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
if request.user != obj and obj.friends_count > 0:
return True
else:
return False
Both of these solutions will avoid N+1 queries, but the one you pick depends on what you are trying to achieve.
Described N+1 problem is a number one issue during Django REST Framework performance optimization, so from various opinions, it requires more solid approach then direct prefetch_related() or select_related() in get_queryset() view method.
Based on collected information, here's a robust solution that eliminates N+1 (using OP's code as an example). It's based on decorators and slightly less coupled for larger applications.
Serializer:
class GetAllUsersSerializer(serializers.ModelSerializer):
friends = FriendSerializer(read_only=True, many=True)
# ...
#staticmethod
def setup_eager_loading(queryset):
queryset = queryset.prefetch_related("friends")
return queryset
Here we use static class method to build the specific queryset.
Decorator:
def setup_eager_loading(get_queryset):
def decorator(self):
queryset = get_queryset(self)
queryset = self.get_serializer_class().setup_eager_loading(queryset)
return queryset
return decorator
This function modifies returned queryset in order to fetch related records for a model as defined in setup_eager_loading serializer method.
View:
class GetAllUsers(generics.ListAPIView):
serializer_class = GetAllUsersSerializer
#setup_eager_loading
def get_queryset(self):
return User.objects.all()
This pattern may look like an overkill, but it's certainly more DRY and has advantage over direct queryset modification inside views, as it allows more control over related entities and eliminates unnecessary nesting of related objects.
Using this metaclass DRF optimize ModelViewSet MetaClass
from django.utils import six
#six.add_metaclass(OptimizeRelatedModelViewSetMetaclass)
class MyModelViewSet(viewsets.ModelViewSet):
queryset = MyModel.objects.all()
serializer_class = MyModelSerializer
You can split the view into two query.
First, only get the Users list (without is_friend_already field). This only require one query.
Second, get the friends list of request.user.
Third, modify the results depending on if the user is in the request.user's friend list.
class GetAllUsersSerializer(serializers.ModelSerializer):
...
class UserListView(ListView):
def get(self, request):
friends = request.user.friends
data = []
for user in self.get_queryset():
user_data = GetAllUsersSerializer(user).data
if user in friends:
user_data['is_friend_already'] = True
else:
user_data['is_friend_already'] = False
data.append(user_data)
return Response(status=200, data=data)
I'm using django-rest-framework. I have a model with a relation. I would like to just display the count of related items when a user hits the /modelname/ URL, but show the full related set when a user hits a specific model instance at /modelname/1/.
I can almost get what I want.
I have two serializers, like so:
class DataSetSerializer(serializers.ModelSerializer):
revisions = serializers.RelatedField(source='datasetrevision_set', many=True)
class Meta:
model = DataSet
fields = ('id', 'title', 'revisions')
class ShortDataSetSerializer(serializers.ModelSerializer):
class Meta:
model = DataSet
fields = ('id', 'title', 'revisions')
If I use the short version, I get the count of revisions (it's a calculated field). If I use the long version, I get the full list of related items as "revisions".
Short:
[{"id": 1, "title": "My Data Set", "revisions": 0}]
Long:
[{"id": 1, "title": "My Data Set", "revisions": ["Data Set v1", "Data Set v2"]}]
What I want to do is be able to switch between them based on query parameters (url). I tried to set the serializer_class to the ShortDataSetSerializer when the ID was not present, but it overrode all cases, not just the non-ID case.
class DataSetViewSet(viewsets.ModelViewSet):
serializer_class = DataSetSerializer
model = DataSet
def get_queryset(self):
try:
id = self.kwargs['id']
queryset = DataSet.objects.filter(id=id)
except KeyError:
queryset = DataSet.objects.all()
# We want to only list all of the revision data if we're viewing a
# specific set, but this overrides for all cases, not just the one
# we want.
self.serializer_class = ShortDataSetSerializer
return queryset
Is there a way I can make this work? I realize I may be approaching this in a totally ridiculous manner, but it seems like there should be an easy solution.
The data example I gave rather abbreviated compared to the real data I'm working with. The end goal is to show a subset of fields in list view, and every field in the GET for a specific ID. This is a read-only API, so I don't need to worry about POST/PUT/DELETE.
You could do it by overriding the get_serializer_class method:
class DataSetViewSet(viewsets.ModelViewSet):
model = DataSet
def get_queryset(self):
queryset = DataSet.objects.all()
if self.kwargs.get('id'):
queryset = queryset.filter(pk=self.kwargs.get('id'))
return queryset
def get_serializer_class(self):
return DataSetSerializer if 'id' in self.kwargs else ShortDataSetSerializer
I think one easy solution for this problem would be to use class based generic views instead of a viewset.
You can use a list create api view with serializer_class as ShortDataSetSerializer. So when you get the list of data it will have the count of revisions. Also if you want the post request to work on the same url you will then have to override the get_serializer_class method to set the serializer_class based on request type.
For the retrieve view you can use the serializer_class as DataSetSerializer. It will have a list of revisions instead of count.
Checkout generic views api guide on DRF docs website.
Also, you can override the list and retrieve methods on the viewset, but I would prefer to use class based views since DRF has a lot of additional functionalities attached to the request functions like get, put etc.(or list, detail) and it is better not to override them.
Thank you, Benjamin. That didn't do what quite I was looking for. Ultimately what I had to do was this (with the same serializers as above):
class DataSetViewSet(viewsets.ModelViewSet):
model = DataSet
def list(self, request):
queryset = DataSet.objects.all()
serializer = ShortDataSetSerializer(queryset, many=True)
return Response(serializer.data)
def detail(self, request, id=None):
queryset = DataSet.objects.get(id=id)
serializer = DataSetSerializer(queryset)
return Response(serializer.data)
And in the urls.py:
url(r'^sets/$', views.DataSetViewSet.as_view({'get': 'list'})),
url(r'^sets/(?P<id>\d+)/$', views.DataSetViewSet.as_view({'get': 'detail'})),