We Keep Coding

Serializing model queryset showing empty [OrderedDict()]

I am building a blog app with React and Django and I am serializing model's instances saved by particular user, First I am just trying to test with .all() then I am planning to filter by specific user But when I serialize queryset with Serializer like:
class BlogSerializerApiView(viewsets.ModelViewSet):
serializer_class = BlogSerializer
def get_queryset(self, *args, **kwargs):
queryset = Blog.objects.all()
output_serializer = BlogSerializer(queryset, many=True)
print(output_serializer.data)
return "Testing"
It is showing in console:
[OrderedDict(), OrderedDict()]
and when I access it like
print(output_serializer)
Then it is showing:
BlogSerializer(<QuerySet [<Blog: user_1 - Blog_title>, <Blog: user_2 - second_blog_title>]>, many=True):
serializer.py:
class BlogSerializer(serializers.Serializer):
class Meta:
model = Blog
fields = ['title']
models.py:
class Blog(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
title = models.CharField(max_length=30, default='')
def __str__(self):
return f"{self.user} - {self.title}"
What I am trying to do:
I am trying to serialize queryset to show on page in react frontend, I will relate with specific user later.
I have tried many times by changing CBV serialization method by generics.ListAPIView instead of viewsets.ModelViewSet but still same thing.

There is a concept error here. The get_queryset function is not supposed to return serialized data. It must return a QuerySet of model objects.
To achieve what you want you can just do:
class BlogSerializerApiView(viewsets.ModelViewSet):
serializer_class = BlogSerializer
def get_queryset(self, *args, **kwargs):
return Blog.objects.all()
The Django Rest Framework will take care of serializing data.
In fact, you can even do it way more simple. Defining the view's queryset field like this:
class BlogSerializerApiView(viewsets.ModelViewSet):
queryset = Blog.objects.all()
serializer_class = BlogSerializer
Additional:
You said you will relate to current user later. You could achieve that in fact in the get_queryset method filtering aginst the user
class BlogSerializerApiView(viewsets.ModelViewSet):
serializer_class = BlogSerializer
def get_queryset(self, *args, **kwargs):
return Blog.objects.filter(user_id=USER_ID)
Hope this helps!

I was using
class BlogSerializer(serializers.Serializer):
.......
so it was showing empty results (no idea why, I think its deprecated)
After replaceing it with
class BlogSerializer(serializers.HyperlinkedModelSerializer):
It worked

Django Rest Framework with shortuuid, generics.RetrieveUpdateDestroyAPIView returning 404 {"detail": "Not found."}

I was remaking a social media site as a revision of Django and the rest framework, I didn't want to use the django default linear id count and didn't like how long the uuid library's ids was, so I used the shortuuid library. I've used them on the posts and the comments just to keep the anonymity of the count of both posts and comments. On the posts side everything works for the CRUD stuff (which should be proof that the issue isn't from the shortuuid library, as far as I know), although with the comments the Create Retrieve works perfectly but the Update Destroy doesn't. so here is the code we are working with:
starting with the models to know what kind of data we are working with (models.py):
from shortuuid.django_fields import ShortUUIDField
... # posts likes etc
class Comment(models.Model):
id = ShortUUIDField(primary_key=True, length=8, max_length=10)
user = models.ForeignKey(User, on_delete=models.CASCADE)
post = models.ForeignKey(Post, on_delete=models.CASCADE)
body = models.TextField(max_length=350)
created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
active = models.BooleanField(default=True)
class Meta:
ordering = ['created']
def __str__(self):
return f'on {self.post} by {self.user}'
objects = models.Manager()
serializers.py:
class CommentSerializer(ModelSerializer):
username = SerializerMethodField()
def get_username(self, comment):
return str(comment.user)
class Meta:
model = Comment
fields = ['id', 'user', 'post', 'username', 'body', 'created', 'updated']
read_only_fields = ['id', 'post', 'user', 'username']
now with the routing (urls.py):
from django.urls import path
from .views import *
urlpatterns = [
...
path('<str:pk>/comments/' , Comments),
path('<str:pk>/comments/create/', CreateComment),
path('<str:pk>/comments/<str:cm>/', ModifyComment),
# pk = post ID
# cm = comment ID
]
views.py:
class ModifyComment(generics.RetrieveUpdateDestroyAPIView):
serializer_class = CommentSerializer
permission_classes = [permissions.AllowAny]
def get_queryset(self):
post = Post.objects.get(pk=self.kwargs['pk'])
comment = Comment.objects.get(post=post, pk=self.kwargs['cm'])
return comment
def perform_update(self, serializer):
print(Post.objects.all())
post = Post.objects.get(pk=self.kwargs['pk'])
comment = Comment.objects.filter(pk=self.kwargs['cm'], post=post)
if self.request.user != comment.user:
raise ValidationError('you can\'t edit another user\'s post')
if comment.exists():
serializer.save(user=self.request.user, comment=comment)
else:
raise ValidationError('the comment doesnt exist lol')
def delete(self, request, *args, **kwargs):
comment = Comment.objects.filter(user=self.request.user, pk=self.kwargs['cm'])
if comment.exists():
return self.destroy(request, *args, **kwargs)
else:
raise ValidationError("you can\'t delete another user\'s post")
ModifyComment = ModifyComment.as_view()
and the response to going to the url '<str:pk>/comments/<str:cm>/' comment of some post we get this:
side note, the perform_update function doesn't seem to be called ever, even putting a print statement at the beginning of the function doesn't get printed so the issue may have to do with the get_queryset even though I've tried using the normal queryset=Comment.object.all() and making the get_queryset function return the comment with the correct params but I couldn't make it work

For individual objects you need to overwrite the get_object method.
You are performing the request GET /str:pk/comments/str:cm/, this calls the retrieve method on the view, which in turn calls get_object. The default behaviour is trying to find a Comment object with id equal to pk since it's the first argument, since you need to filter through a different model you need to overwrite it.
classy drf is a good website for seing how the internals of the clases work.

How to change permissions depend on different instance using Django Rest Framework?

I'm using Django rest framework, and my model is like this, Every Act can have more than one post.
class Act(models.Model):
user = models.ForeignKey("common.MyUser", related_name="act_user")
act_title = models.CharField(max_length=30)
act_content = models.CharField(max_length=1000)
act_type = models.IntField()
class Post(models.Model):
user = models.ForeignKey("common.MyUser", related_name="post_user")
act = models.ForeignKey("activities.Act", related_name="post_act")
post_title = models.CharField(max_length=30, blank=True)
post_content = models.CharField(max_length=140)
my view.py in DRF:
class PostList(generics.ListCreateAPIView):
permission_classes = (permissions.IsAuthenticatedOrReadOnly,)
queryset = Post.objects.all()
serializer_class = PostAllSerializer
def perform_create(self, serializer): #self is a instance of class or is a class here?
serializer.save(user=self.request.user)
This works fine, but what I want now is if act_type = 1 means this is a private Act and only the act author can create post under this act.I wonder how to use different permission_classes depend on different Act.Maybe looks like:
class PostList(generics.ListCreateAPIView):
if self.act_type == 1:
permission_classes = (permissions.IsAuthenticatedOrReadOnly,IsActCreatorOrReadOnly)
else
permission_classes = (permissions.IsAuthenticatedOrReadOnly,)
queryset = Post.objects.all()
serializer_class = PostAllSerializer
def perform_create(self, serializer): #self is a instance of class or is a class here?
serializer.save(user=self.request.user)
And I also want to know how to write this permissions.py:
class IsActCreatorOrReadOnly(permissions.BasePermission):
"""
Custom permission to only allow owners of an object to edit it.
"""
def has_object_permission(self, request, view, obj):
if request.method in permissions.SAFE_METHODS:
return True
return obj.act.user == request.user
I don't know what obj really means here, and the error tell me obj.act doesn't exist.
EDIT
Here is my postSerializer.
class PostAllSerializer(serializers.ModelSerializer):
"""Posts api fields"""
post_user = UserSerializer(source="user", read_only=True)
post_author = serializers.ReadOnlyField(source='user.user_name')
class Meta:
model = Post
fields = ("id", "act", "post_author", "post_title", "post_content",)
I tried this, but not working, I still can create the post even I'm not the author of the Act(but the act_id is wrong):
def create(self, request, *args, **kwargs):
act_type = request.data.get("act_type")
if act_type == 0:
act_id = request.data.get("act")
act = Act.objects.get(pk=act_id)
if request.user != act.user:
return Response(status=403)
return super().create(request)

For using different permission classes, there is the get_permissions method that you can overwrite on your PostList view:
def get_permissions(self):
if self.request.method == 'POST':
return (OnePermission(),)
elif # other condition if you have:
return (AnotherPermission(),)
return (YetAnotherPermission(),)
However, in your case you can't use object level permissions, because you don't have an object instance yet. From the DRF docs (highlights by me):
REST framework permissions also support object-level permissioning. Object level permissions are used to determine if a user should be allowed to act on a particular object, which will typically be a model instance.
Object level permissions are run by REST framework's generic views when .get_object() is called.
When doing a POST request, you don't have any object yet, thus the object level permissions won't be invoked.
One way you could achieve what you want is by checking it in the create method of PostList view. Something like this (hypothetical code):
class PostList(generics.ListCreateAPIView):
...
def create(self, request):
act_id = request.data.get('act') # depending on your PostSerializer, the logic of getting act id can vary a little
act = Act.objects.get(pk=act_id) # assuming act always exists, otherwise account for in-existing act
if act.user != request.user:
return Response({details: "You shall not pass!!!", status=200) # change to a status and message you need here
# logic of Post creation here
Good luck!

Optimizing database queries in Django REST framework

I have the following models:
class User(models.Model):
name = models.Charfield()
email = models.EmailField()
class Friendship(models.Model):
from_friend = models.ForeignKey(User)
to_friend = models.ForeignKey(User)
And those models are used in the following view and serializer:
class GetAllUsers(generics.ListAPIView):
authentication_classes = (SessionAuthentication, TokenAuthentication)
permission_classes = (permissions.IsAuthenticated,)
serializer_class = GetAllUsersSerializer
model = User
def get_queryset(self):
return User.objects.all()
class GetAllUsersSerializer(serializers.ModelSerializer):
is_friend_already = serializers.SerializerMethodField('get_is_friend_already')
class Meta:
model = User
fields = ('id', 'name', 'email', 'is_friend_already',)
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
if request.user != obj and Friendship.objects.filter(from_friend = user):
return True
else:
return False
So basically, for each user returned by the GetAllUsers view, I want to print out whether the user is a friend with the requester (actually I should check both from_ and to_friend, but does not matter for the question in point)
What I see is that for N users in database, there is 1 query for getting all N users, and then 1xN queries in the serializer's get_is_friend_already
Is there a way to avoid this in the rest-framework way? Maybe something like passing a select_related included query to the serializer that has the relevant Friendship rows?

Django REST Framework cannot automatically optimize queries for you, in the same way that Django itself won't. There are places you can look at for tips, including the Django documentation. It has been mentioned that Django REST Framework should automatically, though there are some challenges associated with that.
This question is very specific to your case, where you are using a custom SerializerMethodField that makes a request for each object that is returned. Because you are making a new request (using the Friends.objects manager), it is very difficult to optimize the query.
You can make the problem better though, by not creating a new queryset and instead getting the friend count from other places. This will require a backwards relation to be created on the Friendship model, most likely through the related_name parameter on the field, so you can prefetch all of the Friendship objects. But this is only useful if you need the full objects, and not just a count of the objects.
This would result in a view and serializer similar to the following:
class Friendship(models.Model):
from_friend = models.ForeignKey(User, related_name="friends")
to_friend = models.ForeignKey(User)
class GetAllUsers(generics.ListAPIView):
...
def get_queryset(self):
return User.objects.all().prefetch_related("friends")
class GetAllUsersSerializer(serializers.ModelSerializer):
...
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
friends = set(friend.from_friend_id for friend in obj.friends)
if request.user != obj and request.user.id in friends:
return True
else:
return False
If you just need a count of the objects (similar to using queryset.count() or queryset.exists()), you can include annotate the rows in the queryset with the counts of reverse relationships. This would be done in your get_queryset method, by adding .annotate(friends_count=Count("friends")) to the end (if the related_name was friends), which will set the friends_count attribute on each object to the number of friends.
This would result in a view and serializer similar to the following:
class Friendship(models.Model):
from_friend = models.ForeignKey(User, related_name="friends")
to_friend = models.ForeignKey(User)
class GetAllUsers(generics.ListAPIView):
...
def get_queryset(self):
from django.db.models import Count
return User.objects.all().annotate(friends_count=Count("friends"))
class GetAllUsersSerializer(serializers.ModelSerializer):
...
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
if request.user != obj and obj.friends_count > 0:
return True
else:
return False
Both of these solutions will avoid N+1 queries, but the one you pick depends on what you are trying to achieve.

Described N+1 problem is a number one issue during Django REST Framework performance optimization, so from various opinions, it requires more solid approach then direct prefetch_related() or select_related() in get_queryset() view method.
Based on collected information, here's a robust solution that eliminates N+1 (using OP's code as an example). It's based on decorators and slightly less coupled for larger applications.
Serializer:
class GetAllUsersSerializer(serializers.ModelSerializer):
friends = FriendSerializer(read_only=True, many=True)
# ...
#staticmethod
def setup_eager_loading(queryset):
queryset = queryset.prefetch_related("friends")
return queryset
Here we use static class method to build the specific queryset.
Decorator:
def setup_eager_loading(get_queryset):
def decorator(self):
queryset = get_queryset(self)
queryset = self.get_serializer_class().setup_eager_loading(queryset)
return queryset
return decorator
This function modifies returned queryset in order to fetch related records for a model as defined in setup_eager_loading serializer method.
View:
class GetAllUsers(generics.ListAPIView):
serializer_class = GetAllUsersSerializer
#setup_eager_loading
def get_queryset(self):
return User.objects.all()
This pattern may look like an overkill, but it's certainly more DRY and has advantage over direct queryset modification inside views, as it allows more control over related entities and eliminates unnecessary nesting of related objects.

Using this metaclass DRF optimize ModelViewSet MetaClass
from django.utils import six
#six.add_metaclass(OptimizeRelatedModelViewSetMetaclass)
class MyModelViewSet(viewsets.ModelViewSet):
queryset = MyModel.objects.all()
serializer_class = MyModelSerializer

You can split the view into two query.
First, only get the Users list (without is_friend_already field). This only require one query.
Second, get the friends list of request.user.
Third, modify the results depending on if the user is in the request.user's friend list.
class GetAllUsersSerializer(serializers.ModelSerializer):
...
class UserListView(ListView):
def get(self, request):
friends = request.user.friends
data = []
for user in self.get_queryset():
user_data = GetAllUsersSerializer(user).data
if user in friends:
user_data['is_friend_already'] = True
else:
user_data['is_friend_already'] = False
data.append(user_data)
return Response(status=200, data=data)

Django rest framework populate fields of nested objects programmatically

When fields need to be filled programmatically in Django Rest Framework, the pre_save method may be overridden in the APIView, and the needed fields can be populated there, like:
def pre_save(self, obj):
obj.owner = self.request.user
This works great for flat objects, but in case of nested situations, the nested object cannot be accessed in the pre_save method. The only solution I found so far is to override the save_object method, and check if the object is an instance of the nested class, and if so, populate that field there. Although this works, I don't like the solution, and would like to know if anyone found a better way?
Demonstrating the situation:
class Notebook(models.Model):
owner = models.ForeignKey(User)
class Note(models.Model):
owner = models.ForeignKey(User)
notebook = models.ForeignKey(Notebook)
note = models.TextField()
class NoteSerializer(serializers.ModelSerializer):
owner = serializers.Field(source='owner.username')
class Meta:
model = Note
fields = ('note', 'owner')
class NotebookSerializer(serializers.ModelSerializer):
notes = NoteSerializer(many=True)
owner = serializers.Field(source='owner.username')
class Meta:
model = Notebook
fields = ('notes', 'owner')
def save_object(self, obj, **kwargs):
if isinstance(obj, Note):
obj.owner = obj.notebook.owner
return super(NotebookSerializer, self).save_object(obj, **kwargs)
class NotebookCreateAPIView(CreateAPIView):
model = Notebook
permission_classes = (IsAuthenticated,)
serializer_class = NotebookSerializer
def pre_save(self, obj):
obj.owner = self.request.user
Before asking why don't I use different endpoints for creating notebooks and notes separately, let me say that I do that, but I also need a functionality to provide initial notes on creation of the notebook, so that's why I need this kind of endpoint as well.
Also, before I figured out this hackish solution, I actually expected that I will have to override the save_object method of the NoteSerializer class itself, but it turned out in case of nested objects, it won't even be called, only the root object's save_objects method, for all the nested objects, but I guess it was a design decision.
So once again, is this solvable in a more idiomatic way?

You can access the request in your serializer context.
So my approach to this would be:
class NoteSerializer(serializers.ModelSerializer):
owner = serializers.Field(source='owner.username')
def restore_object(self, attrs, instance=None):
instance = super(NoteSerializer, self).restore_object(attrs, instance)
instance.owner = self.context['request'].user
return instance
class Meta:
model = Note
fields = ('note', 'owner')
And the same on the NotebookSerializer.
The Serializer context will be made available to all used serializers in the ViewSet.