We Keep Coding

Django ManyToMany field filter

So I have this system where my Post object has a ManyToMany field and it's called Saves. So like for example on Reddit you can save a post. So I got it working and users can save posts, and it adds them to the ManyToMany field. However, I want to filter out these posts and only show the posts where said user is in the ManyToMany field.
Here is my models.py
class Post(models.Model):
author = models.ForeignKey(User,related_name='posts',on_delete=models.CASCADE)
saves = models.ManyToManyField(User,blank=True,related_name='post_saves')
I have the saves field connected to the User model Django provides.
And here is my views.py
class PostSaveRedirect(RedirectView):
def get_redirect_url(self,*args,**kwargs):
pk = self.kwargs.get("pk")
slug = self.kwargs.get("slug")
obj = get_object_or_404(Post,pk=pk,slug=slug)
url_ = obj.get_absolute_url()
user = self.request.user
if user.is_authenticated:
if user in obj.saves.all():
obj.saves.remove(user)
else:
obj.saves.add(user)
return url_
So this is all working fine, it adds the user to the ManyToMany field, but now I want to know how I can filter out posts and only display ones where the user is in the ManyToMany field.
Here is my saved posts view.
class PostSaveListView(ListView):
model = Post
template_name = 'mainapp/post_saved.html'
paginate_by = 10
queryset = models.Post.objects.all()
def get(self,request):
posts = Post.objects.all()
return render(request, self.template_name)
def get_queryset(self):
return Post.objects.filter().order_by('-published_date')
So with Post.objects.all(), how can I change it so it will filter to my needs? This is a similar queryset for a user post list view I have
I have been Googling and reading up the docs and other articles but have not found anything that has been able to show me how to filter a ManyToMany field. Any help would be much appreciated

edit your model like this:
class PostSaveListView(ListView):
model = Post
template_name = 'mainapp/post_saved.html'
paginate_by = 10
-
def get(self,request):
posts = Post.objects.all()
return render(request, self.template_name)
def get_queryset(self):
object_list = Post.objects.filter(saves__in=[self.request.user]).order_by('-published_date').distinct()
return object_list

The easiest and quickest option for you would be to use the filtering option in like this:
def get_queryset(self):
return Post.objects.filter(saves__in=[self.request.user]).order_by('-published_date')
Please notice the list inclusion for the user object, as that option only filters from lists.
You may consider also adding .distinct() call to the filter also to avoid repetition of objects.

Django Rest Framework: advanced queryset filtering based on another model

I am using Django Rest Framework as a backend for an app.
I have a User that has one Wallet. Then I have Item. If a User wants an Item it creates an instance in his/her Wallet called WalletItem. All works well.
Now I want to limit the number of items for the User using an attribute limit_usage.
First, I added a check to post method adding new instance that checks the number of item instances in User's Wallet. So the user gets 403 when trying to add third WalletItem if limit_usage == 2 for this Item.
I would like to override a get_queryset() method or queryset in list()/retrieve() methods so that if anonymous user calls /items/ there are unfiltered items in response. However if the user is authenticated I would like to filter only those Items that s/he is allowed to put in the Wallet, i.e. those that have not exceeded limit_usage for current user.
class Wallet(models.Model):
user = models.OneToOneField('auth.User', related_name='wallet')
class Item(models.Model):
valid_from = models.DateTimeField()
valid_to = models.DateTimeField()
limit_usage = models.PositiveSmallIntegerField(default=0)
class WalletItem(models.Model):
wallet = models.ForeignKey('Wallet', related_name='%(class)ss')
offer = models.ForeignKey('Item', related_name='offer')
class ItemViewSet(viewsets.ReadOnlyModelViewSet):
queryset = Item.objects.all().order_by('-created_at')
serializer_class = ItemSerializer
def list(self, request, *args, **kwargs):
time_now = now()
self.queryset = self.queryset.filter(
valid_from__lte=time_now,
valid_to__gte=time_now,
)
serializer = self.get_serializer(self.queryset, many=True)
return Response(serializer.data)
I created a method of Item class that should have helped me but I realized I cannot use it in the queryset:
def is_visible_for_user(self, user=None):
if not self.limit_usage or not user:
return True
ct = WalletItem.objects.filter(item=self, wallet=user.wallet).count()
return self.limit_usage > ct
So I can iterate through the queryset to see if each item can be visible for the user, however I cannot construct the queryset out of this filtered list.
I found something similar here on SO: Django REST Framework : filtering with another table but the response did not help me.

You firstly need to check if user is authenticated, it not, then return every Item. Then to filter out Item objects if corresponding WalletItem objects exceeded its limit.
from django.db.models import Count, F, Sum
...
class ItemViewSet(viewsets.ReadOnlyModelViewSet):
def get_queryset(self):
queryset = super().get_queryset()
user = self.request.user
if user.is_anonymous:
return queryset
queryset = queryset.annotate(user_wallet_items=Sum(
Case(
When(walletitem__wallet_id=user.wallet_id, then=1),
default=0, output_field=IntegerField()
)) \
.filter(user_wallet_items__lte=F('limit_usage'))
return queryset
I suggest you to move your filtration based on current time to the same get_queryset() method, since it's belong there.
Note: i've not tested this approach.

Django Admin, sort with custom function

How can I sort a column in Django admin by some simple custom method?
(All the answers I got through was by using annotate but I don't know how to use it my case).
Assume the model
class Shots(models.Model):
hits = models.PositiveIntegerField()
all = models.PositiveIntegerField()
In admin site I would like to sort by hits to all ratio:
class ShotAdmin(admin.ModelAdmin):
list_display = ['ratio']
def ratio(self, obj):
return obj.hits / obj.all * 100
I know the ratio is not a field in DB thus simple ratio.admin_order_field = 'ratio' won't work and I need to somehow append this as a field but I have no idea how.

By following:
The accepted answer to this post:
Django admin: how to sort by one of the custom list_display fields that has no database field
The
How to execute arithmetic operations between Model fields in django
(Disclaimer: I have composed that Q&A style example)
We can compose a solution to your problem:
from django.db.models import F
class ShotsAdmin(admin.ModelAdmin):
list_display = ('get_ratio',)
def get_queryset(self, request):
qs = super(ShotsAdmin, self).get_queryset(request)
qs = qs.annotate(ratio=F('hits') * 100 / F('all'))
return qs
def get_ratio(self, obj):
return obj.ratio
get_ratio.admin_order_field = 'ratio'
Explanation:
The get_queryset method will annotate a new field named ratio to
your queryset. That field's value is the application of your ratio function on the hits and all fields.
The get_ratio function returns the aforementioned field from a queryset instance.
Finally: get_ratio.admin_order_field = 'ratio' sets the ratio field as the ordering field for your queryset on the admin panel.

Optimizing database queries in Django REST framework

I have the following models:
class User(models.Model):
name = models.Charfield()
email = models.EmailField()
class Friendship(models.Model):
from_friend = models.ForeignKey(User)
to_friend = models.ForeignKey(User)
And those models are used in the following view and serializer:
class GetAllUsers(generics.ListAPIView):
authentication_classes = (SessionAuthentication, TokenAuthentication)
permission_classes = (permissions.IsAuthenticated,)
serializer_class = GetAllUsersSerializer
model = User
def get_queryset(self):
return User.objects.all()
class GetAllUsersSerializer(serializers.ModelSerializer):
is_friend_already = serializers.SerializerMethodField('get_is_friend_already')
class Meta:
model = User
fields = ('id', 'name', 'email', 'is_friend_already',)
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
if request.user != obj and Friendship.objects.filter(from_friend = user):
return True
else:
return False
So basically, for each user returned by the GetAllUsers view, I want to print out whether the user is a friend with the requester (actually I should check both from_ and to_friend, but does not matter for the question in point)
What I see is that for N users in database, there is 1 query for getting all N users, and then 1xN queries in the serializer's get_is_friend_already
Is there a way to avoid this in the rest-framework way? Maybe something like passing a select_related included query to the serializer that has the relevant Friendship rows?

Django REST Framework cannot automatically optimize queries for you, in the same way that Django itself won't. There are places you can look at for tips, including the Django documentation. It has been mentioned that Django REST Framework should automatically, though there are some challenges associated with that.
This question is very specific to your case, where you are using a custom SerializerMethodField that makes a request for each object that is returned. Because you are making a new request (using the Friends.objects manager), it is very difficult to optimize the query.
You can make the problem better though, by not creating a new queryset and instead getting the friend count from other places. This will require a backwards relation to be created on the Friendship model, most likely through the related_name parameter on the field, so you can prefetch all of the Friendship objects. But this is only useful if you need the full objects, and not just a count of the objects.
This would result in a view and serializer similar to the following:
class Friendship(models.Model):
from_friend = models.ForeignKey(User, related_name="friends")
to_friend = models.ForeignKey(User)
class GetAllUsers(generics.ListAPIView):
...
def get_queryset(self):
return User.objects.all().prefetch_related("friends")
class GetAllUsersSerializer(serializers.ModelSerializer):
...
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
friends = set(friend.from_friend_id for friend in obj.friends)
if request.user != obj and request.user.id in friends:
return True
else:
return False
If you just need a count of the objects (similar to using queryset.count() or queryset.exists()), you can include annotate the rows in the queryset with the counts of reverse relationships. This would be done in your get_queryset method, by adding .annotate(friends_count=Count("friends")) to the end (if the related_name was friends), which will set the friends_count attribute on each object to the number of friends.
This would result in a view and serializer similar to the following:
class Friendship(models.Model):
from_friend = models.ForeignKey(User, related_name="friends")
to_friend = models.ForeignKey(User)
class GetAllUsers(generics.ListAPIView):
...
def get_queryset(self):
from django.db.models import Count
return User.objects.all().annotate(friends_count=Count("friends"))
class GetAllUsersSerializer(serializers.ModelSerializer):
...
def get_is_friend_already(self, obj):
request = self.context.get('request', None)
if request.user != obj and obj.friends_count > 0:
return True
else:
return False
Both of these solutions will avoid N+1 queries, but the one you pick depends on what you are trying to achieve.

Described N+1 problem is a number one issue during Django REST Framework performance optimization, so from various opinions, it requires more solid approach then direct prefetch_related() or select_related() in get_queryset() view method.
Based on collected information, here's a robust solution that eliminates N+1 (using OP's code as an example). It's based on decorators and slightly less coupled for larger applications.
Serializer:
class GetAllUsersSerializer(serializers.ModelSerializer):
friends = FriendSerializer(read_only=True, many=True)
# ...
#staticmethod
def setup_eager_loading(queryset):
queryset = queryset.prefetch_related("friends")
return queryset
Here we use static class method to build the specific queryset.
Decorator:
def setup_eager_loading(get_queryset):
def decorator(self):
queryset = get_queryset(self)
queryset = self.get_serializer_class().setup_eager_loading(queryset)
return queryset
return decorator
This function modifies returned queryset in order to fetch related records for a model as defined in setup_eager_loading serializer method.
View:
class GetAllUsers(generics.ListAPIView):
serializer_class = GetAllUsersSerializer
#setup_eager_loading
def get_queryset(self):
return User.objects.all()
This pattern may look like an overkill, but it's certainly more DRY and has advantage over direct queryset modification inside views, as it allows more control over related entities and eliminates unnecessary nesting of related objects.

Using this metaclass DRF optimize ModelViewSet MetaClass
from django.utils import six
#six.add_metaclass(OptimizeRelatedModelViewSetMetaclass)
class MyModelViewSet(viewsets.ModelViewSet):
queryset = MyModel.objects.all()
serializer_class = MyModelSerializer

You can split the view into two query.
First, only get the Users list (without is_friend_already field). This only require one query.
Second, get the friends list of request.user.
Third, modify the results depending on if the user is in the request.user's friend list.
class GetAllUsersSerializer(serializers.ModelSerializer):
...
class UserListView(ListView):
def get(self, request):
friends = request.user.friends
data = []
for user in self.get_queryset():
user_data = GetAllUsersSerializer(user).data
if user in friends:
user_data['is_friend_already'] = True
else:
user_data['is_friend_already'] = False
data.append(user_data)
return Response(status=200, data=data)

Override queryset filters while generating django admin URLs

My models.py looks like this:
class Person(models.Model):
Name = models.CharField(max_length=100)
class Lecture(models.Model):
Speaker = model.ForeignKey(Person)
Topic = models.CharField(max_length=100)
Choices = ((1,"Upcoming"),(2,"In Progress",),(3,"Completed"))
Status = models.SmallIntegerField(choices=Choices, default=1, max_length=1)
My admin.py looks like this:
class LectureAdmin(admin.ModelAdmin):
def get_queryset(self):
return Lecture.objects.exclude(Status='Completed')
So my change list view in the django admin for the Lecture model shows only Lectures in "Upcoming" and "In Progress" status. This works fine.
Now I need to get the URL for the list of all lectures to be passed as a view somewhere else.The standard way of doing this in the django admin is by reversing the URL, so I do this:
urlresolvers.reverse('admin:%s_%s_changelist' % (app_label, model_name))
However, when I do this,I get the the filtered Queryset with Lectures in "Completed" state missing.How do I construct a url reverse function to get entire Lecture queryset and not the filtered queryset?

Here's a workaround, looks ugly, I understand.
Add all GET parameter to the changelist url:
url = urlresolvers.reverse('admin:%s_%s_changelist' % (app_label, model_name))
url += '?all'
Call get_queryset() on super(), exclude Completed status only if there is no all in request.GET:
class LectureAdmin(admin.ModelAdmin):
def get_queryset(self, request):
qs = super(LectureAdmin, self).get_queryset(request)
if 'all' not in request.GET:
qs = qs.exclude(Status='Completed')
return qs
UPD (applying other filters from request.GET):
from xadmin.plugin.related import RELATE_PREFIX # or just set RELATE_PREFIX = '_rel_'
qs = qs.filter(**{key[len(RELATE_PREFIX):]: value
for key, value in request.GET.iteritems()
if key.startswith(RELATE_PREFIX)})
** unpacks the dictionary into keyword arguments.
Hope it works for you.

get_queryset() is the basic queryset used in admin listing, thus you wo'nt be able to get all the records if you override it this way.
Possible solutions:
use filters ( https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.filter_vertical ) to exclude unwanted records (these with Status='Completed'
or
create proxy model for Lecture, register it in admin and use modified get_queryset() in given listing. Proxy model is required because each model can have registered only single AdminModel class
models.py
class IncompletedLecture(Lecture):
class Meta:
proxy = True
admin.py
class IncompletedAdmin(admin.ModelAdmin):
def get_queryset():
return Lecture.query.exclude(Status='Completed')
admin.site.register(IncompletedLecture, IncompletedAdmin)