We Keep Coding

DRF: Filter several fields in one request

I have to implement set of filters (see picture). My code works fine for 1 filter e.g.
http://127.0.0.1:8000/api/constructions?field=developer&value=1 -- filter by developer with id =1
I want to filter by several filters in one request.
I can use something like this
http://127.0.0.1:8000/api/constructions?field=field1_field2&value=value1_value2
Split field1_field2 --> [field1, field2] and so on (it's not perfect)
Is there better way to solve my issue?
views.py
class ConstructionView(viewsets.ModelViewSet):
serializer_class = ConstructionSerializer
queryset = Construction.objects.all()
pagination_class = BasePagination
def list(self, request):
field = request.GET.get('field', None)
value = request.GET.get('value', None)
if field is not None and value is not None:
queryset = Construction.objects.filter(**{field:value})
else:
queryset = Construction.objects.all()
page = self.paginate_queryset(queryset)
if page is not None:
serializer = ConstructionSerializer(page, many=True)
return self.get_paginated_response(serializer.data)
else:
serializer = ConstructionSerializer(queryset, many=True)
return Response(serializer.data, status=status.HTTP_200_OK)

If you have a list at your query_params you can use that approach. You can check if your value of your query_params is a list or string and apply it to a filter. <fieldname>__in works for list.
custom_filter = {'field__in': request.query_params.get('field'), 'value__in': request.query_params.get('value')}
queryset = Construction.objects.filter(**custom_filter )
Maybe djangorestframework-queryfields helps your for a bunch of common work.

If you want to get multiple object by their ID, you can try something like this:
Construction.objects.filter(id__in = [1,2])
It will return objects with 1,2 IDs

fetch ids with commas in django

view.py
def get(self, *args, **kwargs):
res = Question.objects.all()
# queryset = super().ListAPIView(*args, **kwargs)
if 'tag' in self.request.GET:
# split and directly put in main query
res = res.filter(
Tag_name=self.request.GET['tag']
)
if 'order_by' in self.request.GET:
res = res.order_by(
self.request.GET['order_by'])
serializer = QuestionSerializers(res, many=True)
return Response(serializer.data)
here I am trying to fetch question with some input tag ids and order by some input , how can I use multiple tag ids in url like
http://127.0.0.1:8000/?tag=1,2&order_by=name
so I get all objects with tag ids 1 and 2.

You can pass multiple get parameters with the same name:
http://127.0.0.1:8000/?tag=1&tag=2&order_by=name
In your view you can access the list using the getlist method:
def get(self, *args, **kwargs):
# returns empty list if no tags are provided
tags = self.request.GET.getlist('tag')
# you can set a default field to order_by, if no field is provided
order_by = self.request.GET.get('order_by', 'id')
# convert tags, currently strings, to int
tags = [int(tag) for tag in tags]
# not sure about this part would need to check your models
res = Question.objects.filter(tag__id__in=tags).order_by(order_by)
serializer = QuestionSerializers(res, many=True)
return Response(serializer.data)

You need to split values of tag by , and filter all tags with id__in filter like this
def get(self, *args, **kwargs):
res = Question.objects.all()
# queryset = super().ListAPIView(*args, **kwargs)
if self.request.GET.get('tag'):
# split and directly put in main query
res = res.filter(
tag_id__in=self.request.GET.get('tag').split(',')
)
if self.request.GET.get('order_by'):
res = res.order_by(
self.request.GET['order_by'])
serializer = QuestionSerializers(res, many=True)
return Response(serializer.data)

Well you can have a workaround(or better to say more django rest framework oriented way to achieve this) using ListAPIView of DRF like:
from rest_framework.generics import ListAPIView
class TagListAPIView(ListAPIView):
serializer_class = QuestionSerializers
filter_backends = (OrderingFilter)
ordering_fields = ['name', ...any other fields you wanna put here as an option for ordering]
ordering = 'name' #setting the default ordering
def get_queryset(self):
ids = self.request.query_params.get('tag', None)
if not ids:
return Question.objects.none() #Empty queryset
return Question.objects.filter(id__in=ids.split(','))
Haven't tested the code but yeah it should be somewhere around this and this code does not do all the proper input handling if some garbage input is coming from user(which is the case manytimes).
You could write more strong code for this by incorporating django-filters and putting a id field in that filter. Which would give you all the error validation and handling.
I have provided a links to DRF's doc regarding filter, ordering and all. https://www.django-rest-framework.org/api-guide/filtering/#orderingfilter
Have a look, its great content.
Ping me if anything unclear.

How to incorporate data from two distinct sources (that don't have a RDBMS relationship) in a single serializer?

I'm trying to serialize some objects whose data is stored in 2 databases, linked by common UUIDs. The second database DB2 stores personal data, so it is run as a segregated microservice to comply with various privacy laws. I receive the data as a decoded list of dicts (rather than an actual queryset of model instances). How can I adapt the ModelSerializer to serialize this data?
Here's a minimal example of interacting with DB2 to get the personal data:
# returns a list of dict objects, approx representing PersonalData.__dict__
# `custom_filter` is a wrapper for the Microservice API using `requests`
personal_data = Microservice.objects.custom_filter(uuid__in=uuids)
And here's a minimal way of serializing it, including the date of birth:
class PersonalDataSerializer(serializers.Serializer):
uuid = serializers.UUIDField() # common UUID in DB1 and DB2
dob = serializers.DateField() # personal, so can't be stored in DB1
In my application, I need to serialize the Person queryset, and related personal_data, into one JSON array.
class PersonSerializer(serializers.ModelSerializer):
dob = serializers.SerializerMethodField()
# can't use RelatedField for `dob` because the relationship isn't
# codified in the RDBMS, due to it being a separate Microservice.
class Meta:
model = Person
# A Person object has `uuid` and `date_joined` fields.
# The `dob` comes from the personal_data, fetched from the Microservice
fields = ('uuid', 'date_joined', 'dob',)
def get_dob(self):
raise NotImplementedError # for the moment
I don't know if there's a nice DRF way to link the two. I definitely don't want to be sending (potentially thousands of) individual requests to the microservice by including a single request in get_dob. The actual view just looks like this:
class PersonList(generics.ListAPIView):
model = Person
serializer_class = PersonSerializer
def get_queryset(self):
self.kwargs.get('some_filter_criteria')
return Person.objects.filter(some_filter_criteria)
Where should the logic go to link the microservice data into the serializer, and what should it look like?

I suggest you to override the serializer and your list method.
Serializer:
class PersonSerializer(models.Serializer):
personal_data = serializers.DictField()
class Meta:
model = Person
make a function to add personal_data dictionary to persons object. Use this method before giving the list of person objects to the serializer.
def prepare_persons(persons):
person_ids = [p.uuid for p in persons]
personal_data_list = Microservice.objects.custom_filter(uuid__in=person_ids)
personal_data_dict = {pd['uuid']: pd for pd in personal_data_list}
for p in persons:
p.personal_data = personal_data_dict[p.id]
return persons
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
page = self.paginate_queryset(queryset)
if page is not None:
page = prepare_persons(page)
serializer = self.get_serializer(page, many=True)
return self.get_paginated_response(serializer.data)
else:
persons = prepare_persons(queryset)
serializer = self.get_serializer(persons, many=True)
return Response(serializer.data)

Because you want to only hit your database one time, a good way to add your extra data to your queryset is by adding a custom version of ListModelMixin to your ViewSet that includes extra context:
class PersonList(generics.ListAPIView):
...
def list(self, request, *args, **kwargs):
queryset = self.filter_queryset(self.get_queryset())
# Pseudo-code for filtering, adjust to work for your use case
filter_criteria = self.kwargs.get('some_filter_criteria')
personal_data = Microservice.objects.custom_filter(filter_criteria)
page = self.paginate_queryset(queryset)
if page is not None:
serializer = self.get_serializer(
page,
many=True,
context={'personal_data': personal_data}
)
return self.get_paginated_response(serializer.data)
serializer = self.get_serializer(
queryset,
many=True,
context={'personal_data': personal_data}
)
return Response(serializer.data)
Then, access the extra context in your serializer by overriding the to_representation method:
def to_representation(self, instance):
"""Add `personal_data` to the object from the Microservice"""
ret = super().to_representation(instance)
personal_data = self.context['personal_data']
ret['personal_data'] = personal_data[instance.uuid]
return ret

Django REST specify which fields when viewing a list

By default, when asking a list of a model like "/cars", django rest outputs all of the model data. I want it to output only pk's on a list request, and full model data on a detail request. I am using ModelSerializer and ModelViewSet.
PS. Is it supposed to be like this by design? Pulling so much unneeded data seems like such a waste.
Thanks :)

You can simply override the 'list' method of viewset to get desired response like this:
from rest_framework.response import Response
def list(self, request, *args, **kwargs):
pks = []
qs = self.get_queryset()
for obj in qs:
pks.append(obj.pk)
return Response(data=pks)

I customized the code to suit my requirement. I created the following 2 mixins. Make your APIListView extend FieldFilterMixin and your serializer extend SerializerFieldsMixin. Pass fl in GET request with comma separated values to get the required fields in response.
class SerializerFieldsMixin(object):
"""
Return only the fields asked for.
Don't return any extra fields in serializer.
"""
def get_fields(self):
all_fields = super(SerializerFieldsMixin,self).get_fields()
asked_fields = self.context.get('asked_fields')
if not asked_fields:
return all_fields
all_fields = OrderedDict([(k,v) for k,v in all_fields.items() if k in asked_fields])
return all_fields
class FieldFilterMixin(object):
"""
To be used with List/Retrieve views.
Set class attribute fields for the fields you want to display.
Or override get_required_fields to customize.
"""
def get_required_fields(self):
if self.request.GET.has_key("fl"):
return self.request.GET["fl"]
return []
def get_serializer_context(self):
methods_to_act_on = ["GET","HEAD"]
context = super(FieldFilterMixin,self).get_serializer_context()
asked_fields = self.get_required_fields()
if asked_fields and self.request.method in methods_to_act_on:
context["asked_fields"] = asked_fields
return context

Add user specific fields to Django REST Framework serializer

I want to add a field to a serializer that contains information specific to the user making the current request (I don't want to create a separate endpoint for this). Here is the way I did it:
The viewset:
class ArticleViewSet(viewsets.ModelViewSet):
queryset = Article.objects.all()
serializer_class = ArticleSerializer
filter_class = ArticleFilterSet
def prefetch_likes(self, ids):
self.current_user_likes = dict([(like.article_id, like.pk) for like in Like.objects.filter(user=self.request.user, article_id__in=ids)])
def get_object(self, queryset=None):
article = super(ArticleViewSet, self).get_object(queryset)
self.prefetch_likes([article.pk])
return article
def paginate_queryset(self, queryset, page_size=None):
page = super(ArticleViewSet, self).paginate_queryset(queryset, page_size)
if page is None:
return None
ids = [article.pk for article in page.object_list]
self.prefetch_likes(ids)
return page
The serializer:
class ArticleSerializer(serializers.ModelSerializer):
class Meta:
model = Article
def to_native(self, obj):
ret = super(ArticleSerializer, self).to_native(obj)
if obj:
view = self.context['view']
ret['has_liked'] = False
if hasattr(view, 'current_user_liked'):
ret['has_liked'] = obj.pk in view.current_user_liked
return ret
Is there a better place to inject the prefetching of liked articles, or a nicer way to do this in general?

you can do it with SerializerMethodField
Example :
class PostSerializer(serializers.ModelSerializer):
fav = serializers.SerializerMethodField('likedByUser')
def likedByUser(self, obj):
request = self.context.get('request', None)
if request is not None:
try:
liked=Favorite.objects.filter(user=request.user, post=obj.id).count()
return liked == 1
except Favorite.DoesNotExist:
return False
return "error"
class Meta:
model = Post
then you should call serializer from view like this:
class PostView(APIVIEW):
def get(self,request):
serializers = PostSerializer(PostObjects,context={'request':request})

I'd be inclined to try and put as much of this as possible on the Like model object and then bung the rest in a custom serializer field.
In serializer fields you can access the request via the context parameter that they inherit from their parent serializer.
So you might do something like this:
class LikedByUserField(Field):
def to_native(self, article):
request = self.context.get('request', None)
return Like.user_likes_article(request.user, article)
The user_likes_article class method could then encapsulate your prefetching (and caching) logic.
I hope that helps.

According to the Django Documentation - SerializerMethodField, I had to change the code of rapid2share slightly.
class ResourceSerializer(serializers.ModelSerializer):
liked_by_user = serializers.SerializerMethodField()
def get_liked_by_user(self, obj : Resource):
request = self.context.get('request')
return request is not None and obj.likes.filter(user=request.user).exists()