We Keep Coding

Save query string data using rest framework in Django

I need to save to a database the query string data coming from a GET request using Django Rest framework.
Something like this:
URL: "http://127.0.0.1:8000/snippets/snippets/?longitude=123123&latitude=456456"
class SnippetList(generics.ListCreateAPIView):
queryset = Snippet.objects.all()
serializer_class = SnippetSerializer
#code here for saving the query string data to database, in this case save:
#{
#"longitude":123123,
#"latitude":456456
#}
It's just like saving the params data like it were a POST request.

I need to save to a database the query string data coming from a GET request using Django Rest framework.
This is against the specifications of the HTTP protocol. Indeed, a GET request should only be used to retrieve data, not update data. For that a POST, PUT, PATCH or DELETE request should be used. The querystring is normally used to filter, not to store data.
If you really want to do this, you can override the .get(…) handler and
# Warning: against the HTTP protocol specifications!
class SnippetCreateView(generics.CreateAPIView):
queryset = Snippet.objects.all()
serializer_class = SnippetSerializer
# {
# "longitude":123123,
# "latitude":456456
# }
def get(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.query_params)
serializer.is_valid(raise_exception=True)
self.perform_create(serializer)
headers = self.get_success_headers(serializer.data)
return Response(
serializer.data, status=status.HTTP_201_CREATED, headers=headers
)
But this is really not a good idea.

Caching must be implemented at view level, in models or in serializers in Django

I have a web application purely based on REST API using Django Rest Framework. I have seen that at most of the places the response for my APIs is not changing or not changing frequently, so I'm thinking to cache such API's and for that, I'm using https://pypi.org/project/redis/ package. So my question here is what could be the better way to implement caching, it should be at the view level, at the model level, or in the serializer. How it could be done?

You can use Cache for APIview and ViewSets with decorators like cache_page.
NOTE: The cache_page decorator only caches the GET and HEAD responses with status 200.
Ex:
from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_page
class UserViewSet(viewsets.ViewSet):
# Cache requested url for each user for 2 hours
#method_decorator(cache_page(60*60*2))
def list(self, request, format=None):
content = {
'user_feed': request.user.get_user_feed()
}
return Response(content)
Or
class PostView(APIView):
# Cache page for the requested url
#method_decorator(cache_page(60*60*2))
def get(self, request, format=None):
content = {
'title': 'Post title',
'body': 'Post content'
}
return Response(content)
DRF cache docs.
settings.py:
# Enable memcache
# https://devcenter.heroku.com/articles/memcache#using_memcache_from_python
CACHES = {
'default': {
'BACKEND': 'django_pylibmc.memcached.PyLibMCCache'
}
}
If you want store just result of a method or a query and store for furture requests, you can just define a key for it and set that result to that key with cache.set(cache_key, result, expire_time) and then get it(cache.get(cache_key)) whenever you want.
Remember you should define a cache backend for your results.Proper and better solution is to use message brokers like redis or memcached to store cache.based on your needs.
//Updated//
check if data is already cashed or not.
class WeatherView(APIView):
def get(self):
if(cache.get('weatherdata') == None):
url = 'https://api.openweathermap.org/data/2.5/forecast?q=' + '...';
serialized_data = urlopen(url).read()
data = json.loads(serialized_data)
print(data)
cache.set('weatherdata', data, 3600)
else:
data = cache.get('weatherdata')
serializer_class = WeatherSerializer(data)
responseJSON = JSONRenderer().render(serializer_class.data)
return Response(responseJSON)

Django Rest Framework - Using Session and Token Auth

I am trying to get this working but don't know if this is possible. It should be doing it like this.
I developed a web app using Django + Rest-Framework + jQuery, and I want to have an external application to consume the same REST API, using JWT Tokens for authentication.
My current config is like this.
settings.py
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
'rest_framework.authentication.SessionAuthentication',
'rest_framework_simplejwt.authentication.JWTAuthentication',
],
'DEFAULT_RENDERER_CLASS': [
'rest_framework.renderers.JSONRenderer',
]
}
SIMPLE_JWT = {
'AUTH_HEADER_TYPES': ('Bearer',),
}
views.py
class ListFileView(APIView):
permission_classes = (IsAuthenticated,)
def get(self, request, *args, **kwargs):
user = request.user
if user:
obj = Foo.objects.filter(owner=user)
serializer = FooSerializer(obj, many=True)
data = serializer.data
return Response(data, status=status.HTTP_200_OK)
return Response({'detail': 'You have no access to files.'}, status=status.HTTP_400_BAD_REQUEST)
The tricky part is that when I use:
permission_classes = (IsAuthenticated,)
I can perform ajax calls from an external application (using a valid JWT token), but jQuery calls from within the app (with an authenticated user) fail with:
{"detail":"Authentication credentials were not provided."}
And on the other hand, if I use autentication_classes instead of permission_classes:
authentication_classes = (SessionAuthentication, BasicAuthentication)
I can perform ajax calls from within the web app using jQuery, but external calls fail with the same 403 error.
I tried using both like this:
class ListFileView(APIView):
authentication_classes = (SessionAuthentication, BasicAuthentication)
permission_classes = (IsAuthenticated,)
def get(self, request, *args, **kwargs):
...
but the external calls are also rejected.
Is it possible to have these two types of Auth working together in the same class view, or should I separate into two endpoints?
EDIT
Example calls from within the app with jQuery:
<script type="text/javascript">
function loadTblDocs() {
$("#tbl-docs > tbody").empty();
$.ajaxSetup({
headers: { "X-CSRFToken": '{{csrf_token}}' }
});
$.ajax({
type: 'GET',
contentType: "application/json; charset=utf-8",
url: "/api/list/",
dataType: "json",
success: function (data) {
console.log(data);
}
});
};
</script>
And externally via VBA code:
Set web = CreateObject("WinHttp.WinHttpRequest.5.1")
web.Open "GET", "/api/list/", False
web.SetRequestHeader "Authorization", "Bearer <JWT_TOKEN_HERE>"
web.Send

I couldn't work out exactly what is going on in your case, because the behavior in the 3 cases you shared does not seem to be consistent, but going to explain simply how authentication and authorization is determined in DRF, this might help you figure the issue out.
You should be able to use two authentication classes with no problems. With DRF, authentication works like this:
At each request, DRF goes over the provided authentication classes, in the order they are defined. For each class, there are 3 cases:
If it can authenticate the request with the current class, DRF sets request.user. From this point on, this request is authenticated.
If no authentication credentials are present, DRF skips that class
If authentication credentials are present but invalid, such as an invalid JWT token in Authorization header, DRF raises an exception and returns a 403 response.
DRF views normally use the DEFAULT_AUTHENTICATION_CLASSES variable defined in the settings file, but if you provide them in a view, settings are overridden.
Authorization comes into play when you add permission_classes to your views. permission_classes control access to your views, so when you add IsAuthenticated to a view's permission classes, DRF rejects the request with a 403 response if you try to access that view without an authenticated user.
So in your initial case, your internal AJAX request failing in this case suggests that there is no authenticated user data in your request session. I do not know what could be causing this. Perhaps you do not update request session in your login view (Django's login method should do this automatically).
In your second case, you remove permission_classes from the view, so the view will serve the request regardless if there is an authenticated user in the request or not. So it is expected that your internal AJAX request succeeds here, but I do not know why your external request fails in this case.
In your third case, from the point of your internal AJAX request, the scenario seems to be the same as the first case, so I do not know why your internal AJAX request succeeds in this case but not in the first case. The external request failing here is expected because you added the IsAuthenticated permission class to the view, but did not include JWTAuthentication in the authentication_classes, so your request can not be authenticated with your JWT token here.

Django Rest Framework - Updating related model using ModelSerializer and ModelViewSet

BACKGROUND
I have two serializers: PostSerializer and PostImageSerializer which both inherit DRF ModelSerializer. The PostImage model is linked with Post by related_name='photos'.
Since I want the serializer to perform update, PostSerializer overrides update() method from ModelSerializer as stated in official DRF doc.
class PostSerializer(serializers.ModelSerializer):
photos = PostImageSerializer(many=True)
class Meta:
model = Post
fields = ('title', 'content')
def update(self, instance, validated_data):
photos_data = validated_data.pop('photos')
for photo in photos_data:
PostImage.objects.create(post=instance, image=photo)
return super(PostSerializer, self).update(instance, validated_data)
class PostImageSerializer(serializer.ModelSerializer):
class Meta:
model = PostImage
fields = ('image', 'post')
I have also defined a ViewSet which inherits ModelViewSet.
class PostViewSet(viewsets.ModelViewSet):
queryset = Post.objects.all()
serializer_class = PostSerializer
Finally the PostViewSet is registered to DefaultRouter. (Omitted code)
Goal
The goals are simple.
Send PUT request via PostMan with url like 'PUT http://localhost:8000/api/posts/1/'
Since Image files should be included, request would be done by form-data like below.
Problem
I'm getting 400 Response with error message as following.
{
"photos": [
"This field is required."
],
"title": [
"This field is required."
],
"content": [
"This field is required."
]
}
(Should you plz note that the error messages might not exactly fit with DRF error messages since they are translated.)
It is obvious that none of my PUT fields are applied.
So I have been digging around Django rest framework source code itself and found out serializer validation in ViewSet update() method continues to fail.
I doubt that because I PUT request not by JSON but by form-data using key-value pair so request.data is not properly validated.
However, I should contain multiple images in the request which means plain JSON would not work.
What would be the most clear solutions for this case?
Thank you.
Update
As Neil pointed out, I added print(self) at the first line of update() method of PostSerializer. However nothing printed out on my console.
I think this is due to my doupt above because perform_update() method which calls serializer update() method is called AFTER serializer is validated.
Therefore the main concept of my question could be narrowed to the followings.
How should I fix requested data fields so that validation inside update() method of ModelViewSet could pass?
Do I have to override update() method of ModelViewSet(not the one from ModelSerializer)?
Thanks again.

First of all you need to set header:
Content-Type: multipart/form-data;
But maybe if you set form-data in postman, this header should be
default.
You can't send images as a json data (unless you encode it to string and decode on server side to image eg. base64).
In DRF PUT by default requires all fields. If you want to set only partial fields you need to use PATCH.
To get around this and use PUT to update partial fields you have two options:
edit update method in viewset to partial update serializer
edit router to always call partial_update method in serializers which is more advanced
You can override viewset update method to always update serializer partial (changing only provided fields):
def update(self, request, *args, **kwargs):
partial = True # Here I change partial to True
instance = self.get_object()
serializer = self.get_serializer(instance, data=request.data, partial=partial)
serializer.is_valid(raise_exception=True)
self.perform_update(serializer)
return Response(serializer.data)
Add
rest_framework.parsers.MultiPartParser
to the main settings file to the REST_FRAMEWORK dict:
REST_FRAMEWORK = {
...
'DEFAULT_PARSER_CLASSES': (
'rest_framework.parsers.JSONParser',
'rest_framework.parsers.MultiPartParser',
)
}
Looking at your serializers it's weird that you don't get error from PostSerializer because you don't add "photos" field to Meta.fields tuple.
More advices from me in this case:
add required=False to your photos field (unless you want this to be required)
as wrote above add photos field to you Meta.fields tuple fields = ('title', 'content', 'photos',)
add default None value for your validated_data.pop('photos'), then check photos data is provided before loop.

The solution is somewhat a mixture or #Neil and #mon's answers. However I'll straighten out a bit more.
Analysis
Right now Postman submits form data which contains 2 key-value pairs(Please refer to the photo which I uploaded in my original question). One is "photos" key field linked with multiple photo files and the other is "data" key field linked with one big chunk of 'JSON-like string'. Although this is a fair method POSTing or PUTting data along with files, DRF MultiPartParser or JSONParser won't parse these properly.
The reason why I got the error message was simple. self.get_serializer(instance, data=request.data, partial=partial method inside ModelViewSet(especially UpdateModelMixin) couldn't understand request.data part.
Currently request.data from submitted form data looks like below.
<QueryDict: { "photos": [PhotoObject1, PhotoObject2, ... ],
"request": ["{'\n 'title': 'title test', \n 'content': 'content test'}",]
}>
Watch the "request" part carefully. The value is a plain string object.
However my PostSerializer expects the request.data to look something like below.
{ "photos": [{"image": ImageObject1, "post":1}, {"image": ImageObject2, "post":2}, ... ],
"title": "test title",
"content": "test content"
}
Therefore, let's do some experiment and PUT some data in accordance with above JSON form.
i.e
{ "photos": [{"image": "http://tny.im/gMU", "post": 1}],
"title" : "test title",
"content": "test content"
}
You'll get an error message as following.
"photos": [{"image": ["submitted data is not a file."]}]
Which means every data is submitted properly but the image url http://tny.im/gMU is not a file but string.
Now the reason of this whole problem became clear. It is the Parser which needs to be fixed so that the Serializer could understand submitted form data.
Solution
1. Write new parser
New parser should parse 'JSON-like' string to proper JSON data. I've borrowed the MultipartJSONParser from here.
What this parser does is simple. If we submit 'JSON-like' string with the key 'data', call json from rest_framework and parse it. After that, return the parsed JSON with requested files.
class MultipartJsonParser(parsers.MultiPartParser):
# https://stackoverflow.com/a/50514022/8897256
def parse(self, stream, media_type=None, parser_context=None):
result = super().parse(
stream,
media_type=media_type,
parser_context=parser_context
)
data = {}
data = json.loads(result.data["data"])
qdict = QueryDict('', mutable=True)
qdict.update(data)
return parsers.DataAndFiles(qdict, result.files)
2. Redesign Serializer
Official DRF doc suggests nested serializers to update or create related objects. However we have a significant drawback that InMemoryFileObject cannot be translated into a proper form which the serializer expects. To do this, we should
Override updatemethod of ModelViewSet
Pop out 'photos' key-value pair from request.data
Translate popped 'photos' pairs into the list of dictionaries containing 'image' and 'post' keys.
Append the result to request.data with a key name 'photos'. This is because our PostSerializer expects the key name to be 'photos'.
However basically request.data is a QuerySet which is immutable by default. And I am quite skeptical if we must force-mutate the QuerySet. Therefore, I'll rather commission PostImage creation process to update() method of the ModelViewSet. In this case, we don't need to define nested serializer anymore.
Simply just do this:
class PostSerializer(serializers.ModelSerializer):
class Meta:
model = Post
fields = '__all__'
class PostImageSerializer(serializer.ModelSerializer):
class Meta:
model = PostImage
fields = '__all__'
3. Override update() method from ModelViewSet
In order to utilize our Parser class, we need to explicitly designate it. We will consolidate PATCH and PUT behaviour, so set partial=True. As we saw earlier, Image files are carried with the key 'photos' so pop out the values and create each Photo instance.
Finally, thanks to our newly designed Parser, plain 'JSON-like' string would be transformed into regular JSON data. So just simply put eveything into serializer_class and perform_update.
class PostViewSet(viewsets.ModelViewSet):
queryset = Post.objects.all()
serializer_class = PostSerializer
# New Parser
parser_classes = (MultipartJsonParser,)
def update(self, request, *args, **kwargs):
# Unify PATCH and PUT
partial = True
instance = self.get_object()
# Create each PostImage
for photo in request.data.pop("photos"):
PostImage.objects.create(post=instance, image=photo)
serializer = self.get_serializer(instance, data=request.data, partial=partial)
serializer.is_valid(raise_exception=True)
# Do ViewSet work.
self.perform_update(serializer)
return Response(serializer.data)
Conclusion
The solution works, but I'm not sure this is the cleanest way of saving foreign key related models. I get a strong feeling that it is the serializer that should save the related model. Just as the doc stated, data other than files are saved that way. If somebody could tell me more subtle way to do this, I would be deeply appreciated.

How to pass django rest framework response to html?

How to pass django restframework response for any request to html.
Example: A list which contains objects, and html be articles.html.
I tried by using rest framework Response :
data= {'articles': Article.objects.all() }
return Response(data, template_name='articles.html')
I am getting this error :
""" AssertionError at /articles/
.accepted_renderer not set on Response """
Where i went wrong, please suggest me.

If it's a function based view, you made need to use an #api_view decorator to display properly. I've seen this particular error happen for this exact reason (missing API View declaration in function based views).
from rest_framework.decorators import api_view
# ....
#api_view(['GET', 'POST', ])
def articles(request, format=None):
data= {'articles': Article.objects.all() }
return Response(data, template_name='articles.html')

In my case just forgot to set #api_view(['PUT']) on view function.
So,
.accepted_renderer The renderer instance that will be used to render the response not set for view.
Set automatically by the APIView or #api_view immediately before the response is returned from the view.

Have you added TemplateHTMLRenderer in your settings?
http://www.django-rest-framework.org/api-guide/renderers/#setting-the-renderers

You missed TemplateHTMLRenderer decorator:
#api_view(('GET',))
#renderer_classes((TemplateHTMLRenderer,))
def articles(request, format=None):
data= {'articles': Article.objects.all() }
return Response(data, template_name='articles.html')