We Keep Coding

Generating OpenAPI Schema in Django Rest Framework: URL Keyword in Update ViewSet

I'm trying to do something that should be pretty straight forward but I'm running into a problem that I can't make heads or tails of. I'm hoping someone can help me sort it out.
I've created some API endpoints using DRF. Naturally, I have Create, Update, Retrieve, and Delete endpoints that I've constructed with the django_rest_framework.generics. I need to send some extra data into the Serializer from the ViewSet on both the CreateAPIView and UpdateAPIView. To accomplish this, I've extended the get_serializer_context() method on those ViewSets.
I'm now trying to create OpenAPI documentation for those said endpoints. However, when I run generateschema in the manage.py it returns an error. This error occurs in each of the UpdateAPIViews where I've extended the get_serializer_context() method. An example of this error is shown below.
AssertionError: Expected view UpdateStaffViewSet to be called with a URL keyword argument named "staff_id". Fix your URL conf, or set the `.lookup_field` attribute on the view correctly.
Naturally, we'd think that I goofed the URL keyword or the lookup_field, just as the error states. Unless I'm looking over something obvious though, I just cannot see what the issue is. These endpoints function perfectly when tested using curl or postman. I do know that if I remove the extended get_serializer_context() method, the generateschema command works fine. (But then an test using postman/curl will fail.) Why in the heck would that extended method matter!?
urls.py
url_patterns = [
path('v1/staff/retrieve/<int:staff_id>/', RetrieveStaffViewSet.as_view()),
path('staff/update/<int:staff_id>/', UpdateStaffViewSet.as_view()),
...
]
viewsets
class UpdateRetrieveBase:
serializer_class = StaffSerializer
queryset = Staff.objects.select_related('user')
lookup_url_kwarg = 'staff_id'
lookup_field = 'pk'
class UpdateStaffViewSet(UpdateRetrieveBase, UpdateAPIView):
def get_serializer_context(self):
context = super().get_serializer_context()
context['validation_refs'] = get_validation_refs(staff=self.get_object())
return context
class RetrieveStaffViewSet(UpdateRetrieveBase, RetrieveAPIView):
pass

Alright well after all that, I found my own answer. I'll post it here in case someone else also gets stumped.
Since the generateschema command runs a test but then doesn't include URL kwargs, we cannot call get_object() in the viewset. The test must account for that but doesn't consider that we may be calling that method anywhere else. To fix this in my particular case I edited the code as thus.
viewset
class UpdateAEAContractViewSet(UpdateRetrieveBase, UpdateAPIView):
def get_serializer_context(self):
context = super().get_serializer_context()
contract_id = self.kwargs.get(self.lookup_url_kwarg)
context['validation_refs'] = get_validation_refs(contract_id=contract_id)
return context
validation ref subjob
def get_validation_refs(staff_id=None):
staff = Staff.objects.get(pk=staff_id) if staff_id else None
...

Django REST Framework - can't override list in ListAPIView

I am using Django REST Framework to create an endpoint that will produce a PDF document. The PDF document will have information that corresponds to a particular Department. I have two desired functionalities -- to be able to download a PDF document, and to be able to preview the document within the browser.
Since the PDF document changes over time based on data that is added to the app, the document needs to be generated in real time when it is requested. As a first step, I'm trying to have the document be generated in a remote file storage location when the following endpoint is hit by a GET request:
departments/<department_pk>/result/preview
Since my endpoint should only take GET requests, I am using a ListAPIView. I'm trying to override the list method so that my custom document generation logic is executed, but it looks like the method is never called. How can I have some custom document generation logic be inserted into my endpoint, so that it is executed when the endpoint is hit by a GET request?
api/urls.py
url(r'^departments/(?P<department_pk>[0-9]+)/result/preview',
include(result_document_urls.result_document_preview_router.urls,
document_app/urls.py
result_document_preview_router = routers.DefaultRouter()
result_document_preview_router.register(r'^', ResultDocumentDetailView.as_view(),
base_name='Department')
document_app/views.py
class ResultDocumentDetailView(generics.ListAPIView):
queryset = Department.objects.all()
lookup_field = 'department_pk'
lookup_url_kwarg = 'department_pk'
def list(self, request, department_pk):
queryset = self.get_queryset()
import ipdb; ipdb.set_trace() # this break point is never hit
department = get_object_or_404(queryset, department_pk=department_pk)
...generate document logic...
return Response(status=status.HTTP_200_OK)

replace list method with below code, I think it will work
class ResultDocumentDetailView(generics.ListAPIView):
queryset = Department.objects.all()
lookup_field = 'department_pk'
lookup_url_kwarg = 'department_pk'
def list(self, request, *args, **kwargs):
queryset = self.get_queryset()
import ipdb; ipdb.set_trace() # this break point is never hit
department = get_object_or_404(
queryset, department_pk=kwargs.get('department_pk')
)
...generate document logic...
return Response(status=status.HTTP_200_OK)
for more reference see the overrinding method "list"
https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/mixins.py#L35

In your document_app/urls.py, you are incorrectly passing ResultDocumentDetailView as an argument instead of a viewset.
Router while registering accepts a ViewSet instead of an APIView.
There are two mandatory arguments to the register() method:
prefix - The URL prefix to use for this set of routes.
viewset - The viewset class.
Also, since you are only interested in the retrieve method, you can just create a ResultDocumentRetrieveView and add its corresponding url to your urls.py without the need of creating a ResultDocument router. (Routers are generally used when you want to handle both list and detail requests.)
class ResultDocumentRetrieveView(generics.RetrieveAPIView):
queryset = Department.objects.all()
lookup_field = 'department_pk'
lookup_url_kwarg = 'department_pk'
def retrieve(self, request, department_pk):
department = self.get_object()
...generate document logic...
return Response(status=status.HTTP_200_OK)
urls.py
url(r'^departments/(?P<department_pk>[0-9]+)/result/preview', ResultDocumentRetrieveView.as_view())

What is the "format" parameter used for in Django REST Framework views?

In the DRF documentation example found here:
class SnippetList(APIView):
"""
List all snippets, or create a new snippet.
"""
def get(self, request, format=None):
snippets = Snippet.objects.all()
serializer = SnippetSerializer(snippets, many=True)
return Response(serializer.data)
def post(self, request, format=None):
serializer = SnippetSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
The above sample has an unused format parameter in both the get() and post() methods. I have stepped through the rest_framework source and I cannot find any instance where this parameter is handed off to the get() or post() method by the dispatcher. Is this parameter required to be present in the method definition? If so, what is it used for? When is it used?

I found that the format parameter is used when you use format suffixes. So now I wanted to find out how this was used because nothing in the handler code seems to be using it but somehow it still works (magic behavior).
I went ahead and used pdb to trace execution of the view and found that APIView.initial() extracts this parameter from the kwargs and the uses it in the call to APIView.perform_content_negotiation() which selects the renderer and media type to use in the response.

Basiclly this "format" parameter is used to define the output response format, like: csv, json, etc

Consider example:
suppose you want the 'html' format of response which you receive your api endpoint
So for this, you can easily make a query from the url itself like
https://api.example.com/users/?format=html
Now if your views doesn't have the parameter as format=None then this could work.
But to disable this behaviour you must turn format=None

Nested resources in Django REST Framework

I wish to implement my new API with a nested resource.
Example: /api/users/:user_id/posts/
Will evaluate to all of the posts for a specific user. I haven't seen an working example for this use case, maybe this isn't the right way for implementing rest API?

As commented by Danilo, the #link decorator got removed in favor of #list_route and #detail_route decorators.
Update: #detail_route & #list_route got deprecated in favor of #action.
Here's the alternate solutions:
Solution 1:
#detail_route()
def posts(self, request, pk=None):
owner = self.get_object()
posts = Post.objects.filter(owner=owner)
context = {
'request': request
}
post_serializer = PostSerializer(posts, many=True, context=context)
return Response(post_serializer.data)
Solution 2:
Try drf-nested-routers. Haven't tried this out yet, but looks promising, many are already using it. Looks like an advanced version of what we are already trying to achieve.
Hope this helps.

To map /api/users/:user_id/posts/ you can decorate a posts method inside your ViewSet with #link()
from rest_framework.decorators import link
from rest_framework.response import Response
class UserViewSet(viewsets.ModelViewSet):
model = User
serializer_class = UserSerializer
# Your regular ModelViewSet things here
# Add a decorated method like this
#link()
def posts(self, request, pk):
# pk is the user_id in your example
posts = Post.objects.filter(owner=pk)
# Or, you can also do a related objects query, something like:
# user = self.get_object(pk)
# posts = user.post_set.all()
# Then just serialize and return it!
serializer = PostSerializer(posts)
return Response(serializer.data)

As commented by Danilo Cabello earlier you would use #detail_route or #list_route instead of #link(). Please read the documentation for "Routers", section "Extra link and actions" and "ViewSets", section "Marking extra actions for routing" for detailed explanations.

Django and Middleware which uses request.user is always Anonymous

I'm trying to make middleware which alters some fields for the user based on subdomain, etc...
The only problem is the request.user always comes in as AnonymousUser within the middleware, but is then the correct user within the views. I've left the default authentication and session middleware django uses within the settings.
There is a similar question here: Django, request.user is always Anonymous User
But doesn't overly answer the total question because I'm not using different authentication methods, and djangos authentication is running before I invoke my own middleware.
Is there a way, while using DRF, to get the request.user within the middleware? I'll show some sample code here:
class SampleMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
#This will be AnonymousUser. I need it to be the actual user making the request.
print (request.user)
def process_response(self, request, response):
return response
with process_request:
class SampleMiddleware(object):
def process_request(self, request):
#This will be AnonymousUser. I need it to be the actual user making the request.
print (request.user)
def process_response(self, request, response):
return response

I've solved this problem by getting DRF token from the requests and loading request.user to the user associated to that model.
I had the default django authentication and session middleware, but it seems DRF was using it's token auth after middleware to resolve the user (All requests were CORS requests, this might have been why). Here's my updated middleware class:
from re import sub
from rest_framework.authtoken.models import Token
from core.models import OrganizationRole, Organization, User
class OrganizationMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
header_token = request.META.get('HTTP_AUTHORIZATION', None)
if header_token is not None:
try:
token = sub('Token ', '', header_token)
token_obj = Token.objects.get(key = token)
request.user = token_obj.user
except Token.DoesNotExist:
pass
#This is now the correct user
print (request.user)
This can be used on process_view or process_request as well.
Hopefully this can help someone out in the future.

Came across this today while having the same problem.
TL;DR;
Skip below for code example
Explanation
Thing is DRF have their own flow of things, right in the middle of the django request life-cycle.
So if the normal middleware flow is :
request_middleware (before starting to work on the request)
view_middleware (before calling the view)
template_middleware (before render)
response_middleware (before final response)
DRF code, overrides the default django view code, and executes their own code.
In the above link, you can see that they wrap the original request with their own methods, where one of those methods is DRF authentication.
So back to your question, this is the reason using request.user in a middleware is premature, as it only gets it's value after view_middleware** executes.
The solution I went with, is having my middleware set a LazyObject.
This helps, because my code (the actual DRF ApiVIew) executes when the actual user is already set by DRF's authentication.
This solution was proposed here together with a discussion.
Might have been better if DRF had a better way to extend their functionality, but as things are, this seems better than the provided solution (both performance and readability wise).
Code Example
from django.utils.functional import SimpleLazyObject
def get_actual_value(request):
if request.user is None:
return None
return request.user #here should have value, so any code using request.user will work
class MyCustomMiddleware(object):
def process_request(self, request):
request.custom_prop = SimpleLazyObject(lambda: get_actual_value(request))

The accepted answer only takes TokenAuthentication in consideration - in my case, there are more authentication methods configured. I thus went with initializing the DRF's Request directly which invokes DRF's authentication machinery and loops through all configured authentication methods.
Unfortunately, it still introduces an additional load on the database since the Token object must be queried (the accepted answer has this problem as well). The trick with SimpleLazyObject in this answer is a much better solution, but it didn't work for my use case because I need the user info in the middleware directly - I'm extending the metrics in django_prometheus and it processes the request before get_response is called.
from rest_framework.request import Request as RestFrameworkRequest
from rest_framework.views import APIView
class MyMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
drf_request: RestFrameworkRequest = APIView().initialize_request(request)
user = drf_request.user
...
return self.get_response(request)

Based on Daniel Dubovski's very elegant solution above, here's an example of middleware for Django 1.11:
from django.utils.functional import SimpleLazyObject
from organization.models import OrganizationMember
from django.core.exceptions import ObjectDoesNotExist
def get_active_member(request):
try:
active_member = OrganizationMember.objects.get(user=request.user)
except (ObjectDoesNotExist, TypeError):
active_member = None
return active_member
class OrganizationMiddleware(object):
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
# Code to be executed for each request before
# the view (and later middleware) are called.
request.active_member = SimpleLazyObject(lambda: get_active_member(request))
response = self.get_response(request)
# Code to be executed for each request/response after
# the view is called.
return response

Daniel Dubovski's solution is probably the best in most cases.
The problem with the lazy object approach is if you need to rely on the side effects. In my case, I need something to happen for each request, no matter what.
If I'd use a special value like request.custom_prop, it has to be evaluated for each request for the side effects to happen. I noticed that other people are setting request.user, but it doesn't work for me since some middleware or authentication class overwrites this property.
What if DRF supported its own middleware? Where could I plug it in? The easiest way in my case (I don't need to access the request object, only the authenticated user) seems to be to hook into the authentication class itself:
from rest_framework.authentication import TokenAuthentication
class TokenAuthenticationWithSideffects(TokenAuthentication):
def authenticate(self, request):
user_auth_tuple = super().authenticate(request)
if user_auth_tuple is None:
return
(user, token) = user_auth_tuple
# Do stuff with the user here!
return (user, token)
Then I could just replace this line in my settings:
REST_FRAMEWORK = {
"DEFAULT_AUTHENTICATION_CLASSES": (
#"rest_framework.authentication.TokenAuthentication",
"my_project.authentication.TokenAuthenticationWithSideffects",
),
# ...
}
I'm not promoting this solution, but maybe it will help someone else.
Pros:
It to solves this specific problem
There's no double authentication
Easy to maintain
Cons:
Not tested in production
Things happen in an unexpected place
Side effects...

I know it's not exactly answering the 'can we access that from the middleware' question, but I think it's a more elegant solution VS doing the same work in the middleware VS what DRJ does in its base view class. At least for what I needed, it made more sense to add here.
Basically, I'm just overriding the method 'perform_authentication()' from DRF's code, since I needed to add more things related to the current user in the request. The method just originally call 'request.user'.
class MyGenericViewset(viewsets.GenericViewSet):
def perform_authentication(self, request):
request.user
if request.user and request.user.is_authenticated():
request.my_param1 = 'whatever'
After that in your own views, instead of settings APIView from DRF as a parent class, simply set that class as a parent.

I wasn't quite happy with the solutions out there. Here's a solution that uses some DRF internals to make sure that the correct authentication is applied in the middleware, even if the view has specific permissions classes. It uses the middleware hook process_view which gives us access to the view we're about to hit:
class CustomTenantMiddleware():
def process_view(self, request, view_func, view_args, view_kwargs):
# DRF saves the class of the view function as the .cls property
view_class = view_func.cls
try:
# We need to instantiate the class
view = view_class()
# And give it an action_map. It's not relevant for us, but otherwise it errors.
view.action_map = {}
# Here's our fully formed and authenticated (or not, depending on credentials) request
request = view.initialize_request(request)
except (AttributeError, TypeError):
# Can't initialize the request from this view. Fallback to using default permission classes
request = APIView().initialize_request(request)
# Here the request is fully formed, with the correct permissions depending on the view.
Note that this doesn't avoid having to authenticate twice. DRF will still happily authenticate right afterwards.

I had the same issue and decided to change my design. Instead of using a Middleware I simply monkey-patch rest_framework.views.APIView.
In my case I needed to patch check_permissions but you can patch whatever fits your problem. Have a look at the the source code.
settings.py
INSTALLED_APPS = [
..
'myapp',
]
myapp/patching.py
import sys
from rest_framework.views import APIView as OriginalAPIView
class PatchedAPIView(OriginalAPIView):
def check_permissions(self, request):
print(f"We should do something with user {request.user}"
return OriginalAPIView.check_permissions(self, request)
# We replace the Django REST view with our patched one
sys.modules['rest_framework'].views.APIView = PatchedAPIView
myapp/__init__.py
from .patching import *