We Keep Coding

How should I continue a Python Social Auth Partial Pipeline

The application I am working has an overwritten endpoint for the Python Social Auth /complete/<backend>/ endpoint.
within our urls.py:
urlspatterns = [
...
# Override of social_auth
url(r'^api/v1/auth/oauth/complete/(?P<backend>[^/]+)/$',
social_auth_complete,
name='social_complete'),
...
]
within views.py:
from social_django.views import complete
def social_auth_complete(request, backend, *args, **kwargs):
"""Overwritten social_auth_complete."""
# some custom logic getting variables from session (Unrelated).
response = complete(request, backend, *args, **kwargs)
# Some custom logic adding args to the redirect (Unrelated).
We are attempting to implement a partial pipeline method. The first time the endpoint is called everything works as expected.
#partial
def required_info(strategy, details, user=None, is_new=False, *args, **kwargs):
"""Verify the user has all the required information before proceeding."""
if not is_new:
return
for field in settings.SOCIAL_USER_REQUIRED_DATA:
if not details.get(field):
data = strategy.request_data().get(field)
if not data:
current_partial = kwargs.get('current_partial')
social_provider = kwargs.get('backend')
return strategy.redirect(f'.../?partial_token={partial_token}&provider={social_provider}'
else:
details[field] = data
This redirects the user to the front end in which they fill out a form which calls a POST request to orginal API api/v1/auth/oauth/complete/(?P<backend>[^/]+)/ with the following in the data:
{
'required_fieldX': 'data',
...
'partial_token': '',
}
Key Issues
Two things go wrong; When I pdb into required_info there is never any data within strategy.request_data(). There is still data within the kwargs['request'].body and I can take the data out there.
However
But I am afraid that the second time around we never get into this block of code from social-core:
partial = partial_pipeline_data(backend, user, *args, **kwargs)
if partial:
user = backend.continue_pipeline(partial)
# clean partial data after usage
backend.strategy.clean_partial_pipeline(partial.token)
else:
user = backend.complete(user=user, *args, **kwargs)
I know this to be true because when I interrogate the database the original Partial object still exists as if backend.strategy.clean_partial_pipeline(partial.token) was never called.
Final Questions
Why is the social_django.views.complete not processing the POST request as expected and as it appears to be in all the example applications. Is there an issue from our overwriting it? Should I just create a separate endpoint to handle the POST request and if so how do mimic all that goes on within #psa such that I can call backend.continue_pipeline(partial)?

I think there's only one issue here, and that's that the Django strategy doesn't look into request.body when loading the request data, you can see the method in charge here. There you can see that it looks for request.GET and/or request.POST, but not body.
You can easily overcome this by defining your custom strategy that extends from the built-in one, and override the request_data method to look for the values in request.body. Then define the SOCIAL_AUTH_STRATEGY to point to your class.

What's the most elegant way to convert requests' response to DRF response in Django?

Consider the following flow:
public client ----> DRF API on Service A ------> DRF API on Service B
Some of the DRF API on Service A merely proxying to Service B, so in the particular API on Service A looks like this:
class SomeServiceAPI(APIView):
def get(request):
resp = requests.get('http://service-b.com/api/...')
return Response(resp.json())
While this works on normal status, but it has a few issues:
It doesn't proxy the actual status code from service b.
Unnecessary round-trip of json serialization within Response()
If service b returns a non-json error, service does not return actual error from service b.
The question is, is there a better way to do it? I had a look at Django Rest Framework Proxy project, but I am not entirely sure if it actually suits my use case here.

You can solve the status code part by modifying your Response:
return Response(resp.json(), status=resp.status_code)
For the second part though, this is the essence of Proxying... (True, sometimes you want to manipulate the request and/or the response in the middleman of the proxy, but what you do is the essence).
Notes:
The DRF Proxy that you are suggesting seems to do the job just
fine, without the need for you to write a specific view just for the
roundtrip.
There exist another tool, DRF Reverse Proxy which is a DRF port of Django Revproxy and you may want to consider.
The general idea of both of the above is that you create a URL path specifically to Proxy the path to another API:
DRF Proxy:
Add your proxy to settings.py:
REST_PROXY = {
'HOST': 'http://service-b.com/api/'
}
In urls.py:
url(
r'^somewere_in_a/$',
ProxyView.as_view(source='somewere_in_b/'),
name='a_name'
)
DRF Reverse Proxy:
Pretty much similar with the above, without the settings part:
url(
r'^(?P<path>.*)$',
ProxyView.as_view(upstream='http://service-b.com/api/somewere_in_b/'),
name='a_name'
)
Opinion: the DRF Proxy seems more solid...

I had a look at both existing packages mentioned in John's answer but they don't seem to perfectly suit in my use case, so I have created a simple wrapper to proxy the requests' response to DRF response.
# encoding: utf-8
from __future__ import unicode_literals
from __future__ import absolute_import
from rest_framework.response import Response
from requests.models import Response as RResponse
class InCompatibleError(Exception):
pass
class DRFResponseWrapper(Response):
"""
Wraps the requests' response
"""
def __init__(self, data, *args, **kwargs):
if not isinstance(data, RResponse):
raise InCompatibleError
status = data.status_code
content_type = data.headers.get('content_type')
try:
content = data.json()
except:
content = data.content
super(DRFResponseWrapper, self).__init__(content, status=status, content_type=content_type)
And use as below:
resp = requests.get(
'{}://{}/api/v5/business/'.format(settings.SEARCH_HOST_SCHEMA, settings.SEARCH_HOST),
params=request.query_params
)
return DRFResponseWrapper(resp)

DRF testing views with versioning: versioned url retrieval

I created some tests for my views before. Like that
class TestUserRegistrationViewUserCreate(APITestCase):
def setUp(self):
self.factory = APIRequestFactory()
def test_create_user(self):
data = {
'phone_number': '+79513332211',
'password': 'qwerty'
}
request = self.factory.post(reverse('user'), data=data)
response = CustomUserAPIView.as_view()(request)
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
Everything worked great, until I was asked to add API versioning.
DRF supports versioning natively http://www.django-rest-framework.org/api-guide/versioning/
so I just went with it and added namespace-based versioning to my APIs with
REST_FRAMEWORK = {
'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.NamespaceVersioning'
}
Now I need to rewrite my views unit tests to support versioning.
This problem is that in order to get versioned url through reverse, I have to use
from rest_framework.reverse import reverse
reverse('bookings-list', request=request)
like in the docs.
But I don't have a request objects in the tests, as I'm making one myself and versioned url required for making it.
What should I do?
P.S. I can implement versioning without using DRF one, with view decorator and a couple of utils functions and solve this problem, but it feels bad for me as I'm reinventing the wheel. Also, I might forget some edge cases too.

I use reverse('<VERSION>:<VIEW_NAME>') in my test cases.

Pretty late but for those having similar issues you can pass the version while calling the view -
response = CustomUserAPIView.as_view()(request, version='1.0')

Django and Middleware which uses request.user is always Anonymous

I'm trying to make middleware which alters some fields for the user based on subdomain, etc...
The only problem is the request.user always comes in as AnonymousUser within the middleware, but is then the correct user within the views. I've left the default authentication and session middleware django uses within the settings.
There is a similar question here: Django, request.user is always Anonymous User
But doesn't overly answer the total question because I'm not using different authentication methods, and djangos authentication is running before I invoke my own middleware.
Is there a way, while using DRF, to get the request.user within the middleware? I'll show some sample code here:
class SampleMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
#This will be AnonymousUser. I need it to be the actual user making the request.
print (request.user)
def process_response(self, request, response):
return response
with process_request:
class SampleMiddleware(object):
def process_request(self, request):
#This will be AnonymousUser. I need it to be the actual user making the request.
print (request.user)
def process_response(self, request, response):
return response

I've solved this problem by getting DRF token from the requests and loading request.user to the user associated to that model.
I had the default django authentication and session middleware, but it seems DRF was using it's token auth after middleware to resolve the user (All requests were CORS requests, this might have been why). Here's my updated middleware class:
from re import sub
from rest_framework.authtoken.models import Token
from core.models import OrganizationRole, Organization, User
class OrganizationMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
header_token = request.META.get('HTTP_AUTHORIZATION', None)
if header_token is not None:
try:
token = sub('Token ', '', header_token)
token_obj = Token.objects.get(key = token)
request.user = token_obj.user
except Token.DoesNotExist:
pass
#This is now the correct user
print (request.user)
This can be used on process_view or process_request as well.
Hopefully this can help someone out in the future.

Came across this today while having the same problem.
TL;DR;
Skip below for code example
Explanation
Thing is DRF have their own flow of things, right in the middle of the django request life-cycle.
So if the normal middleware flow is :
request_middleware (before starting to work on the request)
view_middleware (before calling the view)
template_middleware (before render)
response_middleware (before final response)
DRF code, overrides the default django view code, and executes their own code.
In the above link, you can see that they wrap the original request with their own methods, where one of those methods is DRF authentication.
So back to your question, this is the reason using request.user in a middleware is premature, as it only gets it's value after view_middleware** executes.
The solution I went with, is having my middleware set a LazyObject.
This helps, because my code (the actual DRF ApiVIew) executes when the actual user is already set by DRF's authentication.
This solution was proposed here together with a discussion.
Might have been better if DRF had a better way to extend their functionality, but as things are, this seems better than the provided solution (both performance and readability wise).
Code Example
from django.utils.functional import SimpleLazyObject
def get_actual_value(request):
if request.user is None:
return None
return request.user #here should have value, so any code using request.user will work
class MyCustomMiddleware(object):
def process_request(self, request):
request.custom_prop = SimpleLazyObject(lambda: get_actual_value(request))

The accepted answer only takes TokenAuthentication in consideration - in my case, there are more authentication methods configured. I thus went with initializing the DRF's Request directly which invokes DRF's authentication machinery and loops through all configured authentication methods.
Unfortunately, it still introduces an additional load on the database since the Token object must be queried (the accepted answer has this problem as well). The trick with SimpleLazyObject in this answer is a much better solution, but it didn't work for my use case because I need the user info in the middleware directly - I'm extending the metrics in django_prometheus and it processes the request before get_response is called.
from rest_framework.request import Request as RestFrameworkRequest
from rest_framework.views import APIView
class MyMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
drf_request: RestFrameworkRequest = APIView().initialize_request(request)
user = drf_request.user
...
return self.get_response(request)

Based on Daniel Dubovski's very elegant solution above, here's an example of middleware for Django 1.11:
from django.utils.functional import SimpleLazyObject
from organization.models import OrganizationMember
from django.core.exceptions import ObjectDoesNotExist
def get_active_member(request):
try:
active_member = OrganizationMember.objects.get(user=request.user)
except (ObjectDoesNotExist, TypeError):
active_member = None
return active_member
class OrganizationMiddleware(object):
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
# Code to be executed for each request before
# the view (and later middleware) are called.
request.active_member = SimpleLazyObject(lambda: get_active_member(request))
response = self.get_response(request)
# Code to be executed for each request/response after
# the view is called.
return response

Daniel Dubovski's solution is probably the best in most cases.
The problem with the lazy object approach is if you need to rely on the side effects. In my case, I need something to happen for each request, no matter what.
If I'd use a special value like request.custom_prop, it has to be evaluated for each request for the side effects to happen. I noticed that other people are setting request.user, but it doesn't work for me since some middleware or authentication class overwrites this property.
What if DRF supported its own middleware? Where could I plug it in? The easiest way in my case (I don't need to access the request object, only the authenticated user) seems to be to hook into the authentication class itself:
from rest_framework.authentication import TokenAuthentication
class TokenAuthenticationWithSideffects(TokenAuthentication):
def authenticate(self, request):
user_auth_tuple = super().authenticate(request)
if user_auth_tuple is None:
return
(user, token) = user_auth_tuple
# Do stuff with the user here!
return (user, token)
Then I could just replace this line in my settings:
REST_FRAMEWORK = {
"DEFAULT_AUTHENTICATION_CLASSES": (
#"rest_framework.authentication.TokenAuthentication",
"my_project.authentication.TokenAuthenticationWithSideffects",
),
# ...
}
I'm not promoting this solution, but maybe it will help someone else.
Pros:
It to solves this specific problem
There's no double authentication
Easy to maintain
Cons:
Not tested in production
Things happen in an unexpected place
Side effects...

I know it's not exactly answering the 'can we access that from the middleware' question, but I think it's a more elegant solution VS doing the same work in the middleware VS what DRJ does in its base view class. At least for what I needed, it made more sense to add here.
Basically, I'm just overriding the method 'perform_authentication()' from DRF's code, since I needed to add more things related to the current user in the request. The method just originally call 'request.user'.
class MyGenericViewset(viewsets.GenericViewSet):
def perform_authentication(self, request):
request.user
if request.user and request.user.is_authenticated():
request.my_param1 = 'whatever'
After that in your own views, instead of settings APIView from DRF as a parent class, simply set that class as a parent.

I wasn't quite happy with the solutions out there. Here's a solution that uses some DRF internals to make sure that the correct authentication is applied in the middleware, even if the view has specific permissions classes. It uses the middleware hook process_view which gives us access to the view we're about to hit:
class CustomTenantMiddleware():
def process_view(self, request, view_func, view_args, view_kwargs):
# DRF saves the class of the view function as the .cls property
view_class = view_func.cls
try:
# We need to instantiate the class
view = view_class()
# And give it an action_map. It's not relevant for us, but otherwise it errors.
view.action_map = {}
# Here's our fully formed and authenticated (or not, depending on credentials) request
request = view.initialize_request(request)
except (AttributeError, TypeError):
# Can't initialize the request from this view. Fallback to using default permission classes
request = APIView().initialize_request(request)
# Here the request is fully formed, with the correct permissions depending on the view.
Note that this doesn't avoid having to authenticate twice. DRF will still happily authenticate right afterwards.

I had the same issue and decided to change my design. Instead of using a Middleware I simply monkey-patch rest_framework.views.APIView.
In my case I needed to patch check_permissions but you can patch whatever fits your problem. Have a look at the the source code.
settings.py
INSTALLED_APPS = [
..
'myapp',
]
myapp/patching.py
import sys
from rest_framework.views import APIView as OriginalAPIView
class PatchedAPIView(OriginalAPIView):
def check_permissions(self, request):
print(f"We should do something with user {request.user}"
return OriginalAPIView.check_permissions(self, request)
# We replace the Django REST view with our patched one
sys.modules['rest_framework'].views.APIView = PatchedAPIView
myapp/__init__.py
from .patching import *

Django Nose how to write this test?

I'm completely new to testing in Django. I have started by installing nose and selenium and now I want to test the following code (below) It sends an SMS message.
This is the actual code:
views.py
#login_required
def process_all(request):
"""
I process the sending for a single or bulk message(s) to a group or single contact.
:param request:
"""
#If we had a POST then get the request post values.
if request.method == 'POST':
batches = Batch.objects.for_user_pending(request.user)
for batch in batches:
ProcessRequests.delay(batch)
batch.complete_update()
return HttpResponseRedirect('/reports/messages/')
So where do I start? This is what I have done so far...
1) created a folder called tests and added init.py.
2) created a new python file called test_views.py (I'm assuming that's correct).
Now, how do I go about writing this test?
Could someone show me with an example of how I write the test for the view above?
Thank you :)

First of all, you don't need selenium for testing views. Selenium is a tool for high-level in-browser testing - it's good and useful when you are writing UI tests simulating a real user.
Nose is a tool that makes testing easier by providing features like automatic test discovery, supplies a number of helper functions etc. The best way to integrate nose with your django project is to use django_nose package. All you have to do is to:
add django_nose to INSTALLED_APPS
define TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'
Then, every time you run python manage.py test <project_name> nose will be used to run your tests.
So, speaking about testing this particular view, you should test:
login_required decorator work - in other words, unauthenticated user will be redirected to the login page
if request.method is not POST, no messages sent + redirect to /reports/messages
sending SMS messages when POST method is used + redirect to /reports/messages
Testing first two statements is pretty straight-forward, but in order to test the last statement you need to provide more details on what is Batch, ProcessRequests and how does it work. I mean, you probably don't want to send real SMS messages during testing - this is where mocking will help. Basically, you need to mock (replace with your own implementation on the fly) Batch, ProcessRequests objects. Here's an example of what you should have in test_views.py:
from django.contrib.auth.models import User
from django.core.urlresolvers import reverse
from django.test.client import Client
from django.test import TestCase
class ProcessAllTestCase(TestCase):
def setUp(self):
self.client = Client()
self.user = User.objects.create_user('john', 'lennon#thebeatles.com', 'johnpassword')
def test_login_required(self):
response = self.client.get(reverse('process_all'))
self.assertRedirects(response, '/login')
def test_get_method(self):
self.client.login(username='john', password='johnpassword')
response = self.client.get(reverse('process_all'))
self.assertRedirects(response, '/reports/messages')
# assert no messages were sent
def test_post_method(self):
self.client.login(username='john', password='johnpassword')
# add pending messages, mock sms sending?
response = self.client.post(reverse('process_all'))
self.assertRedirects(response, '/reports/messages')
# assert that sms messages were sent
Also see:
https://docs.djangoproject.com/en/dev/topics/testing/
Django Testing Guide
Getting Started with Python Mock
Hope that helps.