We Keep Coding

Indirect way of usine Model.objects.all() in a formset

I'm using something like this to populate inlineformsets for an update view:
formset = inline_formsetfactory(Client, Groupe_esc, form=GroupEscForm, formset=BaseGroupEscInlineFormset, extra=len(Groupe.objects.all()))
(Basically I need as many extra form as there are entries in that table, for some special processing I'm doing in class BaseGroupEscInlineFormset(BaseInlineFormset)).
That all works fine, BUT if I pull my code & try to makemigrations in order to establish a brand new DB, that line apparently fails some django checks and throws up a "no such table (Groupe)" error and I cannot makemigrations. Commenting that line solves the issues (then I can uncomment it after making migration). But that's exactly best programming practices.
So I would need a way to achieve the same result (determine the extra number of forms based on the content of Groupe table)... but without triggering that django check that fails. I'm unsure if the answer is django-ic or pythonic.
E.g. perhaps I could so some python hack that allows me to specific the classname without actually importing Groupe, so I can do my_hacky_groupe_import.Objects.all(), and maybe that wouldn't trigger the error?
EDIT:
In forms.py:
from .models import Client, Groupe
class BaseGroupEscInlineFormset(BaseInlineFormSet):
def get_form_kwargs(self, index):
""" this BaseInlineFormset method returns kwargs provided to the form.
in this case the kwargs are provided to the GroupEsForm constructor
"""
kwargs = super().get_form_kwargs(index)
try:
group_details = kwargs['group_details'][index]
except Exception as ex: # likely this is a POST, but the data is already in the form
group_details = []
return {'group_details':group_details}
GroupeEscFormset = inlineformset_factory(Client, Groupe_esc,
form=GroupeEscForm,
formset=BaseGroupEscInlineFormset,
extra=len(Groupe.objects.all()),
can_delete=False)

The issue as already outlined is that your code is written at the module level and it executes a query when the migrations are not yet done, giving you an error.
One solution as I already pointed in the comment would be to write the line to create the formset class in a view, example:
def some_view(request):
GroupeEscFormset = inlineformset_factory(
Client,
Groupe_esc,
form=GroupeEscForm,
formset=BaseGroupEscInlineFormset,
extra=len(Groupe.objects.all()),
can_delete=False
)
Or if you want some optimization and want to keep this line at the module level to not keep recreating this formset class, you can override the __init__ method and accept extra as an argument (basically your indirect way to call Model.objects.all()):
class BaseGroupEscInlineFormset(BaseInlineFormSet):
def __init__(self, *args, extra=3, **kwargs):
self.extra = extra
super().__init__(*args, **kwargs)
...
GroupeEscFormset = inlineformset_factory(Client, Groupe_esc,
form=GroupeEscForm,
formset=BaseGroupEscInlineFormset,
can_delete=False)
# In your views:
def some_view(request):
formset = GroupeEscFormset(..., extra=Groupe.objects.count()) # count is better if the queryset is needed only to get a count

Permission class on #detail_route not working, Django Rest Framework

I'm new to DRF, but I'm trying to use a permission class on a #detail_route using the method in this stack thread: Using a permission class on a detail route
My code currently looks like this :
#detail_route(methods=['GET'], permission_classes=[IsStaffOrRestaurantUser])
def restaurant_dishes_ready_for_pickup(self, request, pk=None):
...stuff....
class IsStaffOrRestaurantUser(permissions.BasePermission):
def has_object_permission(self, request, view, obj):
print(request)
print(view)
print(obj)
return False
The print statements never get executed... I'm probably missing something but I've looked through the documentation and can't really figure it out, is my approach right at all? Thanks!
EDIT:
I realize in our code already that we have this snippet in our Viewset, is it possible to override this in the decorator?
def get_permissions(self):
# Limit to listing and getting for non Admin user
if self.request.method in permissions.SAFE_METHODS:
return (permissions.AllowAny(),)
return (IsAdminUser(),)

Not sure if it's the most elegant solution, but you might be able to upgrade get_permissions() like so:
def get_permissions(self):
# check additional route specifics
path = self.request.path
if ("restaurant_dishes_ready_for_pickup" in path):
return (IsStaffOrRestaurantUser,)
# Limit to listing and getting for non Admin user
if (self.request.method in permissions.SAFE_METHODS):
return (permissions.AllowAny,)
return (IsAdminUser,)
PS: Also maybe return permission class objects instead of instances in get_permissions().

Quote from the documentation:
If you're writing your own views and want to enforce object level permissions, or if you override the get_object method on a generic view, then you'll need to explicitly call the .check_object_permissions(request, obj) method on the view at the point at which you've retrieved the object.
So you'll need to call explicitly the permission check.
Note that you could have that for free if you were using a RetrieveAPIView instead of a function based view for example.

What is the proper way of testing throttling in DRF?

What is the proper way of testing throttling in DRF? I coulnd't find out any answer to this question on the net. I want to have separate tests for each endpoint since each one has custom requests limits (ScopedRateThrottle).
The important thing is that it can't affect other tests - they have to somehow run without throttling and limiting.

An easy solution is to patch the get_rate method of your throttle class. Thanks to tprestegard for this comment!
I have a custom class in my case:
from rest_framework.throttling import UserRateThrottle
class AuthRateThrottle(UserRateThrottle):
scope = 'auth'
In your tests:
from unittest.mock import patch
from django.core.cache import cache
from rest_framework import status
class Tests(SimpleTestCase):
def setUp(self):
cache.clear()
#patch('path.to.AuthRateThrottle.get_rate')
def test_throttling(self, mock):
mock.return_value = '1/day'
response = self.client.post(self.url, {})
self.assertEqual(
response.status_code,
status.HTTP_400_BAD_REQUEST, # some fields are required
)
response = self.client.post(self.url, {})
self.assertEqual(
response.status_code,
status.HTTP_429_TOO_MANY_REQUESTS,
)
It is also possible to patch the method in the DRF package to change the behavior of the standard throttle classes: #patch('rest_framework.throttling.SimpleRateThrottle.get_rate')

Like people already mentioned, this doesn't exactly fall within the scope of unit tests, but still, how about simply doing something like this:
from django.core.urlresolvers import reverse
from django.test import override_settings
from rest_framework.test import APITestCase, APIClient
class ThrottleApiTests(APITestCase):
# make sure to override your settings for testing
TESTING_THRESHOLD = '5/min'
# THROTTLE_THRESHOLD is the variable that you set for DRF DEFAULT_THROTTLE_RATES
#override_settings(THROTTLE_THRESHOLD=TESTING_THRESHOLD)
def test_check_health(self):
client = APIClient()
# some end point you want to test (in this case it's a public enpoint that doesn't require authentication
_url = reverse('check-health')
# this is probably set in settings in you case
for i in range(0, self.TESTING_THRESHOLD):
client.get(_url)
# this call should err
response = client.get(_url)
# 429 - too many requests
self.assertEqual(response.status_code, 429)
Also, regarding your concerns of side-effects, as long as you do user creation in setUp or setUpTestData, tests will be isolated (as they should), so no need to worry about 'dirty' data or scope in that sense.
Regarding cache clearing between tests, I would just add cache.clear() in tearDown or try and clear the specific key defined for throttling.

I implemented my own caching mechanism for throttling based on the user and the parameters with which a request is called. You can override SimpleRateThrottle.get_cache_key to get this behavior.
Take this throttle class for example:
class YourCustomThrottleClass(SimpleRateThrottle):
rate = "1/d"
scope = "your-custom-throttle"
def get_cache_key(self, request: Request, view: viewsets.ModelViewSet):
# we want to throttle the based on the request user as well as the parameter
# `foo` (i.e. the user can make a request with a different `foo` as many times
# as they want in a day, but only once a day for a given `foo`).
foo_request_param = view.kwargs["foo"]
ident = f"{request.user.pk}_{foo_request_param}"
# below format is copied from `UserRateThrottle.get_cache_key`.
return self.cache_format % {"scope": self.scope, "ident": ident}
In order to clear this in a TestCase I call the following method in each test method as required:
def _clear_throttle_cache(self, request_user, foo_param):
# we need to clear the cache of the throttle limits already stored there.
throttle = YourCustomThrottleClass()
# in the below two lines mock whatever attributes on the request and
# view instances are used to calculate the cache key in `.get_cache_key`
# which you overrode. Here we use `request.user` and `view.kwargs["foo"]`
# to calculate the throttle key, so we mock those.
pretend_view = MagicMock(kwargs={foo: foo_param})
pretend_request = MagicMock(user=request_user)
# this is the method you overrode in `YourCustomThrottleClass`.
throttle_key = throttle.get_cache_key(pretend_request, pretend_view)
throttle.cache.delete(user_key)

This is an amendment on yofee's post which got me 90% there. When using a throttle, custom or otherwise, with a set rate, get_rate is never called. As shown below from the source.
def __init__(self):
if not getattr(self, 'rate', None):
self.rate = self.get_rate()
Hence when one is mocking a throttle with a set rate that is not None, I would recommend patching the rate attribute directly.
...
with mock.patch.object(AuthRateThrottle, 'rate', '1/day'):
...

Django Rest Framework: How can I patch a serializer for unit testing, when it's used in the serializer_class attribute?

I have a serializer with an create() function. When I do a POST request, then I want to get this function called and create a new object. When I do it in the browser, it works and it also calls the function. But inside the test, it says that the function is not called. I think that I have done something wrong with the patch, because in the API it is only set as an serializer_class and the class is likely called somewhere inside the framework. Another thought was, that I do not need to test this, because it should be guaranteed by the rest_framework, that if I do it this way, the framework should call the function with the correct parameters.
# serializers.py
class FooSerializer(models.ModelSerializer):
class Meta:
...
def create(self, validated_data):
...
# apis.py
class FooAPI(generics.CreateAPIView):
serializer_class = FooSerializer
# tests.py
#patch('apis.FooSerializer'):
def test_that_create_is_called(self, mock):
mock.create = MagicMock()
mock.create.return_value = Foo() # Foo is my model
response = self.client.post('/foo', {name: 'Test'})
self.assertTrue(mock.create.called) # => Output says "False is not true"

Your current code is mocking the entire Serializer object, which is probably overkill and could stop the create method ever being called if it's expected to be called by internal logic on the serializer.
Instead you want to just patch a single method - like this:
#patch('apis.FooSerializer', 'create')
Now your test method receives the MagicMock object instance that has replaced the create method.
So your test method becomes:
def test_that_create_is_called(self, mock_method):
response = self.client.post('/foo', {name: 'Test'})
self.assertTrue(mock_method.called)

In Django how to avoid boilerplate code for getting model instance by pk in a view

Here is example code:
def someview(request):
try:
instance = SomeModel.objects.get(id=request.GET.get('id'))
except SomeModel.DoesNotExist:
instance = None
except ValueError:
# This error may occur if user manually enter invalid (non-integer)
# id value (intentionally or not) in a browser address bar, e.g.
# http://example.com/?id=2_foo instead of http://example.com/?id=2
# This raises ValueError: invalid literal for int() with base 10: '2_'
instance = None
...
Is there a best practice to get a model instance by pk without writing this boilerplate code over and over? Should I use some predefined shortcut in Django or just roll my own?
I was sure that I should use Django's DetailView or SingleObjectMixin but curiously enough it doesn't handle the ValueError exception from my example https://github.com/django/django/blob/master/django/views/generic/detail.py#L50
Is it implied that I have to specify correct integer regexp for pk kwarg in urlconf? Ok, likely. But what if I get pk from request querystring?
UPD I have special logic to do with instance either it's None or not.

You can also use Django's built in shorcut get_object_or_404() that it's designed for this specifically. That function will raise an Http404 exception in case the object doesn't exist. If you want to get None instead of raising the exception, you can create a helper function to accomplish it very easily:
def get_object_or_none(klass, *args, **kwargs):
try:
return get_object_or_404(klass, *args, **kwargs)
except Http404:
return None
Hope this helps!

The first part of your try/except block can be simplified by using django-annoying:
from annoying.functions import get_object_or_None
instance = get_object_or_None(SomeModel, id=request.GET.get('id'))
FYI, you can also just extract get_object_or_None from the package (see source).

There are many generic class based views that might be helpful, in your case DetailView could work.
from django.views.generic.detail import DetailView
class SomeModelDetailView(DetailView):
model = SomeModel
You can overwrite get_object method to change default behaviour.
def get_object(self, queryset=None):
return SomeModel.objects.get(pk=self.get_pk())
And lastly if object is none you should probably display custom 404 page.