We Keep Coding

Assertion error parsing response to POST request from DRF viewset with DRF APIClient in tests

I've run into a strange issue with Django Rest Framework testing engine. The weird thing is that everything used to work fine with Django 3 and this issue turned up after I migrated to Django 4. Apart from testing, everything works well, and responds to queries as expected.
The problem
I'm using DRF APIClient to make queries for unit tests. While GET requests perform predictably, I fail to make POST requests work.
Here is some minimalistic example code I created to figure out the issue. The versions I'm using:
Python 3.9
Django==4.0.3
djangorestframework==3.13.1
from django.db import models
from django.urls import include, path
from django.utils import timezone
from rest_framework import routers, serializers, viewsets
router = routers.DefaultRouter()
# models.py
class SomeThing(models.Model):
created_at = models.DateTimeField(default=timezone.now)
title = models.CharField(max_length=100, null=True, blank=True)
# serializers.py
class SomeThingSerializer(serializers.ModelSerializer):
class Meta:
fields = "__all__"
model = SomeThing
# views.py
class SomeThingViewSet(viewsets.ModelViewSet):
queryset = SomeThing.objects.all().order_by('id')
serializer_class = SomeThingSerializer
# urls.py
router.register("some-things", SomeThingViewSet, basename="some_thing")
app_name = 'question'
urlpatterns = (
path('', include(router.urls)),
)
Here is my test case:
import json
from django.contrib.auth import get_user_model
from rest_framework import status
from rest_framework.test import APITestCase, APIClient
class TestUserView(APITestCase):
self.some_user = get_user_model().objects.create(login="some_user#test.ru")
#staticmethod
def get_client(user):
client = APIClient()
client.force_authenticate(user=user)
return client
def test_do_something(self):
client = self.get_client(self.compliance_chief)
url = reverse('question:some_things-list')
resp = client.post(
path=url,
data=json.dumps({"title": "Created Something"}),
content_type="application/json",
)
assert resp.status_code == status.HTTP_201_OK
(Yes, I have to use some authentication to get access to the data, but I don't think it is relevant to the problem.) To which I receive a lengthy traceback, ending with an assertion error:
File "/****/****/****/venv/lib/python3.9/site-packages/django/test/client.py", line 82, in read
assert (
AssertionError: Cannot read more than the available bytes from the HTTP incoming data.
As it is really fairly long, I'll leave it just in case in a gist without posting it here.
Steps to fix
The problem clearly happens after the correct response is returned by the viewset. To make sure the response is correct I made a slight customisation in the create method to print out the response before it is returned, like so:
class SomeThingViewSet(viewsets.ModelViewSet):
queryset = SomeThing.objects.all().order_by('id')
serializer_class = SomeThingSerializer
def create(self, request, *args, **kwargs):
response = super().create(request, *args, **kwargs)
print("THIS IS THE RESPONSE FROM THE VIEWSET", response)
return response
And, sure enough, the result is correct:
THIS IS THE RESPONSE FROM THE VIEWSET <Response status_code=201, "text/html; charset=utf-8">
Which makes me think something goes wrong at the parsing stage (actually, the traceback implies the same). I tried to tweak the way I build the query, namely:
using format instead of content type like so: resp = client.post(path=url, data={"title": "Created Something"}, format="json")
using the .generic method instead of .post like so: resp = client.generic(method="POST", path=url, data=json.dumps({"title": "Created Something"}), content_type="application/json")
The result is the same.
From googling I found out that this error indeed has occasionally occurred in connection with DRF APIClient and Django, but really long ago (like this discussion, which claims that the issue was fixed in the later versions of Django).
I'm sure the reason for this behaviour is rather obvious (some stupid mistake most likely) and the solution must be very simple, but so far I've failed to find it. I would be very grateful if somebody shared their experience, if there is any, of dealing with such an issue, or their considerations as to where to move from this deadlock.

Alright, the mystery's been resolved and I'm going to share it here in case somebody runs into something similar, although it would take quite a coincidence, so it is unlikely.
Long story short: I messed up the source code of my Django4.0.3. installed in this project.
Now, how it happened. While I was testing some stuff, I ran into an error, which I failed to locate, so I went along the whole chain of events checking if the output was what I expected it to be. Soon enough I found myself checking the output from functions in the libraries installed under my virtual environment. I realise it's a malpractice to directly modify their code, but as I was working in my local environment with an option to reinstall everything at any moment, I decided it was fine to play with them. As it resulted in nothing, I removed all the code I had added (or so I thought).
After a while I realised what caused the initial error (an overlooked condition in my testing setup), fixed it and tried to run the test. That's when the problem in question showed up.
Later I found out that the same very test performs correctly in an identical environment. Then I suspected that I broke something in my local library code. Next I simply compared the code I had dealt with in my local environment with the code from the official source and soon enough I established the offending line. It happened to be in django/test/client.py, in the definition of the RequestFactory.generic method. Something like this:
...
if not r.get("QUERY_STRING"):
# WSGI requires latin-1 encoded strings. See get_path_info().
query_string = parsed[4].encode().decode("iso-8859-1")
r["QUERY_STRING"] = query_string
req = self.request(**r)
return self.request(**r)
...
The offending line (which I added and forgot to remove) was req = self.request(**r). After I deleted it, everything returned back to normal.

How to prevent Error: 403 in ModelAdmin.autocomplete_fields?

ModelAdmin.autocomplete_fields looks to be very simple to implement into the Django admin:
class UserAdmin(admin.ModelAdmin):
autocomplete_fields = ['material']
admin.site.register(User, UserAdmin)
class MaterialAdmin(admin.ModelAdmin):
search_fields = ['name']
admin.site.register(Material, MaterialAdmin)
It renders the field correctly (as a search field instead of a dropdown), but the search field says "The results could not be loaded" and inspect shows:
*/admin/autocomplete/ 403 (Forbidden)
jquery.js:9203
I assume there is a csrf issue receiving data from the Material model. I looked into ways to exempt this request from csrf but couldn't figure out how to do it through ModelAdmin.autocomplete_fields.
I tried using django-autocomplete-light as well and couldn't get it working.

That's because of the conflict between DAL and Django 3.2 + versions. If you turn DAL off it can solve this problem. DAL's js overloads Django's one and that's it. To know more just follow the link to
Dal's github issue

If you implement your own UserAdmin either make sure search_fields is defined, as requested in the documentation, or use Django's UserAdmin as a base class.
Also, if you just upgraded your Django version and it stopped working, clear your cache to load the proper javascript files.

Django's user_logged_in signal not working when running a test case

I have this signal:
#receiver(user_logged_in)
def on_user_login(sender, request, **kwargs):
if request.user.is_superuser:
It all works when running the app. But when testing, this piece of code fails:
self.client.force_login(...)
The test fails with:
AttributeError: HttpRequest object has no attribute user
If I remove the signal, that test case runs just fine.
I have Django 1.10

In my test I was testing one of the admin views. I wanted to login the user first before doing this change. What I did not realize was that I really wanted to test just the admin view, not logging in.
So instead of using client for testing, I used RequestFactory and passed the user in it. No logging in necessary.
The original error was due to the fact that client in django tests does not add the user to the request.

Django unit-testing client.login not working

I am using django 1.6 and I have the following test code:
def tes_stuff(self):
new_user=EndUser.objects.create(username="test", firstname="test", email="t#t.com", password="test")
self.assertTrue(self.client.login(username="test", password="test"))
When I run it I get an arror that says:
AssertionError: False is not true
I am unsure why it is not logging the user in. Perhaps it has to do with the password? EndUser is a model I created, but it extends from the normal user model in Django

Try using the create_user method. Link to the docs: create_user

Well I think i solved it, I added a save:
def tes_stuff(self):
new_user=EndUser.objects.create(username="test", firstname="test", email="t#t.com", password="test")
new_user.save()
self.assertTrue(self.client.login(username="test", password="test"))
It seems that this fixed it, I dont get errors now.

Django admin problems

I have the following stack trace when trying to access the django admin default app. Does anyone know how to fix it? I have defined the field date_created clearly in my models.py for the catalog app. I'm not sure where else I need to define it?
ImproperlyConfigured at /admin
'ProductAdmin.exclude' refers to field 'date_created' that is missing from the form.
Request Method: GET
Request URL: http://localhost:8000/admin
Django Version: 1.3.1
Exception Type: ImproperlyConfigured
Exception Value:
'ProductAdmin.exclude' refers to field 'date_created' that is missing from the form.
Exception Location: /Library/Python/2.6/site-packages/django/contrib/admin/validation.py in check_formfield, line 362
Python Executable: /usr/bin/python
Python Version: 2.6.1
Python Path:
['/Users/christopherfarm/Desktop/ecomstore',
'/Library/Python/2.6/site-packages/python_dateutil-1.5-py2.6.egg',
'/Library/Python/2.6/site-packages/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg',
'/Library/Python/2.6/site-packages/django_db_log-2.2.1-py2.6.egg',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python26.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-old',
'/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload',
'/Library/Python/2.6/site-packages',
'/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC',
'/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']
Server time: Mon, 31 Oct 2011 16:02:11 -0500

My guess is that you've set auto_now_add or auto_add in your models.py field, thus it's not a part of your form in the first place to exclude.
Try removing it from your exclude statement.

Did you make the change after the initial syncdb? If so, you need to do a reset on the app in question so that the tables get dropped, re-created, and (if you have fixtures) reloaded.
Alternatively, you can go into the dbshell for your app and alter the table yourself using SQL.

I have the very same error. I came back to the project I was not dealing with for a while. And I run it with django==1.5 instead of django==1.6 on which is was working.
so to summarize in my case it was connected with django version on 1.5 I got error like:
ExhibitorAdmin.exclude' refers to field 'collectedcontact' that is missing from the form.
while, when switching to django==1.6 this problem for me disappears immediately.