We Keep Coding

Why is my Django DRF RequestsClient request failing in tests?

I have a few Django Rest Framework API endpoints I want to test.
They are behind an authenticated page. I'm trying to use the DRF RequestsClient to do that.
My code looks like this:
from rest_framework.test import APITestCase
from rest_framework.test import RequestsClient
from django.urls import reverse
import requests
URL_PREFIX="http://testserver"
API_TOKEN="tfhgffhgf675h"
class APITests(APITestCase):
#staticmethod
def _get_full_url(url, *args, **kwargs):
return URL_PREFIX + reverse(url, *args, **kwargs)
def setUp(self):
self.client = RequestsClient()
def test_stuff(self):
url = self._get_full_url("ahs:agena_results-list")
# Raw requests library - works fine
# Note: Testing shows that the RequestsClient() seems to do some sort of magic to
# be able to resolve http://testserver. A raw requests GET request must hit it at
# 127.0.0.1, and specify the port
response = requests.get("http://127.0.0.1:8000", headers={"Authorization": f"Token {API_TOKEN}"})
# RequestsClient() - doesn't work
response = self.client.get(url, headers={"Authorization": f"Token {API_TOKEN}"})
My RequestsClient request doesn't work, but my raw requests request works fine.
The exception raised when using the RequestsClient is pretty cryptic:
Internal Server Error: /ahs/api/agena_results/
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1361, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 83, in _execute
self.db.validate_no_broken_transaction()
File "/usr/local/lib/python3.9/site-packages/django/db/backends/base/base.py", line 480, in validate_no_broken_transaction
raise TransactionManagementError(
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 497, in dispatch
self.initial(request, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 414, in initial
self.perform_authentication(request)
File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 324, in perform_authentication
request.user
File "/usr/local/lib/python3.9/site-packages/rest_framework/request.py", line 227, in user
self._authenticate()
File "/usr/local/lib/python3.9/site-packages/rest_framework/request.py", line 380, in _authenticate
user_auth_tuple = authenticator.authenticate(self)
File "/usr/local/lib/python3.9/site-packages/rest_framework/authentication.py", line 196, in authenticate
return self.authenticate_credentials(token)
File "/usr/local/lib/python3.9/site-packages/rest_framework/authentication.py", line 201, in authenticate_credentials
token = model.objects.select_related('user').get(key=key)
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 492, in get
num = len(clone)
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 302, in __len__
self._fetch_all()
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1507, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 57, in __iter__
results = compiler.execute_sql(
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1364, in execute_sql
cursor.close()
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 83, in close
while self.nextset():
File "/usr/local/lib/python3.9/site-packages/MySQLdb/cursors.py", line 137, in nextset
nr = db.next_result()
MySQLdb._exceptions.OperationalError: (2006, '')
.
----------------------------------------------------------------------
Ran 1 test in 30.774s
The strange thing is that if I remove the Authorization header from the RequestsClient request, it doesn't crash, but does return a 403, so that doesn't help me.
I also tried setting the Authorization header on the RequestsClient this way, but that didn't help either:
self.client.headers.update({'Authorization': f'Token {API_TOKEN}'})
Any idea what I'm doing wrong?

Using APILiveServerTestCase instead of APITestCase when using RequestsClient solves the problem.
Apparently, the poor documentation around this has been known for some time now, but does not appear to have improved since then.

Cannot force an update in save() with no primary key

I am working with a custom build user model in Django and there was some message in the terminal at the time of migrate given in the stackoverflow link.
However, I have failed to solve that and it didn't hamper to run the project, so I actually ignore it and continue with my code. Now when I try to log-in to the admin panel(http://127.0.0.1:8000/admin/) it shows the following error:
Internal Server Error: /admin/login/
Traceback (most recent call last):
File "G:\Python\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "G:\Python\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "G:\Python\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "G:\Python\lib\site-packages\django\views\decorators\cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\contrib\admin\sites.py", line 407, in login
return LoginView.as_view(**defaults)(request)
File "G:\Python\lib\site-packages\django\views\generic\base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\utils\decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "G:\Python\lib\site-packages\django\views\decorators\debug.py", line 76, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\utils\decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "G:\Python\lib\site-packages\django\utils\decorators.py", line 130, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\utils\decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "G:\Python\lib\site-packages\django\views\decorators\cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\contrib\auth\views.py", line 63, in dispatch
return super().dispatch(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\views\generic\base.py", line 97, in dispatch
return handler(request, *args, **kwargs)
File "G:\Python\lib\site-packages\django\views\generic\edit.py", line 142, in post
return self.form_valid(form)
File "G:\Python\lib\site-packages\django\contrib\auth\views.py", line 92, in form_valid
auth_login(self.request, form.get_user())
File "G:\Python\lib\site-packages\django\contrib\auth\__init__.py", line 131, in login
user_logged_in.send(sender=user.__class__, request=request, user=user)
File "G:\Python\lib\site-packages\django\dispatch\dispatcher.py", line 173, in send
return [
File "G:\Python\lib\site-packages\django\dispatch\dispatcher.py", line 174, in <listcomp>
(receiver, receiver(signal=self, sender=sender, **named))
File "G:\Python\lib\site-packages\django\contrib\auth\models.py", line 20, in update_last_login
user.save(update_fields=['last_login'])
File "G:\Python\lib\site-packages\django\contrib\auth\base_user.py", line 66, in save
super().save(*args, **kwargs)
File "G:\Python\lib\site-packages\django\db\models\base.py", line 745, in save
self.save_base(using=using, force_insert=force_insert,
File "G:\Python\lib\site-packages\django\db\models\base.py", line 782, in save_base
updated = self._save_table(
File "G:\Python\lib\site-packages\django\db\models\base.py", line 847, in _save_table
raise ValueError("Cannot force an update in save() with no primary key.")
ValueError: Cannot force an update in save() with no primary key.

I was facing the same issue,I searched everywhere.
I found the answer in the link I will attach below.
1-to change the database name.
2-makemigrations and migrate.
but it failed to work until I read towen's comment! I was always creating
the super user before
migrating, which is wrong!
you have make the migrations before creating the super user.
so
python manage.py makemigrations
then
python manage.py migrate
then all the tables are ready to work properly, now we can create the super user, and it will have the primary key and it will work
python manage.py createsuperuser
https://github.com/nesdis/djongo/issues/3

TypeError: __str__ returned non-string (type NoneType) With Django third party package Models

I installed the django_message package into my django app using django 2.0 and while it works on my development machine, it throws the following errors on production when I try to open the list of messages from Django Admin console.
TypeError: __str__ returned non-string (type NoneType)
[22/Mar/2019 15:08:12] ERROR [django.request:118] Internal Server Error: /admin/django_messages/message/
Traceback (most recent call last):
File "/opt/site_env/lib/python3.6/site-packages/django/core/handlers/exception.py", line 35, in inner
response = get_response(request)
File "/opt/site_env/lib/python3.6/site-packages/django/core/handlers/base.py", line 128, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/site_env/lib/python3.6/site-packages/django/core/handlers/base.py", line 126, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/options.py", line 574, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/utils/decorators.py", line 142, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/sites.py", line 223, in inner
return view(request, *args, **kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/utils/decorators.py", line 62, in _wrapper
return bound_func(*args, **kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/utils/decorators.py", line 142, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "/opt/site_env/lib/python3.6/site-packages/django/utils/decorators.py", line 58, in bound_func
return func.__get__(self, type(self))(*args2, **kwargs2)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/options.py", line 1570, in changelist_view
cl = self.get_changelist_instance(request)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/options.py", line 705, in get_changelist_instance
self,
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/views/main.py", line 76, in __init__
self.queryset = self.get_queryset(request)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/views/main.py", line 320, in get_queryset
filters_use_distinct) = self.get_filters(request)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/views/main.py", line 130, in get_filters
self.model, self.model_admin, field_path=field_path
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/filters.py", line 157, in create
return list_filter_class(field, request, params, model, model_admin, field_path=field_path)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/filters.py", line 168, in __init__
self.lookup_choices = self.field_choices(field, request, model_admin)
File "/opt/site_env/lib/python3.6/site-packages/django/contrib/admin/filters.py", line 195, in field_choices
return field.get_choices(include_blank=False)
File "/opt/site_env/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 812, in get_choices
limit_choices_to)]
File "/opt/site_env/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 811, in <listcomp>
for x in rel_model._default_manager.complex_filter(
File "/opt/site_env/lib/python3.6/site-packages/django/utils/encoding.py", line 34, in smart_text
return force_text(s, encoding, strings_only, errors)
File "/opt/site_env/lib/python3.6/site-packages/django/utils/encoding.py", line 67, in force_text
s = str(s)
TypeError: __str__ returned non-string (type NoneType)
The biggest concern is that this is not the only model that is throwing the error. I have the same from django-allauth throwing the same error with SocialAccount.
Another concern is that I don't think I should be fixing the codes from this packages but I am left with nothing else.
Also to note, that none of the models for the application itself causes these error.
Only applies to django_allauth and django_messages. However, the failure doesn't happen in my development machine.
The mysql table for django_messages is empty (without records).
Is there a config that causes these kinds of errors or somethings.
I am using django 2.0 and python 3.6 in both enviroments.

Figured out.
It turns out that all the models failing are related to the User model whose str() method points to a field that has its database column as NULLs.
Solution: I have to make that field not nullable at the database level and at django level by assigning default value at the pre_save phase.
It worked.

KeyError: 'wizard_activation_wizard'

I recently upgraded from Django 1.6.11 to Django 1.8.9, and installed django-formtools==1.0 as part of the process. I am getting this error, which does not point to a single line of my code. Has anyone come across this?
Internal Server Error: /accounts/activation-process/maria-cristo-2/
Traceback (most recent call last):
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/formtools/wizard/views.py", line 237, in dispatch
response = super(WizardView, self).dispatch(request, *args, **kwargs)
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/django/views/generic/base.py", line 89, in dispatch
return handler(request, *args, **kwargs)
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/formtools/wizard/views.py", line 300, in post
return self.render_done(form, **kwargs)
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/formtools/wizard/views.py", line 358, in render_done
self.storage.reset()
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/formtools/wizard/storage/base.py", line 34, in reset
wizard_files = self.data[self.step_files_key]
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/formtools/wizard/storage/session.py", line 13, in _get_data
return self.request.session[self.prefix]
File "/home/vagrant/virtualenvs/staging/local/lib/python2.7/site-packages/django/contrib/sessions/backends/base.py", line 48, in __getitem__
return self._session[key]
KeyError: 'wizard_activation_wizard'

python oauthlib: in escape ValueError "Only unicode objects are escapable"

I'm using python-social-auth to login with social networks from my Django application. On my local machine everything works fine, but when I deploy to a server I get the following error:
oauthlib.oauth1.rfc5849.utils in escape
ValueError: Only unicode objects are escapable. Got None of type <type 'NoneType'>.
Stacktrace:
File "django/core/handlers/base.py", line 112, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "social/apps/django_app/utils.py", line 45, in wrapper
return func(request, backend, *args, **kwargs)
File "social/apps/django_app/views.py", line 12, in auth
return do_auth(request.social_strategy, redirect_name=REDIRECT_FIELD_NAME)
File "social/actions.py", line 25, in do_auth
return strategy.start()
File "social/strategies/base.py", line 66, in start
return self.redirect(self.backend.auth_url())
File "social/backends/oauth.py", line 99, in auth_url
token = self.set_unauthorized_token()
File "social/backends/oauth.py", line 158, in set_unauthorized_token
token = self.unauthorized_token()
File "social/backends/oauth.py", line 177, in unauthorized_token
method=self.REQUEST_TOKEN_METHOD)
File "social/backends/base.py", line 202, in request
response = request(method, url, *args, **kwargs)
File "requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "requests/sessions.py", line 349, in request
prep = self.prepare_request(req)
File "requests/sessions.py", line 287, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "requests/models.py", line 291, in prepare
self.prepare_auth(auth, url)
File "requests/models.py", line 470, in prepare_auth
r = auth(self)
File "requests_oauthlib/oauth1_auth.py", line 87, in __call__
unicode(r.url), unicode(r.method), None, r.headers)
File "oauthlib/oauth1/rfc5849/__init__.py", line 293, in sign
request.oauth_params.append(('oauth_signature', self.get_oauth_signature(request)))
File "oauthlib/oauth1/rfc5849/__init__.py", line 128, in get_oauth_signature
uri, headers, body = self._render(request)
File "oauthlib/oauth1/rfc5849/__init__.py", line 199, in _render
headers = parameters.prepare_headers(request.oauth_params, request.headers, realm=realm)
File "oauthlib/oauth1/rfc5849/utils.py", line 31, in wrapper
return target(params, *args, **kwargs)
File "oauthlib/oauth1/rfc5849/parameters.py", line 57, in prepare_headers
escaped_value = utils.escape(value)
File "oauthlib/oauth1/rfc5849/utilsy", line 56, in escape
'Got %s of type %s.' % (u, type(u)))
Some packages from requirements.txt file:
Django==1.6.6
google-api-python-client==1.1
oauth2==1.5.211
oauthlib==0.6.3
python-openid==2.2.5
python-social-auth==0.1.26
requests==2.4.0
requests-oauthlib==0.4.1
six==1.7.3

I got this error when I did not provide my OAuth Consumer key and secrets in my django settings file. It's essentially a "Required setting not found" error. Is it worth checking all your settings copied across successfully when you deployed?

I've upgraded to python-social-auth and had the same problem with Linkedin login. Parameter names in settings are changed in the latest version so instead:
LINKEDIN_CONSUMER_KEY and LINKEDIN_CONSUMER_SECRET it'd be used: SOCIAL_AUTH_LINKEDIN_KEY and SOCIAL_AUTH_LINKEDIN_SECRET.

the problem comes to you if this statement have spelling mistake:
tweepy.OAuthHandler(consumer_key,consumer_secret)