We Keep Coding

How can I solve the Not found problem when getting from pytest-django through pk?

I have a problem with django-pytest
I'm using, djnago-rest-framework
There is a problem testing the details. As shown in the code below, I entered the same details, detail1, detail2, and detail3 codes. However, only detail1 succeeds and detail2, detail3 indicates that '/api/v1/stats/1/' could not be found. It also occurs when implementing delete. I am curious about the cause and solution of this error.
enter image description here
// tests/test_apis.py
import json
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APITestCase
from stats.models import Stats
class StatsApiTests(APITestCase):
def setUp(self):
Stats.objects.get_or_create(blockshots=1, memo='test1')
Stats.objects.get_or_create(blockshots=2, memo='test2')
self.create_read_url = reverse('api:stats:stats-list')
self.read_update_delete_url = reverse('api:stats:stats-detail', kwargs={'pk': '1'})
def test_detail1(self):
response = self.client.get(self.read_update_delete_url)
data = json.loads(response.content)
content = {
'blockshots': 1,
'memo': 'test1',
}
self.assertEqual(data, content)
def test_detail2(self):
response = self.client.get(self.read_update_delete_url)
data = json.loads(response.content)
content = {
'blockshots': 1,
'memo': 'test1',
}
self.assertEqual(data, content)
def test_detail3(self):
response = self.client.get(self.read_update_delete_url)
data = json.loads(response.content)
content = {
'blockshots': 1,
'memo': 'test1',
}
self.assertEqual(data, content)
def test_list(self):
response = self.client.get(self.create_read_url)
self.assertContains(response, 'test1')
self.assertContains(response, 'test2')

Its hard to know what your actual implementation for read_update_delete_url, hence I assume it is looking up the resource by primary key. In that case, you can simply add the primary key in the url like this:
stat_one, _ = Stats.objects.get_or_create(blockshots=1, memo='test1')
stat_two, _ = Stats.objects.get_or_create(blockshots=2, memo='test2')
self.read_update_delete_url = reverse('api:stats:stats-detail', kwargs={'pk': stat_one.pk})
Basically, get_or_create returns the object and the state of the object (created or not). You can use the object's id as the parameter of reverse function.

Flask App Route returning "Bad Request" error

I'm learning full stack in Flask and am having trouble with a particular route from an API. The API being developed is a list of books and in particular I am trying to reach the data for a particular book, say book with ID = 8. The URI is http://127.0.0.1:5000/books/8. However this returns a 400 error (bad request).
I really can't spot what is going wrong. I have defined the route '/books/int:book_id' with methods GET and PATCH, so I would expect the route to work. I also see errors when I test the route with curl, for example:
curl -X PATCH -H "Content-Type: application/json" -d '{"rating":"1"}' http://127.0.0.1:5000/books/8
See below for the particular route in question:
#app.route('/books/<int:book_id>', methods=['GET', 'PATCH'])
def update_book_rating(book_id):
body = request.get_json()
try:
book = Book.query.filter_by(Book.id==book_id).one_or_none()
if book is None:
abort(404)
if 'rating' in body:
book.rating = int(body.get('rating'))
book.update() #Class book in models.py has an update method which executes a commit()
return jsonify({
'success': True,
'id': book.id
})
except Exception as e:
print(e)
abort(400)
If it helps, I am also adding the full code. Note that the Book object is defined in a separate file, which I won't put here.
import os
from flask import Flask, request, abort, jsonify
from flask_sqlalchemy import SQLAlchemy # , or_
from flask_cors import CORS
import random
from models import setup_db, Book
BOOKS_PER_SHELF = 8
# #TODO: General Instructions
# - As you're creating endpoints, define them and then search for 'TODO' within the frontend to update the endpoints there.
# If you do not update the endpoints, the lab will not work - of no fault of your API code!
# - Make sure for each route that you're thinking through when to abort and with which kind of error
# - If you change any of the response body keys, make sure you update the frontend to correspond.
def paginate_books(request, selection):
page = request.args.get('page', 1, type=int)
start = (page - 1) * BOOKS_PER_SHELF
end = start + BOOKS_PER_SHELF
books = [book.format() for book in selection]
current_books = books[start:end]
return current_books
def create_app(test_config=None):
# create and configure the app
app = Flask(__name__)
setup_db(app)
CORS(app)
# CORS Headers
#app.after_request
def after_request(response):
response.headers.add("Access-Control-Allow-Headers", "Content-Type,Authorization,true")
response.headers.add("Access-Control-Allow-Methods", "GET,PUT,PATCH,POST,DELETE,OPTIONS")
return response
# #TODO: Write a route that retrivies all books, paginated.
# You can use the constant above to paginate by eight books.
# If you decide to change the number of books per page,
# update the frontend to handle additional books in the styling and pagination
# Response body keys: 'success', 'books' and 'total_books'
# TEST: When completed, the webpage will display books including title, author, and rating shown as stars
#app.route('/books', methods=['GET'])
def get_books():
selection = Book.query.order_by(Book.id).all()
current_books = paginate_books(request, selection)
if len(current_books) == 0:
abort(404)
return jsonify({
'success': True,
'books': current_books,
'total_books': len(Book.query.all())
})
# #TODO: Write a route that will update a single book's rating.
# It should only be able to update the rating, not the entire representation
# and should follow API design principles regarding method and route.
# Response body keys: 'success'
# TEST: When completed, you will be able to click on stars to update a book's rating and it will persist after refresh
#app.route('/books/<int:book_id>', methods=['GET', 'PATCH'])
def update_book_rating(book_id):
body = request.get_json()
try:
book = Book.query.filter_by(Book.id==book_id).one_or_none()
if book is None:
abort(404)
if 'rating' in body:
book.rating = int(body.get('rating'))
book.update() #Class book in models.py has an update method which executes a commit()
return jsonify({
'success': True,
'id': book.id
})
except Exception as e:
print(e)
abort(400)
# #TODO: Write a route that will delete a single book.
# Response body keys: 'success', 'deleted'(id of deleted book), 'books' and 'total_books'
# Response body keys: 'success', 'books' and 'total_books'
#app.route('/delete/<int:book_id>', methods=['DELETE'])
def delete_book(book_id):
try:
book = Book.query.filter_by(Book.id==book_id).one_or_none()
if book is None:
abort(404)
book.delete()
selection = Book.query.order_by(Book.id).all()
current_books = paginate_books(request, selection)
return jsonify({
'success': True,
'deleted': book_id,
'books': current_books,
'total_books': len(Book.query.all())
})
except:
abort(422)
# TEST: When completed, you will be able to delete a single book by clicking on the trashcan.
# #TODO: Write a route that create a new book.
# Response body keys: 'success', 'created'(id of created book), 'books' and 'total_books'
# TEST: When completed, you will be able to a new book using the form. Try doing so from the last page of books.
# Your new book should show up immediately after you submit it at the end of the page.
#app.route('/books', methods=['POST'])
def create_book():
body = request.get_json()
new_title = body.get('title', None)
new_author = body.get('author', None)
new_rating = body.get('rating', None)
try:
book = Book(title=new_title, author=new_author, rating=new_rating)
book.insert()
selection = Book.query.order_by(Book.id).all()
current_books = paginate_books(request, selection)
return jsonify({
'success': True,
'created': book.id,
'books': current_books,
'total_books': len(Book.query.all())
})
except:
abort(422)
#app.errorhandler(400)
def bad_request(error):
return jsonify({
'success': False,
'error': 400,
'message': 'Server cannot or will not process the request due to client error (for example, malformed request syntax, invalid request message framing, or deceptive request routing).'
}), 400
#app.errorhandler(404)
def not_found(error):
return jsonify({
'success': False,
'error': 404,
'message': 'resource not found'
}), 404
#app.errorhandler(405)
def not_found(error):
return jsonify({
'success': False,
'error': 405,
'message': 'method not allowed'
}), 405
#app.errorhandler(422)
def unprocessable(error):
return jsonify({
'success': False,
'error': 422,
'message': 'unprocessable'
}), 422
return app

When using django.test.client.post I get Bad Request

I use pytest-django to do unit test on my django project. The view is
def news(request):
"""
Interface for newslist
"""
page = 1
if request.method == 'POST':
content = request.body
try:
content = json.loads(content)
except ValueError as error:
return err_response("Value Error of post: {}".format(error))
if 'page' in content:
page = content['page']
articlelist = Article.objects.all().order_by('-time')
paginator = Paginator(articlelist, 10)
try:
current_list = paginator.page(page)
except InvalidPage as error:
return err_response(error)
# coping with the paginator
...
newsnum = len(Article.objects.all())
return JsonResponse({
'newsnum': newsnum,
'pagelist': list(pagelist),
'data': [{
'title': newsitem.title,
'source': newsitem.source,
'time': newsitem.time.strftime("%Y-%m-%d %H:%M:%S"),
'content': newsitem.content,
'href': newsitem.href,
'image': newsitem.image,
} for newsitem in current_list]
}, status=200)
When I use pytest-django to test it
#pytest.mark.django_db
def test_view_news(client):
"""
Test view news
"""
url = reverse("news")
data = {
'page': 1
}
response = client.post(url, data=data)
assert response.status_code == 200
It gives Bad Request and code 400. But when I use client.get(), the response is normal (code 200).In the settings, I already set
DEBUG = True
ALLOWED_HOSTS = ['*']
Can anyone tells me what happened?

Override the default server_name used by client
Just override the default server name used by client to match one of those you have in allowed hosts e.g
ALLOWED_HOSTS = ['localhost',...]
...
.....
response = client.post(url, data, SERVER_NAME='localhost')

Middleware logging limitation Django 1.11

Background:
I have integration test which is working fine, but later on failed when I
added the customized middleware
def test_mobile_update_customer(self):
user = User.objects.create(username='warhead')
self.client.force_authenticate(user=user)
mommy.make(Customer, family_name='IBM', created_user=self.soken_staff, updated_user=self.soken_staff)
data = {
"family_name": "C0D1UM"
}
customer = Customer.objects.first()
res = self.client.patch(reverse('api:customer-detail', kwargs={'pk': customer.id}), data=data)
self.assertEqual(200, res.status_code)
customer.refresh_from_db()
self.assertEqual('C0D1UM', customer.family_name)
self.assertEqual('spearhead', customer.created_user.username)
self.assertEqual('warhead', customer.updated_user.username)
Problem:
The middleware break it with Exception
File "/Users/el/Code/norak-cutter/soken/soken-web/soken_web/middleware.py", line 47, in process_request
data['PATCH'] = json.loads(request.body)
File "/Users/el/.pyenv/versions/soken/lib/python3.6/site-packages/django/http/request.py", line 264, in body
raise RawPostDataException("You cannot access body after reading from request's data stream")
django.http.request.RawPostDataException: You cannot access body after reading from request's data stream
The problem is data has been read before the RESTful api do the job.
Then the program raises an exception.
def process_request(self, request):
if request.path.startswith('/api/'):
data = collections.OrderedDict()
data["user"] = request.user.username
data["path"] = request.path
data["method"] = request.method
data["content-type"] = request.content_type
if request.method == 'GET':
data['GET'] = request.GET
elif request.method == 'POST':
data['POST'] = request.POST
# https://stackoverflow.com/questions/4994789/django-where-are-the-params-stored-on-a-put-delete-request
# elif request.method == 'PUT':
# data['PUT'] = json.loads(request.body)
# test_mobile_update_customer
# raise RawPostDataException("You cannot access body after reading from request's data stream")
# django.http.request.RawPostDataException: You cannot access body after reading from request's data stream
# elif request.method == 'PATCH':
# data['PATCH'] = json.loads(request.body)
elif request.method == 'DELETE':
pass
self.__uuid = str(uuid.uuid4())
request_logger.info(f'{self.__uuid} {json.dumps(data)}')
Update2
Attempt:
I change client constructor refer to https://github.com/encode/django-rest-framework/issues/2774
def test_mobile_update_customer(self):
user = User.objects.create(username='warhead')
# self.client.force_authenticate(user=user)
from django.test import Client
client = Client()
client.force_login(user)
mommy.make(Customer, family_name='IBM', created_user=self.soken_staff, updated_user=self.soken_staff)
data = {
"family_name": "C0D1UM"
}
customer = Customer.objects.first()
res = client.patch(reverse('api:customer-detail', kwargs={'pk': customer.id}), data=data, content_type='application/json')
self.assertEqual(200, res.status_code)
customer.refresh_from_db()
self.assertEqual('C0D1UM', customer.family_name)
self.assertEqual('spearhead', customer.created_user.username)
self.assertEqual('warhead', customer.updated_user.username)
It does not work. The Client misinterpret the payload
>>> res.content
b'{"detail":"JSON parse error - Expecting property name enclosed in double quotes: line 1 column 2 (char 1)"}'
Workaround:
I don't know this is correct for all cases or not, but I workaround by this
test.py
https://gist.github.com/elcolie/88eb7c90cfca1c369a020ac232c7fbcc
middleware.py
https://gist.github.com/elcolie/5d9b1a2f890a0efcb46fdb95c0e90908
result.py
https://gist.github.com/elcolie/298e595b404c1a5839ed8dd584d2f07f
Question:
How do I do integration test of PATCH with the same time that testcase will not break by my middleware?
Right now I have to choose either one of them.

Unit Testing Bootstrap Modal Redirect

I've been unable resolve this issue on IRC, hoping I could find some guidance here. I have the following test:
def test_validation_errors_return_hops_list_page(self):
response = self.client.post(
'/beerdb/add/hops',
data={
'name': '',
'min_alpha_acid': '',
'max_alpha_acid': '',
'country': '',
'comments': ''
}, follow=True
)
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, 'homebrewdatabase/addhops.html')
name_validation_error = escape("A hop name is required")
min_alpha_acid_error = escape("You must enter a min alpha acid")
max_alpha_acid_error = escape("You must enter a max alpha acid")
country_error = escape("You must enter a country")
comments_error = escape("You must enter a comment")
self.assertContains(response, name_validation_error)
self.assertContains(response, min_alpha_acid_error)
self.assertContains(response, max_alpha_acid_error)
self.assertContains(response,country_error)
self.assertContains(response, comments_error)
It's failing on self.assertContains(response, name_validation_error). Here's the trace back:
Failure
Traceback (most recent call last):
File "/Users/USER/workspace/PycharmProjects/hashtagbrews/homebrewdatabase/tests/test_views.py", line 189, in test_validation_errors_return_hops_list_page
self.assertContains(response, name_validation_error)
File "/Users/USER/workspace/Envs/hashtagbrews/lib/python3.4/site-packages/django/test/testcases.py", line 398, in assertContains
msg_prefix + "Couldn't find %s in response" % text_repr)
AssertionError: False is not true : Couldn't find 'A hop name is required' in response
My view in views.py renders the hops.html template with errors when the form is invalid:
def addhops(request):
add_form = HopForm(request.POST or None)
if request.method == 'POST':
if add_form.is_valid():
Hop.objects.create(name=request.POST['name'],
min_alpha_acid=request.POST['min_alpha_acid'],
max_alpha_acid=request.POST['max_alpha_acid'],
country=request.POST['country'],
comments=request.POST['comments']
)
return redirect('hops_list')
else:
hops_list = Hop.objects.all()
return render(request, 'homebrewdatabase/hops.html', {'hops': hops_list, 'form': add_form})
return render(request, 'homebrewdatabase/addhops.html', {'form': add_form})
When I manually click through the site, I get exactly what I'm looking for: a redirect from the modal to the main hops page list with a Bootstrap alert box at the top containing an unordered list of errors from add_hops.errors. I've printed out the response after the post request (self.client.post('url', data={invalid data})) and it only contains the modal form. What would be the proper method to finish this test? Or do I need to rewrite my form validation?

The issue here, as identified in the comments, is that the Django test client runs a GET request on the addhops view method after executing the post request. According to the view logic, if the method isn't POST, then it returns the bootstrap modal, which does not contain the form errors by design. So the test will fail when using the test client. However, the test can be altered to, instead, use the HttpRequest object to send invalid data in a POST request and then assert that the content contains the form errors. So I rewrote the test to use the following, which passes--
def test_validation_errors_return_hops_list_page(self):
request = HttpRequest()
request.method = 'POST'
request.POST['name'] = ''
request.POST['min_alpha_acid'] = ''
request.POST['max_alpha_acid'] = ''
request.POST['country'] = 'USA'
request.POST['comments'] = ''
response = addhops(request)
self.assertEqual(response.status_code, 200)
name_validation_error = escape("A hop name is required")
min_alpha_acid_error = escape("You must enter a min alpha acid")
max_alpha_acid_error = escape("You must enter a max alpha acid")
comments_error = escape("You must enter a comment")
self.assertContains(response, name_validation_error)
self.assertContains(response, min_alpha_acid_error)
self.assertContains(response, max_alpha_acid_error)
self.assertContains(response, comments_error)
I cannot assert which template was used as assertTemplateUsed is a method that belongs to the test client, but my functional tests with selenium should be enough to check that the required elements are in the rendered view. This may have to be changed in the future, but for now, it's enough to work with.