I'm running into weird behavior when writing nested structures with django-rest and then trying to test them using django-rest's test client. The nested child object should be optional.
Here's a sample serializer:
from rest_framework import serializers
class OptionalChildSerializer(serializers.Serializer):
field_b = serializers.IntegerField()
field_c = serializers.IntegerField()
class Meta:
fields = ('field_b', 'field_c', )
class ParentSerializer(serializers.Serializer):
field_a = serializers.IntegerField()
child = OptionalChildSerializer(required=False, many=False)
class Meta:
fields = ('a', 'child',)
def perform_create(self, serializer):
# TODO: create nested object.
pass
(I've omitted the code in perform_create, as it's not relevant to the question).
Now, passing a normal dict as data argument works just fine:
ser = ParentSerializer(data=dict(field_a=3))
ser.is_valid(raise_exception=True)
But passing a QueryDict instead will fail:
from django.http import QueryDict
ser = ParentSerializer(data=QueryDict("field_a=3"))
ser.is_valid(raise_exception=True)
ValidationError: {'child': {'field_b': [u'This field is required.'], 'field_c': [u'This field is required.']}}
On the actual web site, the API gets a normal dict and thus works fine. During testing however, using something like client.post('url', data=dict(field_a=3)) will result in a QueryDict being passed to the view, and hence not work.
So my question is: what's the difference between the QueryDict and normal dict? Or am I approaching this the wrong way?
DRF defines multiple parser classes for parsing the request content having different media types.
request.data will normally be a QueryDict or a normal dictionary depending on the parser used to parse the request content.
JSONParser:
It parses the JSON request content i.e. content with .media_type as application/json.
FormParser
It parses the HTML form content. Here, request.data is populated with a QueryDict of data. These have .media_type as application/x-www-form-urlencoded.
MultiPartParser
It parses multipart HTML form content, which supports file uploads. Here also, both request.data is populated with a QueryDict. These have
.media_type as multipart/form-data.
FileUploadParser
It parses raw file upload content. The request.data property is a dictionary with a single key file containing the uploaded file.
How does DRF determines the parser?
When DRF accesses the request.data, it examines the Content-Type header on the incoming request and then determines which parser to use to parse the request content.
You will need to set the Content-Type header when sending the data otherwise it will use either a multipart or a form parser to parse the request content and give you a QueryDict in request.data instead of a dictionary.
As per DRF docs,
If you don't set the content type, most clients will default to using
'application/x-www-form-urlencoded', which may not be what you wanted.
So when sending json encoded data, also set the Content-Type header to application/json and then it will work as expected.
Why request.data is sometimes QueryDict and sometimes dict?
This is done because different encodings have different datastructures and properties.
For example, form data is an encoding that supports multiple keys of the same value, whereas json does not support that.
Also in case of JSON data, request.DATA might not be a dict at all, it could be a list or any of the other json primitives.
Check out this Google Groups thread about the same.
What you need to do?
You can add format='json' in the tests when POSTing the data which will set the content-type as well as serialize the data correctly.
client.post('url', format='json', data=dict(field_a=3))
You can also send JSON-encoded content with content-type argument.
client.post('url', json.dumps(dict(field_a=3)), content_type='application/json')
The django-rest test client doesn't automatically serialize data as json, but uses multipart/form, which results in a QueryDict.
There is, however, a format option, described in the docs. The following test code works fine:
client.post('url', format='json', data=dict(field_a=3))
I'm still puzzled on the different serializer behavior between a normal dict and a QueryDict, though...
Thanks Rajesh for pointing me in the right direction!
Related
I have a model with API for an ImageField. I need to send image fetched via post method on the template and send it via post request to the API created. The image fetched has a type InMemoryUploadedFile, if I try to send it directly, I get 406 because of failed serializer validation. So I tried making a PIL object out of it and tried sending. I checked the JS counterpart of the code and it just takes the file from the input field and sends it directly to the same field and it works.
Is there a way I can just send an image file object via post request to fail serializer validation.
category_thumbnail = request.FILES.get('category_thumbnail')
category_thumbnail = Image.open(category_thumbnail)
data = {
'category_thumbnail': category_thumbnail
}
This would give me 406.
I also tried converting image string to a base64 byte object.
category_thumbnail = request.FILES.get('category_thumbnail')
category_thumbnail = Image.open(category_thumbnail)
with io.BytesIO() as output:
category_thumbnail.save(output, format="GIF")
contents = output.getvalue()
category_thumbnail = base64.b64decode(contents)
data = {
'category_thumbnail': category_thumbnail
}
but this would give me 406 too.
I wonder if there's a way I can just send the image file object taken from InMemoryUploadedFile.
Also tried
category_thumbnail = request.FILES.get('category_thumbnail').file
if the code is in API, you should use request.data instead of request.FILES
if the code is in normal html view, like a form. then it should be using an HTML form with enctype="multipart/form-data" or FormData generated by javascripts.
And in views:
obj.category_thumbnail = request.FILES['category_thumbnail']
it should be used directly with rest_framework, no matter it is InMemoryUploadedFile or UploadedFile.
If, for some reason, you are using serializer rather than form in view.
Then the view should be derived from APIView if class based. Or use #api_view decorator to function based view.
I have a JSON string that I compute from a Pandas dataframe
aggr.aggregated.to_json(orient='values')
I cannot directly provide aggr.aggregated to a standard Python JSON serializer because it would not follow the orient='values' rules and would do so differently.
I want to serve my own JSON string as a response from a Django view:
return JsonResponse(aggr.aggregated.to_json(orient='values'))
However, in the code above, Django tried to serialize my JSON string.
How can I use JsonResponse exclusively to set the Content-Type header to application/json but not to serialize a string that is already serialized?
There is no benefit in using JsonResponse if you don't want it to encode the JSON for you.
Just use HttpResponse and set the content-type header yourself:
return HttpResponse(
aggr.aggregated.to_json(orient='values'),
content_type='application/json'
)
I'm using Django==1.10.5 and djangorestframework==3.5.3.
I have a few ModelViewSets that handle JSON API requests properly. Now, I'd like to use TemplateRenderer to add HTML rendering to those same ModelViewSets. I started with the list endpoint, and created a simple template that lists the available objects. I implemented the get_template_names to return the template I created.
Accessing that endpoint through the browser works fine when there are no objects to list, so everything related to setting up HTML renderers alongside APIs seems to work.However, when tere are objects to return the same endpoint fails with the following error:
ValueError: dictionary update sequence element #0 has length XX; 2 is required
Where XX is the number of attributes the object has.
This documentation section suggests the view function should act slightly differently when returning an HTML Response object, but I assume this is done by DRF's builtin views when necessary, so I don't think that's the issue.
This stackoverflow Q/A also looks relevant but I'm not quite sure it's the right solution to my problem.
How can I make my ModelViewSets work with both HTML and JSON renderers?
Thanks!
DRF has a brief explanation of how to do this in their documentation.
I think you'd do something like this...
On the client side, tell your endpoint what type of response you want:
fetch(yourAPIUrl, {
headers: {
'Accept': 'application/json'
// or 'Accept': 'text/html'
}
})
In your view, just check that and act accordingly:
class FlexibleAPIView(APIView):
"""
API view that can render either JSON or HTML.
"""
renderer_classes = [TemplateHTMLRenderer, JSONRenderer]
def get(self, request, *args, **kwargs):
queryset = Things.objects.all()
# If client wants HTML, give them HTML.
if request.accepted_renderer.format == 'html':
return Response({'things': queryset}, template_name='example.html')
# Otherwise, the client likely wants JSON so serialize the data.
serializer = ThingSerializer(instance=queryset)
data = serializer.data
return Response(data)
I have a Django application which must have the following behavior: if a request has no Content-Type header, it returns an error response.
In order to test this behavior, I need to make an HTTP request without a Content-Type header.
I am using the Client class in the django.test module. This has many methods, including this one:
post(path, data=None, content_type=MULTIPART_CONTENT, follow=False, secure=False, **extra)
Makes a POST request on the provided path and returns a Response object, which is documented below.
[...]
If you provide content_type (e.g. text/xml for an XML payload), the contents of data will be sent as-is in the POST request, using content_type in the HTTP Content-Type header.
If you don’t provide a value for content_type, the values in data will be transmitted with a content type of multipart/form-data. In this case, the key-value pairs in data will be encoded as a multipart message and used to create the POST data payload.
The documentation says that a Content-Type header is always set on the request, irrespective of whether I pass a content_type argument.
So what other ways do I have to construct a request, such that it does not have a Content-Type header?
You can build a customised request instance through the class RequestFactory.
Once generated, you can modify the request instance before passing it to the view.
Using the example in the RequestFactory documentation page as a starting point, you can do:
from django.test import TestCase, RequestFactory
from .views import my_view
class SimpleTest(TestCase):
def setUp(self):
# Every test needs access to the request factory.
self.factory = RequestFactory()
def test_details(self):
# Create an instance of a POST request.
request = self.factory.post('/your/url', data={'your':'data'})
# NOW you can customise your request instance!
# (i.e. remove the Content-Type header)
request.META.pop('CONTENT_TYPE', None)
# Actually use the request to test my_view()
# as if it were deployed at /customer/details
response = my_view(request)
self.assertEqual(response.status_code, 400)
The request.META is just a standard Python dictionary (as explained here), so you can use
del request.META['CONTENT_TYPE']
instead of pop() to remove it, but only if you are deadly sure that the key will be in the dictionary.
I know this is several years old, but I had the same question and found the real answer, i.e. how to do this with the test client:
client.get(url, content_type=None)
At least on Django 2.0, that makes a request with no content type header.
I just set up an apache server with django, and to test it, made a very simple function in views.py
channel = rabbit_connection()
#csrf_protect
#csrf_exempt
def index(request):
data={'text': 'Food truck is awesome! ', 'email': 'bob#yahoo.com', 'name': 'Bob'}
callback(json.dumps(data))
context = RequestContext(request)
return render_to_response('index.html', context_instance=context)
This function works fine if I send a GET or POST request to the server. However I would like to get this data from POST request. Assuming I send request like this:
import pycurl
import simplejson as json
data = json.dumps({'name':'Bob', 'email':'bob#yahoo.com', 'text': u"Food truck is awesome!"})
c = pycurl.Curl()
c.setopt(c.URL, 'http://ec2-54-......compute-1.amazonaws.com/index.html')
c.setopt(c.POSTFIELDS, data)
c.setopt(c.VERBOSE, True)
for i in range(100):
c.perform()
What I would like to have in the view is something like this:
if request.method == 'POST':
data = ?????? # Something that will return me my dictionary
Just in case:
It is always will be in JSON format and the fields are unknown.
data= request.POST.get('data','')
Will return you a single value (key=data) from your dictionary. If you want the entire dictionary, you simply use request.POST. You are using the QueryDict class here:
In an HttpRequest object, the GET and POST attributes are instances of django.http.QueryDict. QueryDict is a dictionary-like class customized to deal with multiple values for the same key. This is necessary because some HTML form elements, notably , pass multiple values for the same key.
QueryDict instances are immutable, unless you create a copy() of them. That means you can’t change attributes of request.POST and request.GET directly.
-Django Docs
If the data posted is in JSON format, you need to deserialize it:
import simplejson
myDict = simplejson.loads(request.POST.get('data'))