In my tests, I send mock data of models that I've passed through the serializer. The serializer.data looks something like this
{
"field": None
}
However, the data that my API receives is formatted like
{
"field": "None"
}
which is a problem because I'm trying to specify a foreign key that is allowed to be null. Shouldn't the APIClient convert None into null instead of unicode?
Is there any way to fix this or get around it?
Here's my serializer
class MyModelSerializer(serializers.ModelSerializer):
field = serializers.PrimaryKeyRelatedField(
queryset=OtherModel.objects.all(), required=False, allow_null=True)
And my create method in a viewset
def create(self, request):
model = MyModel()
serializer = MyModelSerializer(model, data=request.data)
if serializer.is_valid():
serializer.save(owner=request.user)
return Response(serializer.data, status=201)
return Response(serializer.errors, status=406)
Also my model class
class MyModel(models.Model):
field= models.OneToOneField(
OtherModel, blank=True, null=True)
In addition to what Kevin already said, you can force the APIClient to send JSON using the parameter format='json'.
See the documentation.
The problem here is that the APIClient is sending data to the view as form-data by default, which doesn't have a concept of None or null, so it is converted to the unicode string None.
The good news is that Django REST framework will coerce a blank string to None for relational fields for this very reason. Alternatively, you can use JSON and actually send None or null, which should work without issues.
In addition to existing answers,
if you are expecting a null, this probably means you expect your api to receive json.
If that's the case, you may want to configure the test request default format to json instead of form-data:
In your setting.py:
REST_FRAMEWORK = {
...
'TEST_REQUEST_DEFAULT_FORMAT': 'json'
}
This way, no need to add format='json' to each request
Related
I am deserializing JSON data from file to a model object. I'm using a Django Rest Framework serializer.
The JSON is quite large and I expected some fields that needed tweaking. However, the error messages are not helpful, contrary to those in the docs.
The serializer:
class ChannelSerializer(serializers.ModelSerializer):
class Meta:
model = Channel
fields = '__all__'
The test:
class UnitTests(unittest.TestCase):
def testDeserializeChannel(self):
json = Path(Dirs.CHANNELS_METADATA_TEST, 'A SIMULATED REALITY/youtube/A SIMULATED REALITY.json')
with open(json, 'rb') as file:
stream = io.BytesIO(file.read())
parsed_channel = JSONParser().parse(stream)
serializer = ChannelSerializer(data=parsed_channel, many=True)
if not serializer.is_valid(): # required for serializer.save() to work
print(serializer.error_messages, "\n")
print(serializer.errors)
channel = serializer.validated_data.save()
self.assertEqual(channel[0].title, "A SIMULATED REALITY")
self.assertEqual(channel[0].original_url, "https://www.youtube.com/channel/UCBT3VPJlhJHeIvmDadeZB5w")
The error messages:
{'required': 'This field is required.', 'null': 'This field may not be null.', 'not_a_list': 'Expected a list of items but got type "{input_type}".', 'empty': 'This list may not be empty.', 'max_length': 'Ensure this field has no more than {max_length} elements.', 'min_length': 'Ensure this field has at least {min_length} elements.'}
{'non_field_errors': [ErrorDetail(string='Expected a list of items but got type "dict".', code='not_a_list')]}
We can see required and null errors but without mentioning of the fields.
I suspect the 'all' bit in the serializer is the cause, contrary to explicitly defined field. Can somebody comfirm? And more importantly, can it be fixed?
Kr,
You can use DRF's APIException to create your custom exceptions:
from rest_framework.exceptions import APIException
class MyCustomException(APIException):
status_code = 400
default_detail = 'This serializer is not valid my friend...'
default_code = 'bad_request'
Usage:
. . .
if not serializer.is_valid():
raise MyCustomException()
. . .
Outcome if serializer is not valid:
"detail": {"This serializer is not valid my friend..."}
Like this you can use custom exceptions all around your code. :)
I have a Twitter authentication view that doesn't use a viewset so the auth can be handled on the backend. The view takes in the oauth_token & uses Twython to get the profile & create a Twitter model.
Currently I just return status 201 on success, but to alleviate the need for another request after creation, I'd like to return the created model. I have a TwitterSerializer already which defines the fields that I want to include, so I'd like to be able to reuse this if possible.
TwitterSerializer
class TwitterSerializer(serializers.ModelSerializer):
class Meta:
model = Twitter
fields = (
"id",
"twitter_user_id",
"screen_name",
"display_name",
"profile_image_url",
)
When I try to use this, I get the error that Instance of TwitterSerializer is not JSON serializable.
serialized = TwitterSerializer(instance=twitter)
return JsonResponse({ "created": serialized })
I could return a serialized instance of the model using serializers.serialize()
serialized = serializers.serialize('json', [twitter, ])
serialized = serialized[0]
return JsonResponse({ "created": serialized })
I could pass the fields kwarg to serialize() but I don't want to have to repeat myself if I don't have to. So would it be possible to re-use my TwitterSerializer in this case? I'm having trouble finding a direct answer since most docs assume you'll be using a ViewSet when using serializerss understandably, and this feels like an edge case. I'm open to suggestions for refactoring this approach as well!
After serialization, you can get your data using data attribute of serializer like this.
serialized = TwitterSerializer(instance=twitter)
return JsonResponse({ "created": serialized.data })
You should use Django rest Response instead of JsonResponse like this
from rest_framework response
serialized = TwitterSerializer(instance=twitter)
return response.Response({ "created": serialized.data })
BACKGROUND
I have two serializers: PostSerializer and PostImageSerializer which both inherit DRF ModelSerializer. The PostImage model is linked with Post by related_name='photos'.
Since I want the serializer to perform update, PostSerializer overrides update() method from ModelSerializer as stated in official DRF doc.
class PostSerializer(serializers.ModelSerializer):
photos = PostImageSerializer(many=True)
class Meta:
model = Post
fields = ('title', 'content')
def update(self, instance, validated_data):
photos_data = validated_data.pop('photos')
for photo in photos_data:
PostImage.objects.create(post=instance, image=photo)
return super(PostSerializer, self).update(instance, validated_data)
class PostImageSerializer(serializer.ModelSerializer):
class Meta:
model = PostImage
fields = ('image', 'post')
I have also defined a ViewSet which inherits ModelViewSet.
class PostViewSet(viewsets.ModelViewSet):
queryset = Post.objects.all()
serializer_class = PostSerializer
Finally the PostViewSet is registered to DefaultRouter. (Omitted code)
Goal
The goals are simple.
Send PUT request via PostMan with url like 'PUT http://localhost:8000/api/posts/1/'
Since Image files should be included, request would be done by form-data like below.
Problem
I'm getting 400 Response with error message as following.
{
"photos": [
"This field is required."
],
"title": [
"This field is required."
],
"content": [
"This field is required."
]
}
(Should you plz note that the error messages might not exactly fit with DRF error messages since they are translated.)
It is obvious that none of my PUT fields are applied.
So I have been digging around Django rest framework source code itself and found out serializer validation in ViewSet update() method continues to fail.
I doubt that because I PUT request not by JSON but by form-data using key-value pair so request.data is not properly validated.
However, I should contain multiple images in the request which means plain JSON would not work.
What would be the most clear solutions for this case?
Thank you.
Update
As Neil pointed out, I added print(self) at the first line of update() method of PostSerializer. However nothing printed out on my console.
I think this is due to my doupt above because perform_update() method which calls serializer update() method is called AFTER serializer is validated.
Therefore the main concept of my question could be narrowed to the followings.
How should I fix requested data fields so that validation inside update() method of ModelViewSet could pass?
Do I have to override update() method of ModelViewSet(not the one from ModelSerializer)?
Thanks again.
First of all you need to set header:
Content-Type: multipart/form-data;
But maybe if you set form-data in postman, this header should be
default.
You can't send images as a json data (unless you encode it to string and decode on server side to image eg. base64).
In DRF PUT by default requires all fields. If you want to set only partial fields you need to use PATCH.
To get around this and use PUT to update partial fields you have two options:
edit update method in viewset to partial update serializer
edit router to always call partial_update method in serializers which is more advanced
You can override viewset update method to always update serializer partial (changing only provided fields):
def update(self, request, *args, **kwargs):
partial = True # Here I change partial to True
instance = self.get_object()
serializer = self.get_serializer(instance, data=request.data, partial=partial)
serializer.is_valid(raise_exception=True)
self.perform_update(serializer)
return Response(serializer.data)
Add
rest_framework.parsers.MultiPartParser
to the main settings file to the REST_FRAMEWORK dict:
REST_FRAMEWORK = {
...
'DEFAULT_PARSER_CLASSES': (
'rest_framework.parsers.JSONParser',
'rest_framework.parsers.MultiPartParser',
)
}
Looking at your serializers it's weird that you don't get error from PostSerializer because you don't add "photos" field to Meta.fields tuple.
More advices from me in this case:
add required=False to your photos field (unless you want this to be required)
as wrote above add photos field to you Meta.fields tuple fields = ('title', 'content', 'photos',)
add default None value for your validated_data.pop('photos'), then check photos data is provided before loop.
The solution is somewhat a mixture or #Neil and #mon's answers. However I'll straighten out a bit more.
Analysis
Right now Postman submits form data which contains 2 key-value pairs(Please refer to the photo which I uploaded in my original question). One is "photos" key field linked with multiple photo files and the other is "data" key field linked with one big chunk of 'JSON-like string'. Although this is a fair method POSTing or PUTting data along with files, DRF MultiPartParser or JSONParser won't parse these properly.
The reason why I got the error message was simple. self.get_serializer(instance, data=request.data, partial=partial method inside ModelViewSet(especially UpdateModelMixin) couldn't understand request.data part.
Currently request.data from submitted form data looks like below.
<QueryDict: { "photos": [PhotoObject1, PhotoObject2, ... ],
"request": ["{'\n 'title': 'title test', \n 'content': 'content test'}",]
}>
Watch the "request" part carefully. The value is a plain string object.
However my PostSerializer expects the request.data to look something like below.
{ "photos": [{"image": ImageObject1, "post":1}, {"image": ImageObject2, "post":2}, ... ],
"title": "test title",
"content": "test content"
}
Therefore, let's do some experiment and PUT some data in accordance with above JSON form.
i.e
{ "photos": [{"image": "http://tny.im/gMU", "post": 1}],
"title" : "test title",
"content": "test content"
}
You'll get an error message as following.
"photos": [{"image": ["submitted data is not a file."]}]
Which means every data is submitted properly but the image url http://tny.im/gMU is not a file but string.
Now the reason of this whole problem became clear. It is the Parser which needs to be fixed so that the Serializer could understand submitted form data.
Solution
1. Write new parser
New parser should parse 'JSON-like' string to proper JSON data. I've borrowed the MultipartJSONParser from here.
What this parser does is simple. If we submit 'JSON-like' string with the key 'data', call json from rest_framework and parse it. After that, return the parsed JSON with requested files.
class MultipartJsonParser(parsers.MultiPartParser):
# https://stackoverflow.com/a/50514022/8897256
def parse(self, stream, media_type=None, parser_context=None):
result = super().parse(
stream,
media_type=media_type,
parser_context=parser_context
)
data = {}
data = json.loads(result.data["data"])
qdict = QueryDict('', mutable=True)
qdict.update(data)
return parsers.DataAndFiles(qdict, result.files)
2. Redesign Serializer
Official DRF doc suggests nested serializers to update or create related objects. However we have a significant drawback that InMemoryFileObject cannot be translated into a proper form which the serializer expects. To do this, we should
Override updatemethod of ModelViewSet
Pop out 'photos' key-value pair from request.data
Translate popped 'photos' pairs into the list of dictionaries containing 'image' and 'post' keys.
Append the result to request.data with a key name 'photos'. This is because our PostSerializer expects the key name to be 'photos'.
However basically request.data is a QuerySet which is immutable by default. And I am quite skeptical if we must force-mutate the QuerySet. Therefore, I'll rather commission PostImage creation process to update() method of the ModelViewSet. In this case, we don't need to define nested serializer anymore.
Simply just do this:
class PostSerializer(serializers.ModelSerializer):
class Meta:
model = Post
fields = '__all__'
class PostImageSerializer(serializer.ModelSerializer):
class Meta:
model = PostImage
fields = '__all__'
3. Override update() method from ModelViewSet
In order to utilize our Parser class, we need to explicitly designate it. We will consolidate PATCH and PUT behaviour, so set partial=True. As we saw earlier, Image files are carried with the key 'photos' so pop out the values and create each Photo instance.
Finally, thanks to our newly designed Parser, plain 'JSON-like' string would be transformed into regular JSON data. So just simply put eveything into serializer_class and perform_update.
class PostViewSet(viewsets.ModelViewSet):
queryset = Post.objects.all()
serializer_class = PostSerializer
# New Parser
parser_classes = (MultipartJsonParser,)
def update(self, request, *args, **kwargs):
# Unify PATCH and PUT
partial = True
instance = self.get_object()
# Create each PostImage
for photo in request.data.pop("photos"):
PostImage.objects.create(post=instance, image=photo)
serializer = self.get_serializer(instance, data=request.data, partial=partial)
serializer.is_valid(raise_exception=True)
# Do ViewSet work.
self.perform_update(serializer)
return Response(serializer.data)
Conclusion
The solution works, but I'm not sure this is the cleanest way of saving foreign key related models. I get a strong feeling that it is the serializer that should save the related model. Just as the doc stated, data other than files are saved that way. If somebody could tell me more subtle way to do this, I would be deeply appreciated.
I create realtime chat application using websocket frontend(angular) backend(Django).. I want to store messages in to db(mySql).. when I trying to store message from angular to django.. it give me error:
non_field_errors:
["Expected a list of items but got type "dict"."]
so what is wrong?
model.py
class msg(models.Model):
name = models.ForeignKey(User, on_delete=models.CASCADE)
receiver = models.CharField(max_length=20)
text = models.CharField(max_length=1200)
myDate = models.DateTimeField()
serializer.py
class MesSerializer(serializers.ModelSerializer):
name = serializers.SlugRelatedField(many=False, slug_field='name', queryset=User.objects.all())
class Meta:
model = msg
fields = '__all__'
view.py
class msg_list(APIView):
def get(self, request, format=None):
mes = msg.objects.all()
serializer = MesSerializer(mes, many=True) # convert into JSON
return Response(serializer.data)
def post(self, request, formate = None):
serializer = MesSerializer(data=request.data, many = True) #type list
if serializer.is_valid():
serializer.save()
return JsonResponse(serializer.data, status=201)
return JsonResponse(serializer.errors, status=400)
The trouble is not with the slug. It is that you have used many = True in the view when you pass the data to the serializer, but you are in fact only sending a single message - which is why it is a dict and not a list. Remove that parameter.
def post(self, request, formate = None):
serializer = MesSerializer(data=request.data)
I recently came across a problem which is similar to the OP. Hence would like to share my experience and solution.
I have a dictionary of items, each of them unique. I wanted to use PUT to update an existing item. So I used objects.filter to fetch the object based on the name passed via JSON Request(I know I should have used pk, but I didn't because the items are unique and will remain so). Then I created a Django REST Serializer class object to save the updated object but I failed. This is because I wasn't using many = True. But when I did use it, I faced another error:
"non_field_errors": [
"Expected a list of items but got type \"dict\"."
]
So I finally removed both many=True as well objects.filter.
Instead I used objects.get. This solved the problem because objects.get returns the required object which I want to update whereas objects.filter returns a queryset object and not the actual object that I want to update. Of course objects.get will fail if I have multiple results, in which case I need to ensure there is a pk. Then again in my case objects.get will never return more than one object.
Hope this post helps someone and saves a lot of their time. :-)
I'm making a Django application, where I am sending requests through rest API. I am using POST and GET and all works pretty well, but when I am trying to use PATCH (as I must firstly upload field "start_time", and then add field "time"), I'm getting following error:
match = time_re.match(value)
TypeError: expected string or buffer
Surely it is about views.py, but I cannot find clear recipe how to do that, thus I don't know where I am wrong.
Thank you.
Views.py
...
elif request.method = 'PATCH':
serializer = TABLESerializer(data=request.data, partial=True)
if serializer.is_valid():
obj = TABLE.objects.get(start_time=request.data['start_time'])
obj.time = serializer['time']
obj.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
Serializers.py
class TABLESerilizer(serializers.serializer):
start_time = serializers.DateTimeField(format = None, allow_null=True)
time = serializers.TimeField(format=None, required=False)
models.py
class TABLE(models.Model):
start_time=models.DateTimeField(primary_key=True)
time = models.TimeField(null= True, blank= True, default = '00:00:00')
format - A string representing the output format. If not specified,
this defaults to the same value as the TIME_FORMAT settings key, which
will be 'iso-8601' unless set.
But in your case you set it to None. we get error when python try to parse the data into time object with format None format must me like HH:MM:SS, etc.
update your serializer like below
from rest_framework.settings import api_settings
class TABLESerilizer(serializers.serializer):
start_time = serializers.DateTimeField(format=api_settings.TIME_FORMAT)
time = serializers.TimeField(format=api_settings.TIME_FORMAT)
References: http://www.django-rest-framework.org/api-guide/fields/#timefield
https://github.com/django/django/blob/master/django/utils/dateparse.py#L80