We are creating an API that needs to allow a user to update a record. In many cases, like a status update or name change, only one field will change. This seems an appropriate use-case scenario for a PATCH request. As I understand it this is a 'partial' update.
We've implemented Django's REST Framework and run into this issue. For a record such as a "AccountUser" I want to change a name field only so I send the following request:
PATCH /api/users/1/ HTTP/1.1
Host: localhost
X-CSRFToken: 111122223333444455556666
Content-Type: application/json;charset=UTF-8
Cache-Control: no-cache
{ "fullname": "John Doe" }
The record obviously has other attributes including a couple of 'related' fields such as 'account' which is required for a new record. When submit the request, the response is a 400 error with the following body:
{ "account": [ "This field is required." ] }
The serializer for the user looks like this:
class AccountUserSerializer(serializers.ModelSerializer):
account = serializers.PrimaryKeyRelatedField()
class Meta:
model = AccountUser
fields = ('id', 'account', 'fullname', ... )
depth = 1
And the model looks like this:
class AccountUser(models.Model):
''' Account User'''
fullname = models.CharField(max_length=200,
null=True,blank=True)
account = models.ForeignKey(Account,
on_delete=models.PROTECT
)
objects = AccountUserManager()
def __unicode__(self):
return self.email
class Meta:
db_table = 'accounts_account_user'
Am I doing something wrong here or is it wrong to expect to be able to update a single field on a record this way. Thanks! This community rocks!
EDIT:
Requested - AccountUserManager:
class AccountUserManager(BaseUserManager):
def create_user(self, email, account_name):
username = hash_email_into_username(email)
...
account = Account.objects.get(name=account_name)
account_user = AccountUser(email=email,user=user,account=account)
account_user.save()
return account_user
It doesn't look like your manager is filtering the user. I'd encourage you to use pdb and set a breakpoint in your view code and step through to see why its attempting to create a new record. I can vouch that we use PATCH to complete partial updates all the time and only send a few fields to update without issue.
Only other thought is that you're some how sending a value for account (like null) that's triggering the validation error even though you're listed example only shows sending the fullname field.
See my answer about partial updates. Also you can see the drf docs and this one docs
In my case, all I had to do was add required=False to my serializer field's arguments.
A novice mistake to be sure, but I thought I'd mention it here in case it helps anyone.
I came here because I got into a similar problem. CREATE is fine but PATCH isn't.
Since OP doesn't post the ViewSets and I suspend if he is using a custom ViewSets, which I got a mistake when override the update function:
class MyViewSet(ModelViewSet):
def update(self, request, *args, **kwargs):
// Do some checking
return super().update(request, args, kwargs)
My mistake is in the call to super(). Missing the asterisks. This fix my problem
return super().update(request, *args, **kwargs)
you can simply add this line after the fields variable in serializers.py
class AccountUserSerializer(serializers.ModelSerializer):
account = serializers.PrimaryKeyRelatedField()
class Meta:
model = AccountUser
fields = ('id', 'account', 'fullname', ... )
depth = 1
extra_kwargs = {
'account' : {'required' : False}
}
Related
I would like to be able to parse the data from a post request in my django rest api project by sending it to my function that will return true or false if the number is valid before saving it to the database and if it's wrong send a custom bad request message to the client that did the request.
I've been told I can overwrite the create method todo this but I'm not sure how to go about it.
My code so far looks like this:
class Messages(models.Model):
phone_number = models.CharField(max_length=256, default='')
message_body = models.CharField(max_length=256, default='')
created = models.DateTimeField(auto_now_add=True)
def __str__(self):
return self.phone_number + ' ' + self.message_body + ' ' + self.created
def save(self, force_insert=False, force_update=False, using=None, update_fields=None):
# I assume this is where I would do the check before saving it but not sure how? example would be like:
# if numberValid(self.phone_number):
# then save to the database
# else:
# then send back a bad request?
super(Messages, self).save(force_update=force_update)
send_sms(self.phone_number, self.message_body)
def delete(self, using=None, keep_parents=False):
super(Messages, self).delete(using=using, keep_parents=keep_parents)
So basically just would like some direction on how to solve this problem. Even helpful links would be appreciated. I did look on stackoverflow but was not successful, maybe I don't know how to phrase the question right when searching.
You can use DRF Serializer's validation. For example, create a serializer, and add a validation method naming validate_<field_name>. Then add the validation code there:
import re
class MessagesSerializer(serializers.ModelSerializer):
class Meta:
model = Messages
fields = "__all__"
def validate_phone_number(self, value):
rule = re.compile(r'(^[+0-9]{1,3})*([0-9]{10,11}$)')
if not rule.search(value):
raise serializers.ValidationError("Invalid Phone Number")
return value
And use it in the view:
class SomeView(APIView):
def post(self, request, *args, **kwargs):
serializer = MessagesSerializer(
data=request.data
)
if serializer.is_valid(): # will call the validate function
serializer.save()
return Response({'success': True})
else:
return Response(
serializer.errors,
status=status.HTTP_400_BAD_REQUEST
)
Check the official documentation for how this is to be done: https://docs.djangoproject.com/en/2.2/ref/models/instances/#django.db.models.Model.clean
This method should be used to provide custom model validation, and to modify attributes on your model if desired. For instance, you could use it to automatically provide a value for a field, or to do validation that requires access to more than a single field:
def clean(self):
# Don't allow draft entries to have a pub_date.
if self.status == 'draft' and self.pub_date is not None:
raise ValidationError(_('Draft entries may not have a publication date.'))
# Set the pub_date for published items if it hasn't been set already.
if self.status == 'published' and self.pub_date is None:
self.pub_date = datetime.date.today()
Implement a clean method that will raise a ValidationError if it detects a problem with the data. You can then catch this in the view by calling model_obj.full_clean():
from django.core.exceptions import NON_FIELD_ERRORS, ValidationError
try:
article.full_clean()
except ValidationError as e:
non_field_errors = e.message_dict[NON_FIELD_ERRORS]
Model.save() is an option although it's more common to validate input data, like a phone number being posted, in the DRF Serializer.
Where to perform checks is a decision you can make based on the principal of separation of concerns.
You want to validate the fields before saving.
There are quite a few techniques to do that.
using serializers. If you are using django rest framework, then you can easily use serilizers for validating.
https://www.django-rest-framework.org/api-guide/validators/
django model validation. This is by overriding some of the available methods in models class.
https://docs.djangoproject.com/en/2.2/ref/models/instances/#validating-objects
For your scenario i would suggest second option. Override the method clean_fields as in documentation. Then call the method just before saving.
Situation
While working with validation in the Django REST Framework's ModelSerializer, I have noticed that the Meta.model fields are always validated, even when it does not necessarily make sense to do so. Take the following example for a User model's serialization:
I have an endpoint that creates a user. As such, there is a password field and a confirm_password field. If the two fields do not match, the user cannot be created. Likewise, if the requested username already exists, the user cannot be created.
The user POSTs improper values for each of the fields mentioned above
An implementation of validate has been made in the serializer (see below), catching the non-matching password and confirm_password fields
Implementation of validate:
def validate(self, data):
if data['password'] != data.pop('confirm_password'):
raise serializers.ValidationError("Passwords do not match")
return data
Problem
Even when the ValidationError is raised by validate, the ModelSerializer still queries the database to check to see if the username is already in use. This is evident in the error-list that gets returned from the endpoint; both the model and non-field errors are present.
Consequently, I would like to know how to prevent model validation until after non-field validation has finished, saving me a call to my database.
Attempt at solution
I have been trying to go through the DRF's source to figure out where this is happening, but I have been unsuccessful in locating what I need to override in order to get this to work.
Since most likely your username field has unique=True set, Django REST Framework automatically adds a validator that checks to make sure the new username is unique. You can actually confirm this by doing repr(serializer()), which will show you all of the automatically generated fields, which includes the validators.
Validation is run in a specific, undocumented order
Field deserialization called (serializer.to_internal_value and field.run_validators)
serializer.validate_[field] is called for each field
Serializer-level validators are called (serializer.run_validation followed by serializer.run_validators)
serializer.validate is called
So the problem that you are seeing is that the field-level validation is called before your serializer-level validation. While I wouldn't recommend it, you can remove the field-level validator by setting extra_kwargs in your serilalizer's meta.
class Meta:
extra_kwargs = {
"username": {
"validators": [],
},
}
You will need to re-implement the unique check in your own validation though, along with any additional validators that have been automatically generated.
I was also trying to understand how the control flows during serializer validation and after carefully going through the source code of djangorestframework-3.10.3 I came up with below request flow diagram. I have described the flow and what happens in the flow to the best of my understanding without going into too much detail as it can be looked up from source.
Ignore the incomplete method signatures. Only focusing on what methods are called on what classes.
Assuming you have an overridden is_valid method on your serializer class (MySerializer(serializers.Serializer)) when you call my_serializer.is_valid() the following takes place.
MySerializer.is_valid() is executed.
Assuming you are calling the super class (BaseSerializer) is_valid method (like: super(MySerializer, self).is_valid(raise_exception) in your MySerializer.is_valid() method, that will be called.
Now since MySerializer is extending serializers.Serializer, the run_validation() method from serializer.Serializers is called. This is validating only the data dict the first. So we haven't yet started field level validations.
Then the validate_empty_values from fields.Field gets called. This again happens on the entire data and not a single field.
Then the Serializer.to_internal_method is called.
Now we loop over each fields defined on the serializer. And for each field, first we call the field.run_validation() method. If the field has overridden the Field.run_validation() method then that will be called first. In case of a CharField it is overridden and calls the run_validation method of Field base class. Step 6-2 in the figure.
On that field we again call the Field.validate_empty_values()
The to_internal_value of the type of field is called next.
Now there is a call to the Field.run_validators() method. I presume this is where the additional validators that we add on the field by specifying the validators = [] field option get executed one by one
Once all this is done, we are back to the Serializer.to_internal_value() method. Now remember that we are doing the above for each field within that for loop. Now the custom field validators you wrote in your serializer (methods like validate_field_name) are run. If an exception occurred in any of the previous steps, your custom validators wont run.
read_only_defaults()
update validate data with defaults I think
run object level validators. I think the validate() method on your object is run here.
I don't believe the above solutions work any more. In my case, my model has fields 'first_name' and 'last_name', but the API will only receive 'name'.
Setting 'extra_kwargs' and 'validators' in the Meta class seems to have no effect, first_name and last_name are allways deemed required, and validators are always called. I can't overload the first_name/last_name character fields with
anotherrepfor_first_name = serializers.CharField(source=first_name, required=False)
as the names make sense. After many hours of frustration, I found the only way I could override the validators with a ModelSerializer instance was to override the class initializer as follows (forgive the incorrect indentation):
class ContactSerializer(serializers.ModelSerializer):
name = serializers.CharField(required=True)
class Meta:
model = Contact
fields = [ 'name', 'first_name', 'last_name', 'email', 'phone', 'question' ]
def __init__(self, *args, **kwargs):
self.fields['first_name'] = serializers.CharField(required=False, allow_null=True, allow_blank=True)
self.fields['last_name'] = serializers.CharField(required=False, allow_null=True, allow_blank=True)
return super(ContactSerializer, self).__init__(*args, **kwargs)
def create(self, validated_data):
return Contact.objects.create()
def validate(self, data):
"""
Remove name after getting first_name, last_name
"""
missing = []
for k in ['name', 'email', 'question']:
if k not in self.fields:
missing.append(k)
if len(missing):
raise serializers.ValidationError("Ooops! The following fields are required: %s" % ','.join(missing))
from nameparser import HumanName
names = HumanName(data['name'])
names.capitalize()
data['last_name'] = names.last
if re.search(r'\w+', names.middle):
data['first_name'] = ' '.join([names.first, names.middle])
else:
data['first_name'] = names.first
del(data['name'])
return data
Now the doc says that allowing blank and null with character fields is a no no, but this is a serializer, not a model, and as the API gets called by all kinds of cowboys, I need to cover my bases.
Here's the approach that worked for me.
Use a sentinel error type that gets caught in an overridden view function
The sentinel is raised from the custom serializer
The sentinel error type:
from django.core.exceptions import ValidationError
class CustomValidationErrors(ValidationError):
""" custom validation error for the api view to catch the status code """
And in the serializer we override errors, _errors, validated_data, and _validated_data, as well as is_valid:
class CustomSerializer(serializers.ModelSerializer):
# fields that usually run validation before parent serializer validation
child_field1 = Child1Serializer()
child_field2 = Child2Serializer()
# override DRF fields
errors = {}
_errors = None
validated_data = {}
_validated_data = []
def is_valid(self, *args, **kwargs):
# override drf.serializers.Serializer.is_valid
# and raise CustomValidationErrors from parent validate
self.validate(self.initial_data)
return not bool(self.errors)
def validate(self, attrs):
self._errors = {}
if len(attrs.get("child_field1", {}).get("name", "")) > 100:
self._errors.update({"child_field1": {"name": "child 1 name > 100"}})
if len(attrs.get("child_field2", {}).get("description", "")) > 1000:
self._errors.update({"child_field2.description": "child 2 description > 100"})
if len(self._errors):
# set the overriden DRF values
self.errors = self._errors
# raise the sentinel error type
raise CustomValidationErrors(self._errors)
# set the overriden DRF serializer values
self._errors = None
self.validated_data = attrs
self._validated_data = [[k, v] for k, v in attrs.items()]
return attrs
class Meta:
model = CustomModel
And in the view we can override the default method, and catch the sentinel error type:
class CustomSerializerView(ListCreateAPIView):
serializer_class = CustomeSerializer
def post(self, *args, **kwargs):
try:
# if this fails for any exception
# other than CustomValidationErrors
# it will return the default error
return super().post(*args, **kwargs)
except CustomValidationErrors as e:
############
# returns a 400 with the following
# {"child_field1":
# [[{"name": "child 1 name > 100"}]],
# "child_field2.description":
# [["child 2 description > 100"]]
# }
############
return Response(e.error_dict, status=400)
drf version:
djangorestframework==3.11.0
I have a form that looks like this:
class ContactForm(forms.ModelForm):
error_messages = {
'duplicate_name': 'A backup contact with that name already exists for this location',
'missing_location': 'No location supplied.'
}
class Meta:
fields = ('name', 'notification_preference', 'email', 'phone')
model = Contact
········
def clean_name(self):
# FIXME: Location validation shouldn't be happening here, but it's difficult to get
# form fields for foreign key relationships to play nicely with Tastypie
print dir(self)
if 'location' in self.data:
location = Location.objects.get(pk=self.uri_to_pk(self.data['location']))
else:
raise forms.ValidationError(self.error_messages['missing_location'])
# No duplicate names in a given location
if 'name' in self.cleaned_data and Contact.objects.filter(name=self.cleaned_data['name'], location=location).exists():
raise forms.ValidationError(self.error_messages['duplicate_name'])
return self.cleaned_data
I'm using it to validate calls to my TastyPie API. The clean_name method is meant to prevent POST requests from happening if they post a contact with the same name to the same location. This works perfectly as long as I'm making a POST request.
If I make a PATCH however, changing, say, the email and phone field on an already existent contact, the clean_name logic is still fired. Since the name already exists for a given location, it raises a validation error.
Should I be overriding something other than clean_name? Can I change the way PATCH works so it ignores certain validations?
Yes, if you're checking values between fields, I'd recommend implementing a general def clean(self, data) that checks the values don't conflict.
Regarding checking for duplicates, I'd advise you to use an .exclude(pk=instance.pk) in your exists() queryset to prevent mistakenly detecting an update the the model as a duplicate. Looking at the tastypie validation source, it adds the appropriate self.instance for the object being updated.
qs = Contact.objects.filter(name=self.cleaned_data.get('name', ''), location=location)
if self.instance and self.instance.pk:
qs = qs.exclude(pk=self.instance.pk)
if qs.exists():
raise forms.ValidationError(...)
Since my model's custom save method takes request.user as an argument I'm unable to do POST/PUT requests.
TypeError at /api/obsadmin/observation/23
save() takes at least 2 arguments (1 given)
I'm using SessionAuthentication() and have included the CSRF token.
Here's the relevant model part:
def save(self, user, owner=None, *args, **kwargs):
self.updated_by = user.id
super(ObsModel, self).save(*args, **kwargs)
And the resource:
class ObservationResource2(ModelResource):
comments = fields.ToManyField(CommentResource2, 'comments', full=True, null=True)
class Meta:
queryset = Observation.objects.filter(is_verified=True)
authentication = SessionAuthentication()
authorization = DjangoAuthorization()
resource_name = 'observation'
always_return_data = True
I've just achieved this same end goal by using the built-in hydrate methods to modify the data prior to saving. The current request is available in bundle.request inside the hydrate methods. See the docs here.
I have a Friend model exposed via FriendResource that I want to link to the creating Django user via a user ForeignKey field.
My example Resource code:
class FriendResource(ModelResource):
class Meta:
queryset = Friend.objects.all()
resource_name = 'friend'
excludes = ['slug',]
authentication = SessionAuthentication()
authorization = DjangoAuthorization()
always_return_data = True
def get_object_list(self, request):
return super(FriendResource, self).get_object_list(request).filter(user=request.user)
def hydrate(self, bundle):
bundle.obj.user = bundle.request.user
return bundle
Hope that helps!
You could override the default save() method on your ModelResource subclass. Looking at the default implementation shows that save() is called with a bundle object which has both the request and the object to be saved.
Unfortunately, there's no easy way to change this without copying most of that code because changing a Django model's save() signature is fairly uncommon. You might be able to do something like this, although I'd recommend testing it carefully:
from functools import partial
try:
old_save = bundle.obj.save
bundle.obj.save = partial(old_save, user=bundle.request.user)
return super(FooResource, self).save(bundle)
finally:
bundle.obj.save = old_save
References:
obj_create: docs source
obj_update: docs source
save: source
In a model's Meta class, I define a unique_together. I have a ModelForm based on this model. When I call is_valid on this ModelForm, an error will automatically raised if unique_together validation fails. That's all good.
Now my problem is that I'm not satisfied with the default unique_together error message. I want to override it. How can I do that? For a field related error, I can easily do that by setting error_messages on the field parameters. But unique_together is a non field error. How can I override a non field error message?
You can do this since Django 1.7
from django.forms import ModelForm
from django.core.exceptions import NON_FIELD_ERRORS
class ArticleForm(ModelForm):
class Meta:
error_messages = {
NON_FIELD_ERRORS: {
'unique_together': "%(model_name)s's %(field_labels)s are not unique.",
}
}
Update 2016/10/20: See jifeng-yin's even nicer answer below for Django >= 1.7
The nicest way to override these error messages might be to override the unique_error_message method on your model. Django calls this method to get the error message whenever it encounters a uniqueness issue during validation.
You can just handle the specific case you want and let all other cases be handled by Django as usual:
def unique_error_message(self, model_class, unique_check):
if model_class == type(self) and unique_check == ('field1', 'field2'):
return 'My custom error message'
else:
return super(Project, self).unique_error_message(model_class, unique_check)
For DRF serializers you can use this
from rest_framework import serializers
class SomeSerializer(serializers.ModelSerializer):
class Meta:
model = Some
validators = [
serializers.UniqueTogetherValidator(
queryset=model.objects.all(),
fields=('field1', 'field2'),
message="Some custom message."
)
]
Here is the original source.
After a quick check, it seems that unique_together validation errors are hard-coded deep in django.db.models.Model.unique_error_message :
def unique_error_message(self, model_class, unique_check):
opts = model_class._meta
model_name = capfirst(opts.verbose_name)
# A unique field
if len(unique_check) == 1:
field_name = unique_check[0]
field_label = capfirst(opts.get_field(field_name).verbose_name)
# Insert the error into the error dict, very sneaky
return _(u"%(model_name)s with this %(field_label)s already exists.") % {
'model_name': unicode(model_name),
'field_label': unicode(field_label)
}
# unique_together
else:
field_labels = map(lambda f: capfirst(opts.get_field(f).verbose_name), unique_check)
field_labels = get_text_list(field_labels, _('and'))
return _(u"%(model_name)s with this %(field_label)s already exists.") % {
'model_name': unicode(model_name),
'field_label': unicode(field_labels)
}
So maybe you should try to override this method from your model, to insert your own message !?
However, I haven't tried, and it seems a rather brutal solution ! But if you don't have something better, you might try...
Notice: A lot had changed in Django since this answer. So better check other answers...
If what sebpiq is true( since i do not check source code), then there is one
possible solution you can do, but it is the hard way...
You can define a validation rule in your form, as it described here
You can see examples of validation with more than one field, so by using this method, you can define a unique together check before standard django unique check executed...
Or the worst one, you can do a validation in your view before you try to save the objects...
You might take a look at overriding django/db/models/base.py:Model._perform_unique_checks() in your model.
In that method you can get the "original" errors:
errors = super(MyModel, self)._perform_unique_checks(unique_checks)
-- then modify and return them upwards.