We Keep Coding

Django Rest Framework: source / re-name without re-confirming the field settings

I'd like to change many fields name in DRF ModelSerializer without the need to re-typing the fields.
According a post on SO (ref), one can re-name a field name within the serializer by using source, such as:
newName = serializers.CharField(source='old_name')
However, this method takes away the benefits of using a ModelSerializer as you essentially do the work twice. This become heavy when you have many fields adhering to one internal naming convention but want to display another naming convention within the API.
in my case, I have a model field such as:
product_uid = models.UUIDField(primary_key=False, unique=True, default=uuid.uuid4, editable=False)
In the API, I'd like the field to be called 'uid'.
If I would do the following:
uid = serializers.UUIDField(source=product_uid)
would result in editable=True
Is there a way to reference to a ModelField and keep its definition intact according the Model (as you normally do when using serializers.ModelSerializer) but only change the name, e.g. something like: uid = serializers.ModelField(source=product_uid) ?

If you want your field not to be editable, you can use the read_only parameter (https://www.django-rest-framework.org/api-guide/fields/#read_only)
You can try:
uid = serializers.UUIDField(source=product_uid, read_only=True)
You can also use the ModelSerializer using extra_kwargs :
class MySerializer(serializers.ModelSerializer):
class Meta:
model = MyModel
fields = ['product_uid', 'field_a', 'field_b']
extra_kwargs = {'product_uid': {'source': 'uid'}} # Add `'read_only': True` if needed
If you have many fields, you can generate extra_kwargs programmatically

Django - validate unique for a calculated field in the Model and also in the ModelForm

TL;DR both my model and my form calculate the value of the field number_as_char. Can I avoid the double work, but still check uniqueness when using the model without the form?
I use Python 3 and Django 1.11
My model looks as follows:
class Account(models.Model):
parent_account = models.ForeignKey(
to='self',
on_delete=models.PROTECT,
null=True,
blank=True)
number_suffix = models.PositiveIntegerField()
number_as_char = models.CharField(
max_length=100,
blank=True,
default='',
unique=True)
#classmethod
def get_number_as_char(cls, parent_account, number_suffix):
# iterate over all parents
suffix_list = [str(number_suffix), ]
parent = parent_account
while parent is not None:
suffix_list.insert(0, str(parent.number_suffix))
parent = parent.parent_account
return '-'.join(suffix_list)
def save(self, *args, **kwargs):
self.number_as_char = self.get_number_as_char(
self.parent_account, self.number_suffix)
super().save(*args, **kwargs)
The field number_as_char is not supposed to be set by the user because it is calculated based on the selected parent_account: it is obtained by chaining the values of the field number_suffix of all the parent accounts and the current instance.
Here is an example with three accounts:
ac1 = Account()
ac1.parent_account = None
ac1.number_suffix = 2
ac1.save()
# ac1.number_as_char is '2'
ac2 = Account()
ac2.parent_account = ac1
ac2.number_suffix = 5
ac2.save()
# ac2.number_as_char is '2-5'
ac3 = Account()
ac3.parent_account = ac2
ac3.number_suffix = 1
ac3.save()
# ac3.number_as_char is '2-5-1'
It is NOT an option to drop the field and use a model property instead, because I need to ensure uniqueness and also use that field for sorting querysets with order_by().
My form looks as follows:
class AccountForm(forms.ModelForm):
class Meta:
model = Account
fields = [
'parent_account', 'number_suffix', 'number_as_char',
]
widgets = {
'number_as_char': forms.TextInput(attrs={'readonly': True}),
}
def clean(self):
super().clean()
self.cleaned_data['number_as_char'] = self.instance.get_number_as_char(
self.cleaned_data['parent_account'], self.cleaned_data['number_suffix'])
I included number_as_char in the form with widget attribute readonly and I use the forms clean() method to calculate number_as_char (it has to be calculated before validating uniqueness).
This all works (the model and the form), but after validating the form, the value of number_as_char will be calculated again by the models save() method. Its not a big problem, but is there a way to avoid this double calculation?
If I remove the calculation from the forms clean() method, then the uniqueness will not be validated with the new value (it will only check the old value).
I don't want to remove the calculation entirely from the model because I use the model in other parts without the form.
Do you have any suggestions what could be done differently to avoid double calculation of the field?

I can't see any way around doing this in two places (save() and clean()) given that you need it to work for non-form-based saves as well).
However I can offer two efficiency improvements to your get_number_as_char method:
Make it a cached_property so that the second time it is called, you simply return a cached value and eliminate double-calculation. Obviously you need to be careful that this isn't called before an instance is updated, otherwise the old number_as_char will be cached. This should be fine as long as get_number_as_char() is only called during a save/clean.
Based on the information you've provided above you shouldn't have to iterate over all the ancestors, but can simply take the number_as_char for the parent and append to it.
The following incorporates both:
#cached_property
def get_number_as_char(self, parent_account, number_suffix):
number_as_char = str(number_suffix)
if parent_account is not None:
number_as_char = '{}-{}'.format(parent_account.number_as_char, number_as_char)
return number_as_char
To be sure that the caching doesn't cause problems you could just clear the cached value after you're done saving:
def save(self, *args, **kwargs):
self.number_as_char = self.get_number_as_char(
self.parent_account, self.number_suffix)
super().save(*args, **kwargs)
# Clear the cache, in case something edits this object again.
del self.get_number_as_char

I tinkered with it a bit, and I think I found a better way.
By using the disabled property on the number_as_char field of your model form, you can entirely ignore users input (and make the field disabled in a single step).
Your model already calculates the number_as_char attribute in the save method. However, if the Unique constraint fails, then your admin UI will throw a 500 error. However, you can move your field calculation to the clean() method, leaving the save() method as it is.
So the full example will look similar to this:
The form:
class AccountForm(forms.ModelForm):
class Meta:
model = Account
fields = [
'parent_account', 'number_suffix', 'number_as_char',
]
widgets = {
'number_as_char': forms.TextInput(attrs={'disabled': True}),
}
The model:
class Account(models.Model):
# ...
def clean(self):
self.number_as_char = self.get_number_as_char(
self.parent_account, self.number_suffix
)
super().clean()
That way anything that generates form based on your model will throw a nice validation error (provided that it uses the built-in model validation, which is the case for Model Forms).
The only downside to this is that if you save a model that triggers the validation error, you will see an empty field instead of the value that failed the validation - but I guess there is some nice way to fix this as well - I'll edit my answer if I also find a solution to this.

After reading all the answers and doing some more digging through the docs, I ended up using the following:
#samu suggested using the models clean() method and #Laurent S suggested using unique_together for (parent_account, number_suffix). Since only using unique_together doesn't work for me because parent_account can be null, I opted for combining the two ideas: checking for existing (parent_account, number_suffix) combinations in the models clean() method.
As a consecuence, I removed number_as_char from the form and it is now only calculated in the save() method. By the way: thanks to #solarissmoke for suggesting to calculated it based on the first parent only, not iterating all the way to the top of the chain.
Another consecuence is that I now need to explicitly call the models full_clean() method to validate uniqueness when using the model without the form (otherwise I will get the database IntegrityError), but I can live with that.
So, now my model looks like this:
class Account(models.Model):
parent_account = models.ForeignKey(
to='self',
on_delete=models.PROTECT,
null=True,
blank=True)
number_suffix = models.PositiveIntegerField()
number_as_char = models.CharField(
max_length=100,
default='0',
unique=True)
def save(self, *args, **kwargs):
if self.parent_account is not None:
self.number_as_char = '{}-{}'.format(
self.parent_account.number_as_char,
self.number_suffix)
else:
self.number_as_char = str(self.number_suffix)
super().save(*args, **kwargs)
def clean(self):
qs = self._meta.model.objects.exclude(pk=self.pk)
qs = qs.filter(
parent_account=self.parent_account,
number_suffix=self.number_suffix)
if qs.exists():
raise ValidationError('... some message ...')
And my form ends up like this:
class AccountForm(forms.ModelForm):
class Meta:
model = Account
fields = [
'parent_account', 'number_suffix',
]
EDIT
I'll mark my own answer as accepted, because non of the suggestions fully suited my needs.
However, the bounty goes to #samus answer for pointing me in the right direction with using the clean() method.

Another way - probably not as good though - would be to use Django signals. You could make a pre_save signal that would set the correct value for number_as_char field on the instance that's about to get saved.
That way you don't have to have it done in a save() method of your model, OR in the clean() method of your ModelForm.
Using signals should ensure that any operation that uses the ORM to manipulate your data (which, by extend, should mean all ModelForms as well) will trigger your signal.
The disadvantage to this approach is that it is not clear from the code directly how is this property generated. One has to stumble upon the signal definition in order to discover that it's even there. If you can live with it though, I'd go with signals.

Django Rest update many to many by id

I'm novice in python and django rest. But I'm confused. What is the best way to update many to many relation in django rest framework.
I read the docs
http://www.django-rest-framework.org/api-guide/relations/#manytomanyfields-with-a-through-model
By default, relational fields that target a ManyToManyField with a through model specified are set to read-only.
If you explicitly specify a relational field pointing to a ManyToManyField with a through model, be sure to set read_only to True.
So if I have a code
class Master(models.Model):
# other fields
skills = models.ManyToManyField(Skill)
class MasterSerializer(serializers.ModelSerializer):
skills = SkillSerializer(many=True, read_only=False)
This will return skills as list of objects. And I don't have a way to update them. As far as I understood Django prefers work with objects vs object id when it comes to M2M. If I work with yii or rails I will work with "through" models. I would like to get skill_ids field. That I could read and write. And I can do this for write operation
class MasterSerializer(serializers.ModelSerializer):
skill_ids = serializers.ListField(write_only=True)
def update(self, instance, validated_data):
# ...
validated_data['skill_ids'] = filter(None, validated_data['skill_ids'])
for skill_id in validated_data['skill_ids']:
skill = Skill.objects.get(pk=skill_id)
instance.skills.add(skill)
return instance
But I cannot make it return skill_ids in field. And work for read and write operations.

A few things to note.
First, you don't have an explicit through table in your example. Therefore you can skip that part.
Second, you are trying to use nested serializers which are far more complex than what you're trying to achieve.
You can simply read/write related id by using a PrimaryKeyRelatedField:
class MasterSerializer(serializers.ModelSerializer):
skills_ids = serializers.PrimaryKeyRelatedField(many=True, read_only=False, queryset=Skill.objects.all(), source='skills')
Which should be able to read/write:
{id: 123, first_name: "John", "skill_ids": [1, 2, 3]}
Note that the mapping from JSON's "skill_ids" to model's "skills" is done by using the optional argument source

I will try to bring some light in terms of design: in Django if you specify the model for a ManyToManyRelation, then the relation field on the model becomes read-only. If you need to alter the associations you do it directly on the through model, by deleting or registering new records.
This means that you may need to use a completely different serializer for the through model, or to write custom update/create methods.
There are some sets back with custom through model, are you sure you're not good enough with the default implementation of ManyToManyFields ?

tl;dr:
For a much simpler, one-liner solution for M2M, I sussed out a solution of the form:
serializer = ServiceSerializer(instance=inst, data={'name':'updated', 'countries': [1,3]}, partial=True)
if serializer.is_valid():
serializer.save()
For a more complete example, I have included the following:
models.py
from django.db import models
class Country(models.Model):
name = models.CharField(max_length=50, null=False, blank=False)
class Service(models.Model):
name = models.CharField(max_length=20, null=True)
countries = models.ManyToManyField('Country')
serializers.py
from rest_framework import serializers
from .models import *
class CountrySerializer(serializers.ModelSerializer):
class Meta:
model = Country
fields = ('name',)
class ServiceSerializer(serializers.ModelSerializer):
class Meta:
model = Service
fields = ('name', 'countries',)
Make sure some dummy service and country instances are created for testing. Then you can update an instance in a function like so:
Update example
# get an object instance by key:
inst = ServiceOffering.objects.get(pk=1)
# Pass the object instance to the serializer and a dictionary
# Stating the fields and values to update. The key here is
# Passing an instance object and the 'partial' argument:
serializer = ServiceSerializer(instance=inst, data={'name':'updated', 'countries': [1,3]}, partial=True)
# validate the serializer and save
if serializer.is_valid():
serializer.save()
return 'Saved successfully!'
else:
print("serializer not valid")
print(serializer.errors)
print(serializer.data)
return "Save failed"
If you inspect the relevant tables, the updates are carried through including to the M2M bridging table.
To extend this example, we could create an object instance in a very similar way:
### Create a new instance example:
# get the potential drop down options:
countries = ['Germany', 'France']
# get the primary keys of the objects:
countries = list(Country.objects.filter(name__in=countries).values_list('pk', flat=True))
# put in to a dictionary and serialize:
data = {'countries': countries, 'name': 'hello-world'}
serializer = ServiceOfferingSerializer(data=data)

I have dealt with this issue for quite some time and I have found that the best way to solve the general problem of updating any many to many field is by working around it.
In my case there is a model called Listing and a user can make a Subscription(the other model) to an instance of the Listing model. The Subscription works with a Generic Foreign Key and the Listing imports the Subscriptions of the users via Many2Many.
Instead of making a PUT request to the Listing Model via API, I simply add the Subscription instance to the right model in the POST Method of the API View of Subscription. Here is my adjusted code:
#Model
class Listing(models.Model):
#Basics
user = models.ForeignKey(settings.AUTH_USER_MODEL)
slug = models.SlugField(unique=True, blank=True)
timestamp = models.DateTimeField(auto_now_add=True, auto_now=False)
#Listing
title = models.CharField(max_length=200)
price = models.CharField(max_length=50, null=True, blank=True)
subscriptions = models.ManyToManyField(Subscription, blank=True)
class Subscription(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
timestamp = models.DateTimeField(auto_now_add=True)
#Views
class APISubscriptionCreateView(APIView): #Retrieve Detail
def post(self, request, format=None):
serializer = SubscriptionCreateSerializer(data=request.data)
if serializer.is_valid():
sub = serializer.save(user=self.request.user)
object_id = request.data['object_id']
lis = Listing.objects.get(pk=object_id)
lis.subscriptions.add(sub)
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
I hope this will help, it took me a while to figure this out

Different serializers for serializing/deserializing using Django REST framework

I've got a model with a recursive relationship to itself:
class Tweet(models.Model):
text = models.CharField(max_length=140)
original = models.ForeignKey("self", null=True, blank=True)
And a serializer that renders the original Tweet inline:
class TweetSerializer(serializers.ModelSerializer):
class Meta:
model = Tweet
fields = ('id', 'text', 'original', 'original_id')
original_id = serializers.IntegerField(source='original_id', required=False)
def to_native(self, obj):
ret = super(TweetSerializer, self).to_native(obj)
del ret['original_id']
return ret
TweetSerializer.base_fields['original'] = TweetSerializer(source='original', read_only=True)
As you can see I've also got an original_id field that is removed in to_native. The purpose of original_id is to allow me to set the original_id of a new tweet, rather than having to supply a full blown Tweed object to the original field. You could say that I'm using it as a write only field.
This seems a bit clunky though. Is there a better way to do it?

OK, two points:
Have you tried using PrimaryKeyRelatedField for your original_id? It would seem to target your use-case specifically. Combined with the depth option it may give you everything you need.
You can switch serializers (e.g. based on request method) by overriding get_serializer_class() on your view. Not sure if you'll get the exact behaviour you want here though.

Reordering fields in Django model

I want to add few fields to every model in my django application. This time it's created_at, updated_at and notes. Duplicating code for every of 20+ models seems dumb. So, I decided to use abstract base class which would add these fields. The problem is that fields inherited from abstract base class come first in the field list in admin. Declaring field order for every ModelAdmin class is not an option, it's even more duplicate code than with manual field declaration.
In my final solution, I modified model constructor to reorder fields in _meta before creating new instance:
class MyModel(models.Model):
# Service fields
notes = my_fields.NotesField()
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
last_fields = ("notes", "created_at", "updated_at")
def __init__(self, *args, **kwargs):
new_order = [f.name for f in self._meta.fields]
for field in self.last_fields:
new_order.remove(field)
new_order.append(field)
self._meta._field_name_cache.sort(key=lambda x: new_order.index(x.name))
super(MyModel, self).__init__(*args, **kwargs)
class ModelA(MyModel):
field1 = models.CharField()
field2 = models.CharField()
#etc ...
It works as intended, but I'm wondering, is there a better way to acheive my goal?

I was having the very same problem, but I found these solutions to be problematic, so here's what I did:
class BaseAdmin(admin.ModelAdmin):
def get_fieldsets(self, request, obj = None):
res = super(BaseAdmin, self).get_fieldsets(request, obj)
# I only need to move one field; change the following
# line to account for more.
res[0][1]['fields'].append(res[0][1]['fields'].pop(0))
return res
Changing the fieldset in the admin makes more sense to me, than changing the fields in the model.

If you mainly need the ordering for Django's admin you could also create your "generic"-admin class via sub-classing Django's admin class. See http://docs.djangoproject.com/en/dev/intro/tutorial02/#customize-the-admin-form for customizing the display of fields in the admin.
You could overwrite the admin's __init__ to setup fields/fieldsets on creation of the admin instance as you wish. E.g. you could do something like:
class MyAdmin(admin.ModelAdmin):
def __init__(self, model, admin_site):
general_fields = ['notes', 'created_at', 'updated_at']
fields = [f.name for f in self.model._meta.fields if f.name not in general_fields]
self.fields = fields + general_fields
super(admin.ModelAdmin, self).__init__(model, admin_site)
Besides that i think it's not a good practice to modify the (private) _field_name_cache!

I ALSO didn't like the other solutions, so I instead just modified the migrations files directly.
Whenever you create a new table in models.py, you will have to run "python manage.py makemigrations" (I believe this in Django >= v1.7.5). Once you do this, open up the newly created migrations file in your_app_path/migrations/ directory and simply move the rows to the order you want them to be in. Then run "python manage.py migrate". Voila! By going into "python manage.py dbshell" you can see that the order of the columns is exactly how you wanted them!
Downside to this method: You have to do this manually for each table you create, but fortunately the overhead is minimal. And this can only be done when you're creating a new table, not to modify an existing one.