DRF multiple update with reduced database hits - python

I'm using DRF's example of multiple updates which works fine except every self.child.update is a separate update query to the database.
Is there a way I can rewrite this to call the updates as one query as a bulk update?
class BookListSerializer(serializers.ListSerializer):
def update(self, instance, validated_data):
book_mapping = {book.id: book for book in instance}
data_mapping = {item['id']: item for item in validated_data}
ret = []
for book_id, data in data_mapping.items():
book = book_mapping.get(book_id, None)
ret.append(self.child.update(book, data))
return ret
class BookSerializer(serializers.Serializer):
id = serializers.IntegerField()
class Meta:
list_serializer_class = BookListSerializer

Django has a command bulk_update
Docs: https://docs.djangoproject.com/en/4.1/ref/models/querysets/#bulk-update
I'd write up a full example but I'm not sure what validated_data looks like

A few suggestions:
- Use ModelSerializer, as mentioned. It will handle updates/creates automatically and more efficiently.
- Use model validation (clean methods) instead of serializer validation for database-level validation.
- Use bulk_create for the creation case (instead of a loop).
- Use select_related/prefetch_related to reduce queries when fetching the instance.
- Use .update() or F-expressions to update fields, instead of re-getting the instance from the DB.
- Use partial=True on the serializer if you expect partial updates.

Related

Graphene Django without Django Model?

I've successfully used Graphene-Django to successfully build several GraphQL calls. In all of those cases I populated, in whole or in part, a Django model and then returned the records I populated.
Now I have a situation where I'd like to return some data that I don't wish to store in the Django model. Is this possible to do with Graphene?
Robert
Robert_LY answered his own question perfectly in the comments, I'd just like to expand his solution.
My database-less model WordForm is generated automatically, without storing it in a database. I define it as a Django model as follows:
from django.db import models
class WordForm(models.Model):
value = models.CharField(max_length=100)
attributes = models.CharField(max_length=100)
In the schema I define the node and query like this:
class WordFormNode(DjangoObjectType):
class Meta:
model = WordForm
interfaces = (relay.Node, )
class Query(AbstractType):
word_forms = List(WordFormNode,query=String(),some_id=String())
def resolve_word_forms(self, args, context, info):
query= args['query']
some_id = from_global_id(args['some_id'])[1]
word_forms = []
# some logic to make WordForm objects with
# WordForm(value=value,attributes=attributes),
# then append them to list word_forms
return word_forms
You can pass as many arguments as you like to the List and access them in resolve_word_forms.
When you map your Django model to a GraphQL, it create a new model with GraphQL object types from the introspection of the Django model..
And nothing prevent you to combine this model with with plain GraphQL objects types, or mapped from an other third party persistence model

How to automatically fill-in model fields in Django rest_framework serializer?

Let's assume I have a model like this:
class Data(models.Model):
a = models.CharField()
b = models.CharField()
c = models.IntegerField()
I would like to setup a serializer in such a way that it automatically fills in field c and it is not required for a POST. I tried to overwrite the create function of the serializer, but it doesn't work:
class DataSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Data
fields = ('a', 'b')
def create(self, validated_data, **kwargs):
Data.objects.c = 5
return Data.objects.create(**validated_data)
However, if I try this, I end up with an IntegrityError: NOT NULL constraint failed: model_data.c. What is the syntax that I have to use here?
EDIT: Updated formatting.
The reason you're getting the error because field c is not set to null = True - as such an error is raised at the validation stage even before the serializer hits the create method.
Bear in mind that the process goes like this:
Submit serializer data
field-level validation happens - this includes checks for null integrity, min/max length etc and also any custom field validations defined in def validate_<field_name>
object-level validation happens - this calls the def validate method
validated data is passed to the save method, depending on how you designed the serializer - it will save the instance, or route the data to either create or update
All of the info regarding this can be found in Django's and DRF's docs.
A few things to consider:
are you setting a global default for that field? If so, set the default in your models - c = models.IntegerField(default=a_number_or_a_callable_that_returns_an_integer)
do you intend to display the field? If so, include c in your fields and add one more Meta attribute - read_only_fields = ('c',)
If it's neither of the above, you might want to override the validate_c method
Apologies for the poor formatting, typing it on my phone - will update once I get to a computer
In your code Data.objects.c = 5 does nothing.
If you want to set this value yourself use validated_data['c'] = 5 or Data.objects.create(c=5, **validated_data) (just not both at the same time).
Rather than doing this in the serializer, there are hooks in the generic views that allow you to pass values to the serializer. So in your case you might have:
class DataViewSet(ModelViewSet):
# ...
def perform_create(self, serializer):
serializer.save(c=5)
See the "Save and deletion hooks" section here

How to handle concurrency with django queryset get method?

I'm using django (1.5 with mysql) select_for_update method for fetching data from one model and serve this data to user upon request, but when two user request at simultaneously it returns same data for both of the user, see the sample code below
models.py
class SaveAccessCode(models.Model):
code = models.CharField(max_length=10)
class AccessCode(models.Model):
code = models.CharField(max_length=10)
state = models.CharField(max_length=10, default='OPEN')
views.py
def view(request, code):
# for example code = 234567
acccess_code = AccessCode.objects.select_for_update().filter(
code=code, state='OPEN')
acccess_code.delete()
SaveAccessCode.objects.create(code=code)
return
Concurrent request will generate two records of SaveAccessCode with same code, Please guide me how to handle this scenario in better way
You need to set some flag on the model when doing select_for_update, something like:
qs.first().update(is_locked=True)`
and before that should do select like
qs = self.select_for_update().filter(state='OPEN', is_locked=False).order_by('id')
Then after the user, I presume, has done something with it and saved, set the flag is_locked=False and save.
Also make the fetch_available_code as a #staticmethod.

Validating a Django model field based on another field's value?

I have a Django app with models accessible by both Django REST Framework and a regular form interface. The form interface has some validation checks before saving changes to the model, but not using any special Django framework, just a simple local change in the view.
I'd like to apply the same validation to forms and REST calls, so I want to move my validation into the model. I can see how to do that for simple cases using the validators field of the Field, but in one case I have a name/type/value model where the acceptable values for 'value' change depending on which type is selected. The validator doesn't get sent any information about the model that the field is in, so it doesn't have access to other fields.
How can I perform this validation, without having essentially the same code in a serializer for DRF and my POST view for the form?
I dug around codebase of drf a little bit. You can get values of all fields using following approach. Doing so, you can throw serialization error as
{'my_field':'error message} instead of {'non_field_error':'error message'}.
def validate_myfield(self, value):
data = self.get_initial() # data for all the fields
#do your validation
However, if you wish to do it for ListSerializer, i.e for serializer = serializer_class(many=True), this won't work. You will get list of empty values.
In that scenario, you could write your validations in def validate function and to avoid non_field_errors in your serialization error, you can raise ValidationError with error message as a dictionary instead of string.
def validate(self, data):
# do your validation
raise serializers.ValidationError({"your_field": "error_message"})
The validation per-field doesn't get sent any information about other fields, when it is defined like this:
def validate_myfield(self, value):
...
However, if you have a method defined like this:
def validate(self, data):
...
Then you get all the data in a dict, and you can do cross-field validation.
You can use the required package for your cross-field validation. It allows you to express your validation rules declaratively in python. You would have something like this with DRF:
class MySerializer(serializers.Serializer):
REQUIREMENTS = (
Requires("end_date", "start_date") +
Requires("end_date", R("end_date") > R("start_date")) +
Requires("end_date", R("end_date") < today.date() + one_year) +
Requires("start_date", R("start_date") < today.date() + one_year)
)
start_date = serializers.DateField(required=False, null=True, blank=True)
end_date = serializers.DateField(required=False, null=True, blank=True)
def validate(self, data):
self.REQUIREMENTS.validate(data) # handle validation error
You could put the REQUIREMENTS on your Model and have both your DRF and Django Form validate your data using it.
Here is a blog post explaining more

Django Rest Framework - Read nested data, write integer

So far I'm extremely happy with Django Rest Framework, which is why I alsmost can't believe there's such a large omission in the codebase. Hopefully someone knows of a way how to support this:
class PinSerializer(serializers.ModelSerializer):
item = ItemSerializer(read_only=True, source='item')
item = serializers.IntegerSerializer(write_only=True)
class Meta:
model = Pin
with the goal
The goal here is to read:
{pin: item: {name: 'a', url: 'b'}}
but to write using an id
{pin: item: 10}
An alternative would be to use two serializers, but that looks like a really ugly solution:
django rest framework model serializers - read nested, write flat
Django lets you access the Item on your Pin with the item attribute, but actually stores the relationship as item_id. You can use this strategy in your serializer to get around the fact that a Python object cannot have two attributes with the same name (a problem you would encounter in your code).
The best way to do this is to use a PrimaryKeyRelatedField with a source argument. This will ensure proper validation gets done, converting "item_id": <id> to "item": <instance> during field validation (immediately before the serializer's validate call). This allows you to manipulate the full object during validate, create, and update methods. Your final code would be:
class PinSerializer(serializers.ModelSerializer):
item = ItemSerializer(read_only=True)
item_id = serializers.PrimaryKeyRelatedField(write_only=True,
source='item',
queryset=Item.objects.all())
class Meta:
model = Pin
fields = ('id', 'item', 'item_id',)
Note 1: I also removed source='item' on the read-field as that was redundant.
Note 2: I actually find it rather unintuitive that Django Rest is set up such that a Pin serializer without an Item serializer specified returns the item_id as "item": <id> and not "item_id": <id>, but that is beside the point.
This method can even be used with forward and reverse "Many" relationships. For example, you can use an array of pin_ids to set all the Pins on an Item with the following code:
class ItemSerializer(serializers.ModelSerializer):
pins = PinSerializer(many=True, read_only=True)
pin_ids = serializers.PrimaryKeyRelatedField(many=True,
write_only=True,
source='pins',
queryset=Pin.objects.all())
class Meta:
model = Item
fields = ('id', 'pins', 'pin_ids',)
Another strategy that I previously recommended is to use an IntegerField to directly set the item_id. Assuming you are using a OneToOneField or ForeignKey to relate your Pin to your Item, you can set item_id to an integer without using the item field at all. This weakens the validation and can result in DB-level errors from constraints being violated. If you want to skip the validation DB call, have a specific need for the ID instead of the object in your validate/create/update code, or need simultaneously writable fields with the same source, this may be better, but I wouldn't recommend anymore. The full line would be:
item_id = serializers.IntegerField(write_only=True)
If you are using DRF 3.0 you can implement the new to_internal_value method to override the item field to change it to a PrimaryKeyRelatedField to allow the flat writes. The to_internal_value takes unvalidated incoming data as input and should return the validated data that will be made available as serializer.validated_data. See the docs: http://www.django-rest-framework.org/api-guide/serializers/#to_internal_valueself-data
So in your case it would be:
class ItemSerializer(ModelSerializer):
class Meta:
model = Item
class PinSerializer(ModelSerializer):
item = ItemSerializer()
# override the nested item field to PrimareKeyRelatedField on writes
def to_internal_value(self, data):
self.fields['item'] = serializers.PrimaryKeyRelatedField(queryset=Item.objects.all())
return super(PinSerializer, self).to_internal_value(data)
class Meta:
model = Pin
Two things to note: The browsable web api will still think that writes will be nested. I'm not sure how to fix that but I only using the web interface for debug so not a big deal. Also, after you write the item returned will have flat item instead of the nested one. To fix that you can add this code to force the reads to use the Item serializer always.
def to_representation(self, obj):
self.fields['item'] = ItemSerializer()
return super(PinSerializer, self).to_representation(obj)
I got the idea from this from Anton Dmitrievsky's answer here: DRF: Simple foreign key assignment with nested serializers?
You can create a Customized Serializer Field (http://www.django-rest-framework.org/api-guide/fields)
The example took from the link:
class ColourField(serializers.WritableField):
"""
Color objects are serialized into "rgb(#, #, #)" notation.
"""
def to_native(self, obj):
return "rgb(%d, %d, %d)" % (obj.red, obj.green, obj.blue)
def from_native(self, data):
data = data.strip('rgb(').rstrip(')')
red, green, blue = [int(col) for col in data.split(',')]
return Color(red, green, blue)
Then use this field in your serializer class.
I create a Field type that tries to solve the problem of the Data Save requests with its ForeignKey in Integer, and the requests to read data with nested data
This is the class:
class NestedRelatedField(serializers.PrimaryKeyRelatedField):
"""
Model identical to PrimaryKeyRelatedField but its
representation will be nested and its input will
be a primary key.
"""
def __init__(self, **kwargs):
self.pk_field = kwargs.pop('pk_field', None)
self.model = kwargs.pop('model', None)
self.serializer_class = kwargs.pop('serializer_class', None)
super().__init__(**kwargs)
def to_representation(self, data):
pk = super(NestedRelatedField, self).to_representation(data)
try:
return self.serializer_class(self.model.objects.get(pk=pk)).data
except self.model.DoesNotExist:
return None
def to_internal_value(self, data):
return serializers.PrimaryKeyRelatedField.to_internal_value(self, data)
And so it would be used:
class PostModelSerializer(serializers.ModelSerializer):
message = NestedRelatedField(
queryset=MessagePrefix.objects.all(),
model=MessagePrefix,
serializer_class=MessagePrefixModelSerializer
)
I hope this helps you.

Categories