We Keep Coding

Trying to set user field in the nested form of a django nested inline formset - fails

I followed this: https://www.yergler.net/2009/09/27/nested-formsets-with-django/ and this: django inline formsets with a complex model for the nested form and overall my code works great.
class Account(models.Model):
user_username = models.ForeignKey(User, on_delete=models.CASCADE)
account_name = models.CharField(max_length=30)
class Classification(models.Model):
user_username=models.ForeignKey(User, on_delete=models.CASCADE)
data_id=models.ForeignKey(ImportData, on_delete=models.CASCADE)
class ImportData(models.Model):
user_username = models.ForeignKey(User, on_delete=models.CASCADE)
data_id = models.UUIDField(
primary_key=True, default=uuid.uuid4, editable=False)
ClassificationFormset = inlineformset_factory(ImportData, Classification, exclude=('user_username',), extra=1)
# below is just what came from the nested formset links above: pasted here for easy reference.
class BaseNestedTransactionFormset(BaseInlineFormSet):
def add_fields(self, form, index):
# allow the super class to create the fields as usual
super(BaseNestedTransactionFormset, self).add_fields(form, index)
try:
instance = self.get_queryset()[index]
pk_value = instance.pk
except IndexError:
instance=None
pk_value = hash(form.prefix)
transaction_data = None
if (self.data):
transaction_data = self.data;
# store the formset in the .nested property
form.nested = [
CategoryFormset(data=transaction_data,
instance = instance,
prefix = 'CAT_%s' % pk_value)]
def is_valid(self):
result = super(BaseNestedTransactionFormset, self).is_valid()
for form in self.forms:
if hasattr(form, 'nested'):
for n in form.nested:
# make sure each nested formset is valid as well
result = result and n.is_valid()
return result
def save_new(self, form, commit=True):
"""Saves and returns a new model instance for the given form."""
instance = super(BaseNestedTransactionFormset, self).save_new(form, commit=commit)
# update the form’s instance reference
form.instance = instance
# update the instance reference on nested forms
for nested in form.nested:
nested.instance = instance
# iterate over the cleaned_data of the nested formset and update the foreignkey reference
for cd in nested.cleaned_data:
cd[nested.fk.name] = instance
return instance
def save_all(self, commit=True):
"""Save all formsets and along with their nested formsets."""
# Save without committing (so self.saved_forms is populated)
# — We need self.saved_forms so we can go back and access
# the nested formsets
objects = self.save(commit=False)
# Save each instance if commit=True
if commit:
for o in objects:
o.save()
# save many to many fields if needed
if not commit:
self.save_m2m()
# save the nested formsets
for form in set(self.initial_forms + self.saved_forms):
# if self.should_delete(form): continue
for nested in form.nested:
nested.save(commit=commit)
ImportTransactionFormset = inlineformset_factory(Account, ImportData, exclude=('user_username',), formset=BaseNestedTransactionFormset, extra=0)
My template has a table that displays the import data formset... user selects the account and the table shows all the imported data from that account. For each of these row forms, there is a hidden row underneath... user clicks a button to show that hidden row. The hidden row displays the nested classification formset.
If include the user_username field in the template and allow for it to be part of the nested formset in the template, i can set is accordingly in the html form and the formsets save no problem.
However: I want to be able to exclude the user_username field from the template and have my view or some other method under the BaseNestedTransactionFormset class set the value of the user_username field to request.user value for whoever is logged in at that time.
I tried to override the clean method, but cleaned_data kicks back an error because the form doesnt validate; the field is required. I can't seem to figure out a good way to do this.
If this was a normal formset, not too hard to do. I would just set the field by modifying what comes back from POST. I have never worked with nested inline formsets, and the prefixes and indeces in the field names have got me. I've been at this for a couple of days and can't seem to be getting anywhere.
I am also contemplating just getting rid of that field from the classification model, since it is already tied to the ImportData model which is linked to the logged in user regardless. I'm just thinking i may run into this at some point again, so maybe good to solve.
Thanks in advance.

Django - validate unique for a calculated field in the Model and also in the ModelForm

TL;DR both my model and my form calculate the value of the field number_as_char. Can I avoid the double work, but still check uniqueness when using the model without the form?
I use Python 3 and Django 1.11
My model looks as follows:
class Account(models.Model):
parent_account = models.ForeignKey(
to='self',
on_delete=models.PROTECT,
null=True,
blank=True)
number_suffix = models.PositiveIntegerField()
number_as_char = models.CharField(
max_length=100,
blank=True,
default='',
unique=True)
#classmethod
def get_number_as_char(cls, parent_account, number_suffix):
# iterate over all parents
suffix_list = [str(number_suffix), ]
parent = parent_account
while parent is not None:
suffix_list.insert(0, str(parent.number_suffix))
parent = parent.parent_account
return '-'.join(suffix_list)
def save(self, *args, **kwargs):
self.number_as_char = self.get_number_as_char(
self.parent_account, self.number_suffix)
super().save(*args, **kwargs)
The field number_as_char is not supposed to be set by the user because it is calculated based on the selected parent_account: it is obtained by chaining the values of the field number_suffix of all the parent accounts and the current instance.
Here is an example with three accounts:
ac1 = Account()
ac1.parent_account = None
ac1.number_suffix = 2
ac1.save()
# ac1.number_as_char is '2'
ac2 = Account()
ac2.parent_account = ac1
ac2.number_suffix = 5
ac2.save()
# ac2.number_as_char is '2-5'
ac3 = Account()
ac3.parent_account = ac2
ac3.number_suffix = 1
ac3.save()
# ac3.number_as_char is '2-5-1'
It is NOT an option to drop the field and use a model property instead, because I need to ensure uniqueness and also use that field for sorting querysets with order_by().
My form looks as follows:
class AccountForm(forms.ModelForm):
class Meta:
model = Account
fields = [
'parent_account', 'number_suffix', 'number_as_char',
]
widgets = {
'number_as_char': forms.TextInput(attrs={'readonly': True}),
}
def clean(self):
super().clean()
self.cleaned_data['number_as_char'] = self.instance.get_number_as_char(
self.cleaned_data['parent_account'], self.cleaned_data['number_suffix'])
I included number_as_char in the form with widget attribute readonly and I use the forms clean() method to calculate number_as_char (it has to be calculated before validating uniqueness).
This all works (the model and the form), but after validating the form, the value of number_as_char will be calculated again by the models save() method. Its not a big problem, but is there a way to avoid this double calculation?
If I remove the calculation from the forms clean() method, then the uniqueness will not be validated with the new value (it will only check the old value).
I don't want to remove the calculation entirely from the model because I use the model in other parts without the form.
Do you have any suggestions what could be done differently to avoid double calculation of the field?

I can't see any way around doing this in two places (save() and clean()) given that you need it to work for non-form-based saves as well).
However I can offer two efficiency improvements to your get_number_as_char method:
Make it a cached_property so that the second time it is called, you simply return a cached value and eliminate double-calculation. Obviously you need to be careful that this isn't called before an instance is updated, otherwise the old number_as_char will be cached. This should be fine as long as get_number_as_char() is only called during a save/clean.
Based on the information you've provided above you shouldn't have to iterate over all the ancestors, but can simply take the number_as_char for the parent and append to it.
The following incorporates both:
#cached_property
def get_number_as_char(self, parent_account, number_suffix):
number_as_char = str(number_suffix)
if parent_account is not None:
number_as_char = '{}-{}'.format(parent_account.number_as_char, number_as_char)
return number_as_char
To be sure that the caching doesn't cause problems you could just clear the cached value after you're done saving:
def save(self, *args, **kwargs):
self.number_as_char = self.get_number_as_char(
self.parent_account, self.number_suffix)
super().save(*args, **kwargs)
# Clear the cache, in case something edits this object again.
del self.get_number_as_char

I tinkered with it a bit, and I think I found a better way.
By using the disabled property on the number_as_char field of your model form, you can entirely ignore users input (and make the field disabled in a single step).
Your model already calculates the number_as_char attribute in the save method. However, if the Unique constraint fails, then your admin UI will throw a 500 error. However, you can move your field calculation to the clean() method, leaving the save() method as it is.
So the full example will look similar to this:
The form:
class AccountForm(forms.ModelForm):
class Meta:
model = Account
fields = [
'parent_account', 'number_suffix', 'number_as_char',
]
widgets = {
'number_as_char': forms.TextInput(attrs={'disabled': True}),
}
The model:
class Account(models.Model):
# ...
def clean(self):
self.number_as_char = self.get_number_as_char(
self.parent_account, self.number_suffix
)
super().clean()
That way anything that generates form based on your model will throw a nice validation error (provided that it uses the built-in model validation, which is the case for Model Forms).
The only downside to this is that if you save a model that triggers the validation error, you will see an empty field instead of the value that failed the validation - but I guess there is some nice way to fix this as well - I'll edit my answer if I also find a solution to this.

After reading all the answers and doing some more digging through the docs, I ended up using the following:
#samu suggested using the models clean() method and #Laurent S suggested using unique_together for (parent_account, number_suffix). Since only using unique_together doesn't work for me because parent_account can be null, I opted for combining the two ideas: checking for existing (parent_account, number_suffix) combinations in the models clean() method.
As a consecuence, I removed number_as_char from the form and it is now only calculated in the save() method. By the way: thanks to #solarissmoke for suggesting to calculated it based on the first parent only, not iterating all the way to the top of the chain.
Another consecuence is that I now need to explicitly call the models full_clean() method to validate uniqueness when using the model without the form (otherwise I will get the database IntegrityError), but I can live with that.
So, now my model looks like this:
class Account(models.Model):
parent_account = models.ForeignKey(
to='self',
on_delete=models.PROTECT,
null=True,
blank=True)
number_suffix = models.PositiveIntegerField()
number_as_char = models.CharField(
max_length=100,
default='0',
unique=True)
def save(self, *args, **kwargs):
if self.parent_account is not None:
self.number_as_char = '{}-{}'.format(
self.parent_account.number_as_char,
self.number_suffix)
else:
self.number_as_char = str(self.number_suffix)
super().save(*args, **kwargs)
def clean(self):
qs = self._meta.model.objects.exclude(pk=self.pk)
qs = qs.filter(
parent_account=self.parent_account,
number_suffix=self.number_suffix)
if qs.exists():
raise ValidationError('... some message ...')
And my form ends up like this:
class AccountForm(forms.ModelForm):
class Meta:
model = Account
fields = [
'parent_account', 'number_suffix',
]
EDIT
I'll mark my own answer as accepted, because non of the suggestions fully suited my needs.
However, the bounty goes to #samus answer for pointing me in the right direction with using the clean() method.

Another way - probably not as good though - would be to use Django signals. You could make a pre_save signal that would set the correct value for number_as_char field on the instance that's about to get saved.
That way you don't have to have it done in a save() method of your model, OR in the clean() method of your ModelForm.
Using signals should ensure that any operation that uses the ORM to manipulate your data (which, by extend, should mean all ModelForms as well) will trigger your signal.
The disadvantage to this approach is that it is not clear from the code directly how is this property generated. One has to stumble upon the signal definition in order to discover that it's even there. If you can live with it though, I'd go with signals.

How to delete many to many when unassociated in Django?

We have a tagging system for users to filter their files by defined tags.
Here's how the models are set up:
class Tags(models.Model):
name = models.CharField(max_length=100)
user = models.ForeignKey(User)
class Files(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=100)
tags = models.ManyToManyField(Tags, null=True, blank=True)
Now, because tags are not required, when we remove tags from a file they don't get deleted. This leaves a bunch of tags saved on our database and we want to clean them up.
I've tried redefining the save method on the Files model, and the clean method.
I've tried connecting an m2m_changed signal on the Files model: https://docs.djangoproject.com/en/dev/ref/signals/#m2m-changed
Last thing I tried was a pre_save signal: https://docs.djangoproject.com/en/dev/ref/signals/#pre-save
I was planning to iterate over the tags and delete the ones with empty files_set, but using these methods I can't reliably figure that out (i.e. I end up removing tags that aren't associated but are about to be associated (because m2m_changed fires several times with different actions)).
Here's what I thought would work:
def handle_tags (sender, instance, *args, **kwargs) :
action = kwargs.get('action')
if action == 'post_clear':
# search through users tags... I guess?
tags = Tags.objects.filter(user=instance.user)
for tag in tags:
if not tag.files_set.exists():
tag.delete()
return
m2m_changed.connect(handle_tags, sender=Files.tags.through)
But, as I said, it will delete a tag before it is added (and if it's added, we obviously don't want to delete it).

You we're on the right track when using the m2m_changed signal.
Your problem is that when responding to the post_clear signal the tags have already been deleted so you won't be able to access them like that.
You actually need to dispatch your method before the tags are deleted, which means handling the pre_clear signal.
Something like this:
#receiver(m2m_changed, sender=Files.tags.through)
def handle_tags(sender, **kwargs):
action = kwargs['action']
if action == "pre_clear":
tags_pk_set = kwargs['instance'].tags.values_list('pk')
elif action == "pre_remove":
tags_pk_set = kwargs.get('pk_set')
else:
return
# I'm using Count() just so I don't have to iterate over the tag objects
annotated_tags = Tags.objects.annotate(n_files=Count('files'))
unreferenced = annotated_tags.filter(pk__in=tags_pk_set).filter(n_files=1)
unreferenced.delete()
I've also added the handling of the pre_remove signal in which you can use the pk_set argument to get the actual tags that will be removed.
UPDATE
Of course the previous listener won't delete the unreferenced tags when deleting the files, since it's only handling the pre_clear and pre_remove signals from the Tags model. In order to do what you want, you should also handle the pre_delete signal of the Files model.
In the code below I've added an utility function remove_tags_if_orphan, a slightly modified version of handle_tags and a new handler called handle_file_deletion to remove the tags which will become unreferenced once the File is deleted.
def remove_tags_if_orphan(tags_pk_set):
"""Removes tags in tags_pk_set if they're associated with only 1 File."""
annotated_tags = Tags.objects.annotate(n_files=Count('files'))
unreferenced = annotated_tags.filter(pk__in=tags_pk_set).filter(n_files=1)
unreferenced.delete()
# This will clean unassociated Tags when clearing or removing Tags from a File
#receiver(m2m_changed, sender=Files.tags.through)
def handle_tags(sender, **kwargs):
action = kwargs['action']
if action == "pre_clear":
tags_pk_set = kwargs['instance'].tags.values_list('pk')
elif action == "pre_remove":
tags_pk_set = kwargs.get('pk_set')
else:
return
remove_tags_if_orphan(tags_pk_set)
# This will clean unassociated Tags when deleting/bulk-deleting File objects
#receiver(pre_delete, sender=Files)
def handle_file_deletion(sender, **kwargs):
associated_tags = kwargs['instance'].tags.values_list('pk')
remove_tags_if_orphan(associated_tags)
Hope this clears things up.

Just to sum up with hopefully a cleaner answer:
from django.db.models.signals import m2m_changed
from django.dispatch import receiver
class Tags(models.Model):
name = models.CharField(max_length=100)
user = models.ForeignKey(User)
class Files(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=100)
tags = models.ManyToManyField(Tags, null=True, blank=True)
#receiver(m2m_changed, sender=Files.tags.through)
def delete_orphean_dateranges(sender, **kwargs):
if kwargs['action'] == 'post_remove':
Tags.objects.filter(pk__in=kwargs['pk_set'], files_set=None).delete()
post_remove ensure that the callback is fired when a Tag was disassociated from a File

I think you go deeper than it required. Just define related_name for Tag, and process post_save signal from File.
class Files(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=100)
tags = models.ManyToManyField(Tags, null=True, blank=True, related_name='files')
def clean_empty_tags(sender, instance, *args, **kwargs):
Tags.objects.filter(user=instance.user, files=None).delete()
post_save.connect(clean_empty_tags, sender=Files)

How to use RelatedManager add() method in save()

Banging head against the wall again.
I'm trying to add tags using other known fields
# models.py
class MyModel(models.Model):
...
tags = models.ManyToManyField(Tag, blank=True)
field_m2m = models.ManyToManyField('M2mModel', blank=True)
field_fk = models.ForeignKey('FkModel', blank=True, null=True)
...
def save(self, *args, **kwargs):
for inst in self.field_m2m.all():
self.tags.add(Tag.objects.get(name=inst.name))
self.tags.add(Tag.objects.get(name=self.field_fk.name))
super(MyModel, self).save(*args, **kwargs)
class FkModel(models.Model):
name = models.CharField(max_length=255, unique=True)
...
class M2mModel(models.Model):
name = models.CharField(max_length=255, unique=True)
...
I am 100% sure my field_m2m and field_fk aren't empty and what's not less important: there are instances corresponding to EXISTING tags. I have other functions covering this part well. I have also tried hardcoding the strings (Tag.objects.get(name="mystring")) to be 101% sure.
Yet, no tags are assigned through admin panel.
I tried to go through the steps in shell and it works there.
>>> m = MyModel.objects.get(name='something')
>>> t = Tag.objects.get(name='sometag')
>>> m.tags.add(t)
>>> m.tags.all()
[<Tag: sometag>]
How to make it work from save() method?
Also until the the model instance is created for the first time, traceback is complaining about: "<MyModel: Something>" needs to have a value for field "mymodel" before this many-to-many relationship can be used.
I guess I should save the model instance before even doing aforementioned assignments, right? How can I do it all at once?

Seems to me that your MyModel instance must be saved into database before saving any relationships. It makes sense because for the relationships, the MyModel's id is needed. So you can change the order of the save method like this:
def save(self, *args, **kwargs):
# notice that super class save go first.
super(MyModel, self).save(*args, **kwargs)
# also this cicle doesn't make sense since self is a newly
# created instance so it won't have anythin in field_m2m.all()
for inst in self.field_m2m.all():
self.tags.add(Tag.objects.get(name=inst.name))
# this should work ok if get returns any tag.
self.tags.add(Tag.objects.get(name=self.field_fk.name))
Hope this helps!

Figured it out thanks to this hint: https://stackoverflow.com/a/6200451/1344854
Model save() method stays default. First I tied a tag to my FkModel and M2mModel instances. The rest of the job is done in ModelAdmin.
# models.py
class FkModel(models.Model):
name = models.CharField(max_length=255, unique=True)
tag = models.ForeignKey(Tag, blank=True, null=True, related_name='fk')
...
class M2mModel(models.Model):
name = models.CharField(max_length=255, unique=True)
tag = models.ForeignKey(Tag, blank=True, null=True, related_name='m2m')
...
# admin.py
class MyModelAdmin(admin.ModelAdmin):
def save_model(self, request, obj, form, change):
form.save() # as I learned, without saving at this point, obj.field_m2m.all() will be empty
tags = []
if obj.field_m2m.all():
tags += [m2m.tag for m2m in obj.field_m2m.all()]
if obj.field_fk:
tags += [obj.field_fk.tag]
form.cleaned_data['tags'] = tags
super(MyModelAdmin, self).save_model(request, obj, form, change)

Getting current logged_in user in Django

I'm developping a blog with Django, and I'd like my posts to be related to their author. I've created an author field in my Blog model class, which is a ForeignKey pointing on a User object. Problem is, South database migration django tool asks me for a default value to give to it. Actually, I'd like it to be the id of the logged in user who edited the post, but can't see how to make it. Any idea?
This is my Blog model class:
class Blog(models.Model):
PUBLISHING_STATUS = (
('P', 'Published'),
('D', 'Draft'),
('AA', 'Awaiting Approval'),
)
title = models.CharField(max_length=128, unique=True)
slug = models.SlugField(max_length=160, unique=True)
header_image = models.ImageField(upload_to='images/uploaded', max_length=256, null=True)
body = models.TextField()
author = models.ForeignKey(User)
status = models.CharField(max_length=2, choices=PUBLISHING_STATUS, null=True)
posted = models.DateTimeField('date posted')
category = models.ForeignKey('blog.Category')
tags = models.ManyToManyField('blog.Tags')
def __unicode__(self):
return self.title
def get_absolute_url(self):
return '/blog/%s/view_post' % self.slug
Thanks!

If you want the field to be required you will have to set a default value when migrating. For the reason you don't have stored any data anywhere about who created the post, you ca only set it to a specific user, which in this case probably should be your administrator user!
You could then edit the posts in the admin and set them to the correct author, if necessary!
If you are looking for a way to auto-populate the field upon saving a new post in the admin, you have to override the save_model method on your ModelAdmin:
class BlogPostAdmin(model_admin):
def save_model(self, request, instance, form, change):
user = request.user
instance = form.save(commit=False)
if not change or not instance.author:
instance.author = user
instance.save()
form.save_m2m()
return instance

Problem is, South database migration django tool asks me for a default value to give to it.
Why? Because you have old data that you're trying to preserve? Perhaps you shouldn't be trying to keep any old data.
Actually, I'd like it to be the id of the logged in user who edited the post, but can't see how to make it. Any idea?
That's what view functions are for. You don't do this in the model. You do this in the view function that handles the Form.

You should do this as a two step process. First, do a schema migration with any default value to create the new structure. Then, do a data migration to update the author column with the correct value from the user table.