We Keep Coding

Django models default function run for every current object

models.py
class Subscription(models.Model):
#... many fields ...
# I added this field when I already had many objects
uniqueSubscriptionId = models.CharField(default=generateUniqueSubscription, max_length=30)
generateUniqueSubscription
from django.utils.crypto import get_random_string
def generateUniqueSubscription():
return get_random_string(20)
The Problem is that, when I run migrations, all of my old objects get the same uniqueSubscriptionId. I want each and every single old object to get a unique uniqueSubscriptionId.
How can I do that?

Here's what I did:
models.py
def updateOldSubscriptionObjs(apps, schema_editor):
old_subscription_model = apps.get_model("app_label", "Profile")
for obj in old_subscription_model.objects.all():
obj.uniqueSubscriptionId = generateUniqueSubscription()
obj.save()
class Subscription(models.Model):
#... many fields ...
# I added this field when I already had many objects
uniqueSubscriptionId = models.CharField(default=generateUniqueSubscription, max_length=30)
Then I ran makemigrations:
python manage.py makemigrations
Then edited the latest migration file:
class Migration(migrations.Migration):
dependencies = [
# forget this
]
operations = [
# .... many things ...
migrations.RunPython(updateOldProfileObjs)
]
Then ran migrate:
python manage.py migrate
And voila, all old objects got updated, and also, any new object will also get updated as I specified default.
If you are lazy like me, and don't want to do these things, then open django python shell:
python manage.py shell
and then execute this function in shell:
def updateOldSubscriptionObjs():
for obj in Subscription.objects.all():
obj.uniqueSubscriptionId = generateUniqueSubscription()
obj.save()
I wish if there was some built-in django feature for this.

Extend django-import-export's import form to specify fixed value for each imported row

I am using django-import-export 1.0.1 with admin integration in Django 2.1.1. I have two models
from django.db import models
class Sector(models.Model):
code = models.CharField(max_length=30, primary_key=True)
class Location(models.Model):
code = models.CharField(max_length=30, primary_key=True)
sector = ForeignKey(Sector, on_delete=models.CASCADE, related_name='locations')
and they can be imported/exported just fine using model resources
from import_export import resources
from import_export.fields import Field
from import_export.widgets import ForeignKeyWidget
class SectorResource(resources.ModelResource):
code = Field(attribute='code', column_name='Sector')
class Meta:
model = Sector
import_id_fields = ('code',)
class LocationResource(resources.ModelResource):
code = Field(attribute='code', column_name='Location')
sector = Field(attribute='sector', column_name='Sector',
widget=ForeignKeyWidget(Sector, 'code'))
class Meta:
model = Location
import_id_fields = ('code',)
and import/export actions can be integrated into the admin by
from django.contrib import admin
from import_export.admin import ImportExportModelAdmin
class SectorAdmin(ImportExportModelAdmin):
resource_class = SectorResource
class LocationAdmin(ImportExportModelAdmin):
resource_class = LocationResource
admin.site.register(Sector, SectorAdmin)
admin.site.register(Location, LocationAdmin)
For Reasons™, I would like to change this set-up so that a spreadsheet of Locations which does not contain a Sector column can be imported; the value of sector (for each imported row) should be taken from an extra field on the ImportForm in the admin.
Such a field can indeed be added by overriding import_action on the ModelAdmin as described in Extending the admin import form for django import_export. The next step, to use this value for all imported rows, is missing there, and I have not been able to figure out how to do it.

EDIT(2): Solved through the use of sessions. Having a get_confirm_import_form hook would still really help here, but even better would be having the existing ConfirmImportForm carry across all the submitted fields & values from the initial import form.
EDIT: I'm sorry, I thought I had this nailed, but my own code wasn't working as well as I thought it was. This doesn't solve the problem of passing along the sector form field in the ConfirmImportForm, which is necessary for the import to complete. Currently looking for a solution which doesn't involve pasting the whole of import_action() into an ImportMixin subclass. Having a get_confirm_import_form() hook would help a lot here.
Still working on a solution for myself, and when I have one I'll update this too.
Don't override import_action. It's a big complicated method that you don't want to replicate. More importantly, as I discovered today: there are easier ways of doing this.
First (as you mentioned), make a custom import form for Location that allows the user to choose a Sector:
class LocationImportForm(ImportForm):
sector = forms.ModelChoiceField(required=True, queryset=Sector.objects.all())
In the Resource API, there's a before_import_row() hook that is called once per row. So, implement that in your LocationResource class, and use it to add the Sector column:
def before_import_row(self, row, **kwargs):
sector = self.request.POST.get('sector', None)
if contract:
self.request.session['import_context_sector'] = sector
else:
# if this raises a KeyError, we want to know about it.
# It means that we got to a point of importing data without
# contract context, and we don't want to continue.
try:
sector = self.request.session['import_context_sector']
except KeyError as e:
raise Exception("Sector context failure on row import, " +
f"check resources.py for more info: {e}")
row['sector'] = sector
(Note: This code uses Django sessions to carry the sector value from the import form to the import confirmation screen. If you're not using sessions, you'll need to find another way to do it.)
This is all you need to get the extra data in, and it works for both the dry-run preview and the actual import.
Note that self.request doesn't exist in the default ModelResource - we have to install it by giving LocationResource a custom constructor:
def __init__(self, request=None):
super()
self.request = request
(Don't worry about self.request sticking around. Each LocationResource instance doesn't persist beyond a single request.)
The request isn't usually passed to the ModelResource constructor, so we need to add it to the kwargs dict for that call. Fortunately, Django Import/Export has a dedicated hook for that. Override ImportExportModelAdmin's get_resource_kwargs method in LocationAdmin:
def get_resource_kwargs(self, request, *args, **kwargs):
rk = super().get_resource_kwargs(request, *args, **kwargs)
rk['request'] = request
return rk
And that's all you need.

Django Custom Migration Not Executing

So I added a new "status" field to a django database table. This field needed a default value, so I defaulted it to "New", but I then added a custom migration file that calls the save() method on all of the objects in that table, as I have the save() overridden to check a different table and pull the correct status from that. However, after running this migration, all of the statuses are still set to "New", so it looks like the save isn't getting executed. I tested this by manually calling the save on all the objects after running the migration, and the statuses are updated as expected.
Here's the table model in models.py:
class SOS(models.Model):
number = models.CharField(max_length=20, unique=True)
...
# the default="New" portion is missing here because I have a migration to remove it after the custom migration (shown below) that saves the models
status = models.CharField(max_length=20)
def save(self, *args, **kwargs):
self.status = self.history_set.get(version=self.latest_version).status if self.history_set.count() != 0 else "New"
super(SOS, self).save(*args, **kwargs)
And here is the migration:
# Generated by Django 2.0.5 on 2018-05-23 13:50
from django.db import migrations, models
def set_status(apps, schema_editor):
SOS = apps.get_model('sos', 'SOS')
for sos in SOS.objects.all():
sos.save()
class Migration(migrations.Migration):
dependencies = [
('sos', '0033_auto_20180523_0950'),
]
operations = [
migrations.RunPython(set_status),
]
So it seems pretty clear to me that I'm doing something wrong with the migration, but I matched it exactly to what I see in the Django Documentation and I also compared it to this StackOverflow answer, and I can't see what I'm doing wrong. There are no errors when I run the migrations, but the custom one I wrote does run pretty much instanteously, which seems strange, as when I do the save manually, it takes about 5 seconds to save all 300+ entries.
Any suggestions?
P.S. Please let me know if there are any relevant details I neglected to include.

When you run migrations and get Model from apps you can not use custom managers or custom save or create or something like that. This model only have the fields and that's all. If you want to achieve what you want you should add your logic into you migrations like this:
# comment to be more than 6 chars...
def set_status(apps, schema_editor):
SOS = apps.get_model('sos', 'SOS')
for sos in SOS.objects.all():
if sos.history_set.exists():
sos.status = sos.history_set.get(version=sos.latest_version).status
else:
sos.status = "New"
sos.save()

Django migration with uuid field generates duplicated values

I have a uuid field (not a primary key). The generated migration is:
from __future__ import unicode_literals
from django.db import migrations, models
import uuid
class Migration(migrations.Migration):
dependencies = [
....
]
operations = [
...
migrations.AddField(
model_name='device',
name='uuid',
field=models.UUIDField(default=uuid.uuid4, unique=True),
),
...
]
But when doing python manage.py migrate it is crashing with:
django.db.utils.IntegrityError: could not create unique index
"restaurants_device_uuid_key" DETAIL: Key
(uuid)=(f3858ded-b8e0-4ac0-8436-8a61b10efc73) is duplicated.
Strangely enough, the problem does not seem to occur with primary keys (which are maybe created by the database, and not internally by django?)
How can I add a uuid field, and make sure that migrations work?

Here is an example doing everything in one single migration thanks to a RunPython call.
# -*- coding: utf-8 -*
from __future__ import unicode_literals
from django.db import migrations, models
import uuid
def create_uuid(apps, schema_editor):
Device = apps.get_model('device_app', 'Device')
for device in Device.objects.all():
device.uuid = uuid.uuid4()
device.save()
class Migration(migrations.Migration):
dependencies = [
('device_app', 'XXXX'),
]
operations = [
migrations.AddField(
model_name='device',
name='uuid',
field=models.UUIDField(blank=True, null=True),
),
migrations.RunPython(create_uuid),
migrations.AlterField(
model_name='device',
name='uuid',
field=models.UUIDField(unique=True)
)
]

(Answer taken from the first comment)
See the django docs - Migrations that add unique fields
They recommend changing your single migration into three separate migrations:
Create field, set to null but not unique
Generate unique UUIDs
Alter the field to be unique

In the mode, you have configured, that you want unique values for the uuid fields, but with default values(the same for all). So if you have two 'device' objects in the database, the migrations add 'uuid' field to them with the default 'uuid.uuid4' value and when it tries to set it to the second one, it crashes because of the unique constrains.
If you drop your db and create new objects probably there will be not problems but thats not a solution for production db obviously :D.
A better solution is to create a data migration which sets different uuid value (generated by the default 'uuid' library) to every existing object in the database. You can read more about data migrations here:
https://docs.djangoproject.com/en/1.10/topics/migrations/#data-migrations
Then, when you create new objects, django will generate different uuid automatically. ;)
For the primary keys: Django adds it to the model by default.

You can provide a management command to populate the uuid field after uu_id column is created in the model but this has to be done after migrating the model and setting the field default as None:
from django.core.management.base import BaseCommand
from django.apps import apps
import uuid
class Command(BaseCommand):
def handle(self, *args, **options):
classes()
def classes():
app_models = apps.get_app_config('appname').get_models()
for model in app_models:
field = None
try:
field = model._meta.get_field('uu_id')
except:
pass
if field:
uu_id_list = list(model.objects.all().values_list('uu_id',flat=True))
if None in uu_id_list:
for row in model.objects.all():
row.uu_id = uuid.uuid4()
row.save()

Django 1.8 'default' option only executes external function once

I am trying to add a UUID field to an existing table. I specified that default = uuid.uuid4 however, it Django doesn't seem to call uuid.uuid4 function for every row. So when I migrate I keep getting duplicated uuid error.
My Django version is 1.8.2.
from django.db import models, migrations
import uuid
class Migration(migrations.Migration):
dependencies = [
('conv', '0008_video_video_uri'),
]
operations = [
migrations.AddField(
model_name='conversation',
name='channel_id',
field=models.UUIDField(unique=True, default=uuid.uuid4, editable=False),
),
]
Below the error:
> > File "/home/yonk/projects/trailerapp/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py",
> line 64, in execute
> return self.cursor.execute(sql, params) django.db.utils.IntegrityError: could not create unique index
> "conv_conversation_channel_id_68f7d58df7c78d61_uniq" DETAIL: Key
> (channel_id)=(5f512cbe-e514-4bf5-bf5a-3efd1a94e401) is duplicated.

Here you have django docs describing exactly what you want:
https://docs.djangoproject.com/en/1.8/howto/writing-migrations/#migrations-that-add-unique-fields
You will need two migration files.
First one Adds fields, also change unique=True to null=True so django won't try to use default value...
Second migration populates the field.
So second migration should look like this:
def gen_uuid(apps, schema_editor):
MyModel = apps.get_model('myapp', 'MyModel')
for row in MyModel.objects.all():
row.uuid = uuid.uuid4()
row.save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0004_add_uuid_field'),
]
operations = [
# omit reverse_code=... if you don't want the migration to be reversible.
migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
]

In order to get a new value each time, you will need to set the default to a callable, otherwise as you have observed, uuid.uuid4 will be calculated once and then that single value will be used each time.
This other StackOverflow question and answer shows how to pass a callable as the default value.
EDIT: This answer only applies to versions of Django 1.7 and lower.