We Keep Coding

Best practice when add a new unique field to an existing django model

I have an existing model that looks somewhat like the following...
class Resource(models.Model):
id = models.AutoField(primary_key=True)
We have been using this for some time, and now have ~1M instances of these Resource objects (and associated ForeignKey/else usages) in our database.
I now have a need to track another ID on this model, one that I want to enforce is unique.
other_id = models.IntegerField(unique=True)
This other_id information is currently stored in some external CSVs, and I want to (at some point in the process) load this information in to all existing Resource instances.
After adding the above field, Django's makemigrations works just fine. However when I go to apply said migration against an existing database I get an error indicating that I need to provide a default to use for all existing Resource instances. I'm sure many of you have seen something similar.
What is the best approach to getting around this limitation? Some methods I have thought of...
Remove the unique=True requirement
apply the migration
externally load in the other_id value to all existing models (through some management command, or 1-off script)
add the unique=True back in and apply the migration
Dump all existing data to JSON
flush all tables
apply the migration (with unique=True)
write a script that loads the data back in, adding the correct other_id value
(unsure if this is possible) - Write some custom migration logic to automatically reference these external CSVs to load other_id values when I run manage.py migrate. This could hit issues if (at some point in the future) someone re-runs these migrations and this part fails (cannot find existing resource id in the CSVs to pull out other_id).
All of these feel complicated, but then again I guess what I am trying to do isn't the simplest thing either.
Any ideas? I have to imagine someone has had to work around a similar issue in the past.
Thanks!

Actually, the source or your issue is not the unique constraint by itself but the fact that your field doesn't allow nulls and has no default value - you'd have the very same error with a non-unique field.
The proper solution here is to allow the field to be null (null=True) and default it to None (which will translate to sql "null"). Since null values are excluded from unique constraints (at least if your db vendor respects SQL standard), this allow you to apply the schema change while still making sure you cannot have a duplicate for non-null values.
Then you may want a data migration to load the known "other_id" values, and eventually a third schema migration to disallow null values for this field - if and only if you know you have filled this field for all records.

Django has something called Data Migrations where you create a migration file that modifies/remove/add data to your database as you apply your migrations.
In this case you would create 3 different migrations:
Create a migration that allow null values with null=True.
Create a data migration that populate the data.
Create a migration that disallow null values by removing the null=True added in step 1.
As you then run python manage.py migrate it would apply all of the migrations in step 1-3 in the correct order.
Your data migration would look something like this:
from django.db import migrations
def populate_reference(apps, schema_editor):
MyModel = apps.get_model('yourappname', 'MyModel')
for obj in MyModel.objects.all():
obj.other_id = random_id_generator()
obj.save()
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(populate_reference),
]
You can create an empty migration file using the ./manage.py makemigrations --empty yourappname command.

Django automatically create primary keys for existing database tables

I have an existing database that I'm trying to access with Django. I used python manage.py inspectdb to create the models for the database. Currently I'm able to import the models into the python shell however when I try to access any of the actual objects in any way, I get this error OperationalError: (1054, "Unknown column 'some_table.id' in 'field list'"). I see that the table in the database in fact does not have an id field. How can I fix this? Do I need to update the managed field in the Meta class and run a migration so it can create this field automatically?

From the Django documentation: This feature is meant as a shortcut, not as definitive model generation. See the documentation of inspectdb for more information. (Reference: https://docs.djangoproject.com/en/1.8/howto/legacy-databases/)
You're going to need to manually clean up the models and migrate. The line you'll have to add for adding the "id" field is:
id = models.IntegerField(primary_key=True)
Warning: I'd definitely create a copy of the database to toy with, rather than the original. This will likely take you some trial and error to get right. After you're absolutely sure you have it right, you can changed Managed=True, but be VERY careful!

members_data.dob may not be NULL in Django

Using Django 1.4 in my app I defined a model called Member and another called Data.Every member has basic like ID and it is related to a Data object that contains additional variables describing the "member".
I had initially created the member model without specifying that the dob variable could be NULL. I have since then changed this to allow for blank or null but I was still getting the members_data.dob may not be NULL error.
I thought it was because I needed to run a syncdb command, which I did, however this did not fix the problem.
dob = models.CharField(max_length=200, blank=True, null=True)
Any ideas? Thanks
ps. If you want to get an overall picture of what I am trying to implement please refer to: Can I use JSON data to add new objects in Django?
Thanks so much.

The syncdb command only creates tables if they do not exist. It does not handle migrations for you. You have a few options:
If there is no important data in the table, drop the table and run syncdb again to recreate it.
Update the column to allow null in a db shell. The correct command depends on which database you are using.
Use a migration tool, like South.
To drop the table in sqlite:
Open a dbshell
./manage.py dbshell
Drop the table
drop table <tablename>

The field does not have a default specified, yet is NOT NULL

I have no idea what Django is trying to tell me. I have a model, WeekTwo, which inherits from Week, which inherits from modelsModel. I have another model, UserProfile. I want to use WeekTwo as a OneToOne Key in UserProfile, so I inserted the following line of code:
weekTwo = models.OneToOneField(WeekTwo)
However, when I try to migrate my database using python manage.py schemamigration my_app --auto, I get the following error:
The field 'UserProfile.weekTwo' does not have a default specified, yet is NOT NULL.
I tried adding default=0 to my weekTwo declaration, but now I'm getting this error when I try the schema migration:
IntegrityError: column weekTwo_id is not unique
Moreover, south is now telling me that I am in an interim state between migrations and that I might be able to recover. I literally have no idea what any of this means.

Be aware before going further, be sure that if South already did any migration that has been failed, It's better to redo|recover to the last migration that was working.
You have two options here, at first you can do Data Migration. Look at ref also.
In Second way You can make weekTwo field null and blank first
weekTwo = models.OneToOneField(WeekTwo, null=True, blank=True)
Then let South generate a migration for you by
python manage.py schemamigration my_app --auto
I'm sure South won't complain about it now, Then
python manage.py migrate
If everything is okay now, You can now get back and change to weekTwo field to
weekTwo = models.OneToOneField(WeekTwo)
And generate migration then migrate them.
Anyway, When south find out your field in not NULL and doesn't have a default value, at the schemamigration step it will suggest you to provide a value for it, Again here your field is OneToOneField, because even south gives you chance to provide a default value for your existed record on model again uniqueness of weekTwo field will raise an error.
I think still you have to go with Data Migration if second way didn't work, Or gives it a shot and try the second way this time instead of make it null, blank change the whole Field type. Try it with;
weekTwo = models.ForeignKey(WeekTwo)
But keep in mind Data migration would be definitely smarter and standard way here.

Update Field schema in Django Model

I'm trying to update a field key from the following in the model:
name = models.CharField(max_length=255, unique = True)
to:
name = models.CharField(max_length=255, unique = False)
However I need to update the mysql db without losing any of the current data. How would I go about doing this? I tried $ python manage.py syncdb but it doesn't seem to update the key.

That is because syncdb doesn't do that. It only creates new tables, but doesn't change the existing if your model changes.
On a dev environment, to update your database after changing your model, you can use python manage.py reset appname to empty out your database, and use syncdb again.
Otherwise, you have to use tools such as South. South was made to handle migrations, but also changing the database when the model changes.
There is also django-evolutions that does just what you want, but I still recommend South as its migrations features are almost always going to be of some use.

You could use PHPMyAdmin and remove the unique index from this field.