Related
Previously we made integer field like:
cart_id = models.IntegerField(_('cart_id'), null=True)
But now I want to make this field foreign key:
cart = models.ForeignKey(Cart, null=True, db_column='cart_id')
The problem is that in the migration it generates two operations for deleting field and creating new one:
operations = [
migrations.RemoveField(
model_name='order',
name='cart_id',
),
migrations.AddField(
model_name='order',
name='cart',
field=models.ForeignKey(db_column=b'cart_id', to='cart.Cart', null=True),
preserve_default=True,
),
]
Is there any way to make it as alter field?
First add the ForeignKey. Set the default blank=True and run migrations.
Then run this code to fill the previous instances (python manage.py shell):
m = Order.objects.all()
for i in m:
c = Cart.object.get(id=i.cart_id)
i.cart = c
i.save()
Once done check if the ForeignKey is filled in the admin.
You can remove blank=True in ForeignKey it is a required field.
My current project uses multi-table inheritance models:
from django.db import models
class Place(models.Model):
name = models.CharField(max_length=50)
address = models.CharField(max_length=80)
class Restaurant(Place):
serves_hot_dogs = models.BooleanField(default=False)
serves_pizza = models.BooleanField(default=False)
class Cinema(Place):
sells_tickets = models.BooleanField(default=False)
sells_popcorn = models.BooleanField(default=False)
I want to switch to abstract base classes instead. Since my model is already deployed I need to write some custom migrations to convert the above schema to this one:
from django.db import models
class AbstractPlace(models.Model):
name = models.CharField(max_length=50)
address = models.CharField(max_length=80)
class Meta:
abstract = True
class Restaurant(AbstractPlace):
serves_hot_dogs = models.BooleanField(default=False)
serves_pizza = models.BooleanField(default=False)
class Cinema(AbstractPlace):
sells_tickets = models.BooleanField(default=False)
sells_popcorn = models.BooleanField(default=False)
Does anyone have any advice on the steps to take to achieve this?
I recently tackled this exact problem, which I solved by writing and running the migration in the code block below - loosely translated to fit the models in your case.
I'm pretty sure that it's not possible to alter the tables of the old Restaurant and Cinema models directly, as if you try to add fields to them, they will collide with the existing fields of the base model, and if you try to "decouple" the derived models from the base model by e.g. by manually setting abstract=True in the base model's options, Django reports that it's unable to find the base models of Restaurant and Cinema. (These issues might be caused by a bug, for all I know.) To circumvent this problem, I created new tables for the derived models, copied the data from the old tables to the new ones, deleted the old tables, and renamed the new tables to match the names of the old ones.
I got large parts of the code below from code generated by Django, which can be reproduced by creating a temporary migration (before creating one with the code below) which only deletes Restaurant, Cinema and Place, running makemigrations, and copying the CreateModel()s and AlterField()s (for related fields pointing to Restaurant or Cinema) from the generated migration.
For the record, I'm using Django 3.1.4.
from django.db import migrations, models
def copy_objects_from_restaurant_and_cinema_to_restaurant_tmp_and_cinema_tmp(apps, schema_editor):
Restaurant_Tmp = apps.get_model('<app name>', 'Restaurant_Tmp')
Cinema_Tmp = apps.get_model('<app name>', 'Cinema_Tmp')
Restaurant = apps.get_model('<app name>', 'Restaurant')
Cinema = apps.get_model('<app name>', 'Cinema')
# The `_meta.fields` list includes the PK
copy_objects_from_old_model_to_new_model(Restaurant, Restaurant_Tmp, Restaurant_Tmp._meta.fields)
copy_objects_from_old_model_to_new_model(Cinema, Cinema_Tmp, Cinema_Tmp._meta.fields)
def copy_objects_from_old_model_to_new_model(old_model, new_model, fields_to_copy):
field_names = [field.name for field in fields_to_copy]
for old_obj in old_model.objects.all():
old_obj_field_dict = {
field_name: getattr(old_obj, field_name)
for field_name in field_names
}
new_model.objects.create(**old_obj_field_dict)
def copy_objects_from_restaurant_tmp_and_cinema_tmp_to_restaurant_and_cinema(apps, schema_editor):
Restaurant_Tmp = apps.get_model('<app name>', 'Restaurant_Tmp')
Cinema_Tmp = apps.get_model('<app name>', 'Cinema_Tmp')
Restaurant = apps.get_model('<app name>', 'Restaurant')
Cinema = apps.get_model('<app name>', 'Cinema')
copy_objects_from_old_model_to_new_model(Restaurant_Tmp, Restaurant, Restaurant_Tmp._meta.fields)
copy_objects_from_old_model_to_new_model(Cinema_Tmp, Cinema, Cinema_Tmp._meta.fields)
class Migration(migrations.Migration):
dependencies = [
('<app name>', '<last migration>'),
]
operations = [
migrations.CreateModel(
name='Restaurant_Tmp',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(max_length=50)),
('address', models.CharField(max_length=80)),
('serves_hot_dogs', models.BooleanField(default=False)),
('serves_pizza', models.BooleanField(default=False)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='Cinema_Tmp',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(max_length=50)),
('address', models.CharField(max_length=80)),
('sells_tickets', models.BooleanField(default=False)),
('sells_popcorn', models.BooleanField(default=False)),
],
options={
'abstract': False,
},
),
migrations.RunPython(copy_objects_from_restaurant_and_cinema_to_restaurant_tmp_and_cinema_tmp, migrations.RunPython.noop),
# Update foreign keys to reference the non-abstract models directly,
# instead of through the (automatically generated) `place_ptr` field of the old models
<
Run `migrations.AlterField()` here for each related field (like ForeignKey) of other models that point to Restaurant or Cinema,
but change their `to` argument from e.g. `<app name>.restaurant` to `<app name>.restaurant_tmp`
>
migrations.RunPython(migrations.RunPython.noop, copy_objects_from_restaurant_tmp_and_cinema_tmp_to_restaurant_and_cinema),
migrations.DeleteModel(
name='Restaurant',
),
migrations.DeleteModel(
name='Cinema',
),
migrations.DeleteModel(
name='Place',
),
migrations.RenameModel(
old_name='Restaurant_Tmp',
new_name='Restaurant',
),
migrations.RenameModel(
old_name='Cinema_Tmp',
new_name='Cinema',
),
]
Note that the migration I originally wrote was only tested to work using SQLite; other database management systems might not accept such a large variety of migration operations, and you might have to split it into multiple migrations. (I'm somewhat unsure what exactly could cause this problem, but I can recall that I've experienced it with PostgreSQL.)
Please let me know if this solves your problem! 😊
I have a Django models with a duplicate field name trading I didn't noticed at the time I ran makemigration followed by the migrate command (both commands didn't throw any error). As you can see in the model.py below, the name "trading" is used for a BooleanField and a CharField.
But now I would like to remove or rename one of them and Django throws an error saying:
django.db.utils.DataError: invalid input syntax for type boolean: "future"
I'm not familiar with Postgres SQL and have no clue how to solve this problem without destroying and recreating the databse from scratch.
class Account(models.Model):
exchange = models.ForeignKey(Exchange, on_delete=models.CASCADE, related_name='account', null=True)
strategy = models.ForeignKey(Strategy, on_delete=models.CASCADE, related_name='account', null=True)
limit_order, credentials, trading = [models.BooleanField(default=True) for i in range(3)]
trading = models.CharField(max_length=12, null=True, blank=True, choices=[('future', 'future'),
('swap', 'swap')
])
I've try to first rename the CharField from trading to instrument but in the migration file it looks like Django wants to add a new field. The problem is that everything I modify in my models generate an error. How can I fix this issue while preserving the database?
This is the first migration file that generated the error:
# Generated by Django 3.0.6 on 2020-07-16 10:11
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('trading', '0011_auto_20200714_0758'),
]
operations = [
migrations.AddField(
model_name='account',
name='instrument',
field=models.CharField(blank=True, choices=[('future', 'future'), ('swap', 'swap')], max_length=12, null=True),
),
migrations.AlterField(
model_name='account',
name='trading',
field=models.BooleanField(default=True),
),
]
Thank you
I answer my own question since I successfully fixed the problem by:
keep the first migration file that caused the problem and removing the others
remove all reference to the duplicate fields in my models
then modify the file as bellow
From :
operations = [
migrations.AddField(
model_name='account',
name='instrument',
field=models.CharField(blank=True, choices=[('future', 'future'), ('swap', 'swap')], max_length=12, null=True),
),
migrations.AlterField(
model_name='account',
name='trading',
field=models.BooleanField(default=True),
),
]
To :
operations = [
migrations.RemoveField(
model_name='account',
name='trading',
),
]
How can I stop Django 2.2.4 from trying to create a database column that already exists when making a model managed?
I have 2 models, ticket and message, which were connected to tables in a third-party database so the models were created with managed=False. I'm moving away from the third-party tool. The ticket model was change to managed=True a while ago by somebody else, and now I'm trying to do the same with the message model.
These are the relevant parts of the model:
from django.db import models
class Message(models.Model):
mid = models.BigAutoField(db_column='MID', primary_key=True)
ticket = models.ForeignKey('Ticket', on_delete=models.CASCADE, db_column='TID')
author = models.CharField(db_column='AUTHOR', max_length=32)
date = models.DateTimeField(db_column='DATE')
internal = models.CharField(db_column='INTERNAL', max_length=1)
isoper = models.CharField(db_column='ISOPER', max_length=1)
headers = models.TextField(db_column='HEADERS')
msg = models.TextField(db_column='MSG')
class Meta:
# managed = False
db_table = 'messages'
permissions = (
("can_change_own_worked_time", "Can change own worked time"),
("can_change_own_recently_worked_time", "Can change own recently worked time"),
("can_change_subordinate_worked_time", "Can change subordinate worked time"),
)
This are the migrations that get generated by commenting out managed=False:
# Generated by Django 2.2.4 on 2020-06-18 20:56 (0017_auto_20200618_1656)
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('troubleticket', '0016_auto_20200511_1644'),
]
operations = [
migrations.AlterModelOptions(
name='message',
options={'permissions': (('can_change_own_worked_time', 'Can change own worked time'), ('can_change_own_recently_worked_time', 'Can change own recently worked time'), ('can_change_subordinate_worked_time', 'Can change subordinate worked time'))},
),
]
# Generated by Django 2.2.4 on 2020-06-18 21:14 (0018_message_ticket)
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('troubleticket', '0017_auto_20200618_1656'),
]
operations = [
migrations.AddField(
model_name='message',
name='ticket',
field=models.ForeignKey(db_column='TID', default=1, on_delete=django.db.models.deletion.CASCADE, to='troubleticket.Ticket'),
preserve_default=False,
),
]
When I try to apply those migrations I get this error:
django.db.utils.OperationalError: (1060, "Duplicate column name 'TID'")
The initial migration didn't include the TID column and neither do any of the subsequent migrations so I understand why Django thinks it's a new column. But it isn't a new column (the model has had it since the first time it was committed to the git repo) so I also understand why MySQL is throwing an error.
This is the initial migration:
# Generated by Django 2.0.8 on 2018-08-20 14:43 (0001_initial)
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = [
]
operations = [
migrations.CreateModel(
name='Message',
fields=[
('mid', models.BigAutoField(db_column='MID', primary_key=True, serialize=False)),
('author', models.CharField(db_column='AUTHOR', max_length=32)),
('date', models.DateTimeField(db_column='DATE')),
('internal', models.CharField(db_column='INTERNAL', max_length=1)),
('isoper', models.CharField(db_column='ISOPER', max_length=1)),
('headers', models.TextField(db_column='HEADERS')),
('msg', models.TextField(db_column='MSG')),
],
options={
'managed': False,
'db_table': 'messages',
},
),
migrations.CreateModel(
name='Ticket',
fields=[
('id', models.BigIntegerField(db_column='ID', primary_key=True, serialize=False)),
('accesskey', models.CharField(db_column='ACCESSKEY', max_length=64)),
('open', models.DateTimeField(db_column='OPEN')),
('updated', models.DateTimeField(db_column='UPDATED')),
('closed', models.DateTimeField(db_column='CLOSED', null=True)),
('status', models.CharField(db_column='STATUS', max_length=3)),
('oper', models.CharField(db_column='OPER', max_length=32)),
('email', models.CharField(db_column='EMAIL', max_length=128)),
('name', models.CharField(db_column='NAME', max_length=128)),
('subject', models.CharField(db_column='SUBJECT', max_length=255)),
('lname', models.CharField(db_column='LNAME', max_length=50)),
('company', models.CharField(db_column='C0', max_length=255)),
('type', models.CharField(db_column='C1', max_length=255)),
('c2', models.CharField(db_column='C2', max_length=255)),
('c3', models.DecimalField(db_column='C3', decimal_places=2, max_digits=6)),
('c4', models.CharField(db_column='C4', max_length=255)),
('pending', models.CharField(db_column='C5', max_length=255)),
('c6', models.CharField(db_column='C6', max_length=255)),
('c7', models.CharField(db_column='C7', max_length=255)),
('c8', models.CharField(db_column='C8', max_length=255)),
('cc', models.CharField(db_column='C9', max_length=255)),
('grp', models.CharField(db_column='GRP', max_length=10)),
('item', models.CharField(db_column='ITEM', max_length=255)),
],
options={
'managed': False,
'db_table': 'tickets',
},
),
]
2 Seconds after writing my comment I figured out how to solve the problem.
When 'migrating' from another ORM to Django always consider the following aspects.
The following is my recommendation of order, but I'm still learning how to use django migrations, so keep that in mind.
1. Consider which fields of the model are currently known to django
It is important to note, that the actual content of the database does not matter to django when calculating which columns to add or alter. This means that, in the initial migration of the django models, where managed is still =False, the columns are "created" and registered by django. When makemigrations calculates which columns to add, it only takes the columns mentioned in the initial migration as given. Not the actual contents of the db.
Knowing that, we can now go field for field and decide for each one according to point 2.
2. Consider which fields of the model should or should not be created by django
Now, generally, when setting a Model managed=True fields will fall into 3 categories.
2.1 The field is currently in the model definition, the database, and is also in the initial commit.
For this case, no actions need to be taken.
2.2 The field is currently in the model definition, the database, but is not in the initial commit.
For this case, the field definition must be added to the original initial migration script. As such, the migrations.CreateModel call might look like this:
...
('field_already_present', models.FloatField(blank=True, null=True)),
...
2.3 The field is currently in the database, but not in the model definition or the initial commit
If the field is not needed in the django application it can be left out.
If it is needed at some point it will have to be added to the model as well as the first initial migration. That way, django makemigrations will not attempt to create the field.
3. Set managed=True
Now set managed=True and make your migrations. Do so twice! The first time, makemigrations will set the model to managed, then it will add fields not in the initial commit.
After this, the model can be treated same as a normally created and managed-from-the-start django model.
I have a very similar situation like this: Django migration strategy for renaming a model and relationship fields
I need to rename Foo to Bar.
We have an identical myapp:
class Foo(models.Model):
name = models.CharField(unique=True, max_length=32)
description = models.TextField(null=True, blank=True)
But I have in my myotherapp a ManyToMany field:
class AnotherModel(models.Model):
foo = models.ForeignKey(Foo)
is_awesome = models.BooleanField()
class YetAnotherModel(models.Model):
foo = models.ManyToManyField(Foo, blank=True, null=True) # Here!
is_ridonkulous = models.BooleanField()
I tried rename:
foo = models.ManyToManyField(Foo, blank=True, null=True)
to IntegerField() but doesn't work. How can I do that?
This is how I did it. Note: I was NOT in production, so I did not have to worry about information already in the tables. If you currently have data that you need to keep in the linking table, back up your data first. Also, I was using Django 1.9, but I think that everything referenced here is in 1.8 too.
The issue with the many-to-many relationship is the intermediate tables. Using RemoveField and AddField handled that.
The myapp migration for the model rename probably looks something like this:
class Migration(migrations.Migration):
dependencies = [
('Foo', '0001_initial.py'),
]
operations = [
migrations.RenameModel(
old_name='Foo',
new_name='Bar',
),
]
Next you would run:
python manage.py makemigrations --empty myotherapp
Then you would put this code in the new migration:
dependencies = [
('myotherapp', '0001_initial.py'),
('myapp', '0002_whateverthismigrationwasnamed')
]
operations = [
migrations.RemoveField(
model_name='YetAnotherModel',
name='Foo'
),
migrations.AddField(
model_name='YetAnotherModel',
name='Bar',
field=models.ManyToManyField(blank=True, null=True, to='myapp.Bar'),
),
]
It's important to make sure that you add the rename model migration from myapp as a dependency so it runs first.
Now, if you are in production you should NOT use this method without taking precautions. It straight up deletes the linking table. This from the django docs on RemoveField:
Bear in mind that when reversed, this is actually adding a field to a model. The operation is reversible (apart from any data loss, which of course is irreversible) if the field is nullable or if it has a default value that can be used to populate the recreated column.
If you are in production, you will want to take steps to backup the data so you can restore it into the new table.