All the previous answers I came across are not clear or old.
I have a third-party package installed and migrations ran.
Problem is the third-party package uses an Interger field to reference users(based on the assumption that the app is using the default django user), but in my case I am using a uuid for user IDs
package models.py
class UserDashboardModule(models.Model):
title = models.CharField(verbose_name=_('Title'), max_length=255)
user = models.PositiveIntegerField(verbose_name=_('user'))
column = models.PositiveIntegerField(verbose_name=_('column'))
order = models.IntegerField(verbose_name=_('order'))
collapsed = models.BooleanField(verbose_name=_('collapsed'), default=False)
...
# Migrations of third-party package
operations = [
migrations.CreateModel(
name='UserDashboardModule',
fields=[
('id', models.AutoField(verbose_name='ID', primary_key=True, serialize=False, auto_created=True)),
('title', models.CharField(verbose_name='Title', max_length=255)),
('module', models.CharField(verbose_name='module', max_length=255)),
('app_label', models.CharField(verbose_name='application name', max_length=255, blank=True, null=True)),
('user', models.PositiveIntegerField(verbose_name='user')),
]
),
]
My user model is like this
class User(AbstractUser):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
...
this setup makes it impossible to use the third-party package and I will like to maintain my uuids on users table.
What I want to do is to somehow override this migration and turn the user field to a uuid instead of an interger field.
PS: I have tried creating an empty migration in my users app and wrote the migration but it didn't work.
don't know this is gonna work or not , you can set id of user model as whatever this app is comfortable to work with . then make a migration file that depends on the last migration of other app then convert your model and foreign key to whatever you want it to be .
This can be done with django's MIGRATION_MODULES setting.
For example, I needed to override the migrations in the puput application, for a very similar reason to yours (they hard coded the name of what should have been a swappable dependency model).
I copied all of puput's migration files to a directory in one of my own apps, in my case app/custom_puput_migrations and edited them as I saw fit.
Then in settings I put:
MIGRATION_MODULES = {'puput': 'app.custom_puput_migrations')
Then I recreated my database from scratch and away it went.
The downside of this is that you have to maintain those migrations; each time you upgrade the third party app version, you need to check for and copy across any migrations they may have made.
Related
Suppose I have a reusable app, that defines a model Person and a model Invite.
A Person has a OneToOne field to AUTH_USER_MODEL and defines some basic fields (such as birthday). It is a swappable model, so that a project which uses this app can easily add other fields (such as gender, etc.)
In my reusable app, I define a setting that provides the swapping model (otherwise, a default one will be used, exactly as django.contrib.auth does it.
The Invite model has a OneToOneField to the swappable Person model and an email field. (I think, it's quite clear what this model is for). The model itself is swappable as well, but I don't think that this makes any difference for the kind of problem I am facing.
reusable app models:
class AbstractPerson(models.Model):
user = models.OneToOneField(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True, blank=True, related_name='person')
birthdate = models.DateField()
class Meta:
abstract = True
class Person(AbstractPerson):
class Meta(AbstractPerson.Meta):
swappable = 'REUSABLEAPP_PERSON_MODEL'
class AbstractInvite(models.Model):
email = models.EmailField()
person = models.OneToOneField(settings.REUSABLEAPP_PERSON_MODEL, on_delete=models.CASCADE, null=False, related_name='+')
class Meta:
abstract = True
class Invite(AbstractInvite):
class Meta(AbstractInvite.Meta):
swappable = 'REUSABLEAPP_INVITE_MODEL'
If I create the initial migration for my reusable app (using a dummy project and not swapping out my models), I get the following migration for my reusable app:
class Migration(migrations.Migration):
initial = True
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='Person',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('birthdate', models.DateField()),
('user', models.OneToOneField(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='person', to=settings.AUTH_USER_MODEL)),
],
options={
'abstract': False,
'swappable': 'REUSABLEAPP_PERSON_MODEL',
},
),
migrations.CreateModel(
name='Invite',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('email', models.EmailField(max_length=254)),
('person', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='+', to=settings.REUSABLEAPP_PERSON_MODEL)),
],
options={
'abstract': False,
'swappable': 'REUSABLEAPP_INVITE_MODEL',
},
),
]
If I then include my reusable app in another project, and swap out the Person and Invite model, I get an error when running makemigrations:
ValueError: The field myreusable_app.Invite.person was declared with a lazy reference to 'tester.myperson', but app 'tester' isn't installed.
(tester is the app that defines the swapped models, obviously)
If I delete the migration from my reusable app, and run makemigrations again, it works. the created migration is almost identical to the one above, with the exception of a new dependency:
migrations.swappable_dependency(settings.REUSABLEAPP_PERSON_MODEL),
The migration created in the tester app looks as follows:
class Migration(migrations.Migration):
initial = True
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='MyPerson',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('birthdate', models.DateField()),
('user', models.OneToOneField(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='person', to=settings.AUTH_USER_MODEL)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='MyInvite',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('email', models.EmailField(max_length=254)),
('person', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='+', to=settings.REUSABLEAPP_PERSON_MODEL)),
],
options={
'abstract': False,
},
),
]
I have looked up what swappable_dependency actually does: It looks only which app name is defined (let's say, the setting is tester.mymodel), and creates a dependency to this apps initial migration.
Now, if I delete the created migration from my tester app, I cannot run makemigrations again, I get the same error as above.
To clarify up until this point: Everything works as desired, if I delete the initial migration (and therefore all future migrations!) from my reusable app.
The problem, as I understand it, is the following:
The reusable app has a dependency to the initial migration of the client application that defines the swapped models. But this migration does not exist yet (heck, I am trying to create it!), so makemigration failes. (running makemigrations tester does not help).
But this almost exact same thing works flawlessly when swapping out the standard User model for a custom one. Furthermore, I do not fully understand why the error message states that the tester app is not installed. It definitly is inside my INSTALLED_APPS and it is picked up by the django-ecosystem.
After a few hours, I came up with a possible (but hacky) workaround:
Remove my reusable app from INSTALLED_APPS
Create MyInvite and MyPerson in the tester app (they both inherit from django.models.Model
Create those models by running makemigrations tester
Add my reusable app to INSTALLED_APPS
Define the swap settings
Change the inheritance of my models to their respective abstract counterparts
Run makemigrations again.
This works, because the initial migration of my reusable app now can fullfill the dependency to the swapped models by looking at the initial migration of tester that defines the models with the same name that is defined in the swap variables.
But I am sure that there must be a better way to do that.
This leaves me with the following questions:
How can I handle foreign key relationships to swapped models?
Why can't I create migrations for one app without looking at the migrations of other apps?
Unfortunately, I have to answer my own question.
This package provides a public interface to the swapping API. I have learned, that looking at closed issues is sometimes more helpful than reading the open ones.
Especially #12, and the open #10 answer my question. To sum it up, what works for the AUTH_USER_MODEL setting, does not work for any other swappable model, because the lazy_reference error is ignored, when the target model is set as AUTH_USER_MODEL within django, here:
# There shouldn't be any operations pending at this point.
from django.core.checks.model_checks import _check_lazy_references
ignore = {make_model_tuple(settings.AUTH_USER_MODEL)} if ignore_swappable else set()
errors = _check_lazy_references(self, ignore=ignore)
if errors:
raise ValueError("\n".join(error.msg for error in errors))
The proposed solution would be a registry, where all swapped models are registered and then looked up, to ignore the lazy reference error. Unfortunately, there seems to be no plans from the django devs to support this, as I have not found any open issues or feature requests for that.
I have a rather annoying issue when trying to send my merge to my automated tests on circle CI.
Just for the context, I've inherited a project where the authors are no longer working at my current work.
I'm working on django and I've done a merge, from my local dev branch to my local master branch. The merge went well. However, when starting the django server through a manage.py runserver, it gives me the warning Your project may not work properly until you apply the migrations for app(s)[...].
When doing the manage.py migrate, I'm running into the first issue:
1- django.db.utils.ProgrammingError: relation "cms_disclaimerpanel" already exists
I fix the issue by manually editing the migration file, commenting the following lines
# migrations.CreateModel(
# name='DisclaimerPanel',
# fields=[
# ('abstractpanel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='cms.AbstractPanel')),
# ('title', models.CharField(blank=True, max_length=1024, verbose_name='title')),
# ('show_title', models.BooleanField(default=True, verbose_name='show title')),
# ('subtitle', models.TextField(blank=True, verbose_name='content')),
# ('show_subtitle', models.BooleanField(default=True, verbose_name='show subtitle')),
# ('alignment', models.CharField(choices=[('left', 'left'), ('center', 'center')], default='center', max_length=10, verbose_name='text alignment')),
# ('button', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='cms.Link')),
# ],
# options={
# 'verbose_name': 'Disclaimer Panel',
# },
# bases=('cms.abstractpanel',),
# )
Then the second issue happened, while carrying my manage.py migrate
2 - ProgrammingError: column "http_request_lang" of relation "cms_dynamicsettings" does not exist
I fix the issue by manually editing the migration file, commenting the following lines
#operations = [
# migrations.RemoveField(
# model_name='dynamicsettings',
# name='http_request_lang',
# ),
#]
The manage.py was able to run entirely. Then I ran manage.py makemigrations and it gives me this last file
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('cms', '0088_merge_20190411_1655'),
]
operations = [
migrations.CreateModel(
name='DisclaimerPanel',
fields=[
('abstractpanel_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='cms.AbstractPanel')),
('title', models.CharField(blank=True, max_length=1024, verbose_name='title')),
('show_title', models.BooleanField(default=True, verbose_name='show title')),
('subtitle', models.TextField(blank=True, verbose_name='content')),
('show_subtitle', models.BooleanField(default=True, verbose_name='show subtitle')),
('alignment', models.CharField(choices=[('left', 'left'), ('center', 'center')], default='center', max_length=10, verbose_name='text alignment')),
('button', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='cms.Link')),
],
options={
'verbose_name': 'Disclaimer Panel',
},
bases=('cms.abstractpanel',),
),
migrations.RemoveField(
model_name='dynamicsettings',
name='http_request_lang',
),
]
With the changes above, I can runmanage.py runserver.
I'm then adding these 3 files to my merged branch and create a distant repository for my branch.
Each new repository created is run through the unit test and there lies the issue as it does not take into account my three newly commited files.
It gives me the following error, the same error as in the point 2 (see above).
ERROR: relation "cms_dynamicsettings" does not exist at character 1508
Using my dev environnement as a template, my guess would be that circle ci is replicating the same issue I've encountered and that I manually fixed.
The questions are the following:
Is there a way to drop my model cms_disclaimer before it runs through that damn migration file? If yes, how?
Is there a way to not take into account the migration file and tell it not to drop the column http_request_lang
My last question is why the manage.py makemigrations does not see the changes in the database?
One more information:
The database was built with the branch master. I checkout on another branch based on master , merged my dev branch into master and then do manage.py migrate.
Any info will be more than welcome as I'm loosing my sanity.
Thanks.
The procedure in such a situation is to ensure you first get to a state that is in sync with your database (assuming you can't drop your database because the system is in production).
Check in your production database the table "django_migrations" and look at the last migration that was applied for each of your apps.
In your code, delete all migration files that were added after that last migration. Do that for each app. Make sure all the migration files applied to your database are present in your code repo.
Run manage.py migrate, which should do nothing ("nothing to migrate") if your migration files are in sync with your db.
Run manage.py makemigrations which will create one extra migration file that reflects all the changes in your models with respect to your database.
Run manage.py migrate and everything should be working now.
Some things to worry about when doing this:
Make sure all the databases you are using (in all environments) are in sync with the production database.
Make sure no one is working on a different branch where some other migrations might exist.
I'm trying to work on a Django project on a different computer than the one I usually do. However, when trying to run the following migration:
from django.conf import settings
import django.contrib.postgres.fields
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('auth', '0010_auto_20180727_1345'),
('lucy_web', '0183_auto_20180814_1505'),
]
operations = [
migrations.CreateModel(
name='GoogleCredentials',
fields=[
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('user', models.OneToOneField(limit_choices_to={'is_staff': True}, on_delete=django.db.models.deletion.CASCADE, primary_key=True, serialize=False, to=settings.AUTH_USER_MODEL)),
('token', models.CharField(max_length=255, null=True)),
('refresh_token', models.CharField(max_length=255, null=True)),
('token_uri', models.CharField(max_length=255, null=True)),
('client_id', models.CharField(max_length=255, null=True)),
('client_secret', models.CharField(max_length=255, null=True)),
('scopes', django.contrib.postgres.fields.ArrayField(base_field=models.CharField(max_length=255), null=True, size=None)),
],
options={
'abstract': False,
},
),
]
I get this error message:
django.db.migrations.exceptions.NodeNotFoundError: Migration lucy_web.0184_googlecredentials dependencies reference nonexistent parent node ('auth', '0010_auto_20180727_1345')
Following https://groups.google.com/forum/#!topic/django-users/m59NufO1GI8, I went into the django/contrib/auth/migrations directory and found that indeed, there are 9 migrations, but the 10th migration is not there:
Kurts-MacBook-Pro:auth kurtpeek$ ls migrations
0001_initial.py
0002_alter_permission_name_max_length.py
0003_alter_user_email_max_length.py
0004_alter_user_username_opts.py
0005_alter_user_last_login_null.py
0006_require_contenttypes_0002.py
0007_alter_validators_add_error_messages.py
0008_alter_user_username_max_length.py
0009_alter_user_last_name_max_length.py
__init__.py
__pycache__/
Kurts-MacBook-Pro:auth kurtpeek$ pwd
/Users/kurtpeek/.local/share/virtualenvs/lucy-web-oCa8O1zi/lib/python3.7/site-packages/django/contrib/auth
The problem is, I haven't checked the virtual env into version control, and I don't have access to the other computer at the moment. I also feel like I shouldn't have to check in the Django source code for the project, though.
My question is: how did this situation likely come to pass? What I suspect is that it is associated with the way the django.contrib.auth.User model is customized in the project. We have a lucy_web/models/user.py with content similar to the following:
from django.contrib.auth.models import User
from django.contrib.auth.signals import user_logged_in, user_logged_out, user_login_failed
from django.db.models.signals import pre_save, post_save
from django.dispatch import receiver
from auditlog.registry import auditlog
from lucy_web.models import LucyGuide
from crequest.middleware import CrequestMiddleware
from lucy_web.lib import intercom
# overrides built in user to provide reasonable unique constraints
User._meta.get_field('username')._unique = True
# TODO(kayla): fix this, the unique constraint on email doesn't seem to be working in prod
# I suspect the username unique constraint only works because it's the default setting
User._meta.get_field('email')._unique = True
auditlog.register(User)
#property
def family(self):
if hasattr(self, 'employee_family'):
return self.employee_family
elif hasattr(self, 'partner_family'):
return self.partner_family
else:
return None
#property
def is_employee(self):
return hasattr(self, 'employee_family')
#property
def user_apn(self):
return self.userapn_set.order_by("created_at").last()
#property
def lucy_guide(self):
try:
return self.lucyguide
except LucyGuide.DoesNotExist:
return None
def user__str__(self):
"""
User model's __str__ method.
We use a different name than '__str__' because dunder names
are reserved by Python and subject to breakage without warning
(cf. https://www.python.org/dev/peps/pep-0562/#backwards-compatibility-and-impact-on-performance).
"""
return f"{self.first_name} {self.last_name}".strip()
#property
def profile(self):
if hasattr(self, 'user_profile'):
return self.user_profile
else:
return None
#property
def using_app(self):
if hasattr(self, 'user_profile'):
return self.user_profile.using_app
return False
#property
def activation_code(self):
if hasattr(self, 'user_profile'):
return self.user_profile.activation_code
return False
User.add_to_class('family', family)
User.add_to_class('is_employee', is_employee)
User.add_to_class('user_apn', user_apn)
User.add_to_class('lucy_guide', lucy_guide)
User.add_to_class('__str__', user__str__)
User.add_to_class('profile', profile)
User.add_to_class('using_app', using_app)
In short, we use the add_to_class method to add properties to Django's User model. This does not seem to be one of the recommended methods described in https://docs.djangoproject.com/en/2.1/topics/auth/customizing/#extending-user. Is this perhaps the source of the problem?
If you are using add_to_class to the Django User model it will detect a change in the external package and apply the migration in the external packages migrations directory (usually in your virtual environment).
It will be referenced in the migration file provided (lucy_web.0184_googlecredentials) at the top if the User model is required for the later migration to be created. The user migration number chosen is the last available at the time of creation to ensure compatibility.
As a bad workaround you could make migrations on the new computer which should create the new user model model migration. Find the file name and alter it temporarily in the migration file lucy_web.0184_googlecredentials
I also used add_to_class() and created new project in the same enviroment.
It need to delete all migrations and *.pyc in subfolders "migrations" in your env. Leave only __init__.py files.
...\Envs\blah-blah-env\Lib\site-packages\django\contrib\...
e.g. path (one of them):
c:\Users\user\Envs\blah-blah-env\Lib\site-packages\django\contrib\auth\migrations\
If you get after (source):
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration admin.0001_initial is applied before its dependency account.0001_initial on database 'default'
delete db or clear migration table, delete migration files in your project. Good luck!
I have a very similar situation like this: Django migration strategy for renaming a model and relationship fields
I need to rename Foo to Bar.
We have an identical myapp:
class Foo(models.Model):
name = models.CharField(unique=True, max_length=32)
description = models.TextField(null=True, blank=True)
But I have in my myotherapp a ManyToMany field:
class AnotherModel(models.Model):
foo = models.ForeignKey(Foo)
is_awesome = models.BooleanField()
class YetAnotherModel(models.Model):
foo = models.ManyToManyField(Foo, blank=True, null=True) # Here!
is_ridonkulous = models.BooleanField()
I tried rename:
foo = models.ManyToManyField(Foo, blank=True, null=True)
to IntegerField() but doesn't work. How can I do that?
This is how I did it. Note: I was NOT in production, so I did not have to worry about information already in the tables. If you currently have data that you need to keep in the linking table, back up your data first. Also, I was using Django 1.9, but I think that everything referenced here is in 1.8 too.
The issue with the many-to-many relationship is the intermediate tables. Using RemoveField and AddField handled that.
The myapp migration for the model rename probably looks something like this:
class Migration(migrations.Migration):
dependencies = [
('Foo', '0001_initial.py'),
]
operations = [
migrations.RenameModel(
old_name='Foo',
new_name='Bar',
),
]
Next you would run:
python manage.py makemigrations --empty myotherapp
Then you would put this code in the new migration:
dependencies = [
('myotherapp', '0001_initial.py'),
('myapp', '0002_whateverthismigrationwasnamed')
]
operations = [
migrations.RemoveField(
model_name='YetAnotherModel',
name='Foo'
),
migrations.AddField(
model_name='YetAnotherModel',
name='Bar',
field=models.ManyToManyField(blank=True, null=True, to='myapp.Bar'),
),
]
It's important to make sure that you add the rename model migration from myapp as a dependency so it runs first.
Now, if you are in production you should NOT use this method without taking precautions. It straight up deletes the linking table. This from the django docs on RemoveField:
Bear in mind that when reversed, this is actually adding a field to a model. The operation is reversible (apart from any data loss, which of course is irreversible) if the field is nullable or if it has a default value that can be used to populate the recreated column.
If you are in production, you will want to take steps to backup the data so you can restore it into the new table.
I'm working on this project and i'm extending Django's base user model in order to have emails as usernames. I've got the following project structure with two apps (client and showroom)
. project
.. client
... models
.... Client
.. showroom
... models
.... Image
Client inheritates AbstractBaseUser like this:
class Client(AbstractBaseUser):
email = models.CharField(max_length=255, unique=True)
firstname = models.CharField(max_length=50)
etc...
Image has a Foreign Key to my Client model:
class Image(models.Model):
client = models.ForeignKey(_('Client'), settings.AUTH_USER_MODEL, blank=True, null=True, limit_choices_to=Q(groups__name = 'website_user'))
etc...
And in my settings.py (which is not called settings.py, don't think it's relevant but just in case) I have got this:
INSTALLED_APPS = (
'django.contrib.auth',
etc...
'client',
'showroom',
etc...
)
AUTH_USER_MODEL = 'client.Client'
Now, when I try to run the project, syncdb, migrate or whatever else that has to do with the database, I get this error:
showroom.Image.client: (fields.E300) Field defines a relation with model 'Client', which is either not installed, or is abstract.
Of course, when I remove the foreign key to Client in my Image model, everything works fine.
I have googled this a lot and most solutions suggest that my apps are not properly installed, but they seem to be as shown in my config file. So I guess this has something to do with inheriting django's AbstractBaseUser, but i can't see why this wont work for me, as my code is very similar to the one in the official docs.
Anyway, thanks in advance for your help.
First argument of ForeignKey should be a model or a name of a model. You pass _('Client') what I think is verbose_name.
Try this:
client = models.ForeignKey(settings.AUTH_USER_MODEL, verbose_name=_('Client'), blank=True, null=True, limit_choices_to=Q(groups__name = 'website_user'))