Related
I'm curious how other django developers manage multiple code branches (in git for instance) with migrations.
My problem is as follows:
- we have multiple feature branches in git, some of them with django migrations (some of them altering fields, or removing them altogether)
- when I switch branches (with git checkout some_other_branch) the database does not reflect always the new code, so I run into "random" errors, where a db table column does not exist anymore, etc...
Right now, I simply drop the db and recreate it, but it means I have to recreate a bunch of dummy data to restart work. I can use fixtures, but it requires keeping track of what data goes where, it's a bit of a hassle.
Is there a good/clean way of dealing with this use-case? I'm thinking a post-checkout git hook script could run the necessary migrations, but I don't even know if migration rollbacks are at all possible.
Migrations rollback are possible and usually handled automatically by django.
Considering the following model:
class MyModel(models.Model):
pass
If you run python manage.py makemigrations myapp, it will generate the initial migration script.
You can then run python manage.py migrate myapp 0001 to apply this initial migration.
If after that you add a field to your model:
class MyModel(models.Model):
my_field = models.CharField()
Then regenerate a new migration, and apply it, you can still go back to the initial state. Just run
python manage.py migrate myapp 0001 and the ORM will go backward, removing the new field.
It's more tricky when you deal with data migrations, because you have to write the forward and backward code.
Considering an empty migration created via python manage.py makemigrations myapp --empty,
you'll end up with something like:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
def forward(apps, schema_editor):
# load some data
MyModel = apps.get_model('myapp', 'MyModel')
while condition:
instance = MyModel()
instance.save()
def backward(apps, schema_editor):
# delete previously loaded data
MyModel = apps.get_model('myapp', 'MyModel')
while condition:
instance = MyModel.objects.get(myargs)
instance.delete()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0003_auto_20150918_1153'),
]
operations = [
migrations.RunPython(forward, backward),
]
For pure data-loading migrations, you usually don't need the backward migration.
But when you alter the schema and update existing rows
(like converting all values in a column to slug), you'll generally have to write the backward step.
In our team, we try to avoid working on the same models at the same time to avoid collision.
If it is not possible, and two migration with the same number (e.g 0002) are created,
you can still rename one of them to change the order in which they will be applied (also remember to update
the dependencies attribute on the migration class to your new order).
If you end up working on the same model fields at the same time in different features,
you'll still be in trouble, but it may mean these features are related and should be handled
together in a single branch.
For the git-hooks part, it's probably possible to write something, Assuming your are on branch mybranch
and want to check out another feature branch myfeature:
Just before switching, you dump the list of currently applied migrations into
a temporary file mybranch_database_state.txt
Then, you apply myfeature branch migrations, if any
Then, when checking back mybranch, you reapply your previous database state
by looking to the dump file.
However, it seems a bit hackish to me, and it would probably be really difficult to handle properly all scenarios:
rebasing, merging, cherry-picking, etc.
Handling the migrations conflicts when they occurs seems easier to me.
I don't have a good solution to this, but I feel the pain.
A post-checkout hook will be too late. If you are on branch A and you check out branch B, and B has fewer migrations than A, the rollback information is only in A and needs to be run before checkout.
I hit this problem when jumping between several commits trying to locate the origin of a bug. Our database (even in development trim) is huge, so dropping and recreating isn't practical.
I'm imagining a wrapper for git-checkout that:
Notes the newest migration for each of your INSTALLED_APPS
Looks in the requested branch and notes the newest migrations there
For each app where the migrations in #1 are farther ahead than in #2, migrate back to the highest migration in #2
Check out the new branch
For each app where migrations in #2 were ahead of #1, migrate forward
A simple matter of programming!
For simple changes I rely on migration rollback, as discussed by Agate.
However, if I know a feature branch is going to involve highly invasive database changes, or if it will involve a lot of data migration, I like to create a clone of the local (or remote dev) database as soon as I start the new branch. This may not always be convenient, but especially for local development using sqlite it is just a matter op copying a file (which is not under source control).
The first commit on the new branch then updates my Django settings (local/dev) to use the cloned database. This way, when I switch branches, the correct database is selected automatically. No need to worry about rolling back schema changes, missing data, etc. No complicated stuff.
After the feature branch has been fully merged, the cloned database can be removed.
So far I have found two Github projects (django-south-compass and django_nomad) that try to solve the issue of migrating between dev branches and there is a couple of answers on Stack Overflow.
Citing an article on Medium, most of the solutions boil down to one of the following concepts:
Dropping all the tables and reapply migrations in the target branch from scratch. When the tables are created from scratch, all the data will be lost and needs to be recreated as well. This can be handled with fixtures and data migrations but managing them, in turn, will become a nightmare, not to mention that it will take some time (...)
Have a separate database for each branch and change the settings file with the target branch’s settings every time the branch is switched using tools like sed. This can be done with a post_checkout hook. Maintaining one large database for each branch would be very storage-intensive. Also, checking out individual commit IDs might potentially produce the same errors.
Finding the differences in migrations between the source and target branch, and apply the differences. We can do so with post_checkout script but there is a small issue. This post explains the issue in detail. To summarize the issue, post_checkout is run after all the files in the target branch are checked out, which includes migration files. If the target branch doesn’t contain all the migrations in the source branch when we run python manage.py migrate app1 Django won’t be able to find the missing migrations which are needed to apply reverse migrations. We have to temporarily checkout migration files in the source branch, run python manage.py migrate and checkout migration files in the target branch. django-south-compass does something very similar but is available only for up to python 2.6.
Using a management command (which uses python git module), find all the migration operations differences between the source branch and the merge-base of the source branch and target branch and notify the user of these changes. If these changes don’t interfere with the reason for branch change, the user can go ahead and change the branch. Else, using another management command, un-apply all migration till merge base, switch branch, and apply the migrations in the target branch. There will be a small data loss and if the two branches haven’t diverged a lot, is manageable. django_nomad does some of this work.
Keep a track of applied and unapplied migrations in files and use this data to populate the tables when switching branches.
I've just started working with Django, and when setting up my models and db I picked a name which I would like to change now. Is it OK to just edit models.py (rename the class), then run makemigrations and migrate on it? I have a table set up in the db (SQLite), but no entries yet.
I'm new to database migration too. Does this cover what I want to do?
Thanks.
After renaming of model in the models.py, run makemigrations. The question should appear "Did you rename the xxx model to yyy? [y/N]" Press y and it will be enough in your case.
UPD:
In general, you should run makemigrations and migrate every time when you change code in models.py. You should manually edit migrations files (created by makemigrations) only in few cases, when the Django isn't smart enough to understand what you want to do.
I recently switched from Django 1.6 to 1.7, and I began using migrations (I never used South).
Before 1.7, I used to load initial data with a fixture/initial_data.json file, which was loaded with the python manage.py syncdb command (when creating the database).
Now, I started using migrations, and this behavior is deprecated :
If an application uses migrations, there is no automatic loading of fixtures.
Since migrations will be required for applications in Django 2.0, this behavior is considered deprecated. If you want to load initial data for an app, consider doing it in a data migration.
(https://docs.djangoproject.com/en/1.7/howto/initial-data/#automatically-loading-initial-data-fixtures)
The official documentation does not have a clear example on how to do it, so my question is :
What is the best way to import such initial data using data migrations :
Write Python code with multiple calls to mymodel.create(...),
Use or write a Django function (like calling loaddata) to load data from a JSON fixture file.
I prefer the second option.
I don't want to use South, as Django seems to be able to do it natively now.
Update: See #GwynBleidD's comment below for the problems this solution can cause, and see #Rockallite's answer below for an approach that's more durable to future model changes.
Assuming you have a fixture file in <yourapp>/fixtures/initial_data.json
Create your empty migration:
In Django 1.7:
python manage.py makemigrations --empty <yourapp>
In Django 1.8+, you can provide a name:
python manage.py makemigrations --empty <yourapp> --name load_intial_data
Edit your migration file <yourapp>/migrations/0002_auto_xxx.py
2.1. Custom implementation, inspired by Django' loaddata (initial answer):
import os
from sys import path
from django.core import serializers
fixture_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '../fixtures'))
fixture_filename = 'initial_data.json'
def load_fixture(apps, schema_editor):
fixture_file = os.path.join(fixture_dir, fixture_filename)
fixture = open(fixture_file, 'rb')
objects = serializers.deserialize('json', fixture, ignorenonexistent=True)
for obj in objects:
obj.save()
fixture.close()
def unload_fixture(apps, schema_editor):
"Brutally deleting all entries for this model..."
MyModel = apps.get_model("yourapp", "ModelName")
MyModel.objects.all().delete()
class Migration(migrations.Migration):
dependencies = [
('yourapp', '0001_initial'),
]
operations = [
migrations.RunPython(load_fixture, reverse_code=unload_fixture),
]
2.2. A simpler solution for load_fixture (per #juliocesar's suggestion):
from django.core.management import call_command
fixture_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '../fixtures'))
fixture_filename = 'initial_data.json'
def load_fixture(apps, schema_editor):
fixture_file = os.path.join(fixture_dir, fixture_filename)
call_command('loaddata', fixture_file)
Useful if you want to use a custom directory.
2.3. Simplest: calling loaddata with app_label will load fixtures from the <yourapp>'s fixtures dir automatically :
from django.core.management import call_command
fixture = 'initial_data'
def load_fixture(apps, schema_editor):
call_command('loaddata', fixture, app_label='yourapp')
If you don't specify app_label, loaddata will try to load fixture filename from all apps fixtures directories (which you probably don't want).
Run it
python manage.py migrate <yourapp>
Short version
You should NOT use loaddata management command directly in a data migration.
# Bad example for a data migration
from django.db import migrations
from django.core.management import call_command
def load_fixture(apps, schema_editor):
# No, it's wrong. DON'T DO THIS!
call_command('loaddata', 'your_data.json', app_label='yourapp')
class Migration(migrations.Migration):
dependencies = [
# Dependencies to other migrations
]
operations = [
migrations.RunPython(load_fixture),
]
Long version
loaddata utilizes django.core.serializers.python.Deserializer which uses the most up-to-date models to deserialize historical data in a migration. That's incorrect behavior.
For example, supposed that there is a data migration which utilizes loaddata management command to load data from a fixture, and it's already applied on your development environment.
Later, you decide to add a new required field to the corresponding model, so you do it and make a new migration against your updated model (and possibly provide a one-off value to the new field when ./manage.py makemigrations prompts you).
You run the next migration, and all is well.
Finally, you're done developing your Django application, and you deploy it on the production server. Now it's time for you to run the whole migrations from scratch on the production environment.
However, the data migration fails. That's because the deserialized model from loaddata command, which represents the current code, can't be saved with empty data for the new required field you added. The original fixture lacks necessary data for it!
But even if you update the fixture with required data for the new field, the data migration still fails. When the data migration is running, the next migration which adds the corresponding column to the database, is not applied yet. You can't save data to a column which does not exist!
Conclusion: in a data migration, the loaddata command introduces potential inconsistency between the model and the database. You should definitely NOT use it directly in a data migration.
The Solution
loaddata command relies on django.core.serializers.python._get_model function to get the corresponding model from a fixture, which will return the most up-to-date version of a model. We need to monkey-patch it so it gets the historical model.
(The following code works for Django 1.8.x)
# Good example for a data migration
from django.db import migrations
from django.core.serializers import base, python
from django.core.management import call_command
def load_fixture(apps, schema_editor):
# Save the old _get_model() function
old_get_model = python._get_model
# Define new _get_model() function here, which utilizes the apps argument to
# get the historical version of a model. This piece of code is directly stolen
# from django.core.serializers.python._get_model, unchanged. However, here it
# has a different context, specifically, the apps variable.
def _get_model(model_identifier):
try:
return apps.get_model(model_identifier)
except (LookupError, TypeError):
raise base.DeserializationError("Invalid model identifier: '%s'" % model_identifier)
# Replace the _get_model() function on the module, so loaddata can utilize it.
python._get_model = _get_model
try:
# Call loaddata command
call_command('loaddata', 'your_data.json', app_label='yourapp')
finally:
# Restore old _get_model() function
python._get_model = old_get_model
class Migration(migrations.Migration):
dependencies = [
# Dependencies to other migrations
]
operations = [
migrations.RunPython(load_fixture),
]
Inspired by some of the comments (namely n__o's) and the fact that I have a lot of initial_data.* files spread out over multiple apps I decided to create a Django app that would facilitate the creation of these data migrations.
Using django-migration-fixture you can simply run the following management command and it will search through all your INSTALLED_APPS for initial_data.* files and turn them into data migrations.
./manage.py create_initial_data_fixtures
Migrations for 'eggs':
0002_auto_20150107_0817.py:
Migrations for 'sausage':
Ignoring 'initial_data.yaml' - migration already exists.
Migrations for 'foo':
Ignoring 'initial_data.yaml' - not migrated.
See django-migration-fixture for install/usage instructions.
In order to give your database some initial data, write a data migration.
In the data migration, use the RunPython function to load your data.
Don't write any loaddata command as this way is deprecated.
Your data migrations will be run only once. The migrations are an ordered sequence of migrations. When the 003_xxxx.py migrations is run, django migrations writes in the database that this app is migrated until this one (003), and will run the following migrations only.
The solutions presented above didn't work for me unfortunately. I found that every time I change my models I have to update my fixtures. Ideally I would instead write data migrations to modify created data and fixture-loaded data similarly.
To facilitate this I wrote a quick function which will look in the fixtures directory of the current app and load a fixture. Put this function into a migration in the point of the model history that matches the fields in the migration.
On Django 2.1, I wanted to load some models (Like country names for example) with initial data.
But I wanted this to happen automatically right after the execution of initial migrations.
So I thought that it would be great to have a sql/ folder inside each application that required initial data to be loaded.
Then within that sql/ folder I would have .sql files with the required DMLs to load the initial data into the corresponding models, for example:
INSERT INTO appName_modelName(fieldName)
VALUES
("country 1"),
("country 2"),
("country 3"),
("country 4");
To be more descriptive, this is how an app containing a sql/ folder would look:
Also I found some cases where I needed the sql scripts to be executed in a specific order. So I decided to prefix the file names with a consecutive number as seen in the image above.
Then I needed a way to load any SQLs available inside any application folder automatically by doing python manage.py migrate.
So I created another application named initial_data_migrations and then I added this app to the list of INSTALLED_APPS in settings.py file. Then I created a migrations folder inside and added a file called run_sql_scripts.py (Which actually is a custom migration). As seen in the image below:
I created run_sql_scripts.py so that it takes care of running all sql scripts available within each application. This one is then fired when someone runs python manage.py migrate. This custom migration also adds the involved applications as dependencies, that way it attempts to run the sql statements only after the required applications have executed their 0001_initial.py migrations (We don't want to attempt running a SQL statement against a non-existent table).
Here is the source of that script:
import os
import itertools
from django.db import migrations
from YourDjangoProjectName.settings import BASE_DIR, INSTALLED_APPS
SQL_FOLDER = "/sql/"
APP_SQL_FOLDERS = [
(os.path.join(BASE_DIR, app + SQL_FOLDER), app) for app in INSTALLED_APPS
if os.path.isdir(os.path.join(BASE_DIR, app + SQL_FOLDER))
]
SQL_FILES = [
sorted([path + file for file in os.listdir(path) if file.lower().endswith('.sql')])
for path, app in APP_SQL_FOLDERS
]
def load_file(path):
with open(path, 'r') as f:
return f.read()
class Migration(migrations.Migration):
dependencies = [
(app, '__first__') for path, app in APP_SQL_FOLDERS
]
operations = [
migrations.RunSQL(load_file(f)) for f in list(itertools.chain.from_iterable(SQL_FILES))
]
I hope someone finds this helpful, it worked just fine for me!. If you have any questions please let me know.
NOTE: This might not be the best solution since I'm just getting started with Django, however I still wanted to share this "How-to" with you all since I didn't find much information while googling about this.
In my opinion fixtures are a bit bad. If your database changes frequently, keeping them up-to-date will came a nightmare soon. Actually, it's not only my opinion, in the book "Two Scoops of Django" it's explained much better.
Instead I'll write a Python file to provide initial setup. If you need something more I'll suggest you look at Factory boy.
If you need to migrate some data you should use data migrations.
There's also "Burn Your Fixtures, Use Model Factories" about using fixtures.
Although #rockallite's answer is excellent, it does not explain how to handle fixtures that rely on natural keys instead of integer pk values.
Simplified version
First, note that #rockallite's solution can be simplified by using unittest.mock.patch as a context manager, and by patching apps instead of _get_model:
...
from unittest.mock import patch
...
def load_fixture(apps, schema_editor):
with patch('django.core.serializers.python.apps', apps):
call_command('loaddata', 'your_data.json', ...)
...
This works well, as long as your fixtures do not rely on natural keys.
If they do, you're likely to see a DeserializationError: ... value must be an integer....
The problem with natural keys
Under the hood, loaddata uses django.core.serializers.deserialize() to load your fixture objects.
The deserialization of fixtures based on natural keys relies on two things:
the presence of a get_by_natural_key() method on the model's default manager
the presence of a natural_key() method on the model itself
The get_by_natural_key() method is necessary for the deserializer to know how to interpret the natural key, instead of an integer pk value.
Both methods are necessary for the deserializer to get existing objects from the database by natural key, as also explained here.
However, the apps registry which is available in your migrations uses historical models, and these do not have access to custom managers or custom methods such as natural_key().
Possible solution: step 1
The problem of the missing get_by_natural_key() method from our custom model manager is relatively easy to solve:
Just set use_in_migrations=True on your custom manager, as described in the documentation.
This ensures that your historical models can access the current get_by_natural_key() during migrations, and fixture loading should now succeed.
However, your historical models still don't have a natural_key() method. As a result, your fixtures will be treated as new objects, even if they are already present in the database.
This may lead to a variety of errors if the data-migration is ever re-applied, such as:
unique-constraint violations (if your models have unique-constraints)
duplicate fixture objects (if your models do not have unique-constraints)
"get returned multiple objects" errors (due to duplicate fixture objects created previously)
So, effectively, you're still missing out on a kind of get_or_create-like behavior during deserialization.
To experience this, just apply a data-migration as described above (in a test environment), then roll back the same data-migration (without removing the data), then re-apply the data-migration.
Possible solution: step 2
The problem of the missing natural_key() method from the model itself is a bit more difficult to solve.
One solution would be to assign the natural_key() method from the current model to the historical model, for example:
...
from unittest.mock import patch
from django.apps import apps as current_apps
from django.core.management import call_command
...
def load_fixture(apps, schema_editor):
def _get_model_patch(app_label):
""" add natural_key method from current model to historical model """
historical_model = apps.get_model(app_label=app_label)
current_model = current_apps.get_model(app_label=app_label)
historical_model.natural_key = current_model.natural_key
return historical_model
with patch('django.core.serializers.python._get_model', _get_model_patch):
call_command('loaddata', 'your_data.json', ...)
...
Notes:
For clarity, I omitted things like error handling and attribute checking from the example. You should implement those where necessary.
This solution uses the current model's natural_key method, which may still lead to trouble in certain scenarios, but the same goes for Django's use_in_migrations option for model managers.
I have changed the name of an app in Django by renaming its folder, imports and all its references (templates/indexes). But now I get this error when I try to run python manage.py runserver
Error: Could not import settings 'nameofmynewapp.settings' (Is it on sys.path?): No module named settings
How can I debug and solve this error? Any clues?
Follow these steps to change an app's name in Django:
Rename the folder which is in your project root
Change any references to your app in their dependencies, i.e. the app's views.py, urls.py , manage.py , and settings.py files.
Edit the database table django_content_type with the following command: UPDATE django_content_type SET app_label='<NewAppName>' WHERE app_label='<OldAppName>'
Also, if you have models, you will have to rename the model tables. For postgres, use ALTER TABLE <oldAppName>_modelName RENAME TO <newAppName>_modelName. For mysql too, I think it is the same (as mentioned by #null_radix).
(For Django >= 1.7) Update the django_migrations table to avoid having your previous migrations re-run: UPDATE django_migrations SET app='<NewAppName>' WHERE app='<OldAppName>'. Note: there is some debate (in comments) if this step is required for Django 1.8+; If someone knows for sure please update here.
If your models.py 's Meta Class has app_name listed, make sure to rename that too (mentioned by #will).
If you've namespaced your static or templates folders inside your app, you'll also need to rename those. For example, rename old_app/static/old_app to new_app/static/new_app.
For renaming django models, you'll need to change django_content_type.name entry in DB. For postgreSQL, use UPDATE django_content_type SET name='<newModelName>' where name='<oldModelName>' AND app_label='<OldAppName>'
Update 16Jul2021: Also, the __pycache__/ folder inside the app must be removed, otherwise you get EOFError: marshal data too short when trying to run the server. Mentioned by #Serhii Kushchenko
Meta point (If using virtualenv): Worth noting, if you are renaming the directory that contains your virtualenv, there will likely be several files in your env that contain an absolute path and will also need to be updated. If you are getting errors such as ImportError: No module named ... this might be the culprit. (thanks to #danyamachine for providing this).
Other references: you might also want to refer to the below links for a more complete picture:
Renaming an app with Django and South
How do I migrate a model out of one django app and into a new one?
How to change the name of a Django app?
Backwards migration with Django South
Easiest way to rename a model using Django/South?
Python code (thanks to A.Raouf) to automate the above steps (Untested code. You have been warned!)
Python code (thanks to rafaponieman) to automate the above steps (Untested code. You have been warned!)
New in Django 1.7 is a app registry that stores configuration and provides introspection. This machinery let's you change several app attributes.
The main point I want to make is that renaming an app isn't always necessary: With app configuration it is possible to resolve conflicting apps. But also the way to go if your app needs friendly naming.
As an example I want to name my polls app 'Feedback from users'. It goes like this:
Create a apps.py file in the polls directory:
from django.apps import AppConfig
class PollsConfig(AppConfig):
name = 'polls'
verbose_name = "Feedback from users"
Add the default app config to your polls/__init__.py:
default_app_config = 'polls.apps.PollsConfig'
For more app configuration: https://docs.djangoproject.com/en/1.7/ref/applications/
Fun problem! I'm going to have to rename a lot of apps soon, so I did a dry run.
This method allows progress to be made in atomic steps, to minimise disruption for other developers working on the app you're renaming.
See the link at the bottom of this answer for working example code.
Prepare existing code for the move:
Create an app config (set name and label to defaults).
Add the app config to INSTALLED_APPS.
On all models, explicitly set db_table to the current value.
Doctor migrations so that db_table was "always" explicitly defined.
Ensure no migrations are required (checks previous step).
Change the app label:
Set label in app config to new app name.
Update migrations and foreign keys to reference new app label.
Update templates for generic class-based views (the default path is <app_label>/<model_name>_<suffix>.html)
Run raw SQL to fix migrations and content_types app (unfortunately, some raw SQL is unavoidable). You can not run this in a migration.
UPDATE django_migrations
SET app = 'catalogue'
WHERE app = 'shop';
UPDATE django_content_type
SET app_label = 'catalogue'
WHERE app_label = 'shop';
Ensure no migrations are required (checks previous step).
Rename the tables:
Remove "custom" db_table.
Run makemigrations so django can rename the table "to the default".
Move the files:
Rename module directory.
Fix imports.
Update app config's name.
Update where INSTALLED_APPS references the app config.
Tidy up:
Remove custom app config if it's no longer required.
If app config gone, don't forget to also remove it from INSTALLED_APPS.
Example solution: I've created app-rename-example, an example project where you can see how I renamed an app, one commit at a time.
The example uses Python 2.7 and Django 1.8, but I'm confident the same process will work on at least Python 3.6 and Django 2.1.
In case you are using PyCharm and project stops working after rename:
Edit Run/Debug configuration and change environment variable DJANGO_SETTINGS_MODULE, since it includes your project name.
Go to Settings / Languages & Frameworks / Django and update the settings file location.
In many cases, I believe #allcaps's answer works well.
However, sometimes it is necessary to actually rename an app, e.g. to improve code readability or prevent confusion.
Most of the other answers involve either manual database manipulation or tinkering with existing migrations, which I do not like very much.
As an alternative, I like to create a new app with the desired name, copy everything over, make sure it works, then remove the original app:
Start a new app with the desired name, and copy all code from the original app into that. Make sure you fix the namespaced stuff, in the newly copied code, to match the new app name.
makemigrations and migrate. Note: if the app has a lot of foreign keys, there will likely be issues with clashing reverse accessors when trying to run the initial migration for the copied app. You have to rename the related_name accessor (or add new ones) on the old app to resolve the clashes.
Create a data migration that copies the relevant data from the original app's tables into the new app's tables, and migrate again.
At this point, everything still works, because the original app and its data are still in place.
Now you can refactor all the dependent code, so it only makes use of the new app. See other answers for examples of what to look out for.
Once you are certain that everything works, you can remove the original app. This involves another (schema) migration.
This has the advantage that every step uses the normal Django migration mechanism, without manual database manipulation, and we can track everything in source control. In addition, we keep the original app and its data in place until we are sure everything works.
Re-migrate approach for a cleaner plate.
This can painlessly be done IF other apps do not foreign key models from the app to be renamed. Check and make sure their migration files don't list any migrations from this one.
Backup your database. Dump all tables with a) data + schema for possible circular dependencies, and b) just data for reloading.
Run your tests.
Check all code into VCS.
Delete the database tables of the app to be renamed.
Delete the permissions: delete from auth_permission where content_type_id in (select id from django_content_type where app_label = '<OldAppName>')
Delete content types: delete from django_content_type where app_label = '<OldAppName>'
Rename the folder of the app.
Change any references to your app in their dependencies, i.e. the app's views.py, urls.py , 'manage.py' , and settings.py files.
Delete migrations: delete from django_migrations where app = '<OldAppName>'
If your models.py 's Meta Class has app_name listed, make sure to rename that too (mentioned by #will).
If you've namespaced your static or templates folders inside your app, you'll also need to rename those. For example, rename old_app/static/old_app to new_app/static/new_app.
If you defined app config in apps.py; rename those, and rename their references in settings.INSTALLED_APPS
Delete migration files.
Re-make migrations, and migrate.
Load your table data from backups.
If you use Pycharm, renaming an app is very easy with refactoring(Shift+F6 default) for all project files.
But make sure you delete the __pycache__ folders in the project directory & its sub-directories. Also be careful as it also renames comments too which you can exclude in the refactor preview window it will show you.
And you'll have to rename OldNameConfig(AppConfig): in apps.py of your renamed app in addition.
If you do not want to lose data of your database, you'll have to manually do it with query in database like the aforementioned answer.
Simple 4 steps to rename an existing app in Django project without any pain or data loss.
Step 1
Rename the app folder. For this example,"old_app" is our old name and "new_app" is our new name".
mv ./old_app ./new_app
Step 2
Update all imports referencing the old folder to reference the new.
For example:
# Before
from myproject.old_app import models
# After
from myproject.new_app import models
Step 3
Update old app name references within the Django migrations.
Example changes you'll likely have to make:
# Before
dependencies = [
('old_app', '0023_auto_20200403_1050'),
]
# After
dependencies = [
('new_app', '0023_auto_20200403_1050'),
]
# Before
field = models.ForeignKey(
default=None, on_delete=django.db.models.deletion.CASCADE,
to='old_app.Experiment'
)
# After
field = models.ForeignKey(
default=None, on_delete=django.db.models.deletion.CASCADE,
to='new_app.Experiment'
)
Step 4
Make a commit at this point.
Then, however you run your application migrations in a deployed environment, run django_rename_app before you run your migrations in that process.
i.e Before "python manage.py migrate --noinput", as the example below shows.
# Before
python manage.py collectstatic --noinput
python manage.py migrate --noinput
gunicorn my_project.wsgi:application
# After
python manage.py collectstatic --noinput
python manage.py rename_app old_app new_app
python manage.py migrate --noinput
gunicorn my_project.wsgi:application
This will update the app name in the following internal Django database tables:
django_content_type
django_migrations
And rename the prefix of all of your tables to start with your new app name, rather than the old one.
That's it.
Note : Its better to use your IDE's/text editor's find & replace function for making changes in the various files.
My simple alternative that avoids manual database manipulation (similar to https://stackoverflow.com/a/64133141/1608851), works well in a CI/CD, and is reversible.
Create an empty migration in the "old" app that renames the tables, content type, and migration records. Include a reverse migration.
Example:
from django.db import migrations
sql = """
ALTER TABLE oldapp_examplemodel RENAME TO newapp_examplemodel;
UPDATE django_migrations SET app ='newapp' WHERE app ='oldapp';
UPDATE django_content_type SET app_label ='newapp' WHERE app_label ='oldapp';
"""
reverse_sql = """
ALTER TABLE newapp_examplemodel RENAME TO oldapp_examplemodel;
UPDATE django_migrations SET app ='oldapp' WHERE app ='newapp';
UPDATE django_content_type SET app_label ='oldapp' WHERE app_label ='newapp';
"""
class Migration(migrations.Migration):
dependencies = [
('oldapp', '0001_initial'),
]
operations = [
migrations.RunSQL(sql, reverse_sql)
]
Set the model table name to the new_app.
Example:
class ExampleModel(models.Model):
# normal column definitions...
class Meta:
db_table = "newapp_examplemodel" # add this line to the meta class
Commit, test, and deploy the above changes.
Delete the migration created in step 1.
Rename/move the old app and fix all references (see https://stackoverflow.com/a/8408131/1608851 for more details), including references in migrations.
Remove the db_table overrides from the model meta classes.
Commit, test, and deploy.
Notes:
The django_migrations table will retain a row with the old app, because that row is added after our new migration was run, but before it was registered under the new app name. It should be okay to leave that as is.
Deleting the migration is due to the above point, but also to prevent future issues when reverse migrating, but also because
Reversing this app rename requires checking out / deploying the commit from step 3, then reversing our migration, then checking out / deploying the commit before that
Why not just use the option Find and Replace. (every code editor has it)?
For example Visual Studio Code (under Edit option):
You just type in old name and new name and replace everyhting in the project with one click.
NOTE: This renames only file content, NOT file and folder names. Do not forget renaming folders, eg. templates/my_app_name/ rename it to templates/my_app_new_name/
I added some models in my models.py and I want to add an admin class to use a wysiwyg-editor in text-fields.
Well, I know that Django itself doesn't support migrations and I've used South, but it doesn't work either.
South doesn't "see" the change.
Could it be, that South just detects changes to fields, but not if I add a new class?
How can I tweak Django to detect such changes?
syncdb and South are only concerned with descendants of Model in apps listed in INSTALLED_APPS. Everything else is handled by Django directly.
You seem to be very confused, unfortunately. Of course Django reads the code in models.py - otherwise what would be the point of it? Django uses that code initially to define the model SQL when doing syncdb, but it doesn't modify existing database tables in subsequent calls to syncdb - hence the need for South.
But naturally, Django uses models.py and admin.py and all the other Python code to define its own configuration and state. (And note that admin classes are not defined in models.py but in admin.py.)
If you are not seeing changes, you will need to restart your server.
I'm fairly sure that if you follow the steps as outlined in the tutorial to create an admin app it'll just work. Migration isn't an issue as the admin app creates new tables rather than altering the existing one.