I'm having trouble with Django south migrations. It may be related to how we've laid our project out but it was working previously, and it works fine locally.
I pushed new code last night that contained a migration in my_app app. So in my local environment...
$ ./manage.py migrate --list
socialaccount
(*) 0001_initial
(*) 0002_genericmodels
(*) 0003_auto__add_unique_socialaccount_uid_provider
(*) 0004_add_sites
(*) 0005_set_sites
(*) 0006_auto__del_field_socialapp_site
(*) 0007_auto__add_field_socialapp_client_id
(*) 0008_client_id
(*) 0009_auto__add_field_socialtoken_expires_at
(*) 0010_auto__chg_field_socialtoken_token
(*) 0011_auto__chg_field_socialtoken_token
payments
(*) 0001_initial
users
(*) 0001_initial
my_app
(*) 0001_initial
(*) 0002_auto__add_organizerapplication
In heroku, it doesn't recognize my_app as an app with migrations. When I attempt to migrate that app....
$ heroku run my_app/manage.py migrate my_app --app=my_app
Running my_app/manage.py migrate my_app attached to terminal... up, run.5016
The app 'my_app' does not appear to use migrations.
./manage.py migrate [appname] [migrationname|zero] [--all] [--list] [--skip] [--merge] [--no-initial-data] [--fake] [--db-dry-run] [--database=dbalias]
If I list the migrations, you'll notice my_app isn't listed...
$ heroku run my_app/manage.py migrate --list --app=my_app
Running my_app/manage.py migrate --list attached to terminal... up, run.8264
socialaccount
(*) 0001_initial
(*) 0002_genericmodels
(*) 0003_auto__add_unique_socialaccount_uid_provider
(*) 0004_add_sites
(*) 0005_set_sites
(*) 0006_auto__del_field_socialapp_site
(*) 0007_auto__add_field_socialapp_client_id
(*) 0008_client_id
(*) 0009_auto__add_field_socialtoken_expires_at
(*) 0010_auto__chg_field_socialtoken_token
(*) 0011_auto__chg_field_socialtoken_token
payments
(*) 0001_initial
users
(*) 0001_initial
I'm not sure it's relevant but my_app is listed in my INSTALLED_APPS when I check.
UPDATE
The issue was that this particular migrations dir was missing __init__.py. Running commands like convert_to_south through Heroku don't impact this as local file changes don't stick. Pushing the repo again with that file got the migrations recognized. I then just had to fake the first migration and I was good.
Make sure you have a init.py file in the migrations folder of the app you want to migrate. South will work locally, but not in production on heroku. Simply copy an init.py file from one of your apps, and put it into the migrations folder for the app you are getting the error for. Push that change to production, and then migrate.
Related
I am a beginner at the django and module and web applications.
Here is the traceback and the data:
OperationalError at /admin/learning_logs/example/
no such column: learning_logs_example.entry_id
Request Method: GET
Request URL: http://localhost:8000/admin/learning_logs/example/
Django Version: 1.11.7
Exception Type: OperationalError
Exception Value:
no such column: learning_logs_example.entry_id
Exception Location: C:\Users\Bryan\Desktop\LEARNI~1\ll_env\lib\site-
packages\django\db\backends\sqlite3\base.py in execute, line 328
Python Executable: C:\Users\Bryan\Desktop\LEARNI~1\ll_env\Scripts\python.exe
Python Version: 3.6.2
Python Path:
['C:\\Users\\Bryan\\Desktop\\learning log',
'C:\\Users\\Bryan\\Desktop\\LEARNI~1\\ll_env\\Scripts\\python36.zip',
'C:\\Users\\Bryan\\AppData\\Local\\Programs\\Python\\Python36-32\\DLLs',
'C:\\Users\\Bryan\\AppData\\Local\\Programs\\Python\\Python36-32\\lib',
'C:\\Users\\Bryan\\AppData\\Local\\Programs\\Python\\Python36-32',
'C:\\Users\\Bryan\\Desktop\\LEARNI~1\\ll_env',
'C:\\Users\\Bryan\\Desktop\\LEARNI~1\\ll_env\\lib\\site-packages']
Server time: Sat, 18 Nov 2017 18:36:14 +0000
For some reason, when I click that link that says "Examples" below,
it says this
I tried migrating but it says
(venv) C:\...\learning log>python manage.py makemigrations
No changes detected
(venv) C:\...\learning log>python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
No migrations to apply.
How did I get it to do that?
Well first I added the example entry/form with a TextField and then I added a "example". I added a ForeignKey object with the parameter as Entry.
I went back to edit my "example" but then the exception showed up.
EDIT:
Well janos wanted the output when I run python manage.py showmigrations learning_logs so here it is:
admin
[X] 0001_initial
[X] 0002_logentry_remove_auto_add
auth
[X] 001_initial
[X] 0001_initial
[X] 0002_alter_permission_name_max_length
[X] 0003_alter_user_email_max_length
[X] 0004_alter_user_username_opts
[X] 0005_alter_user_last_login_null
[X] 0006_require_contenttypes_0002
[X] 0007_alter_validators_add_error_messages
[X] 0008_alter_user_username_max_length
contenttypes
[X] 0001_initial
[X] 0002_remove_content_type_name
learning_logs
[X] 0001_initial
sessions
[X] 0001_initial
Well I hope that helped.
I recreated the exception page right here
As the error message says,
the column entry_id in the table learning_logs_example does not exist.
This is not normal.
If schema migrations were created correctly and applied correctly,
then this should not happen.
Moreover, looking at this:
(venv) C:\...\learning log>python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
No migrations to apply.
The Apply all migrations line should contain the name of your app.
The existing names are standard Django components.
The name of your app is probably learning_log or similar.
In short,
it seems that migrations don't exist for your app.
To debug this,
first of all check the output of showmigrations for your app:
./manage.py showmigrations learning_log
If you don't get an error, then please add the output to your question.
If you get an error CommandError: No migrations present for: ...,
then create the migrations:
./manage.py makemigrations learning_log
And then check the output of showmigrations again,
and please add the output to your question.
If the output looks something like this:
learning_log
[X] 0001_initial
...
That means the database has some records about applied migrations,
and in fact Django thinks that the migrations are correctly applied.
(When in fact they are not, based on your main error.)
With the information I have so far,
it seems your database got out of sync with the model:
some fields have been added to the model without applying to the database,
and the migrations were not correctly saved.
One simple solution could be to manually add the missing columns.
You can find the correct schema by looking at the migration file:
./manage.py sqlmigrate learning_log 0001
From this you should be able to find how the missing entry_id field is declared,
and then manually add it to the table with a query ALTER TABLE learning_logs_example ADD COLUMN entry_id ...,
where the rest of the parameters depend on the definition of the column in the output of sqlmigrate.
If the problem is only one or a few missing fields,
then this workaround may be enough to have something working.
(If not, then jump to the next section.)
You should not stop there though,
because the model may need some indexes as well.
You could either study further output of sqlmigrate and recreate all other objects that depend on learning_logs_example,
such as indexes, triggers.
But a cleaner solution will be to recreate all the tables of the app,
following these steps:
Dump the current data: ./manage.py dumpdata learning_log > learning_log.json
Drop all tables of the app
Recreate the tables of the app: ./manage.py migrate --fake learning_log zero; ./manage.py migrate learning_log
Restore the data: ./manage.py loaddata learning_log
If the workaround of adding columns manually doesn't work
(you stumble into more complicated problems),
then you have two choices:
If you cannot afford to lose data, well then you will have to track down and solve every single problem manually. After that's done, you should dump and restore the tables of the app as I explained in the previous section
If it's ok to lose data, then you can follow the procedure in the previous section, without the dumping and restoring steps. That is, drop and recreate the tables of the app
Finally,
to avoid further problems,
you need to make sure that all migrations are in sync with the model,
and are correctly added to version control.
A good way to test that is install the Django project in a different folder on your computer.
If that doesn't go smoothly,
then the setup is still not good.
Let me know if any of the above needs more clarification.
Is this the first time you are creating migrations for your application? If so, you need to specify the app name to make the migrations:
python manage.py makemigrations my-app
If data isn't matter, you can just drop your database file from here:
C:\Users\bryan_8nva8ki\Desktop\learning logs\db.sqlite3
(I saw that path from your logs)
Then you should make migrate again. Hope it will helps!
I have 2 related questions.
I have deleted all my migrations, including the migrations directory, from all the apps in my project. I've started with a clean database. But still, when I run ./manage.py makemigrations, django says there are no changes to be made. How do I completely start over with migrations? No squashing, just starting over.
It seems that when I call makemigrations, django consults the database. I'd like my codebase to be the only source of truth for my migrations. Is there a way to do this?
If an app doesn't have a migrations/ directory with an __init__.py file (even on Python 3), Django won't create any migrations if you don't specify the app name. You either have to create the directories and __init__.py files, or you have to call ./manage.py makemigrations <app_name> for each app, which forces Django to create the directory.
It doesn't. It may connect to the database during the makemigrations command, but the migrations are purely based on your models.
I just clone exiting project from github, and dump mysql database in my local machine. Every thing is working fine. I made some changes in myapp/model.py, like add new tables. After that when run
1. python ./manage.py makemigrations myapp. Then it makes migration files like
Migrations for 'myapp':
0001_initial.py:
- Create model AndroidRegkey
- Create model ApkVersion
.......................
.......................
python manage.py migrate myapp it gives following message
Operations to perform:
Apply all migrations: myapp
Running migrations:
No migrations to apply.
This is the first time migration on my local machine. I already have database. But after Adding new models in model.py first time it does not apply any migration to data base, why?
I also go this link stack-over flow but not working.
When I ran python manage.py migrate --list Getting following result.
admin
[X] 0001_initial
auth
[X] 0001_initial
contenttypes
[X] 0001_initial
intracity
(no migrations)
mailer
[X] 0001_initial
[X] 0002_auto_20150720_1433
sessions
[X] 0001_initial
When you apply migrations, the migrations that have been applied are stored in the database. They will also be exported to your database dump. So your database should already be in the correct state after you import the data locally. Django looks at the relevant table, sees you're up to date, and takes no further action.
I have a Django app that I added South to, performed some migrations with, and runs as expected on my local machine. However, I have had nothing but database errors after pushing my project to Heroku.
In trying to deal with one database error I am experiencing, I attempted a test where I deleted one of my models, pushed the edited models file to Heroku and ran:
heroku run python manage.py schemamigration django_app test_remove_pub --auto
This seemed to work fine. I got back the message:
Running `python manage.py schemamigration apricot_app test_remove_pub --auto` attached
to terminal... up, run.6408
- Deleted model django_app.Publication
- Deleted M2M table for journalists on django_app.Publication
- Deleted M2M table for tags on apricot_app.Publication
Created 0006_test_remove_pub.py. You can now apply this migration with: ./manage.py
migrate django_app
So, South appeared to do everything I expected - it deleted my model and its many to many relationships and made the appropriate migration file. Next, I enter:
heroku run python manage.py migrate django_app
And I get back:
Running `python manage.py migrate django_app` attached to terminal... up, run.4792
Running migrations for django_app:
- Nothing to migrate.
- Loading initial data for django_app.
Installed 0 object(s) from 0 fixture(s)
Why would it say "nothing to migrate" when obviously there are things to migrate??
After you change the models, you need to do :
heroku run python manage.py schemamigration django_app --auto
if anything has changed
and then run
heroku run python manage.py migrate django_app
South applies migrations based on the entries in this database table: south_migrationhistory. So if you want to manually override it,
Remove all the entries with column app_name of the models you changed
Manually remove all the related tables. You can get a list of all the tables by typing this in the django shell:
from django.db.models import get_app, get_models
app = get_app(app_name)
for model in get_models(app, include_auto_created=True):
print model._meta.db_table
Remove the migrations/ folder related to the app
Do a fresh migration: ./manage.py schemamigration app_name --initial
Apply the migration ./manage.py migrate app_name
I've accumulated quite a few migrations using South (0.7) and Django (1.1.2) which are starting to consume quite a bit of time in my unit tests. I would like to reset the baseline and start a fresh set of migrations. I've reviewed the South documentation, done the usual Google/Stackoverflow searching (e.g. "django south (reset OR delete OR remove) migration history") and haven't found anything obvious.
One approach I've contemplated would involve "starting over" by "removing" South or "clearing" the history manually (e.g. clear the db table, remove migration files from the migrations director) and just re-run,
./manage.py schemamigration southtut --initial
So, if anyone has done this before and has some tips/suggestions they would be greatly appreciated.
If you need to selectively (for just one app) reset migrations that are taking too long, this worked for me.
rm <app-dir>/migrations/*
python manage.py schemamigration <app-name> --initial
python manage.py migrate <app-name> 0001 --fake --delete-ghost-migrations
Don't forget to manually restore any dependencies on other apps by adding lines like depends_on = (("<other_app_name>", "0001_initial"),("<yet_another_app_name>", "0001_initial")) to your <app-dir>/migrations/0001_initial.py file, as the first attribute in your migration class just below class Migration(SchemaMigration):.
You can then ./manage.py migrate <app-name> --fake --delete-ghost-migrations on other environments, per this SO answer. Of course if you fake the delete or fake the migrate zero you'll need to manually delete any left-over db tables with a migration like this.
A more nuclear option is to ./manage.py migrate --fake --delete-ghost-migrations on the live deployment server followed by a [my]sqldump. Then pipe that dump into [my]sql on the environments where you need the migrated, fully-populated db. South sacrilege, I know, but worked for me.
EDIT - I'm putting a comment below at the top of this as it's important to read it before the > accepted answer that follows #andybak
#Dominique: Your advice regarding manage.py reset south is dangerous
and may destroy the database if there are any third party apps using
south in the project, as pointed out by #thnee below. Since your
answer has so many upvotes I'd really appreciate it if you could edit
it and add at least a warning about this, or (even better) change it
to reflect #hobs approach (which is just as convenient, but doesn't
affect other apps) - thanks! – chrisv Mar 26 '13 at 9:09
Accepted answer follows below:
First, an answer by the South author:
As long as you take care to do it on all deployments simultaneously, there shouldn't be any problem with this. Personally, I'd do:
rm -r appname/migrations/
./manage.py reset south
./manage.py convert_to_south appname
(Notice that the “reset south” part clears migration records for ALL apps, so make sure you either run the other two lines for all apps or delete selectively).
The convert_to_south call at the end makes a new migration and fake-applies it (since your database already has the corresponding tables). There's no need to drop all the app tables during the process.
Here's what I'm doing on my dev + production server when I need to get rid of all these unneeded dev migrations:
Make sure we have the same DB schema on both sides
delete every migrations folder on both sides
run ./manage.py reset south (as the post says) on both sides = clears the south table *
run ./manage.py convert_to_south on both sides (faking 0001 migration)
then I can re-start to make migrations and push the migrations folders on my server
* except if you want to clean only one app among others, if so you'll need to edit your south_history table and delete only the entries about your app.
Thanks to the answers by Dominique Guardiola and hobs, it helped me solve a hard problem.
However there are a couple of issues with the solution, here is my take on it.
Using manage.py reset south is not a good idea if you have any third party apps that uses South, for example django-cms (basically everything uses South).
reset south will delete all migration history for all apps that you have installed.
Now consider that you upgrade to the latest version of django-cms, it will contain new migrations like 0009_do_something.py. South will surely be confused when you try to run that migration without having 0001 through 0008 in the migration history.
It is much better/safer to selectively reset only the apps that you are maintaining.
First of all, make sure that your apps don't have any desync between migrations on disk, and migrations that have been executed on the database. Otherwise there will be headache.
1. Delete migration history for my apps
sql> delete from south_migrationhistory where app_name = 'my_app';
2. Delete migrations for my apps
$ rm -rf my_app/migrations/
3. Create new initial migrations for my apps
$ ./manage.py schemamigration --initial my_app
4. Fake execute the initial migrations for my apps
This inserts the migrations into south_migrationhistory without touching actual tables:
$ ./manage.py migrate --fake my_app
Step 3 and 4 is actually just a longer variant of manage.py convert_to_south my_app, but I prefer that extra control, in such delicate situation as modifying the production database.
Like thnee (see her answer), we're using a gentler approach to the South author's (Andrew Godwin) suggestion quoted elsewhere here, and we're separating what we do with the code base from what we do to the database, during deployment, because we need the deployments to be repeatable:
What we do in the code:
# Remove all the migrations from the app
$ rm -fR appname/migrations
# Make the first migration (don't touch the database)
$ ./manage.py schemamigration appname --initial
What we do to the database once that code is deployed
# Fake the migration history, wiping out the rest
$ ./manage.py migrate appname --fake --delete-ghost-migrations
If you are just working on the dev machine, I wrote a management command that does pretty much what Dominique suggested.
http://balzerg.blogspot.co.il/2012/09/django-app-reset-with-south.html
In contrast of the south author suggestion, this will NOT HARM other installed apps using south.
Following is only if you want to reset all apps. Please backup your all databases prior to that work. Also note down your depends_on in initial files if there are any.
For once:
(1) find . -type d -name migrations -exec git rm -rf '{}' \;
(2) find . -type d -name migrations -exec rm -rf '{}' \;
(3) ./manage.py schemamigration <APP_NAME> --initial
(4) [GIT COMMIT]
Test bootstrapping your project before push. Then, for each local/remote machine, apply following:
(5) [GIT PULL]
(6) ./manage.py reset south
(7) ./manage.py migrate --fake
Do initial (3) for each app you want to re-involve. Note that, reset (6) will delete only migration history, therefore not harmful to libraries. Fake migrations (7) will put back migration history of any 3rd party apps installed.
delete necessary file from app folder
instance path
cd /usr/local/lib/python2.7/dist-packages/wiki/south_migrations
wiki -is my app