Recovering from a duplicate migration in Django South - python

Due to merging several feature branches into my project, I have the following migrations:
0001_initial.py
0002_auto__add_field_userprofile_telephone__add_field_userprofile_timezone.py
0003_auto.py
0004_auto__del_field_organisation_admin.py
0005_auto__add_field_organisation_permitted_domains.py
0005_auto__add_field_userprofile_currency.py
Note that I have two duplicate 0005 migrations. These ran fine, and deployed fine on my production system.
$ python manage.py migrate accounts --list [17:11:42]
accounts
(*) 0001_initial
(*) 0002_auto__add_field_userprofile_telephone__add_field_userprofile_timezone
(*) 0003_auto
(*) 0004_auto__del_field_organisation_admin
(*) 0005_auto__add_field_organisation_permitted_domains
(*) 0005_auto__add_field_userprofile_currency
My table has the correct columns:
$ psql
db_my_project=# \d+ accounts_organisation
db_my_project=# \d+ accounts_userprofile
... shows currency and permitted_domain, suggesting the migrations worked correctly
However, if I try to create a new migration, South thinks that I haven't added the column' permitted_domains' to my model:
$ python manage.py schemamigration accounts --auto [17:16:15]
+ Added field permitted_domains on accounts.Organisation
Created 0006_auto__add_field_organisation_permitted_domains.py. You can now apply this migration with: ./manage.py migrate accounts
How do I fix this?

From the docs: http://south.readthedocs.org/en/0.7.6/autodetector.html
When the autodetector runs, it compares your current models with those
frozen in your most recent migration on the app, and if it finds any
changes, yields one or more Actions to the South
migration-file-writer.
The migrations keep a frozen version of the fields within the model in a dict.
Therefore:
In 0005_auto__add_field_organisation_permitted_domains the Organisation class will have a field permitted_domains but in 0005_auto__add_field_userprofile_currency it will not. When you run:
$ python manage.py schemamigrate accounts --auto
this will compare the current state of the code against the record of the fields store in 0005_auto_add_field_userprofile_currency, thus leading south to add the field for a second time.
If you copy the line for the 'permitted_domains' field from 0005_auto__add_field_organisation_permitted_domains to 0005_auto__add_field_userprofile_currency this will solve your problem.

It is a very specific problem, I hope this helps, do the following:
1) Rename this file: 0005_auto__add_field_organisation_permitted_domains to 0006_auto__add_field_organisation_permitted_domains
2) Rename the number of your recently migration file from 0006 to 0007
3) Issue the command python manage.py migrate account 0006 --fake to deceive south.
4) Issue the command python manage.py migrate account 0007
That might get the south engine in sycn again with your application
Hope this helps!

Related

Django 1.8 migrate is not creating tables

yekabathula-macbookair2:roster yekabathula$ python manage.py migrate
Operations to perform:
Synchronize unmigrated apps: staticfiles, messages
Apply all migrations: admin, contenttypes, api, auth, sessions
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying api.0001_initial... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying sessions.0001_initial... OK
yekabathula-macbookair2:roster yekabathula$ python manage.py syncdb
/Library/Python/2.7/site-packages/django/core/management/commands/syncdb.py:24: RemovedInDjango19Warning: The syncdb command will be removed in Django 1.9
warnings.warn("The syncdb command will be removed in Django 1.9", RemovedInDjango19Warning)
Operations to perform:
Synchronize unmigrated apps: staticfiles, messages
Apply all migrations: admin, contenttypes, api, auth, sessions
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
No migrations to apply.
After doing python manage.py migrate, tables are not created in database from my models.py it is able to create other tables from django_session etc. Is there anything else that I need to follow here ?
I was facing a similar problem in Django 1.10 and none of the above solutions worked for me.
What eventually worked was running this command:
python manage.py migrate --fake myappname zero
This reset all migrations (to the zeroth state)
This followed by :
python manage.py migrate myappname
created the tables for me.
If you do not want to roll back to the initial(zero) state but say to the migration number 0005(the last migration that worked), you can instead do this:
python manage.py migrate --fake myappname 0005
And then proceed with the actual migrate:
python manage.py migrate myappname
More details in the docs
In my case the __init__.py file was missing from the APP/migrations/ folder. If you don't have one, all it needs is an empty __init__.py file.
I ran into the same problem. After lots of digging, I found the solution.
I'm using django 1.11.
If you want to start-over,
1)delete all the files in your migrations folder except __init__.py
2)drop database
3)create database
4)python makemigrations
5)python migrate
if you have reset_db, instead of 2nd and 3rd steps you can use reset_db.
python manage.py reset_db
I am using MySQL and get into this issue after deleting 0001_initial.py migration file and all the custom tables evolved from DB to try to regenerate all they...
Solved this issue simply deleting these rows in django_migrations table...
After that, $ python manage.py migrate command regenerates all my custom tables again.
I had a similar problem and just figured it out.I have multiple databases. My local one (the one not being updated) is a MySQL database. The others are MS SQL Server and MySQL. I have routers to the other databases since I do not manage them and had (in Django 1.6) used the routers to indicate allow_sync() = False. With 1.7, I changed that to allow_migrate() = False. BUT I DID NOT ADD A ROUTER FOR MY LOCAL DATABASE. The default appears to be allow_migrate() = False if there is not one. As a result, the migrations just failed silently ( Reference: https://docs.djangoproject.com/en/1.7/topics/db/multi-db/). I added a router for my local DB, setting allow_migrate() to return True and now my migrations actually create my tables.
Change managed = True (if it is set to False) in models.py
class Meta:
managed = False
db_table = 'table_name'
To
class Meta:
managed = True
db_table = 'table_name'
I had a similar issue. I did everything stated above but nothing worked. Then I realized I had not added my model name in admin.py file in my app.
Along with everything stated above you have to also add your model name in admin.py file. You have to add it like this:
admin.site.register(model name)
This can be very frustrating, but here is method that worked for me.
First, if you've deleted the migration file go ahead and create it again
python manage.py makemigrations
Then compare the migration file with the SQL raw code and if it fit then apply migration
python manage.py sqlmigrate [app_name] 0001
python manage.py migrate
or you can simply pipe it from your command line or terminal
python manage.py sqlmigrate [app_name] 0001 | [psql] db_name username
BEGIN
CREATE TABLE
ALTER TABLE
COMMIT
NOTE: In my case I'm using postgresql. Though engine is similar to other database langauge, you might not get expected result.
Delete existing tables of the models in MySQL database.
Delete migration folder under app folder.
Delete all relative migration records in the table
"django_migrations" from MySQL.
Now you get clear model and database. Use
python manage.py makemigrations and python manage.py migrate
to create tables.
Hope to help you.
try this one,
run,
python manage.py makemigrations app_name
above command will make migrations, if successful then run second command or
check if you have typo in Installed_app and also check for AppConfig module
python manage.py migrate app_name
if above was successful, this will create the table in db
This solved the problem for me (I am using MySQL workbench by the way):
Run this sql: SET FOREIGN_KEY_CHECKS = 0;
Select all the tables in your django database (click on the first table, then press and hold shift, then click on the last table). Then right click and choose "Drop n tables" (where n is the number of tables you just selected)
then run python manage.py migrate
Finally restore foreign key check settings by running this sql: SET FOREIGN_KEY_CHECKS = 1;
Note: Before taking this drastic measure, I tried what Paulo Pessoa said in his comment, but still I got "No migrations to apply." messages. However, this solved the issue.
If you want to do it only for your application not from all the apps then Clear your application migrations records from 'django_migrations' table in your DB.
Python manage.py makemigrtions
python
Problem: : When you apply migrations in django for the first time, django creates table of that model in database and marks somewhere in its own file(class):
`initial = True`
When you then tries to alter the schema of that table it firstly checks
if initial = True
if an initial class attribute isn’t found, a migration will be considered “initial”
In case the initial = True we need to use 
python manage.py migrate --fake-initial
For an initial migration Django checks that all of those tables already exist in the database and fake-applies the migration if so. Similarly, for an initial migration that adds one or more fields Django checks that all of the respective columns already exist in the database and fake-applies the migration if so.
Fake initial migration uses both CreateModel() and AddField() methods.
Solution:
>> python manage.py makemigrations <AppName>
>> python manage.py migrate --fake-initial
To avoid deleting critical databases, you may also consider silencing properties below the Class Meta: for example this model:
class Blog(models.Model):
category = models.CharField(max_length=100)
title = models.CharField(max_length=100)
date_added = models.DateTimeField()
merits = models.CharField(max_length=300, blank=True)
demerits = models.CharField(max_length=300, blank=True)
class Meta:
managed = True
db_table = 'weblog'
verbose_name_plural = "Blog"
#property
def content_min(self):
return truncatechars(self.content, 50)
You can then run makemigrations and migrate and your table will be created.
Check if you have added your created app to you installed apps under settings.py.
The list should seem like this:
INSTALLED_APPS = [
'django.contrib.admin','django.contrib.auth','django.contrib.contenttypes',
'django.contrib.sessions', 'django.contrib.messages','django.contrib.staticfiles',
'YourCreatedApp',
]
in my case just one app's tables of project doesn't recreate!
and this line of code recreate them and successfully run!
python manage.py migrate --run-syncdb
however i couldn't find main reason caused this kind of problem!
Delete database
delete migration folder
run migrate command
run makemigrations command
run migrate command
It will create all tables perfectly
Change from sqlite3 to mysql in settings.py
Make sure you have correct information regarding database name and username-password
Delete existing migration
Make migration
And then migrate
Make sure each app's migrations folder has a __init__.py file.
I had a similar issue, the connection with DB was set correctly as all the django admin tables were created on DB side. However, no models appeared in models.py
What worked for me is running in the console:
python manage.py inspectdb > models.py
which wrote everything into a new models.py file which then I replaced with the one I had in the app folder. Then I could alter the
managed = False
into
managed = True
here the link to the documentation: https://docs.djangoproject.com/en/3.1/howto/legacy-databases/
If you are a begginer like me make sure your classes extend models.Model. Then makemigrations and migrate.
class Cook(models.Model):
email = models.Field()
I solved a similar problem by doing this:
Delete all migration files and pycache so what retains is init.py
Comment all implementations of that table in models, serializers, admin, urls and views
Do python manage.py makemigrations
Do python manage.py migrate
Then, uncomment all implementations of that table in models, serializers, admin, urls and views
Then, do python manage.py makemigrations
Then, do python manage.py migrate
To elaborate on #Leman-Kirme 's answer.
I had a similar problem where the table for the model I added into models.py of one of my apps wouldn't get created when making and applying migrations. I tried different solutions but nothing seemed to work and I really didn't want to flush my databases. So what I did was:
Delete all the files in the migrations folder except for _init_.py
Delete all records in databases' django_migrations tables.
Manually make a migration file (copied from another project and changed the content so that migration would only make a model that I was having troubles with). Please, note, that it seems like it should be initial migration, i.e. 0001_initial.py
Run it.
Voila, here comes the table!
If you want / need, instead of deleting previous migrations' files and records in the databases, you could backup them and restore afterwards. You gonna need to edit name of your manually-created migration from 0001_initial to something like <number_of_existing_migrations>_fix
That worked for me (but I ignored step 6), I hope someone finds this useful as well!
Gist: => Delete entry from django_migrations and run the migrate command again.
This answer takes help from other answers, but still writing as there are different cases and my answer might help in your case.
After creating migration, I think by mistake I ran the command python manage.py migrate campaign --fake I am not sure if it did some wrong thing, so now running python manage.py migrate campaign was not creating tableSo what solved for me is--
in mysql shell run
select * from django_migrations; Note the id of last migration which is creating problem in my case it was 62delete from django_migrations where id = 62Now I ran the migration again, and it worked this time. with command python manage.py migrate campaign i.e. required table was created as a result of applying migration
The problem I discovered working with Postgres is that I had not set the schema. I the original question is one of those gotchas that may have a number of different answers, so if the other answers are not working for you just check this.
The "problem" is that Postgres has an additional "namespace" layer which means you can specify the database but there can still be a whole set of tables, still in the same database, but in another schema or namespace.
The tables in this schema may be unreachable because the user being used to connect does not have permissions, or in fact the schema has been specified, but the tables were created in a different schema, or even you haven't specified the schema this time and are looking in the public schema while the tables have been created in another schema.
So the solution is to set the schema correctly, to what you intend, in every location and place that you use it. This boils down to correct database settings in all the settings files you are using, and logging in to the right schema and user when you inspect the database manually.
For example:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'OPTIONS': {'options': f'-c search_path={"foobar"}'},
'NAME': 'foo',
'USER': 'bar',
'PASSWORD': 'barpass',
'HOST': 'baz.example.com',
'PORT': 5432,
},
}
The database name here is foo, the username is bar and the schema name is foobar. Adding this line to the OPTIONS parameter defaults the schema for all operations.
For other usage, within Postgres, you can display the current search_path, and alter it for the current session, or "permanently" for the database:
SHOW search_path;
SET search_path=foobar,public;
ALTER DATABASE foo SET search_path TO foobar;
If your problem was that Django was creating your tables, but just in the wrong schema or under different permissions, I hope this has resolved your issue.
This kind-of thing was happening to me when I would try to reset my DB. Nothing else would work to get django to recognize my tables except:
python manage.py migrate --run-syncdb
In my case those solutions didn't work to me
I have to:
Create a new app: py manage.py startapp new_app_name
Copy all the files in my new app
Run: py manage.py makemigrations
Run: py manage.py migrate
It works to me, I hope it could be useful for you too
Below commands worked for me
python manage.py makemigrations app_name
python manage.py migrate app_name
I had the same issue and lol I just forgot to change my engine to the needed one

Django South migrations after migrate starts from initial

I added field to my model in banks app, run python manage.py schemamigration banks --auto which generates correct migration but after migrate, don't know why South start over from initial migration.
python manage.py migrate banks
Running migrations for banks:
- Migrating forwards to 0038_auto__add_field_offer_description.
> banks:0001_initial
FATAL ERROR - The following SQL query failed: CREATE TABLE `banks_lendingtarget` (`id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `name` varchar(255) NOT NULL UNIQUE)
The error was: (1050, "Table 'banks_lendingtarget' already exists")
Even if I start migrate with specified migration name like:
python manage.py migrate banks 0038_auto__add_field_offer_description
There is the same error, and my question is: why?
I was thinking that recently added initial_data.json to my app can cause this but after renaming it nothing changes. Have anyone of You have same situation? Thanks for help.
This is assuming you are using South 0.8 or so and Django 1.6 or below:
I can't explain why this happened without knowing more about the history of your database and South usage, but to help in diagnosing the issue you could manually inspect the South migration history table in your database.
However, whatever you find out, the remedy is likely to be the same: you can "fake" the migration and skip ahead to the proper place in your migration history.
Try python manage.py migrate banks 0037 --fake. That will take you to just BEFORE the 0038 migration without actually trying to create tables. Of course, this assumes that 0037 is the most recent migration that was successfully applied to your database.

Django and South: South not detecting when django models change

I've wasted all my day trying to make this work, but I couldn't.
Here's what I've done so far:
Created an app and its models (no south at this point)
Installed South on my system
Added 'south' to my project settings
Ran manage.py syncdb to create south migration tables
Converted my app to south using: manage.py convert_to_south myapp. This created a migrations folder and an initial migration file.
Changed my model (added a new field)
Ran manage.py schemamigration myapp --auto
It says NOTHING SEEMS TO HAVE CHANGED.
Here's my original model:
https://gist.github.com/kustomrtr/8f4f4262634904b53d5f
Here's my MODIFIED model (added line 41):
https://gist.github.com/kustomrtr/edab10975803c74b869a
Here's my initial migration file (created when I converted my app to south):
https://gist.github.com/kustomrtr/2a4884be3177225a45e7
Can you help me please!
Thanks in advance.
Someone pointed that the Manage = False in my models could cause the trouble. I tried commenting those lines and guess what:
C:\Users\Kevin\Desktop\difundelo>python manage.py schemamigration registros --auto
? The field 'Post.image' does not have a default specified, yet is NOT NULL.
? Since you are adding this field, you MUST specify a default
? value to use for existing rows. Would you like to:
? 1. Quit now, and add a default to the field in models.py
? 2. Specify a one-off value to use for existing columns now
? Please select a choice:
Voilá! It worked! I can finally keep working on my project!
PS: I don't know why the guy who commented the solution, deleted his comment. Thank you anonymous!!!

How to run test with migrate appname --fake?

My test not working. If I try python manage.py test appname I have this error:
! You *might* be able to recover with: = DROP TABLE "appname_userprofile"; []
= DROP TABLE "appname_table2"; []
= DROP TABLE "appname_table3"; []
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: content:0015_initial
django.db.utils.DatabaseError: table "appname_userprofile" already exists
How to run my python manage.py test appname with
manage.py migrate appname --fake
You'd have to write a custom test runner to selectively add --fake to individual migrations, as far as I know.
You should probably fix your database migrations -- it looks like you have two migrations which are trying to create the same table.
South wants to run all of your migrations, in order, before starting the tests, in order to build up an initial database, and right now it can't do that.
You can disable South completely for unit tests if you put this in your settings.py file (reference):
SOUTH_TESTS_MIGRATE = False
If you do that, then the Django test runner will just create your test database based on your current models, rather than running migrations to build it.

You cannot add a null=False column without a default value

I'm adding a new app, and while setting up the database using South I get the following:
... line 11, in forwards
db.add_column('experiments_dailyreport', 'test_group_size',
orm['experiments.dailyreport:test_group_size'])
You cannot add a null=False column without a default value.
Given that this is a new table with no data in it, is there some way to force this migration?
You can force a migration using:
manage.py migrate --fake django-lean 0005
where 0005 is the version number of the migration. All that matters in your situation are:
having the correct database schema in the end
having South think that all migrations have been run
After that you can run the other migrations as normal. Alternatively, you can remove South, create the latest tables from django-lean using syncdb and then fake all the django-lean migrations.
Lastly, if you're certain that there's something wrong with the migration, it's worth contacting the django-lean developer(s) about this.

Categories