Django migration is not changing database in AWS Elastic Beanstalk - python

I have deployed my Djnago app using AWS Elastic Beanstalk. I have added some data and everything was working fine. Then I have increased max_length of one of the CharField in model.py. I have deployed the app again using 'eb deploy' command. But it did not increase the field length in database. If I try to add subject_id greater than 10, I get this error: django.db.utils.DataError: value too long for type character varying(10)
Model.py:
class Subject(models.Model):
#subject_id = models.CharField(max_length=10, blank=True)
subject_id = models.CharField(max_length=50, blank=True)
0001_initial.py:
#Generated by Django 3.0.4 on 2021-02-18 18:54
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
initial = True
dependencies = [
('accounts', '0001_initial'),]
operations = [
migrations.CreateModel(
name='Subject',
fields=[('subject_id', models.CharField(blank=True, max_length=50)),],),]
I have added migration command in .ebextensions/django_build.config file. The 0001_initial.py file inside migration folder shows the changes. But it is not updated in the database(AWS RDS postgresql). I have checked django_migrations table in postgresql database. It's showing the last migration happened when I first created the instance.
I need to change subject_id field length of Subject model in existing database. Any help will be appreciated.
.ebextensions/django_build.config:
container_commands:
01_create_log_folder:
command: "mkdir -p log && chown webapp:webapp -R log"
02_source_env:
command: "source /etc/profile.d/sh.local"
leader_only: true
03_database_migrations:
command: "source /var/app/venv/*/bin/activate && python3 manage.py makemigrations --noinput && python3 manage.py migrate --noinput && deactivate"
leader_only: true
django_migrations table:
select * from django_migrations;
id | app | name | applied
----+-----------------+------------------------------------------+-----------------------
28 | fileupload | 0001_initial | 2021-01-22 11:42:40.726471+00

Since your revised migration has the same name (0001_initial) as the one you've already applied, it is not being executed. You need to either:
Create a new migration (0002_new_migration) that alters the fields you created in the first one
OR
First rollback the migration you have and then apply the revised one
To rollback the migration you have, you'll need to SSH into the ELB instance:
Login via SSH - check your AWS console for specific instructions
Then run the following to reset the accounts migrations
source /opt/python/run/venv/bin/activate
source /opt/python/current/env
cd /opt/python/current/app
./manage.py migrate accounts zero
When you deploy next, you'll be starting the accounts model from scratch and your new migration will run.
This is no different than when you reverse migrations locally using manage.py and migrate, except you're doing it on your remote instance.

Related

How to fix: Django error: no such table: main.classroom_student__old

I am following this tutorial for learning how to create a Django (v2.0.1) app with multiple user types (teachers and students in this case). I cloned the associated code from the Github Repository, migrated the pre-made migrations and ran the site on localhost using:
python3 manage.py migrate
python3 manage.py runserver
The site works almost perfectly (Teacher sign up, login, quiz creation, and student log in are all in order). However, the student Signup fails with the following error:
OperationalError at /accounts/signup/student/
no such table: main.classroom_student__old
With the traceback pointing eventually to the file
django_school/classroom/forms.py, line 39:
student.interests.add(*self.cleaned_data.get('interests'))
That line comes from the definition of the following class in that forms.py file:
class StudentSignUpForm(UserCreationForm):
interests = forms.ModelMultipleChoiceField(
queryset=Subject.objects.all(),
widget=forms.CheckboxSelectMultiple,
required=True
)
class Meta(UserCreationForm.Meta):
model = User
#transaction.atomic
def save(self):
user = super().save(commit=False)
user.is_student = True
user.save()
student = Student.objects.create(user=user)
student.interests.add(*self.cleaned_data.get('interests'))
return user
What I have tried:
Following answers to the many similar questions on this site, I assumed it was a migration issue so I tried running:
python manage.py migrate --run-syncdb
Operations to perform:
Synchronize unmigrated apps: crispy_forms, humanize, messages, staticfiles
Apply all migrations: auth, classroom, contenttypes, sessions
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Running migrations:
No migrations to apply.
But the error persisted. I then deleted db.sqlite3 as well as all migration files associated with the app classroom. I then ran python3 manage.py makemigrations followed by python manage.py migrate --run-syncdb, again to no avail.
This makes me think that this is an issue with the way the code is adding "interests" to the "student" user object. Indeed, commenting out the line in question stops the error and creates the new student user, however this has the obvious issue that the student has no interests stored.
Running python manage.py sqlmigrate classroom 0001 shows:
...
-- Add field quizzes to student
--
ALTER TABLE "classroom_student" RENAME TO "classroom_student__old";
CREATE TABLE "classroom_student" ("user_id" integer NOT NULL PRIMARY KEY REFERENCES "classroom_user" ("id") DEFERRABLE INITIALLY DEFERRED);
INSERT INTO "classroom_student" ("user_id") SELECT "user_id" FROM "classroom_student__old";
DROP TABLE "classroom_student__old";
COMMIT;
So the database in question (classroom_student__old) is getting created and deleted when another attribute "quizzes" is added to the student user object. Could this be creating the issue?
Instead of downgrading to sqlite 3.24.0, another solution for making the tutorial you listed run successfully is to change the requirements.txt file's django version to 2.2.7 before running the "pip install -r requirements.txt" command.
So, your requirements.txt file for this project would be:
Django==2.2.7
django-crispy-forms==1.7.0
pytz==2017.3
Just to close the question, #SwapnilBhate had the correct answer; it was the sqlite version 3.26.0. After downgrading to 3.24.0, deleting the directory, and reinstalling, everything works perfectly.
1.0) First try:
python manage.py migrate YOURAPPNAME --fake
2.0) If it still doesn't working delete all the migrations files inside your app migrations(except init.py) and pycache folder. After that run makemigrations and migrate.
Use django==2.2.7
then python manage.py migrate,
and finally python runserver

I can't migrate at another database

I have well working project on local. I use postresql.
Ok. I create another database in postgres locally and specify new name/user/password in settings.py of project.
When I do
$ python manage.py makemigrations
I get error as
relation 'report_person' not exist
And I have tried delete (and not delete) directory 'migrations'. delete pycache. I tried specify Sqlite3 as database - but the same error.
Why I ask?
My project don't want to work with another database)
I succesfully push project on heroku. Specify parameters in settings.py. Create postgresql. But I can't make migrate on heroku) the same error.
I do makemigrations and git commit before push on heroku, but no result
Help me please.
Thank you
Have a nice day!
try migrating with --fake
python manage.py migrate --fake
python manage.py migrate yourappname --fake
and then again
python manage.py makemigrations
python manage.py migrate
Oh, I can't explain this situation. All problems in one little model.
It is from models.py:
from django import models
class Person(models.Model):
name = models.CharField(max_length=255)
position = models.CharField(max_length=255)
This part of forms.py (crazy):
from django import form
q = models.Person.objects.all()
qty = range(len(q))
PersonForm = type('PersonForm',
(forms.Form,),
{'person'+'_'+str(q[i].id): forms.BooleanField(label=q[i].name,
required=False) for i in qty})
And I must before migrations delete this 'PersonForm'. Then migrate will be succesfully in new any database. And after migrations I put this form again.
Ok. I don't undestand, why?)

How to recreate a deleted table with Django Migrations?

There are two models Groups and Students and only one table for Groups of them, the Students table was deleted.
How to make Django recreate the deleted table? If I do makemigrations it prints "No changes detected".
On admin page when I click on the Students table it throws an exception:
relation "students_students" does not exist
In django 1.7 you can try:
1. Delete your migrations folder
2. In the database: DELETE FROM django_migrations WHERE app = 'app_name'.
You could alternatively just truncate this table.
3. python manage.py makemigrations
4. python manage.py migrate --fake
If you are working in django 1.9.5 this is the 100 % solution for this problem:
1. Delete your migrations folder
2. In the database: DELETE FROM django_migrations WHERE app = 'app_name'.
You could alternatively just truncate this table.
3. python manage.py makemigrations app_name
4. python manage.py migrate
This works 100% for me!
Jan 2021
I had a migration problem and I had to drop/delete a table by pgadmin. Then, when I makemigrations and migrate the table wasn't recreated. In this way, I've found this procedure which worked for me:
python manage.py migrate --fake app_name zero
python manage.py migrate app_name
[NOTE]
If you don't have the intended migration file, create that before the above commands by python manage.py makemigrations
If you don't want to roll back to the initial(zero) state use the number of migration file instead of zero, e.g. python manage.py migrate --fake myappname 0005
I tested this approach in Django 2.2
Read More
There isn't an easy way to get Django to recreate a table that you have deleted manually. Once your database is altered manually, Django's view of the database (from migrations) is different from reality, and it can be tricky to fix.
If you run the sqlmigrate command, it will show you the required SQL to create the table. You can run the sql in a database shell. Assuming your app name is students, and the migration that created the table was 00XX_create_students.py, you would do:
./manage.py sqlmigrate students 00XX_create_students
Be careful if there are foreign keys to or from the students table, the constraints will have to be created as well.
The only way that worked for me:
rm -r <app-name>/migrations/
python manage.py makemigrations <app-name>
python manage.py sqlmigrate <app-name> 0001_initial
Copy what it prints out (or, depending on what you have actually removed from the DB, only part of the SQL queries).
Apply those copied queries to your DB:
psql -U user_name -h 127.0.0.1 database_name
Paste what you have copied from the SQL queries printout.
Commit the queries.
And that's it - your missing tables are created.
The answer that worked for me is as follows:
Assume in your database a table of a model has been deleted and you need to re-create, then do the following.
comment out the model in models.py that creates the table that has been deleted (either the model class or a line that creates a table like a = models.ManyToManyField(...))
run: python manage.py makemigrations <app-name>, where <app-name> is the name of of the app where you have models.py
run: python manage.py migrate --fake <app-name>
un-comment the model in models.py
run: python manage.py makemigrations <app-name>
run: python manage.py migrate <app-name> (without the --fake)
and you the table should be back in the database. But any data that was in the table will be lost.
Delete the migration folder from your migration app folder and simply run the migration commands:
python3 manage.py makemigrations appname
python3 manage.py migrate
I just deleted my migrations folder, dropped the whole database, then i made migration for the app
python3 manage.py makemigration
python3 manage.py migrate
and it came back.
Rename the deleted table name to some_new_name in the models.py and run:
python3 manage.py makemigration
python3 manage.py migrate
again rename the some_new_name table to the original name and run
python3 manage.py makemigration
python3 manage.py migrate
finally, go to the dbshell and drop the table some_new_name
I create table manualy and it helps.
For Django 1.10.4
I deleted the db.sqlite3 file from the project folder and then ran the following commands:
python manage.py makemigrations app_name
python manage.py migrate
Django 1.11.2 using MariaDB, accidental drop of database table.
To recreate table, try the following:
1/ Delete all except for init.py in your app/migrations directory
2/ select * from django_migrations; delete from django_migrations where app = 'yourapp';
3/ Check your model is good and run: python manage.py makemigrations
4/ python manage.py migrate
Works for me!
if you have created your classes and performed the migration operation, and then you want to add items to your classes, empty the migration folder with this command beforehand.
In Django 3, I proceeded according to the following steps and it worked 100%
python manage.py makemigrations appname --empty
python manage.py makemigrations appname
python manage.py migrate
Actually, the above methods did not work for me, so I just perform the below workaround as I did not want to manually write the whole query to create the table.
So I changed the database in the settings file and re-ran the migrations command after deleting the migrations folder, then just performed the python migrate command it created new tables in another database then from there just opened the table in query view, copied the script, and inserted the table in my main database.
Another Simple way to do this is
Go to your migrations folder.
Search for the file which contains the code to create the Students table in DB.
Remove the code snippet from the file and save it.
Then run py manage.py makemigrations and py manage.py migrate again
This worked for me :)
In this case, you need to trick django!
Do one thing...
copy the "students" model in models.py with other name like
"studentscopy".
Now run --> python manage.py makemigration
It will create a new migration in migration package of your app. Open
that last migration and rename "studentscopy" back to "students"
in that file.
Now run --> python manage.py migrate
It will create the table again with "students" name and at last delete that "studentscopy" model from your models.py file.
Below steps solved the problem for me
Delete all migrations files
python manage.py makemigrations (Create your initial migration file 0001_inital.py)
python manage.py migrate --fake <app_name> zero
( Tell Django to mark the migrations as having been applied or unapplied, but without actually running the SQL to change your database schema. This brings the migrations to zeroth state. Otherwise Django will think there's no change to apply )
python manage.py migrate

Django 1.8 OperationalError: no such column:

I'm using django 1.8 and I'm having problems adding to my models.py. Currently it's:
from django.db import models
# Create your models here.
class Company(models.Model):
role = models.CharField(max_length=32, blank=True)
name = models.CharField(max_length=70, blank=True)
and it works perfectly fine but whenever I try to add to this and then run the server I get
OperationalError: no such column [added element]
For example I added founder = models.CharField(max_length=200, blank=True) and I ran the program and I got
django.db.utils.OperationalError: no such column: companies_company.founder
Run in your console this commands:
manage.py makemigrations app_name
manage.py migrate app_name
Every time when you change model in your app you should migrate changes to your db using makemigration and migrate commands. When you adding a new column to your db table you must add value of this column to all existing rows. You can do it by seting default value in your new field in your model.
Or set values when run migrate command ( django automatically propose this)
You can read about this in docs
This type of problem occurs when there are some operations on the model field in some other files such as forms.py or views.py other than models.py when you run makemigrations. If you read the traceback carefully you can figure it out from which file the problem is originating.
For example, if the traceback tells you some complaints in forms.py which may happen to use some of the model fields, just comment out the code that is working on the model fields and rerun makemigrations again. Hopefully it resolves the issue.
If it does, you can then remove the comments that you added before.
You have to re sync the database using python manage.py makemigrations
Maybe not optimal, but it worked for me doing that (and the default thing did not work):
delete all files apart from the init.py in migrations folder
delete the db.sqlite file
doing: python manage.py runserver (without this one previous to migrate, migrations, that did not work, I think it's linked to the views / templates calling tags, so has to run first)
python manage.py migrate
python manage.py makemigrations
python manage.py migrate
python manage.py runserver
python manage.py createsuperuser
I found that clearing all data with python3 manage.py sqlflush and then python3 manage.py flush
Then makemigrations, sqlmigrate, and then migrate
It will delete all data in the SQL database including users and objects

South Migration Issues

I have looked at these questions; http://south.readthedocs.org/en/latest/tutorial/part1.html, South ignores change in field default value in Python / Django & Django-south not detecting DB changes and many more all over SO but I can't seem to fix my problem.
I have an existing model with data in it's tables, and I'm adding another model via Foreign key to it. I have run schema migrations and migrations, but nothing has proved to work. This is the code:
class UserNote(models.Model):
user = models.ForeignKey(User)
description = models.TextField(blank=True, null=True)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class UserNoteType(models.Model):
name = models.CharField(max_length=20)
I need to add UserNoteType to UserNote as Foreign key,and every attempt to add the field results in "not installed", "not defined". I've been battling with this for hours now, any help would assist greatly.
EDIT:
Error I recieve when I try create a Schema-migration:
CommandError: One or more models did not validate:
auth.usernote: 'note_type' has a relation with model , which has either not been installed or is abstract.
Have you tried to convert your existing app for South with the following command ?
python manage.py convert_to_south my_app
More informations can be find here
The UserNote model was attached to the Django UserAuth application. This resulted with South looking in the wrong application each time. I fixed this issue by creating a custom command to manually insert the new column note_type to the UserNote table.
Thank you so much for your help.
For your reference
If you are creating new project, beginning it's self you need to setup south, Try below:
python manage.py syncdb
python manage.py schemamigration --initial firstapp
python manage.py migrate forecasts --fake
python manage.py schemamigration --auto firstapp
python manage.py migrate google
After the development side you are planning to include south, then try below
./manage.py schemamigration --auto firstapp
./manage.py migrate firstapp

Categories