My flask application now has 20+ migrations built with flask-migrate and they all have hashed file names like: 389d9662fec7_.py
I want to double check the settings on the latest migration that I ran, but don't want to open every file to look for the correct one. I could create a new dummy migration and look at what it references as the down_revision but that seems clunky.
I'm using flask-script, flask-migrate, and flask-sqlalchemy
My question is: How can I quickly find the latest migration that I created?
./manage.py db history -r current: will show the migrations in the order they will be applied. -r current: shows only the migrations since the currently applied one.
./manage.py db heads will show the most recent migration for each branch (typically there's only one branch). ./manage.py db upgrade would apply all migrations to get to the head.
Use the -v flag to get verbose output, including the full path to the migration.
You can also check in your database and the current version should be displayed in a table called alembic_version.
Related
I am trying to change some field names of Django models, but for some reasons Django does not want to see my migrations. I made sure the following things are properly set up:
myApp.apps.MyAppConfig is added to INSTALLED_APPS in settings.py
A migration folder with an empty __init__.py is present in my app folder
In myApp folder, I also have an empty __init__.py
The whole procedure works just fine locally with a SQLite database and I can change field names and run python manage.py makemigrations and python manage.py migrate to migrate the database. However in production where I use Docker Compose for orchestration and Postgres as database only the first migration works fine. Afterwards when I change any model field name and try to run docker-compose exec projectname django-admin makemigrations or docker-compose exec projectname python manage.py makemigrations or add the app name behind these commands nothing helps.
Then Postgres can't handle the request, because it doesn't know the new field name:
2022-03-11 14:40:22.136 UTC [81] ERROR: column model_name.new_field_name does not exist at character 357
What am I missing here? How can I get Django to migrate?
I had the same issues when using version control to submit changes to another environment.
What probably happens is that django believes it already made the migration, because of the information that is being passed along when you push those changes. What you need to do is to correct that behavior manually deleting those migrations or, if you don't need to keep information yet, force the migration from the beginning.
You may want to read the docs for future use, and I used the answers in this questions to solve the same issue.
At the end I was able to solve this issue by adding a Docker volume for the migrations folder. I believe a lot of times people put their entire code in a persistent volume, which would've also prevented this issue. However at least everything that needs to be persistent (obviously) needs a persistent volume.
I get the following error when I run any python manage.py function:
raise NodeNotFoundError(self.error_message, self.key, origin=self.origin)
django.db.migrations.exceptions.NodeNotFoundError: Migration auth.0010_user_following dependencies reference nonexistent parent node ('accounts', '0002_auto_20180615_2021')
It happened after I tried to reset my migrations by manually deleting the migration files in the migration folders (except the init files - no other files are left in the migration folders).
I have tried dropping the database with python manage.py flush, which also doesn't run.
Any suggestions? Thanks!
SOLUTION:
After a week of google search I ended up reconstructing the referenced migration files manually using the documentation: https://docs.djangoproject.com/en/2.0/howto/writing-migrations/
After that the manage.py migrate and makemigration functions worked again. Never delete migration files without taking a backup first!
SOLUTION: After a week of google search I ended up reconstructing the referenced migration files manually using the documentation: https://docs.djangoproject.com/en/2.0/howto/writing-migrations/
After that the manage.py migrate and makemigration functions worked again. Never delete migration files without taking a backup first!
Unfortunately, manually deleting migrations doesn't reset them. The database knows which migrations have been run, and the error you're seeing is from Django trying to check whether the state of the models in your models modules matches the state of the migrations that have been run (which is to say, whether or not you need to create migrations to match) and also whether there are migrations that have been created but not run--these cases would create warnings. In trying to check these things, it tries to load migrations and can't find any of them.
If you want to reset your migrations, and just have a single migration per app to go from an empty database to your current schema in a single step, I recommend using the squash migrations command. You'll need to have the migrations files back first, though.
Alternatively, if you do want to drop and re-create the database altogether, you'll need to do that outside of the management commands, since those do the above checks when they run. Then you can have an empty database and run ./manage.py makemigrations and you'll get initial migrations that represent models as they are.
I faced the exact same error, and found the constructing the whole database a little complicated
I found a way to simply delete the whole database and reload it , (if its useful for you) using the following commands:
Step 1:
find . -path "/migrations/.py" -not -name "init.py" -delete
find . -path "/migrations/.pyc" -delete
Step 2:
Delete the database (sqlite3 in my case)
Step 3:
Run
python manage.py makemigrations
python manage.py migrate
They should now run successfully
If you have manually deleted your migrations folder, here are options to directly get them back instead of manually re-creating them. Can't imagine the pain.
Reset that deletion
To get them back you can do a git reset on the deleted folder.
git checkout -- initial/path/to/migrations
This should recover them if they were being tracked. This also assumes changes were not yet commited. If they were commited, see this answer
In case the migrations were .gitignored
If the migrations were not tracked by git, you can result to get a clean working tree where django hadn't indexed the run migrations
Clone the repository anew in a separate directory. Run the migrations python manage.py makemigrations && python manage.py migrate
Copy the deleted migrations to your migration-deleted working tree (So your can continue where you had left off)
The above option will work if the changes were unpushed to the remote. Otherwise, you'll have to checkout a commit made before the deletions.
The provided options assume you were using git
i am just uninstalled my current Django
then re-installed
its worked for me
pip3 uninstall Django
pip3 install Django
I am using flask-migrate to update the changes in my database. I ran this command. and then this command
$python manage.py db init
$python manage.py db migrate
I get the error below
alembic.util.exc.CommandError: Can't locate revision identified by 'e462fd034cc1'
I looked on stackoverflow for similar problems where it was suggested to deleted the migrations folder which i did but still same error is coming again and again.
What should i do.
Your database is out of sync with your migrations repository. For some reason the latest migration id stored in the database is not the migration id of a migration in your repository. This means that you probably deleted or modified the migration repository by hand and made it inconsistent with the current state of your database.
If this is a scratch database, maybe deleting and running the migrations again will fix the problem and give you a valid database.
Just see what is your last migration number.
Open migrations/versions folder in any file manager and sort by date.
For me, for example, it e222b725dce9_.py
Then change the value in version_num column in alembic_version table
Note that I delete underscrore in the end
Then run python manage.py db migrate and python manage.py db upgrade
All must pass successfully
I am using Django to build a website with MySQL. Now as I am learning so I need to change the Model very often so I want that all tables get cleared and new table get created.
But syncdb doesn't touch existing tables. Is there any better way to handle this problem?
If you don't care about data:
Best way would be to drop the database and run syncdb again. Or you can run:
For Django >= 1.5
python manage.py flush
For Django < 1.5
python manage.py reset appname
(you can add --no-input to the end of the command for it to skip the interactive prompt.)
If you do care about data:
From the docs:
syncdb will only create tables for
models which have not yet been
installed. It will never issue ALTER
TABLE statements to match changes made
to a model class after installation.
Changes to model classes and database
schemas often involve some form of
ambiguity and, in those cases, Django
would have to guess at the correct
changes to make. There is a risk that
critical data would be lost in the
process.
If you have made changes to a model
and wish to alter the database tables
to match, use the sql command to
display the new SQL structure and
compare that to your existing table
schema to work out the changes.
https://docs.djangoproject.com/en/dev/ref/django-admin/
Reference: FAQ - https://docs.djangoproject.com/en/dev/faq/models/#if-i-make-changes-to-a-model-how-do-i-update-the-database
People also recommend South ( http://south.aeracode.org/docs/about.html#key-features ), but I haven't tried it.
Using Django Extensions, running:
./manage.py reset_db
Will clear the database tables, then running:
./manage.py syncdb
Will recreate them (south may ask you to migrate things).
I think Django docs explicitly mention that if the intent is to start from an empty DB again (which seems to be OP's intent), then just drop and re-create the database and re-run migrate (instead of using flush):
If you would rather start from an empty database and re-run all
migrations, you should drop and recreate the database and then run
migrate instead.
So for OP's case, we just need to:
Drop the database from MySQL
Recreate the database
Run python manage.py migrate
Quickest (drops and creates all tables including data):
./manage.py reset appname | ./manage.py dbshell
Caution:
Might not work on Windows correctly.
Might keep some old tables in the db
You can use the Django-Truncate library to delete all data of a table without destroying the table structure.
Example:
First, install django-turncate using your terminal/command line:
pip install django-truncate
Add "django_truncate" to your INSTALLED_APPS in the settings.py file:
INSTALLED_APPS = [
...
'django_truncate',
]
Use this command in your terminal to delete all data of the table from the app.
python manage.py truncate --apps app_name --models table_name
Is there SQLAlchemy automigration tool like South for Django?
I looked to sqlalchemy-migrate but it doesn't seem to generate sql update scripts automatically or upgrade downgrade DB
Looks like with sqlalchemy-migrate you need to
a) manually copy your old model to a new file
b) crate new model in application and copy it to a new file
c) write manually create/drop/alter tables in python sqlalchemy extended dialect
d) generate sql alter script
e) run command to execute alter sql script
As for me it doesn't solve the problem and only adds overhead, as I can simply do d) manually and it will be much faster then do a), b), c) manually just to d) that you can do in one step.
Is there any auto migration libraries for SQLAlchemy like South for Django, or many RoR-like migration tools?
What I need is to change SQLAlchemy model in python app, run tool and it will compare current DB schema to new DB schema that new model should use, and create Alter scripts that I can adjust manually and execute.
Is there any solution like this in Python?
You can perform automatic migrations with Alembic. I use it in two large-scale projects I am currently working on. The automatic migration generator can recognize:
Table additions and removals
Column additions and removals
Change of nullable status on columns
Basic changes in indexes, explicitly-named unique constraints, and foreign keys
(see also: https://alembic.sqlalchemy.org/en/latest/autogenerate.html)
Install alembic
pip install alembic
or (depending on the version of Python you are using):
pip3 install alembic
Configure alembic
Execute the following command in your project:
alembic init alembic
This will set up alembic for your project, in a folder called alembic.
You will then need to edit the generated alembic.ini configuration file.
In the file env.py, tell Alembic where to find SQLAlchemy's metadata object in your project.
(see also: https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file)
Generate the migration
Simply execute the following command line:
alembic revision --autogenerate -m "Message for this migration"
Or even (not recommended):
alembic revision --autogenerate
Upgrade the database
After this, I upgrade the database with this simple command from the folder containing the alembic.ini configuration file:
alembic upgrade head
(see also: http://rodic.fr/blog/automatic-migrations-sqlalchemy-alembic/)
There is Alembic which looks very promising, but the problem is (for now) that the support for SQlite databases is very limited.
No there is none at this moment. Alembic require you to write code regarding adding/deleting/altering table structure or creating/dropping table. So nothing like django south exists for sqlalchemy.
Have you looked into using sqlalchemy-migrate?
http://shane.caraveo.com/2010/09/13/database-migrations-for-sqlalchemy-part-duex/