I'm following this tutorial.. and the initial auto generate is perfect.. it basically creates the migration file with the upgrade and downgrade methods just fine.
so let's say this is the migration file's revision number: 3e96cf22770b.. all of my upgrade statements look like this:
def upgrade():
### commands auto generated by Alembic - please adjust! ###
op.create_table('hashtag',
sa.Column('id', sa.VARCHAR(), autoincrement=False, nullable=False),
sa.Column('text', sa.VARCHAR(), autoincrement=False, nullable=True),
sa.PrimaryKeyConstraint('id', name=u'hashtag_pkey')
)
and my downgrade statement looks like this:
def downgrade():
### commands auto generated by Alembic - please adjust! ###
op.drop_table('user_access_token')
Now I made a simple modification to my models.py file, this is what it looks like on git:
- verificationCode = Column(String())
+ isVerified = Column(Boolean())
The thing is, I have no idea how to run an autogenerate statement that actually just give me a delta migration file.. ie i just want a migration file that replaces one column by another..
i tried setting the current revision to 3e96cf22770b then running
python migrate.py db revision --autogenerate
but then it keeps on creating duplicates of the initial migration file (ie migrating the entire database schema) rather than just the delta.. ideas?
Alembic automatically generates a migration script (using the --autogenerate flag) by observing the current DB schema (it actually connects the DB and fetches the schema) and the new model (in your python code). So when you want to create a new migration script, make sure your database is at the previous schema (3e96cf22770b in your case).
Not sure how you tried to set the current schema, but you can check for the schema at alembic_version table in your DB.
You should be able to run:
python migrate.py db migrate
and that should create a new migration file for you. Once you get that you can run:
python migrate.py db upgrade
and that will upgrade your database.
Before you upgrade your db, look at the migration file and see if it's doing what you're wanting to do.
Related
I want to add alembic to an existing ,sqlalchemy using, project, with a working production db. I fail to find what's the standard way to do a "zero" migration == the migration setting up the db as it is now (For new developers setting up their environment)
Currently I've added import the declarative base class and all the models using it to the env.py , but first time alembic -c alembic.dev.ini revision --autogenerate does create the existing tables.
And I need to "fake" the migration on existing installations - using code. For django ORM I know how to make this work, but I fail to find what's the right way to do this with sqlalchemy/alembic
alembic revision --autogenerate inspects the state of the connected database and the state of the target metadata and then creates a migration that brings the database in line with metadata.
If you are introducing alembic/sqlalchemy to an existing database, and you want a migration file that given an empty, fresh database would reproduce the current state- follow these steps.
Ensure that your metadata is truly in line with your current database(i.e. ensure that running alembic revision --autogenerate creates a migration with zero operations).
Create a new temp_db that is empty and point your sqlalchemy.url in alembic.ini to this new temp_db.
Run alembic revision --autogenerate. This will create your desired bulk migration that brings a fresh db in line with the current one.
Remove temp_db and re-point sqlalchemy.url to your existing database.
Run alembic stamp head. This tells sqlalchemy that the current migration represents the state of the database- so next time you run alembic upgrade head it will begin from this migration.
New installation: applying the migration
Simply run alembic upgrade head against an empty database. This will apply all the migrations (in your case, the initial migration as it's the only one) to the database.
If you want to do this from code rather than from shell, you can do it the following way:
from alembic.config import Config
from alembic import command
alembic_cfg = Config("/path/to/yourapp/alembic.ini")
command.upgrade(alembic_cfg, "head")
Existing installation: faking the migration
SQL way
One way would be running this SQL against the database:
CREATE TABLE IF NOT EXISTS alembic_version (
version_num VARCHAR(32) NOT NULL
);
INSERT INTO alembic_version (version_num) VALUES ('your initial migration version');
The first statement creates the table that alembic uses to track your database/migration state. The second statement basically tells alembic that your database state corresponds to the version of your initial migration, or, in other words, fakes the migration.
Alembic way
Alembic has a stamp command, which basically does the same thing. It can be called from shell as alembic stamp head, or from code (taken from the cookbook):
from alembic.config import Config
from alembic import command
alembic_cfg = Config("/path/to/yourapp/alembic.ini")
command.stamp(alembic_cfg, "head")
I have 6 models in two files(benef and petition) in addition to the user information. There are some inter dependencies. When I changed the model, there was some error. so I thought of starting afresh and want to drop all tables.
I ran sqlflush and Sqlclear with following result.
Sqlflush result is
BEGIN;
DELETE FROM "django_admin_log";
DELETE FROM "auth_permission";
DELETE FROM "auth_group";
DELETE FROM "auth_group_permissions";
DELETE FROM "django_session";
DELETE FROM "auth_user_groups";
DELETE FROM "auth_user_user_permissions";
DELETE FROM "benef_beneficiary_information";
DELETE FROM "petition_employer";
DELETE FROM "petition_job";
DELETE FROM "nashvegas_migration";
DELETE FROM "benef_beneficiary";
DELETE FROM "auth_user";
DELETE FROM "benef_beneficiaryname";
DELETE FROM "petition_petition";
DELETE FROM "django_content_type";
COMMIT;
Finished "C:\pyProjs\immiFile\manage.py sqlflush" execution.
Sqlclear benef result is
BEGIN;
DROP TABLE "benef_beneficiary_information";
DROP TABLE "benef_beneficiary";
DROP TABLE "benef_beneficiaryname";
COMMIT;
Finished "C:\pyProjs\immiFile\manage.py sqlclear benef" execution.
sqlclear petition result is
BEGIN;
DROP TABLE "petition_petition";
DROP TABLE "petition_job";
DROP TABLE "petition_employer";
COMMIT;
Finished "C:\pyProjs\immiFile\manage.py sqlclear petition" execution.
But then when I run the project and go to admin, I still see the old tables and when I click on them, the error related to field comes, which was originally caused by model change. The data is not relevant
OperationalError at /admin/benef/beneficiary/
no such column: benef_beneficiary.last_edited_by_id
I want to start afresh. What is the solution?
I am using Django 1.8 and Python 2.7
From the documentation:
sqlflush: Prints the SQL statements that would be executed for the flush command.
It just prints the statements that would be executed. It doesn't touch the database (same goes for sqlclear). You need to use flush instead.
Also note this from the documentation on flush:
Removes all data from the database and re-executes any
post-synchronization handlers. The table of which migrations have been
applied is not cleared.
If you would rather start from an empty database and re-run all
migrations, you should drop and recreate the database and then run
migrate instead.
I want to start afresh. What is the solution?
The sure fire way, which will delete everything (including your data):
Shut down the application (if you are using runserver, make sure its stopped)
Delete the database from the database server. If you are using sqlite, simply delete the db file.
Delete all migration directories created under your app(s).
Type ./manage.py makemigrations
Type ./manage.py migrate
I recently reinstalled my OS and lost some old data from a local postgre db. But I backed up all the migration files using Alembic. And now I just want to restore the database schema from the migration files, not data. Is that possible?
It is doable. All migration files have a revision number. Your first migration file has something like:
revision = '22a39a2bf2ed'
down_revision = None
and your second revision file has something like:
revision = '507003430224'
down_revision = '22a39a2bf2ed'
As you can see all revision files are linked.
The only thing that you need to do is make your first migration file manually, then run
alembic upgrade head
Then you need to replace the content of this file with you previous first migration file. Then open your second migration file and replace downgrade_version number with this new number.
Now you should be able to run
alembic upgrade head
again and your database should be upgraded
My goal is to add a second database to my current project, using alembic. I have the default alembic folder, and using 'alembic init alembic_second' I created a second folder structure. I modified the env.py in the second folder and the root alembic.ini. When I run
alembic -n 'alembic_second' revision -m "create second" --head=base --version-path=alembic_second/versions --autogenerate
the output is:
postgresql:// (all the correct second database connection stuff)
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
ERROR [alembic.util] Target database is not up to date.
FAILED: Target database is not up to date.
postgresql://(all the correct second database info)
The solution here doesn't work for me because my new versions folder is empty, my problem is I can't run my FIRST migration on this new database. As you can see in my terminal input, I am specifying the new versions folder.
Also, I put a print statement in my second env.py, and I am successfully seeing that, so it is hitting the correct env.py.
Any ideas on how to get past this error and create my first revision?
Thank you!
It turns out that after I sorted through differences between the two database structures, the core problem was the same as the one I linked to, I just needed to add all of the proper flags to make it work with two databases.
Running:
alembic -n 'alembic_second' stamp head --version-path=alembic_second/versions
solved my problems.
Everything I found about this via searching was either wrong or incomplete in some way. So, how do I:
delete everything in my postgresql database
delete all my alembic revisions
make it so that my database is 100% like new
This works for me:
1) Access your session, in the same way you did session.create_all, do session.drop_all.
2) Delete the migration files generated by alembic.
3) Run session.create_all and initial migration generation again.
If you want to do this without dropping your migration files the steps are:
Drop all the tables in the db that are to be recreated.
Truncate the alembic_version table (so it will start from the beginning) - this is where the most recent version is kept.
The you can run:
alembic upgrade head
and everything will be recreated. I ran into this problem when my migrations got in a weird state during development and I wanted to reset alembic. This worked.
No idea how to mess with alembic, but for the database, you can just log into the SQL console and use DROP DATABASE foo.
Or were you wanting to clear out all the data, but leave the tables there? If so, Truncating all tables in a postgres database has some good answers.