I have a Flask project with MySQL database and uses SQLAlchemy as ORM and Flask-Migrate*for migrations.
I wrote my models and when I run the migrations, migration file is empty because existing tables are out of Flask-Migrate's control so I actually have to delete those tables to let migration tool to detect and create them again. But the problem is that I do not want to delete and create my tables.
So is there a way that I can sync my model with my existing table ?
EDIT :
I just found out that in env.py file, it's possible to specify tables that exists
and it will not create those tables :
metadata.reflect(engine, only=["table1", "table2"])
Thanks for the answer.
Automatic migrations are by definition generated as a delta between your models and your database. If you already have a database that was created before you started using Flask-Migrate/Alembic then you can begin tracking migrations from that point on.
If you want to generate an initial migration that takes you to your current version the easiest way is to delete all the tables, as you suggested. To avoid losing your data I can suggest two ideas:
backup your database before deleting your tables, then restore after the migration was generated.
temporarily point your application at an empty database (a different one). Once the migration is generated point it back to your database.
I hope this helps.
Related
I am new to Python ORM world.
I wanted to generate model (code) from the existing database in Alembic or SQLAlchemy.
I could find any docs about that. Just simply having already created database, I would like Alembic to generate me classes for that database for each table.
Is there any way to achieve this?
Alembic is used for migration. You can read more about it here Alembic Project Description
SQLAlchemy is an ORM that translates Python classes to tables on relational databases not vice versa.
This online tool create-class-from-database-table may help you to get classes from database tables
I'm a few weeks into Python/Django and encountering an annoying problem. I have some existing databases set up in settings.py, everything looks good, I've even accessed the databases using connections[].cursor()
But the databases (and data) are not making their way into models that I want to use, despite doing the makemigrations and migrate commands. I was able to use py manage.py inspectdb --database-dbname and copied that class information manually into my models.py, but that didn't work either (typing py manage.py inspectdb on its own does not pull up these databases, I was only able to view by that --database extension). So I'm stumped, as it seems I'm doing all the right steps but not able to use these existing databases in Django.
Any other hints and steps I can take are welcome!
(Almost) all the tutorials, examples, and third-party app you'll find on the internet, and most of the Django documentation assume you use one database for your app. That's because it's fairly tricky and unusual to use multiple databases in one app.
But it's not impossible to use multiple databases and the documentation contains instructions on how to do this and what changes you'll need to make to make it work.
IMO, these are the pre-conditions to use multiple databases in one project:
The databases contain explicitly unrelated information, i.e. you won't have SQL relationships between tables in different databases. One database may contain a table with a column that maps to a column in a table in another database, but they aren't explicit (no ForeignKey or ManyToManyField in your models).
You don't need to mix databases in one query: This basically derives from the previous condition. It just means that if you need to get objects from one database that depend on the rows coming from another database, you establish the relationship in python. E.g. fetching as list of names from one database and using that list to filter a queryset on the other database.
For example, if you have an existing database that contains Strava routes (which are regularly updated via some external mechanism) and your app is a broader app that helps users getting to know their neighbourhood where they can recommend locations and things to do, being able to offer a list of routes with a starting point nearby might be something you'd want to show.
Now that you know this, the way to go is described in the doc linked above:
Create a database router so that queries for certain models are automatically routed to the correct database. E.g. Route.objects.filter(start_city=city) would automatically fetch routes from your Strava routes database.
If you need to save information about a route in your app, save it in a model in the default database and use a unique identifier of the route that will map to the strava database. Use separate queries (no relationships) to fetch information about a specific route.
That being said, if the Strava database is not regularly updated via 3rd channels and its purpose is just to pre-populate your default database, then export the data from the Strava database as json and import it into your django db using manage.py loaddata or a migration file, the latter being more flexible as to the structure of the json file.
In my Django app, I want to add a couple of fields to my existing models and possibly create a new class. I just want to test the new feature and approve if it works.
I can revert the code using git easily. But if I make a makemigrations+migrate then my MySQL database will change and reversing the changes looks like manual deletion of tables and reverting to an old state using a command like django-admin migrate [app_label] [migration_name] (In some cases it looks really cumbersome, example).
I'm wondering if there is any safe practice to try manipulating the database and revert it back to it's initial state safely.
Probable solution #1:
You can utilize the test database that gets created when using django.test.TestCase:
Tests that require a database (namely, model tests) will not use your
“real” (production) database. Separate, blank databases are created
for the tests.
Create some unit tests for your project and make your migrations (without migrating to your production DB, just keep the migrations). Then:
If the database does not exist, it will first be created. Any
migrations will also be applied in order to keep it up to date.
Usually, the database gets destroyed at the end of your tests, but you can keep it between runs:
You can prevent the test databases from being destroyed by using them
test --keepdb option. This will preserve the test database between
runs.
With this trick you can test every migration you make in a fake DB and when you do finalize your model and you have all the migrations history complete, you can migrate on your production DB.
Probable solution #2:
You can make a copy of your database as #albar suggests and have it as a back up while you are working on your new migrations.
Break stuff as much as you want and when you are set and done, replace the "battered" DB with your back up and apply your migration history to it.
I have a django app which consists of 17 models. Now I have realized that these models should be in 3 different apps(not in the original app). So now I would like to migrate these models out of the original app to these 3 different apps. How do I do that?
There exists foreign key, generic foreign key and ManyToMany relationships among the models. I also have data in the database(MySql), so I would like the data to be preserved during migration.
I have installed south for migrations, but don't know how to use it for solving this issue. I have gone through this similar question but could not find an answer that would solve my problem. Would be thankful for any help !
In my opinion, you have two ways of completing this task as stated below:
Move the models and add Meta.db_table to refer the existing sql table as needed as #kroolik suggested
Perform a three steps migration
The former is easier while the later could be better as tables would be named as you expect.
First of all, you mention you already has south installed. The first step would be to create the initial migration for the existing app. Take a look to the south tutorial. Then you must apply that migration, but as you already has the tables in db it would fail unless you include --fake flag.
After that you need to create the three apps you mention, and their models. Also create and apply (this time without fake flag) the initial migration for them.
Next step is write a datamigration. You must write it manually, although you can create the skeleton with datamigration. You must write "by hand" the migration.
Now you are almost done, the only remaining thing is remove the original tables. You can just remove those models, and create an "auto" schemamigration.
Don't forget to apply the migrations with migrate command. Also as #Bibhas mention a copy of database and/or a dump of it is a pretty good idea.
I have some models I'm working with in a new Django installation. Is it possible to change the fields without losing app data?
I tried changing the field and running python manage.py syncdb. There was no output from this command.
Renavigating to admin pages for editing the changed models caused TemplateSyntaxErrors as Django sought to display fields that didn't exist in the db.
I am using SQLite.
I am able to delete the db file, then re-run python manage.py syncdb, but that is kind of a pain. Is there a better way to do it?
Django does not ever alter an existing database column. Syncdb will create tables, but it does not do 'migrations' as found in Rails, for instance. If you need something like that, check out Django South.
See the docs for syndb:
Syncdb will not alter existing tables
syncdb will only create tables for models which have not yet been installed. It will never issue ALTER TABLE statements to match changes made to a model class after installation. Changes to model classes and database schemas often involve some form of ambiguity and, in those cases, Django would have to guess at the correct changes to make. There is a risk that critical data would be lost in the process.
If you have made changes to a model and wish to alter the database tables to match, use the sql command to display the new SQL structure and compare that to your existing table schema to work out the changes.
You have to change the column names in your DB manually through whatever administration tools sqlite provides. I've done this with MySQL, for instance, and since MySQL lets you change column names without affecting your data, it's no problem.
Of course there is.
Check out South
You'll have to manually update the database schema/layout, if you're only talking about adding/removing columns.
If you're attempting to rename a column, you'll have to find another way.
You can use the python manage.py sql [app name] (http://docs.djangoproject.com/en/dev/ref/django-admin/#sql-appname-appname) command to see what the new SQL should look like, to see what columns, of what type/specification Django would have you add, and then manually run corresponding ALTER TABLE commands.
There are some apps/projects that enable easier model/DB management, but Django doesn't support this out of the box, on purpose/by design.