I'm new. Bear with me.
I'm developing a Flask application using SQLAlchemy as an ORM and up until today I have been using SQLite for convenience. I'm now putting the application live on Digital Ocean and want to use MySQL instead of SQLite.
I have MySQL installed on my ubuntu VPS on Digital Ocean and it seems like it is configured properly. However, obviously I have to create the database tables, so I can save the data inside.
Question: Is there a way for me to migrate my models.py, so the database tables are created from what I have written in models.py or do I have to create all the database tables myself manually in MySQL?
You can see the application live here: http://workflowforum.dk/ and I have made a small test to see if there is a database connection here: http://workflowforum.dk/testdb
Models.py (Only user model):
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from flask.ext.script import Manager
from flask.ext.migrate import Migrate, MigrateCommand
from datetime import datetime, date
from hashlib import md5
from bcrypt import hashpw, gensalt
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:password#localhost/database'
db = SQLAlchemy(app)
migrate = Migrate(app, db)
manager = Manager(app)
manager.add_command('db', MigrateCommand)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
slug = db.Column(db.String(80))
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(80))
password = db.Column(db.String(80), unique=False)
admin = db.Column(db.Boolean(), default=False)
join_date = db.Column(db.DateTime)
last_seen = db.Column(db.DateTime)
topics = db.relationship('Topic')
posts = db.relationship('Post')
picture = db.Column(db.Boolean(), default=False)
title = db.Column(db.String(80))
company = db.Column(db.String(80))
summary = db.Column(db.String(80))
class Category(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80), unique=True)
description = db.Column(db.String(180), unique=False)
topics = db.relationship('Topic', backref="category")
class Topic(db.Model):
id = db.Column(db.Integer, primary_key=True)
slug = db.Column(db.String(255), unique=True)
title = db.Column(db.String(80), unique=False)
description = db.Column(db.Text, unique=False)
pub_date = db.Column(db.DateTime)
last_update = db.Column(db.DateTime)
user_id = db.Column(db.String(80), db.ForeignKey('user.id'))
category_id = db.Column(db.Integer, db.ForeignKey('category.id'))
views = db.Column(db.Integer, default=0)
locked = db.Column(db.Boolean(), default=False)
pinned = db.Column(db.Boolean(), default=False)
user = db.relationship('User')
posts = db.relationship('Post')
Views.py (Only database test):
#app.route('/testdb')
def testdb():
if db.session.query("1").from_statement("SELECT 1").all():
return 'It works.'
else:
return 'Something is broken.'
UPDATE after Lukas comment:
When trying to db.create_all() I get this traceback:
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1005, "Can't create table 'pwforum.topic' (errno: 150)") [SQL: u'\nCREATE TABLE topic (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\tslug VARCHAR(255), \n\ttitle VARCHAR(80), \n\tdescription TEXT, \n\tpub_date DATETIME, \n\tlast_update DATETIME, \n\tuser_id VARCHAR(80), \n\tcategory_id INTEGER, \n\tviews INTEGER, \n\tlocked BOOL, \n\tpinned BOOL, \n\tPRIMARY KEY (id), \n\tUNIQUE (slug), \n\tFOREIGN KEY(user_id) REFERENCES user (id), \n\tFOREIGN KEY(category_id) REFERENCES category (id), \n\tCHECK (locked IN (0, 1)), \n\tCHECK (pinned IN (0, 1))\n)\n\n']
The db.create_all suggestion in the comments is usually what you do when you don't use a migration framework. But it looks like you are using Flask-Migrate as a database migration framework here. The problem with doing create_all is that your migration scripts are skipped, so any fixes that you need to make to convert your sqlite models to MySQL will not be reflected in the migration scripts.
My recommendation is that you create an empty MySQL database, configure it in your Flask app, and then generate the MySQL tables simply by calling:
$ ./manage.py db upgrade
When you do this, Flask-Migrate will start running the migration scripts one by one and in order.
If you get failures, that it is probably because some of your migration scripts have changes that are compatible with sqlite but not with MySQL. You will need to fix all these problems in your migration scripts, until you get all of them to run cleanly, at which point you will have a complete MySQL database to use in your application.
Related
I have a problem creating relationships between my tables in flask-sqlalchemy. I have a table with project overview, and from there on out I want to dynamically create new experiment tables with a relationship to my project overview. However, when I try to define the relationship, sqlalchemy throws the following error:
sqlalchemy.exc.InvalidRequestError: When initializing mapper mapped class Projects->projects, expression 'Experiment_Overview' failed to locate a name ('Experiment_Overview'). If this is a c
lass name, consider adding this relationship() to the <class 'app.Projects'> class after both dependent classes have been defined.
This seems to be the case because the class Experiment_Overview(db.Model) does not exist yet, which is correct since it will be dynamically generated later on through user input. How can I mitigate this error?
import os
from flask import Flask, render_template, redirect, request, url_for
from flask_sqlalchemy import SQLAlchemy
from datetime import datetime
app = Flask(__name__)
Bootstrap(app)
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///DATA/DB.db"
db = SQLAlchemy(app)
def TableCreator(tablename):
class Experiment_Overview(db.Model):
__tablename__ = tablename
creation_date = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
experiment_name = db.Column(db.String(30), unique=False, nullable=False, primary_key=True)
projectname = db.Column(db.String(150), db.ForeignKey('projects.projectname'), nullable=False, unique=True)
return MyTable
class Projects(db.Model):
projectname = db.Column(db.String(150), unique=True, nullable=False, primary_key=True)
creation_date = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
experiments = db.relationship('Experiment_Overview', backref="experiments", lazy=True, uselist=False)
def __init__(self, owner, projectname, status, created_at):
self.owner=owner
self.projectname=projectname
self.status=status
self.created_at=created_at
db.create_all()
Generally speaking, you don't dynamically create tables; you usually shouldn't create or drop tables while your program is running.
Additionally, I believe it's impossible to create a true relationship that links back to an entire table. Relationships/Foreign Keys are for linking between rows in tables.
Don't worry, there are easier ways to achieve the behavior that you are looking for here.
From your question it sounds like you can have multiple Projects, and each project can have multiple Experiments within the project.
This would make the relationship between a Project and its Experiments a One-To-Many relationship.
If this is the case, you would need one Projects table (which you have already in your code), and you would also have one Experiments table.
Each row in the Projects table represents one Project.
Each row in the Experiments table represents one Experiment. The Experiments table will have a column containing a foreign key linking back to the Project the Experiment is linked to.
I've modified your code according to the One-To-Many example code given in the SQLAlchemy documentation that I linked above.
Note the addition of the back_populates option to the relationship(), this allows bi-directional knowledge of the relationship: the Experiment know what project it belongs to, and the Project know what Experiments it has.
import os
from flask import Flask, render_template, redirect, request, url_for
from flask_sqlalchemy import SQLAlchemy
from datetime import datetime
app = Flask(__name__)
Bootstrap(app)
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///DATA/DB.db"
db = SQLAlchemy(app)
class Projects(db.Model):
__tablename__ = "projects"
projectname = db.Column(db.String(150), unique=True, nullable=False, primary_key=True)
creation_date = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
experiments = db.relationship("Experiment", back_populates="project")
def __init__(self, owner, projectname, status, created_at):
self.owner=owner
self.projectname=projectname
self.status=status
self.created_at=created_at
class Experiment(db.Model):
__tablename__ = "experiments"
creation_date = db.Column(db.DateTime, nullable=False, default=datetime.utcnow)
experiment_name = db.Column(db.String(30), unique=False, nullable=False, primary_key=True)
projectname = db.Column(db.String(150), db.ForeignKey('projects.projectname'), nullable=False)
project = relationship("Project", back_populates="experiments")
db.create_all()
I am trying to configure a MySQL schema using Flask-SQLAlchemy. I have a schema called testdb and some tables. I will list one table, User.
This code, so far, creates all of the tables needed but only when testdb already exists. Is there a way to check and create testdb before I connect?
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:password#localhost/testdb'
db = SQLAlchemy(app)
class User(db.Model):
userid = db.Column(db.Integer, primary_key=True, autoincrement=True)
username = db.Column(db.String(16), unique=True, nullable=False)
password = db.Column(db.String(16), unique=False, nullable=False)
email = db.Column(db.String(80), unique=True, nullable=False)
createdDate = db.Column(db.DateTime, default=datetime.utcnow, nullable=False)
lastUpdated = db.Column(db.DateTime, default=datetime.utcnow, nullable=False)
db.create_all()
Desired command:
CREATE SCHEMA IF NOT EXISTS `testdb` ;
I solved this thanks to #hygorxaraujo
See the code below:
import sqlachemy
engine = sqlalchemy.create_engine('mysql://root:password#localhost') # connect to server
engine.execute("CREATE SCHEMA IF NOT EXISTS `testdb`;") #create db
engine.execute("USE testdb;") # select new db
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:mypassword#localhost/testdb'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False #Turn off annoying message
db = SQLAlchemy(app)
My flask application has a single database(db1) before, now I bind a new database(db2) to it, both has 10 tables.
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://root#localhost:3306/db1'
SQLALCHEMY_BINDS = {
'test': 'mysql+pymysql://root#localhost:3306/db2'
}
db = SQLAlchemy()
class table1(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
.......
class table10(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
class table11(db.Model):
__bind_key__ = 'db2'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
......
class table20(db.Model):
__bind_key__ = 'db2'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
I want to use alembic autogenerate function to auto detecting and generating migrations for db 1 and db2 separately, but db.metadata will get all tables metadata, but how to just get bind db metadata?
Thanks #davidism help! I can use include_symbol to make it.
def include_symbol(tablename, schema):
return tablename in ('table1', 'table2'.......'table10') # for db1
# return tablename not in ('table1', 'table2'.......'table10') # for db2
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
include_symbol=include_symbol
)
You can't because that feature doesn't exist yet. Currently, there is one metadata instance for all models across all binds. As long as all the models have unique names, this isn't a huge problem.
When we apply this patch and make a new release, each bind will have its own metadata. You will then be able to access it with db.get_metadata(bind='db2').
I am new to Flask.
I am working on Flask-SQLAlchemy,and i also tried Flask-Migrate.
It's handy to create production and test environments quickly.
But i think it's not as convenient as Django-Migration.
When i tried to create many-to-many data model.
I got following error sometimes:
sqlalchemy.exc.IntegrityError: (_mysql_exceptions.IntegrityError) (1215, 'Cannot add foreign key constraint') [SQL: u'
CREATE TABLE ad_accounts (
access_token_id INTEGER,
ad_account_id INTEGER,
FOREIGN KEY(access_token_id) REFERENCES fb_access_token (id),
FOREIGN KEY(ad_account_id) REFERENCES ad_account (id)
)
']
My models:
from sqlalchemy.dialects.mysql import BIGINT
ad_accounts = db.Table('ad_accounts',
db.Column('access_token_id', db.Integer, db.ForeignKey('fb_access_token.id')),
db.Column('ad_account_id', db.Integer, db.ForeignKey('ad_account.id'))
)
class AdAccount(db.Model):
__bind_key__ = 'ads'
__tablename__ = 'ad_account'
id = db.Column(db.Integer, primary_key=True)
account_id = db.Column(BIGINT(unsigned=True), default=None)
account_status = db.Column(db.Integer, default=None)
business_name = db.Column(db.String(255), default='')
owner = db.Column(db.String(255), default='')
timezone_name = db.Column(db.String(255), default='')
created_time = db.Column(db.Integer, default=0)
activities = db.relationship('Activity', backref='ad_account', lazy='dynamic')
class FbAccessToken(db.Model):
__bind_key__ = 'ads'
__tablename__ = 'fb_access_token'
id = db.Column(db.Integer, primary_key=True)
admin_id = db.Column(db.Integer, db.ForeignKey('admin_user.admin_id'))
# fb_account_id = db.Column(db.String(32), default='')
ad_accounts = db.relationship('AdAccount', secondary=ad_accounts, backref='access_token_list', lazy='dynamic')
update_time = db.Column(db.Integer, default=0)
page_id = db.Column(BIGINT(unsigned=True), default=0)
current_account_id = db.Column(BIGINT(unsigned=True), nullable=True)
When every time i was running 'python app.py db upgrade' in product envirment,i was afraid to break it down.
And someone told me that i should alter table by manual SQL.
I am confused now,i want to do it in a convenient way,i used to do it in django-migration.
Suppose I have already created database and tables in product envirment, do I still have to execute create_all or 'python app.py db upgrade'(In Flask-Migrate)?
And how to add a comment on a column in Flask-Migrate?
On an existing database, in production, you obviously don't want to recreate your database schema and lose any existing data. Look into database migrations, here are the relevant packages:
flask-migrate (it uses alembic internally)
sqlalchemy-migrate
I am not sure if I got well what you want.
I think flask Migrate is a good tool but it have some limitations as stated Here.
Basically , since alembic doesn't detect every migration you have to edit the migration script by adding foreign keys names when using them as it says in alembic documentation here
.
I face the same problem with my migration and I resolve it by editing the migration script in folder '/migration/version' by manually adding foreign keys name! with this line of code
sa.ForeignKeyConstraint(['access_token_id'], ['fb_access_token.id'],use_alter=True, name='fk_fb_access_token.id_id' ),
or if you don't like to edit migration script you can use the object
ForeignKeyConstraint
instead of
db.ForeignKey()
and give it all the parameter specially name when defining your foreign keys
But In general, it's because you need to name the foreign keys and don't use the default names
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(64), unique=True)
# 是不是应该加密下，不能明文存储？应该设置多长的空间？ 14.7.18 4：22 by lee
password = db.Column(db.String(100))
nickname = db.Column(db.String(64))
school = db.Column(db.String(20))
sex = db.Column(db.String(5))
status = db.Column(db.String(10))
grade = db.Column(db.String(18))
I have a database remains. Then I add model to models.py:
class PubSquare(db.Model):
id = db.Column(db.Integer, primary_key=True)
author_id = db.Column(db.Integer, db.ForeignKey('user.id'))
author = db.relationship('User', backref=db.backref('publish'))
subject = db.Column(db.String(100))
timestamp = db.Column(db.DateTime, default=datetime.datetime.now)
Then I run migrate script, it call bug:
NoReferencedTableError: Foreign key associated with column 'pub_square.author_id' could not find table 'user' with which to generate a foreign key to target column 'id'
Befor this time, I can run migrate script successfully for serveral times.But this time, when it refer to foreignkey relationship, it doesn't work.
to prove my models code is right, I re-create the database, it works.
So, it's the flask-migrate calls to this bug.
#knight We have migrated for many times.'user' table is in the database. But I found that if I
code like
author_id = db.Column(db.Integer)
and migrate, It's nothing wrong. And than add the code like
author_id = db.Column(db.Integer, db.ForeignKey('user.id'))
and migrate again, It passed. It's strange. I don't know why exactly.
Our migrate code is
api.update_db_from_model(SQLALCHEMY_DATABASE_URI, SQLALCHEMY_MIGRATE_REPO, db.metadata)
From what I can see the user table is not being created (could not find table 'user' with which to generate a foreign key to target column 'id'). Try migrating the User first and, therefore, making the user table, and then doing the PubSquare.
EDIT: Have you tried reading the docs? http://sqlalchemy-migrate.readthedocs.org/en/v0.7.1/changeset.html#constraint seems to help.