I am having a lot of trouble implementing Joined Table Inheritance as laid out in the SQLAlchemy documentation here!.
I am building a web app using Flask and therefore using the Flask-SQLAlchemy library. I have been able to successfully replicate the inheritance when using just SQLAlchemy. My issue arises when I start implementing it with Flask-SQLAlchemy. It seems as if Flask-SQLAlchemy makes some fundamental changes that inhibit this kind of inheritance out of the box.
To be more specific, when I query a subclass using Flask-SQLAlchemy, an InstrumentedList does indeed get returned. The problem is that it never casts to the correct subclass.
I have searched for a long time reading all the info I could possibly find but to no avail. I have tried using the with_polymorphic function from sqlalchemy but, again, to no avail.
If someone could point me in the correct direction I would greatly, greatly appreciate it. Ideally, if someone could post a minimalist, working code snippet for Joined Table Inheritance using Flask-SQLAlchemy that would be much appreciated.
Thanks in advance
Daniel
The concept of joined table inheritance is so neat from a database design perspective. Let us take this example below:
# Base class
class Employee(UserMixin, db.Model)
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(64), index=True, unique=True)
email = db.Column(db.String(120), index=True, unique=True)
password_hash = db.Column(db.String(128))
type = db.Column(db.String(64))
__mapper_args__ = {
'polymorphic_identity': 'employee',
'polymorphic_on': 'type'
}
def __repr__(self):
return f'Employee: {self.username}'
# Child1
class Manager(Employee):
id = db.Column(db.Integer, db.ForeignKey('employee.id'), primary_key=True)
department = db.Column(db.String(64))
__mapper_args__ = {
'polymorphic_identity': 'manager',
'polymorphic_load': 'inline'
}
def __init__(self, department):
super().__init__(username, email, department)
self.department = department
def __repr__(self):
return f'Manager: {self.department}'
# Child2
class Engineer(Employee):
id = db.Column(db.Integer, db.ForeignKey('employee.id'), primary_key=True)
stack = db.Column(db.String(64))
__mapper_args__ = {
'polymorphic_identity': 'engineer',
'polymorphic_load': 'inline'
}
def __init__(self, stack):
super().__init__(username, email, stack)
self.stack = stack
def __repr__(self):
return f'Engineer: {self.stack}'
There is a parent table called Employee that is polymorphically inherited by two children, Manager and Engineer. The base class defines common fields. The child models define fields that are specific to them.
Each model has an identity mapped to the discriminator column called type. This discriminator column typically stores strings such as in the case above, "employee", "manager" and "engineer".
To load inheritance hierarchies, polymorphic loading usually comes with an additional problem of which subclass attributes are to be queried upfront and which are to be loaded later. When an attribute of a particular subclass is queried up front, it can be used in the query as something to filter on, and it will be loaded when we get our objects back. On the other hand, if it is not queried up front, it gets loaded later when we first need to access it.
Thankfully, there are ways we can handle this problem. In the sample models shown above, I have specified that the children models should individually participate in the polymorphic loading by default using the mapper.polymorphic_load parameter.
If I query a particular subclass, I should be able to access all its information, which includes the data stored in the Employee table.
Updating The Models
A simple route to update a manager would look like this:
#app.route('/register/user/general-info', methods=['GET', 'POST'])
def register_user():
"""Registration URL"""
if current_user.is_authenticated:
return redirect(url_for('index'))
form = RegistrationForm()
if form.validate_on_submit():
session['username'] = form.username.data
session['email'] = form.email.data
session['identity'] = form.identity.data
session['password'] = form.password.data
if session['identity'].lower() == 'manager':
flash('Complete your registration as manager')
return redirect(url_for('register_manager'))
if session['identity'].lower() == 'engineer':
flash('Complete your registration as engineer')
return redirect(url_for('register_engineer'))
return render_template('register.html', title='General User Data', form=form)
#app.route('/register/user/manager', methods=['GET', 'POST'])
def register_manager():
if current_user.is_authenticated:
return redirect(url_for('index'))
form = ManagerForm()
if form.validate_on_submit():
manager = Manager(
username = session['username'],
email = session['email'],
department = form.department.data
)
manager.set_password(session['password'])
db.session.add(manager)
db.session.commit()
del session['username']
del session['email']
del session['identity']
del session['password']
flash(f'Registration successful. Login to continue')
return redirect(url_for('login'))
return render_template('register_manager.html', title='Manager Details', form=form)
Here, I am using two view functions, one to capture general user data, then the other for manager-specific details. In the end, I've added a manager object with all the info the Manager model would need.
Querying
In an active Python prompt, we can do the following:
>>> managers = Manager.query.all()
>>> for m in managers:
... m.username, m.type, m.department
# Output
('manager_username', 'manager', 'manager_department')
All fields about the manager, including the department which is specific to the Manager model, are accessible.
Related
Is there a way to select the db bind based on user selection? In my case, I want to change the db bind based on user selected country. If US, connect to db_us, and if UK, select db_uk.
However, as soon as the application loads, I call db.init_app() which already selects db_us and the models are set, the blueprints are configured and then the login form with country selection is loaded. The user selects UK and enters username and password, but despite having the country selected in session I am not able to reset the models with the new bind_key.
Here is some code:
# app.py
def create_app(config=None, app_name=None, blueprints=None):
db.init_app(app)
admin.add_view(UsersAdmin(db.session))
admin.init_app(app)
...
# /partners/model.py
class Partners(db.Model):
__tablename__ = 'partners'
__table_args__ = {'extend_existing': True}
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
# /user/model.py
class Users(db.Model, UserMixin):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
email = db.Column(db.String(STRING_LEN), unique=True)
partner = db.Column(db.Integer, db.ForeignKey("PARTNERS.id"))
partners = db.relationship("Partners", uselist=True, backref="user")
# Customized User model admin
class UsersAdmin(ModelView):
# set the form fields to use
form_columns = ('name', 'email', 'partners')
column_list = ('id', 'name', 'email', 'stores')
# Override query methods to add database switchers
def _run_view(self, fn, *args, **kwargs):
selectedCountry = session['selectedcountry']
with db.context(bind=selectedCountry):
return super()._run_view(fn, *args, **kwargs)
It still queries against US db.
SQLAlchemy with multiple binds - Dynamically choose bind to query helped me, but I still have an issue with using db bind with a flask-admin ModelView. I am now able to use the correct db to login but in Flask Admin page User Create form, the form field Partners drop down list always loads US relevant options. How can I set bind in ModelView?
# /frontend/views.py
def login():
...
user, authenticated = Users.authenticate(form.login.data, form.password.data, country=form.country.data)
if user and authenticated:
...
session['selectedcountry'] = form.country.data
...
# /user/model.py
#classmethod
def authenticate(cls, login, password, country):
with db.context(bind=country):
user = cls.query.filter(Users.email.ilike(login), Users.country.ilike(country)).first()
...
After applying the suggestion from Aaron regarding overriding UserAdmin._run_view(), I still find view does not query against the db in context. The select list for partners is still loaded from the default db.
Additional issues that I saw was that, even though authentication works fine for each db, the logged in user and email displayed post login is from the other db. For example, I logged in with UK Admin User of db_uk but the name and email is from db_us. See images below. The first image is from the application and second one is from db_uk and db_us respectively. The number of users listed is correct.
Specify the context in lms_signallingSession.execute() for it to work globally, and then you can remove with db.context(bind=selectedCountry): everywhere except Users.authenticate().
class lms_signallingSession(SignallingSession):
...
def execute(self, *args, **kwargs):
from flask import session
selectedCountry = session['selectedcountry']
with self.db.context(bind=selectedCountry):
return super().execute(*args, **kwargs)
It's called whenever you access the DB.
Otherwise, you need to specify the context explicitly in all places that are relevant for it to not be "unreliable". In particular, you are missing it in:
UsersAdmin._run_view()
def load_user(id):
I have two user models for flask_login.:
class Admin(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(30))
password_hash = db.Column(db.String(200))
class Merchant(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(30))
password_hash = db.Column(db.String(200))
Now I want to load user in session:
#login_manager.user_loader
def load_user(user_id):
pass
I want to know how to load user from two models.
Here's a solution I've been using so far; I don't know it's flaws, but it's the answer you're looking for.
Assuming you have multiple account types, the key is to use the session to store that account type upon login, and use it like this:
#login_manager.user_loader
def load_user(user_id):
if session['account_type'] == 'Admin':
return Admin.query.get(int(user_id))
elif session['account_type'] == 'Merchant':
return Merchant.query.get(int(user_id))
else:
return None
Providing routes and html is not necessary; you could implement them as you wish, either by:
Creating different routes for different user types.
Adding a select field in the login form with one route for all user types.
This thread provides further information about sessions and how to secure them; you should check it out.
I understand that your choice to keep your classes separated, but consider merging the common attributes together in one parent class leaving only the id to prevent foreign keys problems, like this:
class Person(db.Model):
__abstract__ = True
name = db.Column(db.String(30))
password_hash = db.Column(db.String(200))
class Admin(Person, UserMixin):
id = db.Column(db.Integer, primary_key=True)
class Merchant(Person, UserMixin):
id = db.Column(db.Integer, primary_key=True)
As the parent table is abstract it won't be created, but its children will.
#You can create a permission
admin_permission = Permission(RoleNeed('admin'))
#protect a view with a principal for that need
#app.route('/admin')
#admin_permission.required()
def do_admin_index():
return Response('Only if you are an admin)
Say I have a set of users, a set of games, and I track whether a user has finished a game in a separate table (name: 'game_progress'). I want it to be that whenever a user is created, the 'game_progress' table is auto-populated with her ID and a 'No' against all the available games. (I know that I can wait until she starts a game to create the record, but, I need this for an altogether different purpose.) How would I go about doing this?
I tried using the after_insert() event. But, then I can't retrieve the ID of the User to insert into 'game_progress'. I don't want to use after_flush (even if I can figure out how to do it) because it may be a bit of an overkill, as the user creation operation doesn't happen that often.
class Game(db.Model):
__tablename__ = 'games'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(30))
class User(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(30))
class GameProgress(db.Model):
__tablename__ = 'game_progress'
user_id = db.Column(db.Integer, db.ForeignKey('users.id'), primary_key=True)
game_id = db.Column(db.Integer, db.ForeignKey('games.id'), primary_key=True)
game_finished = db.Column(db.Boolean)
#event.listens_for(User, "after_insert")
def after_insert(mapper, connection, target):
progress_table = GameProgress.__table__
user_id = target.id
connection.execute(
progress_table.insert().\
values(user_id=user_id, game_id=1, game_finished=0)
)
db.create_all()
game = Game(name='Solitaire')
db.session.add(game)
db.session.commit()
user = User(name='Alice')
db.session.add(user)
db.session.commit()
You don't need to do anything fancy with triggers or event listeners at all, you can just set up the relations and then make related objects in the constructor for User. As long as you have defined relationships (which you're not doing at present, you'd only added the foreign keys), then you don't need User to have an id to set up the associated objects. Your constructor can just do something like this:
class User(db.Model):
def __init__(self, all_games, **kwargs):
for k,v in kwargs.items():
setattr(self, k, v)
for game in all_games:
self.game_progresses.append( GameProgress(game=game, \
user=self, game_finished=False) )
When you commit the user, you'll also commit a list of GameProgress objects, one for each game. But the above depends on you setting up relationships on all your objects. You need to add the below to GameProgress class
game = relationship("Game", backref="game_progresses")
user = relationship("User", backref="game_progresses")
And pass in a list of games to user when you make your user:
all_games = dbs.query(Game).all()
new_user = User(all_games=all_games, name="Iain")
Once that's done you can just add GameProgress objects to the instrumented list user.game_progresses and you don't need to have committed anything before the first commit. SQLA will chase through all the relationships. Basically any time you need to muck with an id directly, ask yourself if you're using the ORM right, you rarely need to. The ORM tutorial on the SQLA docs goes through this very well. There are lots of options you can pass to relationships and backrefs to get the cascading doing what you want.
I am stumped on testing a POST to add a category to the database where I've used Flask_WTF for validation and CSRF protection. For the CRUD operations pm my website. I've used Flask, Flask_WTF and Flask-SQLAlchemy. It is my first independent project, and I find myself a little at a lost on how to test the Flask-WTForm validate_on_submit function.
Here's are the models:
class Users(db.Model):
id = db.Column(db.Integer, primary_key=True, unique=True)
name = db.Column(db.String(80), nullable=False)
email = db.Column(db.String(250), unique=True)
class Category(db.Model):
id = db.Column(db.Integer, primary_key=True, unique=True)
name = db.Column(db.String(250), nullable=False, unique=True)
users_id = db.Column(db.Integer, db.ForeignKey('users.id'))
Here's the form:
class CategoryForm(Form):
name = StringField(
'Name', [validators.Length(min=4, max=250, message="name problem")])
And here's the controller:
#category.route('/category/add', methods=['GET', 'POST'])
#login_required
def addCategory():
""" Add a new category.
Returns: Redirect Home.
"""
# Initiate the form.
form = CategoryForm()
# On POST of a valid form, add the new category.
if form.validate_on_submit():
category = Category(
form.name.data, login_session['users_id'])
db.session.add(category)
db.session.commit()
flash('New Category %s Successfully Created' % category.name)
return redirect(url_for('category.showHome'))
else:
# Render the form to add the category.
return render_template('newCategory.html', form=form)
How do I write a test for the if statement with the validate_on_submit function?
You should have different configurations for your app, depending if you are local / in production / executing unit tests. One configuration you can set is
WTF_CSRF_ENABLED = False
See flask-wtforms documentation.
Using py.test and a conftest.py recommended by Delightful testing with pytest and SQLAlchemy, here's a test that confirms the added category.
def test_add_category_post(app, session):
"""Does add category post a new category?"""
TESTEMAIL = "test#test.org"
TESTUSER = "Joe Test"
user = Users.query.filter(Users.email==TESTEMAIL).first()
category = Category(name="Added Category", users_id=user.id)
form = CategoryForm(formdata=None, obj=category)
with app.test_client() as c:
with c.session_transaction() as sess:
sess['email'] = TESTEMAIL
sess['username'] = TESTUSER
sess['users_id'] = user.id
response = c.post(
'/category/add', data=form.data, follow_redirects=True)
assert response.status_code == 200
added_category = Category.query.filter(
Category.name=="Added Category").first()
assert added_category
session.delete(added_category)
session.commit()
Note that the new category is assigned to a variable and then used to create a form. The form's data is used in the post.
Working on the comments of #mas I got to this solution which worked for me:
topic_name = "test_topic"
response = fixt_client_logged_in.post('/create', data={"value":topic_name}, follow_redirects=True)
I am using this form class:
class SimpleSubmitForm(FlaskForm):
value = StringField(validators=[DataRequired()])
submit = SubmitField()
In this html file:
{{form.hidden_tag()}}
{{form.value.label("Topic", class="form-label")}}
{{form.value(value=topic_name, class="form-control")}}
<br/>
{{form.submit(value="submit", class="btn btn-primary")}}
Note that I am using the hidden_tag for the CSRF security, however when testing I have this extra line that de-activates it:
app.config['WTF_CSRF_ENABLED']=False
I have no idea how it actually works under the hood but my hypothesis is this: The wtform FlaskForm object looks at the "data" attribute of the request, which should be a dict. It then looks for keys in that dict that have the same name as its attributes. If it finds a key with the same name then it assigns that value to its attribute.
I am attempting to declare a User model in my flask application in order to implement login with the Flask-Login extension. From the flask documentation regarding sql alchemy there is this example which I have used for another model called employees. Here is the code:
class Employee(db.Model):
__tablename__ = "employees"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(200))
title = db.Column(db.String(200))
email = db.Column(db.String(200))
department = db.Column(db.String(200))
def __init__(self, name, title, email, department):
self.name = name
self.title = title
self.email = email
self.department = department
def __repr__(self):
return '<Employee %r>' % self.name
This is taken from this page as an exmample Flask SQL-Alchemy Docs
I am confused because I am also using alembic to run migrations so by using alembic revision -m "create user table I have already created a 'users' table. I created my first model's table (employees) by using the recommendation of the Flask Sql-Alchemy guide as such:
from yourapplication.database import init_db
init_db()
Here is the confusion. I now need to establish a User model for authentication. How do I do this? Here is my code so far:
class User(flask_login.UserMixin):
def __init__(self, username, password):
self.id = username
self.password = password
Notice that I have the User class inherit from flask_login.Usermixin. I need this in order for Flask Login to work so now its not instantiated like the Employee model from db.Model. But the thing is is that I created an alembic migration to establish my User model. Why am I not able to query the database. What piece of the puzzle am I missing?
All UserMixin does is provide a common interface that any user model needs to implement to work with Flask-Login. It does not set up a SQLAlchemy model, it mixes in to a model.
class User(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False, unique=True)
active = db.Column(db.Boolean, nullable=False, default=True)
#property
def is_active(self):
# override UserMixin property which always returns true
# return the value of the active column instead
return self.active
SQLAlchemy maps Python classes to database tables. To create these tables if they don't exist, call db.create_all(). If a table already exists, it will not be altered if you changed the model, you need to drop and re-create the table.
Since dropping tables means you lose all your data, it's a better idea to migrate the schema and data as you change your model. Alembic will generate scripts representing these changes. Instead of using create_all, use Alembic to generate a migration creating the table, and then run the migration.
alembic revision --autogenerate -m 'create user'
alembic upgrade head
Now that you have a user model, and have created the table with an initial migration script, you can perform queries on the table and do login related operations on the instances.
# create a user
u = User(name='davidism')
db.session.add(u)
db.session.commit()
# query all users
users = User.query.all()
# update a user
u = User.query.filter_by(name='davidism').one()
u.active = False
db.session.commit()
# check if a user is active like Flask-Login
print(u.is_active)