flask-admin custom form action and kick off background processes - python

I'm fairly new to flask-admin and would like to create a batch action for a view that will primarily do two things:
update the Status of the selected record(s) in the table that the action is "Running"
kick off a deamon or background process that runs some queries, optimizations, etc.
Edit: For #2) I'll probably want to use Celery. If that's too much weight for standalone question, I'm happy to focus on simply #1), which is: How do I simply update the records I've selected? Seems super trivial but nothing is working.
Edit #2: I found this question which seems to answer the question fairly simply, however I don't undestand what and where the transaction_service.recalculate_transaction is: Flask Admin extend "with select"-dropdown menu with custom button
Here's what I have thus far, but I keep getting a 302 status from the action. So nothing actually happens.
Any help will be greatly appreciated.
__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SECRET_KEY'] = '12345'
app.config['DEBUG'] = True
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:password#localhost/testdb'
db = SQLAlchemy(app)
import test_project.views
views.py
from test_project import app, db
from flask_socketio import SocketIO, emit
from flask import render_template, url_for, redirect
from flask_admin import Admin
from flask_admin.actions import action
from flask_admin.contrib.sqla import ModelView
from test_project.db import TargetTable
socketio = SocketIO(app)
admin = Admin(app, name='Test Tool', template_mode='bootstrap4')
class CustomView(ModelView):
# Not really using yet.
pass
class TargetTableAdmin(CustomView):
form_excluded_columns = ['status']
column_display_pk = True
create_modal = True
can_edit = False
can_delete = False
def on_model_change(self, form, model, is_created):
###############
##### Note: ### When a record is created the Status is set to "Not Run"
###############
model.status = 'Not Run' #["Running", "Completed", "Failed", "Not Run"]
################################
# HOW DO I GET THIS TO WORK?? ##
################################
#action('run', 'Run')
def run_target(self, ids):
query = TargetTable.query.filter(TargetTable.id.in_(ids))
for target in query.all():
target.status = 'Running'
db.session.commit()
# THIS DOESN'T WORK :(
admin.add_view(TargetTableAdmin(TargetTable, db.session, category='Target'))
db.py
from test_project import app
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
from flask_migrate import Migrate
db = SQLAlchemy(app)
migrate = Migrate(app, db)
class TargetTable(db.Model):
id = db.Column(db.Integer, primary_key=True)
start_date = db.Column(db.DateTime, nullable=False)
end_date = db.Column(db.DateTime, nullable=False)
status = db.Column(db.String(100), nullable=True)
def __init__(self, start_date, end_date, status):
self.start_date = start_date
self.end_date = end_date
self.status = status

OK.
So ultimately #1 was easy to answer, as I mention in my comments above. I was importing the db from the __init__.py file, where I should have been importing from the db.py file. I'm not sure I need to initialize the DB in the __init__.py file, but that's a different question. Here is the fix:
views.py
from test_project import app #### -> remove import here, db
from flask_socketio import SocketIO, emit
from flask import render_template, url_for, redirect
from flask_admin import Admin
from flask_admin.actions import action
from flask_admin.contrib.sqla import ModelView
from test_project.db import TargetTable, db #### <-- add import here
This gets harder? to answer because there are lots of ways to skin this cat. You could use a couple options / packages:
Celery / Redis
Threading
Flask-Executor
Flask-SocketIO
For now, I'm using Flask-SocketIO. And specifically I'm using a function in that package called: start_background_task. The function is non-blocking and once the function is called the page will refresh all the while the process runs in the background. It's possible this might scale, but I'm not too concerned with that right now as this is just an internal tool with minimal users at the moment. Updated file:
views.py:
# Imports above ^
socketio = SocketIO(app)
admin = Admin(app, name='Test Tool', template_mode='bootstrap4')
class CustomView(ModelView):
# Not really using yet.
pass
def testFunct(record):
for i in range(10):
print(i)
time.sleep(1)
updateRecord = TargetTable.query.filter( (TargetTable.id == record.id) ).first()
updateRecord.status = "Completed"
db.session.commit()
class TargetTableAdmin(CustomView):
form_excluded_columns = ['status']
column_display_pk = True
create_modal = True
can_edit = False
can_delete = False
def on_model_change(self, form, model, is_created):
###############
##### Note: ### When a record is created the Status is set to "Not Run"
###############
model.status = 'Not Run' #["Running", "Completed", "Failed", "Not Run"]
###############
# IT WORKS!! ##
###############
#action('run', 'Run')
def run_target(self, ids):
query = TargetTable.query.filter(TargetTable.id.in_(ids))
for t in query.all():
t.status = 'Running'
db.session.commit()
# Kick off background task:
task = socketio.start_background_task(testFunct, t)
admin.add_view(TargetTableAdmin(TargetTable, db.session, category='Target'))

Related

db.create_all() not generating db

I'm trying to test Flask with SQLAlchemy and I stumbeld accross this problem. First, I have to note that I read all of the related threads and none of them solves my problem. I have a problem that db.create_all() doesn't generate the table I defined. I have model class in file person.py:
from website import db
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String, nullable=False)
password = db.Column(db.String)
width = db.Column(db.Integer)
height = db.Column(db.Integer)
agent = db.Column(db.String)
user_data_dir = db.Column(db.String)
And in my website.py which is the file from where I launch the app:
from flask import Flask, jsonify, render_template, request
from flask_sqlalchemy import SQLAlchemy
# create the extension
db = SQLAlchemy()
def start_server(host, port, debug=False):
from person import Person
# create the app
app = Flask(__name__,
static_url_path='',
static_folder='web/static',
template_folder='web/templates')
# configure the SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///database0.db"
# initialize the app with the extension
db.init_app(app)
print('initialized db')
print('creating tables...')
with app.app_context():
db.create_all()
db.session.add(Person(username="example33"))
db.session.commit()
person = db.session.execute(db.select(Person)).scalar()
print('persons')
print(person.username)
if __name__ == '__main__':
start_server(host='0.0.0.0', port=5002, debug=True)
I think the problem might be that the Person class is not importing properly, because when I put the class inside the start_server function it executes fine and creates the table, but I don't know why this is happening. I followed all the advice and imported it before everything, and also I share the same db object between the 2 files
There is probably a better way to do this but this is the only way I could get this to work. You need to create a models.py file or w.e you wanna call it. Then all your database stuff goes in there. The db engine, ALL your models and a function to initialize it all. The reason is, you are having import issues where Person is imported but not fully and so the db doesn't have it in its metadata.
models.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Person(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String, nullable=False)
password = db.Column(db.String)
width = db.Column(db.Integer)
height = db.Column(db.Integer)
agent = db.Column(db.String)
user_data_dir = db.Column(db.String)
# All other models
def initialize_db(app: Flask):
db.init_app(app)
with app.app_context():
db.create_all()
main.py
from flask import Flask
import models
def start_server(host, port, debug=False):
app = Flask(__name__)
# configure the SQLite database, relative to the app instance folder
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///database0.db"
# initialize the app with the extension
models.initialize_db(app)
db = models.db
with app.app_context():
db.session.add(models.Person(username="example33"))
db.session.commit()
person = db.session.execute(db.select(models.Person)).scalar()
print('persons')
print(person.username)
if __name__ == '__main__':
start_server(host='0.0.0.0', port=5002, debug=True)
I am reading the documentation,
which explains that the function will
Create all tables stored in this metadata.
That leads me to believe Person is not associated with the db metadata.
You mentioned
when I put the class inside the start_server function it ... creates the table
Your from person import Person is nice enough,
but I suspect we wanted a simple import person.
In many apps the idiom would be import models.
Failing that, you may be able to point
create_all in the right direction
with this optional parameter:
tables – Optional list of Table objects, which is a subset of the total tables in the MetaData
Please let us know
what technical approach worked for you.

ImportError: cannot import name 'UserModel'

HELP, I have this appp.py file:
from flask import Flask, jsonify, request, make_response
import json
from flask_restful import Api
from flask_sqlalchemy import SQLAlchemy
import models,resources
app = Flask(__name__)
api = Api(app)
api.add_resource(resources.UserRegistration, '/registration')
api.add_resource(resources.UserLogin, '/login')
api.add_resource(resources.UserLogoutAccess, '/logout/access')
api.add_resource(resources.UserLogoutRefresh, '/logout/refresh')
api.add_resource(resources.TokenRefresh, '/token/refresh')
api.add_resource(resources.AllUsers, '/users')
api.add_resource(resources.SecretResource, '/secret')
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///app.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['SECRET_KEY'] = 'waaahawhawaahhawhaw'
db = SQLAlchemy(app)
#app.before_first_request
def create_tables():
db.create_all()
#app.route('/')
def index():
return jsonify({'message': 'Hell to the World!'})
if __name__ == '__main__':
app.run(debug = True)
and here are the models.py file:
from appp import db
class UserModel(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key = True)
username = db.Column(db.String(120), unique = True, nullable = False)
password = db.Column(db.String(120), nullable = False)
def save_to_db(self):
db.session.add(self)
db.session.commit()
#classmethod
def find_by_username(cls, username):
return cls.query.filter_by(username = username).first()
and the resources.py file
from flask_restful import Resource, reqparse
from models import UserModel
parser = reqparse.RequestParser()
parser.add_argument('username', help = 'This field cannot be blank', required = True)
parser.add_argument('password', help = 'This field cannot be blank', required = True)
class UserRegistration(Resource):
def post(self):
data = parser.parse_args()
if UserModel.find_by_username(data['username']):
return {'message': 'User {} already exists'. format(data['username'])}
new_user = UserModel(
username = data['username'],
password = data['password']
)
try:
new_user.save_to_db()
return {
'message': 'User {} was created'.format( data['username'])
}
except:
return {'message': 'Something went wrong'}, 500
Once I try the run the app I get this error message:
ImportError: cannot import name 'UserModel'
Indeed I found other question like mine and they helped me understand why I'm getting this error but none of them helped me work around it.
My guess is that python (or flask) can't load the class UserModel from model.py because of model.py (or the class UserModel) is still initializing and it needs db from appp.py which is waiting for resources.py which cannot be loaded cuz it's waiting for models.py.
How to fix this ???? btw I'm new to all this and I'm just following this tutorial
Here is the project structure
test
|---appp.py
|---models.py
|---resources.py
The 3 files are next to each other in the test folder.
Thank you
To expand, here is an example in the the context of my comment..
from test.models import User, Role, UserRoles,\
Regions, RegionAttributes, CityAttributes,\
UserAttributes, SkillTracker, RegionWar,\
Articles
You requested that I explain the differences with importing.
I'm not sure it's something one can explain in a few words but ill try my best to not overcomplicate it.
You have your project folder called test and writhing this folder is your models.py. Because models file is within the test folder (test is a module in this case) you're importing a class, within a file, within a project directory, which ends up looking like this:
from test import models
--test
|
--models.py
In plain English you could say it means, from the test folder, I want to import the file models.py
Hope this adds some clarity, this would be a well googled search, I'm sure there are lots more better explanations out there and I'm probably missing something out.
The reason why one way didn't work, was because you have to import from a module, while app, models and UserModel are not modules.
You must create a separate file to write the db syntax, then import it in models
in db.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
on main of app.py you can put this code:
if __name__=='__main__':
from db import db
db.init_app(app)
app.run(port=5000)

Getting Flask-Login to work with unittest

I know logins are disabled by default in testing. I'm trying to get them back on, by setting app.config['LOGIN_DISABLED'] to False. This doesn't seem to be working, since current_user still returns None, but will return a User in code outside a test file.
Some relevant bits of code are below. My Flask app object is originally created in my app's main __init__.py, which is imported and re-configured during testing.
===========
init.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager
app = Flask(__name__)
app.config[u'DEBUG'] = settings.debug
db = SQLAlchemy(app)
login_manager = LoginManager()
login_manager.init_app(app)
===========
tests/base.py
from my_app import app, db
import unittest
class BaseTest(unittest.TestCase):
def setUp(self):
app.config['TESTING'] = True
app.config['LOGIN_DISABLED'] = False
#app.login_manager._login_disabled = False #doesn't help either
self.app = app.test_client()
db.create_all()
===========
tests/test_a.py
from flask_login import current_user
from my_app.tests.base import BaseTest
class MyTests(BaseTest):
def test_a(self):
#visit endpoint that calls `login_user(user)`
#printing `current_user` in that endpoint works, but the following line only returns `None`
print current_user
Note: The User should definitely be logged in before the print statement. Using current_user in the endpoint works as expected.
Turns out my problem was due to testing not using the same context, so my solution is similar to what is described here and here. tests/test_a.py now looks like:
from flask_login import current_user
from my_app.tests.base import BaseTest
class MyTests(BaseTest):
def test_a(self):
with self.app:
print current_user
Strangely, app.config['LOGIN_DISABLED'] = False and app.login_manager._login_disabled = False don't matter after this change - both can be removed.

use db without requiring app.app_context()- Flask

I have an app like this:
myapp/app/init.py:
import sqlite3
from contextlib import closing
from flask import Flask, g
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager
# from app.models import db
from database import db
application = Flask(__name__)
application.config.from_object('config')
application.debug = True
db.init_app(application)
login_manager = LoginManager()
login_manager.init_app(application)
from app import views
myapp/database.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
myapp/app/models.py:
from database import db
from app import application
class CRUDMixin(object):
...
def delete(self, commit=True):
"""Remove the record from the database."""
with application.app_context():
db.session.delete(self)
return commit and db.session.commit()
class Model(CRUDMixin, db.Model):
"""Base model class that includes CRUD convenience methods."""
__abstract__ = True
def __init__(self, **kwargs):
db.Model.__init__(self, **kwargs)
class User(Model):
"""
:param str email: email address of user
:param str password: encrypted password for the user
"""
__tablename__ = 'users'
email = db.Column(db.String, primary_key=True)
password = db.Column(db.String)
authenticated = db.Column(db.Boolean, default=False)
def is_active(self):
"""True, as all users are active."""
return True
def get_id(self):
"""Return the email address to satisfy Flask-Login's requirements."""
return self.email
def is_authenticated(self):
"""Return True if the user is authenticated."""
return self.authenticated
def is_anonymous(self):
"""False, as anonymous users aren't supported."""
return False
The project I tried to structure after did not require with application.app_context() in the Model helper class. I cannot see any significant differences between my setup and its, and yet without with application.app_context() all over anything related to db I get the usual application not registered on db error. When everything you see in app/models.py and database.py was in app/__init__.py, it worked without requiring any with application.app_context() and I could import db raw in the shell like from myapp.app import db and it worked as is. What can I do to quiet the application not registered on db complaint but be able to use db easily without needing app_context, but still keep a proper directory structure where everything isn't jammed into init? Thank you
Flask-Script gives you a shell.
If you want to do it without Flask-Script, you must set the application context. A normal Python shell doesn't know how to setup your context.
It is easy to mimic the Flask-Script shell.
Create a shell.py file:
from app import application
ctx = application.app_context()
ctx.push()
Run it with python -i and use db with your app context already defined:
$ python -i shell.py
>>> from app import db
>>> db.create_all()

Flask SQLAlchemy Connect to MySQL Database

Setup & Code:
I'm building a basic Flask Application with an an AngularJS front end and I am currently at the point where I need to make a connection to a MySQL database I have hosted with Godaddy phpmyadmin.
This is part of my __init__.py
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
# Create instnace called app
app = Flask(__name__)
app.config['SQLAlchemy_DATABASE_URI'] = 'mysql://username:password##xxxxxx.hostedresource.com/dbname'
# Create SQLAlchemy object
db = SQLAlchemy(app)
# ...
This is my models.py
from app import app, db
class UsersPy(db.Model):
__tablename__ = "userspy"
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String, nullable=False)
password = db.Column(db.String, nullable=False)
def __init__(self, username, password):
self.username = username
self.password = password
def __repr__(self):
return '<title {}'.format(self.username)
This is a snippet from my views.py:
from app import app, db
from app.models import UsersPy
from flask import render_template, request, redirect, url_for, jsonify, session, flash
#app.route('/testdb/')
def testdb():
admin = UsersPy('user1', 'password1')
guest = UsersPy('user2', 'password2')
db.session.add(admin)
db.session.add(guest)
#db.session.merge(admin)
#db.session.merge(guest)
db.session.commit()
results = UsersPy.query.all()
json_results = []
for result in results:
d = {'username': result.username,
'password': result.password}
json_results.append(d)
return jsonify(items=json_results)
Problem:
All of this works well, the users are 'created' and displayed back in JSON format when you visit the /testdb/ location, however the actual database hosted with Godaddy is not being updated so the real connection must not be being made or it is failing for some reason. I have the userspy database table already created but add() and commit() functions are not actually adding users to the database. I can't figure out how to solidify the connection between SQLAlchemy and the MySQL Database. Any help is appreciated, thanks.
A good first step in debugging this would be to set SQLALCHEMY_ECHO to True in your app configuration. This will cause the actual MySQL database statements that SQLAlchemy is executing to be printed out.
Also, it's worth noting that SQLAlchemy_DATABASE_URI is different from SQLALCHEMY_DATABASE_URI, which is what the Flask-SQLAlchemy documentation states the config key should be.

Categories