We Keep Coding

Getting 'RuntimeError: Working outside of application context.' when trying to import function from another Blueprint

Initial Notes: The project uses Blueprints and below are the file structure and extracts of the code used...
File Structure:
/app
├── flaskapp/
│ ├── posts/
│ │ ├── __init__.py
│ │ ├── forms.py
│ │ ├── routes.py
│ │ ├── utils.py
│ ├── users/
│ │ ├── __init__.py
│ │ ├── forms.py
│ │ ├── routes.py
│ │ ├── utils.py
│ ├── main/
│ │ ├── __init__.py
│ │ ├── crons.py
│ │ ├── routes.py
│ ├── templates/
│ │ ├── users.html
│ ├── __init__.py
│ ├── config.py
│ ├── models.py
├── run.py
posts/utils.py
# Function to get all posts from DB
def get_all_posts():
post = Post.query.order_by(Post.id.asc())
return post
users/routes.py
# Importing 'get_all_posts' function from 'posts/utils.py'
from flaskapp.posts.utils import get_all_posts
users = Blueprint('users', __name__)
#All Users Route + Related Posts
#posts.route("/posts", methods=['GET'])
#login_required
def all_users():
users = User.query.order_by(User.id.asc())
return render_template('users.html', USERS=users, POSTS=get_all_posts())
main/crons.py
# Importing 'get_all_posts' function from 'posts/utils.py'
from flaskapp.posts.utils import get_all_posts
# A function to be called using 'scheduler' from 'flaskapp/__init__.py' on launch
def list_expired_posts():
posts = get_all_posts()
for p in posts:
if p.expired == 1:
print(p.title)
scheduler = BackgroundScheduler()
scheduler.add_job(func=list_expired_posts, trigger="interval", seconds=60)
scheduler.start()
# Terminate Scheduler on APP termination
atexit.register(lambda: scheduler.shutdown())
flaskapp/init.py
# __init__.py Main
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager
from flaskapp.config import Config
db = SQLAlchemy()
login_manager = LoginManager()
login_manager.login_view = 'users.login'
login_manager.login_message_category = 'warning'
# Create APP Function
def create_app(config_class=Config):
app = Flask(__name__)
# Import Configuration from Config File Class
app.config.from_object(Config)
db.init_app(app)
login_manager.init_app(app)
# Import Blueprint objects
from flaskapp.posts.routes import posts
from flaskapp.users.routes import users
from flaskapp.main.routes import main
# Register Blueprints
app.register_blueprint(posts)
app.register_blueprint(users)
app.register_blueprint(main)
# END
return(app)
# Calling scheduler function 'list_expired_posts' FROM '/main/crons.py' as a scheduled job to be triggered on app initiation
from flaskapp.main.crons import list_expired_posts
list_expired_posts()
Explanation:
I have the function 'get_all_posts()' located in 'posts/utils.py' which works fine when I import it and use it in another blueprint's routes.py file (ex.** users/routes.py**) as shown above.
But I'm getting the below error when importing the same function in another blueprint (specifically.** main/crons.py**) as shown above.
I'm trying to use the 'get_all_posts()' function from 'posts/utils.py' within the 'list_expired_posts()' in 'main/crons.py' and then calling the 'list_expired_posts()' function from 'flaskapp/init.py' to trigger it on launch and keep on executing it every 60 minutes.
Error:
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
the current application. To solve this, set up an application context
with app.app_context(). See the documentation for more information.
*Conclusion Notes + Steps attempted: *
I have already eliminated the 'Scheduler' temporarily and tried working only with the function itself and not even calling it from 'flaskapp/init.py'.
I have also tried moving the below code to the 'def create_app(config_class=Config)' section without any luck
from flaskapp.main.crons import list_expired_posts
list_expired_posts()
I have also tried creating a specific Blueprint for 'crons.py' and registering it to my 'flaskapp/init.py' but still got the same result.
As a final outcome, I am trying to call the 'get_all_posts()' FROM 'posts/utils.py', then filter out the 'expired posts' using the 'list_expired_posts()' function FROM 'main/crons.py' and schedule it to print the title of the expired posts every 60 minutes.
Since I've eliminated the scheduler already to test out, I'm quite sure this is not a scheduler issue but some import mixup I'm not figuring out.
I am also aware that the 'list_expired_posts()' can become as another function in 'posts/utils.py' and then directly call the function from there using the scheduler which I've also tried but keep getting the same error.
I also tried manually configuring the app's context as instructed in other posts but I keep getting the same error.
with app.app_context():
I'm not really a Python pro and I always try seeking multiple online resources prior to posting a question here but it seems like i'm out of luck this time. Your help is truly appreciated.

There is no blueprint in crons.py, which is expected as it's about scheduling, not routing.
Not being in a route, you get the error that the application context is missing, because effectively everything not in a route does not have an application context.
You might want to look into Flask-APScheduler, which you'd initialise in a similar way to Flask-SQLAlchemy and then use in your crons.py, it should carry the application context transparently.
PS. In users/routes.py you define a users Blueprint, but them seem to use a posts blueprint in the route decorator.
Ok, sorry for delay but here is a demo of using Flask-SQLAlchemy and Flask-APSscheduler to access the app context from scheduled tasks.
from flask import Blueprint, Flask
from flask_apscheduler import APScheduler
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
scheduler = APScheduler()
users = Blueprint("users", __name__)
class User(db.Model):
id = db.Column(db.Integer(), primary_key=True)
username = db.Column(db.String(100), unique=True, nullable=False)
def __repr__(self):
return "<Name %r>" % self.id
def get_all_users():
return User.query.all()
#users.route("/all", methods=["GET"])
def handle_users_all():
return get_all_users()
#scheduler.task("cron", id="job_get_all_users", second="*/10")
def scheduled_get_all_users():
with scheduler.app.app_context():
get_all_users()
def create_app():
app = Flask(__name__)
app.register_blueprint(users)
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:////tmp/74975778.db"
db.init_app(app)
with app.app_context():
db.create_all()
app.config["SCHEDULER_API_ENABLED"] = True
scheduler.init_app(app)
scheduler.start()
return app
NB. this does not do anything useful in the scheduled task, I leave it up to you, but it can execute the utility function get_all_users on schedule.

I also don't think this is directly related to the scheduler. I think it works in your users/routes.py because it is inside a route. So it will only get called when you visit that route. And by then your app is fully created and has its context and whatever.
But in main/crons.py it is not inside a route so I think it will try to run it when it passes this import line in your flaskapp/init.py
from flaskapp.main.crons import list_expired_posts
Maybe you could try this: putting the scheduler part into a function
main/crons.py
# Importing 'get_all_posts' function from 'posts/utils.py'
from flaskapp.posts.utils import get_all_posts
# A function to be called using 'scheduler' from 'flaskapp/__init__.py' on launch
def list_expired_posts():
posts = get_all_posts()
for p in posts:
if p.expired == 1:
print(p.title)
def initiate_scheduler():
scheduler = BackgroundScheduler()
scheduler.add_job(func=list_expired_posts, trigger="interval", seconds=60)
scheduler.start()
and then call that function in your run.py after you create the app
run.py
# first create the app
app = create_app()
# then start the scheduler
initiate_scheduler()
if __name__ == '__main__':
app.run()
# if you visit '/posts' in a browser it will call the get_all_posts()
# at this point in time after all the other code has ran.

Register Tortoise signals in fast API

I just created a signal to generate a token when the user is created, I use tortoise.signals to generate it in a separate file, my project structure is like this
├── Makefile
├── README.md
├── core
│ ├── __init__.py
│ ├── authentication.py
│ ├── email.py
│ ├── models.py
│ ├── router.py
│ ├── schema.py
│ ├── signals.py
│ └── views.py
├── main.py
├── requirements.txt
├── static
└── templates
└── verification.html
And here is my signal file
# signals.py
from tortoise.signals import post_save
from tortoise import BaseDBAsyncClient
from typing import Type, List, Optional
from .models import UserModel, business_pydantic
from .email import send_email
#post_save(sender=UserModel)
async def create_business(
sender: "Type[UserModel]",
instance: UserModel,
created: bool,
using_db: Optional[BaseDBAsyncClient],
update_fields: List[str],
) -> None:
if created:
business_objects: UserModel = await UserModel.create(
business_name = instance.username, owner_id = instance
)
await business_pydantic.from_tortoise_orm(business_objects)
# send email
await send_email([instance.email], instance)
and I import the module in function create_app
from dotenv import dotenv_values
from fastapi import FastAPI
from tortoise.contrib.fastapi import register_tortoise
from fastapi.staticfiles import StaticFiles
from .router import router
def create_app():
app = FastAPI()
# mount static files
app.mount("/static", StaticFiles(directory="static"), name="static")
import signals # <- import my signal file here
register_tortoise(
app=app,
db_url="postgres://postgres:1234#localhost:5432/ecommerce_db",
modules={"models": ["core.models"]},
generate_schemas=True,
add_exception_handlers=True,
)
# add routes
app.include_router(router)
return app
My problem is that the signal that I created in the signals.py file is not read/executed, how do I make the functions that I make executed properly if I make them in a separate file, is there a way to register so that the signal is read in fast API?
Thanks!

How should the startup of a flask app be structured?

I have built a flask app that I have been starting from an if __name__ == '__main__': block, as I saw in a tutorial. When the time came to get the app to launch from wsgi for production use, I had to remove the port and host options from app.run(), and make changes to the structure of the launching code that I am not too sure about. I am now adding test cases, which adds more ways to launch and access the app (with app.test_client(), with app.test_request_context(), and who knows what else.) What is the right / recommended way to structure the code that creates and launches the application, so that it behaves correctly (and consistently) when launched stand-alone, from wsgi, and for testing?
My current structure is as follows:
def create_app():
"""
Create and initialize our app. Does not call its run() method
"""
app = Flask(__name__)
some_initialization(app, "config_file.json")
return app
app = create_app()
...
# Services decorated with #app.route(...)
...
if __name__ == "__main__":
# The options break wsgi, I had to use `run()`
app.run(host="0.0.0.0", port=5555)
To be clear, I have already gotten wsgi and tests to work, so this question is not about how to do that; it is about the recommended way to organize the state-creating steps so that the result behaves as a module, the app object can be created as many times as necessary, service parameters like port and server can be set, etc. What should my code outline actually look like?
In addition to the launch flag issue, the current code creates an app object (once) as a side effect of importing; I could create more with create_app() but from mycode import app will retrieve the shared object... and I wonder about all those decorators that decorated the original object.
I have looked at the documentation, but the examples are simplified, and the full docs present so many alternative scenarios that I cannot figure out the code structure that the creators of Flask envisioned. I expect this is a simple and it must have a well-supported code pattern; so what is it?

Disclaimer While this isn't the only struture for Flask, this has best suited my needs and is inspired from the Flask officials docs of
using a Factory
Pattern
Project Structure
Following the structure from the Documentation
/home/user/Projects/flask-tutorial
├── flaskr/
│ ├── __init__.py
│ ├── db.py
│ ├── schema.sql
│ ├── auth.py
│ ├── blog.py
│ ├── templates/
│ │ ├── base.html
│ │ ├── auth/
│ │ │ ├── login.html
│ │ │ └── register.html
│ │ └── blog/
│ │ ├── create.html
│ │ ├── index.html
│ │ └── update.html
│ └── static/
│ └── style.css
├── tests/
│ ├── conftest.py
│ ├── data.sql
│ ├── test_factory.py
│ ├── test_db.py
│ ├── test_auth.py
│ └── test_blog.py
├── venv/
├── setup.py
└── MANIFEST.in
flaskr/, a Python package containing your application code and files.
flaskr will contain the factory to generate flask app instances that can be used by WGSI servers and will work with tests, orms (for migrations) etc.
flaskr/__init__.py contains the factory method
The Factory
The factory is aimed at configuring and creating a Flask app. This means you need to pass all required configurations in one of the many ways accepted by Flask
The dev Flask server expects the function create_app() to be present in the package __init__.py file. But when using a production server like those listed in docs you can pass the name of the function to call.
A sample from the documentation:
# flaskr/__init__.py
import os
from flask import Flask
def create_app(test_config=None):
# create and configure the app
app = Flask(__name__, instance_relative_config=True)
app.config.from_mapping(
SECRET_KEY='dev',
DATABASE=os.path.join(app.instance_path, 'flaskr.sqlite'),
)
if test_config is None:
# load the instance config, if it exists, when not testing
app.config.from_pyfile('config.py', silent=True)
else:
# load the test config if passed in
app.config.from_mapping(test_config)
# ensure the instance folder exists
try:
os.makedirs(app.instance_path)
except OSError:
pass
# a simple page that says hello
#app.route('/hello')
def hello():
return 'Hello, World!'
return app
when running a dev flask server, set environment variables as described
$ export FLASK_APP=flaskr
$ export FLASK_ENV=development
$ flask run
The Routes
A Flask app may have multiple modules that require the App object for functioning, like #app.route as you mentioned in the comments. To handle this gracefully we can make use of Blueprints. This allows us to keep the routes in a differnt file and register them in create_app()
from flask import Blueprint, render_template, abort
from jinja2 import TemplateNotFound
simple_page = Blueprint('simple_page', __name__,
template_folder='templates')
#simple_page.route('/', defaults={'page': 'index'})
#simple_page.route('/<page>')
def show(page):
try:
return render_template(f'pages/{page}.html')
except TemplateNotFound:
abort(404)
and we can modify the create_app() to register blueprint as follows:
def create_app(test_config=None):
app = Flask(__name__, instance_relative_config=True)
# configure the app
.
.
.
from yourapplication.simple_page import simple_page
app.register_blueprint(simple_page)
return app
You will need to locally import the blueprint to avoid circular imports. But this is not graceful when having many blueprints. Hence we can create an init_blueprints(app) function in the blueprints package like
# flaskr/blueprints/__init__.py
from flaskr.blueprints.simple_page import simple_page
def init_blueprints(app):
with app.app_context():
app.register_blueprint(simple_page)
and modify create_app() as
from flaskr.blueprints import init_blueprints
def create_app(test_config=None):
app = Flask(__name__, instance_relative_config=True)
# configure the app
.
.
.
init_blueprints(app)
return app
this way your factory does not get cluttered with blueprints. And you can handle the registration of blueprints inside the blueprint package as per your choice. This also avoids circular imports.
Other Extensions
Most common flask extensions support the factory pattern that allows you to create an object of an extension and then call obj.init_app(app) to initialize it with Flask App. Takeing Marshmallow here as an exmaple, but it applies to all. Modify create_app() as so -
ma = Marshmallow()
def create_app(test_config=None):
app = Flask(__name__, instance_relative_config=True)
# configure the app
.
.
.
init_blueprints(app)
ma.init_app(app)
return app
you can now import ma from flaskr in which ever file required.
Production Server
As mentioned initialially, the production WSGI serevrs will call the create_app() to create instances of Flask.
using gunicorn as an example, but all supported WSGI servers can be used.
$ gunicorn "flaskr:create_app()"
You can pass configurations as per gunicorn docs, and the same can be achieved within a script too.

What I did was:
class App:
def __init__(self):
# Various other initialization (e.g. logging, config, ...)
...
self.webapp = self._start_webapp(self.app_name, self.app_port, self.log)
pass
def _start_webapp(self, app_name: str, app_port: Optional[int], log: logging):
log.info('Running webapp...')
webapp = Flask(app_name)
# other flask related code
...
webapp.run(debug=False, host='0.0.0.0', port=app_port)
return webapp
pass
if __name__ == '__main__':
app = App()
This way you can add optional parameters to the init to override during tests or override via config change and even create additional types of endpoints in the same application, if you need.

Initializing database with flask

I have the following code:
init.py:
"""Initialize app."""
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def create_app():
"""Construct the core application."""
app = Flask(__name__, instance_relative_config=False)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///db.sqlite'
app.config['RECAPTCHA_PUBLIC_KEY'] = '6LcmEeoUAAAAAIbdhgkFBvz676UCRJMjnSx8H6zy'
app.config['RECAPTCHA_PARAMETERS'] = {'size': '100%'}
db.init_app(app)
# blueprint for auth routes in our app
from .auth import auth as auth_blueprint
app.register_blueprint(auth_blueprint)
# blueprint for non-auth parts of app
from .__main__ import main as main_blueprint
app.register_blueprint(main_blueprint)
with app.app_context():
# Import parts of our application
from . import routes
return app
and I try to initialize the db with the following code:
from realProject import db, create_app
db.create_all(app=create_app())
all of the scripts are in realProject folder
but when trying to compile the last code I get this error:
ModuleNotFoundError: No module named 'realProject'
What am I doing wrong?

You need to follow this structure :
|__project_name
|__app.py ----------> The main file from where you run your app
|__app -----> This is the app folder
├── templates
│ └── index.html
└── __init__.py -----------> The __init__.py should be inside the app folder for it to be imported
└── routes.py
And then do this in your main file :
from app import db, create_app
db.create_all(app=create_app())

Share sqlalchemy models between flask and other apps

I have a running Flask application that is set up according to a combination of best practices we found online and in Miguel Grinberg's "Flask Web Development" book.
We now need a second Python application, that is NOT a web app, and that needs access to the same models as the Flask application. We wanted to re-use the same models ofcourse, so both apps can benefit from the shared code.
We have removed dependencies on the flask-sqlalchemy extention (which we used before, when we had just the Flask application). And replaced it with the SQLalchemy Declarative extension described here, which is a bit simpler (Flask-SQLalchemy adds a few specific things to standard SQLAlchemy)
In line with the example we have created a database.py file in the root. In our case there are two things different from the Declarative extension example: I put the engine and session in a class, because all of our models use db.session, instead of db_session, and I pass a dictionary with configuration values to the init(), so that I can re-use this database.py from both Flask as well as another application, using a different configuration. it looks like this:
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
class Database(object):
def __init__(self, cfg):
self.engine = create_engine(cfg['SQLALCHEMY_DATABASE_URI'], convert_unicode=True)
self.session = scoped_session(sessionmaker(autocommit=False, autoflush=False, bind=self.engine))
class Model(object):
pass
Base = declarative_base()
So now we come to the actual problem. Flask creates a dictionary-like object containing configuration options, and adds them as a property to the app-instance. It loads them from an instance folder, a config.py in the root of the site and from environment variables. I need to pass in the configuration dictionary from Flask, so I need Flask to FIRST load and assemble the configuration, and after that initialise the database, and have a (configured) db object in the root of the app file. However, we follow the Application factory pattern, so we can use different configurations for different situations (test, production, development).
This means our app/__init__.py looks something like this (simplified):
from flask import Flask
from database import Database
from flask.ext.mail import Mail
from flask_bcrypt import Bcrypt
from config import config
mail = Mail()
bcrypt = Bcrypt()
def create_app(config_name):
app = Flask(__name__, instance_relative_config=True)
if not config_name:
config_name = 'default'
app.config.from_object(config[config_name])
app.config.from_pyfile('config.py')
config[config_name].init_app(app)
db = Database(app.config)
mail.init_app(app)
bcrypt.init_app(app)
#app.teardown_appcontext
def shutdown_session(exception=None):
db.session.remove()
from main import main as main_blueprint
app.register_blueprint(main_blueprint)
return app
But the db (that the models import from ..), now needs to be inside the create_app() function, because that's where Flask loads the configuration. If I would instantiate the db object outside of the create_app() function, it will be importable from the models, but it is not configured!
an example model looks like this, and as you can see, it expects a "db" in the root of the app:
from . base_models import areas
from sqlalchemy.orm import relationship, backref
from ..utils.helper_functions import newid
from .. import db
class Areas(db.Model, areas):
"""Area model class.
"""
country = relationship("Countries", backref=backref('areas'))
def __init__(self, *args, **kwargs):
self.area_id = newid()
super(Areas, self).__init__(*args, **kwargs)
def __str__(self):
return u"{}".format(self.area_name).encode('utf8')
def __repr__(self):
return u"<Area: '{}'>".format(self.area_name).encode('utf8')
So my question is, how can I have a db instance that can be configured externally (by either Flask or another app), and still use the Application Factory Pattern?
edit: The code-example was incorrect, it had an import for Flask-SQLalchemy which was replaced by from database import Database. Sorry for any confusion.

The Flask-SQLAlchemy extension, like most Flask extensions, should be created outside the factory, then initialized in the factory using init_app. This is so that you can use the db object before an app is created.
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
db.init_app(app)
return app
Your Flask app, like any properly designed Python project, should be an installable package. This is simple to do: make sure your project layout makes sense, then add a basic setup.py file.
project/
my_flask_package/
__init__.py # at the most basic, this contains create_app and db
setup.py
from setuptools import setup, find_packages
setup(
name='my_flask_package',
version='1.0',
packages=find_packages(),
install_requires=['flask', 'flask-sqlalchemy'],
)
$ python setup.py sdist
Now you can install your Flask app, along with it's database, for use in other projects. Install and import it in your second project's virtualenv, then create and push an app to initialize it.
$ pip install my_flask_package-1.0.tar.gz
from my_flask_package import db, create_app
create_app().app_context().push()
db.session.query(...)
If you're concerned about overhead involved in creating your application, you could add arguments to the create_app function to control what gets initialized. For most cases this shouldn't be an issue though.

I ran into the same problem.
If you turn on "SQLALCHEMY_ECHO" you'll likely see that a new transaction is started but the corresponding COMMIT/ ROLLBACK is missing.
For what i found out, it has something to do with two SQLAlchemy instances which you also create, once in your model file and once in your web.py. Most likely it's because you interact with your web.py's session and if you query your models there is some context switched which will receive the COMMIT.
I fixed the issue by importing "db" from models and then init it by calling db.init_app(app). According to the logs, committing now works fine.
The #app.teardown_appcontext shouldn't be necessary as it is set up in Flask-SQLAlchemy's SQLAlchemy class (https://github.com/mitsuhiko/flask-sqlalchemy/blob/master/flask_sqlalchemy/init.py)

You can easily share. I will show how. Considering this Flask app:
.
├── config.py
├── db
│   └── test.db
├── do_somenthing2.py ============> Here is run some script 2
├── do_something.py ============> Here is run some script
├── machinelearning
│   ├── models
│   │   ├── restore.py
│   │   ├── train.py
│   │   └── utils.py
│   └── save
│   └── test.ckpt
├── runserver.py ============> Here is run your app
├── test.py
└── web
├── __init__.py
├── api
│   ├── __init__.py
│   ├── app.py ============> Here is app = Flask(__name__)
│   ├── client.py
│   ├── models.py ==========> Here is db = SQLAlchemy(app)
│   ├── sample.json
│   └── utils.py
└── frontend
├── __init__.py
└── routes.py
runserver.py
import os
from config import DEBUG
from web.api.app import app
from web.api.client import *
if __name__ == "__main__":
app.run(debug=DEBUG)
OK. Now you want do use the same models to do another thing. For example: train a machine, serve and save in database (ORM) using the same models.
You can import the app and use the app.test_request_context(). Like that:
do_something.py
from web.api.app import app
from web.api.models import db, user
def do_something():
q = db.session.query(User)\
.filter(User.Name.ilike('Andre'))
for i in q.all():
print (i.Name)
with app.test_request_context():
do_something()
do_something2.py (real example)
from web.api.app import app
from web.api.models import *
def save(df):
passengers = []
records = df.to_dict('records')
for row in records:
p = Passenger(row)
passengers.append(p)
for p in passengers:
db.session.add(p)
db.session.commit()
from ml.models import train, restore
with app.test_request_context():
print ('Trainning model. It will take a while... (~ 5 minutos)')
train.run()
print ('Saving model...')
save(restore.run())
print ('Saved!')
Many answers recommend use (Importing files from different folder):
import sys
sys.path.append('../')
But I disagree when you have a Flask app and other scripts, because you will get crazy solving the relative references.
The approach to install your Flask app, along with it's database, for use in other projects, is another option.
Here you can find a documentation about the packages and modules.
Packages are a way of structuring Python’s module namespace by using
“dotted module names”. For example, the module name A.B designates a
submodule named B in a package named A. Just like the use of modules
saves the authors of different modules from having to worry about each
other’s global variable names, the use of dotted module names saves
the authors of multi-module packages like NumPy or Pillow from having
to worry about each other’s module names.