This question already has answers here:
Are global variables thread-safe in Flask? How do I share data between requests?
(4 answers)
Store large data or a service connection per Flask session
(1 answer)
Closed 3 years ago.
Similar to this question, I want to share one variable among an entire Flask application (all requests).
I am compiling a large data file (considerable overhead) and want to do the compilation as little as possible - storing the result in a variable so it can be accessed later. Using flask.g won't work because it is wiped clean between requests.
In a Reddit post six years ago, Armin Ronacher (creator of Flask) said:
You can just attach it to the application object and access it via
flask.current_app.whatever_variable.
Here are my two questions:
A) Is this still the recommended practice?
B) Are these two methods (attaching to current_app or app inside an application factory) equivalent?
Using current_app:
from flask import current_app
current_app.my_var = <<file>>
Using an application factory:
def create_app(config_filename):
app = Flask(__name__)
app.config.from_pyfile(config_filename)
app.my_var = <<file>>
...
return app
Normally in my projects i create a Globals.py to use globals variables in every part of my program, like this:
from celery import Celery
from flask_babel import Babel
from flask_marshmallow import Marshmallow
from flask_socketio import SocketIO
from flask import json, Flask
app: Flask = None
marshmallow: Marshmallow = Marshmallow()
socket = SocketIO(async_mode="eventlet", json=json)
celery: Celery
babel = Babel()
current_user = None
def init_globals(currentApp):
global marshmallow
global app
global socket
global celery
global babel
app = currentApp
marshmallow.init_app(app)
babel.init_app(app)
socket.init_app(app)
celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
celery.conf.update(app.config)
And do this in main.py
import Globals
app = Flask(__name__)
Globals.init_globals(app)
This question already has answers here:
Display the contents of a log file as it is updated
(3 answers)
Closed 3 years ago.
I have script which executes for 10 minutes and generates logs/outputs while executing..i want these logs to be passed to html file using flask.
You can use ajax methods in your javascript code to ask server for new logs/outputs with any period.
You can open WebSocket and write your logs to client in real-time. Look at example on flask-socketIO documentation: https://flask-socketio.readthedocs.io/en/latest/ or somewhere else.
Here is example how to send message to client:
from flask import Flask, render_template
from flask_socketio import SocketIO, send
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
#socketio.on('message')
def handle_message(message):
send(message)
if __name__ == '__main__':
socketio.run(app)
I have created a flask application using Blueprints.
This application receives data via paho.mqtt.client.
This is also the trigger to processes the data and run processes afterwards.
'system' is a blueprint containing mqtt.py and functions.py
functions.py contains the function to process the data once received
mqtt.py contains the definition of the mqtt client
mqtt.py
from app.system import functions
import paho.mqtt.client as mqtt
#....
def on_message(mqttc,obj,msg):
try:
data = json.loads(msg.payload.decode('utf-8'))
# start main process
functions.process(data)
except Exception as e:
print("error: ", e)
pass
Once I receive data and the on_message callback is triggered I get an out of application context error:
error: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
How can i get the application context within the on_message callback?
I tried importing current_app and using something like this
from flask import current_app
#...
def on_message(mqttc,obj,msg):
try:
data = json.loads(msg.payload.decode('utf-8'))
app = current_app._get_current_object()
with app.app_context():
# start main process
functions.process(data)
I still get the same error
There is this package - https://flask-mqtt.readthedocs.io/en/latest/ - that might help, but it only works with one worker instance.
Most of the time you set the application context when you create the app object.
So wherever you create your app is where you should initialize the extension. In your case it sounds like functions.py needs mqtt.py to carry out its logic, so you should initialize your mqtt client in your application creation.
From the flask docs - http://flask.pocoo.org/docs/1.0/appcontext/
If you see that error while configuring your application, such as when
initializing an extension, you can push a context manually since you
have direct access to the app. Use app_context() in a with block, and
everything that runs in the block will have access to current_app.
def create_app():
app = Flask(__name__)
with app.app_context():
#init_db()
initialize mqtt client
return app
I' trying to combine two independent Flask apps like the example below:
from geventwebsocket import WebSocketServer, Resource
...
server = WebSocketServer(('', 8080), Resource({
'/': frontend,
'/one': flask_app_one,
'/two': flask_app_two}))
server.serve_forever()
Inside each Flask app I declare the full path, isn't that suppose to be relative path, inside flask_app_one:
from flask import Flask
app = Flask(__name__)
#app.route('/one/ping')
def ping():
return 'hello\n'
Why I should specify in #app.route('/one/ping') instead of just #app.route('/ping') since all traffic to /one will be forwarded to the corresponding app?
Let me know if you need any additional info I kept my example clean
Thank you
Finally I have managed to do it with the so called Application Dispatching and the resources found in this page:
http://flask.pocoo.org/docs/0.10/patterns/appdispatch/#app-dispatch
Thanks
I am building a website using python Flask. Everything is going good and now I am trying to implement celery.
That was going good as well until I tried to send an email using flask-mail from celery. Now I am getting an "working outside of application context" error.
full traceback is
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/task/trace.py", line 228, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/celery/task/trace.py", line 415, in __protected_call__
return self.run(*args, **kwargs)
File "/home/ryan/www/CG-Website/src/util/mail.py", line 28, in send_forgot_email
msg = Message("Recover your Crusade Gaming Account")
File "/usr/lib/python2.7/site-packages/flask_mail.py", line 178, in __init__
sender = current_app.config.get("DEFAULT_MAIL_SENDER")
File "/usr/lib/python2.7/site-packages/werkzeug/local.py", line 336, in __getattr__
return getattr(self._get_current_object(), name)
File "/usr/lib/python2.7/site-packages/werkzeug/local.py", line 295, in _get_current_object
return self.__local()
File "/usr/lib/python2.7/site-packages/flask/globals.py", line 26, in _find_app
raise RuntimeError('working outside of application context')
RuntimeError: working outside of application context
This is my mail function:
#celery.task
def send_forgot_email(email, ref):
global mail
msg = Message("Recover your Crusade Gaming Account")
msg.recipients = [email]
msg.sender = "Crusade Gaming stuff#cg.com"
msg.html = \
"""
Hello Person,<br/>
You have requested your password be reset. <a href="{0}" >Click here recover your account</a> or copy and paste this link in to your browser: {0} <br />
If you did not request that your password be reset, please ignore this.
""".format(url_for('account.forgot', ref=ref, _external=True))
mail.send(msg)
This is my celery file:
from __future__ import absolute_import
from celery import Celery
celery = Celery('src.tasks',
broker='amqp://',
include=['src.util.mail'])
if __name__ == "__main__":
celery.start()
Here is a solution which works with the flask application factory pattern and also creates celery task with context, without needing to use app.app_context(). It is really tricky to get that app while avoiding circular imports, but this solves it. This is for celery 4.2 which is the latest at the time of writing.
Structure:
repo_name/
manage.py
base/
base/__init__.py
base/app.py
base/runcelery.py
base/celeryconfig.py
base/utility/celery_util.py
base/tasks/workers.py
So base is the main application package in this example. In the base/__init__.py we create the celery instance as below:
from celery import Celery
celery = Celery('base', config_source='base.celeryconfig')
The base/app.py file contains the flask app factory create_app and note the init_celery(app, celery) it contains:
from base import celery
from base.utility.celery_util import init_celery
def create_app(config_obj):
"""An application factory, as explained here:
http://flask.pocoo.org/docs/patterns/appfactories/.
:param config_object: The configuration object to use.
"""
app = Flask('base')
app.config.from_object(config_obj)
init_celery(app, celery=celery)
register_extensions(app)
register_blueprints(app)
register_errorhandlers(app)
register_app_context_processors(app)
return app
Moving on to base/runcelery.py contents:
from flask.helpers import get_debug_flag
from base.settings import DevConfig, ProdConfig
from base import celery
from base.app import create_app
from base.utility.celery_util import init_celery
CONFIG = DevConfig if get_debug_flag() else ProdConfig
app = create_app(CONFIG)
init_celery(app, celery)
Next, the base/celeryconfig.py file (as an example):
# -*- coding: utf-8 -*-
"""
Configure Celery. See the configuration guide at ->
http://docs.celeryproject.org/en/master/userguide/configuration.html#configuration
"""
## Broker settings.
broker_url = 'pyamqp://guest:guest#localhost:5672//'
broker_heartbeat=0
# List of modules to import when the Celery worker starts.
imports = ('base.tasks.workers',)
## Using the database to store task state and results.
result_backend = 'rpc'
#result_persistent = False
accept_content = ['json', 'application/text']
result_serializer = 'json'
timezone = "UTC"
# define periodic tasks / cron here
# beat_schedule = {
# 'add-every-10-seconds': {
# 'task': 'workers.add_together',
# 'schedule': 10.0,
# 'args': (16, 16)
# },
# }
Now define the init_celery in the base/utility/celery_util.py file:
# -*- coding: utf-8 -*-
def init_celery(app, celery):
"""Add flask app context to celery.Task"""
TaskBase = celery.Task
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
with app.app_context():
return TaskBase.__call__(self, *args, **kwargs)
celery.Task = ContextTask
For the workers in base/tasks/workers.py:
from base import celery as celery_app
from flask_security.utils import config_value, send_mail
from base.bp.users.models.user_models import User
from base.extensions import mail # this is the flask-mail
#celery_app.task
def send_async_email(msg):
"""Background task to send an email with Flask-mail."""
#with app.app_context():
mail.send(msg)
#celery_app.task
def send_welcome_email(email, user_id, confirmation_link):
"""Background task to send a welcome email with flask-security's mail.
You don't need to use with app.app_context() here. Task has context.
"""
user = User.query.filter_by(id=user_id).first()
print(f'sending user {user} a welcome email')
send_mail(config_value('EMAIL_SUBJECT_REGISTER'),
email,
'welcome', user=user,
confirmation_link=confirmation_link)
Then, you need to start the celery beat and celery worker in two different cmd prompts from inside the repo_name folder.
In one cmd prompt do a celery -A base.runcelery:celery beat and the other celery -A base.runcelery:celery worker.
Then, run through your task that needed the flask context. Should work.
Flask-mail needs the Flask application context to work correctly. Instantiate the app object on the celery side and use app.app_context like this:
with app.app_context():
celery.start()
I don't have any points, so I couldn't upvote #codegeek's above answer, so I decided to write my own since my search for an issue like this was helped by this question/answer: I've just had some success trying to tackle a similar problem in a python/flask/celery scenario. Even though your error was from trying to use mail while my error was around trying to use url_for in a celery task, I suspect the two were related to the same problem and that you would have had errors stemming from the use of url_for if you had tried to use that before mail.
With no context of the app present in a celery task (even after including an import app from my_app_module) I was getting errors, too. You'll need to perform the mail operation in the context of the app:
from module_containing_my_app_and_mail import app, mail # Flask app, Flask mail
from flask.ext.mail import Message # Message class
#celery.task
def send_forgot_email(email, ref):
with app.app_context(): # This is the important bit!
msg = Message("Recover your Crusade Gaming Account")
msg.recipients = [email]
msg.sender = "Crusade Gaming stuff#cg.com"
msg.html = \
"""
Hello Person,<br/>
You have requested your password be reset. <a href="{0}" >Click here recover your account</a> or copy and paste this link in to your browser: {0} <br />
If you did not request that your password be reset, please ignore this.
""".format(url_for('account.forgot', ref=ref, _external=True))
mail.send(msg)
If anyone is interested, my solution for the problem of using url_for in celery tasks can be found here
In your mail.py file, import your "app" and "mail" objects. Then, use request context. Do something like this:
from whateverpackagename import app
from whateverpackagename import mail
#celery.task
def send_forgot_email(email, ref):
with app.test_request_context():
msg = Message("Recover your Crusade Gaming Account")
msg.recipients = [email]
msg.sender = "Crusade Gaming stuff#cg.com"
msg.html = \
"""
Hello Person,<br/>
You have requested your password be reset. <a href="{0}" >Click here recover your account</a> or copy and paste this link in to your browser: {0} <br />
If you did not request that your password be reset, please ignore this.
""".format(url_for('account.forgot', ref=ref, _external=True))
mail.send(msg)
The answer provided by Bob Jordan is a good approach but I found it very hard to read and understand so I completely ignored it only to arrive at the same solution later myself. In case anybody feels the same, I'd like to explain the solution in a much simpler way. You need to do 2 things:
create a file which initializes a Celery app
# celery_app_file.py
from celery import Celery
celery_app = Celery(__name__)
create another file which initializes a Flask app and uses it to monkey-patch the Celery app created earlier
# flask_app_file.py
from flask import Flask
from celery_app import celery_app
flask_app = Flask(__name__)
class ContextTask(celery_app.Task):
def __call__(self, *args, **kwargs):
with flask_app.app_context():
return super().__call__(self, *args, **kwargs)
celery_app.Task = ContextTask
Now, any time you import the Celery application inside different files (e.g. mailing/tasks.py containing email-related stuff, or database/tasks.py containg database-related stuff), it'll be the already monkey-patched version that will work within a Flask context.
The important thing to remember is that this monkey-patching must happen at some point when you start Celery through the command line. This means that (using my example) you have to run celery -A flask_app_file.celery_app worker because flask_app_file.py is the file that contains the celery_app variable with a monkey-patched Celery application assigned to it.
Without using app.app_context(), just configure the celery before you register blueprints like below :
celery = Celery('myapp', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0')
From your blueprint where you wish to use celery, call the instance of celery already created to create your celery task.
It will work as expected.