I'm new in Django and I am creating a web application for uni project. I have to send emails periodically, and to do so I'm using a management command, but I don't know how to make it automatically run when I start the server.
I'm working on Pycharm in Windows 8.1
from django.core.mail import send_mail
from django.core.management.base import BaseCommand
from ProgettoDinamici.settings import EMAIL_HOST_USER
from products.models import Notification
from users.models import User
class Command(BaseCommand):
help = 'Sends emails periodically'
def handle(self, *args, **options):
users = User.objects.all()
for u in users:
try:
notify = Notification.objects.filter(receiver=u, read=False)
count = notify.count()
except:
print("No notification found")
try:
if notify:
send_mail(
'E-Commerce',
'You have ' + str(count) + ' notifications.',
EMAIL_HOST_USER,
[u.email],
fail_silently=False,
)
except:
print("error")
For now I tried to use schedule and cron to repeat the send_email every n minutes, but nothing worked and searching online I found out that cron (and cron based) ins't supported by Windows. But this is another problem...
You can use celery for periodic tasks. Just convert the function handle into a celery task and you can schedule cron jobs on that tasks.
You can refer: https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html
So I have a scheduled celery beat task (celery.py):
#app.on_after_configure.connect
def setup_periodic_tasks(sender,
**kwargs):
sender.add_periodic_task(10.0, test_event, name='test')
And the task (events/tasks.py):
#shared_task
def test_event():
from .models import Event
Event.objects.create()
When the event is created, a receiver is fired, that should send a message to a channels group (events/receivers.py):
#receiver(post_save, sender=Event)
def event_post_add(sender, instance, created, *args, **kwargs):
if created:
print("receiver fired")
Group("test").send({
"text": json.dumps({
'type': 'test',
})
})
The main problem is that the receiver is being fired in the celery beat process, and nothing is getting sent via django channels. No error messages, nothing, it's simply not being sent.
How can I integrate these two so that I will be able to send messages to channels from celery background processes?
hi i dont know if you found a solution or not. but as i was stuck on this problem myself so i tried a work around.
i created a view for the message that needs to be send by websocket and make a request to it from celery beat
the view:
def send_message(request,uuid,name):
print('lamo')
ty = f"uuid_{uuid}"
data={
'message':f'{name} Driver is now Avaliable',
'sender':'HQ',
'id':str(uuid),
'to':str(uuid),
'type':'DriverAvailable',
}
channel_layer = get_channel_layer()
async_to_sync(channel_layer.group_send)(
ty,
{'type':'chat_message',
'message':data,
}
)
and the task:
def my_task():
list=[]
for d in Driver_api.objects.all():
if d.available_on !=None:
if d.available_on <= timezone.now():
d.available_on = None
d.save()
uuid = str(d.user.uuid)
requests.get(f'{DOMAIN}message/sendMessage/{uuid}/{d.name}')
logger.info('Adding {0}'.format(d.user.uuid))
return list
sorry for any neglects or overlooks in my approach to the problem.
Signals are actually not being asynchronous in Django.
So in:
#shared_task
def test_event():
from .models import Event
Event.objects.create() # This will fire a signal and the signal will
# still be interpreted by celery
This issue is described at length in the following link:
https://githubmemory.com/repo/CJWorkbench/channels_rabbitmq/issues/37
I've checked the claim of reconnections and suboptimal perfomance of redis_channels (as described in link) and I couldn't find it happening.
here is my code that's working and Celery is sending a message to django channels.
class DjangoView(APIView):
def get(request):
send_message_to_channels.delay()
#shared_task
def send_message_to_channels():
send_test_message("Hello from celery", 200)
def send_test_message(message: str, code: int):
channel_layer = get_channel_layer()
channel_name = "celery-channels-test"
async_to_sync(channel_layer.group_send)(channel_name, {
'type': 'consumer.test.message',
'message': message,
'code': code
})
Pre Requisites to make my code work
install channels_redis (other packages just fail)
define channel_layer in django settings.py pointing to redis database
If this doesn't work I guess simple but very nasty approach would be to fire a request to a view in django like in #cosmo_boi answer
This question already has answers here:
RuntimeError: working outside of application context
(4 answers)
Closed 6 years ago.
I have a project which is structured similar to the overholt and fbone example. I can send emails from all my blueprints fine, but struggle to send outside. E.g. from within a cron script or celery task.
I keep getting the error working outside of application context
app/factory.py
from flask import Flask
from .extensions import mail
def create_app(package_name, package_path, settings_override=None,
register_security_blueprint=True):
app = Flask(package_name, instance_relative_config=True)
mail.init_app(app)
register_blueprints(app, package_name, package_path)
app.wsgi_app = HTTPMethodOverrideMiddleware(app.wsgi_app)
return app
app/extensions.py
from flask_mail import Mail
mail = Mail()
app/frontend/admin.py
bp = Blueprint('admin', __name__, url_prefix='/admin', static_folder='static')
#bp.route('/')
def admin():
msg = Message(......)
mail.send(msg)
Everything up until here works fine. Now I have a separate module in app which has some cron scripts which now fail.
app/cron/alerts.py
from ..extensions import mail
from flask.ext.mail import Message
def alert():
msg = Message('asdfasdf', sender='hello#example.com', recipients=['hello#example.com'])
msg.body = 'hello'
mail.send(msg)
Which produces the error. How can I get around this?
raise RuntimeError('working outside of application context')
RuntimeError: working outside of application context
You need use Flask-Mail:
from flask_mail import Mail
mail = Mail(app)
I would recommend going with celery. For the context problem I have found my solution in the following.
Have a look at this:
Miguel Grinberg's Blogpost on celery
Or if you are using the factory pattern in your application:
Celery with Factory Pattern
The second one is some kind of further/extended reading.
Both of them helped me a lot. (The second one also solved a context issue for me)
I am building a website using python Flask. Everything is going good and now I am trying to implement celery.
That was going good as well until I tried to send an email using flask-mail from celery. Now I am getting an "working outside of application context" error.
full traceback is
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/celery/task/trace.py", line 228, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/celery/task/trace.py", line 415, in __protected_call__
return self.run(*args, **kwargs)
File "/home/ryan/www/CG-Website/src/util/mail.py", line 28, in send_forgot_email
msg = Message("Recover your Crusade Gaming Account")
File "/usr/lib/python2.7/site-packages/flask_mail.py", line 178, in __init__
sender = current_app.config.get("DEFAULT_MAIL_SENDER")
File "/usr/lib/python2.7/site-packages/werkzeug/local.py", line 336, in __getattr__
return getattr(self._get_current_object(), name)
File "/usr/lib/python2.7/site-packages/werkzeug/local.py", line 295, in _get_current_object
return self.__local()
File "/usr/lib/python2.7/site-packages/flask/globals.py", line 26, in _find_app
raise RuntimeError('working outside of application context')
RuntimeError: working outside of application context
This is my mail function:
#celery.task
def send_forgot_email(email, ref):
global mail
msg = Message("Recover your Crusade Gaming Account")
msg.recipients = [email]
msg.sender = "Crusade Gaming stuff#cg.com"
msg.html = \
"""
Hello Person,<br/>
You have requested your password be reset. <a href="{0}" >Click here recover your account</a> or copy and paste this link in to your browser: {0} <br />
If you did not request that your password be reset, please ignore this.
""".format(url_for('account.forgot', ref=ref, _external=True))
mail.send(msg)
This is my celery file:
from __future__ import absolute_import
from celery import Celery
celery = Celery('src.tasks',
broker='amqp://',
include=['src.util.mail'])
if __name__ == "__main__":
celery.start()
Here is a solution which works with the flask application factory pattern and also creates celery task with context, without needing to use app.app_context(). It is really tricky to get that app while avoiding circular imports, but this solves it. This is for celery 4.2 which is the latest at the time of writing.
Structure:
repo_name/
manage.py
base/
base/__init__.py
base/app.py
base/runcelery.py
base/celeryconfig.py
base/utility/celery_util.py
base/tasks/workers.py
So base is the main application package in this example. In the base/__init__.py we create the celery instance as below:
from celery import Celery
celery = Celery('base', config_source='base.celeryconfig')
The base/app.py file contains the flask app factory create_app and note the init_celery(app, celery) it contains:
from base import celery
from base.utility.celery_util import init_celery
def create_app(config_obj):
"""An application factory, as explained here:
http://flask.pocoo.org/docs/patterns/appfactories/.
:param config_object: The configuration object to use.
"""
app = Flask('base')
app.config.from_object(config_obj)
init_celery(app, celery=celery)
register_extensions(app)
register_blueprints(app)
register_errorhandlers(app)
register_app_context_processors(app)
return app
Moving on to base/runcelery.py contents:
from flask.helpers import get_debug_flag
from base.settings import DevConfig, ProdConfig
from base import celery
from base.app import create_app
from base.utility.celery_util import init_celery
CONFIG = DevConfig if get_debug_flag() else ProdConfig
app = create_app(CONFIG)
init_celery(app, celery)
Next, the base/celeryconfig.py file (as an example):
# -*- coding: utf-8 -*-
"""
Configure Celery. See the configuration guide at ->
http://docs.celeryproject.org/en/master/userguide/configuration.html#configuration
"""
## Broker settings.
broker_url = 'pyamqp://guest:guest#localhost:5672//'
broker_heartbeat=0
# List of modules to import when the Celery worker starts.
imports = ('base.tasks.workers',)
## Using the database to store task state and results.
result_backend = 'rpc'
#result_persistent = False
accept_content = ['json', 'application/text']
result_serializer = 'json'
timezone = "UTC"
# define periodic tasks / cron here
# beat_schedule = {
# 'add-every-10-seconds': {
# 'task': 'workers.add_together',
# 'schedule': 10.0,
# 'args': (16, 16)
# },
# }
Now define the init_celery in the base/utility/celery_util.py file:
# -*- coding: utf-8 -*-
def init_celery(app, celery):
"""Add flask app context to celery.Task"""
TaskBase = celery.Task
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
with app.app_context():
return TaskBase.__call__(self, *args, **kwargs)
celery.Task = ContextTask
For the workers in base/tasks/workers.py:
from base import celery as celery_app
from flask_security.utils import config_value, send_mail
from base.bp.users.models.user_models import User
from base.extensions import mail # this is the flask-mail
#celery_app.task
def send_async_email(msg):
"""Background task to send an email with Flask-mail."""
#with app.app_context():
mail.send(msg)
#celery_app.task
def send_welcome_email(email, user_id, confirmation_link):
"""Background task to send a welcome email with flask-security's mail.
You don't need to use with app.app_context() here. Task has context.
"""
user = User.query.filter_by(id=user_id).first()
print(f'sending user {user} a welcome email')
send_mail(config_value('EMAIL_SUBJECT_REGISTER'),
email,
'welcome', user=user,
confirmation_link=confirmation_link)
Then, you need to start the celery beat and celery worker in two different cmd prompts from inside the repo_name folder.
In one cmd prompt do a celery -A base.runcelery:celery beat and the other celery -A base.runcelery:celery worker.
Then, run through your task that needed the flask context. Should work.
Flask-mail needs the Flask application context to work correctly. Instantiate the app object on the celery side and use app.app_context like this:
with app.app_context():
celery.start()
I don't have any points, so I couldn't upvote #codegeek's above answer, so I decided to write my own since my search for an issue like this was helped by this question/answer: I've just had some success trying to tackle a similar problem in a python/flask/celery scenario. Even though your error was from trying to use mail while my error was around trying to use url_for in a celery task, I suspect the two were related to the same problem and that you would have had errors stemming from the use of url_for if you had tried to use that before mail.
With no context of the app present in a celery task (even after including an import app from my_app_module) I was getting errors, too. You'll need to perform the mail operation in the context of the app:
from module_containing_my_app_and_mail import app, mail # Flask app, Flask mail
from flask.ext.mail import Message # Message class
#celery.task
def send_forgot_email(email, ref):
with app.app_context(): # This is the important bit!
msg = Message("Recover your Crusade Gaming Account")
msg.recipients = [email]
msg.sender = "Crusade Gaming stuff#cg.com"
msg.html = \
"""
Hello Person,<br/>
You have requested your password be reset. <a href="{0}" >Click here recover your account</a> or copy and paste this link in to your browser: {0} <br />
If you did not request that your password be reset, please ignore this.
""".format(url_for('account.forgot', ref=ref, _external=True))
mail.send(msg)
If anyone is interested, my solution for the problem of using url_for in celery tasks can be found here
In your mail.py file, import your "app" and "mail" objects. Then, use request context. Do something like this:
from whateverpackagename import app
from whateverpackagename import mail
#celery.task
def send_forgot_email(email, ref):
with app.test_request_context():
msg = Message("Recover your Crusade Gaming Account")
msg.recipients = [email]
msg.sender = "Crusade Gaming stuff#cg.com"
msg.html = \
"""
Hello Person,<br/>
You have requested your password be reset. <a href="{0}" >Click here recover your account</a> or copy and paste this link in to your browser: {0} <br />
If you did not request that your password be reset, please ignore this.
""".format(url_for('account.forgot', ref=ref, _external=True))
mail.send(msg)
The answer provided by Bob Jordan is a good approach but I found it very hard to read and understand so I completely ignored it only to arrive at the same solution later myself. In case anybody feels the same, I'd like to explain the solution in a much simpler way. You need to do 2 things:
create a file which initializes a Celery app
# celery_app_file.py
from celery import Celery
celery_app = Celery(__name__)
create another file which initializes a Flask app and uses it to monkey-patch the Celery app created earlier
# flask_app_file.py
from flask import Flask
from celery_app import celery_app
flask_app = Flask(__name__)
class ContextTask(celery_app.Task):
def __call__(self, *args, **kwargs):
with flask_app.app_context():
return super().__call__(self, *args, **kwargs)
celery_app.Task = ContextTask
Now, any time you import the Celery application inside different files (e.g. mailing/tasks.py containing email-related stuff, or database/tasks.py containg database-related stuff), it'll be the already monkey-patched version that will work within a Flask context.
The important thing to remember is that this monkey-patching must happen at some point when you start Celery through the command line. This means that (using my example) you have to run celery -A flask_app_file.celery_app worker because flask_app_file.py is the file that contains the celery_app variable with a monkey-patched Celery application assigned to it.
Without using app.app_context(), just configure the celery before you register blueprints like below :
celery = Celery('myapp', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0')
From your blueprint where you wish to use celery, call the instance of celery already created to create your celery task.
It will work as expected.
Below is a simple app to send mesg to the browser. if there is a new mesg from the redis channel it will be sent other wise send the last know value in a non-blocking way.
But i am doing something wrong. can someone please help me understand
from gevent import monkey, Greenlet
monkey.patch_all()
from flask import Flask,render_template,request,redirect,url_for,abort,session,Response,jsonify
app = Flask(__name__)
myglobaldict = {'somedata':''}
class RedisLiveData:
def __init__(self, channel_name):
self.channel_name = channel_name
self.redis_conn = redis.Redis(host='localhost', port=6379, db=0)
pubsub = self.redis_conn.pubsub()
gevent.spawn(self.sub, pubsub)
def sub(self,pubsub):
pubsub.subscribe(self.channel_name)
for message in pubsub.listen():
gevent.spawn(process_rcvd_mesg, message['data'])
def process_rcvd_mesg(mesg):
print "Received new message %s " % mesg
myglobaldict['somedata'] = mesg
g = RedisLiveData("test_channel")
#app.route('/latestmessage')
def latestmessage():
return Response(myglobaldict,mimetype="application/json")
if __name__ == '__main__':
app.run()
on the javascript side i am just using a simple $.ajax get to view the messages.
but the client http://localhost:5000/latestmessage has the old message even after the redis update.
It should be the cache issue.
You can add a timestamp or a random number to the request http://localhost:5000/latestmessage?t=timestamp sent from the ajax.
I suggest you to use POST instead of GET as http method, you eliminate the caching problem and some annoying behaviour from browsers like chrome where the requests after the first will wait for the first to complete before being sent to the webserver.
If you want to keep the GET method then you can ask jquery to make the request non cache-able by the browser with the setting parameter cache
$.ajax(..., {cache:false})