I've been building an app that uses flask, flask_jwt_extended and the decorator #jwt_required around protected functions that need an access token to access. From these endpoints, I can use flask_jwt_extended's get_current_user function to fetch the currrent user.
This has been working fine while in development, but now I'm planning to serve a production application using gunicorn. However, when I run the gunicorn server, it seems like the get_current_user function is always returning none. For example, the following code works with python3 -m flask run, but not with gunicorn run:app -b localhost:5000. What could be the problem here?
#jwt_required
def get_user_by_id(user_id: str) -> Dict[str, Any]:
# returns user when using flask run, but not with gunicorn
curr_user = get_current_user()
Class-worker
Do you use multiple workers and sync class ?
Maybe try to look at gunicorn worker-class
If you try to use the sync worker type and set the threads setting to more than 1, the gthread worker type will be used instead.
In your python configuration file or you can change worker-class directly in the command line.
worker_class = 'gthread'
Check your JWT_SECRET_KEY
JWT_SECRET_KEY need to have the same value all over each workers
Related
I have a slow request, and I want to change timeout for the worker during handling of that request and only for that request.
Basically, I have a flask application:
class Slow(Resource):
def post(self):
if slow_condition():
gunicorn.how.to.extend.processing.time.here()
do_something_slow()
api = Api(application)
api.add_resource(Slow, "/slow")
and I want to extend processing time if slow_condition returned True. How can I change timeout for the single request?
No way.
Flask is just web framework. Framework doesn't know anything about production server and settings, workers etc. By the way to change sever configuration you need to reload all workers(restart gunicorn, uWSGI, waitress etc). So you can only increase timeout parameter.
I have a flask server running within a gunicorn.
In my flask application I want to handle large upload files (>20GB), so I plan on letting a celery task do the handling of the large file.
The problem is that retrieving the file from request.files already takes quite long, in the meantime gunicorn terminates the worker handling that request. I could increase the timeout time, but the maximum file size is currently unknown, so I don't know how much time I would need.
My plan was to make the request context available to the celery task, as it was described here: http://xion.io/post/code/celery-include-flask-request-context.html, but I cannot make it work
Q1 Is the signature right?
I set the signature with
celery.signature(handle_large_file, args={}, kwargs={})
and nothing is complaining. I get the arguments I pass from the flask request handler to the celery task, but that's it. Should I somehow get a handle to the context here?
Q2 how to use the context?
I would have thought if the flask request context was available I could just use request.files in my code, but then I get the warning that I am out of context.
Using celery 4.4.0
Code:
# in celery.py:
from flask import request
from celery import Celery
celery = Celery('celery_worker',
backend=Config.CELERY_RESULT_BACKEND,
broker=Config.CELERY_BROKER_URL)
#celery.task(bind=True)
def handle_large_file(task_object, data):
# do something with the large file...
# what I'd like to do:
files = request.files['upfile']
...
celery.signature(handle_large_file, args={}, kwargs={})
# in main.py
def create_app():
app = Flask(__name__.split('.')[0])
...
celery_worker.conf.update(app.config)
# copy from the blog
class RequestContextTask(Task):...
celery_worker.Task = RequestContextTask
# in Controller.py
#FILE.route("", methods=['POST'])
def upload():
data = dict()
...
handle_large_file.delay(data)
What am I missing?
I am using gunicorn to run my Flask application. I would like to register server hooks to perform some action at the start of the application and before shutting down, but I am confused on how to pass variables to these functions and how to extract variables created within them.
In gunicorn.conf.py:
bind = "0.0.0.0:8000"
workers = 2
loglevel = "info"
preload = True
def on_starting(server):
# register some variables here
print "Starting Flask application"
def on_exit(server):
# perform some clean up tasks here using variables from the application
print "Shutting down Flask application"
In app.py, the sample Flask application:
from flask import Flask, request, jsonify
app = Flask(__name__)
#app.route('/hello', methods=['POST'])
def hello_world():
return jsonify(message='Hello World')
if __name__ == '__main__':
app.run(host='0.0.0.0', port=9000, debug=False)
Running gunicorn like so: $ gunicorn -c gunicorn.conf.py app:app
A bit late, but you have access to the flask application instance through server.app.wsgi(). It returns the same instance used by the workers (the one that is also returned by flask.current_app).
def on_exit(server):
flask_app = server.app.wsgi()
# do whatever with the flask app
Put the data you need to pass to the hooks into environment variables.
You can also store the data to be passed to the hooks and from them in files.
What are you trying to achieve is not possible due to the wsgi interface and the way the state is managed between requests.
I am trying to transfer from Cherrypy to Bottle & Gevent(server).
After I run:
application=bottle.default_app() #bottle
WSGIServer(('', port), application, spawn=None).serve_forever() #gevent
I want to restart the server just as if the reloader reloaded the server (but only when I tell the server to).
So I want to access a page with credential request and only after correct authentication will it restart.
Here is my functional example in Cherrypy:
#expose
def reloadMe(self, u=None, p=None):
if u=="username" and p=="password":
engine.restart()
raise HTTPRedirect('/')
More simply I am asking how do I reload this script so that my edits to the source file are implemented but only when I retrieve a "restart" page.
I literally only need the Bottlepy equivalent of
engine.restart() #cherrypy
Does no one know how to do this?
You can write a small shell script to restart gevent wsgi server.
then using this code, you can call the script.
#get('/restartmyserver')
def handler():
http_auth_data = bottle.request.auth() # returns a tuple (username,password) only basic auth.
if http_auth_data[0] == user and http_auth_data[1] == password:
os.system("your_shell_script_to_restart_gevent_wsgi")
bottle.redirect('/')
let me know if you need more info.
I've been working on a new dev platform using nginx/gunicorn and Flask for my application.
Ops-wise, everything works fine - the issue I'm having is with debugging the Flask layer. When there's an error in my code, I just get a straight 500 error returned to the browser and nothing shows up on the console or in my logs.
I've tried many different configs/options.. I guess I must be missing something obvious.
My gunicorn.conf:
import os
bind = '127.0.0.1:8002'
workers = 3
backlog = 2048
worker_class = "sync"
debug = True
proc_name = 'gunicorn.proc'
pidfile = '/tmp/gunicorn.pid'
logfile = '/var/log/gunicorn/debug.log'
loglevel = 'debug'
An example of some Flask code that borks- testserver.py:
from flask import Flask
from flask import render_template_string
from werkzeug.contrib.fixers import ProxyFix
app = Flask(__name__)
#app.route('/')
def index():
n = 1/0
return "DIV/0 worked!"
And finally, the command to run the flask app in gunicorn:
gunicorn -c gunicorn.conf.py testserver:app
Thanks y'all
The accepted solution doesn't work for me.
Gunicorn is a pre-forking environment and apparently the Flask debugger doesn't work in a forking environment.
Attention
Even though the interactive debugger does not work in
forking environments (which makes it nearly impossible to use on
production servers) [...]
Even if you set app.debug = True, you will still only get an empty page with the message Internal Server Error if you run with gunicorn testserver:app. The best you can do with gunicorn is to run it with gunicorn --debug testserver:app. That gives you the trace in addition to the Internal Server Error message. However, this is just the same text trace that you see in the terminal and not the Flask debugger.
Adding the if __name__ ... section to the testserver.py and running python testserver.py to start the server in development gets you the Flask debugger. In other words, don't use gunicorn in development if you want the Flask debugger.
app = Flask(__name__)
app.config['DEBUG'] = True
if __name__ == '__main__':
app.run()
## Tip for Heroku users:
Personally I still like to use `foreman start`, instead of `python testserver.py` since [it sets up all the env variables for me](https://devcenter.heroku.com/articles/config-vars#using-foreman). To get this to work:
Contents of Procfile
web: bin/web
Contents of bin/web, file is relative to project root
#!/bin/sh
if [ "$FLASK_ENV" == "development" ]; then
python app.py
else
gunicorn app:app -w 3
fi
In development, create a .env file relative to project root with the following contents (docs here)
FLASK_ENV=development
DEBUG=True
Also, don't forget to change the app.config['DEBUG']... line in testserver.py to something that won't run Flask in debug mode in production.
app.config['DEBUG'] = os.environ.get('DEBUG', False)
The Flask config is entirely separate from gunicorn's. Following the Flask documentation on config files, a good solution would be change my source to this:
app = Flask(__name__)
app.config.from_pyfile('config.py')
And in config.py:
DEBUG = True
For Heroku users, there is a simpler solution than creating a bin/web script like suggested by Nick.
Instead of foreman start, just use foreman run python app.py if you want to debug your application in development.
I had similiar problem when running flask under gunicorn I didn't see stacktraces in browser (had to look at logs every time). Setting DEBUG, FLASK_DEBUG, or anything mentioned on this page didn't work. Finally I did this:
app = Flask(__name__)
app.config.from_object(settings_map[environment])
if environment == 'development':
from werkzeug.debug import DebuggedApplication
app_runtime = DebuggedApplication(app, evalex=False)
else:
app_runtime = app
Note evalex is disabled because interactive debbugging won't work with forking (gunicorn).
I used this:
gunicorn "swagger_server.__main__:app" -w 4 -b 0.0.0.0:8080
You cannot really run it with gunicorn and for example use the flask reload option upon code changes.
I've used following snippets in my api launchpoint:
app = Flask(__name__)
try:
if os.environ["yourapp_environment"] == "local":
run_as_local = True
# some other local configs e.g. paths
app.logger.info('Running server in local development mode!')
except KeyError as err:
if "yourapp_environment" in err.args:
run_as_local = False
# some other production configs e.g. paths
app.logger.info('No "yourapp_environment env" given so app running server in production mode!')
else:
raise
...
...
...
if __name__ == '__main__':
if run_as_local:
app.run(host='127.0.0.1', port='8058', debug=True)
else:
app.run(host='0.0.0.0')
For above solution you need to give export yourapp_environment = "local" in the console.
now I can run my local as python api.py and prod gunicorn --bind 0.0.0.0:8058 api:app
The else statement app.run() is not actually needed, but I keep it for reminding me about host, port etc.
Try setting the debug flag on the run command like so
gunicorn -c gunicorn.conf.py --debug testserver:app
and keep the DEBUG = True in your Flask application. There must be a reason why your debug option is not being applied from the config file but for now the above note should get you going.