unable to load configuration from uwsgi - python

I have below setup in my Python application
server.py
from bots.flask_app import app
from bots.flask_app.api import api
from bots.flask_app.public import public
from bots import db
from bots.commons.helpers.flask.json.serializer import make_alternative_encoder
from flask_debugtoolbar import DebugToolbarExtension
import logging
import bots.commons.managers.configuration as ConfigurationManager
logger = logging.getLogger()
#######
# Public functions
#######
def setup_db_and_app():
# Flask application bootstrap
config = ConfigurationManager.get_flask_rest_config()
app.config.update(config)
logger.debug('Flask configuration object: %s', app.config)
# MongoDB connection initialization
db.init_app(app)
# Debug toolbar enabled only if Flask in debug mode
if ConfigurationManager.get_raw_flask_rest_config()['flask']['debug']:
DebugToolbarExtension(app)
# Replace the serializer with the custom one (for ObjectId and DateTime serialization)
app.json_encoder = make_alternative_encoder(app.json_encoder)
# Register the components
app.register_blueprint(api)
app.register_blueprint(public)
def start_server():
setup_db_and_app()
logger.debug('Registered routes: %s', app.url_map)
app.run(host='0.0.0.0')
main.py
import bots.flask_app.server as FlaskApp
import bots.commons.managers.log as LogManager
# Logging initialization
LogManager.init_logging()
# Defined in server.py
FlaskApp.start_server()
I am trying to see whether this applicator can bed served by uwsgi as below
uwsgi --socket 0.0.0.0:8080 --protocol=http -w main
The output is as follows
INFO:werkzeug: * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
INFO:werkzeug: * Restarting with stat
unable to load configuration from uwsgi
My questions
1. Where can I find the configurations which are causing this issue?
2. Can main.py be defined as a callable and used as a parameter for -w?
This is an app which is already written by someone and I am trying make this application served through uwsgi.
Any suggestions would be helpful
Thanks

I had the 'unable to load configuration from uwsgi' error too. According to flask uwsgi docs:
Please make sure in advance that any app.run() calls you might have in your application file are inside an if __name__ == '__main__': block or moved to a separate file. Just make sure it’s not called because this will always start a local WSGI server which we do not want if we deploy that application to uWSGI.
I move app.run() to if __name__ == '__main__':, and the problem solved. Maybe you can try to put FlaskApp.start_server() under if __name__ == '__main__':.

I'm a bit late to the party, I've encountered this error when I forgot to remove debug=True from app.run(). It makes sense that you can't run the debug server with uwsgi.

Related

Run flask application with uWSGI [duplicate]

This question already has answers here:
Run code after flask application has started
(7 answers)
Closed last year.
I have a flask application and I would like to run it in a "production" way using uwsgi.
I have my launcher.py:
from app import app
import db
if __name__ == "__main__":
db.init()
app.run()
If I run the application with simply python launcher.py it's all ok. Especially, the db.init() is called correctly.
However, if I run using uwgsi with uwsgi app.ini , db.init() is not called.
Here is app.ini:
[uwsgi]
wsgi-file = launcher.py
callable = app
socket = :8080
I'm new with flask and uwsgi so probably I missed something but I could not find the solution in the different tutorials I read.
Also, in case you need it to understand the project, here is app.py:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def index():
return "hello from flask"
if __name__ == "__main__":
db.init()
app.run()
All files are in the same level in my project:
webserver/
- app.ini
- launcher.py
- app.py
- db.py
So, what am I doing wrong here?
Your code under if __name__ == "__main__": is not executed because uwsgi does not run your script like python app.py. Instead it imports the module specified in wsgi-file and looks for an object specified as callable (app in our case). You can test it by using the following script:
from flask import Flask
app = Flask(__name__)
print(__name__)
if __name__ == '__main__':
app.run()
If you run it with python ./flask_app.py then the output will be
$ python ./flask_app.py
__main__
* Serving Flask app 'app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
(notice __main__ - it means we're running the script). However if you run it with uwsgi the __name__ will be app (it will be the name of your file, so app in my case):
$ uwsgi --http 127.0.0.1:5000 --module app:app
...
*** Operational MODE: single process ***
app
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x7fb2d0c067b0 pid: 46794 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 46794, cores: 1)
Similar output will be if you run your app with FLASK_APP=app flask run - it does not execute script, it just imports it and uses app object from it.
So, in order to initialize database you should either move your db initialization out of if __name__ == "__main__":
from flask import Flask
app = Flask(__name__)
class DB:
def init(self):
self.data = 'Hello, World'
db = DB()
#app.route("/")
def hello_world():
return db.data
db.init()
if __name__ == '__main__':
app.run()
Or add a before_first_request handler:
# replace db.init() with
#app.before_first_request
def init_db():
db.init()
Notice the difference between them - if you put db.init() as a top-level statement it will be executed once you load app.py. If you register a before_first_request callback it will be executed once first request arrives to your application. Pick the one that works best for you.

How do you customize the launching message for Flask?

I'm running a Flask app in Cloud9. Whenever I start my Flask app, it says this message:
* Serving Flask app "app" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://0.0.0.0:80/ (Press CTRL+C to quit)
Is there a way to change this message? I'd like it to say something like this:
Connect to me at http://0.0.0.0:80/!
I've searched stack overflow and the web but couldn't find anything. I'm starting my app with app.run().
Also, is it possible to make the URL cyan?
You can change everything besides Running on http://0.0.0.0:80/ (Press CTRL+C to quit) by changing show_server_banner of flask.cli:
from flask import Flask
import sys
cli = sys.modules['flask.cli']
# put your own message here
cli.show_server_banner = lambda *x: click.echo("My nice message")
app = Flask(__name__)
app.run(host='0.0.0.0', port='80')
To get rid of the Running on http://0.0.0.0:80/ ... message, you can use unittest.mock:
from unittest import mock
from werkzeug._internal import _log
def my_startup_log(*args):
# log all messages except for the * Running on message
if not args[1].startswith(" * Running on"):
return _log(*args)
app = Flask(__name__)
with mock.patch('werkzeug.serving._log') as mocked:
# patch the logger object and replace with own logger
mocked.side_effect = my_startup_logger
app.run(host='0.0.0.0', port='8000')
This is very hacky and depends on the internal implementation of flask. Be careful when using this in production code, as this could easily break.

How to enable logging of Flask app with `gevent.pywsgi.WSGIServer` and `WebSocketHandler`?

The issue should be reproducible with the following two minimal examples:
Minimal example with app.run()
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello():
return 'Hello'
app.run()
Minimal example with gevent.pywsgi.WSGIServer
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello():
return 'Hello'
from gevent import pywsgi
from geventwebsocket.handler import WebSocketHandler
server = pywsgi.WSGIServer(('127.0.0.1', 5000), app, handler_class=WebSocketHandler)
server.serve_forever()
The first 5 lines are identical, so both examples only differ in the way they start the server. Both servers do work, I get "Hello" in the browser. The first example prints:
* Serving Flask app "1" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [20/Jun/2019 23:43:15] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [20/Jun/2019 23:43:19] "GET / HTTP/1.1" 200 -
So the console output shows some information about every request which is handled by the server.
However, with the second example, I don't get any logging output in the console anymore. How do I enable logging for gevent.pywsgi.WSGIServer and WebSocketHandler?
Background (which doesn't matter regarding the issue, I think)
I'm running a Flask app which uses flask_sockets. Because
Werkzeug development server cannot provide the WSGI environ with a websocket interface
I am not able to use the server with app.run() and I'm using gevent.pywsgi.WSGIServer instead. The code of my example above is taken directly from the module's examples at https://github.com/heroku-python/flask-sockets without any modifications.
Logging works differently with gevent.pywsgi.WSGIServer. It uses python logging and is much more sophisticated and flexible.
Here's an example:
from gevent import pywsgi
from geventwebsocket.handler import WebSocketHandler
import logging
logging.basicConfig(level=logging.INFO)
server = pywsgi.WSGIServer(('127.0.0.1', 5000), app, handler_class=WebSocketHandler)
server.serve_forever()
And now wherever you want output, you can do:
logging.info("You can see me now...")
If you want to see startup info from WSGIServer, then set the log level to DEBUG and you can see tons of output.
It should work like that, but it's a bug in WebSocketHandler from the geventwebsocket module which has already been reported: https://gitlab.com/noppo/gevent-websocket/issues/16

Importing CherryPy fails on openshift

I'm running a cherrypy based app on an openshift gear. Recently I've been getting a "503 service temporarily unavailable" error whenever I try to go to the site. Inspecting the logs, I see I'm getting an ImportError where I try to import CherryPy. This is strange - CherryPy is listed as a dependency in my requirements.txt and used to be imported just fine. I double checked to make sure I'm getting the right path to the openshift activate_this.py and it seems to be correct. I'm not quite sure where to look next; any help would be appreciated. Thanks!
The failed import is at line 14 of app.py:
import os
import files
virtenv = os.path.join(os.environ['OPENSHIFT_PYTHON_DIR'], 'virtenv')
virtualenv = os.path.join(virtenv, 'bin', 'activate_this.py')
conf = os.path.join(files.get_root(), "conf", "server.conf")
try:
execfile(virtualenv, dict(__file__=virtualenv))
print virtualenv
except IOError:
pass
import cherrypy
import wsgi
def mount():
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = os.environ['OPENSHIFT_APP_DNS']
cherrypy.config.update({"tools.staticdir.root": files.get_root()})
cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
cherrypy.tree.mount(wsgi.application(), "/", conf)
def start():
cherrypy.engine.start()
def end():
cherrypy.engine.exit()
if __name__ == "__main__":
mount()
start()
UPDATE
I eventually saw (when pushing to the openshift repo using git bash CLI) that the dependency installation from requirements.txt was failing with some exceptions I haven't bothered to look into yet. It then goes on to try to install dependencies in setup.py, and that works just fine.
Regarding the port in use issue...I have no idea. I changed my startup from tree.mount and engine.start to quickstart, and everything worked when I pushed to openshift. Just for kicks (and because I need it to run my tests), I switched back to cherrypy.tree.mount, pushed it, and it worked just fine.
Go figure.
I use the app.py entry point for Openshift. Here are several examples on how I start my server using the pyramid framework on Openshift. I use waitress as the server but I have also used the cherrypy wsgi server. Just comment out the code you don't want.
app.py
#Openshift entry point
import os
from pyramid.paster import get_app
from pyramid.paster import get_appsettings
if __name__ == '__main__':
here = os.path.dirname(os.path.abspath(__file__))
if 'OPENSHIFT_APP_NAME' in os.environ: #are we on OPENSHIFT?
ip = os.environ['OPENSHIFT_PYTHON_IP']
port = int(os.environ['OPENSHIFT_PYTHON_PORT'])
config = os.path.join(here, 'production.ini')
else:
ip = '0.0.0.0' #localhost
port = 6543
config = os.path.join(here, 'development.ini')
app = get_app(config, 'main') #find 'main' method in __init__.py. That is our wsgi app
settings = get_appsettings(config, 'main') #don't really need this but is an example on how to get settings from the '.ini' files
# Waitress (remember to include the waitress server in "install_requires" in the setup.py)
from waitress import serve
print("Starting Waitress.")
serve(app, host=ip, port=port, threads=50)
# Cherrypy server (remember to include the cherrypy server in "install_requires" in the setup.py)
# from cherrypy import wsgiserver
# print("Starting Cherrypy Server on http://{0}:{1}".format(ip, port))
# server = wsgiserver.CherryPyWSGIServer((ip, port), app, server_name='Server')
# server.start()
#Simple Server
# from wsgiref.simple_server import make_server
# print("Starting Simple Server on http://{0}:{1}".format(ip, port))
# server = make_server(ip, port, app)
# server.serve_forever()
#Running 'production.ini' method manually. I find this method the least compatible with Openshift since you can't
#easily start/stop/restart your app with the 'rhc' commands. Mabye somebody can suggest a better way :)
# #Don't forget to set the Host IP in 'production.ini'. Use 8080 for the port for Openshift
# You will need to use the 'pre_build' action hook(pkill python) so it stops the existing running instance of the server on OS
# You also will have to set up another custom action hook so rhc app-restart, stop works.
# See Openshifts Origin User's Guide ( I have not tried this yet)
#Method #1
# print('Running pserve production.ini')
# os.system("pserve production.ini &")
#Method #2
#import subprocess
#subprocess.Popen(['pserve', 'production.ini &'])

Debugging a Flask app running in Gunicorn

I've been working on a new dev platform using nginx/gunicorn and Flask for my application.
Ops-wise, everything works fine - the issue I'm having is with debugging the Flask layer. When there's an error in my code, I just get a straight 500 error returned to the browser and nothing shows up on the console or in my logs.
I've tried many different configs/options.. I guess I must be missing something obvious.
My gunicorn.conf:
import os
bind = '127.0.0.1:8002'
workers = 3
backlog = 2048
worker_class = "sync"
debug = True
proc_name = 'gunicorn.proc'
pidfile = '/tmp/gunicorn.pid'
logfile = '/var/log/gunicorn/debug.log'
loglevel = 'debug'
An example of some Flask code that borks- testserver.py:
from flask import Flask
from flask import render_template_string
from werkzeug.contrib.fixers import ProxyFix
app = Flask(__name__)
#app.route('/')
def index():
n = 1/0
return "DIV/0 worked!"
And finally, the command to run the flask app in gunicorn:
gunicorn -c gunicorn.conf.py testserver:app
Thanks y'all
The accepted solution doesn't work for me.
Gunicorn is a pre-forking environment and apparently the Flask debugger doesn't work in a forking environment.
Attention
Even though the interactive debugger does not work in
forking environments (which makes it nearly impossible to use on
production servers) [...]
Even if you set app.debug = True, you will still only get an empty page with the message Internal Server Error if you run with gunicorn testserver:app. The best you can do with gunicorn is to run it with gunicorn --debug testserver:app. That gives you the trace in addition to the Internal Server Error message. However, this is just the same text trace that you see in the terminal and not the Flask debugger.
Adding the if __name__ ... section to the testserver.py and running python testserver.py to start the server in development gets you the Flask debugger. In other words, don't use gunicorn in development if you want the Flask debugger.
app = Flask(__name__)
app.config['DEBUG'] = True
if __name__ == '__main__':
app.run()
## Tip for Heroku users:
Personally I still like to use `foreman start`, instead of `python testserver.py` since [it sets up all the env variables for me](https://devcenter.heroku.com/articles/config-vars#using-foreman). To get this to work:
Contents of Procfile
web: bin/web
Contents of bin/web, file is relative to project root
#!/bin/sh
if [ "$FLASK_ENV" == "development" ]; then
python app.py
else
gunicorn app:app -w 3
fi
In development, create a .env file relative to project root with the following contents (docs here)
FLASK_ENV=development
DEBUG=True
Also, don't forget to change the app.config['DEBUG']... line in testserver.py to something that won't run Flask in debug mode in production.
app.config['DEBUG'] = os.environ.get('DEBUG', False)
The Flask config is entirely separate from gunicorn's. Following the Flask documentation on config files, a good solution would be change my source to this:
app = Flask(__name__)
app.config.from_pyfile('config.py')
And in config.py:
DEBUG = True
For Heroku users, there is a simpler solution than creating a bin/web script like suggested by Nick.
Instead of foreman start, just use foreman run python app.py if you want to debug your application in development.
I had similiar problem when running flask under gunicorn I didn't see stacktraces in browser (had to look at logs every time). Setting DEBUG, FLASK_DEBUG, or anything mentioned on this page didn't work. Finally I did this:
app = Flask(__name__)
app.config.from_object(settings_map[environment])
if environment == 'development':
from werkzeug.debug import DebuggedApplication
app_runtime = DebuggedApplication(app, evalex=False)
else:
app_runtime = app
Note evalex is disabled because interactive debbugging won't work with forking (gunicorn).
I used this:
gunicorn "swagger_server.__main__:app" -w 4 -b 0.0.0.0:8080
You cannot really run it with gunicorn and for example use the flask reload option upon code changes.
I've used following snippets in my api launchpoint:
app = Flask(__name__)
try:
if os.environ["yourapp_environment"] == "local":
run_as_local = True
# some other local configs e.g. paths
app.logger.info('Running server in local development mode!')
except KeyError as err:
if "yourapp_environment" in err.args:
run_as_local = False
# some other production configs e.g. paths
app.logger.info('No "yourapp_environment env" given so app running server in production mode!')
else:
raise
...
...
...
if __name__ == '__main__':
if run_as_local:
app.run(host='127.0.0.1', port='8058', debug=True)
else:
app.run(host='0.0.0.0')
For above solution you need to give export yourapp_environment = "local" in the console.
now I can run my local as python api.py and prod gunicorn --bind 0.0.0.0:8058 api:app
The else statement app.run() is not actually needed, but I keep it for reminding me about host, port etc.
Try setting the debug flag on the run command like so
gunicorn -c gunicorn.conf.py --debug testserver:app
and keep the DEBUG = True in your Flask application. There must be a reason why your debug option is not being applied from the config file but for now the above note should get you going.

Categories