In a Flask application, I would like to configure two differents file loggers:
One for HTTP access logs (access.log), which will log stuff like:
1.2.3.4 - [11/Jun/2018 09:51:13] "GET /some/path HTTP/1.1" 200 -
One for application logs (my_app.log), which will keep logs defined by me in my code when I'm using current_app.logger.info('some message'):
2018-06-08 15:08:50,083 - flask.app - INFO - some message
How should my configuration looks like to achieve this ? Here is what I tried, without success:
# content of "run.py" :
app = Flask(__name__)
app.logger.removeHandler(default_handler)
# Define 'my_app.log' :
handler = logging.FileHandler('my_app.log')
handler.setLevel(logging.INFO)
formatter = logging.Formatter(app.config['LOGGING_FORMAT'])
handler.setFormatter(formatter)
app.logger.addHandler(handler)
# Define 'access.log' :
access_handler = logging.getLogger('werkzeug')
access_handler = logging.FileHandler('access.log')
access_handler.setLevel(logging.DEBUG)
access_handler.setFormatter(app.config['LOGGING_FORMAT'])
app.logger.addHandler(access_handler)
# Then register my blueprints:
app.register_blueprint(some_blueprint, url_prefix='/')
....
And I run it with python3 run.py. With this config, the only logged stuff are the HTTP access logs in the my_app.log file.
What's wrong with my config ?
I'm building a python application with also a web interface, with Flask web framework.
It runs on Flask internal server in debug/dev mode and in production mode it runs on tornado as wsgi container.
This is how i've set up my logger:
log_formatter = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s')
file_handler = logging.handlers.RotatingFileHandler(LOG_FILE, maxBytes=5 * 1024 * 1024, backupCount=10)
file_handler.setFormatter(log_formatter)
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setFormatter(log_formatter)
log = logging.getLogger('myAppLogger')
log.addHandler(file_handler)
log.addHandler(console_handler)
To add my logger to the Flask app i tried this:
app = Flask('system.web.server')
app.logger_name = 'myAppLogger'
But the log still going to the Flask default log handler, and in addition, I didn't found how to customize the log handlers also for the Tornado web server.
Any help is much appreciated,
thanks in advance
AFAIK, you can't change the default logger in Flask. You can, however, add your handlers to the default logger:
app = Flask('system.web.server')
app.logger.addHandler(file_handler)
app.logger.addHandler(console_handler)
Regarding my comment above - "Why would you want to run Flask in tornado ...", ignore that. If you are not seeing any performance hit, then clearly there's no need to change your setup.
If, however, in future you'd like to migrate to a multithreaded container, you can look into uwsgi or gunicorn.
I managed to do it, multiple handler, each doing their thing, so that ERROR log will not show on the INFO log as well and end up with duplicate info grrr:
app.py
import logging
from logging.handlers import RotatingFileHandler
app = Flask(__name__)
# Set format that both loggers will use:
formatter = logging.Formatter("[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s")
# Set logger A for known errors
log_handler = RotatingFileHandler('errors.log', maxBytes=10000, backupCount=1)
log_handler.setFormatter(formatter)
log_handler.setLevel(logging.INFO)
a = logging.getLogger('errors')
a.addHandler(log_handler)
# Set logger B for bad exceptions
exceptions_handler = RotatingFileHandler('exceptions.log', maxBytes=10000, backupCount=1)
exceptions_handler.setFormatter(formatter)
exceptions_handler.setLevel(logging.ERROR)
b = logging.getLogger('exceptions')
b.addHandler(exceptions_handler)
...
whatever_file_where_you_want_to_log.py
import logging
import traceback
# Will output known error messages to 'errors.log'
logging.getLogger('errors').error("Cannot connect to database, timeout")
# Will output the full stack trace to 'exceptions.log', when trouble hits the fan
logging.getLogger('exceptions').error(traceback.format_exc())
How can I view log messages on Google Cloud?: https://console.cloud.google.com/logs
This is what I see in the terminal when I run dev_appserver.py (locally running):
INFO 2016-05-16 14:00:45,118 module.py:787] default: "GET /static/images/contact.png HTTP/1.1" 304 -
INFO 2016-05-16 14:00:45,128 module.py:787] default: "GET /static/images/email.png HTTP/1.1" 304 -
INFO 2016-05-16 14:00:45,136 module.py:787] default: "GET /static/images/phone.png HTTP/1.1" 304 -
INFO 2016-05-16 14:00:45,487 basehandler.py:19] entering basehandler.py
INFO 2016-05-16 14:00:45,516 module.py:787] default: "GET /static/images/logo-349x209.png HTTP/1.1" 304 -
INFO 2016-05-16 14:00:45,562 requesthandlers.py:26] entering requesthandlers.py
INFO 2016-05-16 14:00:45,563 app.py:28] entering app.py
INFO 2016-05-16 14:00:45,563 app.py:198] Using development database
Both application log messages and request logging is displayed.
However when I view the log of the same code deployed I can only see the requests being logged:
The code I'm using to generate application log messages is something like:
import logging
logger = logging.getLogger("someLogger")
logger.info("entering app.py")
But I've also tried using logging.info(...) directly with the same results.
I've tried finding an answer to this in various resources but I've come up empty-handed, most refer to how to set log level when developing locally.
I'm guessing that I need to enable some setting in order to view application logs on Google Cloud Logs.
Resources that I've looked at:
https://cloud.google.com/logging/docs/view/logs_viewer
https://cloud.google.com/appengine/docs/python/logs/
How to change the logging level of dev_appserver
How do I write to the console in Google App Engine?
Google App Engine - Can not find my logging messages
https://docs.python.org/3/howto/logging.html
App engine groups the logs by request. You need to expand the log using the triangle/pointer on the left of the request in the 'new' GAE log viewer.
Personally I prefer using the old GAE log viewer, but I am unsure how much longer it will be around:
https://appengine.google.com/logs?app_id=s~xxx
(This viewer shows request + logs and allows log expansion)
An easy way to integrate Google Cloud Platform logging into your Python code is to create a subclass from logging.StreamHandler. This way logging levels will also match those of Google Cloud Logging, enabling you to filter based on severity. This solution also works within Cloud Run containers.
Also you can just add this handler to any existing logger configuration, without needing to change current logging code.
import json
import logging
import os
import sys
from logging import StreamHandler
from flask import request
class GoogleCloudHandler(StreamHandler):
def __init__(self):
StreamHandler.__init__(self)
def emit(self, record):
msg = self.format(record)
# Get project_id from Cloud Run environment
project = os.environ.get('GOOGLE_CLOUD_PROJECT')
# Build structured log messages as an object.
global_log_fields = {}
trace_header = request.headers.get('X-Cloud-Trace-Context')
if trace_header and project:
trace = trace_header.split('/')
global_log_fields['logging.googleapis.com/trace'] = (
f"projects/{project}/traces/{trace[0]}")
# Complete a structured log entry.
entry = dict(severity=record.levelname, message=msg)
print(json.dumps(entry))
sys.stdout.flush()
A way to configure and use the handler could be:
def get_logger():
logger = logging.getLogger(__name__)
if not logger.handlers:
gcp_handler = GoogleCloudHandler()
gcp_handler.setLevel(logging.DEBUG)
gcp_formatter = logging.Formatter(
'%(levelname)s %(asctime)s [%(filename)s:%(funcName)s:%(lineno)d] %(message)s')
gcp_handler.setFormatter(gcp_formatter)
logger.addHandler(gcp_handler)
return logger
I have a Flask server running in standalone mode (using app.run()). But, I don't want any messages in the console, like
127.0.0.1 - - [15/Feb/2013 10:52:22] "GET /index.html HTTP/1.1" 200 -
...
How do I disable verbose mode?
You can set level of the Werkzeug logger to ERROR, in that case only errors are logged:
import logging
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
Here is a fully working example tested on OSX, Python 2.7.5, Flask 0.10.0:
from flask import Flask
app = Flask(__name__)
import logging
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
#app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run()
This solution provides you a way to get your own prints and stack traces but without information level logs from flask suck as 127.0.0.1 - - [15/Feb/2013 10:52:22] "GET /index.html HTTP/1.1" 200
from flask import Flask
import logging
app = Flask(__name__)
log = logging.getLogger('werkzeug')
log.disabled = True
None of the other answers worked correctly for me, but I found a solution based off Peter's comment. Flask apparently no longer uses logging for logging, and has switched to the click package. By overriding click.echo and click.secho I eliminated Flask's startup message from app.run().
import logging
import click
from flask import Flask
app = Flask(__name__)
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
def secho(text, file=None, nl=None, err=None, color=None, **styles):
pass
def echo(text, file=None, nl=None, err=None, color=None, **styles):
pass
click.echo = echo
click.secho = secho
#app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run()
Between setting the logging level to ERROR and overriding the click methods with empty functions, all non-error log output should be prevented.
To suppress Serving Flask app ...:
os.environ['WERKZEUG_RUN_MAIN'] = 'true'
app.run()
In case you are using WSGI server , please set the log to None
gevent_server = gevent.pywsgi.WSGIServer(("0.0.0.0", 8080), app, log=None)
#Drewes solution works most of the time, but in some cases, I still tend to get werkzeug logs. If you really don't want to see any of them, I suggest you disabling it like that.
from flask import Flask
import logging
app = Flask(__name__)
log = logging.getLogger('werkzeug')
log.disabled = True
app.logger.disabled = True
For me it failed when abort(500) was raised.
Late answer but I found a way to suppress EACH AND EVERY CONSOLE MESSAGE (including the ones displayed during an abort(...) error).
import os
import logging
logging.getLogger('werkzeug').disabled = True
os.environ['WERKZEUG_RUN_MAIN'] = 'true'
This is basically a combination of the answers given by Slava V and Tom Wojcik
Another reason you may want to change the logging output is for tests, and redirect the server logs to a log file.
I couldn't get the suggestion above to work either, it looks like loggers are setup as part of the app starting. I was able to get it working by changing the log levels after starting the app:
... (in setUpClass)
server = Thread(target=lambda: app.run(host=hostname, port=port, threaded=True))
server.daemon = True
server.start()
wait_for_boot(hostname, port) # curls a health check endpoint
log_names = ['werkzeug']
app_logs = map(lambda logname: logging.getLogger(logname), log_names)
file_handler = logging.FileHandler('log/app.test.log', 'w')
for app_log in app_logs:
for hdlr in app_log.handlers[:]: # remove all old handlers
app_log.removeHandler(hdlr)
app_log.addHandler(file_handler)
Unfortunately the * Running on localhost:9151 and the first health check is still printed to standard out, but when running lots of tests it cleans up the output a ton.
"So why log_names?", you ask. In my case there were some extra logs I needed to get rid of. I was able to find which loggers to add to log_names via:
from flask import Flask
app = Flask(__name__)
import logging
print(logging.Logger.manager.loggerDict)
Side note: It would be nice if there was a flaskapp.getLogger() or something so this was more robust across versions. Any ideas?
Some more key words: flask test log remove stdout output
thanks to:
http://code.activestate.com/lists/python-list/621740/ and
How to change filehandle with Python logging on the fly with different classes and imports
I spent absolute ages trying to get rid of these response logs with all the different solutions, but as it turns out it wasn't Flask / Werkzeug but Gunicorn access logs dumped on stderr...
The solution was replacing the default access log handler with NullHandler by adding this block in the Gunicorn config file:
logconfig_dict = {
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"console": {"class": "logging.StreamHandler", "level": "INFO"},
"null": {"class": "logging.NullHandler"},
},
"loggers": {
"gunicorn.error": {"level": "INFO", "propagate": False, "handlers": ["console"]},
"gunicorn.access": {"level": "INFO", "propagate": False, "handlers": ["null"]},
},
}
somehow none of the above options, including .disabled = True, worked for me.
The following did the trick though:
logging.getLogger('werkzeug').setLevel(logging.CRITICAL)
Using the latest versions as of November 2021 under Python 3.7.3:
pip3 list | grep -E "(connexion|Flask|Werkzeug)"
connexion2 2.10.0
Flask 2.0.2
Werkzeug 2.0.2
Here's the answer to disable all logging, including on werkzeug version 2.1.0 and newer:
import flask.cli
flask.cli.show_server_banner = lambda *args: None
import logging
logging.getLogger("werkzeug").disabled = True
My current workaround to silence Flask as of 2022-10.
It disables the logging and the startup banner:
class WSServer:
...
#staticmethod
def _disabled_server_banner(*args, **kwargs):
"""
Mock for the show_server_banner method to suppress spam to stdout
"""
pass
def _run_server(self):
"""
Winds-up the server.
"""
# disable general logging
log = logging.getLogger('werkzeug')
log.setLevel(logging.CRITICAL)
# disable start-up banner
from flask import cli
if hasattr(cli, "show_server_banner"):
cli.show_server_banner = self._disabled_server_banner
...
A brute force way to do it if you really don't want anything to log into the console beside print() statements is to logging.basicConfig(level=logging.FATAL). This would disable all logs that are of status under fatal. It would not disable printing but yeah, just a thought :/
EDIT:
I realized it would be selfish of me not to put a link to the documentation I used :)
https://docs.python.org/3/howto/logging.html#logging-basic-tutorial
The first point: In according to official Flask documentation, you shouldn't run Flask application using app.run(). The best solution is using uwsgi, so you can disable default flask logs using command "--disable-logging"
For example:
uwsgi --socket 0.0.0.0:8001 --disable-logging --protocol=http -w app:app
I have a pretty weird problem with my logging facility i use inside django/python. The logging is not working anymore since i upgraded to django 1.3. It seems to be related to the logging level and the 'debug=' setting in the settings.py file.
1) When i log INFO messages and debug=False, the logging won't happen, my file doesn't get appended.
2) When i log WARNING messages and debug=False, the logging works perfectly like i want it to, the file gets appended
3) When i log INFO messages and debug=True, the logging seems to work, the file get appended.
How could i log INFO messages with debug=False? It worked before django 1.3... is there somewhere a mysterious setting which do the trick? Underneath there is a sample code:
views.py:
import logging
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(levelname)s %(message)s',
filename='/opt/general.log',
filemode='a')
def create_log_file(filename, log_name, level=logging.INFO):
handler = logging.handlers.TimedRotatingFileHandler(filename, 'midnight', 7, backupCount=10)
handler.setLevel(level)
formatter = logging.Formatter('%(asctime)s %(name)-12s %(levelname)-8s %(message)s', '%a, %Y-%m-%d %H:%M:%S')
handler.setFormatter(formatter)
logging.getLogger(log_name).addHandler(handler)
create_log_file('/opt/test.log', 'testlog')
logger_test = logging.getLogger('testlog')
logger_test.info('testing the logging functionality')
With this code the logging does not work in Django 1.3 with debug set to False in the settings.py file. When i should do like this:
logger_test.warning('testing the logging functionality')
This works perfectly when debug is set to False. The levels DEBUG and INFO aint logging but WARNING,ERROR and CRITICAL are doing their job...
Does anyone have an idea?
Since Django 1.3 contains its own logging configuration, you need to ensure that anything you're doing doesn't clash with it. For example, if the root logger has handlers already configured by Django by the time your module first gets imported, your basicConfig() call won't have any effect.
What you're describing is the normal logging situation - WARNINGs and above get handled, while INFO and DEBUG are suppressed by default. It looks as if your basicConfig() is not having any effect; You should consider replacing your basicConfig() call with the appropriate logging configuration in settings.py, or at least investigate the root logging level and what handlers are attached to it, at the time of your basicConfig() call.