paster serve --reload development.ini
..for debug = true
THis is what I do to load a development server for Pylons.
However, when I do:
print "hello world"
THis message doesn't print out in the console. In Django, it does.
In Pylons logging package is the method to perform logging:
import logging
log = logging.getLogger(__name__)
log.debug('hello world')
This will work as long as you have logging setup configured correctly in your development.ini. I think the code above should be sufficient without any modifications to default configuration. In case it isn't you can call log.info, log.warn, log.error or log.critical instead of log.debug for your message to pass through.
I highly recommend reading this chapter of Pylons Book.
Related
I'm building a website using Flask and I'm now in the process of adding some logging to it for which I found these docs. The basic example is as follows:
if not app.debug:
import logging
from themodule import TheHandlerYouWant
file_handler = TheHandlerYouWant(...)
file_handler.setLevel(logging.WARNING)
app.logger.addHandler(file_handler)
after which you can log using app.logger.error('An error occurred'). This works fine, but apart from the fact that I do not see any advantage over the regular python logging module I also see a major downside: if I want to log outside of a request context (when for example running some code with a cron job) I get errors because I'm using app outside of the request context.
So my main question; why would I use the Flask logger at all? What is the reason that it was ever built?
The Flask logger uses the "generic" Python logging, it's a logging.getLogger(name) like any other.
The logger exists so that Flask app and views can log things that happen during execution. For example, it will log tracebacks on 500 errors during debug mode. The configuration example is there to show how to enable these logs, which are still useful in production, when you are not in debug mode.
Having an internal logger is not unique to Flask, it's the standard way to use logging in Python. Every module defines it's own logger, but the configuration of the logging is only handled by the last link in the chain: your project.
You can also use app.logger for your own messages, although it's not required. You could also create a separate logger for your own messages. In the end, it's all Python logging.
probably I don't quite understand how logging really works in Python. I'm trying to debug a Flask+SQLAlchemy (but without flask_sqlalchemy) app which mysteriously hangs on some queries only if run from within Apache, so I need to have proper logging to get meaningful information. The Flask application by default comes with a nice logger+handler, but how do I get SQLAlchemy to use the same logger?
The "Configuring Logging" section in the SQLAlchemy just explains how to turn on logging in general, but not how to "connect" SQLAlchemy's logging output to an already existing logger.
I've been looking at Flask + sqlalchemy advanced logging for a while with a blank, expressionless face. I have no idea if the answer to my question is even in there.
EDIT: Thanks to the answer given I now know that I can have two loggers use the same handler. Now of course my apache error log is littered with hundreds of lines of echoed SQL calls. I'd like to log only error messages to the httpd log and divert all lower-level stuff to a separate logfile. See the code below. However, I still get every debug message into the http log. Why?
if app.config['DEBUG']:
# Make logger accept all log levels
app.logger.setLevel(logging.DEBUG)
for h in app.logger.handlers:
# restrict logging to /var/log/httpd/error_log to errors only
h.setLevel(logging.ERROR)
if app.config['LOGFILE']:
# configure debug logging only if logfile is set
debug_handler = logging.FileHandler(app.config['LOGFILE'])
debug_handler.setLevel(logging.DEBUG)
app.logger.addHandler(debug_handler)
# get logger for SQLAlchemy
sq_log = logging.getLogger('sqlalchemy.engine')
sq_log.setLevel(logging.DEBUG)
# remove any preconfigured handlers there might be
for h in sq_log.handlers:
sq_log.removeHandler(h)
h.close()
# Now, SQLAlchemy should not have any handlers at all. Let's add one
# for the logfile
sq_log.addHandler(debug_handler)
You cannot make SQLAlchemy and Flask use the same logger, but you can make them writing to one place by add a common Handler. And maybe this article is helpful: https://www.electricmonk.nl/log/2017/08/06/understanding-pythons-logging-module/
By the way, if you want to get all logs in one single request, you can set a uniq name for current thread before request, and add the threadName in you logging's formatter.
Answer to my question at EDIT: I still had "echo=True" set on the create_engine, so what I saw was all the additional output on stderr. echo=False stops that but still logs to debug level DEBUG.
Clear all corresponding handlers created by SqlAlchemy:
logging.getLogger("sqlalchemy.engine.Engine").handlers.clear()
The code above should be called after engine created.
Is there a way to log waitress-serve output into a file?
The current command I use is:
waitress-serve --listen=localhost:8080 --threads=1 my_app_api:app
The application we used was not written with waitress in mind earlier, so we choose to serve it with command line to avoid change (for now at least).
TLDR waitress-serve doesn't provide a way to do it. See the 'how do i get it to log' section.
Background
Per the documentation for the command-line usage of waitress-serve, no - there's no way to setup logging. See arguments docs.
waitress-serve is just an executable to make running your server more convenient. It's source-code is here runner.py. If you read it, you can see it actually is basically just calling from waitress import serve; serve(**args) for you. (That code clip is not literally what it's doing, but in spirit yes).
The documentation for waitress says that it doesn't log http traffic. That's not it's job. But it will log it's own errors or stacktraces. logging docs. If you read the waitress source trying to find when it logs stuff, you'll notice it doesn't seem to log http traffic anywhere github log search. It primarily logs stuff to do with the socket layer.
Waitress does say that if you want to log http traffic, then you need another component. In particular, it points you to pastedeploy docs which is some middle-ware that can log http traffic for you.
The documentation from waitress is actually kind of helpful answering you question, though not direct and explicit. It says
The WSGI design is modular.
per the logging doc
I.e. waitress won't log http traffic for you. You'll need another WSGI component to do that, and because WSGI is modular, you can probably choose a few things.
If you want some background on how this works, there's a pretty good post here leftasexercise.com
OK, how do I get it to log?
Use tee
Basically, if you just want to log the same stuff that is output from waitress-serve then you don't need anything special.
waitress-serve --listen=localhost:8080 --threads=1 my_app_api:app | tee -a waitress-serve.log
Python logging
But if you're actually looking for logging coming from python's standard logger (say you app is making logger calls or you want to log http traffic) then, you can set that up in your python application code. E.g. edit your applications soure-code and get it to setup logging to a file
import logging
logging.basicConfig(filename='app.log', encoding='utf-8', level=logging.DEBUG)
PasteDeploy middleware for http logs
Or if your looking for apache type http logging then you can use something like PasteDeploy to do it. Note, PasteDeploy is another python dependency so you'll need to install it. E.g.
pip install PasteDeploy
Then you need to setup a .ini file that tells PasteDeploy how to start your server and then also tell it to use TransLogger to create apache type http logs. This is explained more detail here logging with pastedeploy The ini file is specific to each app, but from your question is sounds like the ini file should look like:
[app:wsgiapp]
use = my_app_api:app
[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = 8080
[filter:translogger]
use = egg:Paste#translogger
setup_console_handler = False
[pipeline:main]
pipeline = translogger
app
You'll still need to edit the source-code of your app to get PasteDeploy to load the app with your configuration file:
from paste.deploy import loadapp
wsgi_app = loadapp('config:/path/to/config.ini')
Webframework-dependent roll-your-own http logging
Even if you want to log http traffic, you don't necessarily need something like PasteDeploy. For example, if you are using flask as the web-framework, you can write your own http logs using after_request decorator:
#app.after_request
def after_request(response):
timestamp = strftime('[%Y-%b-%d %H:%M]')
logger.error('%s %s %s %s %s %s', timestamp, request.remote_addr, request.method, request.scheme, request.full_path, response.status)
return response
See the full gist at https://gist.github.com/alexaleluia12/e40f1dfa4ce598c2e958611f67d28966
I'm building a website using Flask and I'm now in the process of adding some logging to it for which I found these docs. The basic example is as follows:
if not app.debug:
import logging
from themodule import TheHandlerYouWant
file_handler = TheHandlerYouWant(...)
file_handler.setLevel(logging.WARNING)
app.logger.addHandler(file_handler)
after which you can log using app.logger.error('An error occurred'). This works fine, but apart from the fact that I do not see any advantage over the regular python logging module I also see a major downside: if I want to log outside of a request context (when for example running some code with a cron job) I get errors because I'm using app outside of the request context.
So my main question; why would I use the Flask logger at all? What is the reason that it was ever built?
The Flask logger uses the "generic" Python logging, it's a logging.getLogger(name) like any other.
The logger exists so that Flask app and views can log things that happen during execution. For example, it will log tracebacks on 500 errors during debug mode. The configuration example is there to show how to enable these logs, which are still useful in production, when you are not in debug mode.
Having an internal logger is not unique to Flask, it's the standard way to use logging in Python. Every module defines it's own logger, but the configuration of the logging is only handled by the last link in the chain: your project.
You can also use app.logger for your own messages, although it's not required. You could also create a separate logger for your own messages. In the end, it's all Python logging.
I'm running my app on the GAE development server, with app-engine-patch to run Django.
One of my views is bugged , so I want to log everything that happens.
I added in myapp.views:
import logging
LOG_FILENAME = '/mylog.txt'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)
and my function is:
def function(string):
logging.debug('new call')
#do stuff
#logging.debug('log stuff')
My problem is that I can't find the log. When I run my app I get no errors, but the log is not created.
I also tried various paths: /mylog.txt ,mylog.txt , c:\mylog.txt, c:\complete\path\to \my\app\mylog.txt, but it doesn't work.
On the other hand I tried to create and run a separate test:
#test.py
import logging
LOG_FILENAME = '/mylog.txt'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG)
logging.debug('test')
And the log is created without problems: c:\mylog.txt
I'm not familiar with logging so I don't know if there might be some issues with django, or appengine.
Thanks
You can't write to files on App Engine - thus, any attempt to log to text files is also doomed to failure. Log output will appear on the SDK console in development, or in the logs console in production.
I am guessing the problem is that you put your log configuration in a view. A general rule of thumb for django logging is to set the log configuration in your settings.py.
By default, dev_appserver suppresses the debug log. To turn it on, run it with the --debug option. Note that dev_appserver itself uses the same logger, and will spew lots of debugging stuff you might not care about.
Your logging will print to stderr.