I have the following code for a webserver. I have a problem that nothing is printed in httpd.log- and I'm clueless why. The file gets created and if I add a print()statement to log_message() it is printed out, but nothing ever is written to the file.
from http.server import HTTPServer
from http.server import SimpleHTTPRequestHandler
import logging
from logging.handlers import RotatingFileHandler
class MyHandler(SimpleHTTPRequestHandler):
def __init__(self, *args):
self.logger = logging.getLogger("httpd")
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh = RotatingFileHandler("httpd.log", mode='wa',
maxBytes=1 << 20, backupCount=2)
fh.setFormatter(formatter)
fh.setLevel(logging.INFO)
self.logger.addHandler(fh)
self.logger.info("Creating HTTP Request Handler")
super(SimpleHTTPRequestHandler, self).__init__(*args)
def log_message(self, format, *args):
self.logger.info("%s - - [%s] %s\n" %
(self.address_string(),
self.log_date_time_string(),
format%args))
def main():
server = HTTPServer(('', 80), MyHandler)
server.serve_forever()
if __name__ == '__main__':
main()
This is on Python 3.1.3
A logger can have more than one handler. Different handlers may have different log levels. The logger has its own log level which is unaffected by handlers and passes messages with appropriate log levels to the handlers to do with as they see fit.
So while your handler was interested in INFO level messages your logger was ignoring them.
The default log level of WARNING is inherited from the root logger.
I needed to call self.logger.setLevel(logging.INFO) in addition to calling fh.setLevel()
If someone explains why, I'll accept that answer :)
open(LOGFILE, "w+").write("%s - - [%s] %s\n" % (self.address_string(), self.log_date_time_string(), format%args))
This is something you may be interested.
Related
I created two separate logs with two different names (windows). One is a rotating log and the other not. How can I prevent the data that I write to the rotating log from logging to the non-rotating log?
I am not very log savvy.
importing module
import logging
from logging.handlers import RotatingFileHandler
def main():
def normalLogging(fn1):
global normalLogger
#Create and configure logger
logging.basicConfig(filename=fn1, format='%(asctime)s %(message)s', filemode='w')
#Creating an object
normalLogger=logging.getLogger()
#Setting the threshold of logger to DEBUG
normalLogger.setLevel(logging.DEBUG)
def rotatingLog(fn2):
global rotatingLogger
# start the roll-screen logger too
rotatingLogger = logging.getLogger(fn2)
rotatingLogger.setLevel(logging.DEBUG)
# add a rotating logger
handlerScreen = RotatingFileHandler(fn2, maxBytes=1000, backupCount=3)
rotatingLogger.addHandler(handlerScreen)
def normalTest():
#Test messages to the rotating log
normalLogger.debug("normalLogger.debug - Harmless debug Message")
normalLogger.info("normalLogger.info - Just an information")
normalLogger.warning("normalLogger.warning - Its a Warning")
normalLogger.error("normalLogger.error - Did you try to divide by zero")
normalLogger.critical("normalLogger.critical - Internet is down")
def rotatorTest():
for i in range(1, 100):
#Test messages to the rotating log
rotatingLogger.debug("rotatingLogger.debug %s - Harmless debug Message" % i)
rotatingLogger.info("rotatingLogger.info %s - Just an information" % i)
rotatingLogger.warning("rotatingLogger.warning %s - Its a Warning" % i)
rotatingLogger.error("rotatingLogger.error %s - Did you try to divide by zero" % i)
rotatingLogger.critical("rotatingLogger.critical %s - Internet is down" % i)
# fn2 = "rotatorLog"
rotatingLog("rotatorLog")
rotatorTest()
# fn1 = "normalLog"
normalLogging("normalLog")
normalTest()
rotatorTest()
if __name__ == '__main__':
main()
The rotating log maintains it's own unique data, but the normal log has data from the rotating log. I expected that the data would be unique to each log since I write to them separately, but that's not the case.
All you need to do is rotatingLogger.propagate = False to stop it's logs being sent to the root logger which is used for normalLogger. basicConfig configures the root logger and logging.getLogger without a name returns the root logger.
First: use name for normal logger - ie. logging.getLogger(fn1)
Second: use FileHandler instead of basicConfig for normal logger.
import logging
from logging.handlers import RotatingFileHandler
from logging import FileHandler
def main():
def normalLogging(fn1):
global normalLogger
global normalLogger
normalLogger = logging.getLogger(fn1)
normalLogger.setLevel(logging.DEBUG)
handlerScreen = FileHandler(fn1)
handlerScreen.setFormatter(logging.Formatter('%(asctime)s %(message)s'))
normalLogger.addHandler(handlerScreen)
def rotatingLog(fn2):
global rotatingLogger
rotatingLogger = logging.getLogger(fn2)
rotatingLogger.setLevel(logging.DEBUG)
handlerScreen = RotatingFileHandler(fn2, maxBytes=1000, backupCount=3)
handlerScreen.setFormatter(logging.Formatter('%(asctime)s %(message)s'))
rotatingLogger.addHandler(handlerScreen)
# ... rest ...
As I remember basicConfig creates logger which later is used by all loggers so they use the same settings.
import logging
logger = logging.getLogger("test.conf")
logger.setLevel(logging.DEBUG)
hterm = logging.StreamHandler()
hterm.setLevel(logging.ERROR)
hfile = logging.FileHandler("access.log")
hfile.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
hterm.setFormatter(formatter)
hfile.setFormatter(formatter)
logger.addHandler(hterm)
logger.addHandler(hfile)
class test(object):
def post(self):
try:
logger.info('right')
except:
logger.error('wrong')
why the log ('right') will be printed in the console, i have already set the level, is there any logic i miss?
if replace the class part with the following code , it can reach the aim of mine, but when i add a class, it doesn't work
logger.debug("User %s is loging" % 'jeck')
logger.info("User %s attempted wrong password" % 'fuzj')
logger.warning("user %s attempted wrong password more than 3 times" % 'mary')
logger.error("select db is timeout")
logger.critical("server is down")
in the above code, it will print the critical and the error, and all the level will be written into the file.
You are setting logging level DEBUG on logger and trying to log with info hence 'right' is getting printed. The order is:
DEBUG -> INFO -> ERROR -> CRITICAL.
So if you are setting logging level to DEBUG, any level log proceding the DEBUG will be shown, your case. If you set logging level to ERROR, logs under error and critical will be shown and not under debug and info
I wrote a simple bot.py that defines a logger. The logger worked fine.
But then I needed to keepalive the bot.py script, so I wrote a keepalive.py that is called hourly and uses a sockets to identify when bot.py died, and restart it.
But I don't see the right/pythonic way to setup and share the logger between keepalive.py and bot.py.
I experimented with trying to pass the logger instance to bot.py when it's called from the keepalive.py, and otherwise creating it (if bot.py is run directly), but it would require me pass around the logger as a parameter to all the functions I call (example, the demo() function). This is gross.
Whats the best way to setup a logger once, and then use it everywhere else without doing too much passing around, etc?
bot.py
import logging
def demo():
#this won't work unless I pass in the logger as a parameter
#logger.info("demonstrate the Inner demo() called from Inner main works")
pass
def main(logger=None):
if logger is None:
# Enable logging
logging.basicConfig(
filename='log_experiment.txt',
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
level=logging.INFO)
logger = logging.getLogger(__name__)
logger.info("I created logger instance inside Inner Main()")
else:
logger.info("I am using the logger passed to Inner main() as a parameter")
demo()
if __name__ == "__main__":
main()
keepalive.py
import logging
import socket
import sys
# Enable logging
logging.basicConfig(
filename='log_experiment.txt',
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
level=logging.INFO)
logger = logging.getLogger(__name__)
logger.info("I created logger instance inside keepalive")
lock_socket = None # we want to keep the socket open until the very end of
# our script so we use a global variable to avoid going
# out of scope and being garbage-collected
def is_lock_free():
global lock_socket
lock_socket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
try:
lock_id = "straits.keepalive" # this should be unique. using your username as a prefix is a convention
lock_socket.bind('\0' + lock_id)
logging.debug("Acquired lock %r" % (lock_id,))
return True
except socket.error:
# socket already locked, task must already be running
logging.info("Failed to acquire lock %r" % (lock_id,))
return False
if not is_lock_free():
print("alive")
sys.exit()
print("**RESTART REQUIRED")
# then, either include the rest of your script below,
# or import it, if it's in a separate file:
import inner as bot
bot.main(logger)
I want to log to a text file whatever is printed to sys.stdout. While I understand smarter people can come up with more elegant and Pythonic solutions, here is my solution
class logger:
def __init__(self, filename='log.txt'):
self.logf = open(filename, 'a')
global print
self.__print = print
print = self.lognprint
def __del__(self):
self.logf.close()
def lognprint(self, *args, **keywords):
self.__print(*args, file = self.logf, **keywords)
self.__print(*args, **keywords)
now if anywhere in my code I add
mylog = logger()
anything that is printed afterwards is also logged.
But for many obvious reasons this is not safe/good. For example multiple logger objects can be nasty.
In addition I am inspired by
from __future__ import print_function
(see this for example) and I want to do something similar, so that when I import my module, the builtin print is overridden by my version of print anywhere in the code.
How is that possible?
A similar solution, or logging stuff to a file, which also printed to std.out, is given in the logging cookbook.
Here is how you can simply log stuff to a file called 'spam.log' and also print certain stuff to std.out:
import logging
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
fh.setFormatter(formatter)
# add the handlers to logger
logger.addHandler(ch)
logger.addHandler(fh)
# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')
In this example all messages go to the file, only higher levels go to console.
Instead of placing your code inside the class, put it at the module level. This way it will be executed the first time the module is imported:
# logging.py
print = my_print
I am trying to include simple logging into my application using TimedRotatingFileHandler. However I get the output both into the designated file and into the standard error. I reduced the problem to a small example:
import logging, logging.handlers
import sys
logging.basicConfig(format='%(asctime)s %(message)s')
loghandler = logging.handlers.TimedRotatingFileHandler("logfile",when="midnight")
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.addHandler(loghandler)
for k in range(5):
logger.info("Line %d" % k)
I get 5 log lines both in my 'logfile' and this program's stderr. What am I doing wrong?
This is the way you can have the print just on the log files and not to stdout/stderr:
import logging
from logging.handlers import TimedRotatingFileHandler
logHandler = TimedRotatingFileHandler("logfile",when="midnight")
logFormatter = logging.Formatter('%(asctime)s %(message)s')
logHandler.setFormatter( logFormatter )
logger = logging.getLogger( 'MyLogger' )
logger.addHandler( logHandler )
logger.setLevel( logging.INFO )
for k in range(5):
logger.info("Line %d" % k)
logging.basicConfig sets up a handler that prints to standard error.
logger.addHandler(loghandler) sets up a TimedRotatingFileHandler.
Do you wish to squelch output to standard error?
If so, simply remove the call tologging.basicConfig.