I'm trying to write a module to use it in different scripts
import logging
from logging.handlers import RotatingFileHandler
_logger_name = "Nagios"
_print_format = "%(asctime)s - %(levelname)s - %(message)s"
_level = logging.DEBUG
class Log():
def __init__(self,log_file,logger_name=_logger_name,level=_level):
self.log_file = log_file
self.logger_name = logger_name
self.level = level
def getLog(self):
"""
Return the logging object
"""
_logger = logging.getLogger(self.logger_name)
_logger.setLevel(self.level)
_logger.addHandler(self._rotateLog())
return _logger
def _rotateLog(self):
"""
Rotating the log files if it exceed the size
"""
rh = RotatingFileHandler(self.log_file,
maxBytes=20*1024*1024, backupCount=2)
formatter = logging.Formatter(_print_format)
rh.setFormatter(formatter)
return rh
log = Log("kdfnknf").getLog()
log("hello")
I see the following error:
Traceback (most recent call last):
File "nagiosLog.py", line 45, in <module>
log("hello")
TypeError: 'Logger' object is not callable
Any idea why I'm getting this error,
When debugged using pdb I do see it returns the object and printing the dir(log) I don't see the Logger module in it.
Am I missing something here
log("Hello")
This is wrong.
Correct is
log.info("Hello")
log must be printed with logging level i.e. info/error/warning
See the logging docs:
You have to use a function, you can't just call Logger:
Logger.info(msg, *args, **kwargs)
Logs a message with level INFO on
this logger. The arguments are interpreted as for debug().
or
Logger.warning(msg, *args, **kwargs)
Logs a message with level WARNING on this logger. The arguments are >interpreted as for debug().
so instead, do:
log.info("Test info level logging...")
Related
I would like to call a function everytime, I do any sort of logging using the logging module in python.
As can be seen below, instead of calling copyfiles at each instance of logging, I would like to see if there is way to injest copyfiles to the filehandler in someway or even a wrapper.
from sys import argv
import shutil
import logging
logger = logging.getLogger('')
sh = logging.StreamHandler(sys.stdout)
logfile = 'logs/log-run-{}.log'.format(date.today().strftime('%d-%m-%Y'))
fh = logging.FileHandler(logfile)
formatter = logging.Formatter('[%(asctime)s] %(levelname)s \
[%(filename)s.%(funcName)s:%(lineno)d] \
%(message)s', datefmt='%a, %d %b %Y %H:%M:%S')
logger.setLevel(logging.INFO)
sh.setFormatter(formatter)
fh.setFormatter(formatter)
logger.addHandler(sh)
logger.addHandler(fh)
LOGPATH = args[1]
def copyfiles():
"""
Copy files to azure blob storage
"""
fh.flush()
shutil.copy(logfile, LOGPATH)
logging.info('test')
copyfiles()
logging.info('foobar')
copyfiles()
I tried digging into invoking copyfiles each time logging is called, but I ended up no where.
In case you are wondering why I am copy files from logging, this is why.
Currently this is what I could think of:
Override the flush() of FileHandler by inheriting it into a utility
class like FlushCopyFileHandler class shown in the code below.
Instead of using the FileHandler use the FlushCopyFileHandler class
and all you have to do is call this overridden flush().
"""
from logging import FileHandler
from shutil
class FlushCopyFileHandler(FileHandler):
# These arguments are passed along to the parent class FileHandler.
def __init__(self, *args, **kwargs):
super(FlushCopyFileHandler, self).__init__(filename, *args, **kwargs)
self.copy_destination = "some default destination path" # can also be set from args.
# When this flush() is called it will call the flush() of FileHandler class.
# After that it will call the shutil.copy() method.
def flush(self):
super().flush()
shutil.copy(self.filename, self.copy_destination)
# Use this to change destination path later on.
def set_copy_path(self, destination):
self.copy_destination = destination
"""
I'm trying to create a log in Python 3.x, that writes out to the console. Here is my code:
import logging
import sys
class Temp:
def __init__(self, is_verbose=False):
# configuring log
if (is_verbose):
self.log_level=logging.DEBUG
else:
self.log_level=logging.INFO
log_format = logging.Formatter('[%(asctime)s] [%(levelname)s] - %(message)s')
logging.basicConfig(level=self.log_level, format=log_format)
self.log = logging.getLogger(__name__)
# writing to stdout
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(self.log_level)
handler.setFormatter(log_format)
self.log.addHandler(handler)
# here
self.log.debug("test")
if __name__ == "__main__":
t = Temp(True)
If the line after "here" is entered, Python raises an error:
[2019-01-29 15:54:20,093] [DEBUG] - test
--- Logging error ---
Traceback (most recent call last):
File "C:\Programok\Python 36\lib\logging\__init__.py", line 993, in emit
msg = self.format(record)
File "C:\Programok\Python 36\lib\logging\__init__.py", line 839, in format
return fmt.format(record)
File "C:\Programok\Python 36\lib\logging\__init__.py", line 577, in format
if self.usesTime():
File "C:\Programok\Python 36\lib\logging\__init__.py", line 545, in usesTime
return self._style.usesTime()
File "C:\Programok\Python 36\lib\logging\__init__.py", line 388, in usesTime
return self._fmt.find(self.asctime_search) >= 0
AttributeError: 'Formatter' object has no attribute 'find'
...
I also had some other places in my code that prints to the log, but nothing is written to stdout, even if the line after "here" is removed.
What might be the problem?
The problem comes from the call to basicConfig which sets up a log handler for stderr and also accepts a format string, not a formatter. Because you are doing this work yourself later, you don't need to use the basicConfig function. More information can be found in the python documentation.
Removing the call to basicConfig, and adding a self.log.setLevel(self.log_level) will fix the issue you are seeing.
Working code:
import logging
import sys
class Temp:
def __init__(self, is_verbose=False):
# configuring log
if (is_verbose):
self.log_level=logging.DEBUG
else:
self.log_level=logging.INFO
log_format = logging.Formatter('[%(asctime)s] [%(levelname)s] - %(message)s')
self.log = logging.getLogger(__name__)
self.log.setLevel(self.log_level)
# writing to stdout
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(self.log_level)
handler.setFormatter(log_format)
self.log.addHandler(handler)
# here
self.log.debug("test")
if __name__ == "__main__":
t = Temp(True)
Looking through a similar issue on the Python bug tracker (https://bugs.python.org/issue16368), you can see that the formatter argument is expected to be a string (hence the attempt to invoke find):
log_format = '[%(asctime)s] [%(levelname)s] - %(message)s'
logging.basicConfig(level=self.log_level, format=log_format)
I have a logger and a class DuplicateFilter that filters messages that already were logged once. I would like to include the time when the logging happened into my filter but when I try to access the property asctime I get: AttributeError: 'LogRecord' object has no attribute 'asctime'
Here a small example how I set up my logger:
import logging
import logging.handlers as log_handlers
def setup_logger(filename):
class DuplicateFilter(object):
def __init__(self):
self.msgs = set()
def filter(self, record):
if logger.level == 10:
return True
rv = True
try:
print(record.asctime)
msg = record.threadName + " " + record.msg
if msg in self.msgs:
rv = False
except Exception as e:
print(traceback.format_exc())
return rv
self.msgs.add(msg)
return rv
log_formatter = logging.Formatter("%(asctime)s [%(levelname)-5.5s] [%(threadName)-30.30s] %(message)s")
file_handler = log_handlers.TimedRotatingFileHandler(filename, encoding="UTF-8", when='W6', backupCount=12)
file_handler.setFormatter(log_formatter)
console_handler = logging.StreamHandler()
console_handler.setFormatter(log_formatter)
logger = logging.getLogger(filename)
logger.propagate = False
logger.addHandler(console_handler)
logger.addHandler(file_handler)
logger.setLevel(logging.INFO)
dup_filter = DuplicateFilter()
logger.addFilter(dup_filter)
return logger
log = setup_logger("test.log")
for i in range(3):
log.info("wow")
Now my records look like this: 2018-07-18 14:34:49,642 [INFO ] [MainThread ] wow They clearly have an asctime and I explicitly set the asctime property in the constructor of my Formatter. The only question similar to mine I found says
To have message and asctime set, you must first call self.format(record) inside the emit method
but doesn't the logging.Formatter do that when you specify the log string the way I did with %(asctime)s?
EDIT: running.t was right, I just didn't understand what the documentation meant. I solved it by adding my formater to my filter and calling the format function at the beginning:
def __init__(self, formatter):
self.msgs = {}
self.formatter = formatter
def filter(self, record):
self.formatter.format(record)
In filter objects section of pyton logging module documentation I found following note:
Note that filters attached to handlers are consulted before an event is emitted by the handler, whereas filters attached to loggers are consulted whenever an event is logged (using debug(), info(), etc.), before sending an event to handlers. This means that events which have been generated by descendant loggers will not be filtered by a logger’s filter setting, unless the filter has also been applied to those descendant loggers.
Your filter is added to logger, while formatters are added to handlers. So in my opinion your filter method is applied before any of formatter you specified.
BTW, shouldn't your DuplicateFilter inherit from logging.Filter?
I want to write to a log file some events. In order to do this I've used functions decorators to add the loggin code, and report the function called. But, the output is always the same function, the decorator function _decorador.
I'm using the %(funcName)s parameter in format logging.basicConfig
Output in example.log:
04/21/2014 09:32:41 AM DEBUG This message should go to the log file _decorador
04/21/2014 09:32:41 AM INFO So should this _decorador
04/21/2014 09:32:41 AM WARNING And this, too _decorador
04/21/2014 10:46:23 AM DEBUG This message should go to the log file (debug) _decorador
04/21/2014 10:46:23 AM INFO So should this (info) _decorador
04/21/2014 10:46:23 AM WARNING And this, too (warning) _decorador
Desired output in example.log:
04/21/2014 09:32:41 AM DEBUG This message should go to the log file mi_funcion
04/21/2014 09:32:41 AM INFO So should this mi_funcion
04/21/2014 09:32:41 AM WARNING And this, too mi_funcion
04/21/2014 10:46:23 AM DEBUG This message should go to the log file (debug) mi_funcion
04/21/2014 10:46:23 AM INFO So should this (info) mi_funcion
04/21/2014 10:46:23 AM WARNING And this, too (warning) mi_funcion
My code:
#!usr/bin/python3
# -*- coding: UTF-8 -*-
import logging
FORMAT = '%(asctime)s %(levelname)s %(message)s %(funcName)s'
logging.basicConfig(filename='example.log', level=logging.DEBUG, format=FORMAT, datefmt='%m/%d/%Y %I:%M:%S %p')
# Decorator function, writes in the log file.
def decorador(funcion):
def _decorador(*args, **kwargs):
funcion(*args, **kwargs)
logging.debug('This message should go to the log file (debug)')
logging.info('So should this (info)')
logging.warning('And this, too (warning)')
return _decorador
#decorador
def mi_funcion(arg1, arg2):
print("Code asset: %s; Registry number: s%" % (arg1, arg2))
mi_funcion("18560K", 12405)
It's 2022 and this is still difficult.
Here's a complete example adapted from Using functools.wraps with a logging decorator
from inspect import getframeinfo, stack
import logging
from functools import wraps
class CustomFormatter(logging.Formatter):
"""Custom formatter, overrides funcName with value of name_override if it exists"""
def format(self, record):
if hasattr(record, 'name_override'):
record.funcName = record.name_override
if hasattr(record, 'file_override'):
record.filename = record.file_override
if hasattr(record, 'line_override'):
record.lineno= record.line_override
return super(CustomFormatter, self).format(record)
# setup logger and handler
logger = logging.getLogger(__file__)
handler = logging.StreamHandler()
logger.setLevel(logging.DEBUG)
handler.setLevel(logging.DEBUG)
handler.setFormatter(CustomFormatter('%(asctime)s - %(filename)s:%(lineno)s - %(funcName)s - %(levelname)s - %(message)s'))
logger.addHandler(handler)
def log_and_call(statement):
def decorator(func):
caller = getframeinfo(stack()[1][0])
#wraps(func)
def wrapper(*args, **kwargs):
# set name_override to func.__name__
logger.info(statement, extra={
'name_override': func.__name__,
'file_override': os.path.basename(caller.filename),
'line_override': caller.lineno
})
return func(*args, **kwargs)
return wrapper
return decorator
#log_and_call("This should be logged by 'decorated_function'")
def decorated_function(): # <- the logging in the wrapped function will point to/log this line for lineno.
logger.info('I ran')
decorated_function()
Defining the caller outside of the wrapper function will correctly get the calling function's (i.e. the wrapped function) filename and line number.
You cannot easily change this. The goal of the logging module funcName is to report exact locations of the source code line, not the function it represents. The idea is that you use it in combination with the lineno and filename entries to pinpoint the source code, not what function was called.
In order to achieve this, the log module uses code object introspection to determine the real function name:
def findCaller(self):
"""
Find the stack frame of the caller so that we can note the source
file name, line number and function name.
"""
f = currentframe()
#On some versions of IronPython, currentframe() returns None if
#IronPython isn't run with -X:Frames.
if f is not None:
f = f.f_back
rv = "(unknown file)", 0, "(unknown function)"
while hasattr(f, "f_code"):
co = f.f_code
filename = os.path.normcase(co.co_filename)
if filename == _srcfile:
f = f.f_back
continue
rv = (co.co_filename, f.f_lineno, co.co_name)
break
return rv
Short of reconstructing the _decorador code object you cannot alter what is reported here. Reconstructing the code object can be done; you could build a facade function with exec that calls the decorator, for example. But for this to work with a closure is more work than you should worry about, really.
I'd instead include the function name of the wrapped function:
logging.debug('This message should go to the log file (debug) (function %r)',
funcion)
You can extract the function name from the funcion object:
def decorador(funcion):
def _decorador(*args, **kwargs):
funcion(*args, **kwargs)
logging.debug('This message should go to the log file (debug) %s',
funcion.__name__)
# ...
return _decorador
I get this output after running the modified code:
cat example.log
04/21/2014 11:37:12 AM DEBUG This message should go to the log file (debug) mi_funcion
Question about sharing 'functions' between classes.
Situation:
All my own code is in 1 file
I'm using python-daemon to daemonize my script
That uses a class (Doorcamdaemon) to initiate and run.
It imports another class (Doorcam) which has a looping function
I'm using a sample script for the daemon functions, and it shows how to use the logging module.
The logging works from the main part of the script and in the Doorcamdaemon class, but not in the Doorcam class.
class Doorcamdaemon():
def __init__(self):
#skipping some content, not related to this issue
self.Doorcam=Doorcam()
def run(self):
self.Doorcam.startListening() #looping function
class Doorcam
def __init__(self):
#skipping somecontent, not related to this issue
def startListening(self):
while True:
logger.info('Hello')
app = Doorcamdaemon()
logger = logging.getLogger("DoorcamLog")
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler = logging.FileHandler("/var/log/doorcam.log")
handler.setFormatter(formatter)
logger.addHandler(handler)
daemon_runner = runner.DaemonRunner(app)
daemon_runner.daemon_context.files_preserve=[handler.stream]
daemon_runner.do_action()
The error returned is:
$ ./Doorcam.py start
Traceback (most recent call last):
File "./Doorcam.py", line 211, in <module>
app = Doorcamdaemon()
File "./Doorcam.py", line 159, in __init__
self.doorcam=Doorcam()
File "./Doorcam.py", line 18, in __init__
logger.info('Doorcam started capturing')
NameError: global name 'logger' is not defined
So my obvious question: How can I make it work in the Doorcam class as well?
Try moving the line
app = Doorcamdaemon()
to after the lines that create and set up logger. The traceback is telling you:
logger doesn't exist in line 18 where Doorcam's constructor tries to use it
Doorcamdaemon tries to construct a Doorcam at line 159 in its own constructor
So you can't create a Doorcamdaemon if logger isn't defined yet.
Some of the content you omitted in Doorcam.__init__ was related to this issue :)