I am trying to write unit test case for below logger function and I new to python coding.
def create_logger(filename="__main__"):
# create logger
logger = logging.getLogger(filename)
logger.setLevel(logging.INFO)
# create console handler and set level to debug
console_handler = logging.StreamHandler()
# create formatter
formatter = logging.Formatter(
"%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)
# add formatter to ch
console_handler.setFormatter(formatter)
# add ch to logger
for log_handler in logger.handlers:
logger.removeHandler(log_handler)
logger.addHandler(console_handler)
return logger
Unit test case:
class TestLogger(unittest.TestCase):
def test_create_logger(self):
logger = create_logger("get-list-lambda")
logger.info("logger created successfully")
self.assertEqual("<class 'logging.Logger'>", str(type(logger)))
Now my problem test case coverage is only at 91% due missing unit test case for only one line which is :
logger.removeHandler(log_handler)
Please help me to understand how I can write test case to cover above line.
You can't expect 100% coverage for this example unless you add a handler before the for log_handler in logger.handlers: loop, so that there is actually something to remove. But that would be an artificial thing to do just to increase coverage.
Related
I have a function that sets up logger for my module:
import logging
...
LOGGER = logging.getLogger(__name__)
def setup_logger(level=logging.INFO):
formatter = logging.Formatter(
fmt='%(asctime)s %(levelname)s: %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
handler = logging.StreamHandler()
handler.setLevel(level)
handler.setFormatter(formatter)
LOGGER.setLevel(level)
LOGGER.addHandler(handler)
LOGGER.debug('Logger is set up...')
Now I want to test my code that for example my logger is indeed fires that debug message. I try the following:
def test_setup_logger(self):
with self.assertLogs(LOGGER, level='DEBUG') as cm:
setup_logger(level=10)
self.assertListEqual(cm.output, [2021-11-21 20:39:23 DEBUG: Logger is set up...])
And then I get this message:
AssertionError: Lists differ: ['DEBUG:app.main:Logger is set up...'] != ['2021-11-21 20:39:23 DEBUG: Logger is set up...']
Which apparently means that my LOGGER formatting was not set up. How do I test this properly? Thanks in advance.
First of all your test will never successful because you set static datetime(as str).
cm.output includes printed messages using format {LOG_LEVEL}:{LOGGER_NAME}:{MESSAGE}. Yes, you can check full format using cm.records + handler.formatter. Something like:
class TestExample(unittest.TestCase):
def test_setup_logger(self):
with self.assertLogs(LOGGER) as cm:
now = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
setup_logger(logging.DEBUG)
# Ran 1 test in 0.002s OK...
self.assertEqual(
LOGGER.handlers[1].format(cm.records[0]),
'{} DEBUG: Logger is set up...'.format(now)
)
But... I don't understand what it is for. It looks like you want to write a test for the logging package. Usually you need to check log records(cm.output) after important functionality.
Also you can create a test for the logger configuration. And check only required log records in the other tests. Just an example:
class TestLoggerSetup(unittest.TestCase):
def test_setup_logger(self):
setup_logger(logging.DEBUG)
self.assertTrue(isinstance(LOGGER.handlers[0], logging.StreamHandler))
self.assertEqual(LOGGER.handlers[0].formatter.datefmt, '%Y-%m-%d %H:%M:%S')
self.assertEqual(LOGGER.level, logging.DEBUG)
class TestLogMessages(unittest.TestCase):
def test_my_function(self):
with self.assertLogs(LOGGER) as cm:
LOGGER.info(111)
# or my_function() blablabla... and check only log records:
self.assertListEqual(cm.output, ['INFO:main:111'])
I originally used logging.basicConfig(filename='logs/example.log') to create a log file. After reading the docs I found that it is not recommended to modify the class attributes of logging. Now that I've changed these attributes how can I change them back/reset the logger module?
Output of the code below creates two log files, app.log and example.log. The latter is an artifact of .basicConfig() being called when I first tried to set up the logger.
UPDATE:
grep -R "example.log" /lib/python3.8/ does not output anything so I'm not sure what was changed in the source code of logging to cause the example.log file to be created every time
import logging
import logging.handlers
LOG_FILENAME = 'logs/app.log'
# https://stackoverflow.com/questions/3630774/logging-remove-inspect-modify-handlers-configured-by-fileconfig
# need to run this every time since I changed class attributes of logger
# still creates example.log file
print(logging.getLogger())
root_log = logging.getLogger()
for handler in root_log.handlers:
root_log.removeHandler(handler)
# OR
# logging.basicConfig(force=True) #one way to reset root logger format
# create logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# create console and file handler and set level to info
fh = logging.handlers.RotatingFileHandler(LOG_FILENAME, maxBytes=250000, backupCount=5)
fh.setLevel(logging.INFO)
# ch = logging.StreamHandler()
# ch.setLevel(logging.INFO)
# create formatter
ffh = logging.Formatter('%(asctime)s : %(name)-12s : %(levelname)-8s : %(message)s')
# fch = logging.Formatter('%(name)-12s : %(levelname)-8s : %(message)s')
# add formatter handlers
fh.setFormatter(ffh)
# ch.setFormatter(fch)
# add handler to logger
logger.addHandler(fh)
# logger.addHandler(ch)
logger.info('instance logger')
# logging.shutdown()
I have the following requirements:
To have one global logger which you can configure (setup level, additional handlers,..)
To have per module logger which you can configure (setup level, additional handlers,..)
In other words we need more logs with different configuration
Therefore I did the following
create method to setup logger:
def setup_logger(module_name=None, level=logging.INFO, add_stdout_logger=True):
print("Clear all loggers")
for _handler in logging.root.handlers:
logging.root.removeHandler(_handler)
if add_stdout_logger:
print("Add stdout logger")
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(level)
stdout_handler.setFormatter(logging.Formatter(fmt='%(asctime)-11s [%(levelname)s] [%(name)s] %(message)s'))
logging.root.addHandler(stdout_handler)
print("Set root level log")
logging.root.setLevel(level)
if module_name:
return logging.getLogger(module_name)
else:
return logging.getLogger('global')
Then I create logger as following:
logger_global = setup_logger(level=logging.DEBUG)
logger_module_1 = setup_logger(module_name='module1', level=logging.INFO)
logger_module_2 = setup_logger(module_name='module2', level=logging.DEBUG)
logger_global.debug("This is global log and will be visible because it is setup to DEBUG log")
logger_module_1.debug("This is logger_module_1 log and will NOT be visible because it is setup to INFO log")
logger_module_2.debug("This is logger_module_2 log and will be visible because it is setup to DEBUG log")
Before I will try what works and what not and test it more deeply I want to ask you if this is good practice to do it or do you have any other recommendation how to achieve our requrements?
Thanks for help
Finally I found how to do it:
def setup_logger(module_name=None, level=logging.INFO, add_stdout_logger=True):
custom_logger = logging.getLogger('global')
if module_name:
custom_logger = logging.getLogger(module_name)
print("Clear all handlers in logger") # prevent multiple handler creation
module_logger.handlers.clear()
if add_stdout_logger:
print("Add stdout logger")
stdout_handler = logging.StreamHandler(sys.stdout)
stdout_handler.setLevel(level)
stdout_handler.setFormatter(logging.Formatter(fmt='%(asctime)-11s [%(levelname)s] [%(name)s] %(message)s'))
module_logger.addHandler(stdout_handler)
# here you can add another handlers ,...
# because we use custom handlers which have the different type of log level,
# then our logger has to have the lowest level of logging
custom_logger.setLevel(logging.DEBUG)
return custom_logger
Then simply call the following
logger_module_1 = setup_logger(module_name='module1', level=logging.INFO)
logger_module_2 = setup_logger(module_name='module2', level=logging.DEBUG)
logger_module_1.debug("This is logger_module_1 log and will NOT be visible because it is setup to INFO log")
logger_module_2.debug("This is logger_module_2 log and will be visible because it is setup to DEBUG log")
Currently I'm using logging.getLogger().setLevel(logging.DEBUG) what I think is logging everything where logging level is => DEBUG Is that a correct assumption? I can see a difference when I set logging.DEBUG to logging.ERROR so I guess I'm correct.
Also how do I write these logging rows to a file?
This is a exmaple write log to file
import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# create a file handler
handler = logging.FileHandler('hello.log')
handler.setLevel(logging.INFO)
# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(handler)
logger.info('Hello baby')
More detail:
http://victorlin.me/posts/2012/08/good-logging-practice-in-python/
I have create a global logger using the following:
def logini():
logfile='/var/log/cs_status.log'
import logging
import logging.handlers
global logger
logger = logging.getLogger()
logging.basicConfig(filename=logfile,filemode='a',format='%(asctime)s %(name)s %(levelname)s %(message)s',datefmt='%y%m%d-%H:%M:%S',level=logging.DEBUG,propagate=0)
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=2000000, backupCount=5)
logger.addHandler(handler)
__builtins__.logger = logger
It works, however I am getting 2 outputs for every log, one with the formatting and one without.
I realize that this is being caused by the file rotater as I can comment out the 2 lines of the handler code and then I get a single outputted correct log entry.
How can I prevent the log rotator from outputting a second entry ?
Currently you're configuring two file loggers that point to the same logfile. To only use the RotatingFileHandler, get rid of the basicConfig call:
logger = logging.getLogger()
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=2000000,
backupCount=5)
formatter = logging.Formatter(fmt='%(asctime)s %(name)s %(levelname)s %(message)s',
datefmt='%y%m%d-%H:%M:%S')
handler.setFormatter(formatter)
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
All basicConfig does for you is to provide an easy way to instantiate either a StreamHandler (default) or a FileHandler and set its loglevel and formats (see the docs for more information). If you need a handler other than these two, you should instantiate and configure it yourself.