When I run the below piece of code, the logging is working fine. But when I comment #1) and #2) under setup_logger the log is not displayed.
What does #1) and #2) do here?
import logging
import sys
def get_logger(name):
print('get_logger -- ', name)
log = logging.getLogger("hello.{}".format(name))
return log
def setup_logger():
print('setup_logger')
root = logging.getLogger("")
root.setLevel(logging.ERROR)
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter(
fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
))
root.addHandler(handler)
logger = logging.getLogger("hello") #1
logger.setLevel(logging.DEBUG) #2
LOG = get_logger(__name__)
setup_logger()
print(LOG)
def main():
LOG.debug('hello')
if __name__ == '__main__':
main()
The first line you marked gets a logger with given name. If there currently is no logger with the name you provided, a new one will be created (Also known as Singleton). You can read more about logging.getLogger in the Python documentation :
Return a logger with the specified name or, if name is None, return a logger
which is the root logger of the hierarchy. If
specified, the name is typically a dot-separated hierarchical name
like ‘a’, ‘a.b’ or ‘a.b.c.d’. Choice of these names is entirely up to
the developer who is using logging.
All calls to this function with a given name return the same logger instance.
This means that logger instances never need to be
passed between different parts of an application.
The second line you marked sets the loglevel on the logger instance. If you set the loglevel to DEBUG, all messages you log will be printed to the console / file. If you set it to INFO, all messages except for
debug messages will be logged. And so on. Quote from the docs again:
Sets the threshold for this logger to level. Logging messages which
are less severe than level will be ignored; logging messages which
have severity level or higher will be emitted by whichever handler or
handlers service this logger, unless a handler’s level has been set to
a higher severity level than level.
When a logger is created, the level is set to NOTSET (which causes all
messages to be processed when the logger is the root logger, or
delegation to the parent when the logger is a non-root logger). Note
that the root logger is created with level WARNING.
If you still have questions, I can edit my post to answer them.
Related
It seems recently there were some changes of python logging package. Some code work previously doesn't work now. And I am confused. My python version is Python 2.7.15.
The first example I don't understand is that below only prints "WARNING:root:hello from warn". If I understand correctly, "logging.info" actually calls the root logger, and root logger default to warn level. So the first "hello from info" is ignored, which is fine. But why the second "hello from info" is also not printed?
import logging
logging.info("hello from info")
logging.warn("hello from warn")
logging.basicConfig(level=logging.INFO)
logging.info("hello from info")
The second question is the logging level of Handler versus logger. If we set the log level for both the handler and logger, which one is working? Or what if we just set level for Handler? Take the example as below. We already set the log level for StreamHandler, but the "hello from info" is not printed to stdout. Only the "hello from warn" (Besides, it is not "WARNING:t:hello from warn"). Why is that?
import logging
ch = logging.StreamHandler()
ch.setLevel(logging.INFO)
logger = logging.getLogger('t')
logger.addHandler(ch)
logger.info("hello from info")
logger.warn("hello from warn")
But why the second "hello from info" is also not printed?
Because
logging.info / warn / error / debug call logging.basicConfig under the hood. Example:
def info(msg, *args, **kwargs):
if len(root.handlers) == 0:
basicConfig()
root.info(msg, *args, **kwargs)
logging.basicConfig does not do anything if the root logger is already configured. Quote from the docs:
This function does nothing if the root logger already has handlers configured for it.
So, in your code, the root logger is configured with WARN level when logging.info("hello from info") is executed. The subsequent call of logging.basicConfig(level=logging.INFO) has no effect.
Rule of thumb: configure the loggers (no matter if manually or via logging.basicConfig()) as early as possible in your code.
If we set the log level for both the handler and logger, which one is working?
Both! Logger level and handler level are two different stages of filtering records. The logger level defines what records are actually passed to its handlers, while the handler level defines what records will be handled by the particular handler. Examples:
logger.setLevel(logging.INFO)
handler.setLevel(logging.ERROR)
logger.addHandler(handler)
logger.info('spam')
Since logger has level INFO, it will process spam record and pass it to its handlers. However, handler has level ERROR, so the record will not be processed by handler.
logger.setLevel(logging.WARN)
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
logger.info('spam')
Now the handler will process almost any record, including spam record since its level is INFO, thus greater than DEBUG. However, the handler will never receive spam to process because the logger will not process it, thus not passing spam to its handlers.
logger.setLevel(logging.INFO)
h1 = logging.StreamHandler()
h1.setLevel(logging.CRITICAL)
h2 = logging.FileHandler('some.log')
h2.setLevel(logging.DEBUG)
logger.addHandler(h1)
logger.addHandler(h2)
logger.info('spam')
Now the logger has two handlers, h1 printing records to terminal, h2 writing them to file. The logger will pass only records of level INFO or greater to its handlers. However, you will see only records with level CRITICAL in terminal, but all records in log file.
I created the following script. Could any of you explain to me why the output is like what shows below
Source
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
print('debug', logger.isEnabledFor(logging.DEBUG))
print('info', logger.isEnabledFor(logging.INFO))
print('warning', logger.isEnabledFor(logging.WARNING))
print('error', logger.isEnabledFor(logging.ERROR))
logger.debug('debug')
logger.info('info')
logger.warning('warning')
logger.error('error')
logging.debug('debug')
logging.info('info')
logging.warning('warning')
logging.error('error')
Output
debug True
info True
warning True
error True
warning
error
DEBUG:root:debug
INFO:root:info
WARNING:root:warning
ERROR:root:error
Specifically
what is the difference between logger.info and logging.info here
how come that logger.isEnabledFor(logging.DEBUG) is True while logger.debug('debug') does not show anything
how come that logger.info has no output but logging.info has
A few things to clarify:
Default log level for root logger is WARNING
Root logger is not initialized if you do nothing, that is, without any handlers or formatter set up:
>>> import logging
>>> logging.root.handlers
[]
Okay, but you found out the problem: when logging level set to DEBUG, the root logger is not working as expected. Debug messages are ignored. With the same not configured root logger, warning messages output normally. Why is that?
Keep in mind we don't have any handler for root logger right now. But looking into the code, we do see:
if (found == 0):
if lastResort:
if record.levelno >= lastResort.level:
lastResort.handle(record)
elif raiseExceptions and not self.manager.emittedNoHandlerWarning:
sys.stderr.write("No handlers could be found for logger"
" \"%s\"\n" % self.name)
self.manager.emittedNoHandlerWarning = True
Which means, we have a lastResort for backup if no handler is found. You can refer to the definition of lastResort, it is initialized with logging level WARNING. Meanwhile, debug messages don't have this backup so they are ignored when no handler is set.
For your questions:
These two loggers are identical, since the root logger is returned when getLogger() receives no arguments.
See below:
Logger.isEnabledFor(lvl)
Indicates if a message of severity lvl would
be processed by this logger. This method checks first the module-level
level set by logging.disable(lvl) and then the logger’s effective
level as determined by getEffectiveLevel().
Calling any logging functions in logging module will initialize the root logger with basicConfig() which adds a default handler, so that the subsequent calls on logger will also work.
What you should do is, use logging.basicConfig() to set up a default handler for root logger and messages will be output according to the logger level and message level.
getLogger creates an instance of Logger class if argument name is added. Otherwise it returns root logger. So in this case the program is using the common logger as functions logging.debug, logging.info, logging.warning, logging.info
Forgive me, still learning how to use the logging module. So, I have two handlers, one to log everything to a file and one specifically to log the message when triggered into Hipchat.
I don't quite understand what it means to have root vs non-root loggers. As it is below, it will write what I want to my log file AND displays both messages into the hipchat room. So I found another stack post that says to create two multiple "non root loggers." So I tried it with logger = logging.getLogger('test1') and hipchat_logger = logging.getLogger('test2') but there's no message in Hipchat (it does write to the log file).
logger = logging.getLogger()
hipchat_logger = logging.getLogger()
# File handler
file_handler = logging.FileHandler("mylog.log", "a")
logger.addHandler(file_handler)
logger.setLevel(logging.DEBUG)
# Hipchat handler
hipchat_handler = hiplogging.HipChatHandler(MY_TOKEN, CHATROOM)
hipchat_handler.setLevel(logging.DEBUG)
hipchat_logger.addHandler(hipchat_handler)
# Testing
logger.info("Hi, this is a test. I should not see this in Hipchat")
hipchat_logger.info("Hi, display me in Hipchat")
Problem is in the first two lines. "Multiple calls to getLogger() with the same name will always return a reference to the same Logger object.", cf. here.
So a solution would be (modified after discussion in comments):
logger = logging.getLogger()
hipchat_logger = logging.getLogger("hipchat")
EDIT
After more research on this topic: The problem is certainly not the missing root logger. There is a problem with loglevels:
Logging messages which are less severe than lvl will be ignored. When a logger is created, the level is set to NOTSET (which causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger). Note that the root logger is created with level WARNING.
So what does happen here?
# implicit root logger with loglevel WARNING
# both child-logger: loglevel NOTSET
logger = logging.getLogger('file')
hipchat_logger = logging.getLogger('hipchat')
# ...create and add file handler...
# change 'file'-loglevel to DEBUG
logger.setLevel(logging.DEBUG)
# ...create hipchat handler...
# change loglevel of the handler(!) to DEBUG.
# 'hipchat'-loglevel is still NOTSET
hipchat_handler.setLevel(logging.DEBUG)
# ...add hipchat handler ...
# logger -> loglevel DEBUG -> INFO gets logged.
logger.info("Hi, this is a test. I should not see this in Hipchat")
# hipchat_logger -> loglevel NOTSET -> delegate to root-logger
# root-logger -> loglevel WARNING -> INFO not logged (being below threshold)
hipchat_logger.info("Hi, display me in Hipchat")
So to solve this issue, either explicitly set the loglevel of root logger to DEBUG (as in the code above). Or simply set the loglevel of the second logger accordingly:
hipchat_logger.setLevel(logging.DEBUG)
A question that might remain is answered elsewhere: What is the point of setlevel in a handler?
One with your code problem is that calling logging.getLogger() multiple times returns the same logger.
# both of these are same logger instances
logger = logging.getLogger()
hipchat_logger = logging.getLogger()
To get two different instance, give different names to different loggers. Also, with no name parameter, you get the root logger.
root_logger = logging.getLogger()
logger = logging.getLogger("file")
hipchat_logger = logging.getLogger("hipchat")
#now getting the logger with same name will return same logger
this_is_same_as_logger = logging.getLogger("file")
As the logger.setLevel doc says:
When a logger is created, the level is set to NOTSET (which causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger). Note that the root logger is created with level WARNING.
so I think if I create a root logger, with level NOTSET, the debug and info log will display.
The code use basicConfig to set root logger's level to NOTSET is right:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
logging.basicConfig(level=logging.NOTSET)
log = logging.getLogger()
log.debug('debug')
log.info('info')
log.warning('warning')
log.error('error')
log.critical('critical')
and output is:
DEBUG:root:debug
INFO:root:info
WARNING:root:warning
ERROR:root:error
CRITICAL:root:critical
But if I create a root logger, and add handler with NOTSET level to it, such as:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
log = logging.getLogger()
hd = logging.StreamHandler()
hd.setLevel(logging.NOTSET)
log.addHandler(hd)
log.debug('debug')
log.info('info')
log.warning('warning')
log.error('error')
log.critical('critical')
the output is:
warning
error
critical
but I think it will also output the debug and info message.
OK, I misunderstand the Logger's level and Handler's Level, in the doc:
The setLevel() method, just as in logger objects, specifies the lowest severity that will be dispatched to the appropriate destination. Why are there two setLevel() methods? The level set in the logger determines which severity of messages it will pass to its handlers. The level set in each handler determines which messages that handler will send on.
If I change code to this, will be ok:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
log = logging.getLogger()
log.setLevel(logging.NOTSET) # Set Logger's level to NOTSET, default is WARNING
#print "Logger's Level: ", log.level
hd = logging.StreamHandler()
hd.setLevel(logging.NOTSET)
#print "Handler's Level: ", hd.level
log.addHandler(hd)
log.debug('debug')
log.info('info')
log.warning('warning')
log.error('error')
log.critical('critical')
I also thought this was confusing. I interpreted NOTSET as don't produce logs. But...
The level and handlers entries are interpreted as for the root logger, except that if a non-root logger’s level is specified as NOTSET, the system consults loggers higher up the hierarchy to determine the effective level of the logger. source
and
The level is interpreted as for loggers, and NOTSET is taken to mean ‘log everything’. source
This makes sense because NOTSET (0) which is below DEBUG (10)
Let's explain by example that UNSET does NOT do anything when creating concrete instances of loggers (for some mysterious reason):
def logging_unset_level():
"""My sample logger explaining UNSET level
Resources:
- https://stackoverflow.com/questions/21494468/about-notset-in-python-logging
- https://www.youtube.com/watch?v=jxmzY9soFXg&t=468s
- https://github.com/CoreyMSchafer/code_snippets/tree/master/Logging-Advanced
"""
import logging
logger = logging.getLogger(__name__) # loggers are created in hierarchy using dot notation, thus __name__ ensures no name collisions.
print(f'DEFAULT VALUE: logger.level = {logger.level}')
file_handler = logging.FileHandler(filename='my_log.log')
log_format = "{asctime}:{levelname}:{lineno}:{name}:{message}" # see for logrecord attributes https://docs.python.org/3/library/logging.html#logrecord-attributes
formatter = logging.Formatter(fmt=log_format, style='{')
file_handler.setFormatter(fmt=formatter)
stdout_stream_handler = logging.StreamHandler(stream=sys.stdout)
stdout_stream_handler.setLevel(logging.INFO)
log_format = "{name}:{levelname}:-> {message}" # see for logrecord attributes https://docs.python.org/3/library/logging.html#logrecord-attributes
formatter = logging.Formatter(fmt=log_format, style='{')
stdout_stream_handler.setFormatter(fmt=formatter)
logger.addHandler(hdlr=file_handler)
logger.addHandler(hdlr=stdout_stream_handler)
logger.log(logging.NOTSET, 'notset')
logger.debug('debug')
logger.info('info')
logger.warning('warning')
logger.error('error')
logger.critical('critical')
the output to screen is:
DEFAULT VALUE: logger.level = 0
__main__:WARNING:-> warning
__main__:ERROR:-> error
__main__:CRITICAL:-> critical
the output to stdout is:
2020-04-15 17:00:38,384:WARNING:200:__main__:warning
2020-04-15 17:00:38,384:ERROR:201:__main__:error
2020-04-15 17:00:38,384:CRITICAL:202:__main__:critical
as expected from the question's body, when the logger is instantiated its setting is UNSET. If you are using the basic config for some reason it does work as the docs imply it should everything from UNSET to higher get logged BUT when instantiating loggers concretely this setting makes only error messages work for some reason (find me a reference!).
Ok, now if you run it again but increase the level of the top logger everything works as you would expect (except the unset):
logger.setLevel(logging.DEBUG)
The output of stdout (note the stdout handler has level INFO in my example for pedagogical reasons, different from the file handler, trying to show inheritance of the .level value from the top logger):
DEFAULT VALUE: logger.level = 10
__main__:INFO:-> info
__main__:WARNING:-> warning
__main__:ERROR:-> error
__main__:CRITICAL:-> critical
and the output of the logger file my_log.log (this one DOES output the debug statements! :D ):
2020-04-15 17:05:58,782:DEBUG:198:__main__:debug
2020-04-15 17:05:58,784:INFO:199:__main__:info
2020-04-15 17:05:58,784:WARNING:200:__main__:warning
2020-04-15 17:05:58,784:ERROR:201:__main__:error
2020-04-15 17:05:58,784:CRITICAL:202:__main__:critica
is as expected. It inherited the top level loggers level so it printed everything from DEBUG level upwards.
I'm using Python logging, and for some reason, all of my messages are appearing twice.
I have a module to configure logging:
# BUG: It's outputting logging messages twice - not sure why - it's not the propagate setting.
def configure_logging(self, logging_file):
self.logger = logging.getLogger("my_logger")
self.logger.setLevel(logging.DEBUG)
self.logger.propagate = 0
# Format for our loglines
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
# Setup console logging
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(formatter)
self.logger.addHandler(ch)
# Setup file logging as well
fh = logging.FileHandler(LOG_FILENAME)
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)
self.logger.addHandler(fh)
Later on, I call this method to configure logging:
if __name__ == '__main__':
tom = Boy()
tom.configure_logging(LOG_FILENAME)
tom.buy_ham()
And then within say, the buy_ham module, I'd call:
self.logger.info('Successfully able to write to %s' % path)
And for some reason, all the messages are appearing twice. I commented out one of the stream handlers, still the same thing. Bit of a weird one, not sure why this is happening. I'm assuming I've missed something obvious.
You are calling configure_logging twice (maybe in the __init__ method of Boy) : getLogger will return the same object, but addHandler does not check if a similar handler has already been added to the logger.
Try tracing calls to that method and eliminating one of these. Or set up a flag logging_initialized initialized to False in the __init__ method of Boy and change configure_logging to do nothing if logging_initialized is True, and to set it to True after you've initialized the logger.
If your program creates several Boy instances, you'll have to change the way you do things with a global configure_logging function adding the handlers, and the Boy.configure_logging method only initializing the self.logger attribute.
Another way of solving this is by checking the handlers attribute of your logger:
logger = logging.getLogger('my_logger')
if not logger.handlers:
# create the handlers and call logger.addHandler(logging_handler)
If you are seeing this problem and you're not adding the handler twice then see abarnert's answer here
From the docs:
Note: If you attach a handler to a logger and one or more of its
ancestors, it may emit the same record multiple times. In general, you
should not need to attach a handler to more than one logger - if you
just attach it to the appropriate logger which is highest in the
logger hierarchy, then it will see all events logged by all descendant
loggers, provided that their propagate setting is left set to True. A
common scenario is to attach handlers only to the root logger, and to
let propagation take care of the rest.
So, if you want a custom handler on "test", and you don't want its messages also going to the root handler, the answer is simple: turn off its propagate flag:
logger.propagate = False
I'm a python newbie, but this seemed to work for me (Python 2.7)
while logger.handlers:
logger.handlers.pop()
The handler is added each time you call from outside. Try Removeing the Handler after you finish your job:
self.logger.removeHandler(ch)
In my case I'd to set logger.propagate = False to prevent double printing.
In below code if you remove logger.propagate = False then you will see double printing.
import logging
from typing import Optional
_logger: Optional[logging.Logger] = None
def get_logger() -> logging.Logger:
global _logger
if _logger is None:
raise RuntimeError('get_logger call made before logger was setup!')
return _logger
def set_logger(name:str, level=logging.DEBUG) -> None:
global _logger
if _logger is not None:
raise RuntimeError('_logger is already setup!')
_logger = logging.getLogger(name)
_logger.handlers.clear()
_logger.setLevel(level)
ch = logging.StreamHandler()
ch.setLevel(level)
# warnings.filterwarnings("ignore", "(Possibly )?corrupt EXIF data", UserWarning)
ch.setFormatter(_get_formatter())
_logger.addHandler(ch)
_logger.propagate = False # otherwise root logger prints things again
def _get_formatter() -> logging.Formatter:
return logging.Formatter(
'[%(asctime)s] [%(name)s] [%(levelname)s] %(message)s')
This can also happen if you are trying to create a logging object from the parent file. For e.g.
This is the main application file test.py
import logging
# create logger
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
# create console handler and set level to debug
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
# create formatter
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
# add formatter to ch
ch.setFormatter(formatter)
# add ch to logger
logger.addHandler(ch)
def my_code():
# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')
And below is the parent file main.py
import test
test.my_code()
The output of this will print only once
2021-09-26 11:10:20,514 - simple_example - DEBUG - debug message
2021-09-26 11:10:20,514 - simple_example - INFO - info message
2021-09-26 11:10:20,514 - simple_example - WARNING - warn message
2021-09-26 11:10:20,514 - simple_example - ERROR - error message
2021-09-26 11:10:20,514 - simple_example - CRITICAL - critical message
But if we had a parent logging object, then it will be printed twice. For e.g. if this is the parent file
import test
import logging
logging.basicConfig(level=logging.DEBUG,format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
test.my_code()
The the output will be
2021-09-26 11:16:28,679 - simple_example - DEBUG - debug message
2021-09-26 11:16:28,679 - simple_example - DEBUG - debug message
2021-09-26 11:16:28,679 - simple_example - INFO - info message
2021-09-26 11:16:28,679 - simple_example - INFO - info message
2021-09-26 11:16:28,679 - simple_example - WARNING - warn message
2021-09-26 11:16:28,679 - simple_example - WARNING - warn message
2021-09-26 11:16:28,679 - simple_example - ERROR - error message
2021-09-26 11:16:28,679 - simple_example - ERROR - error message
2021-09-26 11:16:28,679 - simple_example - CRITICAL - critical message
2021-09-26 11:16:28,679 - simple_example - CRITICAL - critical message
A call to logging.debug() calls logging.basicConfig() if there are no root handlers installed. That was happening for me in a test framework where I couldn't control the order that test cases fired. My initialization code was installing the second one. The default uses logging.BASIC_FORMAT that I didn't want.
It seems that if you output something to the logger (accidentally) then configure it, it is too late. For example, in my code I had
logging.warning("look out)"
...
ch = logging.StreamHandler(sys.stdout)
root = logging.getLogger()
root.addHandler(ch)
root.info("hello")
I would get something like (ignoring the format options)
look out
hello
hello
and everything was written to stdout twice. I believe this is because the first call to logging.warning creates a new handler automatically, and then I explicitly added another handler. The problem went away when I removed the accidental first logging.warning call.
I was struggling with the same issue in the context of multiple processes. (For the code see the docs which I was following almost verbatim.) Namely, all log messages originating from any of the child processes got duplicated.
My mistake was to call worker_configurer(),
def worker_configurer(logging_queue):
queue_handler = logging.handlers.QueueHandler(logging_queue)
root = logging.getLogger()
root.addHandler(queue_handler)
root.setLevel(level)
both in the child processes and also in the main process (since I wanted the main process to log stuff, too). The reason this led to trouble (on my Linux machine) is that on Linux the child processes got started through forking and therefore inherited the existing log handlers from the main process. That is, on Linux the QueueHandler got registered twice.
Now, preventing the QueueHandler from getting registered twice in the worker_configurer() function is not as trivial as it seems:
Logger objects like the root logger root have a handlers property but it is undocumented.
In my experience, testing whether any([handler is queue_handler for handler in root.handlers]) (identity) or any([handler == queue_handler for handler in root.handlers]) (equality) fails after forking, even if root.handlers seemingly contains the same QueueHandler. (Obviously, the previous two expressions can be abbreviated by queue_handler in root.handlers, since the in operator checks for both identity and equality in the case of lists.)
The root logger gets modified by packages like pytest, so root.handlers and root.hasHandlers() are not very reliable to begin with. (They are global state, after all.)
The clean solution, therefore, is to replace forking with spawning to prevent these kinds of multiprocessing bugs right from the start (provided you can live with the additional memory footprint, of course). Or to use an alternative to the logging package that doesn't rely on global state and instead requires you to do proper dependency injection but I'm digressing… :)
With that being said, I ended up going for a rather trivial check:
def worker_configurer(logging_queue):
queue_handler = logging.handlers.QueueHandler(logging_queue)
root = logging.getLogger()
for handler in root.handlers:
if isinstance(handler, logging.handlers.QueueHandler):
return
root.addHandler(queue_handler)
root.setLevel(level)
Obviously, this will have nasty side effects the second I decide to register a second queue handler somewhere else.
From the docs:
"Loggers have the following attributes and methods. Note that Loggers should NEVER be instantiated directly, but always through the module-level function logging.getLogger(name).
Multiple calls to getLogger() with the same name will always return a reference to the same Logger object".
Make sure you don't initialise your loggers with the same name
I advise you to initialise the logger with __name__ as name param i.e:
import logging
logger = logging.getLogger(__name__)
NOTE:
even if you init a loggers from other modules with same name, you will still get the same logger, therefore calling i.e logger.info('somthing') will log as many times as you initiated the logger class.
I was getting a strange situation where console logs were doubled but my file logs were not. After a ton of digging I figured it out.
Please be aware that third party packages can register loggers. This is something to watch out for (and in some cases can't be prevented). In many cases third party code checks to see if there are any existing root logger handlers; and if there isn't--they register a new console handler.
My solution to this was to register my console logger at the root level:
rootLogger = logging.getLogger() # note no text passed in--that's how we grab the root logger
if not rootLogger.handlers:
ch = logging.StreamHandler()
ch.setLevel(logging.INFO)
ch.setFormatter(logging.Formatter('%(process)s|%(levelname)s] %(message)s'))
rootLogger.addHandler(ch)
If you are using any config for logging, For instance log.conf
In .conf file you can do it by adding this line in the [logger_myLogger] section: propagate=0
[logger_myLogger]
level=DEBUG
handlers=validate,consoleHandler
qualname=VALIDATOR
propagate=0
I had the same issue. In my case, it was not due to handlers or duplicate initial configuration but a stupid typo. In main.py I was using a logger object but in my_tool.py I was directly calling to the logging module by mistake, hence after invoking functions from my_tool module everything was messed up and the messages appeared duplicated.
This was the code:
main.py
import logging
import my_tool
logger_name = "cli"
logger = logging.getLogger(logger_name)
logger.info("potato")
logger.debug("potato)
my_tool.function()
logger.info("tomato")
my_tool.py
import logging
logger_name = "cli"
logger = logging.getLogger(logger_name)
# some code
logging.info("carrot")
and the result
terminal
>> potato
>> potato
>> carrot
>> tomato
>> tomato
Adding to others' useful responses...
In case you are NOT accidentally configuring more than one handler on your logger:
When your logger has ancestors(root logger is always one) and they have their own handlers, they will also output when your logger outputs (by default), which will create duplicates. You have two options:
Don't propagate your log event to your ancestors by setting:
my_logger.propagate = False
If you only have one ancestor(root logger), you could directly configure their handler instead. For example:
# directly change the formatting of root's handler
root_logger = logging.getLogger()
roots_handler = root_logger.handlers[0]
roots_handler.setFormatter(logging.Formatter(': %(message)s')) # change format
my_logger = logging.getLogger('my_logger') # my_logger will use new formatting
If you use the standard construction logger = logging.getLogger('mymodule') and then accidentally mistype loggger as logging i.e.
logger = logging.getLogger('mymodule')
# configure your handlers
logger.info("my info message") # won't make duplicate
logging.info("my info message") # will make duplicate logs
then this will cause duplicate messages to come up because the call to logging creates a new logger.