I am importing a third party module that I don't have control over and can't change the source. It does a logging.basicConfig() which is messing up my own logger configuration. I have tried giving a different 'name' to my logger using self.logger = logging.getLogger('some_random_name') which is resulting in every log message printed twice, once with my format, and once with the format set by the basicConfig in that third party module.
Is there a way to ignore the basicConfig from the imported module?
logging.basicConfig() implicitly add handlers to the root logger, so you can just delete those handlers by logging.root.handlers = [].
Further, the root logger may set to an unwanted level, you can also simply set it by logging.root.setLevel(what_ever_you_want).
Even further, if you call logging.info, logging.error etc. without configuring the root logger first, a basicConfig() is called internally, and so a StreamHandler will be implicitly added to the root logger.
Related
I want to print all logging messages from all imported modules. Certain imported modules are not logging.
Note that all the files in the libraries I care about have calls
logger = logging.getLogger(__name__) at the top.
Sorry if this is an easy problem. I've looked through a lot of posts without success.
I observe that the loggers for some modules are not being updated by the call to basicConfig
import logging
import local_util # a local util file
from transformers import T5ForConditionalGeneration
for n in logging.root.manager.loggerDict:
print(logging.getLogger(n))
logging.basicConfig(level=logging.DEBUG)
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
formatter = logging.Formatter("[%(asctime)s] %(levelname)s::%(module)s::%(funcName)s() %(message)s")
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
for n in logging.root.manager.loggerDict:
print(logging.getLogger(n))
In the first call to print(logging.getLogger(n)) almost all loggers are set with level WARNING.
In the second call, most loggers are set to DEBUG (including the one for local_util), except for the transformers library which all remain at level WARNING.
I can get transformers messages to print if I manually cycle through all loggers and reset their levels. Even if I use force=True in the call to basicConfig, the loggers for the transformers library do not get updated.
basicConfig means to affect the root logger. If other child loggers don't set their level (which is default to NOTSET), they use the level of the root logger.
So if transformers has set a WARNING level on its own, it won't use DEBUG level of root.
You can set its level directly as the following:
transformers_logger = logging.getLogger('transformers')
transformers_logger.setLevel(logging.DEBUG)
I've imported a python module, apex (from NVIDIA), which uses its own logger. Unfortunately those logged messages are not getting caught by my main logger, which also writes to a file.
import logging
logger = logging.getLogger(__name__) # my main logger
what I need to do is (somehow) run logging.getLogger("apex.amp") so that I can attach the apex logger to my main logger, and catch the corresponding warnings, etc.
How do I combine the loggers?
You can assign your logger to the 'apex.amp' key of the loggerDict dict of the manager object:
logging.manager.loggerDict['apex.amp'] = logger
I'm importing a module that is logging information at a warning level. I think that the person who wrote this code is logging at a root level i.e. in the code is just:
import logging
logging.warn("foo")
I've tried the below code but it doesn't work, probably because the logging is sent to root or something.
logging.getLogger(module).setLevel(logging.ERROR)
Is there a way that I could disable this module's specific logging?
There are several things confusing here:
logging.getLogger(module).setLevel(logging.ERROR)
The ‘module’ here should be a string, and is usually the full-qualified name of the module. Ie: package.module.
Depending of the format, the name of the Logger is usually printed in the log. That way you can disable it easily.
For instance, if you have something like:
WARNING [foo.bar.baz] the message
The logger name should be foo.bar.baz.
But, if you think it is the root logger, then you can try:
logger = logging.getLogger()
logger .setLevel(logging.ERROR)
Another thing which can be confusing, is the Python warnings. Take a look at this answer to disable them: https://stackoverflow.com/a/14463362
My project is composed by a main.py main script and a module aaa.py . I've succesfully setup a log procedure by using the logging package in main.py, also specifying a log filename.
Which is the cleanest/"correct" way to let the functions contained in aaa.py to write in the same log file?
Use the root logger in main.py by defining
logger = logging.getLogger()
fh = logging.FileHandler("path/to/file")
logger.addHandler(fh)
Then use a module logger in aaa.py by defining
logger = logging.getLogger(__name__)
The logger does not have to have the same name as the module, but it is common practice. The submodule logger will automatically bubble up to the root logger and be sent to any handlers.
To quote from the docs
Child loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed.
If you call only a single python script at once (main.py)
then you simply can define your logging config once; for exemple:
logging.basicConfig(filename=logfilepath, format='%(levelname)s:%(message)s')
and call it wherever you want in the modules, for exemple
logging.<level>("log_string")
To change the logging level of a dependent package that properly names its logger log = logging.getLogger(__name__) is easy: logging.getLogger("name.of.package").setLevel(logging.WARNING).
But if the 3rd party package doesn't name their logger and just logs messages using logging.info("A super loud annoying message!"), how do I change that level? Adding the getLogger(...).setLevel(..) doesn't seem to work because the logger isn't named. Is it possible to change the logging level output of just one package without changing the level for the entire logging module?
If the logger is not named, it just means it is the default logger. You can get it by calling logging.getLogger()
So to set the log level, do this:
logging.getLogger.setLevel(logging.INFO)