I know how to suppress log messages up to a certain log level from an imported module:
import logging
logging.getLogger("module_name").setLevel(logging.WARNING)
I want to know if it's possible to suppress log messages from only a specific function/class in an imported module (and keep all other messages in the module).
I tried this:
logging.getLogger("module_name.function_name").setLevel(logging.WARNING)
but it didn't work. Is it possible?
I came up with this in the end based on logging with filters:
Contents of my_module.py:
import logging
logger = logging.getLogger(__name__)
def func1():
logger.debug('Running func1!')
def func2():
logger.debug('Running func2!')
I want to ignore any messages from func2, but keep any messages from func1.
Contents of main.py:
import logging
import my_module
logging.basicConfig(level=logging.DEBUG)
class IgnoreFunc2(logging.Filter):
def filter(self, record):
return not record.funcName == 'func2'
# this relies on knowing the variable name in my_module
my_module.logger.addFilter(IgnoreFunc2())
def main():
my_module.func1()
my_module.func2()
if __name__ == '__main__':
main()
Output:
DEBUG:my_module:Running func1!
Related
I'm trying to write a file python modules and I need to log the output of different module to different files.
module_a.py
import logging
logging.basicConfig(levle = logging.INFO, filename="a.log")
logger = logging.getLogger(__name__)
def function_a():
logger.info("this is function a")
if __name__ == "__main__":
logger.info("this is module a")
function_a()
module_b.py
import logging
import module_a
logging.basicConfig(levle = logging.INFO, filename="b.log")
logger = logging.getLogger(__name__)
def function_b():
module_a.function_a()
logger.info("this is function b")
if __name__ == "__main__":
logger.info("this is module b")
function_b()
What I want to do is, I wants module_b.py always log to b.log, and
module_a.py always log to a.log, no matter the module was called or imported and then called.
The issue I got now is,
If I run
python3 module_a.py
All good.
If I run
python3 module_b.py
then all logs will be logged into a.log. I understand when import module_a, logger got overwritten. So how can make sure these two modules can always log into the right files?
Thanks
Normally, logging rules are decided by top level scripts. Imported modules just log without worrying about where the logs go. You want something different. So stay clear of basicConfig which is just a helper, and have each module setup its own logger.
In this example, I pulled the setup into a function. Its an easy way to get rid of temporary variables like handler. You may want to add a formatter while you are at it. The function is the same in both scripts. You may want to pull it into a single common utilities module. Or maybe there is a reason to do things differently in different modules. There are so many ways to setup loggers that the function is really just a rough guess of what you'll do in your project.
module_a
import logging
def make_logger(name):
handler = logging.FileHandler(filename=name + ".log")
logger = logging.getLogger(name)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
return logger
logger = make_logger("a")
def function_a():
logger.info("this is function a")
if __name__ == "__main__":
logger.info("this is module a")
function_a()
module_b
import logging
import module_a
def make_logger(name):
handler = logging.FileHandler(filename=name + ".log")
logger = logging.getLogger(name)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
return logger
logger = make_logger("b")
def function_b():
module_a.function_a()
logger.info("this is function b")
if __name__ == "__main__":
logger.info("this is module b")
function_b()
Env Summary:
Hi, I have a Python project, where I request some information from a storage system and display this on screen.
Env:
module executer
Module rest has a class RestThreePar
Module storagebox has a class StorageBox
Module locater has a class Locater
Problem:
I want to implement a single module/class for logging for the hole project, which needs to log stuff to 4 different files (each module has it's own log file). When I import logging separately for each module it works like expected but whenever I try to make a separate class and import that one (like from logger import Logger) it goes wrong (I am not sure but it looks like it writes one line multiple times in the files).
I am not a developer and don't know what to do next.
I have tried to use the python documentation but had no success.
https://docs.python.org/3.6/howto/logging-cookbook.html
import logging
import os
class Logger:
def __init__(self, cls_name, file):
self.cls_name = cls_name
current_dir = os.getcwd()
self.file = f'{current_dir}/logs/{file}'
def log(self):
logger = logging.getLogger(self.cls_name)
logger.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s,%(levelname)s,%(name)s,%(message)s')
file_handler = logging.FileHandler(self.file)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
return logger
def log_info(self, msg):
logger = self.log()
logger.info(msg)
def log_exception(self, msg):
logger = self.log()
logger.exception(msg)
if __name__ == '__main__':
pass
I want to find out how logging should be organised given that I write many scripts and modules that should feature similar logging. I want to be able to set the logging appearance and the logging level from the script and I want this to propagate the appearance and level to my modules and only my modules.
An example script could be something like the following:
import logging
import technicolor
import example_2_module
def main():
verbose = True
global log
log = logging.getLogger(__name__)
logging.root.addHandler(technicolor.ColorisingStreamHandler())
# logging level
if verbose:
logging.root.setLevel(logging.DEBUG)
else:
logging.root.setLevel(logging.INFO)
log.info("example INFO message in main")
log.debug("example DEBUG message in main")
example_2_module.function1()
if __name__ == '__main__':
main()
An example module could be something like the following:
import logging
log = logging.getLogger(__name__)
def function1():
print("printout of function 1")
log.info("example INFO message in module")
log.debug("example DEBUG message in module")
You can see that in the module there is minimal infrastructure written to import the logging of the appearance and the level set in the script. This has worked fine, but I've encountered a problem: other modules that have logging. This can result in output being printed twice, and very detailed debug logging from modules that are not my own.
How should I code this such that the logging appearance/level is set from the script but then used only by my modules?
You need to set the propagate attribute to False so that the log message does not propagate to ancestor loggers. Here is the documentation for Logger.propagate -- it defaults to True. So just:
import logging
log = logging.getLogger(__name__)
log.propagate = False
I want to create a Python logging object in my main program and have logging in both my main program and in the modules it uses at the same logging level. The basic example given in logging documentation is essentially as follows:
main.py:
import logging
import mylib
def main():
logging.basicConfig(level = logging.INFO)
logging.info('Started')
mylib.do_something()
logging.info('Finished')
if __name__ == '__main__':
main()
mylib.py:
import logging
def do_something():
logging.info('Doing something')
This works fine. I am not sure, however, of how to get a Python logging object doing something similar. The following, for example, does not work:
main.py:
import logging
import mylib
def main():
verbose = True
global log
log = logging.getLogger(__name__)
if verbose:
log.setLevel(logging.INFO)
else:
log.setLevel(logging.DEBUG)
log.info('Started')
mylib.do_something()
log.info('Finished')
if __name__ == '__main__':
main()
mylib.py:
import logging
def do_something():
log.info('Doing something')
It does not work because the global log object is not recognised in the mylib.py module. How should I be doing this? My two main goals are
to have the logging level that is set in my main program propagate through to any modules used and
to be able to use log for logging, not "logging" (i.e. log.info("alert") as opposed to logging.info("alert")).
Your application should configure the logging once (with basicConfig or see logging.config) and then in each file you can have:
import logging
log = logging.getLogger(__name__)
# further down:
log.info("alert")
So you use a logger everywhere and you don't directly access the logging module in your codebase. This allows you to configure all your loggers in your logging configuration (which, as said earlier, is configured once and loaded at the beginning of your application).
You can use a different logger in each module as follows:
import logging
LOG = logging.getLogger(__name__)
# Stuff
LOG.info("A log message")
trying to understand and learn how to write packages... testing with something i've always used, logging...
Can you please help me understand why the 'log' variable is not working... and no logging is working on the screen?
Thanks!
main.py :
#!/opt/local/bin/python
import sys
sys.path.append('CLUSTER')
import clusterlogging.differentlogging
clusterlogging.differentlogging.consolelogging()
log.debug("Successfully logged in")
differentlogging.py
#!/opt/local/bin/python
def consolelogging():
import logging
class NullHandler(logging.Handler):
def emit(self, record):
pass
print "Console Logging loaded"
DEFAULTLOGLEVEL=logging.INFO
log = logging.getLogger(__name__)
log.addHandler(NullHandler())
log.debug("Successfully logged in")
def mysqllogging():
print "mysql logging module here"
def sysloglogging():
print "rsyslog logging module here"
output
Console Logging loaded
Traceback (most recent call last):
File "./svnprod.py", line 10, in <module>
log.debug("Successfully logged in")
NameError: name 'log' is not defined
log is a global variable in the differentlogging module. Thus you can access it as
clusterlogging.differentlogging.log.
You could also do something like from clusterlogging.differentlogging import log and then access it as just log.
Edit: actually, on reviewing your code again I don't know what to make of it. Could you please fix up your code indentation so that it makes sense? Are you defining log inside the consolelogging function? If so, you'll need to either make it global with global log or return it from the function and assign it to a variable log on the line where you call the function.
This will return the log array, and you will be able to use the logging function associated.
main.py:
#!/usr/bin/env python
import sys
sys.path.append('CLUSTER')
import clusterlogging.differentlogging
log=clusterlogging.differentlogging.ttylogging()
log.debug("Logging module loaded")
log.info ("It worked")
differentlogging.py :
#!/usr/bin/env python
def ttylogging():
print "Console Logging loaded"
import sys
import logging
class NullHandler(logging.Handler):
def emit(self, record):
pass
DEFAULTLOGLEVEL=logging.INFO
log = logging.getLogger(__name__)
log.addHandler(NullHandler())
log.setLevel(DEFAULTLOGLEVEL)
logStreamHandler = logging.StreamHandler(sys.stdout)
logStreamHandler.setFormatter(logging.Formatter("%(asctime)s %(levelname)5s %(name)s %(lineno)d: %(message)s"))
log.addHandler(logStreamHandler)
return log
def mysqllogging():
print "mysql logging module here"
def sysloglogging():
print "rsyslog logging module here"
Your main.py doesn't do anything to define the name log in the global namespace. Importing a module can define names in the namespace of that module, but can't put anything in the global namespace.
In your main.py you should add this statement:
from clusterlogging.differentlogging import log
By the way, I that is such a long module name, I would use import as:
import clusterlogging.differentlogging as difflogging
log = difflogging.log
EDIT: I originally recommended this but it won't work:
from difflogging import log # doesn't work
You might even want to use a really short name like dl:
import clusterlogging.differentlogging as dl
dl.log('whatever')
Since dl is really short, maybe you don't need to get log bound in the global namespace.
Also, you could get every name from a module by using import * but this is not recommended.
from clusterlogging.differentlogging import * # not recommended
You usually don't want to clutter the global namespace with all the stuff defined in a module. Import just what you need. This is tidier and helps document what you are actually using.