The following dev enviroment is to be considered.
Small number of python modules, each contanining one or more classes. A main module, that calls those classes. A custom logging module called logger with a class called Logger. Assume that I call the main execution class with a logging level of debug. How may I make this sufficient to be that log level inherited to every other call including the rest of the classes, methods in those classes, functions in the main module and so forth...
The Logger objects are called like log=Logger(0) for example (logging level is an int, to use the same syntax that we use in other scripts (shell scripts, not python).
My final picture is to have the code filled with log.log_debug('debug message') and log.log_error('error message') but only actually printing a message when the right log_level is choosen. And if possible just one
from logger import Logger
call within the main module.
Thanks for your time.
==================================
Edit
In the main execution module there is a main() function in which a parser.parse_args() object is returned with an argument --log_level to globally define (at least is my intention) the log_level. There is a default log_level handling (i.e, it is always defined)
I will try to mock a minimum example
import argparse
from logfile from Logfile
from logger import Logger
def argument_parser():
parser=argparse.ArgumentParser()
stuff
return parser.parse_args()
def log_file_stuff():
log_file=Logfile()
log_file.methods() [*]
def main():
args=argument_parser()
# Here log_level is defined with args.log_level
global log
log=log(log_level)
log_file_stuff()
main()
[*] One of those methods may call a Logger object, which I want to be exactly the same as the one defined in the main() function the question is how may I achieve this? without a log return and argument waterfall
Well to answer my own question... I actually implemented a cross module variable dictionary to use it as settings. Those settings have the log_level variable son only with
from main import settings
from logger import Logger
log=Logger(settings['log_level'])
then I have my custom logging class with the user input log_level. Of course settings is constructed with argparse, mainly as vars(args) following the notation in the question.
Related
According to the official logging documentation,
logging.getLogger([name])
Return a logger with the specified name or, if no name is specified, return a logger which is the root logger of the hierarchy. If specified, the name is typically a dot-separated hierarchical name like “a”, “a.b” or “a.b.c.d”. Choice of these names is entirely up to the developer who is using logging.
What is the difference between the name we are passing as argument and the name of the object we are actually catching the logger into?
ie difference between logger and __name__ in the statement
logger = logging.getLogger(__name__)
What is the difference between the name we are passing as argument and the name of the object we are actually catching the logger into?
That's absolutely different things:
logger in your case is the way to access logger object later in your code (most probably, in scope of single file). logger acts as local variable pointing to logger object. You can, actually, call your .log with
getLogger(__name__).info('My data')
but it's just less practical that referencing it by variable name
__name__ is derived name of current module, normally this will be dotted path like project.module.submodule — this can be any string that makes sense to you. __name__ makes a lot of sense usually, so it became sort of convention
One is just a reference - logger, what we call a variable and has nothing to do with the logging process. You could call it babapaloopa and that would not make any difference at all, except make your program cooler when referencing the logging object (though something could be said about good design here). I would stress this is not the name of the object, just the name of a variable that happens to reference the object - you could:
another_reference = logger
and now both another_reference and logger are both pointing to the same logging object, and clearly the variable name is unrelated to the object. Objects do have ids, but that is not something you should tamper with usually.
The name argument will become the actual name associated with the logger, a property of the logger object itself, not just a name of a reference to it - likely stored in the initialization of the object. The fact you use __name__ which is a special string, or some other string is just a detail, the point is this will be the name of the logger from a programming viewpoint.
logger.name
will hold this name after the logger is defined (assuming logger is reference to said logging object of course).
I have the following application structure:
./utils.py
def do_something(logger=None):
if not logger:
logger = logging.getLogger(__name__)
print('hello')
logger.debug('test')
./one.py
from utils import do_something
logger = logging.getLogger(__name__)
do_something(logger=logger)
./two.py
from utils import do_something
logger = logging.getLogger(__name__)
do_something(logger=logger)
Now, then this runs, the logging output will show the names of the respective modules that are using the functions (one and two) rather than utils. I then use this information (the logger's name) to filter the messages.
Is there a way to do this without having to pass the logger though as an argument? Would I have to basically introspect the caller function and initialize a logger based on that? What's the common solution for this type of recipe, when one function is used all over the code base but the loggers should be of those calling the function?
Variable logger being a global, can be accessed from inside do_something() function like this:
logger = logging.getLogger(__name__)
def do_something():
x = logger
After reading this carefully:
What's the common solution for this type of recipe, when one function
is used all over the code base but the loggers should be of those
calling the function?
In simple english would be:
How to access global variable logger from imported function do_something()?
I conclude that there is no other way!
You have to pass logger as an argument for this particular case.
from utils import do_something
logger = logging.getLogger(__name__)
do_something() # logger global is not visible in do_something()
do_something(logger=logger) # the only way
Passing the logger as an argument looks fine to me. But if you have many calls to do_something then passing the same logger in each call is not only too verbose but prone to errors --you might pass the wrong logger by mistake. Also, while you do need to make do_something aware of the logger, doing it at the point where you are calling it probably does not make sense --you want to do something, not mess with loggers. OOP to the rescue: make do_something a method of a class and pass the logger when instantiating the class, then you do not need to pass it to the method.
# utils.py
class UsefulObject:
def __init__(self, logger):
self._logger = logger
def do_something(self):
print('hello')
self._logger.debug('test')
Now in the client module you create an instance of UsefulObject and you can call .do_something on it without passing the logger every time.
# one.py
from utils import UsefulObject
logger = logging.getLogger(__name__)
useful_object = UsefulObject(logger) # do this once
# ...
useful_object.do_something() # do this many times, do not pass the logger
If you don't like the existing solutions, here's a magic way to do it:
def do_something():
logger = inspect.currentframe().f_back.f_globals['log']
...
The calling function must then have a global variable called log.
As a newbie to python I finally managed to create my own customized logger in a class in an extra module (tools.py). One of the main achievements here is that I can set the name of the logger and with the name also the log file name.
I want to instantiate the logger in main in one module (recalc.py) and from here call a function prepData in another module (getData.py). That function prepData in module getData.py is supposed to use the logger instance I instantiated in recalc.py. prepData is calling other function in the same module that shall use the same logger instance.
I fail to get the grips on how do declare / instantiate the logger since only the in module recalc.py with the main function knows the correct name for the logger.
module tools.py
:
class Logg():
:
.
module recalc.py
import tools
from getData import prepData
:
lg = Logg("recalc", ...) # instantiate the logger
:
result = prepData(...)
.
module getData.py
:
def otherFunc()
lg.debug(...)
:
return X
def prepData(...)
lg.info(...)
:
x = otherFunc(...)
:
return RES
I read a lot about (not really existing) global variables in Python, also Python 3, one class used across multiple modules and others. The config.py solution implies to initiate the global variable in config.py itself. But that doesn't work for me as only the main module knows the logger name.
The workaround I have right now is not acceptable in the long run.
I appreciate any help.
I have the following application structure:
./utils.py
def do_something(logger=None):
if not logger:
logger = logging.getLogger(__name__)
print('hello')
logger.debug('test')
./one.py
from utils import do_something
logger = logging.getLogger(__name__)
do_something(logger=logger)
./two.py
from utils import do_something
logger = logging.getLogger(__name__)
do_something(logger=logger)
Now, then this runs, the logging output will show the names of the respective modules that are using the functions (one and two) rather than utils. I then use this information (the logger's name) to filter the messages.
Is there a way to do this without having to pass the logger though as an argument? Would I have to basically introspect the caller function and initialize a logger based on that? What's the common solution for this type of recipe, when one function is used all over the code base but the loggers should be of those calling the function?
Variable logger being a global, can be accessed from inside do_something() function like this:
logger = logging.getLogger(__name__)
def do_something():
x = logger
After reading this carefully:
What's the common solution for this type of recipe, when one function
is used all over the code base but the loggers should be of those
calling the function?
In simple english would be:
How to access global variable logger from imported function do_something()?
I conclude that there is no other way!
You have to pass logger as an argument for this particular case.
from utils import do_something
logger = logging.getLogger(__name__)
do_something() # logger global is not visible in do_something()
do_something(logger=logger) # the only way
Passing the logger as an argument looks fine to me. But if you have many calls to do_something then passing the same logger in each call is not only too verbose but prone to errors --you might pass the wrong logger by mistake. Also, while you do need to make do_something aware of the logger, doing it at the point where you are calling it probably does not make sense --you want to do something, not mess with loggers. OOP to the rescue: make do_something a method of a class and pass the logger when instantiating the class, then you do not need to pass it to the method.
# utils.py
class UsefulObject:
def __init__(self, logger):
self._logger = logger
def do_something(self):
print('hello')
self._logger.debug('test')
Now in the client module you create an instance of UsefulObject and you can call .do_something on it without passing the logger every time.
# one.py
from utils import UsefulObject
logger = logging.getLogger(__name__)
useful_object = UsefulObject(logger) # do this once
# ...
useful_object.do_something() # do this many times, do not pass the logger
If you don't like the existing solutions, here's a magic way to do it:
def do_something():
logger = inspect.currentframe().f_back.f_globals['log']
...
The calling function must then have a global variable called log.
When using logging in python like this
import logging
logging.basicConfig(level=logging.INFO)
logger = None
if <some condition>:
logger = logging.getLogger(__name__)
... some code ...
logger.debug('message')
is it possible to avoid calling logger.debug if it does not exist without if statement?
That's not how you use logging: it's missing the point entirely. The point is to always just log, and then (at the app level) configure handlers appropriately so that they either record the debug level or not. Your app-level configuration can even set the handler for loggers in each module. See the Python logging docs for more info.
You could construct a wrapper object and use that everywhere:
class LoggerWrapper(object):
def __init__(self, logger):
self.logger = logger
def debug(self, *args, **kwargs):
if self.logger is None:
return
self.logger.debug(*args, **kwargs)
my_wrapper = LoggerWrapper(logger)
my_wrapper.debug('message')
That being said, this is not specific to Python's logging module. Any variable var that can also be None will need to be checked before you call a method on it (var.method()).
It may be easier to always define a logger and to simply disable the logging through the logging configuration / settings. That keeps the code clean, all logger calls will succeed but the output won't go anywhere if that particular logger is disabled.
Lastly, you could opt to indeed use an if statement as it's not inherently that much more code:
if logger:
logger.debug('message')
is it possible to avoid calling logger.debug if it does not exist without if statement?
Well, you could (ab)use an if expression, or an or expression, or various other things, but otherwise, not really.
You can, however, always write a wrapper function:
def logdebug(*args):
if logger:
logger.debug(*args)
logdebug('message')
logdebug('other message')
You can do other, similar tricks. For example, instead of having logger = None, you can create an object that has a debug method that does nothing:
class Dummy(object):
def debug(self, *args): pass
logger = Dummy()
Or, even better, always create a logger and configure it with a null handler, or—simplest of all—just set its logging level higher than debug.
But note that none of these will completely replace the if statement. Consider, for example:
if logger:
logger.debug(very_expensive_function())
logdebug(very_expensive_function())
The first one will only call very_expensive_function to create the log message if you actually need it; the second one will always call it and then just throw it away if you don't need it.