According to the official logging documentation,
logging.getLogger([name])
Return a logger with the specified name or, if no name is specified, return a logger which is the root logger of the hierarchy. If specified, the name is typically a dot-separated hierarchical name like “a”, “a.b” or “a.b.c.d”. Choice of these names is entirely up to the developer who is using logging.
What is the difference between the name we are passing as argument and the name of the object we are actually catching the logger into?
ie difference between logger and __name__ in the statement
logger = logging.getLogger(__name__)
What is the difference between the name we are passing as argument and the name of the object we are actually catching the logger into?
That's absolutely different things:
logger in your case is the way to access logger object later in your code (most probably, in scope of single file). logger acts as local variable pointing to logger object. You can, actually, call your .log with
getLogger(__name__).info('My data')
but it's just less practical that referencing it by variable name
__name__ is derived name of current module, normally this will be dotted path like project.module.submodule — this can be any string that makes sense to you. __name__ makes a lot of sense usually, so it became sort of convention
One is just a reference - logger, what we call a variable and has nothing to do with the logging process. You could call it babapaloopa and that would not make any difference at all, except make your program cooler when referencing the logging object (though something could be said about good design here). I would stress this is not the name of the object, just the name of a variable that happens to reference the object - you could:
another_reference = logger
and now both another_reference and logger are both pointing to the same logging object, and clearly the variable name is unrelated to the object. Objects do have ids, but that is not something you should tamper with usually.
The name argument will become the actual name associated with the logger, a property of the logger object itself, not just a name of a reference to it - likely stored in the initialization of the object. The fact you use __name__ which is a special string, or some other string is just a detail, the point is this will be the name of the logger from a programming viewpoint.
logger.name
will hold this name after the logger is defined (assuming logger is reference to said logging object of course).
Related
I get a regular non-root logger of the default type:
logger = logging.getLogger('a')
Now I want to change the type and get a new logger for the same name:
logging.setLoggerClass(type('NewLoggerClass', (logging.Logger,), {}))
logger = logging.getLogger('a')
I would like the second call to getLogger to respect the new logger class I set. However, logger remains the same instance of logging.Logger instead of __main__.NewLoggerClass.
How do I either remove an existing logger for a given name, or replace it with one of a different type?
You can't do this (other than low-level hackery) once a logger has been created. You'll need to call setLoggerClass() before you instantiate any loggers via getLogger().
If in doubt ... "Use the source, Luke!"
So spelunking into logging/__init__.py, one can see that a call to logging.getLogger actually invokes Logger.manager.getLogger.
Logger.manager is itself an instance of Manager that was instatiated on module load.
Hunting in the Manager class shows that it checks for existing logger objects in self.loggerDict. This attribute acts as the 'cache' for instantiated logger class.
So if you want to replace a certain logger with a new logger class, you probably can do it by deleting it first. Maybe like this:
del logging.Logger.manager.loggerDict['a']
logging.setLoggerClass(type('NewLoggerClass', (logging.Logger,), {}))
logger = logging.getLogger('a')
Try it out!
As a newbie to python I finally managed to create my own customized logger in a class in an extra module (tools.py). One of the main achievements here is that I can set the name of the logger and with the name also the log file name.
I want to instantiate the logger in main in one module (recalc.py) and from here call a function prepData in another module (getData.py). That function prepData in module getData.py is supposed to use the logger instance I instantiated in recalc.py. prepData is calling other function in the same module that shall use the same logger instance.
I fail to get the grips on how do declare / instantiate the logger since only the in module recalc.py with the main function knows the correct name for the logger.
module tools.py
:
class Logg():
:
.
module recalc.py
import tools
from getData import prepData
:
lg = Logg("recalc", ...) # instantiate the logger
:
result = prepData(...)
.
module getData.py
:
def otherFunc()
lg.debug(...)
:
return X
def prepData(...)
lg.info(...)
:
x = otherFunc(...)
:
return RES
I read a lot about (not really existing) global variables in Python, also Python 3, one class used across multiple modules and others. The config.py solution implies to initiate the global variable in config.py itself. But that doesn't work for me as only the main module knows the logger name.
The workaround I have right now is not acceptable in the long run.
I appreciate any help.
When using logging in python like this
import logging
logging.basicConfig(level=logging.INFO)
logger = None
if <some condition>:
logger = logging.getLogger(__name__)
... some code ...
logger.debug('message')
is it possible to avoid calling logger.debug if it does not exist without if statement?
That's not how you use logging: it's missing the point entirely. The point is to always just log, and then (at the app level) configure handlers appropriately so that they either record the debug level or not. Your app-level configuration can even set the handler for loggers in each module. See the Python logging docs for more info.
You could construct a wrapper object and use that everywhere:
class LoggerWrapper(object):
def __init__(self, logger):
self.logger = logger
def debug(self, *args, **kwargs):
if self.logger is None:
return
self.logger.debug(*args, **kwargs)
my_wrapper = LoggerWrapper(logger)
my_wrapper.debug('message')
That being said, this is not specific to Python's logging module. Any variable var that can also be None will need to be checked before you call a method on it (var.method()).
It may be easier to always define a logger and to simply disable the logging through the logging configuration / settings. That keeps the code clean, all logger calls will succeed but the output won't go anywhere if that particular logger is disabled.
Lastly, you could opt to indeed use an if statement as it's not inherently that much more code:
if logger:
logger.debug('message')
is it possible to avoid calling logger.debug if it does not exist without if statement?
Well, you could (ab)use an if expression, or an or expression, or various other things, but otherwise, not really.
You can, however, always write a wrapper function:
def logdebug(*args):
if logger:
logger.debug(*args)
logdebug('message')
logdebug('other message')
You can do other, similar tricks. For example, instead of having logger = None, you can create an object that has a debug method that does nothing:
class Dummy(object):
def debug(self, *args): pass
logger = Dummy()
Or, even better, always create a logger and configure it with a null handler, or—simplest of all—just set its logging level higher than debug.
But note that none of these will completely replace the if statement. Consider, for example:
if logger:
logger.debug(very_expensive_function())
logdebug(very_expensive_function())
The first one will only call very_expensive_function to create the log message if you actually need it; the second one will always call it and then just throw it away if you don't need it.
For example, i have these files with classes that i want to use (only example, maybe not working)
# helper_one.py
import logging
logger = logging.getLogger('HelperOne')
class HelperOne:
def __init__(self, data):
self.data = data
def run(self):
logger.debug('HelperOne::Run function with data: ' + self.data)
return True
.
# controller_one.py
import logging
from helper_one import HelperOne
logger = logging.getLogger('ControllerOne')
class ControllerOne:
def __init__(self, data):
self.data = data
def run(self):
logger.debug('ControllerOne::Run function with data: ' + self.data)
ho = HelperOne(self.data + '_data')
return ho.run()
And i have a file which creates Threads
import logging
from controller_one import ControllerOne
# this function creates threads (its just an example)
def run_task(task_id, data):
logging.basicConfig(
filename=os.path.join('logs', str(task_id) + '.log'),
level=logging.DEBUG,
format='%(asctime)s\t%(name)s\t%(levelname)s\t%(message)s'
)
result = ControllerOne(data)
logging.debug('run_task: ' + result)
If i create logging instance like in my example, all log are written in a single file.
How can I, for every thread, create its own logging instance with logging to its own file ?
Regards, Alex.
The logger name (the string passed to the getLogger function) can be anything you want. As the docs explain:
The name is potentially a period-separated hierarchical value, like foo.bar.baz (though it could also be just plain foo, for example). Loggers that are further down in the hierarchical list are children of loggers higher up in the list. For example, given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo. The logger name hierarchy is analogous to the Python package hierarchy, and identical to it if you organise your loggers on a per-module basis using the recommended construction logging.getLogger(__name__). That’s because in a module, __name__ is the module’s name in the Python package namespace.
So, using __name__ as the name is recommended, but not required. And in fact you're already explicitly violating it:
logger = logging.getLogger('ControllerOne')
So, you could just use a separate name per thread, by putting a thread ID or name into the logger name. For example:
logger = logging.getLogger('ControllerOne.{}'.format(threading.get_ident())
Or, in your case, since you seem to have a unique task_id for each thread, you can just use that instead of the thread ID.
Of course this means you have to actually use the logger; you can't just call logging.debug. And you can't rely on the basicConfig; you're going to need to explicitly configure the logger in each thread. Since each thread creates its own independent ControllerOne instance, the obvious place to do that is in ControllerOne.__init__; in that function, you can call getLogger with a logger name that incorporates the thread or task ID, create and set a FileHandler that also incorporates the thread or task ID in the name, and store it as self.logger. Then, whenever you need to log something, you just do self.logger.debug(…).
If that all sounds like gobbledegook to you because you have no idea what a FileHandler is, you probably need to read at least the basic tutorial in the HOWTOs, but the advanced tutorial and cookbook are also very useful.
The following dev enviroment is to be considered.
Small number of python modules, each contanining one or more classes. A main module, that calls those classes. A custom logging module called logger with a class called Logger. Assume that I call the main execution class with a logging level of debug. How may I make this sufficient to be that log level inherited to every other call including the rest of the classes, methods in those classes, functions in the main module and so forth...
The Logger objects are called like log=Logger(0) for example (logging level is an int, to use the same syntax that we use in other scripts (shell scripts, not python).
My final picture is to have the code filled with log.log_debug('debug message') and log.log_error('error message') but only actually printing a message when the right log_level is choosen. And if possible just one
from logger import Logger
call within the main module.
Thanks for your time.
==================================
Edit
In the main execution module there is a main() function in which a parser.parse_args() object is returned with an argument --log_level to globally define (at least is my intention) the log_level. There is a default log_level handling (i.e, it is always defined)
I will try to mock a minimum example
import argparse
from logfile from Logfile
from logger import Logger
def argument_parser():
parser=argparse.ArgumentParser()
stuff
return parser.parse_args()
def log_file_stuff():
log_file=Logfile()
log_file.methods() [*]
def main():
args=argument_parser()
# Here log_level is defined with args.log_level
global log
log=log(log_level)
log_file_stuff()
main()
[*] One of those methods may call a Logger object, which I want to be exactly the same as the one defined in the main() function the question is how may I achieve this? without a log return and argument waterfall
Well to answer my own question... I actually implemented a cross module variable dictionary to use it as settings. Those settings have the log_level variable son only with
from main import settings
from logger import Logger
log=Logger(settings['log_level'])
then I have my custom logging class with the user input log_level. Of course settings is constructed with argparse, mainly as vars(args) following the notation in the question.