I'm using a open-source Python library in my project. This library logs a lot of information using the logging class.
...but I can't see the output or log it to file. I know that i would have to create a logger instance and add a file-handler or a console-handler to it but how can i pass this logger instance to the class? Here's the init snippet of the class that I'm going to be using.
class Periscope:
''' Main Periscope class'''
def __init__(self):
self.config = ConfigParser.SafeConfigParser({"lang": "en"})
if is_local:
self.config_file = os.path.join(bd.xdg_config_home, "periscope", "config")
if not os.path.exists(self.config_file):
folder = os.path.dirname(self.config_file)
if not os.path.exists(folder):
logging.info("Creating folder %s" %folder)
os.mkdir(folder)
logging.info("Creating config file")
configfile = open(self.config_file, "w")
self.config.write(configfile)
configfile.close()
else:
#Load it
self.config.read(self.config_file)
self.pluginNames = self.listExistingPlugins()
self._preferedLanguages = None
Any help?
Thanks guys.
Simplest way will be to use basicConfig function in logging module. Here's what docs are saying:
Does basic configuration for the logging system by creating a StreamHandler with a default Formatter and adding it to the root logger. The function does nothing if any handlers have been defined for the root logger. The functions debug(), info(), warning(), error() and critical() will call basicConfig() automatically if no handlers are defined for the root logger.
This function does nothing if the root logger already has handlers configured.
logging module is designed in a way that configuration is separated from creating log messages, so there's no need of having access to logger instance.
Try setting the level to the lowest possible (DEBUG). This enables all log levels and should give you all logging messages. The simplest way to do default configuration is to use basicConfig()
import logging
logging.basicConfig(level=logging.DEBUG, filename='/path/to/mylog.log')
If the library you are using doesn't override the logging configuration this should be enough to get messages into the log file. If you know the name of the logger the library is using, you can set the level for the library specifically:
logging.getLogger("periscope").setLevel(logging.DEBUG)
Related
I had created a custom logger for my purpose using python and made it a utility. I created its context-based and had it created with custom handlers for different scenarios. I am trying to make my custom logger visible across all modules. But I am not able to this. I don't want to reuse these lines in each of my modules just for the logger and pass on my context and config just for that.
logger = myLogger(config, context) # config has data for context based custom handling
In my main module, I just made the logger object global so that other methods can use the logger without any further add ons. Is there any way I can do the same across modules.
In many similar queries. what is suggested is
logger = logging.getLogger(__name__)
But this does not pass on my custom handlers also.
Can someone please advise how I can achieve this.
Make my custom logger global for my whole run time so that I don't have to declare whenever I have to.
my code is like this
def main():
args=argparse.ArgumentParser
parser.add_argument('-context','--context')
parser.add_argument('-cfg','--cfg')
config=configparser.ConfigParser()
config.read(cfg)
global logger
logger=myLogger(config,context)
## here context is my section name from config. which has details for my current process.
##my myLogger reads from a log file config details in configparser onject
## there i will remove my detault handlers and add my custom handlers and return the logger object back to main so this is how it works
## making logger as global in main makes it visible to other methods in same module as main
## but i am trying to make my logger visible to other modules also if i call the methods from those module
if __name__==__main__:
main()
My way of doing it:
Logging.py
import logging
# Your custom stuff
logger = myLogger(config, context)
Every_other_file.py
from Logging import logger
Edit: Change the config later on
Logging.py
import logging
# Your custom stuff
logger = myLogger(config, context)
def change_config(config):
global logger
logger = myLogger(config, context)
def set_logger(config)
global logger
logger = myLogger(config.context)
** main.py**
config = something
Logging.set_logger(config)
Something like that. My point is you can call a method with it you can change the value
I have a root logger with some handlers on it (SysLogHandler lets say).
And I want to get a child logger from the root logger by calling the Logger.getChild(__name__), but when I do so, I got the new logger with no handlers on it. What I'm doing wrong?
Thanks!
import logging
from logging import handlers
sysh = handlers.SysLogHandler()
logger = logging.getLogger()
logger.addHandler(sysh)
logger.handlers # [<SysLogHandler (NOTSET)>] - everything is ok
c = logger.getChild('some_name')
c.handlers # [] where is the handler(SysLogHandler) from parent(root) logger???
getChild does the same thing that logging.getLogger does. Both call the getLogger method on the logging.Manager instance.The manager either returns an existing logger that has been registered in its loggerDict attribute or instantiates a new logger, registers and returns it. In the latter case, the new logger must be configured by the caller.
getChild is not meant to create a clone of the logger it was called on. It is intended as
[...] a convenience method, useful when the parent logger is named using
e.g. __name__ rather than a literal string.
Given this code I try to have log statements working but I am not able to. The documentation tells me that I do not have to set a level.
When a logger is created, the level is set to NOTSET (which causes all
messages to be processed when the logger is the root logger, or
delegation to the parent when the logger is a non-root logger).
But it did not work without. Therefore I tried to set it to debug. But still no luck.
"""
Experimental Port Fowarding
"""
import logging
def main(config):
""" entry point"""
log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)
log.debug("opening config file...")
config_file = open(config, 'r')
log.debug("config found!")
The logger you are getting doesn't have any handlers. You can check this by doing print(log.handlers) and seeing the output is an empty list ([]).
The simplest way to use the logging library is something like this, where you call logging.basicConfig to set everything up, as shown in the logging module basic tutorial:
"""
Experimental Port Fowarding
"""
import logging
logging.basicConfig(level=logging.DEBUG)
def main(config):
""" entry point"""
logging.debug("opening config file...")
config_file = open(config, 'r')
logging.debug("config found!")
main('test.conf')
This works for me from outside and inside IPython.
If you want to avoid basicConfig for some reason you need to register a handler manually, like this:
import logging
def main(config):
""" entry point"""
log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)
# Minimal change: add StreamHandler to display to stdout
log.addHandler(logging.StreamHandler())
log.debug("opening config file...")
config_file = open(config, 'r')
log.debug("config found!")
By default the logger writes to STDERR stream which usually prints to the console itself.
Basically you can change the log file path by setting:
logging.basicConfig(filename="YourFileName.log")
log = logging.getLogger(__name__)
I have been trying to create a new class of Logger by subclassing logging.Logger . Python version is 3.5
I have several modules in my application and I configure the logging only in the main module where I set the logger class using logging.setLoggerClass(...)
However when I retrieve the same Logger instance from some other module, it still creates a new instance of the Logger class and not the child class instance that I defined.
For example my code is :
# module 1
import logging
class MyLoggerClass(logging.getLoggerClass()):
def __init__(name):
super(MyLoggerClass, self).__init__(name)
def new_logger_method(...):
# some new functionality
if __name__ == "__main__":
logging.setLoggerClass(MyLoggerClass)
mylogger = logging.getLogger("mylogger")
# configuration of mylogger instance
# module 2
import logging
applogger = logging.getLogger("mylogger")
print(type(applogger))
def some_function():
applogger.debug("in module 2 some_function")
When this code is executed, I expect the applogger in module 2 to be of type MyLoggerClass. I intend to use the new_logger_method for some new functionality.
However since applogger is turning out to be of type logging.Logger, when the code is run it throws Logger has no attribute named new_logger_method.
Has anyone ever faced this issue?
Thanks in advance for any help!
Pranav
Instead of attempting to affect the global logger by changing the default logger factory, if you want your module to play nicely with any environment you should define a logger just for your module (and its children) and use it as a main logger for everything else deeper in your module structure. The trouble is that you explicitly want to use a different logging.Logger class than the default/globally defined one and the logging module doesn't provide an easy way to do context-based factory switching so you'll have to do it yourself.
There are many ways to do that but my personal preference is to be as explicit as possible and define your own logger module which you'll then import in your other modules in your package whenever you need to obtain a custom logger. In your case, you can create logger.py at the root of your package and do something like:
import logging
class CustomLogger(logging.Logger):
def __init__(self, name):
super(CustomLogger, self).__init__(name)
def new_logger_method(self, caller=None):
self.info("new_logger_method() called from: {}.".format(caller))
def getLogger(name=None, custom_logger=True):
if not custom_logger:
return logging.getLogger(name)
logging_class = logging.getLoggerClass() # store the current logger factory for later
logging._acquireLock() # use the global logging lock for thread safety
try:
logging.setLoggerClass(CustomLogger) # temporarily change the logger factory
logger = logging.getLogger(name)
logging.setLoggerClass(logging_class) # be nice, revert the logger factory change
return logger
finally:
logging._releaseLock()
Feel free to include other custom log initialization logic in it if you so desire. Then from your other modules (and sub-packages) you can import this logger and use its getLogger() to obtain a local, custom logger. For example, all you need in module1.py is:
from . import logger # or `from package import logger` for external/non-relative use
log = logger.getLogger(__name__) # obtain a main logger for this module
def test(): # lets define a function we can later call for testing
log.new_logger_method("Module 1")
This covers the internal use - as long as you stick to this pattern in all your modules/sub-modules you'll have the access to your custom logger.
When it comes to external use, you can write an easy test to show you that your custom logger gets created and that it doesn't interfere with the rest of the logging system therefore your package/module can be declared a good citizen. Under the assumption that your module1.py is in a package called package and you want to test it as a whole from the outside:
import logging # NOTE: we're importing the global, standard `logging` module
import package.module1
logging.basicConfig() # initialize the most rudimentary root logger
root_logger = logging.getLogger() # obtain the root logger
root_logger.setLevel(logging.DEBUG) # set root log level to DEBUG
# lets see the difference in Logger types:
print(root_logger.__class__) # <class 'logging.RootLogger'>
print(package.module1.log.__class__) # <class 'package.logger.CustomLogger'>
# you can also obtain the logger by name to make sure it's in the hierarchy
# NOTE: we'll be getting it from the standard logging module so outsiders need
# not to know that we manage our logging internally
print(logging.getLogger("package.module1").__class__) # <class 'package.logger.CustomLogger'>
# and we can test that it indeed has the custom method:
logging.getLogger("package.module1").new_logger_method("root!")
# INFO:package.module1:new_logger_method() called from: root!.
package.module1.test() # lets call the test method within the module
# INFO:package.module1:new_logger_method() called from: Module 1.
# however, this will not affect anything outside of your package/module, e.g.:
test_logger = logging.getLogger("test_logger")
print(test_logger.__class__) # <class 'logging.Logger'>
test_logger.info("I am a test logger!")
# INFO:test_logger:I am a test logger!
test_logger.new_logger_method("root - test")
# AttributeError: 'Logger' object has no attribute 'new_logger_method'
I have python project with multiple modules with logging. I perform initialization (reading log configuration file and creating root logger and enable/disable logging) in every module before start of logging the messages. Is it possible to perform this initialization only once in one place (like in one class may be called as Log) such that the same settings are reused by logging all over the project?
I am looking for a proper solution to have only once to read the configuration file and to only once get and configure a logger, in a class constructor, or perhaps in the initializer (__init__.py). I don't want to do this at client side (in __main__ ). I want to do this configuration only once in separate class and call this class in other modules when logging is required.
setup using #singleton pattern
#log.py
import logging.config
import yaml
from singleton_decorator import singleton
#singleton
class Log:
def __init__(self):
configFile = 'path_to_my_lof_config_file'/logging.yaml
with open(configFile) as f:
config_dict = yaml.load(f)
logging.config.dictConfig(config_dict)
self.logger = logging.getLogger('root')
def info(self, message):
self.logger.info(message)
#module1.py
from Log import Log
myLog = Log()
myLog.info('Message logged successfully)
#module2.py
from Log import Log
myLog = Log() #config read only once and only one object is created
myLog.info('Message logged successfully)
From the documentation,
Note that Loggers should NEVER be instantiated directly, but always through the module-level function logging.getLogger(name). Multiple calls to getLogger() with the same name will always return a reference to the same Logger object.
You can initialize and configure logging in your main entry point. See Logging from multiple modules in this Howto (Python 2.7).
I had the same problem and I don't have any classes or anything, so I solved it with just using global variable
utils.py:
existing_loggers = {}
def get_logger(name='my_logger', level=logging.INFO):
if name in existing_loggers:
return existing_loggers[name]
# Do the rest of initialization, handlers, formatters etc...