I want to write into two log files by using two loggers with following .ini config file:
[loggers]
keys=root,teja
[handlers]
keys=fileHandler,tejaFileHandler
[formatters]
keys=simpleFormatter
[logger_teja]
level=DEBUG
handlers=tejaFileHandler
qualname='tejaLogger'
[logger_root]
level=DEBUG
handlers=fileHandler
[handler_fileHandler]
class=logging.FileHandler
level=DEBUG
formatter=simpleFormatter
args=("error.log", "a")
[handler_tejaFileHandler]
class=logging.FileHandler
level=DEBUG
formatter=simpleFormatter
args=("teja.log", "a")
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
And I am using this configuration in my python code as
import logging
import logging.config
# load my module
import my_module
# load the logging configuration
logging.config.fileConfig('logging.ini')
logger1=logging.getLogger('root')
logger1.info('Hi how are you?')
logger2=logging.getLogger('teja')
logger2.debug('checking teja logger?')
I see that logs are written to error.log file whereas no logs are written to teja.log file. Please correct me if I am doing something silly...
You named your logger object 'tejaLogger':
[logger_teja]
level=DEBUG
handlers=tejaFileHandler
qualname='tejaLogger'
# ^^^^^^^^^^^^
Note that the quotes are part of the name.
but your test code picks up teja instead:
logger2=logging.getLogger('teja')
Rename one or the other; although you could use logging.getLogger("'tejaLogger'")
you probably want to drop the quotes and / or rename the logger to what you expected it to be:
[logger_teja]
level=DEBUG
handlers=tejaFileHandler
qualname=teja
It turns out that the problem is on this line (in the [logger_teja] section):
qualname='tejaLogger'
If you add this to your code (it prints all the current loggers):
print(logging.Logger.manager.loggerDict)
You get:
{"'tejaLogger'": <logging.Logger object at 0x7f89631170b8>}
Which means that your logger is called literally 'tejaLogger'. Using:
logger2=logging.getLogger("'tejaLogger'")`
Actually works fine. Either do that, or change qualname='tejaLogger' to qualname=teja.
Related
I am trying to setup a logging configuration to use in my different modules. I have followed different tutorials and stackoverflow posts (here, here and here) to write logs in to a project.log file.
While the information is displayed correctly in the console, the log.conf is read correctly, the project.log is created but is not filled with the warning messages.
Here is how I proceeded:
The log.conf file used to write up the handlers and formatting:
[loggers]
keys=root,sLogger
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=fileFormatter,consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_sLogger]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=WARNING
formatter=fileFormatter
args=('%(logfilename)s', 'w')
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
In the main.py file:
import logging
import logging.config
def main():
logging.config.fileConfig(fname='./log.conf',
defaults={'logfilename': 'project.log')},
disable_existing_loggers=False)
logger = logging.getLogger(__name__)
logger.warning('This is a message')
In my module.py file:
import logging
logger = logging.getLogger(__name__)
logger.warning('Example of a warning message')
I found the problem ! I didn't realize but when defining handlers in the log.conf file, the order of the handlers and formatters is important !
I changed the log.conf changed to:
[loggers]
keys=root,sLogger
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=consoleFormatter, fileFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler, fileHandler
[logger_sLogger]
level=DEBUG
handlers=consoleHandler, fileHandler
qualname=sLogger
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=WARNING
formatter=fileFormatter
args=('%(logfilename)s','a')
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[formatter_consoleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
NOTE the changed order between consoleFormatter and fileHandler in the [formatters] section.
The rest is working perfectly fine.
To make it easier to play around with I put it in a function (which is probably redundant I suppose):
def base_logger(path_conf, fname, savedir):
'''
Base logging function to be started once
in the main script then used in every other
modules.
Args:
- path_conf: str, path to the configuration
file {log.conf}
- fname: str, name of the saved log file
- savedir: str, path of the saving directory
Returns:
- logging object to be used in the other scripts
Example:
In main.py:
main():
base_logger(paht_cong = './log.conf', fname='file', savedir='project)
In module.py:
# AFTER all the import
logger = logging.getLogger(__name__) # To ge the module name reported in the log file
...
logger.error(f'Error message regarding variable {var}')
The logger initialization {logger = logging.getLogger(__name__)},
has to be done in every file that will use logging !
'''
logging.config.fileConfig(fname=path_conf,
defaults={'logfilename': os.path.join(savedir, f'{fname}.log')},
disable_existing_loggers=False)
I would add that the logger = logging.getLogger(__name__) has to be placed after all imports to make sure that the module name displayed in the log is not imported from other modules.
I have a following python logging configuration file which
[loggers]
keys=root,paramiko
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=consoleFormatter,fileFormatter
[logger_paramiko]
level=CRITICAL
handlers=consoleHandler,fileHandler
qualname=paramiko
[logger_root]
level=DEBUG
handlers=consoleHandler,fileHandler
[handler_consoleHandler]
class=StreamHandler
level=INFO
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=fileFormatter
args=('run.log', 'w')
[formatter_consoleFormatter]
format=[%(asctime)s - %(levelname)s] %(message)s
[formatter_fileFormatter]
format=[%(asctime)s - %(pathname)s:%(lineno)s - %(levelname)s] %(message)s
As you can see I'm logging to console and to file. The name of the file is run.log. I want to be able to append/prepend to the file name a timestamp, i.e. name my log file as 2019-08-08__18:13:40-run.log. I searched online, but couldn't find anything. How can I do it through a configuration file?
The way that python reads the args key from the config is to run it through eval() so you can put any valid code there. You can put pretty much anything you want there, something that is very likely different on every run would be this:
args=(str(time.time())+'.log', 'w') # needs import time in the code
args=(str(hash(' '))+'.log', 'w') # works without import, 99% chance to be unique
I configure logging in all modules in a project like this:
import logging
import logging.config
logging.config.fileConfig('../logging.ini',defaults={'name':__name__})
logging.getLogger("cassandra").setLevel("WARNING") #move this line into the logging.ini
Now I would like to move the last line into the following config file. What is the best way of doing this? Right now I copy/pasted this line into each module, although I have a shared config-file. :S
Configs I found give only examples for self-created loggers, but not overwriting properties of imported loggers.
logging.ini
[loggers]
keys=root
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=INFO
handlers=consoleHandler,fileHandler
[handler_consoleHandler]
class=StreamHandler
level=INFO
formatter=simpleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=simpleFormatter
args=('../logs/%(name)s.log',)
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
Try this in your logging.ini:
[loggers]
keys=root,cassandra
[logger_cassandra]
level=WARNING
handlers=consoleHandler,fileHandler
qualname=cassandra
With this config you can write your modules like
import logging.config
logging.config.fileConfig('logging.ini', defaults={'name': __name__})
logging.warning("this is my logging message")
Note: I recommend the usage of logging.dictConfig. It appears to be more straight forward to me and offers a many options for your configuration. You maybe want to checkout this example dict config or even this with colored console logging.
I have configured logging in logging.config file. I have created a class where I access this configuration file, enable/disable the logger, and log some Info messages. I am importing this class in all the modules where I need to do some logging. When I try to log to a file, I get this error message. I am not able to understand what this error means.
File "/usr/local/lib/python3.6/configparser.py", line 959, in
getitem raise KeyError(key) KeyError: 'formatters'
logging.config
[loggers]
keys=root
[handlers]
keys=fileHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=INFO
handlers=fileHandler
[handler_fileHandler]
class=FileHandler
level=INFO
formatter=simpleFormatter
args=('example.log','a')
[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
#Log.py
import logging.config
class Monitor(object):
fileName = path.join(path.split(path.dirname(path.abspath(__file__)))[0], "logging.config")
print (fileName) #prints /usr/local/lib/python3.6/site-packages/myproject-0.0.1-py3.6.egg/MyPackageName/logging.config
logging.config.fileConfig(fileName)
logger = logging.getLogger('root')
logger.disabled = False
#staticmethod
def Log(logMessage):
Monitor.logger.info(logMessage)
#sub.py
import Monitor
class Example
def simplelog(self,message):
Monitor.Log("Logging some message here")
#call some function here
Monitor.Log("Logging some other messages here for example")
I had similar issues when I tried to load config from a python script that was not on the project root directory. and what I figured out was that:
logging.config.fileConfig is dependent on configparser and have issues with initializing with absolute path. Try relative path.
Replace
fileName = path.join(path.split(path.dirname(path.abspath(__file__)))[0], "logging.config")
with some thing like:
## get path tree from project root and replace children from root with ".."
path_rslv = path.split(path.dirname(path.abspath(__file__)))[1:]
fileName = path.join(*[".." for dotdot in range(len(path_rslv)], "logging.config")
I have a logging.conf [1] file to configure the logging in my python application.
...
[handler_file_log_handler]
class=logging.handlers.TimedRotatingFileHandler
level=INFO
formatter=simple
args=('/var/log/myapp/myapp.log',)
As you can see, I'm storing the log file in /var/log/myapp directory.
What I would like is to store it in the user (the user that manages the application) home directory (/home/myuser/.myapp/log). So, my question is:
What should I configure in my logging.conf in order to be able to save the log file of my python application in the user home directory?
[1] - https://docs.python.org/2.6/library/logging.html#configuring-logging
Yes, you can pass any Python expression as argument! Including one that finds the user's home!
args=(f'{os.path.expanduser("~")}/myapp.log',)
For Python <= 3.5 (no f-strings):
args=(os.path.expanduser("~") + '/myapp.log',)
For completeness, here is a minimal example (tested with 3.5 and 3.9):
logging.conf
[loggers]
keys=root
[handlers]
keys=fileLogHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=fileLogHandler
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[handler_fileLogHandler]
class=logging.handlers.TimedRotatingFileHandler
formatter=simpleFormatter
args=(os.path.expanduser("~") + '/myapp.log',)
main.py
import logging
import logging.config
logging.config.fileConfig('logging.conf')
log = logging.getLogger(__name__)
log.info('Testo')
Additional note:
I thought that passing ~/myapp.log as path would work and let the OS take care of the expansion, which actually works in most places in Python. But inexplicably this does not work inside a logging.conf file!
args=('~/myapp.log',)
FileNotFoundError: [Errno 2] No such file or directory:
'/path/to/this/python/project/~/myapp.log' # ???
The user's home directory can be found (on any OS) using:
os.path.expanduser("~")