I originally used logging.basicConfig(filename='logs/example.log') to create a log file. After reading the docs I found that it is not recommended to modify the class attributes of logging. Now that I've changed these attributes how can I change them back/reset the logger module?
Output of the code below creates two log files, app.log and example.log. The latter is an artifact of .basicConfig() being called when I first tried to set up the logger.
UPDATE:
grep -R "example.log" /lib/python3.8/ does not output anything so I'm not sure what was changed in the source code of logging to cause the example.log file to be created every time
import logging
import logging.handlers
LOG_FILENAME = 'logs/app.log'
# https://stackoverflow.com/questions/3630774/logging-remove-inspect-modify-handlers-configured-by-fileconfig
# need to run this every time since I changed class attributes of logger
# still creates example.log file
print(logging.getLogger())
root_log = logging.getLogger()
for handler in root_log.handlers:
root_log.removeHandler(handler)
# OR
# logging.basicConfig(force=True) #one way to reset root logger format
# create logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# create console and file handler and set level to info
fh = logging.handlers.RotatingFileHandler(LOG_FILENAME, maxBytes=250000, backupCount=5)
fh.setLevel(logging.INFO)
# ch = logging.StreamHandler()
# ch.setLevel(logging.INFO)
# create formatter
ffh = logging.Formatter('%(asctime)s : %(name)-12s : %(levelname)-8s : %(message)s')
# fch = logging.Formatter('%(name)-12s : %(levelname)-8s : %(message)s')
# add formatter handlers
fh.setFormatter(ffh)
# ch.setFormatter(fch)
# add handler to logger
logger.addHandler(fh)
# logger.addHandler(ch)
logger.info('instance logger')
# logging.shutdown()
Related
I am using logger in my python 2.7 project on a legacy code. I want to create logs on specific location but python logging module creates the log files at the default place i.e. from where it is executed.
Is there is any way to change this default location?
Below is the initialization of the logger.
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
formatter = logging.Formatter('%(message)s')
file_handler = logging.FileHandler('file.log')
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
Below is an example of how you can set the location and other properties of Python logger:
You can define a get_logger function as follows:
import logging
import os
LOG_DIR = 'log_dir'
LOG_FORMATTER = logging.Formatter('[%(asctime)s] %(levelname)s %(name)s: %(message)s')
def get_logger(log_name, log_dir = LOG_DIR):
if not os.path.isdir(log_dir):
os.mkdir(log_dir)
logger = logging.getLogger(log_name)
logging.basicConfig(level = logging.INFO)
log_handler = logging.FileHandler(os.path.join(log_dir, log_name))
logger.addHandler(log_handler)
log_handler.setFormatter(LOG_FORMATTER)
log_handler.setLevel('INFO')
return logger
Then in the file that you want to make logs, you can do as follows:
logger = get_logger('filename')
If you want to make a logging message, you can then do as follows:
logger.info('logging information!')
Yes, I see python doc says: "Loggers are never instantiated directly, but always through the module-level function logging.getLogger(name)", but I have an issue to debug and want to know the root cause.
here is the example:
#!/usr/bin/python
import logging
logger = logging.getLogger("test")
format = "%(asctime)s [%(levelname)-8s] %(message)s"
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter(format, datefmt="%Y-%m-%d %H:%M:%S"))
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
logger.info("test")
Using logging.getLogger("test") here, log message will not be printed.
If I change logging.getLogger("test") to logging.Logger("test"), the log message will be printed.
#!/usr/bin/python
import logging
logger = logging.Logger("test")
format = "%(asctime)s [%(levelname)-8s] %(message)s"
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter(format, datefmt="%Y-%m-%d %H:%M:%S"))
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
logger.info("test")
Or we can using logging.getLogger("test") and set logger level to logging.DEBUG.
#!/usr/bin/python
import logging
logger = logging.getLogger("test")
format = "%(asctime)s [%(levelname)-8s] %(message)s"
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter(format, datefmt="%Y-%m-%d %H:%M:%S"))
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)
logger.info("test")
The method .getLogger("test") is looking for any existing logger configurations for the name "test" while the .Logger("test") is creating a default logger with the name "test" and sets the default log level to 0. If the getLogger method doesn't find a logger class by that name then it will create a basic logger that will have an effective level of 30 (https://docs.python.org/3/library/logging.html#logging-levels) which will ignore your DEBUG message. You can check via logger.getEffectiveLevel() to notice the difference.
Ideally you would create loggers and set them based on the proper naming/configurations instead of accepting the default configuration.
I have create a global logger using the following:
def logini():
logfile='/var/log/cs_status.log'
import logging
import logging.handlers
global logger
logger = logging.getLogger()
logging.basicConfig(filename=logfile,filemode='a',format='%(asctime)s %(name)s %(levelname)s %(message)s',datefmt='%y%m%d-%H:%M:%S',level=logging.DEBUG,propagate=0)
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=2000000, backupCount=5)
logger.addHandler(handler)
__builtins__.logger = logger
It works, however I am getting 2 outputs for every log, one with the formatting and one without.
I realize that this is being caused by the file rotater as I can comment out the 2 lines of the handler code and then I get a single outputted correct log entry.
How can I prevent the log rotator from outputting a second entry ?
Currently you're configuring two file loggers that point to the same logfile. To only use the RotatingFileHandler, get rid of the basicConfig call:
logger = logging.getLogger()
handler = logging.handlers.RotatingFileHandler(logfile, maxBytes=2000000,
backupCount=5)
formatter = logging.Formatter(fmt='%(asctime)s %(name)s %(levelname)s %(message)s',
datefmt='%y%m%d-%H:%M:%S')
handler.setFormatter(formatter)
handler.setLevel(logging.DEBUG)
logger.addHandler(handler)
All basicConfig does for you is to provide an easy way to instantiate either a StreamHandler (default) or a FileHandler and set its loglevel and formats (see the docs for more information). If you need a handler other than these two, you should instantiate and configure it yourself.
I am already using a basic logging config where all messages across all modules are stored in a single file. However, I need a more complex solution now:
Two files: the first remains the same.
The second file should have some custom format.
I have been reading the docs for the module, bu they are very complex for me at the moment. Loggers, handlers...
So, in short:
How to log to two files in Python 3, ie:
import logging
# ...
logging.file1.info('Write this to file 1')
logging.file2.info('Write this to file 2')
You can do something like this:
import logging
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
def setup_logger(name, log_file, level=logging.INFO):
"""To setup as many loggers as you want"""
handler = logging.FileHandler(log_file)
handler.setFormatter(formatter)
logger = logging.getLogger(name)
logger.setLevel(level)
logger.addHandler(handler)
return logger
# first file logger
logger = setup_logger('first_logger', 'first_logfile.log')
logger.info('This is just info message')
# second file logger
super_logger = setup_logger('second_logger', 'second_logfile.log')
super_logger.error('This is an error message')
def another_method():
# using logger defined above also works here
logger.info('Inside method')
def setup_logger(logger_name, log_file, level=logging.INFO):
l = logging.getLogger(logger_name)
formatter = logging.Formatter('%(message)s')
fileHandler = logging.FileHandler(log_file, mode='w')
fileHandler.setFormatter(formatter)
streamHandler = logging.StreamHandler()
streamHandler.setFormatter(formatter)
l.setLevel(level)
l.addHandler(fileHandler)
l.addHandler(streamHandler)
setup_logger('log1', txtName+"txt")
setup_logger('log2', txtName+"small.txt")
logger_1 = logging.getLogger('log1')
logger_2 = logging.getLogger('log2')
logger_1.info('111messasage 1')
logger_2.info('222ersaror foo')
If I get the logger with a name and add a FileHandler it does not write to the file.
This works and writes correctly to the file:
log = logging.getLogger()
fh = logging.FileHandler(logfile)
log.addHandler(fh)
fh_fmt = logging.Formatter("%(asctime)s (%(levelname)s)\t: %(message)s")
fh.setFormatter(fh_fmt)
log.setLevel(logging.INFO)
This does not write to the file:
log = logging.getLogger(name)
fh = logging.FileHandler(logfile)
log.addHandler(fh)
fh_fmt = logging.Formatter("%(asctime)s (%(levelname)s)\t: %(message)s")
fh.setFormatter(fh_fmt)
log.setLevel(logging.INFO)
The only difference is that I get a 'named' logger.
This is a rather old question, but I believe I found the underlying problem and solution, at least with a newer version of Python.
The second code example starts with log = logging.getLogger(name), where name is presumed to be a string representing the name of the logger. Since a name is provided, this log will not be the root logger. According to Logger.setLevel(level) docs for Python 3.6+,
When a logger is created, the level is set to NOTSET (which causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger).
This tells us that we have to set the level of our logger so that it will actually process the messages instead of passing it to the root logger.
This is a code example I wrote (in Python 3.7) that does not work:
from pathlib import Path
import logging
formatter = logging.Formatter('%(name)s [%(levelname)s] %(message)s')
log_file_dir = Path('./log/')
config_file = 'config_file.txt'
config_file_path = log_file_dir / config_file
logger = logging.getLogger('example_logger')
fh = logging.FileHandler(config_file_path, mode='w')
fh.setLevel(logging.INFO)
fh.setFormatter(formatter)
logger.addHandler(fh)
logger.info('Start Configuration Log')
And this one works by adding one line:
from pathlib import Path
import logging
formatter = logging.Formatter('%(name)s [%(levelname)s] %(message)s')
log_file_dir = Path('./log/')
config_file = 'config_file_2.txt'
config_file_path = log_file_dir / config_file
logger = logging.getLogger('example_logger')
logger.setLevel(logging.INFO) # <------ Or the applicable level for your use-case
fh = logging.FileHandler(config_file_path, mode='w')
fh.setLevel(logging.INFO)
fh.setFormatter(formatter)
logger.addHandler(fh)
logger.info('Start Configuration Log')
Note: The first code example does create the chosen log file, but does not write 'Start Configuration Log' to the file.