I am using python logging module to write to my log files:
logging.config.fileConfig(log_conf_file)
The formatter section in my log_conf_file "logging.conf" looks like:
[formatter_mylog]
format: %(asctime)s %(message)s
This makes my log looks like:
2013-05-02 13:39:55,325 mesagebody blablablabla
How I can customize the asctime format to output something like:
May 02 13:39:55 mesage_body blablablabla
I know if it's in python code, you can do something like:
# create formatter
formatter = logging.Formatter("%(asctime)s;%(message)s",
"%Y-%m-%d %H:%M:%S")
But how do you do it via the logging configuration file ?
Try this:
[formatter_mylog]
format: %(asctime)s %(message)s
datefmt=%m-%d %H:%M:%S
Not sure if this will change the month to the verbal style.
Update:
To implement the verbal style of the month try to use %b (source)
Below should give your expected output
[formatter_mylog]
format: %(asctime)s %(message)s
datefmt=%b %d %H:%M:%S
Related
I have a python flask server that outputs request information. I'm setting the output file name, level, and format using logging.basicConfig:
logging.basicConfig(
filename=log_filename, level=level,
format='%(asctime)s %(processName)s[%(process)d]: %(levelname)5s: %(message)s',
datefmt='%b %d %H: %M: %S')
I want to change the output from something like:
Dec 10 11:13:27 MainProcess[11111]: INFO: 200 GET /ws?token=supersecretkey
To something like:
Dec 10 11:13:27 MainProcess[11111]: INFO: 200 GET /ws?token={hidden}
I suppose I'm looking for some way to grab the log output string and change it. Would a filter be the best idea here?
I want to save my results as a log file, so I am thinking to import logging module. I understand that to output a file, the code is very straightforward.
logging.basicConfig(filename='logger.log', level=logging.INFO)
logging.debug('debug message')
logging.info('info message')
logging.warn('warn message')
logging.error('error message')
logging.critical('critical message')
However, what if I want to output multiple log files? for example, in the following for loop, each iteration will output a log file, how should I do this?
for i in range(1,10):
print (i)
#output a log file to save I value
I tried to use these code, but it's not working.
for i in range(1,10):
filename = str.format('mylog%d.txt' % i)
logging.basicConfig(format=log_fmt, level=logging.DEBUG, filename=filename)
logging.debug('This is debug message')
logging.info('This is info message')
logging.warning('This is warning message')
You're using the format function of strings incorrectly. You're trying to use string interpolation, which is an entirely different method of formatting strings. You should try something like this:
filename = 'mylog{0}.txt'.format(i)
The {0} explicitly states that you should take the first value passed to format. You could leave it as {} if you'd like - it makes no real difference.
About file name:
filename = str.format('mylog%d.txt' % i)
is equal to:
filename = 'mylog%d.txt' % i
For output to multiple files you can use multiple handlers.
Logging class that handle logging.
root_logger = logging.getLogger()
Return you root handler. You can add or remove handlers to logger.
root_logger.handlers
Contains list of handlers of root logger.
first = root_logger.handlers[0]
first.close()
root_logger.removeHandler(first)
remove first handler.
new_handler = logging.FileHandler(file_name)
formatter = logging.Formatter('%(asctime)s ' + ' %(message)s', '%H:%M:%S')
new_handler.setFormatter(formatter)
root_logger.addHandler(new_handler)
Add new formatter to root_handler.
You can output log to any files at the same time.
For more info read:
https://docs.python.org/2/library/logging.html
https://docs.python.org/2/howto/logging-cookbook.html
This is the representation desired for dates:
>>> tz = pytz.timezone('US/Central')
>>> datefmt = '%Y-%m-%d %H:%M:%S.%f%z(%Z)'
>>> datetime.now(tz).strftime(datefmt)
'2017-04-27 15:09:59.606921-0500(CDT)'
This is how it's logged (Python 3.6.0 on Linux):
>>> logrecord_format = '%(asctime)s %(levelname)s %(message)s'
>>> logging.basicConfig(format=logrecord_format, datefmt=datefmt)
>>> logging.error('ruh-roh!')
2017-04-27 15:10:35.%f-0500(CDT) ERROR ruh-roh!
It's not filling the microseconds properly. I've tried changing the logrecord_format to a few other things, but I could not figure it out - how to configure the logger to show microseconds and timezone in the correct way to match the strftime output exactly?
Edit: I could settle for milliseconds with offset, i.e. 2017-04-27 15:09:59,606-0500(CDT). Is that possible? logging provides %(msecs)03d directive, but I can't seem to get the timezone offset to appear after the milliseconds.
Personally, instead of integrating the timezone into the date format, I add it directly to the logged message format. Usually, the timezone should not change during the program execution.
import logging
import time
tz = time.strftime('%z')
fmt = '%(asctime)s' + tz + ' %(levelname)s %(message)s'
logging.basicConfig(format=fmt)
logging.error("This is an error message.")
# 2017-07-28 19:34:53,336+0200 ERROR This is an error message.
It is possible to get a logger by module name. Like this:
logging.getLogger(module_name)
I would like to add module_name to every log record. Is it possible to set up a Formatter object which adds module_name?
You are looking for the %(name)s parameter; add that to your formatter pattern:
FORMAT = "%(name)s: %(message)s"
logging.basicConfig(format=FORMAT)
or when creating a Formatter():
FORMAT = "%(name)s: %(message)s"
formatter = logging.Formatter(fmt=FORMAT)
See the LogRecord attributes reference:
Attribute name: name
Format: %(name)s
Description: Name of the logger used to log the call.
when you initialize the logger (only need to do this once for the app) try this config
logging.basicConfig(
filename='var/bpextract.log',
level=logging.INFO,
format='%(asctime)s %(process)-7s %(module)-20s %(message)s',
datefmt='%m/%d/%Y %H:%M:%S'
)
...later in your code...
log = logging.getLogger("bpextract")
log.info('###### Starting BPExtract App #####')
In logging.basicConfig, you can specify the format:
logging.basicConfig(format='%(name)s\t%(message)s')
I'm configuring my Python logging from a file (see http://www.python.org/doc//current/library/logging.html#configuration-file-format ).
From the example on that page, i have a formatter in the config file that looks like:
[formatter_form01]
format=F1 %(asctime)s %(levelname)s %(message)s
datefmt=
class=logging.Formatter
How do i put a newline in the "format" string that specifies the formatter? Neither \n nor \\n work (e.g. format=F1\n%(asctime)s %(levelname)s %(message)s does not work). Thanks
The logging.config module reads config files with ConfigParser, which has support for multiline values.
So you can specify your format string like this:
[formatter_form01]
format=F1
%(asctime)s %(levelname)s %(message)s
datefmt=
class=logging.Formatter
Multilines values are continued by indenting the following lines (one or more spaces or tabs count as an indent).
The logging configuration file is based on the ConfigParser module. There you'll find you can solve it like this:
[formatter_form01]
format=F1
%(asctime)s %(levelname)s %(message)s
datefmt=
class=logging.Formatter
My best bet would be using a custom formatter (instead of logging.Formatter)... For reference, here's the source code for logging.Formatter.format:
def format(self, record):
record.message = record.getMessage()
if string.find(self._fmt,"%(asctime)") >= 0:
record.asctime = self.formatTime(record, self.datefmt)
s = self._fmt % record.__dict__
if record.exc_info:
# Cache the traceback text to avoid converting it multiple times
# (it's constant anyway)
if not record.exc_text:
record.exc_text = self.formatException(record.exc_info)
if record.exc_text:
if s[-1:] != "\n":
s = s + "\n"
s = s + record.exc_text
return s
It's pretty clear to me that, if self._fmt is read from a text file (single line), no escapping of any kind would be possible. Maybe you can extend from logging.Formatter, override this method and substitute the 4th line for something like:
s = self._fmt.replace('\\n', '\n') % record.__dict__
or something more general, if you want other things to be escaped as well.
EDIT: alternatively, you can do that in the init method, once (instead of every time a message is formatted). But as others already pointed out, the ConfigParser support multiple lines, so no need to go this route...
This might be an easy way:
import logging
logformat = """%(asctime)s ... here you get a new line
... %(thread)d .... here you get another new line
%(message)s"""
logging.basicConfig(format=logformat, level=logging.DEBUG)
I tested, the above setting gives two new lines for each logging message, as it shown in the codes. Note: %(asctime)s and things like this is python logging formatting strings.
import logging
logformat = "%(asctime)s %(message)s\n\r"
logging.basicConfig(level=logging.DEBUG, format=logformat,filename='debug.log', filemode='w')
logging.debug (Your String here)
Debug text in the file will be written with new line.
Just add "\n" before the closing apostrophe of basicConfig function
logging.basicConfig(level=logging.DEBUG, format=' %(levelname)s - %(message)s\n')