I am using python logging module. With this logging module I used to print statements with logging.info func
But problem is that I am not getting stderr messages in the log file. Is there a way I can print stderr with logging module
Following code I am using
import logging
def initialize_logger(output_dir):
'''
This initialise the logger
:param output_dir:
:return:
'''
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
# create console handler and set level to info
handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
handler = logging.FileHandler(output_dir, "w")
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
In the above code, I am sending only logging statements in the log. It means that I am not getting stderr in the same log file as logging file. Is there a way I can use the same file to send stderr. For example, If my code fails, then along-with logging messages it should also send stderr message.
Right now I don't know why this is happening but seems like my framework has suppressed stderr and stdin messages to print on the console.
Is there a way I can log stderr via logging module?
Related
I want to log to stream of io.StringIO, but end up with empty stream. Here is my code:
import logging, io
log_handler = logging.StreamHandler(stream)
log_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
log_handler.setFormatter(log_format)
log_handler.setLevel(logging.INFO)
logger.addHandler(log_handler)
logger.info("This is test info log")
print(stream.getvalue())
What am I doing wrong?
UPDATE
It seems to be working if I replace level with "Warning" like this
log_handler.setLevel(logging.WARNING)
logger.warning("This is test info log")
It also prints into console for some reason
Configure the root logger: logging.basicConfig(level=logging.DEBUG).
Then you can set log_handler.setLevel(logging.INFO) to whatever level you wish, it will capture logs to stream accordingly.
here I have a simple code initializing a new logger and just log a simple info message. But the INFO message turns red in console. I thought I should be gray and I want it gray. Debug mode also returns red message in the console.
import logging
logger = logging.getLogger('main')
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
formatter = logging.Formatter("%(name)s %(asctime)s %(levelname)s %(message)s", datefmt="%Y-%m-%d %H:%M:%S")
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.info('START EXECUTION')
Even when I use this solution How can I color Python logging output? it doesnt help, info and debug messages remain red.
I am using python logging like this:
logging.basicConfig(stream=sys.stdout, level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
log = logging.getLogger()
Everything works fine. only issue is that all log messages are printed to both stdout and stderr. stdout is formatted as I expect. stderr simply prints the message itself in error stream.
I see that log.handlers indeed contains 2 handlers. Removing the stderr one doesn't seem to have effect. Tried something like this:
for handler in log.handlers:
if handler.stream and handler.stream.name == '<stderr>':
log.handlers.remove(handler)
Any way around this issue?
You can use logging.StreamHandler()
like that:
logging.basicConfig(handlers=[logging.StreamHandler()], level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
log = logging.getLogger()
If you also want to that certain level would go stderr (only Error or Error and Warning) and other stdout. you can do something like suggested here by #crosswired:
logger = logging.getLogger("__name__")
logger.setLevel(logging.DEBUG)
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
formatter = logging.Formatter(format)
h1 = logging.StreamHandler(sys.stdout)
h1.setLevel(logging.DEBUG)
h1.addFilter(lambda record: record.levelno <= logging.INFO)
h1.setFormatter(formatter)
h2 = logging.StreamHandler()
h2.setLevel(logging.WARNING)
h2.setFormatter(formatter)
logger.addHandler(h1)
logger.addHandler(h2)
logger.info("info")
logger.error("error")
Python logging split between stdout and stderr
I have the following minimalist example of a logging test based on the Logging Cookbook:
import logging
logger = logging.getLogger('test')
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - '
'%(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)
print(logger.handlers)
logger.debug('hello world')
The above produces the following output:
$ python test_log.py
[<StreamHandler <stderr> (DEBUG)>]
As I've defined a handler and set the log level to debug, I was expecting the hello world log message to show up in the sample above.
If a logger's level isn't explicitly set, the system looks up the level of ancestor loggers until it gets to a logger whose level is explicitly set. In this case, it's the root logger which is the parent of the logger named 'test'. Setting the level of either this logger or the root logger to DEBUG will result in the log message being output. See this part of the documentation for the event information flow in Python logging.
So if we call logger.debug('hello world') we are generating a DEBUG event. However the logger acts as first filter before passing the event to the handlers. So if the logger level is at higher severity (like INFO) it will filter the event and the handler won't receive it, even if the event was matching the handler's level (DEBUG).
So the logger's level should be as low as the lowest level of its handlers. If you don't want the rest of the handlers to log DEBUG messages you would need to explicitly set their level higher, instead of leaving it unset and therefor inherited from the logger.
I want to use StreamHandler logging handler of python.
What i have tried is,
import logging
import sys
mylogger = logging.getLogger("mylogger")
h1 = logging.StreamHandler(stream=sys.stdout)
h1.setLevel(logging.DEBUG)
mylogger.addHandler(h1)
# now trying to log with the created logger
mylogger.debug("abcd") # <no output>
mylogger.info("abcd") # <no output>
mylogger.warn("abcd") # abcd
Am i missing something ? Or doing any wrong ?
Why INFO and DEBUG level logs are not coming on STDOUT ?
You have to set the level of the logger, not only the level of the handler:
mylogger.setLevel(logging.DEBUG)
Here is a nice graphic of the logging workflow, where you can see that either the logger and the handler check for the log level:
http://docs.python.org/2/howto/logging.html#logging-flow
The default logLevel is WARNING, so even if you set the level of your handler to DEBUG, the message will not get through, since your logger suppresses it (it's also by default WARNING).
By the way, you can do some basic formatting with Formatter:
import logging
import sys
mylogger = logging.getLogger("mylogger")
formatter = logging.Formatter('[%(levelname)s] %(message)s')
handler = logging.StreamHandler(stream=sys.stdout)
handler.setFormatter(formatter)
handler.setLevel(logging.DEBUG)
mylogger.addHandler(handler)
mylogger.setLevel(logging.DEBUG)
mylogger.debug("This is a debug message.")
mylogger.info("Some info message.")
mylogger.warning("A warning.")
Will give you the output
[DEBUG] This is a debug message.
[INFO] Some info message.
[WARNING] A warning.