I use the Python logging framework with default settings.
For some data compare reason:I have to compare the log with other data output.
But the python log begin with a default, something like:
INFO:root:post params in transmitter
Can I set the python log output without INFO:root:, like:
post params in transmitter
with my own log only?
Thx a lot!
Sure thing. You could set the format to watever you like:
format: '%(message)s'
Like this:
logging.basicConfig(format='%(message)s', ...)
See the doc for more info: http://docs.python.org/library/logging.config.html
Those "INFO:..." or "DEBUG:..." appear there because some handler defines that. My guess: the default handler is still there.
You can check it by taking a peek at logger.handlers right after you created it.
logger = logging.getLogger()
logger.handlers = [] # This is the key thing for the question!
# Start defining and assigning your handlers here
handler = logging.StreamHandler()
handler.setLevel(logging.INFO)
formatter = logging.Formatter("%(asctime)s: %(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
Also, you could just override the format for that default handler:
if (len(logger.handlers) > 0): # Check here whatever. Or try/except. You get the idea...
formatter = logging.Formatter("%(asctime)s: %(levelname)s - %(message)s")
logger.handlers[0].setFormatter(formatter)
I am not a Python expert so maybe there is a better way to remove or even not create that default handler but this works pretty well for me.
Note: As stated in the docs, the .basicConfig is useful for simple loggers. If you have multiple streams, with multiple formats, it does not work as far as I know and you need to go with the custom handlers.
endDate = '2015-07-24'
logging.basicConfig(filename="win" + strftime("%Y%m%d", localtime()) + ".txt", level=logging.DEBUG,format='%(message)s')
Use a Formatter.
Your really dont need to get into removing INFO word.. . (it shall really help you when your code would more messy and you would be using more stuff than just info like debugging exception etc)
If you want to compare you data with that data you can do something like skipping first 10 character (INFO:ROOT:) and then do whatever you feel like. Umm something like this:
f = open("my.log","a+")
lines = f.readlines()
for line in lines:
if(line[10:] == my_output):
do_whatever_you_feel_like()
Related
TL;DR:
How can I have the stream handler to print logs with colors, and the file handler log them without the literal code characters?
Any approach is fine with me, as long I'm still using standart python libraries.
--
I have been looking how to add color to the logger when it's streamed.
Also I would like to use the standart python logging library.
I found this solution: https://stackoverflow.com/a/7995762/11037602
It served me well, the issue is that my logger logs into stream and into file. And the file gets the escape/colors characters aswell, like this:
[1;33mWARNING[1;0m - run - log
Instead of:
WARNING - run - log
My code:
def setup_logger():
logger = logging.getLogger()
#Code suggested in SO question
logging.addLevelName( logging.WARNING, "\033[1;31m%s\033[1;0m" % logging.getLevelName(logging.WARNING))
logging.addLevelName( logging.ERROR, "\033[1;41m%s\033[1;0m" % logging.getLevelName(logging.ERROR))
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(funcName)s - %(message)s')
fh = logging.FileHandler('file.log')
fh.setFormatter(formatter)
logger.addHandler(fh)
sh = logging.StreamHandler()
sh.setFormatter(formatter)
logger.addHandler(sh)
return logger
As I said, this works, however it also records the characters into the file log.
If you click in 'show 2 more comments' on the question that I linked, user #lesnik suggested put the answer's code inside if statement, I'm assuming he meant something like this:
if sys.stderr.isatty():
logging.addLevelName( logging.WARNING, "\033[1;31m%s\033[1;0m" % logging.getLevelName(logging.WARNING))
logging.addLevelName( logging.ERROR, "\033[1;41m%s\033[1;0m" % logging.getLevelName(logging.ERROR))
EDIT: Corrected the typo (pointed out by #Brad Solomon) but just adding this if statement didn't solve my problem. The characters are still logged in file.
To sum it up, how can I have the stream handler to print logs with colors, and the file handler log them without the literal code characters? Any approach is fine with me, as long I'm still using standart python libraries.
What about using this module?
https://coloredlogs.readthedocs.io/en/latest/
I'm using python's logging library, but I want the debug logs to have a different format than the warning and error logs. Is this possible?
ETA: I want warnings and errors to appear as:
%(levelname)s: %(message)s
but debug statements to appear as
DEBUG: (only Brian cares about this) : %(message)s
all other questions I've seen have been to change the format, but that changes for EVERYTHING.
First of all, double-check if you really need this. A log output with different record formats is prone to be rather hard to read by both humans and machines.
Maybe what you actually need is different formats for different log destinations (console vs file) which will also have different verbosity (the file will have a debug log with additional information).
Now, the way is to use a custom Formatter:
class MultiformatFormatter(logging.Formatter):
def __init__(self,<args>):
<...>
def format(self,record):
if record.levelno <= logging.DEBUG:
s=<generate string one way>
else:
s=<generate string another way>
return s
<...>
#for each handler that this should apply to
handler.setFormatter(MultiformatFormatter(<args>))
I'm trying to remove StreamHandler during runtime of my python code execution.
if (False == consoleOutput):
lhStdout = log.handlers[0] # stdout is the only handler initially
log.removeHandler(lhStdout)
This is working fine. But I don't like that we assume that stdout is the first handler in handler array. Is there a way to query handlers class to find which type it is? Something like this
for handler in log.handlers
if (handler.type == StreamHandler())
<...>
What you're looking for is spelled: if isinstance(handler, StreamHandler): - but I'd really like to know why you want to do such a thing instead of using the sensible solution (ie not configuring a StreamHandler for your logger at all...).
I'm messing around with python a bit, and I want to track the date/time of which some events happen. For now, outputting to the console will work just fine.
Currently, I'm handling it like so:
First, get the formatted date/time wrapped in brackets:
def get_date_time():
now = datetime.datetime.now().strftime("%Y/%m/%d %I:%M:%S")
return "[" + now + "]"
Next, whenever I want to "log" an action, call it like so:
print(get_date_time(), "Outputting whatever text here")
Is there anything wrong with doing it this way? Is there maybe a more efficient/clean way to do it? There may not be, it just has me curious.
On every problem; if you are repating stuff, you are doing it wrong.
For your question; just make it a function, something along the lines of;
def log(s):
print(get_date_time(), s)
Or for bigger projects, use logging module.
Yes, use the Python logging module. http://docs.python.org/2/library/logging.html
You can format all your logs to output datetimes if you like.
import logging
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename='/tmp/myapp.log',
filemode='w')
logging.debug('A debug message')
prints
2004-07-02 13:00:08,743 DEBUG A debug message
as seen here http://docs.python.org/2.4/lib/minimal-example.html
If you wish to formate the datetime, use logging.formatter()
I want to output some strings to a log file and I want the log file to be continuously updated.
I have looked into the logging module pf python and found out that it is
mostly about formatting and concurrent access.
Please let me know if I am missing something or amy other way of doing it
usually i do the following:
# logging
LOG = "/tmp/ccd.log"
logging.basicConfig(filename=LOG, filemode="w", level=logging.DEBUG)
# console handler
console = logging.StreamHandler()
console.setLevel(logging.ERROR)
logging.getLogger("").addHandler(console)
The logging part initialises logging's basic configurations. In the following I set up a console handler that prints out some logging information separately. Usually my console output is set to output only errors (logging.ERROR) and the detailed output in the LOG file.
Your loggings will now printed to file. For instance using:
logger = logging.getLogger(__name__)
logger.debug("hiho debug message")
or even
logging.debug("next line")
should work.
Doug Hellmann has a nice guide.
To add my 10cents with regards to using logging. I've only recently discovered the Logging module and was put off at first. Maybe just because it initially looks like a lot of work, but it's really simple and incredibly handy.
This is the set up that I use. Similar to Mkinds answer, but includes a timestamp.
# Set up logging
log = "bot.log"
logging.basicConfig(filename=log,level=logging.DEBUG,format='%(asctime)s %(message)s', datefmt='%d/%m/%Y %H:%M:%S')
logging.info('Log Entry Here.')
Which will produce something like:
22/09/2015 14:39:34 Log Entry Here.
You can log to a file with the Logging API.
Example: http://docs.python.org/2/howto/logging.html#logging-to-a-file