I need help with logging in python 2.7.
My directory structure is
I have 3 different functions in file called hydra:
lets name them a, b, c
I have create 3 process to run those functions separately.
I have separate logging in those functions, which works just fine and logs in 3 different log files.
Now, I want them all to log on console as well as write in the file.
I start logging as:
logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(name)s.%(funcName)s +%(lineno)s: %(levelname)-8s [%(process)d] %(message)s',
filename=logfile,
filemode='a')
What all I tried:
Creating a stream handler and adding it to the root, but it didn't work for me.
Any help is appreciated.
I am happy to answer any doubts and forgive me if I missed out on any detail.
logging.getLogger().addHandler(logging.StreamHandler())
Adding the above mentioned line in all the 3 functions resolved my issue.
Related
I am working with the Python logging module but I don't know how get writing in different files for each type of logging level.
For example, I want the debug-level messages (and only debug) saved in the file debug.log.
This is the code I have so far, I have tried with handlers but does not work anymore. I don't know If it is possible to do so I wish. This is the code I have using handlers and different files for each level of logging, but it does not work, since the Debug file save messages of all other levels.
import logging as _logging
self.logger = _logging.getLogger(__name__)
_handler_debug = _logging.FileHandler(self.debug_log_file)
_handler_debug.setLevel(_logging.DEBUG)
self.logger.addHandler(_handler_debug)
_handler_info = _logging.FileHandler(self.info_log_file)
_handler_info.setLevel(_logging.INFO)
self.logger.addHandler(_handler_info)
_handler_warning = _logging.FileHandler(self.warning_log_file)
_handler_warning.setLevel(_logging.WARNING)
self.logger.addHandler(_handler_warning)
_handler_error = _logging.FileHandler(self.error_log_file)
_handler_error.setLevel(_logging.ERROR)
self.logger.addHandler(_handler_error)
_handler_critical = _logging.FileHandler(self.critical_log_file)
_handler_critical.setLevel(_logging.CRITICAL)
self.logger.addHandler(_handler_critical)
self.logger.debug("debug test")
self.logger.info("info test")
self.logger.warning("warning test")
self.logger.error("error test")
self.logger.critical("critical test")
And the debug.log contains this:
debug test
info test
warning test
error test
critical test
I'm working with classes, so I have adapted a bit the code
Possible duplicate to: python logging specific level only
Please see the accepted answer there.
You need to use filters for each handler that restrict the output to the log level supplied.
I'm using python's logging library, but I want the debug logs to have a different format than the warning and error logs. Is this possible?
ETA: I want warnings and errors to appear as:
%(levelname)s: %(message)s
but debug statements to appear as
DEBUG: (only Brian cares about this) : %(message)s
all other questions I've seen have been to change the format, but that changes for EVERYTHING.
First of all, double-check if you really need this. A log output with different record formats is prone to be rather hard to read by both humans and machines.
Maybe what you actually need is different formats for different log destinations (console vs file) which will also have different verbosity (the file will have a debug log with additional information).
Now, the way is to use a custom Formatter:
class MultiformatFormatter(logging.Formatter):
def __init__(self,<args>):
<...>
def format(self,record):
if record.levelno <= logging.DEBUG:
s=<generate string one way>
else:
s=<generate string another way>
return s
<...>
#for each handler that this should apply to
handler.setFormatter(MultiformatFormatter(<args>))
I cannot get the logging module to write to a file for the life of me and I have no idea what is the problem.
I am runnning
form = "%(asctime)s - %(levelname)s - %(message)s"
logging.basicConfig(logfile='/home/gabriel/Developement/cl/cl.log',level=logging.DEBUG, format=form)
logging.debug("oh")
logging.info("oh!")
logging.warning("OH!")
logging.error("OH NO!")
I'm completely unsure of what's going on, the file is not created, nor is it written to. However, python does not raise an exception. I've tried running with python 2.7.4 and ipython. Please let me know what diagnostic steps I can take, I wish I could provide more information but I do not know what is relevant...
Change logfile to filename, like so:
logging.basicConfig(filename='/home/gabriel/Developement/cl/cl.log', level=logging.DEBUG, format=form)
You can see the keyword arguments taken by basicConfig here.
In log4net we can change the log level in the xml config file on runtime from DEBUG to ERROR for example and the change take effect immediately.
is there a way to do it in python? i haven't figured out yet how to do it.
In addition, for some reason, when the logging file is set without any folder (see example of the config i use at the bottom - yaml dictionary config file), it fails to rotate files. if i change it to be at any other path other the one that the code is in, it works like a charm and 20 files get created.
i will be happy to get any help.
thanks!
dictionary config extract example:
rotatingFile:
class : logging.handlers.RotatingFileHandler
formatter: jajahFormater
level: DEBUG
filename: chs.log
maxBytes: 20
backupCount: 20
You really should have posted two questions.
The .NET infrastructure watches the config file and automatically loads it to get the changes, but Python doesn't - so you would need to code this yourself.
The chs.log file in the code directory may be held open (e.g. in an editor), which would prevent rollover - I can't tell without more information.
From the docs use Logger.setLevel() at logger level and Handler.setLevel() at handler level
import logging
my_logger = logging.getLogger('myloggername') #Assumes you have configured the logger somewhere
my_logger.setLevel(logging.INFO)
I want to output some strings to a log file and I want the log file to be continuously updated.
I have looked into the logging module pf python and found out that it is
mostly about formatting and concurrent access.
Please let me know if I am missing something or amy other way of doing it
usually i do the following:
# logging
LOG = "/tmp/ccd.log"
logging.basicConfig(filename=LOG, filemode="w", level=logging.DEBUG)
# console handler
console = logging.StreamHandler()
console.setLevel(logging.ERROR)
logging.getLogger("").addHandler(console)
The logging part initialises logging's basic configurations. In the following I set up a console handler that prints out some logging information separately. Usually my console output is set to output only errors (logging.ERROR) and the detailed output in the LOG file.
Your loggings will now printed to file. For instance using:
logger = logging.getLogger(__name__)
logger.debug("hiho debug message")
or even
logging.debug("next line")
should work.
Doug Hellmann has a nice guide.
To add my 10cents with regards to using logging. I've only recently discovered the Logging module and was put off at first. Maybe just because it initially looks like a lot of work, but it's really simple and incredibly handy.
This is the set up that I use. Similar to Mkinds answer, but includes a timestamp.
# Set up logging
log = "bot.log"
logging.basicConfig(filename=log,level=logging.DEBUG,format='%(asctime)s %(message)s', datefmt='%d/%m/%Y %H:%M:%S')
logging.info('Log Entry Here.')
Which will produce something like:
22/09/2015 14:39:34 Log Entry Here.
You can log to a file with the Logging API.
Example: http://docs.python.org/2/howto/logging.html#logging-to-a-file