I created a new service in systemD, so far it does not do anything, I am just trying to check that everything is running step by step.
For that, I am adding logs, however, it does not appear in Syslog, which I understand is the default for services.
When I use simple prints in my code, it does appear in Syslog though
import logging
class recoveryService:
def __init__(self):
self.id = 'integ38'
print self.id # prints to log
logging.info("the id is {}".format(self.id)) #does not print to log
def run(self):
print 'reached run' #prints to log
logging.info('reached run log') #does not print to log
if __name__ == '__main__':
recovery = recoveryService()
recovery.run()
How can I make these loggings appear in syslog?
logging.getLogger().setLevel('INFO') call this once before your first logging call. The default level of the root logger is WARNING so no logs below that level are shown. Also be aware that it is good practice to define a handler or use logging.basicConfig to set up your logging.
Related
I have a Python script and I want that the info method write the messages in the console. But the warning, critical or error writes the messages to a file. How can I do that?
I tried this:
import logging
console_log = logging.getLogger("CONSOLE")
console_log.setLevel(logging.INFO)
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)
console_log.addHandler(stream_handler)
file_log = logging.getLogger("FILE")
file_log.setLevel(logging.WARNING)
file_handler = logging.FileHandler('log.txt')
file_handler.setLevel(logging.WARNING)
file_log.addHandler(file_handler)
def log_to_console(message):
console_log.info(message)
def log_to_file(message):
file_log.warning(message)
log_to_console("THIS SHOULD SHOW ONLY IN CONSOLE")
log_to_file("THIS SHOULD SHOW ONLY IN FILE")
but the message that should be only in the file is going to the console too, and the message that should be in the console, is duplicating. What am I doing wrong here?
What happens is that the two loggers you created propagated the log upwards to the root logger. The root logger does not have any handlers by default, but will use the lastResort handler if needed:
A "handler of last resort" is available through this attribute. This
is a StreamHandler writing to sys.stderr with a level of WARNING, and
is used to handle logging events in the absence of any logging
configuration. The end result is to just print the message to
sys.stderr.
Source from the Python documentation.
Inside the Python source code, you can see where the call is done.
Therefore, to solve your problem, you could set the console_log and file_log loggers' propagate attribute to False.
On another note, I think you should refrain from instantiating several loggers for you use case. Just use one custom logger with 2 different handlers that will each log to a different destination.
Create a custom StreamHandler to log only the specified level:
import logging
class MyStreamHandler(logging.StreamHandler):
def emit(self, record):
if record.levelno == self.level:
# this ensures this handler will print only for the specified level
super().emit(record)
Then, use it:
my_custom_logger = logging.getLogger("foobar")
my_custom_logger.propagate = False
my_custom_logger.setLevel(logging.INFO)
stream_handler = MyStreamHandler()
stream_handler.setLevel(logging.INFO)
file_handler = logging.FileHandler("log.txt")
file_handler.setLevel(logging.WARNING)
my_custom_logger.addHandler(stream_handler)
my_custom_logger.addHandler(file_handler)
my_custom_logger.info("THIS SHOULD SHOW ONLY IN CONSOLE")
my_custom_logger.warning("THIS SHOULD SHOW ONLY IN FILE")
And it works without duplicate and without misplaced log.
I'm using pytest-3.7.1 which has good support for logging, including live logging to stdout during tests. I'm using --log-cli-level=DEBUG to dump all debug-level logging to the console as it happens.
The problem I have is that --log-cli-level=DEBUG turns on debug logging for all modules in my test program, including third-party dependencies, and it floods the log with a lot of uninteresting output.
Python's logging module has the ability to set logging levels per module. This enables selective logging - for example, in a normal Python program I can turn on debugging for just one or two of my own modules, and restrict the log output to just those, or set different log levels for each module. This enables turning off debug-level logging for noisy libraries.
So what I'd like to do is apply the same concept to pytest's logging - i.e. specify a logging level, from the command line, for specific non-root loggers. For example, if I have a module called test_foo.py then I'm looking for a way to set the log level for this module from the command line.
I'm prepared to roll-my-own if necessary (I know how to add custom arguments to pytest), but before I do that I just want to be sure that there isn't already a solution. Is anyone aware of one?
I had the same problem, and found a solution in another answer:
Instead of --log-cli-level=DEBUG, use --log-level DEBUG. It disables all third-party module logs (in my case, I had plenty of matplotlib logs), but still outputs your app logs for each test that fails.
I got this working by writing a factory class and using it to set the level of the root logger to logger.INFO and use the logging level from the command line for all the loggers obtained from the factory. If the logging level from the command line is higher than the minimum global log level you specify in the class (using constant MINIMUM_GLOBAL_LOG_LEVEL), the global log level isn't changed.
import logging
MODULE_FIELD_WIDTH_IN_CHARS = '20'
LINE_NO_FIELD_WIDTH_IN_CHARS = '3'
LEVEL_NAME_FIELD_WIDTH_IN_CHARS = '8'
MINIMUM_GLOBAL_LOG_LEVEL = logging.INFO
class EasyLogger():
root_logger = logging.getLogger()
specified_log_level = root_logger.level
format_string = '{asctime} '
format_string += '{module:>' + MODULE_FIELD_WIDTH_IN_CHARS + 's}'
format_string += '[{lineno:' + LINE_NO_FIELD_WIDTH_IN_CHARS + 'd}]'
format_string += '[{levelname:^' + LEVEL_NAME_FIELD_WIDTH_IN_CHARS + 's}]: '
format_string += '{message}'
level_change_warning_sent = False
#classmethod
def get_logger(cls, logger_name):
if not EasyLogger._logger_has_format(cls.root_logger, cls.format_string):
EasyLogger._setup_root_logger()
logger = logging.getLogger(logger_name)
logger.setLevel(cls.specified_log_level)
return logger
#classmethod
def _setup_root_logger(cls):
formatter = logging.Formatter(fmt=cls.format_string, style='{')
if not cls.root_logger.hasHandlers():
handler = logging.StreamHandler()
cls.root_logger.addHandler(handler)
for handler in cls.root_logger.handlers:
handler.setFormatter(formatter)
cls.root_logger.setLevel(MINIMUM_GLOBAL_LOG_LEVEL)
if (cls.specified_log_level < MINIMUM_GLOBAL_LOG_LEVEL and
cls.level_change_warning_sent is False):
cls.root_logger.log(
max(cls.specified_log_level, logging.WARNING),
"Setting log level for %s class to %s, all others to %s" % (
__name__,
cls.specified_log_level,
MINIMUM_GLOBAL_LOG_LEVEL
)
)
cls.level_change_warning_sent = True
#staticmethod
def _logger_has_format(logger, format_string):
for handler in logger.handlers:
return handler.format == format_string
return False
The above class is then used to send logs normally as you would with a logging.logger object as follows:
from EasyLogger import EasyLogger
class MySuperAwesomeClass():
def __init__(self):
self.logger = EasyLogger.get_logger(__name__)
def foo(self):
self.logger.debug("debug message")
self.logger.info("info message")
self.logger.warning("warning message")
self.logger.critical("critical message")
self.logger.error("error message")
Enable/Disable/Modify the log level of any module in Python:
logging.getLogger("module_name").setLevel(logging.log_level)
I want to find out how logging should be organised given that I write many scripts and modules that should feature similar logging. I want to be able to set the logging appearance and the logging level from the script and I want this to propagate the appearance and level to my modules and only my modules.
An example script could be something like the following:
import logging
import technicolor
import example_2_module
def main():
verbose = True
global log
log = logging.getLogger(__name__)
logging.root.addHandler(technicolor.ColorisingStreamHandler())
# logging level
if verbose:
logging.root.setLevel(logging.DEBUG)
else:
logging.root.setLevel(logging.INFO)
log.info("example INFO message in main")
log.debug("example DEBUG message in main")
example_2_module.function1()
if __name__ == '__main__':
main()
An example module could be something like the following:
import logging
log = logging.getLogger(__name__)
def function1():
print("printout of function 1")
log.info("example INFO message in module")
log.debug("example DEBUG message in module")
You can see that in the module there is minimal infrastructure written to import the logging of the appearance and the level set in the script. This has worked fine, but I've encountered a problem: other modules that have logging. This can result in output being printed twice, and very detailed debug logging from modules that are not my own.
How should I code this such that the logging appearance/level is set from the script but then used only by my modules?
You need to set the propagate attribute to False so that the log message does not propagate to ancestor loggers. Here is the documentation for Logger.propagate -- it defaults to True. So just:
import logging
log = logging.getLogger(__name__)
log.propagate = False
I want to capture log entries into a string, to display in a wx dialog. I just can't get the StringIO to be filled by the log entries... what wrong here?
# prepare logging
log = StringIO.StringIO('Report')
logger = logging.getLogger (__name__)
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(log)
logger.addHandler(handler)
# do something and log it
logging.info('Some log entry')
# display log
handler.flush()
dlg = wx.lib.dialogs.ScrolledMessageDialog(window, log.getvalue(), "Import Report")
dlg.ShowModal()
log.close()
The dialog shows the initial StringIO content ('Report'), but nothing added via the log ('Some log entry').
I looked at this without enlightenment, and read the logging tutorials without hunches, so I turned here.
Thanks for pointers, nobi
logger = logging.getLogger (__name__)
You are passing a name to getLogger, so it is giving you a logger with the specified name (instead of the root logger, this is important later).
logging.info('Some log entry')
You are calling logging.info (note: very different from logger.info). logging.info logs to the root logger. Which is a different entity than your logger.
Either switch that call to logger.info('Some log entry') or just get the root logger from logging.getLogger() by not passing it a string.
Reading back over this I just used the word "logger" about 60 times in about as many contexts, so if this is confusing I can try to clarify.
I'm writing a server app which should be able to log at different levels both on the console and a log file.
The problem is, if logging.basicConfig() is set it will log to the console but it must be set in the main thread.
It can also be set with logging.basicConfig(filename='logger.log') to write to a file.
Setting a handle either for console logging (logging.StreamHandler()) or file logging (logging.FileHandler()) complements the logging.baseconfig() option set.
The problem is, that the settings are not independent.
What I mean is, the loglevel of logging.baseConfig() must include the Handler level, or it wont be logged.
So if I set the baseConfig to log to file, and a StreamHandler to log to console, the file loglevel must be lower than the console level.
(Also, the basicConfig option logs all other logs.)
I tried to create two Handles, one for the console and one for the log file, they work, but whatever log type is specified by basicConfig() will still be displayed duplicating messages.
Is there a way to disable the output of basicConfig() ?
Or any other way to implement these options ?
Thanks.
You don't say in your question exactly what levels you want on your console and file logging. However, you don't need to call basicConfig(), as it's only a convenience function. You can do e.g. (code just typed in, not tested):
import logging
logger = logging.getLogger(__name__)
configured = False
def configure_logging():
global configured
if not configured:
logger.setLevel(logging.DEBUG) # or whatever
console = logging.StreamHandler()
file = logging.FileHandler('/path/to/file')
#set a level on the handlers if you want;
#if you do, they will only output events that are >= that level
logger.addHandler(console)
logger.addHandler(file)
configured = True
Events are passed to the logger first, and if an event is to be processed (due to comparing the level of the logger and the event) then the event is passed to each handler of the logger and all its ancestors' handlers as well. If a level is set on a handler the event may be dropped by that handler, otherwise it will output the event.
Find the below example code to handle the logging with all kind of exceptions
import mysql.connector
import logging
logging.basicConfig(filename=r'C:\Users\root\Desktop\logs.txt',level=logging.DEBUG,format='%(asctime)s,%(levelname)s:%(message)s',datefmt='%d-%m-%Y %H:%M:%S')
while True:
try:
mydb=mysql.connector.connect(host='localhost',user='root',passwd='password123', database='shiva')
mycursor=mydb.cursor()
logging.info("Connected mysql db successfully...\n")
mycursor.execute("show databases")
mycursor.execute("Create table employee(name varchar(20), salary float(20))")
mydb.commit()
except Exception as e:
logging.info("Trying to Connect MysqlDB...")
logging.critical("Error Occured While Connecting...\n\n" "CAUSEDBY: "+str(e))
logging.warning("Check Login Credentials.")