We Keep Coding

Separate local logger with root log python

I am using logging module in python. In my main.py file I am using two logger.
Root logger (To get logs from multiple modules in same directory)
Local logger (To log specific information)
I want information of local logger to be separate from root logger. But when I am creating separate logger. Information of local logger is also present in root logger info.
Here is the sample of how I am doing this
# main.py
import logging
def setup_logger(filename, name = ''):
if name == '':
logging.basicConfig(filename=filename,
format='%(asctime)s %(funcName)s %(levelname)s %(message)s',
filemode='a')
logger = logging.getLogger()
else:
"""
handler = logging.FileHandler(filename, mode = 'a')
handler.setFormatter(logging.Formatter('%(asctime)s %(funcName)s %(levelname)s %(message)s'))
logger = logging.getLogger(name)
logger.addHandler(handler)
"""
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
handler = logging.FileHandler(filename)
handler.setFormatter(formatter)
logger = logging.getLogger(name)
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)
return logger
logger.setLevel(logging.DEBUG)
return logger
logger = setup_logger('main.log')
local_logger = setup_logger('local_log.log', 'local_log')
# other file under root log
logger = logging.getLogger("__main__." + __name__)

You have to stop propagation if you don't want the local loggers to send their logs to the root loggers handlers:
logger.propagate = False
This part of the documentation explains it well: https://docs.python.org/3/howto/logging.html#logging-flow

Location of Python log file should be changed

I am using logger in my python 2.7 project on a legacy code. I want to create logs on specific location but python logging module creates the log files at the default place i.e. from where it is executed.
Is there is any way to change this default location?
Below is the initialization of the logger.
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
formatter = logging.Formatter('%(message)s')
file_handler = logging.FileHandler('file.log')
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)

Below is an example of how you can set the location and other properties of Python logger:
You can define a get_logger function as follows:
import logging
import os
LOG_DIR = 'log_dir'
LOG_FORMATTER = logging.Formatter('[%(asctime)s] %(levelname)s %(name)s: %(message)s')
def get_logger(log_name, log_dir = LOG_DIR):
if not os.path.isdir(log_dir):
os.mkdir(log_dir)
logger = logging.getLogger(log_name)
logging.basicConfig(level = logging.INFO)
log_handler = logging.FileHandler(os.path.join(log_dir, log_name))
logger.addHandler(log_handler)
log_handler.setFormatter(LOG_FORMATTER)
log_handler.setLevel('INFO')
return logger
Then in the file that you want to make logs, you can do as follows:
logger = get_logger('filename')
If you want to make a logging message, you can then do as follows:
logger.info('logging information!')

Python needs to log each script into its own log

I have scripts parent.py and child.py (many childs) and I need to have logs for each, so any logging within parent.py should be in parent.log and child.py should be in child.log
I have the below in each script but I get empty logs... why??
#main.py
import child
handler = logging.FileHandler('logs/main.log')
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %
(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)
child.child_func()
logger.info('testing parent...')
#child.py
handler = logging.FileHandler('logs/child.log')
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %
(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)
def child_func():
logger.info('testing child...')
What I need to have is
#parent.log
{format} testing parent...
#child.log
{format} testing child...

The folks above are right about the default level on the loggers. Also, instead of spreading your configuration around everywhere, I find it more manageable to consolidate logging configuration to be early on in the application. See the example below.
Note: I don't expect this to be selected as an answer. I just wanted to point out what I believe is a better way of organizing the code.
main.py
import logging
import child
logger = logging.getLogger(__name__)
def setup_logging():
main_handler = logging.FileHandler('logs/main.log')
child_handler = logging.FileHandler('logs/child.log')
# Note that you can re-use the same formatter for the handlers.
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")
main_handler.setFormatter(formatter)
child_handler.setFormatter(formatter)
# By default, loggers inherit the level from their parent, so we only need
# to set the level on the root logger if you want to have only one knob to
# control the level.
root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG)
main_logger = logging.getLogger(__name__)
child_logger = logging.getLogger('child')
child_logger.propagate = False
main_logger.addHandler(main_handler)
child_logger.addHandler(child_handler)
def main():
setup_logging()
child.child_func()
logger.info('testing parent...')
if __name__ == '__main__':
main()
child.py
import logging
logger = logging.getLogger(__name__)
def child_func():
logger.info('testing child...')
Setting up a root logger and a child logger (no main)
Here's an example of setting up the root logger to log to logs/main.log, and the child logger to go to logs/child.log
def setup_logging():
root_handler = logging.FileHandler('logs/main.log')
child_handler = logging.FileHandler('logs/child.log')
# Note that you can re-use the same formatter for the handlers.
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")
root_handler.setFormatter(formatter)
child_handler.setFormatter(formatter)
# By default, loggers inherit the level from their parent, so we only need
# to set the level on the root logger if you want to have only one knob to
# control the level.
root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG)
root_logger.addHandler(root_handler)
child_logger = logging.getLogger('child')
child_logger.propagate = False
child_logger.addHandler(child_handler)

You can set severity-level on both handlers and loggers - I believe the logger is set to logging.WARNING by default, so you would only get warning-logs using your code.
You can read more in this thread: What is the point of setLevel in a python logging handler?
import logging
import child
handler = logging.FileHandler('logs/main.log')
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG) # <-- changed
child.child_func()
logger.info('testing parent...')
logger.warning('testing parent...')
logger.debug('testing parent...')
#child.py
import logging
handler = logging.FileHandler('logs/child.log')
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG) # <-- changed
def child_func():
logger.info('testing child...')
logger.warning('testing child...')
logger.debug('testing child...')

How do I use logging in python tornado and write to file

Currently I'm using logging.getLogger().setLevel(logging.DEBUG) what I think is logging everything where logging level is => DEBUG Is that a correct assumption? I can see a difference when I set logging.DEBUG to logging.ERROR so I guess I'm correct.
Also how do I write these logging rows to a file?

This is a exmaple write log to file
import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# create a file handler
handler = logging.FileHandler('hello.log')
handler.setLevel(logging.INFO)
# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(handler)
logger.info('Hello baby')
More detail:
http://victorlin.me/posts/2012/08/good-logging-practice-in-python/

Python flushing log buffer at exit

I wrote a python script which executes a while loop and requires a keyboard interrupt or system shutdown to terminate.
I would like my log file to save the log output; currently the log file gets created, but nothing gets written to it.
The following creates an output file with the contents I expect:
import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# create a file handler
handler = logging.FileHandler('hello.log')
# handler.setLevel(logging.INFO)
# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(handler)
logger.info('Mmmm...donuts')
But when I integrate it into my code, the log file lacks any contents:
from logging import log, FileHandler, getLogger, Formatter, CRITICAL
logger = getLogger(__name__)
logger.setLevel(CRITICAL)
handler = FileHandler("test.log")
formatter = Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(formatter)
logger.info("start")
enter_while_loop()
I believe I should handle this using atexit, but how?
Thank you for your time and consideration.