We Keep Coding

Python logging creates empty log file

I am trying to setup a logging configuration to use in my different modules. I have followed different tutorials and stackoverflow posts (here, here and here) to write logs in to a project.log file.
While the information is displayed correctly in the console, the log.conf is read correctly, the project.log is created but is not filled with the warning messages.
Here is how I proceeded:
The log.conf file used to write up the handlers and formatting:
[loggers]
keys=root,sLogger
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=fileFormatter,consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_sLogger]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=WARNING
formatter=fileFormatter
args=('%(logfilename)s', 'w')
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
In the main.py file:
import logging
import logging.config
def main():
logging.config.fileConfig(fname='./log.conf',
defaults={'logfilename': 'project.log')},
disable_existing_loggers=False)
logger = logging.getLogger(__name__)
logger.warning('This is a message')
In my module.py file:
import logging
logger = logging.getLogger(__name__)
logger.warning('Example of a warning message')

I found the problem ! I didn't realize but when defining handlers in the log.conf file, the order of the handlers and formatters is important !
I changed the log.conf changed to:
[loggers]
keys=root,sLogger
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=consoleFormatter, fileFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler, fileHandler
[logger_sLogger]
level=DEBUG
handlers=consoleHandler, fileHandler
qualname=sLogger
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=WARNING
formatter=fileFormatter
args=('%(logfilename)s','a')
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[formatter_consoleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
NOTE the changed order between consoleFormatter and fileHandler in the [formatters] section.
The rest is working perfectly fine.
To make it easier to play around with I put it in a function (which is probably redundant I suppose):
def base_logger(path_conf, fname, savedir):
'''
Base logging function to be started once
in the main script then used in every other
modules.
Args:
- path_conf: str, path to the configuration
file {log.conf}
- fname: str, name of the saved log file
- savedir: str, path of the saving directory
Returns:
- logging object to be used in the other scripts
Example:
In main.py:
main():
base_logger(paht_cong = './log.conf', fname='file', savedir='project)
In module.py:
# AFTER all the import
logger = logging.getLogger(__name__) # To ge the module name reported in the log file
...
logger.error(f'Error message regarding variable {var}')
The logger initialization {logger = logging.getLogger(__name__)},
has to be done in every file that will use logging !
'''
logging.config.fileConfig(fname=path_conf,
defaults={'logfilename': os.path.join(savedir, f'{fname}.log')},
disable_existing_loggers=False)
I would add that the logger = logging.getLogger(__name__) has to be placed after all imports to make sure that the module name displayed in the log is not imported from other modules.

how to make a configuration file in python Flask for Logging file

Hello guys i have a configuration file named as "LoggingConfig" containing all data please Before dislike this code tell me what you need coze i don't know how to implement the config and use in Flask program.**so please help **This is the config file i have created:
import logging
# create and configure logger
def logger():
logger=logging.getLogger("DATA-MANUFACTURING")
logger.setLevel(logging.DEBUG)
# create console handler and set level to debug
ch=logging.StreamHandler()
# create formatter
lOG_FORMAT= "%(levelname)s - %(name)s - %(asctime)s - %(message)s"
logging.basicConfig(filename="C:\\Users\\rahul\\Desktop\\DATA-MANUFACTURING.log", level = ch.setLevel(logging.DEBUG), format = lOG_FORMAT, filemode = "a")
I want to import to my flask program for this i have written a prog like this:
import logging
import LoggingConfig
import pandas as pd
import numpy as np
import random
import os
from random import randint
from flask import Flask, render_template, request, redirect, make_response
LoggingConfig.logger()
app = Flask(__name__)
APP_ROOT = os.path.dirname(os.path.abspath(__file__))
..................................
..................................
so on.........
if __name__ == '__main__':
app.run(debug=True)
my program have lots of things which i have not given here the only thing i want to make a config file get access in my flask program please if you want me to write whole program here please txt me in text box will add that...thanks

There are different way of configure Logging Files:
Go This site for more info : https://docs.python-guide.org/writing/logging/
For You i prefer INI- format
Go to your program make one more file named as "Yourfilename.ini"
[loggers]
keys=root,DATA-MANUFACTURING
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=fileFormatter,consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_DATA-MANUFACTURING]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=DATA-MANUFACTURING
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=fileFormatter
args=('logfile.log', 'a')
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=
[formatter_consoleFormatter]
format=%(levelname)s - %(message)s
datefmt=
Now in your main program write on the top:
import logging.config
logging.config.fileConfig( 'logging.ini')
logger = logging.getLogger('DATA-MANUFACTURING')
.........................................
and write any logging info whatever you want
logger.info("Loading the UpoadFile page")...so on.

Python- How to configure the log to a file and print to console using ini file

I'm new to Python and I'm trying to log to file and to console,
I saw this question:
logger configuration to log to file and print to stdout
which was very helpful but I Saw that there is a way to configure an ini file and then use it in every module.
Problem is that it seems that it doesn't take the properties from the ini file
and if I don't explicitly define the formatter in the code it uses the default logging without the format that I gave him in the ini file:
configFolder = os.getcwd() + os.sep + 'Configuration'
fileConfig(configFolder + os.sep + 'logging_config.ini')
logger = logging.getLogger(__name__)
# create a file handler
handler = logging.FileHandler('logger.log')
handler.setLevel(logging.INFO)
# add the handlers to the logger
logger.addHandler(handler)
logger.info('hello')
output would be just:
hello
instead of :
2016-08-08 15:16:42,954 - __main__ - INFO - hello
This is my ini file:
[loggers]
keys=root
[handlers]
keys=stream_handler
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream_handler
[handler_stream_handler]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s

Change formatter=formatter to match the name of your formatter: formatter_formatter

Save log files in the user home directory

I have a logging.conf [1] file to configure the logging in my python application.
...
[handler_file_log_handler]
class=logging.handlers.TimedRotatingFileHandler
level=INFO
formatter=simple
args=('/var/log/myapp/myapp.log',)
As you can see, I'm storing the log file in /var/log/myapp directory.
What I would like is to store it in the user (the user that manages the application) home directory (/home/myuser/.myapp/log). So, my question is:
What should I configure in my logging.conf in order to be able to save the log file of my python application in the user home directory?
[1] - https://docs.python.org/2.6/library/logging.html#configuring-logging

Yes, you can pass any Python expression as argument! Including one that finds the user's home!
args=(f'{os.path.expanduser("~")}/myapp.log',)
For Python <= 3.5 (no f-strings):
args=(os.path.expanduser("~") + '/myapp.log',)
For completeness, here is a minimal example (tested with 3.5 and 3.9):
logging.conf
[loggers]
keys=root
[handlers]
keys=fileLogHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=fileLogHandler
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[handler_fileLogHandler]
class=logging.handlers.TimedRotatingFileHandler
formatter=simpleFormatter
args=(os.path.expanduser("~") + '/myapp.log',)
main.py
import logging
import logging.config
logging.config.fileConfig('logging.conf')
log = logging.getLogger(__name__)
log.info('Testo')
Additional note:
I thought that passing ~/myapp.log as path would work and let the OS take care of the expansion, which actually works in most places in Python. But inexplicably this does not work inside a logging.conf file!
args=('~/myapp.log',)
FileNotFoundError: [Errno 2] No such file or directory:
'/path/to/this/python/project/~/myapp.log' # ???

The user's home directory can be found (on any OS) using:
os.path.expanduser("~")

Why non-root logger is not writing any log to log file?

I want to write into two log files by using two loggers with following .ini config file:
[loggers]
keys=root,teja
[handlers]
keys=fileHandler,tejaFileHandler
[formatters]
keys=simpleFormatter
[logger_teja]
level=DEBUG
handlers=tejaFileHandler
qualname='tejaLogger'
[logger_root]
level=DEBUG
handlers=fileHandler
[handler_fileHandler]
class=logging.FileHandler
level=DEBUG
formatter=simpleFormatter
args=("error.log", "a")
[handler_tejaFileHandler]
class=logging.FileHandler
level=DEBUG
formatter=simpleFormatter
args=("teja.log", "a")
[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
And I am using this configuration in my python code as
import logging
import logging.config
# load my module
import my_module
# load the logging configuration
logging.config.fileConfig('logging.ini')
logger1=logging.getLogger('root')
logger1.info('Hi how are you?')
logger2=logging.getLogger('teja')
logger2.debug('checking teja logger?')
I see that logs are written to error.log file whereas no logs are written to teja.log file. Please correct me if I am doing something silly...

You named your logger object 'tejaLogger':
[logger_teja]
level=DEBUG
handlers=tejaFileHandler
qualname='tejaLogger'
# ^^^^^^^^^^^^
Note that the quotes are part of the name.
but your test code picks up teja instead:
logger2=logging.getLogger('teja')
Rename one or the other; although you could use logging.getLogger("'tejaLogger'")
you probably want to drop the quotes and / or rename the logger to what you expected it to be:
[logger_teja]
level=DEBUG
handlers=tejaFileHandler
qualname=teja

It turns out that the problem is on this line (in the [logger_teja] section):
qualname='tejaLogger'
If you add this to your code (it prints all the current loggers):
print(logging.Logger.manager.loggerDict)
You get:
{"'tejaLogger'": <logging.Logger object at 0x7f89631170b8>}
Which means that your logger is called literally 'tejaLogger'. Using:
logger2=logging.getLogger("'tejaLogger'")`
Actually works fine. Either do that, or change qualname='tejaLogger' to qualname=teja.