The execution takes place from Robot Framework, where Test.py has been imported as a library and testLog() is being executed, which in turn imports Logger.py and calls LogMessage().
Test.py
import Logger
def testLog():
Logger.LogMessage("This is the first line of the log file.")
Logger.LogMessage("This is the second line of the log file.")
Logger.LogMessage("This is the third line of the log file.")
Logger.py
import logging
def LogMessage(message):
LOG_FILENAME = "C://Log_Details".log"
logger = logging.getLogger()
logFileHandler = logging.FileHandler(LOG_FILENAME)
logger.addHandler(logFileHandler)
Log_Details.log
This is the first line of the log file.
This is the second line of the log file.
This is the second line of the log file.
This is the third line of the log file.
This is the third line of the log file.
This is the third line of the log file.
The message log section in RIDE logs each line just once during execution, but the file named Log_details.log prints them multiple times, i.e the 1st line gets logged once while the 2nd gets logged twice and so on.
You get 1x message 1, 2x message 2 and 3x message 3.
That is because you perform your logging setup as part of your LogMessage function and in there you add a file log handler every time you log a message... so after first run you have 1 handler that logs your message once, after second call you have 2 handlers that log your message twice and so on...
To avoid that just you want to configure your logger only once.
Just move your logging config to a function that you will call once when you start your script and from then on you can just use:
import logging
log = logging.getLogger(__name__)
log.info('smth')
whenever you feel like logging in any other file in your application.
Related
I am trying to log data to stderr and into a file. The file should contain all log messages, and to stderr should go only the log level configured on the command line. This is described several times in the logging howto - but it does not seem to work for me. I have created a small test script which illustrates my problem:
#!/usr/bin/env python
import logging as l
l.basicConfig(level=100)
logger = l.getLogger("me")
# ... --- === SEE THIS LINE === --- ...
logger.setLevel(l.CRITICAL)
sh = l.StreamHandler()
sh.setLevel(l.ERROR)
sh.setFormatter(l.Formatter('%(levelname)-8s CONSOLE %(message)s'))
logger.addHandler(sh)
fh = l.FileHandler("test.dat", "w")
fh.setLevel(l.DEBUG)
fh.setFormatter(l.Formatter('%(levelname)-8s FILE %(message)s'))
logger.addHandler(fh)
logger.info("hi this is INFO")
logger.error("well this is ERROR")
In line 5th code line I can go for logger.setLevel(l.CRITICAL) or logger.setLevel(l.DEBUG). Both results are unsatisfying.
With logger.setLevel(l.CRITICAL) I get ...
$ python test.py
$ cat test.dat
$
Now with logger.setLevel(l.DEBUG) I get ...
$ python test.py
INFO:me:hi this is INFO
ERROR CONSOLE well this is ERROR
ERROR:me:well this is ERROR
$ cat test.dat
INFO FILE hi this is INFO
ERROR FILE well this is ERROR
$
In one case I see nothing nowhere, in the other I see everything everywhere, and one message is being displayed even twice on the console.
Now I get where the ERROR CONSOLE and ERROR FILE outputs come from, those I expect. I don't get where the INFO:me... or ERROR:me... outputs are coming from, and I would like to get rid of them.
Things I already tried:
Creating a filter as described here: https://stackoverflow.com/a/7447596/902327 (does not work)
Emptying handlers from the logger with logger.handlers = [] (also does not work)
Can somebody help me out here? It seems like a straightforward requirement and I really don't seem to get it.
You can set the root level to DEBUG, set propagate to False and then set the appropriate level for the other handlers.
import logging as l
l.basicConfig()
logger = l.getLogger("me")
# ... --- === SEE THIS LINE === --- ...
logger.setLevel(l.DEBUG)
logger.propagate = False
sh = l.StreamHandler()
sh.setLevel(l.ERROR)
sh.setFormatter(l.Formatter('%(levelname)-8s CONSOLE %(message)s'))
logger.addHandler(sh)
fh = l.FileHandler("test.dat", "w")
fh.setLevel(l.INFO)
fh.setFormatter(l.Formatter('%(levelname)-8s FILE %(message)s'))
logger.addHandler(fh)
logger.info("hi this is INFO")
logger.error("well this is ERROR")
Output:
~$ python test.py
ERROR CONSOLE well this is ERROR
~$ cat test.dat
INFO FILE hi this is INFO
ERROR FILE well this is ERROR
I have used the same Python script on Windows which worked fine and produced several logs during each time it was run. The problem is when I ran the script on Linux the logging produced all of the logs onto one line.
I have tried \n in different places such as in the formatter, in each line itself.
This is how the logging is set up:
# This is the setup of the logging system for the program.
logger = logging.getLogger(__name__)
# Sets the name of the log file to 'login.log'
handler = logging.FileHandler(config.path['logger'])
# Sets up the format of the log file: Time, Function Name, Error Level (e.g. Warning Info, Critical),
# and then the message that follows the format.
formatter = logging.Formatter('%(asctime)-5s %(funcName)-20s %(levelname)-10s %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
# Sets the lowest level of messages to info.
logger.setLevel(logging.INFO)
And here is how each log is made:
logger.warning('%-15s' % client + ' Failed: Logout Error')
Thanks in advance
Brief :
base.py:
import logging
from logging.handlers import TimedRotatingFileHandler
import os
slogFile = os.path.join(os.getcwd(), 'LOGS', 'App_Debug.log')
if True != os.path.isdir(os.path.join(os.getcwd(), 'LOGS')):
os.mkdir(os.path.join(os.getcwd(), 'LOGS'))
logging.basicConfig(filename=slogFile, level=logging.DEBUG)
logging.basicConfig(format='%(asctime)s %(message)s')
logger = logging.getLogger("myApp")
fmt = logging.Formatter(fmt='%(asctime)s %(message)s')
size=1024*1024*1 #1mb file size
logger = logging.getLogger("myApp")
fmt = logging.Formatter(fmt='%(asctime)s %(message)s')
hdlr = logging.handlers.RotatingFileHandler(filename = slogFile ,mode='w', maxBytes=size, backupCount=5, encoding=None, delay=0)
hdlr.setFormatter(fmt)
logger.addHandler(hdlr)</em>
app_main1.py:
import base
base.logger.debug('xxx')
app_main2.py:
import base
base.logger.debug('xxx')
NOTE : for my application i am using another module that also recording the logs into the same file .
I am getting this error:
Traceback (most recent call last):
File "C:\Python27\lib\logging\handlers.py", line 78, in emit
self.doRollover()
File "C:\Python27\lib\logging\handlers.py", line 141, in doRollover
os.rename(self.baseFilename, dfn)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process
Logged from file app_main1.py, line 59
Could you explain this to me?
I want do backup the log file when its reaches the max(1mb) size.
You probably have the log file open in some other program, which is why it can't be renamed. This could be one of your programs, or an anti-virus or full-text indexer operating on disk files.
I got the same error while developing a flask application. To solve the problem, I had to change the environmental variable from
"FLASK_DEBUG=1" to "FLASK_DEBUG=0". The reason being that turning debugging on leads to threading errors. I got the solution after reading this blog
My guess is that since you imported base.py twice, the RotatingFileHandler is setup twice, therefore it is accessed by two processes.
I had a similar problem today due to this reason. Details here.
I would suggest not importing base.py, but creating a child logger in app_main1.py and app_main2.py. You can refer to the documentation here.
Someone inadvertently moved the open log file used by a python program.
The program uses the logging module with a TimedRotatingFileHandler. When the time came to roll-over the file this error was output:
Traceback (most recent call last):
File "/python_root/lib/logging/handlers.py", line 78, in emit
self.doRollover()
File "/python_root/lib/logging/handlers.py", line 338, in doRollover
os.rename(self.baseFilename, dfn)
OSError: [Errno 2] no such file or directory
Logged from file logtest.py, line 16
The error was repeated on each subsequent attempt to log something. The logged messages did not go into the old (moved) log file.
This reproduces the problem (if you move the log file :))
import time
import logging
from logging import handlers
f = logging.Formatter( "%(asctime)s %(message)s" )
h = handlers.TimedRotatingFileHandler(
"testlog", when='s', interval=5, backupCount=10 )
h.setFormatter( f )
logger = logging.getLogger( 'test' )
logger.setLevel( logging.INFO )
logger.addHandler( h )
logger.info( "LOGTEST started" )
for i in range( 10 ):
time.sleep( 5 )
logger.info( "Test logging " + str( i ) )
My concern here is that subsequent log messages are lost. What I'd like to achieve is, in ascending order of preference:
An exception that exits.
An exception I can catch and handle.
The logger displays the error, but subsequent messages go to the old log file.
The logger displays this error, but opens a new log file and continues as normal.
I've skimmed the docs/cookbook for relevant hooks, but nothing's popped out at me. Pointers there are equally welcome.
Thanks for your help,
Jonathan
Exceptions that are raised in doRollover are passed to the handleError method of the handler. You can define a subclass and override this method to do whatever it is you want to do to handle the error.
So I exported some unit tests from the Selenium IDE to Python. Now I'm trying to debug something, and I've noticed that Selenium uses the logging module. There is one particular line in selenium.webdriver.remote.remote_connection that I would really like to see the output of. It is:
LOGGER.debug('%s %s %s' % (method, url, data))
At the top of the file is another line that reads:
LOGGER = logging.getLogger(__name__)
So where is this log file? I want to look at it.
In your unit test script, place
import logging
logging.basicConfig(filename = log_filename, level = logging.DEBUG)
where log_filename is a path to wherever you'd like the log file written to.
Without the call to logging.basicConfig or some such call to setup a logging handler, the LOGGER.debug command does nothing.