So I exported some unit tests from the Selenium IDE to Python. Now I'm trying to debug something, and I've noticed that Selenium uses the logging module. There is one particular line in selenium.webdriver.remote.remote_connection that I would really like to see the output of. It is:
LOGGER.debug('%s %s %s' % (method, url, data))
At the top of the file is another line that reads:
LOGGER = logging.getLogger(__name__)
So where is this log file? I want to look at it.
In your unit test script, place
import logging
logging.basicConfig(filename = log_filename, level = logging.DEBUG)
where log_filename is a path to wherever you'd like the log file written to.
Without the call to logging.basicConfig or some such call to setup a logging handler, the LOGGER.debug command does nothing.
Related
The execution takes place from Robot Framework, where Test.py has been imported as a library and testLog() is being executed, which in turn imports Logger.py and calls LogMessage().
Test.py
import Logger
def testLog():
Logger.LogMessage("This is the first line of the log file.")
Logger.LogMessage("This is the second line of the log file.")
Logger.LogMessage("This is the third line of the log file.")
Logger.py
import logging
def LogMessage(message):
LOG_FILENAME = "C://Log_Details".log"
logger = logging.getLogger()
logFileHandler = logging.FileHandler(LOG_FILENAME)
logger.addHandler(logFileHandler)
Log_Details.log
This is the first line of the log file.
This is the second line of the log file.
This is the second line of the log file.
This is the third line of the log file.
This is the third line of the log file.
This is the third line of the log file.
The message log section in RIDE logs each line just once during execution, but the file named Log_details.log prints them multiple times, i.e the 1st line gets logged once while the 2nd gets logged twice and so on.
You get 1x message 1, 2x message 2 and 3x message 3.
That is because you perform your logging setup as part of your LogMessage function and in there you add a file log handler every time you log a message... so after first run you have 1 handler that logs your message once, after second call you have 2 handlers that log your message twice and so on...
To avoid that just you want to configure your logger only once.
Just move your logging config to a function that you will call once when you start your script and from then on you can just use:
import logging
log = logging.getLogger(__name__)
log.info('smth')
whenever you feel like logging in any other file in your application.
I have used the same Python script on Windows which worked fine and produced several logs during each time it was run. The problem is when I ran the script on Linux the logging produced all of the logs onto one line.
I have tried \n in different places such as in the formatter, in each line itself.
This is how the logging is set up:
# This is the setup of the logging system for the program.
logger = logging.getLogger(__name__)
# Sets the name of the log file to 'login.log'
handler = logging.FileHandler(config.path['logger'])
# Sets up the format of the log file: Time, Function Name, Error Level (e.g. Warning Info, Critical),
# and then the message that follows the format.
formatter = logging.Formatter('%(asctime)-5s %(funcName)-20s %(levelname)-10s %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
# Sets the lowest level of messages to info.
logger.setLevel(logging.INFO)
And here is how each log is made:
logger.warning('%-15s' % client + ' Failed: Logout Error')
Thanks in advance
I try to use the logging of python:
import logging
log = logging.getLogger(__name__)
logInstall = logging.getLogger('/var/log/install.log')
log.info("Everything works great")
logInstall.info("Why is not printed to the log file?")
In the log file of the log, every thing works and it prints: "Everything works great"
But in /var/log/install.log, I don't see anything.
What I do wrong?
Where have you seen that the argument to logging.getLogger() was a file path ??? It's only a name for your logger object:
logging.getLogger([name])
Return a logger with the specified name or, if no name is specified,
return a logger which is the root logger of the hierarchy.
If specified, the name is typically a dot-separated hierarchical name
like “a”, “a.b” or “a.b.c.d”. Choice of these names is entirely up to
the developer who is using logging.
https://docs.python.org/2/library/logging.html#logging.getLogger
Where / how this logger logs to (file, stderr, syslog, mail, network call, whatever) depends on how you configure the logging for your own app - the point here is that you decouple the logger's use (in the libs) from the logger config (in the app).
Note that the usual practice is to use __name__ so you get a logger hierarchy that mimics the packages / modules hierarchy.
import logging, logging.handlers
def main():
ntl = logging.handlers.NTEventLogHandler("Python Logging Test")
logger = logging.getLogger("")
logger.setLevel(logging.DEBUG)
logger.addHandler(ntl)
logger.error("This is a '%s' message", "Error")
if __name__ == "__main__":
main()
The Python (2.7.x) script above writes "This is a 'Error' message" to the Windows Event Viewer. When I run it as a script, I get the expected output. If I convert the script to an executable via PyInstaller, I get an entry in the event log but it says something completely different.
The description for Event ID ( 1 ) in Source ( Python Logging Test ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: This is a 'Error' message.
This is the command I use to convert the script into an executable: pyinstaller.py --onefile --noconsole my_script.py though the command line parameters do not appear to have any impact on this behaviour and it will suffice to just call pyinstaller.py my_script.py.
I would appreciate any help in understanding what is going on and how I go about fixing this.
Final solution
I didn't want to go down the resource hacker route, as that is going to be a difficult step to automate. Instead, the approach I took was to grab the win32service.pyd file from c:\Python27\Lib\site-packages\win32 and place it next to my executable. The script was then modified pass the full path to the copy of the win32service.pyd file and this works in both script and exe form. The final script is included below:
import logging, logging.handlers
import os
import sys
def main():
base_dir = os.path.dirname(sys.argv[0])
dllname = os.path.join(base_dir, "win32service.pyd")
ntl = logging.handlers.NTEventLogHandler("Python Logging Test", dllname=dllname)
logger = logging.getLogger("")
logger.setLevel(logging.DEBUG)
logger.addHandler(ntl)
logger.error("This is a '%s' message", "Error")
if __name__ == "__main__":
main()
Usually, the Windows Event Log doesn't store error messages in plain text, but rather message ID references and insertion strings.
Instead of storing a message like Service foo crashed unexpectedly, it stores a message ID which points to a resource string stored in a DLL. In this case, the resource would be something like Service %s crashed unexpectedly and foo would be stored as insertion string. The program which writes the message registers the resource DLL.
The reason for this is localization. DLLs can store lots of different resources (dialog layout, strings, icons…), and one DLL can contain the same resource in many different languages. The operating system automatically chooses the right resources depending on the system locale. Resource DLLs are used by virtually all Microsoft utilities and core utilities.
Side note: Nowadays, the preferred (and cross-platform) way for localization is gettext.
This is used for the message log as well – ideally, you could open a log from an German Windows installation on an English one with all messages in English.
I suspect that the pywin32 implementation skips that mechanism by only having one single message ID (1) which is just something like "%s". It is stored in win32service.pyd and registered by pywin32. This works fine as long as this file exists on the file system, but breaks as soon as it's hidden inside a PyInstaller executable. I guess you have to embed the message ID into your executable directly.
Edit: suspicion confirmed, the message table is indeed stored inside win32service.pyd
Resource Hacker showing the message table http://media.leoluk.de/evlog_rh.png
Try to copy the message table resource from win32service.pyd to your PyInstaller executable (for example using Resource Hacker).
Looking at the logging handler implementation, this might work:
def __init__(self, appname, dllname=None, logtype="Application"):
logging.Handler.__init__(self)
try:
import win32evtlogutil, win32evtlog
self.appname = appname
self._welu = win32evtlogutil
if not dllname:
dllname = os.path.split(self._welu.__file__)
dllname = os.path.split(dllname[0])
dllname = os.path.join(dllname[0], r'win32service.pyd')
You'd have to set dllname to os.path.dirname(__file__). Use something like this if you want it to continue working for the unfrozen script:
if getattr(sys, 'frozen', False):
dllname = None
elif __file__:
dllname = os.path.dirname(__file__)
ntl = logging.handlers.NTEventLogHandler("Python Logging Test", dllname=dllname)
I am experiencing an issue where I am using the logging module in my app. I am working in Eclipse against the LDT Python (Py 2.7) interface (rather than Pydev) on my MacBook Pro. The logging module works through Eclipse; however, when I transfer my app over to a RHEL5 2.7, logging does not seem to be working at all. It is not throwing any exceptions, it is just not logging anything to console or file (it creates the file though).
Code:
# Initialize logging
log = logging.getLogger('pepPrep')
# Log to stderr
console = logging.StreamHandler()
console.setLevel(logging.INFO)
# Log to file
logname = 'pepPrep.' + datetime.datetime.now().strftime("%Y%m%d_%H:%M") + '.log'
filelog = logging.FileHandler(logname)
filelog.setLevel(logging.DEBUG)
# set a format
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
# tell the handler to use this format
console.setFormatter(formatter)
filelog.setFormatter(formatter)
# add the handler to the root logger
log.addHandler(console)
log.addHandler(filelog)
log.INFO('This is a test')
log.DEBUG('This is a test2')
Any pointers on how I can make this work?
The default threshold for logging is WARNING, so INFO and DEBUG messages are not output by default. To do so, add e.g.
logging.getLogger().setLevel(logging.DEBUG)
to get DEBUG and INFO messages.
You can confirm this is your problem by doing
log.warning('This is a test3')
before adding that setLevel, and confirming that the warning is actually output.