I am trying to write a logging module that I can use over and over again. The logging is not the problem. I want to call my logging module from any script and not have to pass the params for the logging file.
If I have a test script called mytest.py how would I return this name in the import logging module. I have tried this in the logging script itself but it returns the name of that file and not the file I am trying to log.
my_test.py:
from my_logger import logging
info.logging("something here")
print("something here")
if __name__ == "__main__":
logging()
I would expect the log file to be called my_test.log, but currently it is being named logging.log
Here is the part from the logging script:
def logging(filename=False, level=DEFAULT_LOG_LEVEL):
if filename is False:
file_ext = os.path.basename(__file__) # Need this to be my_test.py
filename = ("C:/Users/Logs/{0}").format(file_ext)
"Start logging with given filename and level."
#print(filename)
logging.basicConfig(filename=filename, level=LEVELS[level])
else:
"Start logging with given filename and level."
logging.basicConfig(filename=filename, level=LEVELS[level])
Assuming your compiler is CPython (thanks to Matthew Trevor);
You can use inspect.getouterframes to get the caller's frame, plus the filename and line number etc.
import inspect
def logging(filename=False, level=DEFAULT_LOG_LEVEL):
if filename is False:
caller_file = inspect.getouterframes(inspect.currentframe())[1][3]
# prints /home/foo/project/my_test.py
...
The __file__ binding you use in your logging function will hold the value of the file it's scoped in. You need to pass in the calling module's __file__ to get the behaviour you want:
# my_test.py
...
if __name__ == "__main__":
logging(__file__)
I'm surprised that your log file was called logging.log and not my_logger.log, though, as that's the name of the file in which the logging function is defined.
Related
Building a GUI for users to select Python scripts they want to run. Each script has its own docstring explaining inputs and outputs for the script. I want to display that information in the UI once they've highlighted the script, but not selected to run it, and I can't seem to get access to the docstrings from the base program.
ex.
test.py
"""this is a docstring"""
print('hello world')
program.py
index is test.py for this example, but is normally not known because it's whatever the user has selected in the GUI.
# index is test.py
def on_selected(self, index):
script_path = self.tree_view_model.filePath(index)
fparse = ast.parse(''.join(open(script_path)))
self.textBrowser_description.setPlainText(ast.get_docstring(fparse))
Let's the docstring you want to access belongs to the file, file.py.
You can get the docstring by doing the following:
import file
print(file.__doc__)
If you want to get the docstring before you import it then the you could read the file and extract the docstring. Here is an example:
import re
def get_docstring(file)
with open(file, "r") as f:
content = f.read() # read file
quote = content[0] # get type of quote
pattern = re.compile(rf"^{quote}{quote}{quote}[^{quote}]*{quote}{quote}{quote}") # create docstring pattern
return re.findall(pattern, content)[0][3:-3] # return docstring without quotes
print(get_docstring("file.py"))
Note: For this regex to work the docstring will need to be at the very top.
Here's how to get it via importlib. Most of the logic has been put in a function. Note that using importlib does import the script (which causes all its top-level statements to be executed), but the module itself is discarded when the function returns.
If this was the script docstring_test.py in the current directory that I wanted to get the docstring from:
""" this is a multiline
docstring.
"""
print('hello world')
Here's how to do it:
import importlib.util
def get_docstring(script_name, script_path):
spec = importlib.util.spec_from_file_location(script_name, script_path)
foo = importlib.util.module_from_spec(spec)
spec.loader.exec_module(foo)
return foo.__doc__
if __name__ == '__main__':
print(get_docstring('docstring_test', "./docstring_test.py"))
Output:
hello world
this is a multiline
docstring.
Update:
Here's how to do it by letting the ast module in the standard library do the parsing which avoids both importing/executing the script as well as trying to parse it yourself with a regex.
This looks more-or-less equivalent to what's in your question, so it's unclear why what you have isn't working for you.
import ast
def get_docstring(script_path):
with open(script_path, 'r') as file:
tree = ast.parse(file.read())
return ast.get_docstring(tree, clean=False)
if __name__ == '__main__':
print(repr(get_docstring('./docstring_test.py')))
Output:
' this is a multiline\n docstring.\n'
I wrote the module below that will standardize how my logfiles are written as well as easily changing whether events get printed/written to the logfile or not.
FILE: Logging.py
================
import os
import datetime
import io
class Logfile():
def __init__(self,name):
self.logFile = os.getcwd() + r'\.Log\\' + name + '_' + str(datetime.date.today().year) + ('00' + str(datetime.date.today().month))[-2:] + '.log'
self.printLog = False
self.debug = False
# Setup logFile and consolidated Folder
if not os.path.exists(os.path.dirname(self.logFile)):
os.mkdir(os.path.dirname(self.logFile))
#Check if logfile exists.
if not os.path.exists(self.logFile):
with open(self.logFile, 'w') as l:
pass
# Write LogFile Entry
def logEvent(self, eventText, debugOnly): # Function to add an event to the logfile
# If this is marked as debugging only AND debugging is off
if debugOnly == True and self.debug == False:
return
if self.printLog == True:
print(datetime.datetime.strftime(datetime.datetime.now(), '%m/%d/%Y, %I:%M:%S %p, ') + str(eventText))
with open(self.logFile, 'a') as l:
l.seek(0)
l.write(datetime.datetime.strftime(datetime.datetime.now(), '%m/%d/%Y, %I:%M:%S %p, ') + str(eventText) + '\n')
return
This is very handy but, I am having trouble understanding how to make this available to all of my classes. For example, If i import the following module, I am not sure how to use the logfile i created within my main script.
FILE: HelloWorld.py
===================
class HelloWorld():
def __init__(self):
log.logEvent('You have created a HelloWorld Object!', False)
Main Script Here:
import Logging
from HelloWorld import HelloWorld
log = logging.Logfile
hw = HelloWorld()
^^ Will fail because it does not know log is a thing. What is the proper way to handle these sort of situations?
I believe you're trying to do something like this. (and as a side note, you may want to look into using pythons default logging module)
FILE: HelloWorld.py
===================
# import LogFile
from .Logging import LogFile
# create new LogFile instance
log = LogFile(name='log name')
class HelloWorld():
def __init__(self):
# call logEvent method on your LogFile instance
log.logEvent('You have created a HelloWorld Object!', False)
FILE: Main.py
===================
# import HelloWorld
from .HellowWorld import HellowWorld
# create new HellowWorld instance
hw = HellowWorld()
Also, to create a module you will need to add an __init__.py file in that given directory.
This problem is easily solved by using the built-in "Logging" module. In an answer to the broader "how to use a thing(log) within all of my modules" question, I assume the answer to this can be found by reading through the code in the logging module and mimicking that.
I have a function called get_full_class_name(instance), which returns the full module-qualified class name of instance.
Example my_utils.py:
def get_full_class_name(instance):
return '.'.join([instance.__class__.__module__,
instance.__class__.__name__])
Unfortunately, this function fails when given a class that's defined in a currently running script.
Example my_module.py:
#! /usr/bin/env python
from my_utils import get_full_class_name
class MyClass(object):
pass
def main():
print get_full_class_name(MyClass())
if __name__ == '__main__':
main()
When I run the above script, instead of printing my_module.MyClass, it prints __main__.MyClass:
$ ./my_module.py
__main__.MyClass
I do get the desired behavior if I run the above main() from another script.
Example run_my_module.py:
#! /usr/bin/env python
from my_module import main
if __name__ == '__main__':
main()
Running the above script gets:
$ ./run_my_module.py
my_module.MyClass
Is there a way I could write the get_full_class_name() function such that it always returns my_module.MyClass regardless of whether my_module is being run as a script?
I propose handling the case __name__ == '__main__' using the techniques discussed in Find Path to File Being Run. This results in this new my_utils:
import sys
import os.path
def get_full_class_name(instance):
if instance.__class__.__module__ == '__main__':
return '.'.join([os.path.basename(sys.argv[0]),
instance.__class__.__name__])
else:
return '.'.join([instance.__class__.__module__,
instance.__class__.__name__])
This does not handle interactive sessions and other special cases (like reading from stdin). For this you may have to include techniques like discussed in detect python running interactively.
Following mkiever's answer, I ended up changing get_full_class_name() to what you see below.
If instance.__class__.__module__ is __main__, it doesn't use that as the module path. Instead, it uses the relative path from sys.argv[0] to the closest directory in sys.path.
One problem is that sys.path always includes the directory of sys.argv[0] itself, so this relative path ends up being just the filename part of sys.argv[0]. As a quick hack-around, the code below assumes that the sys.argv[0] directory is always the first element of sys.path, and disregards it. This seems unsafe, but safer options are too tedious for my personal code for now.
Any better solutions/suggestions would be greatly appreciated.
import os
import sys
from nose.tools import assert_equal, assert_not_equal
def get_full_class_name(instance):
'''
Returns the fully-qualified class name.
Handles the case where a class is declared in the currently-running script
(where instance.__class__.__module__ would be set to '__main__').
'''
def get_module_name(instance):
def get_path_relative_to_python_path(path):
path = os.path.abspath(path)
python_paths = [os.path.abspath(p) for p in sys.path]
assert_equal(python_paths[0],
os.path.split(os.path.abspath(sys.argv[0]))[0])
python_paths = python_paths[1:]
min_relpath_length = len(path)
result = None
for python_path in python_paths:
relpath = os.path.relpath(path, python_path)
if len(relpath) < min_relpath_length:
min_relpath_length = len(relpath)
result = os.path.join(os.path.split(python_path)[-1],
relpath)
if result is None:
raise ValueError("Path {} doesn't seem to be in the "
"PYTHONPATH.".format(path))
else:
return result
if instance.__class__.__module__ == '__main__':
script_path = os.path.abspath(sys.argv[0])
relative_path = get_path_relative_to_python_path(script_path)
relative_path = relative_path.split(os.sep)
assert_not_equal(relative_path[0], '')
assert_equal(os.path.splitext(relative_path[-1])[1], '.py')
return '.'.join(relative_path[1:-1])
else:
return instance.__class__.__module__
module_name = get_module_name(instance)
return '.'.join([module_name, instance.__class__.__name__])
I try to use the logging of python:
import logging
log = logging.getLogger(__name__)
logInstall = logging.getLogger('/var/log/install.log')
log.info("Everything works great")
logInstall.info("Why is not printed to the log file?")
In the log file of the log, every thing works and it prints: "Everything works great"
But in /var/log/install.log, I don't see anything.
What I do wrong?
Where have you seen that the argument to logging.getLogger() was a file path ??? It's only a name for your logger object:
logging.getLogger([name])
Return a logger with the specified name or, if no name is specified,
return a logger which is the root logger of the hierarchy.
If specified, the name is typically a dot-separated hierarchical name
like “a”, “a.b” or “a.b.c.d”. Choice of these names is entirely up to
the developer who is using logging.
https://docs.python.org/2/library/logging.html#logging.getLogger
Where / how this logger logs to (file, stderr, syslog, mail, network call, whatever) depends on how you configure the logging for your own app - the point here is that you decouple the logger's use (in the libs) from the logger config (in the app).
Note that the usual practice is to use __name__ so you get a logger hierarchy that mimics the packages / modules hierarchy.
So I exported some unit tests from the Selenium IDE to Python. Now I'm trying to debug something, and I've noticed that Selenium uses the logging module. There is one particular line in selenium.webdriver.remote.remote_connection that I would really like to see the output of. It is:
LOGGER.debug('%s %s %s' % (method, url, data))
At the top of the file is another line that reads:
LOGGER = logging.getLogger(__name__)
So where is this log file? I want to look at it.
In your unit test script, place
import logging
logging.basicConfig(filename = log_filename, level = logging.DEBUG)
where log_filename is a path to wherever you'd like the log file written to.
Without the call to logging.basicConfig or some such call to setup a logging handler, the LOGGER.debug command does nothing.