I wrote the module below that will standardize how my logfiles are written as well as easily changing whether events get printed/written to the logfile or not.
FILE: Logging.py
================
import os
import datetime
import io
class Logfile():
def __init__(self,name):
self.logFile = os.getcwd() + r'\.Log\\' + name + '_' + str(datetime.date.today().year) + ('00' + str(datetime.date.today().month))[-2:] + '.log'
self.printLog = False
self.debug = False
# Setup logFile and consolidated Folder
if not os.path.exists(os.path.dirname(self.logFile)):
os.mkdir(os.path.dirname(self.logFile))
#Check if logfile exists.
if not os.path.exists(self.logFile):
with open(self.logFile, 'w') as l:
pass
# Write LogFile Entry
def logEvent(self, eventText, debugOnly): # Function to add an event to the logfile
# If this is marked as debugging only AND debugging is off
if debugOnly == True and self.debug == False:
return
if self.printLog == True:
print(datetime.datetime.strftime(datetime.datetime.now(), '%m/%d/%Y, %I:%M:%S %p, ') + str(eventText))
with open(self.logFile, 'a') as l:
l.seek(0)
l.write(datetime.datetime.strftime(datetime.datetime.now(), '%m/%d/%Y, %I:%M:%S %p, ') + str(eventText) + '\n')
return
This is very handy but, I am having trouble understanding how to make this available to all of my classes. For example, If i import the following module, I am not sure how to use the logfile i created within my main script.
FILE: HelloWorld.py
===================
class HelloWorld():
def __init__(self):
log.logEvent('You have created a HelloWorld Object!', False)
Main Script Here:
import Logging
from HelloWorld import HelloWorld
log = logging.Logfile
hw = HelloWorld()
^^ Will fail because it does not know log is a thing. What is the proper way to handle these sort of situations?
I believe you're trying to do something like this. (and as a side note, you may want to look into using pythons default logging module)
FILE: HelloWorld.py
===================
# import LogFile
from .Logging import LogFile
# create new LogFile instance
log = LogFile(name='log name')
class HelloWorld():
def __init__(self):
# call logEvent method on your LogFile instance
log.logEvent('You have created a HelloWorld Object!', False)
FILE: Main.py
===================
# import HelloWorld
from .HellowWorld import HellowWorld
# create new HellowWorld instance
hw = HellowWorld()
Also, to create a module you will need to add an __init__.py file in that given directory.
This problem is easily solved by using the built-in "Logging" module. In an answer to the broader "how to use a thing(log) within all of my modules" question, I assume the answer to this can be found by reading through the code in the logging module and mimicking that.
Related
Building a GUI for users to select Python scripts they want to run. Each script has its own docstring explaining inputs and outputs for the script. I want to display that information in the UI once they've highlighted the script, but not selected to run it, and I can't seem to get access to the docstrings from the base program.
ex.
test.py
"""this is a docstring"""
print('hello world')
program.py
index is test.py for this example, but is normally not known because it's whatever the user has selected in the GUI.
# index is test.py
def on_selected(self, index):
script_path = self.tree_view_model.filePath(index)
fparse = ast.parse(''.join(open(script_path)))
self.textBrowser_description.setPlainText(ast.get_docstring(fparse))
Let's the docstring you want to access belongs to the file, file.py.
You can get the docstring by doing the following:
import file
print(file.__doc__)
If you want to get the docstring before you import it then the you could read the file and extract the docstring. Here is an example:
import re
def get_docstring(file)
with open(file, "r") as f:
content = f.read() # read file
quote = content[0] # get type of quote
pattern = re.compile(rf"^{quote}{quote}{quote}[^{quote}]*{quote}{quote}{quote}") # create docstring pattern
return re.findall(pattern, content)[0][3:-3] # return docstring without quotes
print(get_docstring("file.py"))
Note: For this regex to work the docstring will need to be at the very top.
Here's how to get it via importlib. Most of the logic has been put in a function. Note that using importlib does import the script (which causes all its top-level statements to be executed), but the module itself is discarded when the function returns.
If this was the script docstring_test.py in the current directory that I wanted to get the docstring from:
""" this is a multiline
docstring.
"""
print('hello world')
Here's how to do it:
import importlib.util
def get_docstring(script_name, script_path):
spec = importlib.util.spec_from_file_location(script_name, script_path)
foo = importlib.util.module_from_spec(spec)
spec.loader.exec_module(foo)
return foo.__doc__
if __name__ == '__main__':
print(get_docstring('docstring_test', "./docstring_test.py"))
Output:
hello world
this is a multiline
docstring.
Update:
Here's how to do it by letting the ast module in the standard library do the parsing which avoids both importing/executing the script as well as trying to parse it yourself with a regex.
This looks more-or-less equivalent to what's in your question, so it's unclear why what you have isn't working for you.
import ast
def get_docstring(script_path):
with open(script_path, 'r') as file:
tree = ast.parse(file.read())
return ast.get_docstring(tree, clean=False)
if __name__ == '__main__':
print(repr(get_docstring('./docstring_test.py')))
Output:
' this is a multiline\n docstring.\n'
I need to have a couple of functions in Python (either variation) to find and print the name of the file they are stored or called from. For example, consider the following functions are stored in at this address: /my/py/func.py:
def this_file():
# print the address of this file
print('this function is stored at %s' % this_file_address)
and
def that_file():
# print the address of the file that is calling this function
print('this function is called form a file at %s' % that_file_address)
And I have a piece of code stored in /my/py/calls.py:
from func import *
this_file()
that_file()
Now, I want the followings to be printed by the above functions:
/my/py/func.py
/my/py/calls.py
How can I write these functions?
Edit #1
It seems calling that_file() from Jupyter notebooks should be handled differently.
import os
import sys
def this_file():
print(os.path.realpath(__file__))
def that_file():
print(os.getcwd() + "/" + sys.argv[0])
I think this is what you're looking for.
Thanks to #quantik and #Iguananaut (see this), I could find a more general solution that works for calling Python functions from .py and .ipynb files:
func.py
Content:
import os.path
import sys
import urllib.request
import json
def this_file():
# prints the address of this file
print(__file__)
return __file__
def that_file():
# prints the address of the file that is calling this function
if sys.argv[0][-21:]=='ipykernel_launcher.py':
print('Are you calling me from a Jupyter Notebook? Try "that_notebook()" instead.')
return False
else:
print(os.getcwd() + "/" + sys.argv[0])
return os.getcwd() + "/" + sys.argv[0]
def that_notebook(base_url='http://127.0.0.1:8888'):
# prints the address of the notebook that is calling this function
## read more about Jupyter APIL: https://github.com/jupyter/jupyter/wiki/Jupyter-Notebook-Server-API
# See if the url is correct
try:
sessions = json.load(urllib.request.urlopen(base_url+'/api/sessions'))
except:
print('Oops! %s is an invalid URL.' % (base_url+'/api/sessions'))
return False
# See if there is any active session
if len(sessions) == 0:
print('No active session found!')
print('Are you calling me from a Python file? Try "that_file()" instead.')
return False
# In case of multiple active sessions, only print the most recently
latest=max([s['kernel']['last_activity'] for s in sessions])
for s in sessions:
if s['kernel']['last_activity']==latest:
print(s['path'])
return(s['path'])
calls.py
Contents:
from func import *
this_file()
that_file()
that_notebook()
Outputs:
python calls.py
/home/jovyan/work/calls.py
No active session found!
Are you calling me from a Python file? Try "that_file()" instead.
jovyan#c5cd7b908543:~/work$
calls.ipynb
Contents:
from func import *
this_file()
that_file()
that_notebook()
Outputs:
calls.ipynb
/home/jovyan/work/func.py
Are you calling me from a Jupyter Notebook? Try "that_notebook()" instead.
work/calls.ipynb
I am trying to write a logging module that I can use over and over again. The logging is not the problem. I want to call my logging module from any script and not have to pass the params for the logging file.
If I have a test script called mytest.py how would I return this name in the import logging module. I have tried this in the logging script itself but it returns the name of that file and not the file I am trying to log.
my_test.py:
from my_logger import logging
info.logging("something here")
print("something here")
if __name__ == "__main__":
logging()
I would expect the log file to be called my_test.log, but currently it is being named logging.log
Here is the part from the logging script:
def logging(filename=False, level=DEFAULT_LOG_LEVEL):
if filename is False:
file_ext = os.path.basename(__file__) # Need this to be my_test.py
filename = ("C:/Users/Logs/{0}").format(file_ext)
"Start logging with given filename and level."
#print(filename)
logging.basicConfig(filename=filename, level=LEVELS[level])
else:
"Start logging with given filename and level."
logging.basicConfig(filename=filename, level=LEVELS[level])
Assuming your compiler is CPython (thanks to Matthew Trevor);
You can use inspect.getouterframes to get the caller's frame, plus the filename and line number etc.
import inspect
def logging(filename=False, level=DEFAULT_LOG_LEVEL):
if filename is False:
caller_file = inspect.getouterframes(inspect.currentframe())[1][3]
# prints /home/foo/project/my_test.py
...
The __file__ binding you use in your logging function will hold the value of the file it's scoped in. You need to pass in the calling module's __file__ to get the behaviour you want:
# my_test.py
...
if __name__ == "__main__":
logging(__file__)
I'm surprised that your log file was called logging.log and not my_logger.log, though, as that's the name of the file in which the logging function is defined.
I am doing browser automation using python + splinter.
my structure is like this
[root]
+--start.py
+--end.py
+--[module1]
| +--mod11area1.py
| +--mod12area2.py
| +--[module1_2]
| | +--mod121area1.py
| +--[module1_3]
| +--mod131area1.py
+--[module2]
+--mod21area1.py
start.py sets the initialization and opening of the browser
and the inner modules.py performs actions per module
this structure would then be merged into one script upon execute by appending the contents in this fasion:
start.py
mod11area1.py
mod12area2.py
mod121area1.py
mod131area1.py
mod21area1.py
end.py
My question is, is there a better way of doing this? I'm quite new to this and just usually create a single script. since my project keeps on expanding I had to employ several other guys to script with me. Hence I came up with this approach.
No, Python has no simple way to merge scripts into one .py file.
But you can fake it, albeit in a fairly limited way.
Heres an example of how you can define multiple modules (each with their own namespace), in a single file.
But has the following limitations.
No package support(although this could be made to work).
No support for modules depending on eachother(a module can't be imported unless its already defined).
Example - 2 modules, each containing a function:
# Fake multiple modules in a single file.
import sys
_globals_init = None # include ourself in namespace
_globals_init = set(globals().keys())
# ------------------------
# ---- begin
__name__ = "test_module_1"
__doc__ = "hello world"
def test2():
print(123)
sys.modules[__name__] = type(sys)(__name__, __doc__)
sys.modules[__name__].__dict__.update(globals())
[globals().__delitem__(k) for k in list(globals().keys()) if k not in _globals_init]
# ---- end ------------
# ---------------------
# ---- begin
__name__ = "some_other"
__doc__ = "testing 123"
def test1():
print(321)
sys.modules[__name__] = type(sys)(__name__, __doc__)
sys.modules[__name__].__dict__.update(globals())
[globals().__delitem__(k) for k in list(globals().keys()) if k not in _globals_init]
# ---- end ------------
# ----------------
# ---- example use
import test_module_1
test_module_1.test2()
import some_other
some_other.test1()
# this will fail (as it should)
test1()
Note, this isn't good practice, if you have this problem, you're probably better off with some alternative solution (such as using https://docs.python.org/3/library/zipimport.html)
See my GitHub project.
There is likely a better way for your needs. I developed this project/hack for programming contests which only allow the contestant to submit a single .py file. This allows one to develop a project with multiple .py files and then combine them into one .py file at the end.
My hack is a decorator #modulize which converts a function into a module. This module can then be imported as usual. Here is an example.
#modulize('my_module')
def my_dummy_function(__name__): # the function takes one parameter __name__
# put module code here
def my_function(s):
print(s, 'bar')
# the function must return locals()
return locals()
# import the module as usual
from my_module import my_function
my_function('foo') # foo bar
I also have a script which can combine a project of many .py files which import each other into one '.py' file.
For example, assume I had the following directory structure and files:
my_dir/
__main__.py
import foo.bar
fb = foo.bar.bar_func(foo.foo_var)
print(fb) # foo bar
foo/
__init__.py
foo_var = 'foo'
bar.py
def bar_func(x):
return x + ' bar'
The combined file will look as follows. The code on the top defines the #modulize decorator.
import sys
from types import ModuleType
class MockModule(ModuleType):
def __init__(self, module_name, module_doc=None):
ModuleType.__init__(self, module_name, module_doc)
if '.' in module_name:
package, module = module_name.rsplit('.', 1)
get_mock_module(package).__path__ = []
setattr(get_mock_module(package), module, self)
def _initialize_(self, module_code):
self.__dict__.update(module_code(self.__name__))
self.__doc__ = module_code.__doc__
def get_mock_module(module_name):
if module_name not in sys.modules:
sys.modules[module_name] = MockModule(module_name)
return sys.modules[module_name]
def modulize(module_name, dependencies=[]):
for d in dependencies: get_mock_module(d)
return get_mock_module(module_name)._initialize_
##===========================================================================##
#modulize('foo')
def _foo(__name__):
##----- Begin foo/__init__.py ------------------------------------------------##
foo_var = 'foo'
##----- End foo/__init__.py --------------------------------------------------##
return locals()
#modulize('foo.bar')
def _bar(__name__):
##----- Begin foo/bar.py -----------------------------------------------------##
def bar_func(x):
return x + ' bar'
##----- End foo/bar.py -------------------------------------------------------##
return locals()
def __main__():
##----- Begin __main__.py ----------------------------------------------------##
import foo.bar
fb = foo.bar.bar_func(foo.foo_var)
print(fb) # foo bar
##----- End __main__.py ------------------------------------------------------##
__main__()
Instead of appending the contents into a single *.py file, why not just import what you need from the code that the other people in your team write?
File "G:\Python25\Lib\site-packages\PyAMF-0.6b2-py2.5-win32.egg\pyamf\util\__init__.py", line 15, in <module>
ImportError: cannot import name python
How do I fix it?
If you need any info to know how to fix this problem, I can explain, just ask.
Thanks
Code:
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext import webapp
from TottysGateway import TottysGateway
import logging
def main():
services_root = 'services'
#services = ['users.login']
#gateway = TottysGateway(services, services_root, logger=logging, debug=True)
#app = webapp.WSGIApplication([('/', gateway)], debug=True)
#run_wsgi_app(app)
if __name__ == "__main__":
main()
Code:
from pyamf.remoting.gateway.google import WebAppGateway
import logging
class TottysGateway(WebAppGateway):
def __init__(self, services_available, root_path, not_found_service, logger, debug):
# override the contructor and then call the super
self.services_available = services_available
self.root_path = root_path
self.not_found_service = not_found_service
WebAppGateway.__init__(self, {}, logger=logging, debug=True)
def getServiceRequest(self, request, target):
# override the original getServiceRequest method
try:
# try looking for the service in the services list
return WebAppGateway.getServiceRequest(self, request, target)
except:
pass
try:
# don't know what it does but is an error for now
service_func = self.router(target)
except:
if(target in self.services_available):
# only if is an available service import it's module
# so it doesn't access services that should be hidden
try:
module_path = self.root_path + '.' + target
paths = target.rsplit('.')
func_name = paths[len(paths) - 1]
import_as = '_'.join(paths) + '_' + func_name
import_string = "from "+module_path+" import "+func_name+' as service_func'
exec import_string
except:
service_func = False
if(not service_func):
# if is not found load the default not found service
module_path = self.rootPath + '.' + self.not_found_service
import_string = "from "+module_path+" import "+func_name+' as service_func'
# add the service loaded above
assign_string = "self.addService(service_func, target)"
exec assign_string
return WebAppGateway.getServiceRequest(self, request, target)
You need to post your full traceback. What you show here isn't all that useful. I ended up digging up line 15 of pyamf/util/init.py. The code you should have posted is
from pyamf import python
This should not fail unless your local environment is messed up.
Can you 'import pyamf.util' and 'import pyamf.python' in a interactive Python shell? What about if you start Python while in /tmp (on the assumption that you might have a file named 'pyamf.py' in the current directory. Which is a bad thing.)
= (older comment below) =
Fix your question. I can't even tell where line 15 of util/__init__.py is supposed to be. Since I can't figure that out, I can't answer your question. Instead, I'll point out ways to improve your question and code.
First, use the markup language correctly, so that all the code is in a code block. Make sure you've titled the code, so we know it's from util/__init__.py and not some random file.
In your error message, include the full traceback, and not the last two lines.
Stop using parens in things like "if(not service_func):" and use a space instead, so its " if not service_func:". This is discussed in PEP 8.
Read the Python documentation and learn how to use the language. Something like "func_name = paths[len(paths) - 1]" should be "func_name = paths[-1]"
Learn about the import function and don't use "exec" for this case. Nor do you need the "exec assign_string" -- just do the "self.addService(service_func, target)"