Is it possible to create a dynamic localized scope in Python? - python

I have a scenario where I'm dynamically running functions at run-time and need to keep track of a "localized" scope. In the example below, "startScope" and "endScope" would actually be creating levels of "nesting" (in reality, the stuff contained in this localized scope isn't print statements...it's function calls that send data elsewhere and the nesting is tracked there. startScope / endScope just set control flags that are used to start / end the current nesting depth).
This all works fine for tracking the nested data, however, exceptions are another matter. Ideally, an exception would result in "falling out" of the current localized scope and not end the entire function (myFunction in the example below).
def startScope():
#Increment our control object's (not included in this example) nesting depth
control.incrementNestingDepth()
def endScope():
#Decrement our control object's (not included in this example) nesting depth
control.decrementNestingDepth()
def myFunction():
print "A"
print "B"
startScope()
print "C"
raise Exception
print "D"
print "This print statement and the previous one won't get printed"
endScope()
print "E"
def main():
try:
myFunction()
except:
print "Error!"
Running this would (theoretically) output the following:
>>> main()
A
B
C
Error!
E
>>>
I'm quite certain this isn't possible as I've written it above - I just wanted to paint a picture of the sort of end-result I'm trying to achieve.
Is something like this possible in Python?
Edit: A more relevant (albeit lengthy) example of how this is actually being used:
class Log(object):
"""
Log class
"""
def __init__(self):
#DataModel is defined elsewhere and contains a bunch of data structures / handles nested data / etc...
self.model = DataModel()
def Warning(self, text):
self.model.put("warning", text)
def ToDo(self, text):
self.model.put("todo", text)
def Info(self, text):
self.model.put("info", text)
def StartAdvanced(self):
self.model.put("startadvanced")
def EndAdvanced(self):
self.model.put("endadvanced")
def AddDataPoint(self, data):
self.model.put("data", data)
def StartTest(self):
self.model.put("starttest")
def EndTest(self):
self.model.put("endtest")
def Error(self, text):
self.model.put("error", text)
#myScript.py
from Logger import Log
def test_alpha():
"""
Crazy contrived example
In this example, there are 2 levels of nesting...everything up to StartAdvanced(),
and after EndAdvanced() is included in the top level...everything between the two is
contained in a separate level.
"""
Log.Warning("Better be careful here!")
Log.AddDataPoint(fancyMath()[0])
data = getSerialData()
if data:
Log.Info("Got data, let's continue with an advanced test...")
Log.StartAdvanced()
#NOTE: If something breaks in one of the following methods, then GOTO (***)
operateOnData(data)
doSomethingCrazy(data)
Log.ToDo("Fill in some more stuff here later...")
Log.AddDataPoint(data)
Log.EndAdvanced()
#(***) Ideally, we would resume here if an exception is raised in the above localized scope
Log.Info("All done! Log some data and wrap everything up!")
Log.AddDataPoint({"data": "blah"})
#Done
#framework.py
import inspect
from Logger import Log
class Framework(object):
def __init__(self):
print "Framework init!"
self.tests = []
def loadTests(self, file):
"""
Simplifying this for the sake of clarity
"""
for test in file:
self.tests.append(test)
def runTests(self):
"""
Simplifying this for the sake of clarity
"""
#test_alpha() as well as any other user tests will be run here
for test in self.tests:
Log.StartTest()
try:
test()
except Exception,e :
Log.Error(str(e))
Log.EndTest()
#End

You can achieve a similar effect with a context manager using a with statement. Here I use the contextlib.contextmanager decorator:
#contextlib.contextmanager
def swallower():
try:
yield
except ZeroDivisionError:
print("We stopped zero division error")
def foo():
print("This error will be trapped")
with swallower():
print("Here comes error")
1/0
print("This will never be reached")
print("Merrily on our way")
with swallower():
print("This error will propagate")
nonexistentName
print("This won't be reached")
>>> foo()
This error will be trapped
Here comes error
We stopped zero division error
Merrily on our way
This error will propagate
Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
foo()
File "<pyshell#3>", line 10, in foo
nonexistentName
NameError: global name 'nonexistentName' is not defined
It cannot be done with an ordinary function call as in your example. In your example, the function startScope returns before the rest of the body of myFunction executes, so startScope can't have any effect on it. To handle exceptions, you need some kind of explicit structure (either a with statement or a regular try/except) inside myFunction; there's no way to make a simple function call magically intercept exceptions that are raised in its caller.
You should read up on context managers as they seem to fit what you're trying to do. The __enter__ and __exit__ methods of the context manager would correspond to your startScope and endScope. Whether it will do exactly what you want depends on exactly what you want those "manager" functions to do, but you will probably have more luck doing it with a context manager than trying to do it with simple function calls.

Related

How to provide feedback that an error occurred in the error handling methods of a class?

I am working with a class in python that is part of a bigger program. The class is calling different methods.
If there is an error in one of the method I would like code to keep running after, but after the program is finished, I want to be able to see which methods had potential errors in them.
Below is roughly how I am structuring it at the moment, and this solution doesn't scale very well with more methods. Is there a better way to provide feedback (after the code has been fully run) as to which of the method had a potential error?
class Class():
def __init__(self):
try:
self.method_1()
except:
self.error_method1 = "Yes"
break
try:
self.method_2()
except:
self.error_method2 = "Yes"
break
try:
self.method_3()
except:
self.error_method3 = "Yes"
break
Although you could use sys.exc_info() to retrieve information about an Exception when one occurs as I mentioned in a comment, doing so may not be required since Python's standard try/expect mechanism seems adequate.
Below is a runnable example showing how to do so in order to provide "feedback" later about the execution of several methods of a class. This approach uses a decorator function, so should scale well since the same decorator can be applied to as many of the class' methods as desired.
from contextlib import contextmanager
from functools import wraps
import sys
from textwrap import indent
def provide_feedback(method):
""" Decorator to trap exceptions and add messages to feedback. """
#wraps(method)
def wrapped_method(self, *args, **kwargs):
try:
return method(self, *args, **kwargs)
except Exception as exc:
self._feedback.append(
'{!r} exception occurred in {}()'.format(exc, method.__qualname__))
return wrapped_method
class Class():
def __init__(self):
with self.feedback():
self.method_1()
self.method_2()
self.method_3()
#contextmanager
def feedback(self):
self._feedback = []
try:
yield
finally:
# Example of what could be done with any exception messages.
# They could instead be appended to some higher-level container.
if self._feedback:
print('Feedback:')
print(indent('\n'.join(self._feedback), ' '))
#provide_feedback
def method_1(self):
raise RuntimeError('bogus')
#provide_feedback
def method_2(self):
pass
#provide_feedback
def method_3(self):
raise StopIteration('Not enough foobar to go around')
inst = Class()
Output:
Feedback:
RuntimeError('bogus') exception occurred in Class.method_1()
StopIteration('Not enough foobar to go around') exception occurred in Class.method_3()

Python: decorator/wrapper for try/except statement

I have some blocks of code which need to be wrapped by function.
try:
if config.DEVELOPMENT == True:
# do_some_stuff
except:
logger.info("Config is not set for development")
Then I'll do again:
try:
if config.DEVELOPMENT == True:
# do_some_another_stuff
except:
logger.info("Config is not set for development")
So, how can I wrap this "do_some_stuff" and "do_some_another_stuff"?
I'm trying to write function with contextmanager:
#contextmanager
def try_dev_config(name):
try:
if name is not None:
yield
except Exception as e:
print "not dev config"
with try_dev_config("config.DEVELOPMENT"):
# do_some_stuff
And I got an error:
RuntimeError: generator didn't yield
You could pass in a function.
boolean = True
def pass_this_in():
print("I just did some stuff")
def the_try_except_bit(function):
try:
if boolean:
function()
except:
print("Excepted")
# Calling the above code
the_try_except_bit(pass_this_in)
If you want to reduce the "pass_this_in" definition bit, then you can use lambda function definitions:
pass_this_in = lambda : print("I just did some stuff")
I am not sure that a context manager is the good method to achieve what you want. The context manager goal is to provide a mecanism to open/instantiate a resource, give access to it (or not) and close/clean it automatically when you no more need it.
IMHO, what you need is a decorator.
A decorator aims at executing code around a function call. It would force you to put each block of code in a function but I don't think it is so difficult. You can implement it like this:
class Config(object):
"""for demonstration purpose only: used to have a config.DEVELOPMENT value"""
DEVELOPMENT = True
class Logger(object):
"""for demonstration purpose only: used to have a logger.info method"""
#staticmethod
def info(msg):
print("Logged: {}".format(msg))
def check_dev_config(config, logger):
def dev_config_checker(func):
def wrapper(*args, **kwargs):
try:
if config.DEVELOPMENT:
func(*args, **kwargs)
except Exception as err:
logger.info(
"Config is not set for developpement: {}".format(err))
return wrapper
return dev_config_checker
#check_dev_config(Config, Logger)
def do_stuff_1():
print("stuff 1 done")
#check_dev_config(Config, Logger)
def do_stuff_2():
raise Exception("stuff 2 failed")
do_stuff_1()
do_stuff_2()
This code prints
stuff 1 done
Logged: Config is not set for developpement: stuff 2 failed
Explanations:
The check_dev_config function is actually a decorator generator which accepts the config and the logger as arguments.
It returns the dev_config_checker function which is an actual (and parameterised) decorator, and which accepts a function to decorate as argument.
This decorator returns a wrapper function which will actually run code around the decorated function call. In this function, the decorated function is called inside a try/except structure and only if the config.DEVELOPMENT is evaluated to True. In case of exception, the logger is used to log an information.
Each block of code to decorate is put into a function (do_stuff_1, do_stuff_2 and decorated with the check_dev_config decorator generator, giving it the config and the logger.
When decorated functions are called, they are called via their decorator and not directly. As you can see, the do_stuff_2 exception has been catched and the a message has been logged.

Exceptions that reflect error codes of a remote service

I'm working with an external service which reports errors by code.
I have the list of error codes and the associated messages. Say, the following categories exist: authentication error, server error.
What is the smartest way to implement these errors in Python so I can always lookup an error by code and get the corresponding exception object?
Here's my straightforward approach:
class AuthError(Exception):
pass
class ServerError(Exception):
pass
map = {
1: AuthError,
2: ServerError
}
def raise_code(code, message):
""" Raise an exception by code """
raise map[code](message)
Would like to see better solutions :)
Your method is correct, except that map should be renamed something else (e.g. ERROR_MAP) so it does not shadow the builtin of the same name.
You might also consider making the function return the exception rather than raising it:
def error(code, message):
""" Return an exception by code """
return ERROR_MAP[code](message)
def foo():
raise error(code, message)
By placing the raise statement inside foo, you'd raise the error closer to where the error occurred and there would be one or two less lines to trace through if the stack trace is printed.
Another approach is to create a polymorphic base class which, being instantiated, actually produces a subclass that has the matching code.
This is implemented by traversing __subclasses__() of the parent class and comparing the error code to the one defined in the class. If found, use that class instead.
Example:
class CodeError(Exception):
""" Base class """
code = None # Error code
def __new__(cls, code, *args):
# Pick the appropriate class
for E in cls.__subclasses__():
if E.code == code:
C = E
break
else:
C = cls # fall back
return super(CodeError, cls).__new__(C, code, *args)
def __init__(self, code, message):
super(CodeError, self).__init__(message)
# Subclasses with error codes
class AuthError(CodeError):
code = 1
class ServerError(CodeError):
code = 2
CodeError(1, 'Wrong password') #-> AuthError
CodeError(2, 'Failed') #-> ServerError
With this approach, it's trivial to associate error message presets, and even map one class to multiple codes with a dict.

Finding out an exception context

tlndr: how to tell in a function if it's called from an except block (directly/indirectly). python2.7/cpython.
I use python 2.7 and try to provide something similar to py3's __context__ for my custom exception class:
class MyErr(Exception):
def __init__(self, *args):
Exception.__init__(self, *args)
self.context = sys.exc_info()[1]
def __str__(self):
return repr(self.args) + ' from ' + repr(self.context)
This seems to work fine:
try:
1/0
except:
raise MyErr('bang!')
#>__main__.MyErr: ('bang!',) from ZeroDivisionError('integer division or modulo by zero',)
Sometimes I need MyErr to be raised outside of an exception block. This is fine too:
raise MyErr('just so')
#>__main__.MyErr: ('just so',) from None
If, however, there has been a handled exception before this point, it's being incorrectly set as a context of MyErr:
try:
print xxx
except Exception as e:
pass
# ...1000 lines of code....
raise MyErr('look out')
#>__main__.MyErr: ('look out',) from NameError("name 'xxx' is not defined",) <-- BAD
I guess the reason is that sys.exc_info simply returns the "last" and not the "current" exception:
This function returns a tuple of three values that give information about the exception that is currently being handled. <...> Here, “handling an exception” is defined as “executing or having executed an except clause.”
So, my question is: how to tell if the interpreter is executing an except clause (and not has it executed in the past). In other words: is there a way to know in MyErr.__init__ if there is an except up on the stack?
My app is not portable, any Cpython specific hacks are welcome.
This is tested with CPython 2.7.3:
$ python myerr.py
MyErr('bang!',) from ZeroDivisionError('integer division or modulo by zero',)
MyErr('nobang!',)
It works as long as the magic exception is directly created within the scope of an except clause. A little additional code can lift that restriction, though.
import sys
import opcode
SETUP_EXCEPT = opcode.opmap["SETUP_EXCEPT"]
SETUP_FINALLY = opcode.opmap["SETUP_FINALLY"]
END_FINALLY = opcode.opmap["END_FINALLY"]
def try_blocks(co):
"""Generate code positions for try/except/end-of-block."""
stack = []
code = co.co_code
n = len(code)
i = 0
while i < n:
op = ord(code[i])
if op in (SETUP_EXCEPT, SETUP_FINALLY):
stack.append((i, i + ord(code[i+1]) + ord(code[i+2])*256))
elif op == END_FINALLY:
yield stack.pop() + (i,)
i += 3 if op >= opcode.HAVE_ARGUMENT else 1
class MyErr(Exception):
"""Magic exception."""
def __init__(self, *args):
callee = sys._getframe(1)
try:
in_except = any(i[1] <= callee.f_lasti < i[2] for i in try_blocks(callee.f_code))
finally:
callee = None
Exception.__init__(self, *args)
self.cause = sys.exc_info()[1] if in_except else None
def __str__(self):
return "%r from %r" % (self, self.cause) if self.cause else repr(self)
if __name__ == "__main__":
try:
try:
1/0
except:
x = MyErr('bang!')
raise x
except Exception as exc:
print exc
try:
raise MyErr('nobang!')
except Exception as exc:
print exc
finally:
pass
And remember, “Explicit is better than implicit,” so this would be way better if you ask me:
try:
…
except Exception as exc:
raise MyErr("msg", cause=exc)
The following approach might work, although it's a bit long-winded.
Get the code of the current frame from import inspect; inspect.currentframe().f_code
Inspect the bytecode (f_code.co_code), perhaps using dis.dis, to figure out whether the frame is being executed in an except block.
Depending on what you want to do, you might want to go back a frame and see if it wasn't called from an except block.
Ex:
def infoo():
raise MyErr("from foo in except")
try:
nope
except:
infoo()
If none of the frames are in an except block then the sys.exc_info() is outdated.
One solution would be to call sys.exc_clear() after an exception has been handled:
import sys
class MyErr(Exception):
def __init__(self, *args):
Exception.__init__(self, *args)
self.context = sys.exc_info()[1]
def __str__(self):
return repr(self.args) + ' from ' + repr(self.context)
try:
print xxx
except Exception as e:
# exception handled
sys.exc_clear()
raise MyErr('look out')
Gives:
Traceback (most recent call last):
File "test.py", line 18, in <module>
raise MyErr('look out')`
__main__.MyErr: ('look out',) from None
If there is not many places that handle an exception without raising MyErr then it might be more suitable then modifying calls to MyErr providing some constructor argument, or even explicitly handling traceback preservation as in this answer.
I searched through the Python source to see if there was some indicator that was being set when the entering an except block that could be queried by going through the frame sequence from the custom exception's constructor.
I found this fblocktype enum that is stored in a fblockinfo struct :
enum fblocktype { LOOP, EXCEPT, FINALLY_TRY, FINALLY_END };
struct fblockinfo {
enum fblocktype fb_type;
basicblock *fb_block;
};
There is a comment above the fblocktype that describes a frame block :
A frame block is used to handle loops, try/except, and try/finally.
It's called a frame block to distinguish it from a basic block in the
compiler IR.
And then when you go up a bit there is a description of a basic block :
Each basicblock in a compilation unit is linked via b_list in the
reverse order that the block are allocated. b_list points to the next
block, not to be confused with b_next, which is next by control flow.
Also reading some more here about about the Control Flow Graphs :
A control flow graph (often referenced by its acronym, CFG) is a
directed graph that models the flow of a program using basic blocks
that contain the intermediate representation (abbreviated “IR”, and in
this case is Python bytecode) within the blocks. Basic blocks
themselves are a block of IR that has a single entry point but
possibly multiple exit points. The single entry point is the key to
basic blocks; it all has to do with jumps. An entry point is the
target of something that changes control flow (such as a function call
or a jump) while exit points are instructions that would change the
flow of the program (such as jumps and ‘return’ statements). What this
means is that a basic block is a chunk of code that starts at the
entry point and runs to an exit point or the end of the block.
So all this seem to indicate that a frame block in Python's design is treated as a temporary object. It is not directly included in the Control Flow Graph except as part of the encompassing basic block's byte code, so it seems it can not be queried without parsing the frames byte code.
Further, I think the reason in your example the sys.exc_info is showing the exception from the try block is because it stores the last exception of the current basic block, frame blocks are not considered here.
sys.exc_info()
This function returns a tuple of three values that give information
about the exception that is currently being handled. The information
returned is specific both to the current thread and to the current
stack frame. If the current stack frame is not handling an exception,
the information is taken from the calling stack frame, or its caller,
and so on until a stack frame is found that is handling an exception.
Here, “handling an exception” is defined as “executing or having
executed an except clause.” For any stack frame, only information
about the most recently handled exception is accessible.
So when it says stack frame there I think it specifically means basic block and all the "handling an exception" talk means that exceptions in a frame block such as a try/except, for, and etc. bubble up to the basic block above.

I have a Python code that is given below, when I run this global name PDBAttributes is not defined

#!/usr/bin/env python2.7
##-*- mode:python;tab-width:2;indent-tabs-mode:t;show-trailing-whitespace:t;rm-trailing-spaces:t;python-indent:2 -*-'
import noesy
import argparse
import library
parser =argparse.ArgumentParser(description="read pdb file",
add_help=True)
parser.add_argument("file",help="protein pdb file")
library.add_standard_args( parser )
args = parser.parse_args()
def read_structure(pdbfile):
struct=[]
for line in pdbfile:
if len(line):
struct.append(PDBAttributes.read_from_line(line))
return struct
pdb=read_structure(open(args.file,'r'))
class PDBAttributes:
def __init__(self, atomindex=1, atom=noesy.Atom(), atomx=1, atomy=1, atomz=1):
self._atomindex=atomindex
self._atom=atom
self._atomx=atomx
self._atomy=atomy
self._atomz=atomz
def __str__(self):
s='ATOM %(_atomindex)d %(_atom)s at %(_atomx)8.3f %(_atomy)8.3f %(_atomz)8.3f'%self.__dict__
return s
def atom(self):
return self._atom
def atomindex(self):
return self._atomindex
def atomx(self):
return self._atomx
def atomy(self):
return self._atomy
def atomz(self):
return self._atomz
#classmethod
def read_from_line(obj,line):
tags=line.split()
atomindex=int(tags[1])
atom=noesy.Atom(tags[2],int(tags[5]))
atomx=float(tags[6])
atomy=float(tags[7])
atomz=float(tags[8])
obj=PDBAttributes(atomindex, atom, atomx, atomy, atomz)
print obj
class AtomDistance(PDBAttributes):
def distance(self, atom1,atom2):
pass
The NameError you are getting is due to the order you have placed the code in your file.
When you call read_structure to create a value for the pdb variable, it tries to look for PDBAttributes, but it has not been defined yet. If you move that line lower down in the file (below the class definition) you'll avoid that error. Note that it is OK to have the declaration of read_structure above the PDBAttributes class definition, though you might want to move it lower too to make the code easier to understand.
Here's a very simple bit of code that demonstrates the same error:
def foo():
print(foo_text)
foo() # raises a NameError
foo_text = "foo"
Here's a fixed version:
def foo():
print(foo_text)
foo_text = "foo"
foo() # no error, prints "foo"
Move your call of read_structure to follow the definition of the PDBAttributes class.
Also, in the process of reformatting your post, I see that you have mixed tabs and spaces for your indentation. Try reformatting your code to use all spaces for indentation, the recommended form is 4-space indents.
Your definition of all those getter functions looks like Java written in Python - this is a lot of extra code that is often unnecessary in Python. The recommended approach is to omit these all-they-do-is-assign-a-value-to-an-attribute-with-the-same-name-but-with-a-leading-underscore methods and just use attributes with the public names. See Python is Not Java.

Categories