We Keep Coding

Python How to force object instantiation via Context Manager?

I want to force object instantiation via class context manager. So make it impossible to instantiate directly.
I implemented this solution, but technically user can still instantiate object.
class HessioFile:
"""
Represents a pyhessio file instance
"""
def __init__(self, filename=None, from_context_manager=False):
if not from_context_manager:
raise HessioError('HessioFile can be only use with context manager')
And context manager:
#contextmanager
def open(filename):
"""
...
"""
hessfile = HessioFile(filename, from_context_manager=True)
Any better solution ?

If you consider that your clients will follow basic python coding principles then you can guarantee that no method from your class will be called if you are not within the context.
Your client is not supposed to call __enter__ explicitly, therefore if __enter__ has been called you know your client used a with statement and is therefore inside context (__exit__ will be called).
You just need to have a boolean variable that helps you remember if you are inside or outside context.
class Obj:
def __init__(self):
self._inside_context = False
def __enter__(self):
self._inside_context = True
print("Entering context.")
return self
def __exit__(self, *exc):
print("Exiting context.")
self._inside_context = False
def some_stuff(self, name):
if not self._inside_context:
raise Exception("This method should be called from inside context.")
print("Doing some stuff with", name)
def some_other_stuff(self, name):
if not self._inside_context:
raise Exception("This method should be called from inside context.")
print("Doing some other stuff with", name)
with Obj() as inst_a:
inst_a.some_stuff("A")
inst_a.some_other_stuff("A")
inst_b = Obj()
with inst_b:
inst_b.some_stuff("B")
inst_b.some_other_stuff("B")
inst_c = Obj()
try:
inst_c.some_stuff("c")
except Exception:
print("Instance C couldn't do stuff.")
try:
inst_c.some_other_stuff("c")
except Exception:
print("Instance C couldn't do some other stuff.")
This will print:
Entering context.
Doing some stuff with A
Doing some other stuff with A
Exiting context.
Entering context.
Doing some stuff with B
Doing some other stuff with B
Exiting context.
Instance C couldn't do stuff.
Instance C couldn't do some other stuff.
Since you'll probably have many methods that you want to "protect" from being called from outside context, then you can write a decorator to avoid repeating the same code to test for your boolean:
def raise_if_outside_context(method):
def decorator(self, *args, **kwargs):
if not self._inside_context:
raise Exception("This method should be called from inside context.")
return method(self, *args, **kwargs)
return decorator
Then change your methods to:
#raise_if_outside_context
def some_other_stuff(self, name):
print("Doing some other stuff with", name)

I suggest the following approach:
class MainClass:
def __init__(self, *args, **kwargs):
self._class = _MainClass(*args, **kwargs)
def __enter__(self):
print('entering...')
return self._class
def __exit__(self, exc_type, exc_val, exc_tb):
# Teardown code
print('running exit code...')
pass
# This class should not be instantiated directly!!
class _MainClass:
def __init__(self, attribute1, attribute2):
self.attribute1 = attribute1
self.attribute2 = attribute2
...
def method(self):
# execute code
if self.attribute1 == "error":
raise Exception
print(self.attribute1)
print(self.attribute2)
with MainClass('attribute1', 'attribute2') as main_class:
main_class.method()
print('---')
with MainClass('error', 'attribute2') as main_class:
main_class.method()
This will outptut:
entering...
attribute1
attribute2
running exit code...
---
entering...
running exit code...
Traceback (most recent call last):
File "scratch_6.py", line 34, in <module>
main_class.method()
File "scratch_6.py", line 25, in method
raise Exception
Exception

None that I am aware of. Generally, if it exists in python, you can find a way to call it. A context manager is, in essence, a resource management scheme... if there is no use-case for your class outside of the manager, perhaps the context management could be integrated into the methods of the class? I would suggest checking out the atexit module from the standard library. It allows you to register cleanup functions much in the same way that a context manager handles cleanup, but you can bundle it into your class, such that each instantiation has a registered cleanup function. Might help.
It is worth noting that no amount of effort will prevent people from doing stupid things with your code. Your best bet is generally to make it as easy as possible for people to do smart things with your code.

You can think of hacky ways to try and enforce this (like inspecting the call stack to forbid direct calls to your object, boolean attribute that is set upon __enter__ that you check before allowing other actions on the instance) but that will eventually become a mess to understand and explain to others.
Irregardless, you should also be certain that people will always find ways to bypass it if wanted. Python doesn't really tie your hands down, if you want to do something silly it lets you do it; responsible adults, right?
If you need an enforcement, you'd be better off supplying it as a documentation notice. That way if users opt to instantiate directly and trigger unwanted behavior, it's their fault for not following guidelines for your code.

How to call __init__ method of the parent class after dynamic class creation in Python?

I want dynamically create a class from two Base classes (ExampleTestRun and Thread). Main purpose for this is to start ExampleTestRun method "run" in a thread (by calling Thread class method "start"). I never used super() in Python. Maybe with it would be an answer?
from threading import Thread
class BaseTest(object):
def __init__(self):
pass
class ExampleTestRun(BaseTest):
def run(self):
try:
a = int('A')
except Exception as ex:
print ex
if __name__ == "__main__":
test_class = type('TestClass', (ExampleTestRun, Thread), {})
test = test_class()
test.start()
I got error:
Traceback (most recent call last):
File "D:/Dropbox/Workspaces/PyCharmWorkspace/ElgsisTests/src/mcltests/mcltransparent/run.py", line 30, in <module>
test.start()
File "C:\Python27\lib\threading.py", line 737, in start
raise RuntimeError("thread.__init__() not called")
RuntimeError: thread.__init__() not called

It throws that error becuase you are overriding the __init__ of the newly created class. Replacing that re-definition with pass altogether will fix it:
class BaseTest(object):
pass
Of course if you want to expand on the original constructor, you will have to resort to super. So the full code will look like this:
from threading import Thread
class BaseTest(object):
def __init__(self):
super(BaseTest, self).__init__()
class ExampleTestRun(BaseTest):
def run(self):
try:
a = int('A')
except Exception as ex:
print ex
if __name__ == "__main__":
test_class = type('TestClass', (ExampleTestRun, Thread), {})
test = test_class()
test.start()

With statement destructor to catch init exceptions

I use a with statement with the following class.
def __init__(self):
...
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
print "EXIT Shutting the SDK down"
ret = self.sdkobject.ShutDown()
self.error_check(ret)
This catches any error that occur when I am using the object of the class and safely shuts down the SDK that I am using. However, it catch problems when the class is still initializing. I have recently found the "del" function which neatly solves this problem. However, it can't be used in conjunction with the exit function (as the with statement evokes the exit and the del gets an exception). How can I set up a destructor using a with statemtent, which will catch failures even during initialization?

Exceptions in the __init__ need to be dealt with directly in that method:
class YourContextManager(object):
sdkobject = None
def __init__(self):
try:
self._create_sdk_object()
except Exception:
if self.sdkobject is not None:
self.sdkobject.ShutDown()
raise
def _create_sdk_object(self):
self.sdkobject = SomeSDKObject()
self.sdkobject.do_something_that_could_raise_an_exception()
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
print "EXIT Shutting the SDK down"
ret = self.sdkobject.ShutDown()
self.error_check(ret)
Note that the exception is re-raised; you want to give the consumer of the context manager an opportunity to handle the failure to create a context manager.

Create a separate shutdown function that gets called in the try/except block of the __init__ and wherever else you need a proper shutdown.

Catch the exception in __init__ and handle it. __del__ is unnecessary.

Catching exception in context manager __enter__()

Is it possible to ensure the __exit__() method is called even if there is an exception in __enter__()?
>>> class TstContx(object):
... def __enter__(self):
... raise Exception('Oops in __enter__')
...
... def __exit__(self, e_typ, e_val, trcbak):
... print "This isn't running"
...
>>> with TstContx():
... pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __enter__
Exception: Oops in __enter__
>>>
Edit
This is as close as I could get...
class TstContx(object):
def __enter__(self):
try:
# __enter__ code
except Exception as e
self.init_exc = e
return self
def __exit__(self, e_typ, e_val, trcbak):
if all((e_typ, e_val, trcbak)):
raise e_typ, e_val, trcbak
# __exit__ code
with TstContx() as tc:
if hasattr(tc, 'init_exc'): raise tc.init_exc
# code in context
In hind sight, a context manager might have not been the best design decision

Like this:
import sys
class Context(object):
def __enter__(self):
try:
raise Exception("Oops in __enter__")
except:
# Swallow exception if __exit__ returns a True value
if self.__exit__(*sys.exc_info()):
pass
else:
raise
def __exit__(self, e_typ, e_val, trcbak):
print "Now it's running"
with Context():
pass
To let the program continue on its merry way without executing the context block you need to inspect the context object inside the context block and only do the important stuff if __enter__ succeeded.
class Context(object):
def __init__(self):
self.enter_ok = True
def __enter__(self):
try:
raise Exception("Oops in __enter__")
except:
if self.__exit__(*sys.exc_info()):
self.enter_ok = False
else:
raise
return self
def __exit__(self, e_typ, e_val, trcbak):
print "Now this runs twice"
return True
with Context() as c:
if c.enter_ok:
print "Only runs if enter succeeded"
print "Execution continues"
As far as I can determine, you can't skip the with-block entirely. And note that this context now swallows all exceptions in it. If you wish not to swallow exceptions if __enter__ succeeds, check self.enter_ok in __exit__ and return False if it's True.

No. If there is the chance that an exception could occur in __enter__() then you will need to catch it yourself and call a helper function that contains the cleanup code.

I suggest you follow RAII (resource acquisition is initialization) and use the constructor of your context to do the potentially failing allocation. Then your __enter__ can simply return self which should never ever raise an exception. If your constructor fails, the exception may be thrown before even entering the with context.
class Foo:
def __init__(self):
print("init")
raise Exception("booh")
def __enter__(self):
print("enter")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print("exit")
return False
with Foo() as f:
print("within with")
Output:
init
Traceback (most recent call last):
File "<input>", line 1, in <module>
...
raise Exception("booh")
Exception: booh
Edit:
Unfortunately this approach still allows the user to create "dangling" resources that wont be cleaned up if he does something like:
foo = Foo() # this allocates resource without a with context.
raise ValueError("bla") # foo.__exit__() will never be called.
I am quite curious if this could be worked around by modifying the new implementation of the class or some other python magic that forbids object instantiation without a with context.

You could use contextlib.ExitStack (not tested):
with ExitStack() as stack:
cm = TstContx()
stack.push(cm) # ensure __exit__ is called
with ctx:
stack.pop_all() # __enter__ succeeded, don't call __exit__ callback
Or an example from the docs:
stack = ExitStack()
try:
x = stack.enter_context(cm)
except Exception:
# handle __enter__ exception
else:
with stack:
# Handle normal case
See contextlib2 on Python <3.3.

if inheritance or complex subroutines are not required, you can use a shorter way:
from contextlib import contextmanager
#contextmanager
def test_cm():
try:
# dangerous code
yield
except Exception, err
pass # do something

class MyContext:
def __enter__(self):
try:
pass
# exception-raising code
except Exception as e:
self.__exit__(e)
def __exit__(self, *args):
# clean up code ...
if args[0]:
raise
I've done it like this. It calls __exit__() with the error as the argument. If args[0] contains an error it reraises the exception after executing the clean up code.

The docs contain an example that uses contextlib.ExitStack for ensuring the cleanup:
As noted in the documentation of ExitStack.push(), this method can be useful in cleaning up an already allocated resource if later steps in the __enter__() implementation fail.
So you would use ExitStack() as a wrapping context manager around the TstContx() context manager:
from contextlib import ExitStack
with ExitStack() as stack:
ctx = TstContx()
stack.push(ctx) # Leaving `stack` now ensures that `ctx.__exit__` gets called.
with ctx:
stack.pop_all() # Since `ctx.__enter__` didn't raise it can handle the cleanup itself.
... # Here goes the body of the actual context manager.

python: closures and classes

I need to register an atexit function for use with a class (see Foo below for an example) that, unfortunately, I have no direct way of cleaning up via a method call: other code, that I don't have control over, calls Foo.start() and Foo.end() but sometimes doesn't call Foo.end() if it encounters an error, so I need to clean up myself.
I could use some advice on closures in this context:
class Foo:
def cleanup(self):
# do something here
def start(self):
def do_cleanup():
self.cleanup()
atexit.register(do_cleanup)
def end(self):
# cleanup is no longer necessary... how do we unregister?
Will the closure work properly, e.g. in do_cleanup, is the value of self bound correctly?
How can I unregister an atexit() routine?
Is there a better way to do this?
edit: this is Python 2.6.5

Make a registry a global registry and a function that calls a function in it, and remove them from there when necessary.
cleaners = set()
def _call_cleaners():
for cleaner in list(cleaners):
cleaner()
atexit.register(_call_cleaners)
class Foo(object):
def cleanup(self):
if self.cleaned:
raise RuntimeError("ALREADY CLEANED")
self.cleaned = True
def start(self):
self.cleaned = False
cleaners.add(self.cleanup)
def end(self):
self.cleanup()
cleaners.remove(self.cleanup)

I think the code is fine. There's no way to unregister, but you can set a boolean flag that would disable cleanup:
class Foo:
def __init__(self):
self.need_cleanup = True
def cleanup(self):
# do something here
print 'clean up'
def start(self):
def do_cleanup():
if self.need_cleanup:
self.cleanup()
atexit.register(do_cleanup)
def end(self):
# cleanup is no longer necessary... how do we unregister?
self.need_cleanup = False
Lastly, bear in mind that atexit handlers don't get called if "the program is killed by a signal not handled by Python, when a Python fatal internal error is detected, or when os._exit() is called."

self is bound correctly inside the callback to do_cleanup, but in fact if all you are doing is calling the method you might as well use the bound method directly.
You use atexit.unregister() to remove the callback, but there is a catch here as you must unregister the same function that you registered and since you used a nested function that means you have to store a reference to that function. If you follow my suggestion of using a bound method then you still have to save a reference to it:
class Foo:
def cleanup(self):
# do something here
def start(self):
self._cleanup = self.cleanup # Need to save the bound method for unregister
atexit.register(self._cleanup)
def end(self):
atexit.unregister(self._cleanup)
Note that it is still possible for your code to exit without calling ther atexit registered functions, for example if the process is aborted with ctrl+break on windows or killed with SIGABRT on linux.
Also as another answer suggests you could just use __del__ but that can be problematic for cleanup while a program is exiting as it may not be called until after other globals it needs to access have been deleted.
Edited to note that when I wrote this answer the question didn't specify Python 2.x. Oh well, I'll leave the answer here anyway in case it helps anyone else.

Since shanked deleted his posting, I'll speak in favor of __del__ again:
import atexit, weakref
class Handler:
def __init__(self, obj):
self.obj = weakref.ref(obj)
def cleanup(self):
if self.obj is not None:
obj = self.obj()
if obj is not None:
obj.cleanup()
class Foo:
def __init__(self):
self.start()
def cleanup(self):
print "cleanup"
self.cleanup_handler = None
def start(self):
self.cleanup_handler = Handler(self)
atexit.register(self.cleanup_handler.cleanup)
def end(self):
if self.cleanup_handler is None:
return
self.cleanup_handler.obj = None
self.cleanup()
def __del__(self):
self.end()
a1=Foo()
a1.end()
a1=Foo()
a2=Foo()
del a2
a3=Foo()
a3.m=a3
This supports the following cases:
objects where .end is called regularly; cleanup right away
objects that are released without .end being called; cleanup when the last
reference goes away
objects living in cycles; cleanup atexit
objects that are kept alive; cleanup atexit
Notice that it is important that the cleanup handler holds a weak reference
to the object, as it would otherwise keep the object alive.
Edit: Cycles involving Foo will not be garbage-collected, since Foo implements __del__. To allow for the cycle being deleted at garbage collection time, the cleanup must be taken out of the cycle.
class Cleanup:
cleaned = False
def cleanup(self):
if self.cleaned:
return
print "cleanup"
self.cleaned = True
def __del__(self):
self.cleanup()
class Foo:
def __init__(self):...
def start(self):
self.cleaner = Cleanup()
atexit.register(Handler(self).cleanup)
def cleanup(self):
self.cleaner.cleanup()
def end(self):
self.cleanup()
It's important that the Cleanup object has no references back to Foo.

Why don't you try it? It only took me a minute to check.
(Answer: Yes)
However, you can simplify it. The closure isn't needed.
class Foo:
def cleanup(self):
pass
def start(self):
atexit.register(self.cleanup)
And to not cleanup twice, just check in the cleanup method if a cleanup is needed or not before you clean up.