We Keep Coding

Is raising exceptions safe within 'before' callback of a transitions.Machine model?

I am using the transitions FSM library. Imagine having an application FSM using the following code:
from transitions import Machine
import os
class Application(object):
states = ["idle", "data_loaded"]
def __init__(self):
self.data = None
machine = Machine(model=self, states=Application.states, initial="idle")
machine.add_transition("filename_dropped",
source="idle",
dest="data_loaded",
before="load_data",
conditions="is_valid_filename")
self.machine = machine
def drop_filename(self, filename):
try:
self.filename_dropped(filename)
except IOError as exc:
print "Oops: %s" % str(exc)
def load_data(self, filename):
with open(filename) as file:
self.data = file.read()
def is_valid_filename(self, filename):
return os.path.isfile(filename)
It can throw an IOError within load_data. My question is whether it is safe to raise exceptions (either implicitly like in this example or explicitly) from within before callbacks? In case of an IOError the transition is not taking place, the state of this example remains idle and any after callbacks are not being invoked. However, I wonder whether the internal state of the machine instance might get corrupted.
Additional question: Are there any better ways to signal errors with concrete information to the application? In this very example I could use the condition to load the file, but this seems ugly and I would need some additional attribute to keep track of the error etc.
Thanks for any help or advice.

However, I wonder whether the internal state of the machine instance might get corrupted.
It is fine to work with raising Exceptions in callback functions unless you plan to use the queued feature of transitions.
Transitions are executed in the following order:
prepare -> conditions -> before -> on_exit -> set_state -> on_enter -> after
If anything before set_state raises an Exception or a function in conditions does not return True, the transition is halted.
Your model might be in an undefined state though. If you rely on some 'clean up' or 'tear down' in State.on_enter or after:
from transitions import Machine
class Model:
def __init__(self):
self.busy = False
def before(self):
self.busy = True
raise Exception('oops')
def after(self):
# if state transition is done, reset busy
self.busy = False
model = Model()
m = Machine(model, states=['A','B'], initial='A',
transitions=[{'trigger':'go', 'source':'A', 'dest':'B',
'before':'before', 'after':'after'}])
try:
model.go()
except Exception as e:
print "Exception: %s" % e # Exception: oops
print "State: %s" % model.state # State: A
print "Model busy: %r" % model.busy # Model busy: True
Are there any better ways to signal errors with concrete information to the application?
It depends on what you want to achieve. Raising Errors/Exceptions usually halts the execution of a current task. In my oppinion this is pretty much THE way to propagate issues. If you want to handle the error and represent the error as a state, I would not consider using conditions ugly. Valid transitions with the same trigger are evaluated in the order they were added. With this in mind and the use of unless which is negated notation for conditions, your code could look like this:
from transitions import Machine
import os
class Application(object):
states = ["idle", "data_loaded", "filename_invalid", "data_invalid"]
transitions = [
{'trigger': 'filename_dropped', 'source': 'idle',
'dest': 'filename_invalid', 'unless': 'is_valid_filename'},
{'trigger':'filename_dropped', 'source': 'idle',
'dest':'data_invalid', 'unless': 'is_valid_data'},
{'trigger':'filename_dropped', 'source': 'idle',
'dest':'data_loaded'}
]
def __init__(self):
self.data = None
machine = Machine(model=self, states=Application.states,
transitions=Application.transitions, initial="idle")
self.machine = machine
def drop_filename(self, filename):
self.filename_dropped(filename)
if self.is_data_loaded():
print "Data loaded"
# renamed load_data
def is_valid_data(self, filename):
try:
with open(filename) as file:
self.data = file.read()
except IOError as exc:
print "File loading error: %s" % str(exc)
return False
return True
def is_valid_filename(self, filename):
return os.path.isfile(filename)
app = Application()
app.drop_filename('test.txt')
# >>> Data loaded
print app.state
# >>> data_loaded
app.to_idle()
app.drop_filename('missing.txt')
print app.state
# >>> filename_invalid
app.to_idle()
app.drop_filename('secret.txt')
# >>> File loading error: [Errno 13] Permission denied: 'secret.txt'
print app.state
# >>> data_invalid
This will create this state machine:

python 2.7 isinstance fails at dynamically imported module class

I'm currently writing some kind of tiny api to support extending module classes. Users should be able to just write their class name in a config and it gets used in our program. The contract is, that the class' module has a function called create(**kwargs) to return an instance of our base module class, and is placed in a special folder. But the isinstance check Fails as soon as the import is made dynamically.
modules are placed in lib/services/name
module base class (in lib/services/service)
class Service:
def __init__(self, **kwargs):
#some initialization
example module class (in lib/services/ping)
class PingService(Service):
def __init__(self, **kwargs):
Service.__init__(self,**kwargs)
# uninteresting init
def create(kwargs):
return PingService(**kwargs)
importing function
import sys
from lib.services.service import Service
def doimport( clazz, modPart, kw, class_check):
path = "lib/" + modPart
sys.path.append(path)
mod = __import__(clazz)
item = mod.create(kw)
if class_check(item):
print "im happy"
return item
calling code
class_check = lambda service: isinstance(service, Service)
s = doimport("ping", "services", {},class_check)
print s
from lib.services.ping import create
pingService = create({})
if isinstance(pingService, Service):
print "why this?"
what the hell am I doing wrong
here is a small example zipped up, just extract and run test.py without arguments
zip example

The problem was in your ping.py file. I don't know exactly why, but when dinamically importing it was not accepting the line from service import Service, so you just have to change it to the relative path: from lib.services.service import Service. Adding lib/services to the sys.path could not make it work the inheritance, which I found strange...
Also, I am using imp.load_source which seems more robust:
import os, imp
def doimport( clazz, modPart, kw, class_check):
path = os.path.join('lib', modPart, clazz + '.py')
mod = imp.load_source( clazz, path )
item = mod.create(kw)
if class_check(item):
print "im happy"
return item

Icon overlay in windows not reflecting in explorer

I m using python2.7 and py2exe to create dll from my python script.
Successfully created a DLL and registered an entry for my icon overlay status and then restarted windows explorer process through task manager.
Verified whether my entry is in registry and Yes, it was there.
But when i set my status through a python test app script for a specific folder location.
I expected all the files and folders in the selected path should overlayed with my overlay icon.
But No, Icon overlay is not happening at all.
But when i m testing through python script for register entry(without creating DLL) and setting my icon overlay through my test app script.
Yes, Its working perfectly.
I am confused why it is not happening when tried with my DLL???
Below is my python script to register an status entry
import os
import win32traceutil
import pythoncom
import winerror
from win32com.shell import shell, shellcon
from multiprocessing.connection import Client
REG_PATH = r'Software\Microsoft\Windows\CurrentVersion\Explorer\ShellIconOverlayIdentifiers'
REG_KEY = "IconOverlayTest"
class IconOverlay:
_reg_desc_ = 'Icon Overlay COM Server'
_public_methods_ = ['GetOverlayInfo', 'GetPriority','IsMemberOf']
_com_interfaces_ = [shell.IID_IShellIconOverlayIdentifier, pythoncom.IID_IDispatch]
def __init__(self, *_args, **_kwargs):
self._icon = None
self._icon_id = None
raise NotImplementedError
def GetOverlayInfo(self):
return self._icon, 0, shellcon.ISIOI_ICONFILE
def GetPriority(self):
return 0
def IsMemberOf(self, path, _attrs):
if is_member(path, self._icon_id):
return winerror.S_OK
return winerror.E_FAIL
class IconOverlay_test(IconOverlay):
_reg_progid_ = 'a.TestServer1'
_reg_clsid_ = '{8B19F050-8354-11E1-A0FE-5C260A5D15E4}'
def __init__(self):
self._icon = "C:\\Users\\Administrator\\mat\\icon_overlay\\icons\\1.ico"
self._icon_id = 101
classes = [IconOverlay_test,]
def is_member(path, icon_id):
try:
conn = None
conn = Client("\\\\.\\pipe\\test.listener", "AF_PIPE")
conn.send(path)
if conn.poll(3):
reply = conn.recv()
return reply == icon_id
except Exception:
pass
finally:
conn and conn.close()
return False
def DllRegisterServer():
print("Registering %s ......."%IconOverlay._reg_desc_)
import winreg
#winreg = _winreg
for view in [winreg.KEY_WOW64_64KEY, winreg.KEY_WOW64_32KEY]:
for cls in classes:
with winreg.CreateKeyEx(winreg.HKEY_LOCAL_MACHINE, r"%s\%s" %
(REG_PATH, cls._reg_progid_), 0,
winreg.KEY_ALL_ACCESS|view) as hkey:
print(" %s"%cls)
winreg.SetValueEx(hkey, None, 0, winreg.REG_SZ, cls._reg_clsid_)
print("Registration complete: %s" % IconOverlay._reg_desc_)
def DllUnregisterServer():
print("Unregistering %s ......."%IconOverlay._reg_desc_)
import winreg
#winreg = _winreg
for view in [winreg.KEY_WOW64_64KEY, winreg.KEY_WOW64_32KEY]:
for cls in classes:
try:
_key = winreg.DeleteKeyEx(winreg.HKEY_LOCAL_MACHINE, r"%s\%s"
% (REG_PATH, cls._reg_progid_),
winreg.KEY_ALL_ACCESS|view)
except WindowsError as err:
if err.errno != 2:
raise
print("Unregistration complete: %s" % IconOverlay._reg_desc_)
if __name__ == '__main__':
from win32com.server import register
register.UseCommandLine(*classes,
finalize_register = DllRegisterServer,
finalize_unregister = DllUnregisterServer)

This is really painful to get working right, goodluck!
I believe windows will only allow 10 different icons to be registered and it will only work with the first 10 registered alphabetically. Do you already have 10 registered? It's quite easy to exceed 10 if you have dropbox, tortoise-svn etc. installed as each image counts as an entry. If that's the case try putting an underscore or a 0 before the name to make sure it gets priority, although it will mean another icon will lose out- I don't think there's a way around this.
Also sometimes windows doesn't know to refresh the status of the icon. Which version of windows are you running? Some are worse than others. I seem to remember XP isn't very good at this. There's some tricks to get it to update though, you can refresh the window through windows api, but it looks horrible and the whole of explorer flashes. A better way I found is to change the an attribute on the file. This is the trick I used:
import stat,os
file_att= os.stat(path)[0]
if file_att & stat.S_IWRITE:
os.chmod(path,stat.S_IREAD)
os.chmod(path,stat.S_IWRITE)
else:
os.chmod(path,stat.S_IWRITE)
os.chmod(path,stat.S_IREAD)

How to monitor Python files for changes?

I want to restart my Python web application, if code gets changed. But there could be a large number of files that could be changed, since files in imported modules could change ...
How to get the actual file names from imported packages / modules?
How can modified Python files be detected efficiently? Is there a library to do that?

Shameless plug. There's also http://github.com/gorakhargosh/watchdog that I'm working on to do exactly this.
HTH.

gamin is another option which is slightly less Linux-specific.

I'm not sure how you would implement the 'reload application' operation in your circumstance; reloading a changed module with the reload built-in probably won't cut it.
But as far as detecting whether or not there was a change, the following would be one way to approach it.
Most python modules have a __file__ attribute.
All loaded modules are stored in sys.modules.
We can walk through sys.modules at some interval, and look for changes on disk for each module in turn
Sometimes __file__ points to a .pyc file instead of a .py file, so you might have to chop off the trailing c. Sometimes a .pyc file exists but a .py doesn't exist; in a robust system you'd have to allow for this.
A proof of concept of code to do this (not robust):
_module_timestamps = {}
_checking = False
def run_checker():
global _checking
_checking = True
while _checking:
for name, module in sys.modules.iteritems():
if hasattr(module, '__file__'):
filename = module.__file__
if filename.endswith('.pyc'):
filename = filename[:-1]
mtime = os.stat(filename).st_mtime
if name not in _module_timestamps:
_module_timestamps[name] = mtime
else:
if mtime > _module_timestamps[name]:
do_reload(name)
else:
'module %r has no file attribute' % (name,)
time.sleep(1)
def do_reload(modname):
print 'I would reload now, because of %r' % (modname,)
check_thread = threading.Thread(target=run_checker)
check_thread.daemon = True
check_thread.start()
try:
while 1:
time.sleep(0.1)
except KeyboardInterrupt:
print '\nexiting...'

Here's an example of how this could be implemented using pyinotify (ie., on Linux).
from importlib import import_module
class RestartingLauncher:
def __init__(self, module_name, start_function, stop_function, path="."):
self._module_name = module_name
self._filename = '%s.py' % module_name
self._start_function = start_function
self._stop_function = stop_function
self._path = path
self._setup()
def _setup(self):
import pyinotify
self._wm = pyinotify.WatchManager()
self._notifier = pyinotify.ThreadedNotifier(
self._wm, self._on_file_modified)
self._notifier.start()
# We monitor the directory (instead of just the file) because
# otherwise inotify gets confused by editors such a Vim.
flags = pyinotify.EventsCodes.OP_FLAGS['IN_MODIFY']
wdd = self._wm.add_watch(self._path, flags)
def _on_file_modified(self, event):
if event.name == self._filename:
print "File modification detected. Restarting application..."
self._reload_request = True
getattr(self._module, self._stop_function)()
def run(self):
self._module = import_module(self._module_name)
self._reload_request = True
while self._reload_request:
self._reload_request = False
reload(self._module)
getattr(self._module, self._start_function)()
print 'Bye!'
self._notifier.stop()
def launch_app(module_name, start_func, stop_func):
try:
import pyinotify
except ImportError:
print 'Pyinotify not found. Launching app anyway...'
m = import_module(self._module_name)
getattr(m, start_func)()
else:
RestartingLauncher(module_name, start_func, stop_func).run()
if __name__ == '__main__':
launch_app('example', 'main', 'force_exit')
The parameters in the launch_app call are the filename (without the ".py"), the function to start execution and a function that somehow stops the execution.
Here's a stupid example of an "app" that could be (re-)launched using the previous code:
run = True
def main():
print 'in...'
while run: pass
print 'out'
def force_exit():
global run
run = False
In a typical application where you'd want to use this, you'd probably have a main loop of some sort. Here's a more real example, for a GLib/GTK+ based application:
from gi.repository import GLib
GLib.threads_init()
loop = GLib.MainLoop()
def main():
print "running..."
loop.run()
def force_exit():
print "stopping..."
loop.quit()
The same concept works for most other loops (Clutter, Qt, etc).
Monitoring several code files (ie. all files that are part of the application) and error resilience (eg. printing exceptions and waiting in an idle loop until the code is fixed, then launching it again) are left as exercises for the reader :).
Note: All code in this answer is released under the ISC License (in addition to Creative Commons).

This is operating system specific. For Linux, there is inotify, see e.g. http://github.com/rvoicilas/inotify-tools/

python update class instance to reflect change in a class method

As I work and update a class, I want a class instance that is already created to be updated. How do I go about doing that?
class MyClass:
""" """
def __init__(self):
def myMethod(self, case):
print 'hello'
classInstance = MyClass()
I run Python inside of Maya and on software start the instance is created. When I call classInstance.myMethod() it always prints 'hello' even if I change this.
Thank you,
/Christian
More complete example:
class MayaCore:
'''
Super class and foundational Maya utility library
'''
def __init__(self):
""" MayaCore.__init__(): set initial parameters """
#maya info
self.mayaVer = self.getMayaVersion()
def convertToPyNode(self, node):
"""
SYNOPSIS: checks and converts to PyNode
INPUTS: (string?/PyNode?) node: node name
RETURNS: (PyNode) node
"""
if not re.search('pymel', str(node.__class__)):
if not node.__class__ == str and re.search('Meta', str(node)): return node # pass Meta objects too
return PyNode(node)
else: return node
def verifyMeshSelection(self, all=0):
"""
SYNOPSIS: Verifies the selection to be mesh transform
INPUTS: all = 0 - acts only on the first selected item
all = 1 - acts on all selected items
RETURNS: 0 if not mesh transform or nothing is selected
1 if all/first selected is mesh transform
"""
self.all = all
allSelected = []
error = 0
iSel = ls(sl=1)
if iSel != '':
if self.all: allSelected = ls(sl=1)
else:
allSelected.append(ls(sl=1)[0])
if allSelected:
for each in allSelected:
if nodeType(each) == 'transform' and nodeType(each.getShape()) == 'mesh':
pass
else: error = 1
else: error = 1
else: error = 1
if error: return 0
else: return 1
mCore = MayaCore()
The last line is inside the module file (mCore = MayaCore()).
There are tons of methods inside the class so I have removed them to shorten the scrolling :-)
Also there are import statements above the class but they screw up the formatting for some reason. Here they are:
from pymel.all import *
import re
from maya import OpenMaya as om
from our_libs.configobj import ConfigObj
if getMelGlobal('float', "mVersion") >= 2011:
from PyQt4 import QtGui, QtCore, uic
import sip
from maya import OpenMayaUI as omui
Inside Maya, we import this and subclasses of this class upon program start:
from our_maya.mayaCore import *
In other tools we write, we then call mCore.method() on a need basis.
The caveat I am running into is that when I am going back to modify the mCore method and the instance call is already in play, I have to restart Maya for all the instances to get updated with the method change (they will still use the un-modified method).

Alright, trying again, but with a new understanding of the question:
class Foo(object):
def method(self):
print "Before"
f = Foo()
f.method()
def new_method(self):
print "After"
Foo.method = new_method
f.method()
will print
Before
After
This will work with old style classes too. The key is modifying the class, not overriding the class's name.

You'll have to provide more details about what you are doing, but Python instances don't store methods, they always get them from their class. So if you change the method on the class, existing instances will see the new method.

My other answer answers your original question, so I'm leaving it there, but I think what you really want is the reload function.
import our_maya.mayaCore
reload(our_maya.mayaCore)
from our_maya.mayaCore import *
Do that after you change the class definition. Your new method ought to show up and be used by all the existing instances of your class.