We Keep Coding

Class Decorator when Inheriting from another class

Ive been on a tear of writing some decorators recently.
One of the ones I just wrote allows you to put the decorator just before a class definition, and it will cause every method of the class to print some logigng info when its run (more for debugging/initial super basic speed tests during a build)
def class_logit(cls):
class NCls(object):
def __init__(self, *args, **kwargs):
self.instance = cls(*args, **kwargs)
#staticmethod
def _class_logit(original_function):
def arg_catch(*args, **kwargs):
start = time.time()
result = original_function(*args, **kwargs)
print('Called: {0} | From: {1} | Args: {2} | Kwargs: {3} | Run Time: {4}'
''.format(original_function.__name__, str(inspect.getmodule(original_function)),
args, kwargs, time.time() - start))
return result
return arg_catch
def __getattribute__(self, s):
try:
x = super(NCls, self).__getattribute__(s)
except AttributeError:
pass
else:
return x
x = self.instance.__getattribute__(s)
if type(x) == type(self.__init__):
return self._class_logit(x)
else:
return x
return NCls
This works great when applied to a very basic class i create.
Where I start to encounter issues is when I apply it to a class that is inheriting another - for instance, using QT:
#scld.class_logit
class TestWindow(QtGui.QDialog):
def __init__(self):
print self
super(TestWindow, self).__init__()
a = TestWindow()
Im getting the following error... and im not entirely sure what to do about it!
self.instance = cls(*args, **kwargs)
File "<string>", line 15, in __init__
TypeError: super(type, obj): obj must be an instance or subtype of type
Any help would be appreciated!
(Apologies in advance, no matter WHAT i do SO is breaking the formatting on my first bit of code... Im even manually spending 10 minutes adding spaces but its coming out incorrectly... sorry!)

You are being a bit too intrusive with your decorator.
While if you want to profile methods defined on the Qt framework itself, a somewhat aggressive approach is needed, your decorator replaces the entire class by a proxy.
Qt bindings are somewhat complicated indeed, and it is hard to tell why it is erroring when being instantiated in this case.
So - first things first - if your intent would be to apply the decorator to a class hierarchy defined by yourself, or at least one defined in pure Python, a good approach there could be using metaclasses: with a metaclass you could decorate each method when a class is created, and do not mess anymore at runtime, when methods are retrieved from each class.
but Qt, as some other libraries, have its methods and classes defined in native code, and that will prevent you from wrapping existing methods in a new class. So, wrapping the methods on attribute retrieval on __getattribute__ could work.
Here is a simpler approach that instead of using a Proxy, just plug-in a foreign __getattribute__ that does the wrap-with-logger thing you want.
Your mileage may vary with it. Specially, it won't be triggered if one method of the class is called by other method in native code - as this won't go through Python's attribute retrieval mechanism (instead, it will use C++ method retrieval directly).
from PyQt5 import QtWidgets, QtGui
def log_dec(func):
def wraper(*args, **kwargs):
print(func.__name__, args, kwargs)
return func(*args, **kwargs)
return wraper
def decorate(cls):
def __getattribute__(self, attr):
attr = super(cls, self).__getattribute__(attr)
if callable(attr):
return log_dec(attr)
return attr
cls.__getattribute__ = __getattribute__
return cls
#decorate
class Example(QtGui.QWindow):
pass
app = QtWidgets.QApplication([])
w = Example()
w.show()
(Of course, just replace the basic logger by your fancy logger above)

Python - instance of a class from another file accessing a var from the main script

am making a text-based RPG game where i split the game into a lot of files containing scripts for creatures classes and weapons, example: classes/mobs/goblin.py, and i have a global list for entities so when a mob attack something it finds that thing's location from that global list, but the instance class from classes/mobs/default.py can't access the list on the main.py file.
example codes:
main.py:
entity_list = ['test']
x = classes.mobs.default.goblin()
x.attack()
goblin.py:
class goblin():
def __init__(self):
pass
def attack(self):
print(entity_list)
is there a workaround? thanks in advance.

This is a design flaw, and while you can push imports down into functions, it's better to solve the underlying problem.
I think the best option here is to create a Context class to hold the global state, context.py:
_CONTEXT = None
def context():
global _CONTEXT
if _CONTEXT is None:
_CONTEXT = Context()
return _CONTEXT
class Context(object):
def __init__(self):
self.entity_list = ['test']
you can then use it freely in goblin.py:
from path.to.context import context
class Goblin(object):
def attack(self):
print(context().entity_list)
and main.py:
x = classes.mobs.default.Goblin()
x.attack()
if you at some point want to implement save/replay/network play, you can extend the context:
class Context(object):
def __init__(self):
self.__events = []
self.entity_list = ['test']
def _addevent(self, name, *options):
self.__events.append(name, options)
# and/or send the event to other processes/users/etc.
def create(self, cls, *args, **kwargs):
self._addevent('create', cls, args, kwargs)
return cls(*args, **kwargs)
def call(self, obj, method, *args, **kwargs):
self._addevent('call', obj, method, args, kwargs)
return getattr(obj, method)(*args, **kwargs)
your main.py would then look like:
from path.to.context import context
x = context().create(classes.mobs.default.goblin)
context().call(x, 'attack')
the awkwardness in syntax can be hidden with meta classes and decorators if need be.

def attack(self):
from .main import entity_list
print(entity_list)
Importing inside methods or functions can be used to cause delayed imports.
This way, the fact that main.py may have to import goblin.py won't throw you in a circular import situation, since the importing of main.py in the goblin file is delayed until its needed.
And it cause no performance impact as well: the main and goblin modules are already in memory and loaded, the import statement just makes an assignment to an object that is already there.

Python: How do you intercept a method call to change function parameters?

What I am trying to do is write a wrapper around another module so that I can transform the parameters that are being passed to the methods of the other module. That was fairly confusing, so here is an example:
import somemodule
class Wrapper:
def __init__(self):
self.transforms = {}
self.transforms["t"] = "test"
# This next function is the one I want to exist
# Please understand the lines below will not compile and are not real code
def __intercept__(self, item, *args, **kwargs):
if "t" in args:
args[args.index("t")] = self.transforms["t"]
return somemodule.item(*args, **kwargs)
The goal is to allow users of the wrapper class to make simplified calls to the underlying module without having to rewrite all of the functions in the module. So in this case if somemodule had a function called print_uppercase then the user could do
w = Wrapper()
w.print_uppercase("t")
and get the output
TEST
I believe the answer lies in __getattr__ but I'm not totally sure how to use it for this application.

__getattr__ combined with defining a function on the fly should work:
# somemodule
def print_uppercase(x):
print(x.upper())
Now:
from functools import wraps
import somemodule
class Wrapper:
def __init__(self):
self.transforms = {}
self.transforms["t"] = "test"
def __getattr__(self, attr):
func = getattr(somemodule, attr)
#wraps(func)
def _wrapped(*args, **kwargs):
if "t" in args:
args = list(args)
args[args.index("t")] = self.transforms["t"]
return func(*args, **kwargs)
return _wrapped
w = Wrapper()
w.print_uppercase('Hello')
w.print_uppercase('t')
Output:
HELLO
TEST

I would approach this by calling the intercept method, and entering the desired method to execute, as a parameter for intercept. Then, in the intercept method, you can search for a method with that name and execute it.

Since your Wrapper object doesn't have any mutable state, it'd be easier to implement without a class. Example wrapper.py:
def func1(*args, **kwargs):
# do your transformations
return somemodule.func1(*args, **kwargs)
Then call it like:
import wrapper as w
print w.func1('somearg')

How to unit test a decorated method

Assume I have to unit test methodA, defined in the following class:
class SomeClass(object):
def wrapper(fun):
def _fun(self, *args, **kwargs):
self.b = 'Original'
fun(self, *args, **kwargs)
return _fun
#wrapper
def methodA(self):
pass
My test class is as follows:
from mock import patch
class TestSomeClass(object):
def testMethodA(self):
def mockDecorator(f):
def _f(self, *args, **kwargs):
self.b = 'Mocked'
f(self, *args, **kwargs)
return _f
with patch('some_class.SomeClass.wrapper', mockDecorator):
from some_class import SomeClass
s = SomeClass()
s.methodA()
assert s.b == 'Mocked', 's.b is equal to %s' % s.b
If I run the test, I hit the assertion:
File "/home/klinden/workinprogress/mockdecorators/test_some_class.py", line 17, in testMethodA
assert s.b == 'Mocked', 's.b is equal to %s' % s.b
AssertionError: s.b is equal to Original
If I stick a breakpoint in the test, after patching, this is I can see wrapper has been mocked out just fine, but that methodA still references the old wrapper:
(Pdb) p s.wrapper
<bound method SomeClass.mockDecorator of <some_class.SomeClass object at 0x7f9ed1bf60d0>>
(Pdb) p s.methodA
<bound method SomeClass._fun of <some_class.SomeClass object at 0x7f9ed1bf60d0>>
Any idea of what the problem is here?

After mulling over, I've found a solution.
Since monkey patching seems not to be effective (and I've also tried a few
other solutions), I dug into the function internals and that proved to be fruitful.
Python 3
You're lucky - just use the wraps decorator, which creates a __wrapped__ attribute, which in turn contains the wrapped function. See the linked answers above for more details.
Python 2
Even if you use #wraps, no fancy attribute is created.
However, you just need to realise that the wrapper method does nothing but a closure: so you'll be able to find your wrapped function in its func_closure attribute.
In the original example, the wrapped function would be at: s.methodA.im_func.func_closure[0].cell_contents
Wrapping up (ha!)
I created a getWrappedFunction helper along this lines, to ease my testing:
#staticmethod
def getWrappedFunction(wrapper):
return wrapper.im_func.func_closure[0].cell_contents
YMMV, especially if you do fancy stuff and include other objects in the closure.

Run-time-patch python module

I am searching for a way to run a module while replacing imports. This would be the missing magic to implement run_patched in the following pseudocode.
from argparse import ArgumentParser
class ArgumentCounter(ArgumentParser):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
arg_counter = 0
def add_argument(self, *args, **kwargs):
super().add_argument(*args, **kwargs)
arg_counter += 1
def parse_args(self, *args, **kwargs):
super().parse_args(*args, **kwargs)
print(arg_counter)
run_patched('test.test_argparse', ArgumentParser = ArgumentCounter)
I know that single methods could be replaced by assignment, for example stating ArgumentParser.parse_args = print, so I was tempted to mess with globals like sys.modules and then execute the module by runpy.run_module.
Unfortunately, the whole strategy should be able to work in a multithreaded scenario. So the change should only affect the module executed while other parts of the program can continue to use the unpatched module(s) as if they were never touched.