I'm writing a modding API for my game and I want to allow users to tell the game engine to run their mod's function immediately before or after one of the API functions, in other words to "extend" the functions.
Before now modders had to rewrite the function to add functionality to it which meant digging in the game's sourcecode, which I think sucks for them.
Right now I'm thinking of something like this:
def moddersFunction():
pass
myAPI.extendFunction('functionName', 'before-or-after', moddersFunction, extraArgs=[])
Now what will be comparably clean way to implement this?
In other words, what would the internals of myAPI.extendFunction look like if you were to write it?
I'm thinking of having a dictionary which myAPI.extendFunction will add functions to and the functions in my API will check and run the mod's function if it has been set ("registered").
But this means adding the code which checks the dictionary in every single function in my modding API which I want to allow to be extended (even if it's just a single function call which does the check and function calling itself, it seems to much itself).
I'm wondering if there's some neat Python trick which will allow such "extending" of functions without "butchering" (maybe I'm exaggerating) the existing sourcecode of my game's API.
I would do something along the following lines:
class hookable(object):
def __init__(self, fn):
self.pre = []
self.post = []
self.fn = fn
def add_pre(self, hook):
self.pre.append(hook)
def add_post(self, hook):
self.post.append(hook)
def __call__(self, *args, **kwargs):
for hook in self.pre:
hook(*args, **kwargs)
ret = self.fn(*args, **kwargs)
for hook in self.post:
hook(*args, **kwargs)
return ret
Any "extendable" function can now be decorated like so:
#hookable
def square(x):
return x**2
Now you can use square.add_pre() and square.add_post() to register functions that would automatically get called before and after each call to square():
print square(2)
def pre(x): print 'pre', x
def post(x): print 'post', x
square.add_pre(pre)
print square(2)
print square(3)
square.add_post(post)
print square(4)
print square(5)
I would probably do something like this. Obviously what you actually do is specific to your requirements.
class Extend(object):
def __init__(self, original_function):
self.original_function = original_function
def __call__(self, function):
def _wrapped(*args, **kwargs):
yield function(*args, **kwargs)
yield self.original_function(*args, **kwargs)
return _wrapped
import time
#Extend(time.asctime)
def funky_time():
return "This is the time in the UK"
print list(funky_time())
Related
Given the code below, I am trying to identify the arguments sent to decorators that are used by a class.
Specifically, I am trying to identify the value 100 that is passed into the make_hyper decorator that is used in the Dog class.
Optimally, I need to be able to get this value without running the method directly since the actual method code I am working with takes a long time to run.
import inspect
def make_hyper(new_volume):
def decorator(decorated_method):
def wrapped_method(self, *args, **kwargs):
self.volume = new_volume
return decorated_method(self, *args, **kwargs)
return wrapped_method
return decorator
class Dog(object):
def __init__(self, name):
self.name = name
self.volume = 1
#make_hyper(new_volume=100)
def bark(self):
if self.volume >= 10:
print('[{}]: BARK!!'.format(self.name))
elif self.volume >= 5:
print('[{}]: Bark!'.format(self.name))
else:
print('[{}]: Bark'.format(self.name))
I have tried to use inspect.getargspec and a handful of other things and have (done my best to) scour Stackoverflow and the Internet at large but I cannot find a solution.
Any help would be greatly appreciated! Thanks for your time!
UPDATE:
Someone asked me to clarify what I was trying to do. Apologies for not being more clear.
The actual code that the above toy represents is a test automation framework that includes decorators specifying test tags:
class TestTags(object):
WIFI = 'wifi'
BLE = 'ble'
NIGHTLY = 'nightly'
REGRESSION = 'regression'
class TestBase(unittest.TestCase):
<define common test stuff and test decorators>
class ThingTester(TestBase):
#TestBase.tags(TestTags.WIFI, TestTags.BLE, TestTags.REGRESSION)
def test_all_the_things(self):
<test all the things>
# what I'm trying to get
test_tags = ???
print(test_tags) # prints out ('wifi', 'ble', 'regression')
I wrote a utility that goes through all the test modules, classes and individual tests and creates an HTML+JavaScript page that defines a Test Plan. One piece of data that I want to add to the page is what tags are associated with each test.
I found that I can make the tags decorator save an attribute to the class (i.e. self.tags = tags) but this requires the test to be run before the value is saved to the TestBase object and I need to be able to generate the test plan independently of running the tests.
I'm not sure if I'm understanding entirely what you want, but if you intend to inspect the existing test to create a plan or a report, then you probably want to store the decorator arguments in the test function, instead of the class instance, right? For example:
def TestTags(*tags):
def decorator(f):
def wrapper(*args, **kwargs):
# ...
return f(*args, **kwargs)
wrapper.tags = tags
return wrapper
return decorator
#TestTags(TAG1, TAG2)
def myTest(...):
...
print(myTest.tags)
>>> (TAG1, TAG2)
EDIT: Just realized that doesn't work out of the box for instance methods. You can still use it like this though:
def TestTags(*tags):
def decorator(f):
def wrapper(*args, **kwargs):
# ...
return f(*args, **kwargs)
wrapper.tags = tags
return wrapper
return decorator
class TestSuite(...):
#TestTags(TAG1, TAG2)
def myTest(...):
...
print(TestSuite.myTest.tags) # From class function
>>> (TAG1, TAG2)
myTestSuite = TestSuite()
print(myTestSuite.myTest.__func__.tags) # From instance method
>>> (TAG1, TAG2)
I'm trying to replace the marshal_with decorator from flask-restful with a decorator that does something before calling marshal_with. My approach is to try to implement a new decorator that wraps marshal_with.
My code looks like:
from flask.ext.restful import marshal_with as restful_marshal_with
def marshal_with(fields, envelope=None):
def wrapper(f):
print("Do something with fields and envelope")
#wraps(f)
def inner(*args, **kwargs):
restful_marshal_with(f(*args, **kwargs))
return inner
return wrapper
Unfortunately this seems to break things... no error messages but my API returns a null response when it shouldn't be. Any insights on what I'm doing wrong?
I don't know the specifics of marshal_with, but it's entirely possible to use multiple decorators on a single function. For instance:
def decorator_one(func):
def inner(*args, **kwargs):
print("I'm decorator one")
func(*args, **kwargs)
return inner
def decorator_two(text):
def wrapper(func):
def inner(*args, **kwargs):
print(text)
func(*args, **kwargs)
return inner
return wrapper
#decorator_one
#decorator_two("I'm decorator two")
def some_function(a, b):
print(a, b, a+b)
some_function(4, 7)
The output this gives is:
I'm decorator one
I'm decorator two
4 7 11
You can modify this little script by adding print statements after each inner function call to see the exact flow control between each decorator as well.
I was doing a couple things wrong here, first, failing to return the output of restful_marshal_with as jonrsharpe pointed out, secondly, failing to understand a decorator written as a class instead of a function, and how to properly pass values to it. The correct code ended up being:
def marshal_with(fields, envelope=None):
def wrapper(f):
print("Do something with fields and envelope")
#wraps(f)
def inner(*args, **kwargs):
rmw = restful_marshal_with(fields, envelope)
return rmw(f)(*args, **kwargs)
return inner
return wrapper
As you can see, in addition to not returning rmw(), I needed to properly initialize the request_marshal_with class before calling it. Finally, it is important to remember that decorators return functions, therefore the arguments of the original function should be passed to the return value of rmw(f), hence the statement return rmw(f)(*args, **kwargs). This is perhaps more apparent if you take a look at the flask_restful.marshal_with code here.
I am using nose test generators feature to run the same test with different contexts. Since it requires the following boiler plate for each test:
class TestSample(TestBase):
def test_sample(self):
for context in contexts:
yield self.check_sample, context
def check_sample(self, context):
"""The real test logic is implemented here"""
pass
I decided to write the following decorator:
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
The decorator is used in the following manner:
class TestSample(TestBase):
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
When the tests executed an error is thrown specifying that the attribute which is being accessed is not available. However If I change the line which calls the method to the following it works fine:
yield getattr(self, f.__name__), service
I understand that the above snippet creates a bound method where as in the first one self is passed manually to the function. However as far as my understanding goes the first snippet should work fine too. I would appreciate if anyone could clarify the issue.
The title of the question is related to calling instance methods in decorators in general but I have kept the description specific to my context.
You can use functools.partial to tie the wrapped function to self, just like a method would be:
from functools import partial
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield partial(f, self), context
return wrapper
Now you are yielding partials instead, which, when called as yieldedvalue(context), will call f(self, context).
As far as I can tell, some things don't fit together. First, your decorator goes like
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
but you use it like
#with_contexts
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
This is wrong: this calls with_context(test_sample), but you need with_context()(test_sample). So do
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
even if you don't provide the contexts argument.
Second, you decorate the wrong function: your usage shows that the test function yields the check function for each context. The function you want to wrap does the job of the check function, but you have to name it after the test function.
Applying self to a method can be done with partial as Martijn writes, but it can as well be done the way Python does it under the hood: with
method.__get__(self, None)
or maybe better
method.__get__(self, type(self))
you can achieve the same. (Maybe your original version works as well, with yielding the function to be called and the arguments to use. It was not clear to me that this is the way it works.)
I currently have the following code which uses a python library:
f = Foo(original_method, parameters)
I would like to augment original_method, and have a decorator add a few lines of code. Let's call the new decorated method decorated_method. Finally I would like to have something like this:
f = Foo(decorated_method(original_method), parameters)
My questions are: is this possible? how would the decorator look like?
I must say that I can't extend original_method, since it is part of an external library.
Edit: original_method is not executed, is only passed to Foo as a parameter. decorated_method function should do some logging and gather some statistics of the number of calls.
Later edit: the code in examples below works fine. I had a few additional problems because original_method had a few attributes, so this is the final code:
def decorated_method(method):
def _reporter(*args, **kwargs):
addmetric('apicall', method.__name__)
return method(*args, **kwargs)
_reporter.original_method_attribute = method.original_method_attribute
return _reporter
You don't mention what you want decorated_method to do, but this is certainly possible:
def decorated_method(f):
def _wrapped(*args, **kwargs):
print "About to call f!"
ret = f(*args, **kwargs)
print "Just got finished with f, ret = %r" % (ret,)
return ret
return _wrapped
This is just standard decorator structure: A decorator is a function which accepts a function and returns a function.
Absolutely:
def decorated_method(fn):
def inner_method(*args, **kwargs):
print("before calling")
result = fn(*args, **kwargs)
print("after calling")
return result
return inner_method
Once you've got this working, you should look at signature-preserving decorators.
This is what i want to do:
#MyDecorator
def f():
pass
for d in f.decorators:
print d
This is not generally possible without the cooperation of the decorators. For example,
def my_decorator(f):
def wrapper(*args, **kwargs):
return f(*args, **kwargs)
wrapper.decorators = [wrapper]
if hasattr(f, 'decorators'):
wrapper.decorators.extend[f.decorators]
return wrapper
Essentially, all the decorator does is wrap the function as usual and then put a decorators attribute on the wrapper function. It then checks the wrapped function for a similar list and propagates it upwards.
This is pretty useless though
What I think you want is
def my_decorator(f):
def wrapper(args):
return f(args)
wrapper.decorated = f
return wrapper
This will allow you to do stuff like
#my_other_decorator # another decorator following the same pattern
#my_decorator
def foo(args):
print args
foo.decorated(args) # calls the function with the inner decorated function (my_decorator)
foo.decorated.decorated(args) # original function
You can actually abstract this pattern into a decorator
def reversable(decorator):
def wrapper(func):
ret = decorator(func) # manually apply the decorator
ret.decorated = func # save the original function
return ret
return wrapper
Now when you are writing your decorator:
#reversable
def my_decorator(f):
def wrapper(x):
return f(x + 1)
return wrapper
The #MyDecorator syntax is just shorthand for writing the following Python code:
def f():
pass
f = MyDecorator(f)
Written in this form, you can see that the decorators applied to the function are not kept track of in any way. You could make your decorators remember when they're applied (Aaron's answer has a couple good ideas on how to do this), but you'd have to wrap all third-party decorators with your own.