I was looking around the aiortc examples when I notice a decorator that has a method on it:
#pc.on("datachannel")
def on_datachannel(channel):
...
I don't really understand how this work or what does this code do. I've been searching about decorators and I know it's possible to have class decorators but none about using methods. Can anyone elaborate on this?
#foo
def bar(): ...
This syntax is merely sugar for this:
def bar(): ...
bar = foo(bar)
So, this:
#pc.on('datachannel')
def on_datachannel(channel): ...
is the same as:
def on_datachannel(channel): ...
on_datachannel = pc.on('datachannel')(on_datachannel)
pc is some object, pc.on is a method on it, pc.on('datachannel') calls it and it returns a function, pc.on('datachannel')(on_datachannel) calls that returned function passing it the on_datachannel function.
The implementation of pc.on is something like this:
class PC:
def on(self, event):
...
def wrapper(fn):
...
def inner_wrapper(*args, **kwargs):
...
fn(*args, **kwargs)
return inner_wrapper
return wrapper
pc = PC()
All that inner part is entirely a regular decorator accepting arguments. That it's defined on a class makes no difference to it.
Related
I'm going to pass a function dynamically to another class as shown below
class simulator(object):
def __init__(self, fn_):
print(self.test(fn_))
def test(self, fn):
return fn(self, 20)
class t(object):
s = 'def get_fitness(x, y):\n return x+y'
exec(s)
def fnGetFitness(self,genes):
return get_fitness(genes, 10)
simulator(fnGetFitness)
t()
but i face error below:
File "N:/Job/GA/mine/dyn.py", line 25, in fnGetFitness
return get_fitness(genes, 10)
NameError: name 'get_fitness' is not defined
i guess its something related to scopes but i can't handle it
anyone on this?
EDIT :
this is a simpler code, showing the problem :
class t(object):
def __init__(self):
exec('def get_fitness(x, y):\n return x+y')
print(get_fitness(2,3))
t()
nothing to do with exec. What you're doing is equivalent (with safety removed) to:
class t(object):
def get_fitness(x,y):
return x+y
but your method definition is at class level, but not on the simulator class.
simulator(fnGetFitness) calls fnGetFitness out of t class context, so it doesn't know your new function.
That cannot work (also get_fitness should be decorated as #staticmethod because it doesn't have a self parameter)
What works is to define dynamically (or not) the function at global level so class can call it
s = 'def get_fitness(x, y):\n return x+y'
exec(s)
class t(object):
def fnGetFitness(self,genes):
return get_fitness(genes, 10)
simulator(fnGetFitness)
t()
that fixed it, but honestly I'm puzzled about the purpose (already took me a while to figure out how to make something run from your code)
EDIT: a simpler and somehow different (and exec related) code has been posted in comments:
class t(object):
def __init__(self):
exec('def get_fitness(x, y):\n return x+y')
print(get_fitness(2,3))
t()
this raises NameError: name 'get_fitness' is not defined
now this has to do with exec. When __init__ is parsed, get_fitness isn't known because the parser didn't see it as a local variable, even if at the time of execution, it is set in locals() dictionary by exec (related: why is 'ord' seen as an unassigned variable here?).
A workaround is to fetch the function in the local variables like this:
class t(object):
def __init__(self):
exec('def get_fitness(x, y):\n return x+y')
print(locals()["get_fitness"](2,3))
t()
this works & prints 5.
I'm trying to replace the marshal_with decorator from flask-restful with a decorator that does something before calling marshal_with. My approach is to try to implement a new decorator that wraps marshal_with.
My code looks like:
from flask.ext.restful import marshal_with as restful_marshal_with
def marshal_with(fields, envelope=None):
def wrapper(f):
print("Do something with fields and envelope")
#wraps(f)
def inner(*args, **kwargs):
restful_marshal_with(f(*args, **kwargs))
return inner
return wrapper
Unfortunately this seems to break things... no error messages but my API returns a null response when it shouldn't be. Any insights on what I'm doing wrong?
I don't know the specifics of marshal_with, but it's entirely possible to use multiple decorators on a single function. For instance:
def decorator_one(func):
def inner(*args, **kwargs):
print("I'm decorator one")
func(*args, **kwargs)
return inner
def decorator_two(text):
def wrapper(func):
def inner(*args, **kwargs):
print(text)
func(*args, **kwargs)
return inner
return wrapper
#decorator_one
#decorator_two("I'm decorator two")
def some_function(a, b):
print(a, b, a+b)
some_function(4, 7)
The output this gives is:
I'm decorator one
I'm decorator two
4 7 11
You can modify this little script by adding print statements after each inner function call to see the exact flow control between each decorator as well.
I was doing a couple things wrong here, first, failing to return the output of restful_marshal_with as jonrsharpe pointed out, secondly, failing to understand a decorator written as a class instead of a function, and how to properly pass values to it. The correct code ended up being:
def marshal_with(fields, envelope=None):
def wrapper(f):
print("Do something with fields and envelope")
#wraps(f)
def inner(*args, **kwargs):
rmw = restful_marshal_with(fields, envelope)
return rmw(f)(*args, **kwargs)
return inner
return wrapper
As you can see, in addition to not returning rmw(), I needed to properly initialize the request_marshal_with class before calling it. Finally, it is important to remember that decorators return functions, therefore the arguments of the original function should be passed to the return value of rmw(f), hence the statement return rmw(f)(*args, **kwargs). This is perhaps more apparent if you take a look at the flask_restful.marshal_with code here.
I have a python module with lots of functions and I want to apply a decorator for all of them.
Is there a way to patch all of them via monkey-patching to apply this decorator for every function without copy-pasting on line of applying decorator?
In other words, I want to replace this:
#logging_decorator(args)
func_1():
pass
#logging_decorator(args)
func_2():
pass
#logging_decorator(args)
func_3():
pass
#logging_decorator(args)
func_n():
pass
With this:
patch_func():
# get all functions of this module
# apply #logging_decorator to all (or not all) of them
func_1():
pass
func_2():
pass
func_3():
pass
func_n():
pass
I'm really not certain that this is a good idea. Explicit is better than implicit, after all.
With that said, something like this should work, using inspect to find which members of a module can be decorated and using __dict__ to manipulate the module's contents.
import inspect
def decorate_module(module, decorator):
for name, member in inspect.getmembers(module):
if inspect.getmodule(member) == module and callable(member):
if member == decorate_module or member == decorator:
continue
module.__dict__[name] = decorator(member)
Sample usage:
def simple_logger(f):
def wrapper(*args, **kwargs):
print("calling " + f.__name__)
f(*args, **kwargs)
return wrapper
def do_something():
pass
decorate_module(sys.modules[__name__], simple_logger)
do_something()
I ain't gonna be pretty ... but you can list all functions using dir() after their definition. Then I can't think of a way to patch them without a wrapper function.
def patched(func):
#logging_decorator
def newfunc(*args, **kwargs):
return func(*args, **kwargs)
return newfunc
funcs=[f in dir() if not '__' in f]
for f in funcs:
exec(f+'=patched(f)')
I am using nose test generators feature to run the same test with different contexts. Since it requires the following boiler plate for each test:
class TestSample(TestBase):
def test_sample(self):
for context in contexts:
yield self.check_sample, context
def check_sample(self, context):
"""The real test logic is implemented here"""
pass
I decided to write the following decorator:
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
The decorator is used in the following manner:
class TestSample(TestBase):
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
When the tests executed an error is thrown specifying that the attribute which is being accessed is not available. However If I change the line which calls the method to the following it works fine:
yield getattr(self, f.__name__), service
I understand that the above snippet creates a bound method where as in the first one self is passed manually to the function. However as far as my understanding goes the first snippet should work fine too. I would appreciate if anyone could clarify the issue.
The title of the question is related to calling instance methods in decorators in general but I have kept the description specific to my context.
You can use functools.partial to tie the wrapped function to self, just like a method would be:
from functools import partial
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield partial(f, self), context
return wrapper
Now you are yielding partials instead, which, when called as yieldedvalue(context), will call f(self, context).
As far as I can tell, some things don't fit together. First, your decorator goes like
def with_contexts(contexts=None):
if contexts is None:
contexts = ['twitter', 'linkedin', 'facebook']
def decorator(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
for context in contexts:
yield f, self, context # The line which causes the error
return wrapper
return decorator
but you use it like
#with_contexts
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
This is wrong: this calls with_context(test_sample), but you need with_context()(test_sample). So do
#with_contexts()
def test_sample(self, context):
"""The real test logic is implemented here"""
var1 = self.some_valid_attribute
even if you don't provide the contexts argument.
Second, you decorate the wrong function: your usage shows that the test function yields the check function for each context. The function you want to wrap does the job of the check function, but you have to name it after the test function.
Applying self to a method can be done with partial as Martijn writes, but it can as well be done the way Python does it under the hood: with
method.__get__(self, None)
or maybe better
method.__get__(self, type(self))
you can achieve the same. (Maybe your original version works as well, with yielding the function to be called and the arguments to use. It was not clear to me that this is the way it works.)
This is what i want to do:
#MyDecorator
def f():
pass
for d in f.decorators:
print d
This is not generally possible without the cooperation of the decorators. For example,
def my_decorator(f):
def wrapper(*args, **kwargs):
return f(*args, **kwargs)
wrapper.decorators = [wrapper]
if hasattr(f, 'decorators'):
wrapper.decorators.extend[f.decorators]
return wrapper
Essentially, all the decorator does is wrap the function as usual and then put a decorators attribute on the wrapper function. It then checks the wrapped function for a similar list and propagates it upwards.
This is pretty useless though
What I think you want is
def my_decorator(f):
def wrapper(args):
return f(args)
wrapper.decorated = f
return wrapper
This will allow you to do stuff like
#my_other_decorator # another decorator following the same pattern
#my_decorator
def foo(args):
print args
foo.decorated(args) # calls the function with the inner decorated function (my_decorator)
foo.decorated.decorated(args) # original function
You can actually abstract this pattern into a decorator
def reversable(decorator):
def wrapper(func):
ret = decorator(func) # manually apply the decorator
ret.decorated = func # save the original function
return ret
return wrapper
Now when you are writing your decorator:
#reversable
def my_decorator(f):
def wrapper(x):
return f(x + 1)
return wrapper
The #MyDecorator syntax is just shorthand for writing the following Python code:
def f():
pass
f = MyDecorator(f)
Written in this form, you can see that the decorators applied to the function are not kept track of in any way. You could make your decorators remember when they're applied (Aaron's answer has a couple good ideas on how to do this), but you'd have to wrap all third-party decorators with your own.