I am interested in creating a generic function and decorating all other functions with it. The new function will contain arguments from both functions. Example
def decorator(func):
# Some chunk of code
#decorator
def func(a, b):
print a**2, b**2
return a**2, b**2
# Equivalent func of what I want out of the decorator
def equiv_func(a, b, var):
print var # I want to modify all decorated functions to accept a new
# argument and print it using "print var"
print a**2, b**2
return a**2, b**2
This is what I kind of came up with but...
def decor(func):
def modify(var, **kwargs):
print var
x, y = func(**kwargs)
return x, y
return modify
#decor
def func(a, b):
print a**2, b**2
return a**2, b**2
>>> func(var="hi", a=4, b=5)
hi
16 25
I used **kwargs as a way to account for arguments of func, but this forces me to use func(var="hi", a=4, b=5) instead of func(a=4, b=5, var="hi") due to the positioning of **kwargs.
Also, this workaround prevents me from using **kwargs when defining my decorator when I need it since it is already used to input arguments into func. Example
def test(**kwargs):
if 'test_ans' in kwargs:
if kwargs['test_ans'] == 'y':
return True
else:
return False
def decor(func):
def modify(var, **kwargs):
if test(**kwargs): # I need to use **kwargs here, but I also need it to define my
# arguments in func.
print var
x, y = func(**kwargs)
return x, y
return modify
...
# The following is what I expect but due to **kwargs being used for two purposes,
# it doesn't work
>>> func(var='hi', a=4, b=5, test_ans='y')
'hi'
16 25
>>> func(var='hi', a=4, b=5, test_ans='n')
16 25
The examples are just to illustrate of the constraints I face. They are not the actual code (I wouldn't write a test function that uses **kwargs for example).
To put simply, is there any other way to modify func(a, b) to func(a, b, c, d, e, ..., **kwargs) without using kwargs in the decorator?
You can access the parameters that a function accepts with func.func_code.co_varnames. Once you know that, you can filter everything else out.
This is limited in that you would not be able to use this effectively if myfunc accepted **kwargs as a parameter.
def print_decorator(func):
def wrapped(*args, **kwargs):
# a tuple of the names of the parameters that func accepts
func_params = func.func_code.co_varnames
# grab all of the kwargs that are not accepted by func
extra = set(kwargs.keys()) - set(func_params)
for kw in extra:
print kwargs.pop(kw)
return func(*args, **kwargs)
return wrapped
#print_decorator
def myfunc(a, b, c=None):
print 'myfunc was passed: a={}, b={}, c={}'.format(a, b, c)
return a, b, c
>>> myfunc(1, b=2, c=3, d=4, e=5)
4
5
myfunc was passed: a=1, b=2, c=3
You could allow func to be called with
func(a=4, b=5, var="hi")
by making var a keyword argument.
def decor(func):
def modify(*args, **kwargs):
var = kwargs.pop('var', None)
print var
x,y=func(*args, **kwargs)
return x,y
return modify
#decor
def func(a,b):
print a**2,b**2
return a**2,b**2
func(a=4, b=5, var="hi")
func(a=4, b=5)
Related
import functools
def test_decorator(a, b, c):
def decorator(function):
#functools.wraps(function)
def wrapper(*args, **kwargs):
return function(*args, **kwargs)
return wrapper
return decorator
#test_decorator(a=1, b=2, c=3)
def test(d, e, f):
return None
The following method will give the arguments of the function being wrapped:
import inspect
fn = test
list(inspect.signature(fn)._parameters)
Will return d, e, f
How can I also access the parameters of the decorator/wrapper test_decorator i.e. a, b, c
I am trying to understand the logic behind two versions of my script which has decorators used in it. In one of the script, the decorator is invoked automatically without being even called.
Script 1:
def outer_decorator(input_function):
def decorator(*args, **kwargs):
print "Outer Decorator Called"
output = input_function(*args, **kwargs)
return output
return decorator
class X:
#outer_decorator
def inner_decorator(input_function):
def decorator(*args, **kwargs):
output = input_function(*args, **kwargs)
print "Inner Decorator Called"
return output
return decorator
def z(self, a, b):
"""
Function returns a - b
"""
return a - b
x = X()
print(x.z(1, 2))
Output
-1
So whenever I run Script 1, I am creating an object of class X and then calling function z. So, in this case, the inner decorator function is not being used anywhere and hence it is not invoked.
Script 2:
def outer_decorator(input_function):
def decorator(*args, **kwargs):
print "Outer Decorator Called"
output = input_function(*args, **kwargs)
return output
return decorator
class X:
#outer_decorator
def inner_decorator(input_function):
def decorator(*args, **kwargs):
output = input_function(*args, **kwargs)
print "Inner Decorator Called"
return output
return decorator
#inner_decorator
def s(self, a, b):
"""
Function return a + b
"""
return a + b
def z(self, a, b):
"""
Function returns a - b
"""
return a - b
x = X()
print(x.z(1, 2))
So now in the above Script I have introduces another function s which uses #inner_decorator. But I am still only invoking function z of class x. But when I run this script I get the following output.
Outer Decorator Called
-1
So why is #outer_decorator called and is printing? Is it because the decorator used somewhere is processed by the interpreter?
The reason the outer_decorator has been invoked is because of the fact that you are decorating a decorator itself.
def outer_decorator(input_function):
def decorator(*args, **kwargs):
print "Outer Decorator Called"
output = input_function(*args, **kwargs)
return output
return decorator
class X:
#outer_decorator
def inner_decorator(input_function):
def decorator(*args, **kwargs):
output = input_function(*args, **kwargs)
print "Inner Decorator Called"
return output
return decorator
This expression #outer_decorator actually translates to inner_decorator = outer_decorator(inner_decorator),so in your code though you have not explicitly called the function that you decorated with inner_decorator but the moment you decorated the function s with inner_decorator it actually called the decorated version of inner_decorated which in turn called the decorator function defined inside the outer_decorator.
I hope this makes the things clear.
Script 1:
inner_decorator = outer_decorator(inner_decorator) # i.e. a function
# object, no invocation
Script 2:
Here:
#outer_decorator
def inner_decorator(input_function):
this gets invoked:
inner_decorator = outer_decorator(inner_decorator) # i.e. a function
# object, no invocation
But,
here:
#inner_decorator
def s(self, a, b):
this leads to:
s = inner_decorator(s) = outer_decorator(inner_decorator)(s) # causes
# invocation of outer_decorator
How to do this in python?
def f(a, b):
return a, b+2, b+3
def g(a, b, c):
return a+b+c
How to get something like k = f+g hence that
k(a, b) is g(f(a,b))
Note that this is an abstract question. I wonder whether there's a function that can return a concat of f+g or even concat([...]) generally working regardless of the args of f.
In another word, I want a function whose args is f and g and returns k:
def concat(f,g):
return something_equivalent_to_k_above
Answer for original question
Define:
k = lambda x,y:g(*f(x,y))
Example:
>>> k(2,3)
13
Answer for revised question
Define:
concat = lambda a,b:lambda *args: b(*a(*args))
Example:
>>> k = concat(f,g)
>>> k(2,3)
13
There are at least 2 approaches to do this:
Approach A (recommended):
def compound(inner_func, outer_func):
def wrapper(*args, **kwargs):
return outer_func(*inner_func(*args, **kwargs))
return wrapper
Note that inner_func mustn't return a dictionary, in which case we should write return outer_func(**inner_func(*args, **argv)).
Usage:
def f(a, b):
return a, b+2, b+3
def g(a, b, c):
return a+b+c
k = compound(f, g)
print k(1, 2)
# output: 10
Approach B:
First, define a "decorator factory" like this:
def decorator_factory(inner_func):
def decorator(outer_func):
def wrapper(*args, **kwargs):
return outer_func(*inner_func(*args, **kwargs))
return wrapper
return decorator
Then you can use it like this:
def f(a, b):
return a, b+2, b+3
#decorator_factory(f)
def g(a, b, c):
return a+b+c
print g(1, 2)
# output: 10
Why not define a third function as you did with f and g:
>>> def f(a, b):
return a, b+2, b+3
>>> def g(a,b,c):
return a+b+c
>>> def k(a,b):
return g(*f(a,b))
>>> k(2,3)
13
If I want to pass an indentical list of args from one function to another, I can use the locals function. However this falls over if I'm in an instance method, as self cannot be passed as kwarg. I can create a dict of locals and then delete self. I was wondering if there's a more elegant way of doing this though - it seems to come up quite a lot for me.
class MyObj:
def test(self, a=1, b=2, c=3, d=4):
return [a,b,c,d]
class MyVerboseObj(MyObj):
def test(self, a=1, b=2, c=3, d=4):
print "RUNNING TEST a=%d"%a
kwargs = locals()
del kwargs["self"]
return MyObj.test(self, **kwargs)
MyVerboseObj().test()
Is there a better way to pass the identical list args from MyVerboseObj.test to MyObj.test?
** UPDATE **
This is a simplified example to demostrate my question. A more practical example might be this:
class GenericShow:
def __init__(self, path):
self.path = path
def getRenderPath(self, extn="jpg", colspace="rgb"):
return "%s/render_%s.%s"%(self.path, colspace, extn)
def hasGenericShowHandler(self):
try:
import GenericShowHandler
return True
except: pass
class UkShow(GenericShow):
def getRenderPath(self, extn="jpg", colspace="rgb"):
if self.hasGenericShowHandler():
kwargs = locals()
kwargs.pop("self")
return GenericShow.getRenderPath(self, **kwargs)
else:
return "%s/blah/render_%s.%s"%(path, colspace, extn)
show = UkShow("/jobs/test")
show.hasGenericShowHandler()
print show.getRenderPath()
I have a lot of show objects with different vars defined by different modules and different naming conventions - I'm trying to create show objects with common functionality.
If you don't want to override the parent's function (because you want to use it) - simply avoid overriding it by giving it a different name!
class MyObj:
def base_test(self, a=1, b=2, c=3, d=4):
return [a,b,c,d]
class MyVerboseObj(MyObj):
def test(self, a=1, b=2, c=3, d=4):
print "RUNNING TEST a=%d"%a
kwargs = locals()
del kwargs["self"]
return self.base_test(**kwargs)
print MyVerboseObj().test()
Using locals() is rarely a good idea. This approach is confusing and fragile to modifications on the code, because adding any local variable will mess up your call.
I would go for the more explicit:
class MyObj:
def test(self, a=1, b=2, c=3, d=4):
return [a,b,c,d]
class MyVerboseObj(MyObj):
def test(self, a=1, b=2, c=3, d=4):
print "RUNNING TEST a=%d"%a
return MyObj.test(self, a, b, c, d)
MyVerboseObj().test()
I know it is tedious, but it avoids some trouble. Plus it is more efficient.
Alternatively, if you are OK with loosing the signature, you could just accept *args, **kwargs as parameters to your overriding function:
class MyObj:
def test(self, a=1, b=2, c=3, d=4):
return [a,b,c,d]
class MyVerboseObj(MyObj):
def test(self, *args, **kwargs):
print "RUNNING TEST"
return MyObj.test(self, *args, **kwargs)
MyVerboseObj().test()
Accessing a in this version would be a bit of a hassle, but since you are not really interested in the arguments in your second example this might be the easier solution.
As a side note: As your inheritance structure becomes more complicated (i.e. multiple inheritance) you might want to start using the super() method:
class MyObj:
def test(self, a=1, b=2, c=3, d=4):
return [a,b,c,d]
class MyVerboseObj(MyObj):
def test(self, *args, **kwargs):
print "RUNNING TEST"
super(MyObj, self).test(*args, **kwargs)
MyVerboseObj().test()
When I wrap a function with #, how do I make the wrapper function look & feel exactly like the wrapped function? help(function) in particular.
Some code:
>>> def wraps(f):
def call(*args, **kw):
print('in', f, args, kw) # example code. I need to transfer the arguments to another process and pickle them.
return f(*args, **kw)
return call
>>> def g():pass
>>> #wraps
def f(a, b = 1, g = g, *args, **kw):
pass
>>> help(f)
Help on function call in module __main__:
call(*args, **kw) # this line bothers me. It should look different, look below
>>> def f(a, b = 1, g = g, *args, **kw):
pass
>>> help(f)
Help on function f in module __main__:
f(a, b=1, g=<function g at 0x02EE14B0>, *args, **kw) # help(f) should look like this.
Motivation: It would also be nice to see the arguments when the help window pops up, when I type f( * plopp * I see (a, b = 1, g = g, *args, **kw). (in this case in the IDLE Python Shell)
I had a look at the inspect module which helps me with nice formatting. The problem is still there: how do I do this with arguments..
Default mutable argument passing like def f(d = {}): does not need to work since I transfer the arguments to another process and the identity would be lost anyway.
functools.wraps can be used to copy the name and docstring of the function. Copying the original function signature is considerably harder to do from scratch.
If you use the third-party decorator module, however, then
import decorator
#decorator.decorator
def wraps(f):
def call(*args, **kw):
print('in', f, args, kw)
return f(*args, **kw)
return call
def g():pass
#wraps
def f(a, b = 1, g = g, *args, **kw):
pass
help(f)
yields
Help on function f in module __main__:
f(a, b=1, g=<function g>, *args, **kw)
Use functools.wraps:
from functools import wraps
def wrapper(f):
#wraps(f)
def call(*args, **kw):
print('in', f, args, kw)
return f(*args, **kw)
return call
#wrapper
def f(a, b = 1, g = g, *args, **kw):
pass
help(f)
Help on function f in module __main__:
f(a, b=1, g=<function g at 0x7f5ad14a6048>, *args, **kw)
This preserves the __name__ and __doc__ attributes of your wrapped function.
I think the other answers are preferable, but if for some reason you don't want to use an external module, you could always alter your decorator like so:
def wraps(f):
def call(*args, **kw):
print('in', f, args, kw)
return f(*args, **kw)
call.__name__ = f.__name__
call.__doc__ = f.__doc__
return call