I tried to reimplement something like partial (which later will have more behavior). Now in the following example lazycall1 seems to work just as fine as lazycall2, so I don't understand why the documentation of partial suggests using the longer second version. Any suggestions? Can it get me in trouble?
def lazycall1(func, *args, **kwargs):
def f():
func(*args, **kwargs)
return f
def lazycall2(func, *args, **kwargs):
def f():
func(*args, **kwargs)
f.func=func # why do I need that?
f.args=args
f.kwargs=kwargs
return f
def A(x):
print("A", x)
def B(x):
print("B", x)
a1=lazycall1(A, 1)
b1=lazycall1(B, 2)
a1()
b1()
a2=lazycall2(A, 3)
b2=lazycall2(B, 4)
a2()
b2()
EDIT: Actually the answers given so far aren't quite right. Even with double arguments it would work. Is there another reason?
def lazycall(func, *args):
def f(*args2):
return func(*(args+args2))
return f
def sum_up(a, b):
return a+b
plusone=lazycall(sum_up, 1)
plustwo=lazycall(sum_up, 2)
print(plusone(6)) #7
print(plustwo(9)) #11
The only extra thing the second form has, are some extra properties. This might be helpful if you start passing around the functions returned by lazycall2, so that the receiving function may make decisions based on these values.
functools.partial can accept additional arguments - or overridden arguments - in the inner, returned function. Your inner f() functions don't, so there's no need for what you're doing in lazycall2. However, if you wanted to do something like this:
def sum(a, b):
return a+b
plusone = lazycall3(sum, 1)
plusone(6) # 7
You'd need to do what is shown in those docs.
Look closer at the argument names in the inner function newfunc in the Python documentation page you link to, they are different than those passed to the inner function, args vs. fargs, keywords vs. fkeywords. Their implementation of partial saves the arguments that the outer function was given and adds them to the arguments given to the inner function.
Since you reuse the exact same argument names in your inner function, the original arguments to the outer function won't be accessible in there.
As for setting func, args, and kwargs attributes on the outer function, a function is an object in Python, and you can set attributes on it. These attributes allow you to get access to the original function and arguments after you have passed them into your lazycall functions. So a1.func will be A and a1.args will be [1].
If you don't need to keep track of the original function and arguments, you should be fine
with your lazycall1.
Related
I want to write a wrapper function which call one function and pass the results to another function. The arguments and return types of the functions are the same, but I have problem with returning lists and multiple values.
def foo():
return 1,2
def bar():
return (1,2)
def foo2(a,b):
print(a,b)
def bar2(p):
a,b=p
print(a,b)
def wrapper(func,func2):
a=func()
func2(a)
wrapper(bar,bar2)
wrapper(foo,foo2)
I am searching for a syntax which works with both function pairs to use it in my wrapper code.
EDIT: The definitions of at least foo2 and bar2 should stay this way. Assume that they are from an external library.
There is no distinction. return 1,2 returns a tuple. Parentheses do not define a tuple; the comma does. foo and bar are identical.
As I overlooked until JacobIRR's comment, your problem is that you need to pass an actual tuple, not the unpacked values from a tuple, to bar2:
a = foo()
foo2(*a)
a = bar()
bar2(a)
I don't necessarily agree with the design, but following your requirements in the comments (the function definitions can't change), you can write a wrapper that tries to execute each version (packed vs. unpacked) since it sounds like you might not know what the function expects. The wrapper written below, argfixer, does exactly that.
def argfixer(func):
def wrapper(arg):
try:
return func(arg)
except TypeError:
return func(*arg)
return wrapper
def foo():
return 1,2
def bar():
return (1,2)
#argfixer
def foo2(a,b):
print(a,b)
#argfixer
def bar2(p):
a,b=p
print(a,b)
a = foo()
b = bar()
foo2(a)
foo2(b)
bar2(a)
bar2(b)
However, if you aren't able to put the #argfixer on the line before the function definitions, you could alternatively wrap them like this in your own script before calling them:
foo2 = argfixer(foo2)
bar2 = argfixer(bar2)
And as mentioned in previous comments/answers, return 1,2 and return (1,2) are equivalent and both return a single tuple.
This code does not run because of arg differences. It runs if you use def foo2(*args): and def bar2(*p):.
The return 1, 2 and return (1, 2) are equivalent. The comma operator just creates a tuple, whether it is enclosed in parentheses or not.
All programming languages that I know of return a single value, so, since you want to return multiple, those values must be wrapped into a collection type, in this case, a tuple.
The problem is in the way you call the second function. Make it bar2(a) instead of bar2(*a), which breaks the tuple into separate arguments.
How to get the values of the call parameters of a function / method?
It's for a debugging tool, and it would be used in a scenario like this one:
import inspect
def getCallParameter():
stack = inspect.stack()
outer = stack[1] #This is an example, I have a smarter way of finding the right frame
frame = outer.frame
print("") #print at least a dict of the value of the parameters, and at best, give also which parameter are passed explicitly (e.g. with f(2, 4) an input of this kind : "a=2, b=4, c=5(default)")
def f(a, b=4, c=5):
a+=b+c
getCallParameters()
return a
Note : I'm aware of inspect.formatargsvalues() but it doesn't meet my requirements since in the f(2,4) example, it would have printed "a=11, b=4, c=5)"
What I though of, would be to watch on the outer frame the values passed. It's not a problem if I don't get the original state of the passed object, as long as I get the objects initially bound to the variable parameters.
Example:
# with f(4)
def f(a, b=4, c=[])
c.append(5)
getCallParameters() # a=4, b=4, c=[5] is ok even if I would have preferred c=[]
c = [4]
getCallParameters() # here, I expect c=[5] or c=[]
I have a different way in mind to cope with finding exact parameters passed, but I do wonder if it is compatible with your debugging tool. This takes good care of how the function is called. Still, real frame parameters should be retrieved with things like inspect.
def inspect_decorator(func): # decorator
def wrapper(*args, **kwargs): # wrapper function
# TODO: don't print! consume somewhere else
print('{} is called with args={} kwargs={}'.format(func.__name__, args, kwargs))
return func(*args, **kwargs) # actual execution
return wrapper # wrapped function
#inspect_decorator
def f(a, b=4, c=5):
a += b + c
return a
f(3)
f(3, 5)
f(3, b=4)
# f is called with args=(3,) kwargs={}
# f is called with args=(3, 5) kwargs={}
# f is called with args=(3,) kwargs={'b': 4}
In the TODO part, you may use a context stack to keep track of every function interested. But again, it depends on what you have done with the whole project.
I currently have the following structure:
Inside a class I need to handle several types of functions with two special variables and an arbitrary number of parameters. To wrap these for the methods I apply them on I scan the function signatures first (that works very reliable) and decide what the parameters and what my variables are.
I then bind them back with a lambda expression in the following way. Let func(x, *args) be my function, then I'll bind
f = lambda x, t: func(x=x, **func_parameter)
In the case that I get func(x, t, *args) I bind
f = lambda x, t: func(x=x, t=t, **func_parameter)
and similar if I have neither variables.
It is essential that I hand a function of the form f(x,t) to my methods inside that class.
I would like to use functools.partial for that - it is the more pythonic way to do it and the performance when executing is better (the function f is potentially called a couple of million times...). The problem that I have is that I don't know what to do if I have a basis function which is independent of one of the variables t and x, that's why I went with lambda functions at all, they just map the other variable 'blind'. It's still two function calls and while definitions with lambda and partial take the same time, execution is a lot faster with partial.
Does anyone knoe how to use partial in that case? Performance is kind of an issue here.
EDIT: A little later. I figured out that function evaluation with tuple arguments are faster than with keyword arguments, so that was a plus.
And then, in the end, as a user I would just take some of the guess work from Python, i.e. directly define
def func(x):
return 2*x
instead of
def func(x, a):
return a*x
And call it directly. In that way I can use the function directly. Second case would be if I implement the case where x and t are both present as partial mapping.
That might be a compromise.
You could write adapter classes that have an f(x,t) call signature. The result is similar to functools.partial but much more flexible. __call__ gives you a consistent call signature and lets you add, drop, and map parameters. Arguments can be fixed when an instance is made. It seems like it should execute as fast as a normal function, but I have no basis for that.
A toy version:
class Adapt:
'''return a function with call signature f(x,t)'''
def __init__(self, func, **kwargs):
self.func = func
self.kwargs = kwargs
def __call__(self, x, t):
# mapping magic goes here
return self.func(x, t, **self.kwargs)
#return self.func(a=x, b=t, **self.kwargs)
def f(a, b, c):
print(a, b, c)
Usage:
>>> f_xt = Adapt(f, c = 4)
>>> f_xt(3, 4)
3 4 4
>>>
Don't know how you could make that generic for arbitrary parameters and mappings, maybe someone will chime in with an idea or an edit.
So if you end up writing an adapter specific to each function, the function can be embedded in the class instead of an instance parameter.
class AdaptF:
'''return a function with call signature f(x,t)'''
def __init__(self, **kwargs):
self.kwargs = kwargs
def __call__(self, x, t):
'''does stuff with x and t'''
# mapping magic goes here
return self.func(a=x, b=t, **self.kwargs)
def func(self, a, b, c):
print(a, b, c)
>>> f_xt = AdaptF(c = 4)
>>> f_xt(x = 3, t = 4)
3 4 4
>>>
I just kinda made this up from stuff I have read so I don't know if it is viable. I feel like I should give credit to the source I read but for the life of me I can't find it - I probably saw it on a pyvideo.
.
I have a base decorator that takes arguments but that also is built upon by other decorators. I can't seem to figure where to put the functools.wraps in order to preserve the full signature of the decorated function.
import inspect
from functools import wraps
# Base decorator
def _process_arguments(func, *indices):
""" Apply the pre-processing function to each selected parameter """
#wraps(func)
def wrap(f):
#wraps(f)
def wrapped_f(*args):
params = inspect.getargspec(f)[0]
args_out = list()
for ind, arg in enumerate(args):
if ind in indices:
args_out.append(func(arg))
else:
args_out.append(arg)
return f(*args_out)
return wrapped_f
return wrap
# Function that will be used to process each parameter
def double(x):
return x * 2
# Decorator called by end user
def double_selected(*args):
return _process_arguments(double, *args)
# End-user's function
#double_selected(2, 0)
def say_hello(a1, a2, a3):
""" doc string for say_hello """
print('{} {} {}'.format(a1, a2, a3))
say_hello('say', 'hello', 'arguments')
The result of this code should be and is:
saysay hello argumentsarguments
However, running help on say_hello gives me:
say_hello(*args, **kwargs)
doc string for say_hello
Everything is preserved except the parameter names.
It seems like I just need to add another #wraps() somewhere, but where?
I experimented with this:
>>> from functools import wraps
>>> def x(): print(1)
...
>>> #wraps(x)
... def xyz(a,b,c): return x
>>> xyz.__name__
'x'
>>> help(xyz)
Help on function x in module __main__:
x(a, b, c)
AFAIK, this has nothing to do with wraps itself, but an issue related to help. Indeed, because help inspects your objects to provide the information, including __doc__ and other attributes, this is why you get this behavior, although your wrapped function has different argument list. Though, wraps doesn't update that automatically (the argument list) what it really updates is this tuple and the __dict__ which is technically the objects namespace:
WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__doc__',
'__annotations__')
WRAPPER_UPDATES = ('__dict__',)
If you aren't sure about how wraps work, probably it'll help if your read the the source code from the standard library: functools.py.
It seems like I just need to add another #wraps() somewhere, but where?
No, you don't need to add another wraps in your code, help as I stated above works that way by inspecting your objects. The function's arguments are associated with code objects (__code__) because your function's arguments are stored/represented in that object, wraps has no way to update the argument of the wrapper to be like the wrapped function (continuing with the above example):
>>> xyz.__code__.co_varnames
>>> xyz.__code__.co_varnames = x.__code__.co_varnames
AttributeError: readonly attribute
If help displayed that function xyz has this argument list () instead of (a, b, c) then this is clearly wrong! And the same applies for wraps, to change the argument list of the wrapper to the wrapped, would be cumbersome! So this should not be a concern at all.
>>> #wraps(x, ("__code__",))
... def xyz(a,b,c): pass
...
>>> help(xyz)
Help on function xyz in module __main__:
xyz()
But xyz() returns x():
>>> xyz()
1
For other references take a look at this question or the Python Documentation
What does functools.wraps do?
direprobs was correct in that no amount of functools wraps would get me there. bravosierra99 pointed me to somewhat related examples. However, I couldn't find a single example of signature preservation on nested decorators in which the outer decorator takes arguments.
The comments on Bruce Eckel's post on decorators with arguments gave me the biggest hints in achieving my desired result.
The key was in removing the middle function from within my _process_arguments function and placing its parameter in the next, nested function. It kind of makes sense to me now...but it works:
import inspect
from decorator import decorator
# Base decorator
def _process_arguments(func, *indices):
""" Apply the pre-processing function to each selected parameter """
#decorator
def wrapped_f(f, *args):
params = inspect.getargspec(f)[0]
args_out = list()
for ind, arg in enumerate(args):
if ind in indices:
args_out.append(func(arg))
else:
args_out.append(arg)
return f(*args_out)
return wrapped_f
# Function that will be used to process each parameter
def double(x):
return x * 2
# Decorator called by end user
def double_selected(*args):
return _process_arguments(double, *args)
# End-user's function
#double_selected(2, 0)
def say_hello(a1, a2,a3):
""" doc string for say_hello """
print('{} {} {}'.format(a1, a2, a3))
say_hello('say', 'hello', 'arguments')
print(help(say_hello))
And the result:
saysay hello argumentsarguments
Help on function say_hello in module __main__:
say_hello(a1, a2, a3)
doc string for say_hello
I've been playing around in depth with attempting to write my own version of a memoizing decorator before I go looking at other people's code. It's more of an exercise in fun, honestly. However, in the course of playing around I've found I can't do something I want with decorators.
def addValue( func, val ):
def add( x ):
return func( x ) + val
return add
#addValue( val=4 )
def computeSomething( x ):
#function gets defined
If I want to do that I have to do this:
def addTwo( func ):
return addValue( func, 2 )
#addTwo
def computeSomething( x ):
#function gets defined
Why can't I use keyword arguments with decorators in this manner? What am I doing wrong and can you show me how I should be doing it?
You need to define a function that returns a decorator:
def addValue(val):
def decorator(func):
def add(x):
return func(x) + val
return add
return decorator
When you write #addTwo, the value of addTwo is directly used as a decorator. However, when you write #addValue(4), first addValue(4) is evaluated by calling the addValue function. Then the result is used as a decorator.
You want to partially apply the function addValue - give the val argument, but not func. There are generally two ways to do this:
The first one is called currying and used in interjay's answer: instead of a function with two arguments, f(a,b) -> res, you write a function of the first arg that returns another function that takes the 2nd arg g(a) -> (h(b) -> res)
The other way is a functools.partial object. It uses inspection on the function to figure out what arguments a function needs to run (func and val in your case ). You can add extra arguments when creating a partial and once you call the partial, it uses all the extra arguments given.
from functools import partial
#partial(addValue, val=2 ) # you can call this addTwo
def computeSomething( x ):
return x
Partials are usually a much simpler solution for this partial application problem, especially with more than one argument.
Decorators with any kinds of arguments -- named/keyword ones, unnamed/positional ones, or some of each -- essentially, ones you call on the #name line rather than just mention there -- need a double level of nesting (while the decorators you just mention have a single level of nesting). That goes even for argument-less ones if you want to call them in the # line -- here's the simplest, do-nothing, double-nested decorator:
def double():
def middling():
def inner(f):
return f
return inner
return middling
You'd use this as
#double()
def whatever ...
note the parentheses (empty in this case since there are no arguments needed nor wanted): they mean you're calling double, which returns middling, which decorates whatever.
Once you've seen the difference between "calling" and "just mentioning", adding (e.g. optional) named args is not hard:
def doublet(foo=23):
def middling():
def inner(f):
return f
return inner
return middling
usable either as:
#doublet()
def whatever ...
or as:
#doublet(foo=45)
def whatever ...
or equivalently as:
#doublet(45)
def whatever ...