We Keep Coding

Python's `functools.wraps` breaks when keyword only arguments involved

It seems like functools.wraps breaks the wrapped function when keyword only kinds are involved. Did I find a stlib bug, or is there something I'm missing
Consider this code:
from functools import wraps, WRAPPER_ASSIGNMENTS, partial
# First, I need to add `__defaults__` and `__kwdefaults__` to wraps, because they don't come for free...
my_wraps = partial(wraps, assigned=(list(WRAPPER_ASSIGNMENTS) + ['__defaults__', '__kwdefaults__']))
def g(a: float, b=10):
return a * b
def f(a: int, *, b=1):
return a * b
# all is well (for now)...
assert f(3) == 3
assert g(3) == 30
Whether I do this:
g = my_wraps(f)(g)
g(3) # raises TypeError (missing required positional argument 'b'), expected
or this:
f = my_wraps(g)(f)
f(3) # raises TypeError (missing required positional argument 'b'), expected
my wrapped function breaks.

Pythonic way to write wrapper functions

Let's say I have a function foo that gets a few parameters
def foo(width, height, depth=0):
...
I want to write a wrapper function that gets all of foo's parameters and passes them on, e.g.
def goo(width, height, depth=0):
...
foo(width, height, depth)
...
But this is ugly, since I have to repeat the variables and the default values.
What's the idiomatic way to do this in python?
A few options I thought about:
passing to goo a dictionary called foo_params and calling foo(**foo_params) but then is error prone since I don't know
if all the arguments are there
writing another wrapper for foo that checks if the params with default values are None and if so doesn't pass them
Putting the default values as constants so I won't repeat them

You can use *args and **kwargs syntax to pass an unknown amount of arguments and/or keyword arguments:
>>> def dec(func):
def inner(*args, **kwargs):
print('decorated function')
func(*args, **kwargs)
return inner
>>> #dec
def func(a, b):
return a + b
>>> func(1, 2)
decorated function
>>>
One downside to using *args and **kwargs is that you'll lose the orginal function signature of the decorated function. eg:
>>> help(func)
Help on function inner in module __main__:
inner(*args, **kwargs)
>>>
The solution is to use functools.wraps(). It basically copies of the data from the decorated function to the wrapper function:
>>> from functools import wraps
>>>
>>> def dec(func):
#wraps(func)
def inner(*args, **kwargs):
print('decorated function')
func(*args, **kwargs)
return inner
>>> #dec
def func(a, b):
return a + b
>>> func(1, 2)
decorated function
>>>
A you can see below, if you now do help(func) the original signature for func will be displayed:
>>> help(func)
Help on function func in module __main__:
func(a, b)
>>>

I think you are looking for functools's partial function:
from functools import partial
def foo(a,b):
return a + b
goo = partial(foo, b = 1)
goo(5) # returns 6

Python - Iterating over Arguments passed to a Function

Suppose I have the following example:
class foo:
...
def bar(self, w, x, y, z, ...):
self.w = w
self.x = x
self.y = y
self.z = z
...
I wish to reduce the n-number of attribute assignment lines in bar() to one assignment line set using a setattr() cycle through the arguments. Is there a good way to cycle through said arguments for this purpose?
I wish to retain the defined parameter names so as to limit the number of parameters passed to the function as well as the order in which they are passed. I also understand that functions can be handled like objects; so is it possible to obtain a list of the defined parameters as an attribute of the function and iterate through that?

Use locals() and you can get all the arguments (and any other local variables):
class foo:
def bar(self, w, x, y, z):
argdict = {arg: locals()[arg] for arg in ('w', 'x', 'y', 'z')}
for key, value in argdict.iteritems():
setattr(self, key, value)
...
Might be possible to do it more efficiently, and you could inline argdict if you prefer less lines to readability or find it more readable that way.

So you don't have to actually name the arguments explicitly use:
class foo:
def __init__(self, w, x, y, z):
args = locals()# gets a dictionary of all local parameters
for argName in args:
if argName!='self':
setattr(self, argName, args[argName])

The __setattr__ attribute only assigns one attribute at a time, if you want to assign multiple attribute, you can use **kwargs in your function header and for limiting the number of arguments you can simply check the length of kwargs within your function. and call the __setattr__ for each each of the arguments one by one. One good reason for this recipe is that basically assigning attribute to an object without considering anything is not a correct and desirable job, due to a lot of reasons. Thus you have to assign each attribute one at a time by considering all the required conditions.
You can also do this manually by updating the instance dictionary but you should handle the exceptions too.
In [80]: class foo:
def bar(self, **kwargs):
if len(kwargs) != 4:
raise Exception("Please enter 4 keyword argument")
for k, v in kwargs.items():
foo.__setattr__(self, k, v)
....:
In [81]: f = foo()
In [82]: f.bar(w=1, x=2, y=3, z=4)
In [83]: f.w
Out[83]: 1
In [84]: f.bar(w=1, x=2, y=3, z=4, r=5)
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-84-758f669d08e0> in <module>()
----> 1 f.bar(w=1, x=2, y=3, z=4, r=5)
<ipython-input-80-9e46a6a78787> in bar(self, **kwargs)
2 def bar(self, **kwargs):
3 if len(kwargs) != 4:
----> 4 raise Exception("Please enter 4 keyword argument")
5 for k, v in kwargs.items():
6 foo.__setattr__(self, k, v)
Exception: Please enter 4 keyword argument
By using __setatter__ it will take care of the exception automatically:
In [70]: f.bar(1, 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-70-07d1f3c9e27f> in <module>()
----> 1 f.bar(1, 2)
<ipython-input-65-1049e26120c1> in bar(self, *args)
2 def bar(self, *args):
3 for item in args:
----> 4 foo.__setattr__(self, item, item)
5
TypeError: attribute name must be string, not 'int'

How to understand python decorator arguments pass

I try to understand python decorator
def dec(func):
def wrap(*args,**kw):
print args, kw
return func(*args,**kw)
return wrap
#dec
def myfunc(a=1,b=2,c=3):
return a+b+c
>>> myfunc()
() {}
6
>>> myfunc(1,2,3)
(1, 2, 3) {}
6
>>> myfunc(1,2,c=5)
(1, 2) {'c': 5}
8
>>>
When I run myfunc() args and kw are nothing, but when I run myfunc(1,2,3) and myfunc(1,2,c=5), args and kw were passed to dec function.
As I know,
#dec
def myfunc(...)
equals to myfunc = dec(myfunc) <-- no arguments were mentioned here.
How arguments were passed to wrap function in dec? How to understand these?

Not sure if i understand correctly your problem, but the default values for myfunc arguments are known only to myfunc - your decorator has no knowledge of them, so it cannot print them.
That's why:
myfunc()
results in printing:
() {}
Both *args and **kw are empty for the decorator, but the decorated function will use the default values in this case.
In the second and third case you get the parameters printed, as they are explicitly passed to the decorated function:
def wrap(*args,**kw): <- this is what is actually called when you invoke decorated function, no default values here
print args, kw
return func(*args,**kw) <- this is the function you decorate
#if func has default parameter values, then those will be used
#when you don't pass them to the wrapper but only **inside** func
return wrap
Edit:
It looks like you're mistaking calling the decorated function with decorating the function:
myfunc = dec(myfunc)
decorates myfunc using dec and is equivalent to
#dec
def myfunc(...)
On the other hand, after using either of them:
myfunc(whatever)
calls the wrap function defined in your decorator, which will in turn call the original myfunc

Another way to think of it is by saying:
def wrapper(some_function):
def _inner(*args, **kwargs):
#do some work
return func(*args, **kwargs)
return _inner
#wrapper
def foo(a, b, c):
return "Foo called, and I saw %d, %d, %d" % (a, b, c)
...you're getting a result which is roughly similar to the following:
def foo(a, b, c):
#do some work
return "Foo called, and I saw %d, %d, %d" % (a, b, c)
This isn't exactly right because the #do some work is occurring before the actual foo() call, but as an approximation this is what you're getting. For that reason, the wrapper can't really 'see' the default arguments for foo() if any exist. So a better way to think of it might be:
#always execute this code before calling...
def foo(a, b, c):
return "Foo called and I saw %d, %d, %d" % (a, b, c)
So something really basic might look like this.
def wrap(f):
... def _inner(*a, **k):
... new_a = (ar*2 for ar in a)
... new_k = {}
... for k, v in k.iteritems():
... new_k[k] = v*2
... return f(*new_a, **new_k)
... return _inner
...
>>> def foo(a=2, b=4, c=6):
... return "%d, %d, %d" % (a, b, c)
...
>>> foo()
'2, 4, 6'
>>> foo(1, 5, 7)
'1, 5, 7'
>>> foo = wrap(foo) #actually wrapping it here
>>> foo()
'2, 4, 6'
>>> foo(3, 5, 6)
'6, 10, 12'
>>> foo(3, 5, c=7)
'6, 10, 14'
>>>

Decorators are function wrappers. They give back a function that wraps the original one into some pre- and post-processing code, but still need to call the original function (normally with the same argument as you would call it in absence of a decorator).
Python has two types of arguments, positional and keyword arguments (this has nothing to do with decorators, that's generic python basics). * is for positional (internally is a list), ** for keyword (dictionary) arguments. By specifying both you allow your decorator to accept all at all possible types of arguments and pass them through to the underlying function. The contract of the call is, however, still defined by your function. E.g. if it only takes keyword arguments it will fail when the decorator function passes through a positional argument.
In your particular example, you have some pre-processing code (i.e. code that will run before the original function is called). For example in this code you can print out arguments *args that your original function might fail to accept all together because it does not take any position arguments.
You do not necessarily have to pass through *args and **kwargs. In fact you can define a decorator which makes some decisions based on the arguments you pass in about what to provide to the original function as arguments, e.g.
def dec(fun):
def wrap(*args, **kwargs):
if 'a' in kwargs:
return fun(kwargs[a])
else:
return fun(*args)
return wrap

Python Argument Binders

How can I bind arguments to a Python function so that I can call it later without arguments (or with fewer additional arguments)?
For example:
def add(x, y):
return x + y
add_5 = magic_function(add, 5)
assert add_5(3) == 8
What is the magic_function I need here?
It often happens with frameworks and libraries that people accidentally call a function immediately when trying to give arguments to a callback: for example on_event(action(foo)). The solution is to bind foo as an argument to action, using one of the techniques described here. See for example How to pass arguments to a Button command in Tkinter? and Using a dictionary as a switch statement in Python.
Some APIs, however, allow you to pass the to-be-bound arguments separately, and will do the binding for you. Notably, the threading API in the standard library works this way. See thread starts running before calling Thread.start. If you are trying to set up your own API like this, see How can I write a simple callback function?.
Explicitly binding arguments is also a way to avoid problems caused by late binding when using closures. This is the problem where, for example, a lambda inside a for loop or list comprehension produces separate functions that compute the same result. See What do lambda function closures capture? and Creating functions (or lambdas) in a loop (or comprehension).

functools.partial returns a callable wrapping a function with some or all of the arguments frozen.
import sys
import functools
print_hello = functools.partial(sys.stdout.write, "Hello world\n")
print_hello()
Hello world
The above usage is equivalent to the following lambda.
print_hello = lambda *a, **kw: sys.stdout.write("Hello world\n", *a, **kw)

Using functools.partial:
>>> from functools import partial
>>> def f(a, b):
... return a+b
...
>>> p = partial(f, 1, 2)
>>> p()
3
>>> p2 = partial(f, 1)
>>> p2(7)
8

If functools.partial is not available then it can be easily emulated:
>>> make_printer = lambda s: lambda: sys.stdout.write("%s\n" % s)
>>> import sys
>>> print_hello = make_printer("hello")
>>> print_hello()
hello
Or
def partial(func, *args, **kwargs):
def f(*args_rest, **kwargs_rest):
kw = kwargs.copy()
kw.update(kwargs_rest)
return func(*(args + args_rest), **kw)
return f
def f(a, b):
return a + b
p = partial(f, 1, 2)
print p() # -> 3
p2 = partial(f, 1)
print p2(7) # -> 8
d = dict(a=2, b=3)
p3 = partial(f, **d)
print p3(), p3(a=3), p3() # -> 5 6 5

lambdas allow you to create a new unnamed function with fewer arguments and call the function:
>>> def foobar(x, y, z):
... print(f'{x}, {y}, {z}')
...
>>> foobar(1, 2, 3) # call normal function
1, 2, 3
>>> bind = lambda x: foobar(x, 10, 20) # bind 10 and 20 to foobar
>>> bind(1)
1, 10, 20
>>> bind = lambda: foobar(1, 2, 3) # bind all elements
>>> bind()
1, 2, 3
You can also use functools.partial. If you are planning to use named argument binding in the function call this is also applicable:
>>> from functools import partial
>>> barfoo = partial(foobar, x=10)
>>> barfoo(y=5, z=6)
10, 5, 6
Note that if you bind arguments from the left you need to call the arguments by name. If you bind from the right it works as expected.
>>> barfoo(5, 6)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foobar() got multiple values for argument 'x'
>>> f = partial(foobar, z=20)
>>> f(1, 1)
1, 1, 20

This would work, too:
def curry(func, *args):
def curried(*innerargs):
return func(*(args+innerargs))
curried.__name__ = "%s(%s, ...)" % (func.__name__, ", ".join(map(str, args)))
return curried
>>> w=curry(sys.stdout.write, "Hey there")
>>> w()
Hey there

Functors can be defined this way in Python. They're callable objects. The "binding" merely sets argument values.
class SomeFunctor( object ):
def __init__( self, arg1, arg2=None ):
self.arg1= arg1
self.arg2= arg2
def __call___( self, arg1=None, arg2=None ):
a1= arg1 or self.arg1
a2= arg2 or self.arg2
# do something
return
You can do things like
x= SomeFunctor( 3.456 )
x( arg2=123 )
y= SomeFunctor( 3.456, 123 )
y()

The question asks generally about binding arguments, but all answers are about functions. In case you are wondering, partial also works with class constructors (i.e. using a class instead of a function as a first argument), which can be useful for factory classes. You can do it as follows:
from functools import partial
class Animal(object):
def __init__(self, weight, num_legs):
self.weight = weight
self.num_legs = num_legs
animal_class = partial(Animal, weight=12)
snake = animal_class(num_legs = 0)
print(snake.weight) # prints 12