We Keep Coding

Function as an argument of another function

I'm learning this language hence I'm new with Python. The code is:
def add(a, b):
return a + b
def double_add(x, a, b):
return x(x(a, b), x(a, b))
a = 4
b = 5
print(double_add(add, a, b))
The add function is simple, it adds two numbers. The double_add function has three arguments. I understand what is happening (With some doubts). The result is 18. I can't understand how double_add uses add to function.
The question is, what is the connection between these two functions?
It would be helpful if tell me some examples of using a function as an argument of another function.
Thanks in advance.

In python language, functions (and methods) are first class objects. First Class objects are those objects, which can be handled uniformly.
So, you just pass a method as an argument.
Your method will return add(add(4, 5), add(4, 5)) which is add(9, 9) and it's equals to 18.

A function is an object just like any other in Python. So you can pass it as argument, assign attributes to it, and well maybe most importantely - call it. We can look at a simpler example to understand how passing a function works:
def add(a, b):
return a + b
def sub(a, b):
return a - b
def operate(func, a, b):
return func(a, b)
a = 4
b = 5
print(operate(add, a, b))
print(operate(sub, a, b))
operate(print, a, b)
And this prints out:
9
-1
4 5
That is because in each case, func is assigned with the respective function object passed as an argument, and then by doing func(a, b) it actually calls that function on the given arguments.
So what happens with your line:
return x(x(a, b), x(a, b))
is first both x(a, b) are evaluated as add(4, 5) which gives 9. And then the outer x(...) is evaluated as add(9, 9) which gives 18.

If you would add print(x) in the double_add function you would see that it would print <function add at 0x10dd12290>.
Therefore, the code of double_add is basically the same as if you would do following:
print(add(add(a,b), add(a,b))) # returns 18 in your case

Functions are objects in Python, just like anything else such as lists, strings.. and you can pass them same way you do with variables.

The function object add is passed as an argument to double_add, where it is locally referred to as x. x is then called on each, and then on the two return values from that.

def double_add(x, a, b):
return x(x(a, b), x(a, b))
Let's write it differently so it's easier to explain:
def double_add(x, a, b):
result1 = x(a, b)
result2 = x(a, b)
return x(result1, result2)
This means, take the function x, and apply it to the parameters a and b. x could be whatever function here.
print(double_add(add, a, b))
Then this means: call the double_add function, giving itaddas the first parameter. Sodouble_add`, would do:
result1 = add(a, b)
result2 = add(a, b)
return add(result1, result2)

This is a very simple example of what is called "dependency injection". What it means is that you are not explicitly defining an interaction between the two functions, instead you are defining that double_add should use some function, but it only knows what it is when the code is actually run. (At runtime you are injecting the depedency on a specific function, instead of hardcoding it in the function itself),
Try for example the following
def add(a, b):
return a + b
def subtract(a, b):
return a - b
def double_add(x, a, b):
return x(x(a, b), x(a, b))
a = 4
b = 5
print(double_add(add, a, b))
print(double_add(subtract, a, b))
In other words, double_add has become a generic function that will execute whatever you give it twice and print the result

Function that returns a function that returns a function

I've been given this function. It returns the function pair that it also returns the function f I think. That is the part that tricks me, I don't know what f(a, b) is and how to use it.
def cons(a, b):
def pair(f):
return f(a, b)
return pair

To help you understand what is going on, consider the following example:
def cons(a, b):
def pair(f):
return f(a, b)
return pair
def my_func(a, b):
return a + b
# cons returns a function that takes a function arg and calls it with args (a, b),
# in this case (1, 3). Here a, b are considered "closured" variables.
apply_func = cons(1, 3)
print apply_func(my_func) # prints 4

Lets analyse this from inside out:
def cons(a, b):
def pair(f):
return f(a, b)
return pair
The innermost level is return f(a, b) - that obviously calls function f with arguments (a, b) and returns whatever the result of that is.
The next level is pair:
def pair(f):
return f(a, b)
Function pair takes a function as an argument, calls that function with two arguments (a, b) and returns the result. For example:
def plus(x, y):
return x + y
a = 7
b = 8
pair(plus) # returns 15
The outermost level is cons - it constructs function pair which has arbitrary a and b and returns that version of pair. E.g.
pair_2_3 = cons(2,3)
pair_2_3(plus) # returns 5, because it calls plus(2, 3)

. . . I don't know what f(a, b) is and how to use it.
f(a, b) is simply a function call. All the code you provided does is define a function that returns a function. The function returned from the first function, itself returns a function. I assume the way it would be used is perhaps something like:
>>> cons(1, 2)(lambda x, y: x + y)
3
>>>
The above code would be equivalent to:
>>> pair_func = cons(1, 2) # return the `pair` function defined in `cons`
>>> f = lambda x, y: x + y
>>> pair_func(f) # apply the `f` function to the arguments passed into `cons`.
3
>>>
It might also help to note that the pair function defined in this case, is what's know as a closure. Essentially, a closure is a function which has access to local variables from an enclosing function's scope, after the function has finished execution. In your specific case, cons is the enclosing function, pair is the closure, and a and b are the variables the closure is accessing.

Well if you could share the complete question then we might be able to help you better. Meanwhile what I can tell you here is that in the return of pair(f) the program is calling a function f which takes two arguments a and b. This function f(a,b) is called and then its value will be returned to pair(f).
But the point to note here is that in pair function we already have a local variable f, so when we will try to call the function f(a,b) it will give us UnboundedLocalVariable error. Therefore, we will need to change the name of this function from f to something else.

Default values for iterable unpacking

I've often been frustrated by the lack of flexibility in Python's iterable unpacking.
Take the following example:
a, b = range(2)
Works fine. a contains 0 and b contains 1, just as expected. Now let's try this:
a, b = range(1)
Now, we get a ValueError:
ValueError: not enough values to unpack (expected 2, got 1)
Not ideal, when the desired result was 0 in a, and None in b.
There are a number of hacks to get around this. The most elegant I've seen is this:
a, *b = function_with_variable_number_of_return_values()
b = b[0] if b else None
Not pretty, and could be confusing to Python newcomers.
So what's the most Pythonic way to do this? Store the return value in a variable and use an if block? The *varname hack? Something else?

As mentioned in the comments, the best way to do this is to simply have your function return a constant number of values and if your use case is actually more complicated (like argument parsing), use a library for it.
However, your question explicitly asked for a Pythonic way of handling functions that return a variable number of arguments and I believe it can be cleanly accomplished with decorators. They're not super common and most people tend to use them more than create them so here's a down-to-earth tutorial on creating decorators to learn more about them.
Below is a decorated function that does what you're looking for. The function returns an iterator with a variable number of arguments and it is padded up to a certain length to better accommodate iterator unpacking.
def variable_return(max_values, default=None):
# This decorator is somewhat more complicated because the decorator
# itself needs to take arguments.
def decorator(f):
def wrapper(*args, **kwargs):
actual_values = f(*args, **kwargs)
try:
# This will fail if `actual_values` is a single value.
# Such as a single integer or just `None`.
actual_values = list(actual_values)
except:
actual_values = [actual_values]
extra = [default] * (max_values - len(actual_values))
actual_values.extend(extra)
return actual_values
return wrapper
return decorator
#variable_return(max_values=3)
# This would be a function that actually does something.
# It should not return more values than `max_values`.
def ret_n(n):
return list(range(n))
a, b, c = ret_n(1)
print(a, b, c)
a, b, c = ret_n(2)
print(a, b, c)
a, b, c = ret_n(3)
print(a, b, c)
Which outputs what you're looking for:
0 None None
0 1 None
0 1 2
The decorator basically takes the decorated function and returns its output along with enough extra values to fill in max_values. The caller can then assume that the function always returns exactly max_values number of arguments and can use fancy unpacking like normal.

Here's an alternative version of the decorator solution by #supersam654, using iterators rather than lists for efficiency:
def variable_return(max_values, default=None):
def decorator(f):
def wrapper(*args, **kwargs):
actual_values = f(*args, **kwargs)
try:
for count, value in enumerate(actual_values, 1):
yield value
except TypeError:
count = 1
yield actual_values
yield from [default] * (max_values - count)
return wrapper
return decorator
It's used in the same way:
#variable_return(3)
def ret_n(n):
return tuple(range(n))
a, b, c = ret_n(2)
This could also be used with non-user-defined functions like so:
a, b, c = variable_return(3)(range)(2)

Shortest known to me version (thanks to #KellyBundy in comments below):
a, b, c, d, e, *_ = *my_list_or_iterable, *[None]*5
Obviously it's possible to use other default value than None if necessary.
Also there is one nice feature in Python 3.10 which comes handy here when we know upfront possible numbers of arguments - like when unpacking sys.argv
Previous method:
import sys.argv
_, x, y, z, *_ = *sys.argv, *[None]*3
New method:
import sys
match sys.argv[1:]: #slice needed to drop first value of sys.argv
case [x]:
print(f'x={x}')
case [x,y]:
print(f'x={x}, y={y}')
case [x,y,z]:
print(f'x={x}, y={y}, z={z}')
case _:
print('No arguments')

Call another function and optionally keep default arguments

I have a function with one optional argument, like this:
def funA(x, a, b=1):
return a+b*x
I want to write a new function that calls funA and also has an optional argument, but if no argument is passed, I want to keep the default in funA.
I was thinking something like this:
def funB(x, a, b=None):
if b:
return funA(x, a, b)
else:
return funA(x, a)
Is there a more pythonic way of doing this?

I would replace if b with if b is not None, so that if you pass b=0 (or any other "falsy" value) as argument to funB it will be passed to funA.
Apart from that it seems pretty pythonic to me: clear and explicit. (albeit maybe a bit useless, depending on what you're trying to do!)
A little more cryptic way that relies on calling funB with the correct keyword arguments (e.g. funB(3, 2, b=4):
def funB(x, a, **kwargs):
return funA(x, a, **kwargs)

def funA(x, a, b=1):
return a+b*x
def funB(x, a, b=1):
return funA(x, a, b)
Make the default value of b=1 in funB() and then pass it always to funA()

The way you did it is fine. Another way is for funB to have the same defaults as funA, so you can pass the same parameters right through. E.g., if you do def funB(x, a, b=1), then you can always call return funA(x, a, b) just like that.
For simple cases, the above will work fine. For more complex cases, you may want to use *args and **kwargs (explained here and here). Specifically, you can pass in all your keyword arguments as a dictionary (conventionally called kwargs). In this case, each function would set its own independent defaults, and you would just pass the whole dictionary through:
def funA(x, a, **kwargs):
b = kwargs.get("b", 1)
return a+b*x
def funB(x, a, **kwargs):
return funA(x, a, **kwargs)
If kwargs is empty when passed to funB (b is not specified), it will be set to the default in funA by the statement b = kwargs.get("b", 1). If b is specified, it will be passed through as-is. Note that in funB, you can access b with its own, independent default value and still get the behavior you are looking for.
While this may seem like overkill for your example, extracting a couple of arguments at the beginning of a function is not a big deal if the function is complex enough. It also gives you a lot more flexibility (such as avoiding many of the common gotchas).

Using inspect.getargspec, you can get the default values (fourth item of the returned tuple = defaults):
import inspect
def funA(x, a, b=1):
return a + b * x
# inspect.getargspec(funA) =>
# ArgSpec(args=['x', 'a', 'b'], varargs=None, keywords=None, defaults=(1,))
def funcB(x, a, b=inspect.getargspec(funA)[3][0]):
return funA(x, a, b)
OR (in Python 2.7+)
def funcB(x, a, b=inspect.getargspec(funA).defaults[0]):
return funA(x, a, b)
In Python 3.5+, it's recommend to use inspect.signature instead:
def funcB(x, a, b=inspect.signature(funA).parameters['b'].default):
return funA(x, a, b)

Using FunctionType from types, you can just take a function and create a new one specifying the defaults at runtime. You can put all this in a decorator so that at the point of where you write your code it will keep things tidy, whilst still giving the reader a clue about what you are trying to accomplish. It also allows the exact same call signature for funB as funA -- all arguments can be positional, or all arguments can be keywords, or any valid mix thereof, and any arguments with default values are optional. Should play nice with positional arguments (*args) and keyword arguments (**kwargs) too.
import inspect
from types import FunctionType
def copy_defaults(source_function):
def decorator(destination_function):
"""Creates a wrapper for the destination function with the exact same
signature as source_function (including defaults)."""
# check signature matches
src_sig = inspect.signature(source_function)
dst_sig = inspect.signature(destination_function)
if list(src_sig.parameters) != list(dst_sig.parameters):
raise ValueError("src func and dst func do not having matching " \
"parameter names / order")
return FunctionType(
destination_function.__code__,
destination_function.__globals__,
destination_function.__name__,
source_function.__defaults__, # use defaults from src
destination_function.__closure__
)
return decorator
def funA(x, a, b=1):
return a+b*x
#copy_defaults(funA)
def funB(x, a, b):
"""this is fun B"""
return funA(x, a, b)
assert funA(1, 2) == funB(1, 2)
assert funB.__name__ == "funB"
assert funB.__doc__ == "this is fun B"

You can also use:
def funA(x, a, b=1):
return a+b*x
def funB(x, a, b=None):
return funA(*filter(lambda o: o is not None, [x, a, b]))
Version which will not fail if x or a are None:
def funB(x, a, b=None):
return funA(*([x, a]+filter(lambda o: o is not None, [b])))

How to understand python decorator arguments pass

I try to understand python decorator
def dec(func):
def wrap(*args,**kw):
print args, kw
return func(*args,**kw)
return wrap
#dec
def myfunc(a=1,b=2,c=3):
return a+b+c
>>> myfunc()
() {}
6
>>> myfunc(1,2,3)
(1, 2, 3) {}
6
>>> myfunc(1,2,c=5)
(1, 2) {'c': 5}
8
>>>
When I run myfunc() args and kw are nothing, but when I run myfunc(1,2,3) and myfunc(1,2,c=5), args and kw were passed to dec function.
As I know,
#dec
def myfunc(...)
equals to myfunc = dec(myfunc) <-- no arguments were mentioned here.
How arguments were passed to wrap function in dec? How to understand these?

Not sure if i understand correctly your problem, but the default values for myfunc arguments are known only to myfunc - your decorator has no knowledge of them, so it cannot print them.
That's why:
myfunc()
results in printing:
() {}
Both *args and **kw are empty for the decorator, but the decorated function will use the default values in this case.
In the second and third case you get the parameters printed, as they are explicitly passed to the decorated function:
def wrap(*args,**kw): <- this is what is actually called when you invoke decorated function, no default values here
print args, kw
return func(*args,**kw) <- this is the function you decorate
#if func has default parameter values, then those will be used
#when you don't pass them to the wrapper but only **inside** func
return wrap
Edit:
It looks like you're mistaking calling the decorated function with decorating the function:
myfunc = dec(myfunc)
decorates myfunc using dec and is equivalent to
#dec
def myfunc(...)
On the other hand, after using either of them:
myfunc(whatever)
calls the wrap function defined in your decorator, which will in turn call the original myfunc

Another way to think of it is by saying:
def wrapper(some_function):
def _inner(*args, **kwargs):
#do some work
return func(*args, **kwargs)
return _inner
#wrapper
def foo(a, b, c):
return "Foo called, and I saw %d, %d, %d" % (a, b, c)
...you're getting a result which is roughly similar to the following:
def foo(a, b, c):
#do some work
return "Foo called, and I saw %d, %d, %d" % (a, b, c)
This isn't exactly right because the #do some work is occurring before the actual foo() call, but as an approximation this is what you're getting. For that reason, the wrapper can't really 'see' the default arguments for foo() if any exist. So a better way to think of it might be:
#always execute this code before calling...
def foo(a, b, c):
return "Foo called and I saw %d, %d, %d" % (a, b, c)
So something really basic might look like this.
def wrap(f):
... def _inner(*a, **k):
... new_a = (ar*2 for ar in a)
... new_k = {}
... for k, v in k.iteritems():
... new_k[k] = v*2
... return f(*new_a, **new_k)
... return _inner
...
>>> def foo(a=2, b=4, c=6):
... return "%d, %d, %d" % (a, b, c)
...
>>> foo()
'2, 4, 6'
>>> foo(1, 5, 7)
'1, 5, 7'
>>> foo = wrap(foo) #actually wrapping it here
>>> foo()
'2, 4, 6'
>>> foo(3, 5, 6)
'6, 10, 12'
>>> foo(3, 5, c=7)
'6, 10, 14'
>>>

Decorators are function wrappers. They give back a function that wraps the original one into some pre- and post-processing code, but still need to call the original function (normally with the same argument as you would call it in absence of a decorator).
Python has two types of arguments, positional and keyword arguments (this has nothing to do with decorators, that's generic python basics). * is for positional (internally is a list), ** for keyword (dictionary) arguments. By specifying both you allow your decorator to accept all at all possible types of arguments and pass them through to the underlying function. The contract of the call is, however, still defined by your function. E.g. if it only takes keyword arguments it will fail when the decorator function passes through a positional argument.
In your particular example, you have some pre-processing code (i.e. code that will run before the original function is called). For example in this code you can print out arguments *args that your original function might fail to accept all together because it does not take any position arguments.
You do not necessarily have to pass through *args and **kwargs. In fact you can define a decorator which makes some decisions based on the arguments you pass in about what to provide to the original function as arguments, e.g.
def dec(fun):
def wrap(*args, **kwargs):
if 'a' in kwargs:
return fun(kwargs[a])
else:
return fun(*args)
return wrap