I have a class. This class has a list of functions that are to be evaluated by a different program.
class SomeClass(object):
def __init__(self, context):
self.functions_to_evaluate = []
There is a function that adds functions to an instance of SomeClass, via something like:
new_function = check_number(5)
SomeClassInstance.functions_to_evaluate.append(new_function)
Where check_number is a function that will check if number is greater than 10, let's say.
If I take SomeClassInstance.functions_to_evaluate and print it, I get a bunch of python objects, like so:
<some_library.check_number object at 0x07B35B90>
I am wondering if it is possible for me to extract the input given to check_number, so something like:
SomeClassInstance.functions_to_evaluate[0].python_feature() that will return "5" or whatever the input to check_number was to me.
You can use the standard library functools.partial, which creates a new partially applied function *.
>>> from functools import partial
>>> def check_number(input):
... return input > 10
>>> fn = partial(check_number, 5)
>>> fn.args # this attribute gives you back the bound arguments, as a tuple.
(5,)
>>> fn() # calls the function with the bound arguments.
False
*: actually the partial object is not a function instance, but it is a callable, and from a duck-type perspective it's a function.
If new_function = check_number(5) is a closure, then you can extract this value using __closure__[0].cell_contents:
Example:
def foo(x):
def inn(y):
return x
return inn
s = foo(5)
print(s.__closure__[0].cell_contents)
Output:
5
I understand your confusion, but:
new_function = check_number(5)
Is calling the function, and the new_function variable gets assigned the return value of the function.
If you have this check_number function:
def check_number(input):
return input > 10
Then it will return False, and new_function will be False. Never <some_library.check_number object at 0x07B35B90>.
If you're getting <some_library.check_number object at 0x07B35B90> then your check_number() function is returning something else.
There are probably several ways to skin this cat. But I'd observe first and foremost that you're not adding python function objects to the functions_to_evaluate list, you're adding the evaluations of functions.
You could simply add a tuple of function, args to the list:
SomeClassInstace.functions_to_evaluate.append((check_number, 5))
And then you can:
for f, args in SomeClassInstance.functions_to_evaluate:
print(args)
Related
I need to write a python function called 'concat' that takes any two functions as input, and returns a function, which is the concatenated function of the two input functions (i.e. it takes f1 and f2, and returns f1◦f2).
I tried this:
def concat(f1,f2):
return f1(f2)
So for example, if f1 and f2 are:
def f1(x):
return x+2
def f2(x):
return x*2
then, concat(f1,f2) should return: (x*2)+2
I want to be able to use it like this:
a = concat(f1,f2)
a(5)
But I get an error:
TypeError: unsupported operand type(s) for +: 'function' and 'int'
I know I can define the function like this:
def concat(f1,f2,x):
return f1(f2(x))
But that is not what I want; I want to be able to create instances of the concat function, which then can be called with any x.
You need to return a new "wrapper" function. One option is to use a lambda expression:
def concat(f1, f2):
return lambda x: f1(f2(x))
DOCS: https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions
I think what you want is a closure.
def concat(f1, f2):
def f3(x):
return f1(f2(x))
return f3
Functions in Python are considered first class objects. This allows them to be created and manipulated like normal variables. In this case concat is constructing a new function that composes two functions.
The second property being used here is lexical scoping. The new function retains access to the local variables where it was defined not where it is executed. This allows the returned function to be run anywhere without losing access to the composite functions.
Here's a possible solution to compose more than two functions. Using a closure to loop through the functions and use the previous function's result as argument for the next one:
def compose(*fns):
def F(x):
for fn in fns[::-1]:
x = fn(x)
return x
return F
Then define your composition and call:
>>> F = compose(f1, f2, lambda x: x+1)
>>> F(4)
12 # f1(f2(f3(x)))
I want to write a wrapper function which call one function and pass the results to another function. The arguments and return types of the functions are the same, but I have problem with returning lists and multiple values.
def foo():
return 1,2
def bar():
return (1,2)
def foo2(a,b):
print(a,b)
def bar2(p):
a,b=p
print(a,b)
def wrapper(func,func2):
a=func()
func2(a)
wrapper(bar,bar2)
wrapper(foo,foo2)
I am searching for a syntax which works with both function pairs to use it in my wrapper code.
EDIT: The definitions of at least foo2 and bar2 should stay this way. Assume that they are from an external library.
There is no distinction. return 1,2 returns a tuple. Parentheses do not define a tuple; the comma does. foo and bar are identical.
As I overlooked until JacobIRR's comment, your problem is that you need to pass an actual tuple, not the unpacked values from a tuple, to bar2:
a = foo()
foo2(*a)
a = bar()
bar2(a)
I don't necessarily agree with the design, but following your requirements in the comments (the function definitions can't change), you can write a wrapper that tries to execute each version (packed vs. unpacked) since it sounds like you might not know what the function expects. The wrapper written below, argfixer, does exactly that.
def argfixer(func):
def wrapper(arg):
try:
return func(arg)
except TypeError:
return func(*arg)
return wrapper
def foo():
return 1,2
def bar():
return (1,2)
#argfixer
def foo2(a,b):
print(a,b)
#argfixer
def bar2(p):
a,b=p
print(a,b)
a = foo()
b = bar()
foo2(a)
foo2(b)
bar2(a)
bar2(b)
However, if you aren't able to put the #argfixer on the line before the function definitions, you could alternatively wrap them like this in your own script before calling them:
foo2 = argfixer(foo2)
bar2 = argfixer(bar2)
And as mentioned in previous comments/answers, return 1,2 and return (1,2) are equivalent and both return a single tuple.
This code does not run because of arg differences. It runs if you use def foo2(*args): and def bar2(*p):.
The return 1, 2 and return (1, 2) are equivalent. The comma operator just creates a tuple, whether it is enclosed in parentheses or not.
All programming languages that I know of return a single value, so, since you want to return multiple, those values must be wrapped into a collection type, in this case, a tuple.
The problem is in the way you call the second function. Make it bar2(a) instead of bar2(*a), which breaks the tuple into separate arguments.
I have a function - func_main - which has several inputs. One of the inputs is another function - func_mini. The smaller function, func_mini, loads some data, and it needs to be able to load data from different sources depending on the arguments passed to it. The issue is that I am reading the documentation on func_main and it says that it only accepts parameterless functions. How can I get around this?
For example:
def func_main(evaluator = None):
num_list = [1,2,3]
return evaluator(num_list)
def func_mini(data_source = None):
if not data_source:
data_source = config.DATA_SOURCE
return pandas.read_csv(data_source).min
I want do to this:
func_main(func_mini(data_souce='path/to/my/file'))
But func_main documentation says:
"evaluator: A parameterless function that defines and return all data needed in the above model definition."
You can use functools.partial to build a function without parameters from your original func_mini function. Demo:
>>> from functools import partial
>>> def foo(x):
... print(x)
...
>>> foo_argless = partial(foo, 'hi')
>>> foo_argless()
hi
I have a base decorator that takes arguments but that also is built upon by other decorators. I can't seem to figure where to put the functools.wraps in order to preserve the full signature of the decorated function.
import inspect
from functools import wraps
# Base decorator
def _process_arguments(func, *indices):
""" Apply the pre-processing function to each selected parameter """
#wraps(func)
def wrap(f):
#wraps(f)
def wrapped_f(*args):
params = inspect.getargspec(f)[0]
args_out = list()
for ind, arg in enumerate(args):
if ind in indices:
args_out.append(func(arg))
else:
args_out.append(arg)
return f(*args_out)
return wrapped_f
return wrap
# Function that will be used to process each parameter
def double(x):
return x * 2
# Decorator called by end user
def double_selected(*args):
return _process_arguments(double, *args)
# End-user's function
#double_selected(2, 0)
def say_hello(a1, a2, a3):
""" doc string for say_hello """
print('{} {} {}'.format(a1, a2, a3))
say_hello('say', 'hello', 'arguments')
The result of this code should be and is:
saysay hello argumentsarguments
However, running help on say_hello gives me:
say_hello(*args, **kwargs)
doc string for say_hello
Everything is preserved except the parameter names.
It seems like I just need to add another #wraps() somewhere, but where?
I experimented with this:
>>> from functools import wraps
>>> def x(): print(1)
...
>>> #wraps(x)
... def xyz(a,b,c): return x
>>> xyz.__name__
'x'
>>> help(xyz)
Help on function x in module __main__:
x(a, b, c)
AFAIK, this has nothing to do with wraps itself, but an issue related to help. Indeed, because help inspects your objects to provide the information, including __doc__ and other attributes, this is why you get this behavior, although your wrapped function has different argument list. Though, wraps doesn't update that automatically (the argument list) what it really updates is this tuple and the __dict__ which is technically the objects namespace:
WRAPPER_ASSIGNMENTS = ('__module__', '__name__', '__qualname__', '__doc__',
'__annotations__')
WRAPPER_UPDATES = ('__dict__',)
If you aren't sure about how wraps work, probably it'll help if your read the the source code from the standard library: functools.py.
It seems like I just need to add another #wraps() somewhere, but where?
No, you don't need to add another wraps in your code, help as I stated above works that way by inspecting your objects. The function's arguments are associated with code objects (__code__) because your function's arguments are stored/represented in that object, wraps has no way to update the argument of the wrapper to be like the wrapped function (continuing with the above example):
>>> xyz.__code__.co_varnames
>>> xyz.__code__.co_varnames = x.__code__.co_varnames
AttributeError: readonly attribute
If help displayed that function xyz has this argument list () instead of (a, b, c) then this is clearly wrong! And the same applies for wraps, to change the argument list of the wrapper to the wrapped, would be cumbersome! So this should not be a concern at all.
>>> #wraps(x, ("__code__",))
... def xyz(a,b,c): pass
...
>>> help(xyz)
Help on function xyz in module __main__:
xyz()
But xyz() returns x():
>>> xyz()
1
For other references take a look at this question or the Python Documentation
What does functools.wraps do?
direprobs was correct in that no amount of functools wraps would get me there. bravosierra99 pointed me to somewhat related examples. However, I couldn't find a single example of signature preservation on nested decorators in which the outer decorator takes arguments.
The comments on Bruce Eckel's post on decorators with arguments gave me the biggest hints in achieving my desired result.
The key was in removing the middle function from within my _process_arguments function and placing its parameter in the next, nested function. It kind of makes sense to me now...but it works:
import inspect
from decorator import decorator
# Base decorator
def _process_arguments(func, *indices):
""" Apply the pre-processing function to each selected parameter """
#decorator
def wrapped_f(f, *args):
params = inspect.getargspec(f)[0]
args_out = list()
for ind, arg in enumerate(args):
if ind in indices:
args_out.append(func(arg))
else:
args_out.append(arg)
return f(*args_out)
return wrapped_f
# Function that will be used to process each parameter
def double(x):
return x * 2
# Decorator called by end user
def double_selected(*args):
return _process_arguments(double, *args)
# End-user's function
#double_selected(2, 0)
def say_hello(a1, a2,a3):
""" doc string for say_hello """
print('{} {} {}'.format(a1, a2, a3))
say_hello('say', 'hello', 'arguments')
print(help(say_hello))
And the result:
saysay hello argumentsarguments
Help on function say_hello in module __main__:
say_hello(a1, a2, a3)
doc string for say_hello
def apply_twice(func,arg):
return func(func(arg))
def add_five(x):
return x+5
print (apply_twice(add_five,10))
The output I get is 20.
This one is actually confusing me like how is it working.Can anybody explain me how this is working by breaking it down
The function apply_twice(func,arg) takes two arguments, a function object func and an argument to pass to the function func called arg.
In Python, functions can easily be passed around to other functions as arguments, they are not treated differently than any other argument type (i.e first class citizens).
Inside apply_twice, func is called twice in the line:
func(func(arg))
Which, alternatively, can be viewed in a more friendly way as:
res = func(arg)
func(res)
If you replace func with the name of the function passed in add_five you get the following:
res = add_five(arg) # equals: 15
add_five(res) # result: 20
which, of course, returns your expected result.
The key point to remember from this is that you shouldn't think of functions in Python as some special construct, functions are objects just like ints, listss and everything else is.
Expanding the code it executes as follows, starting with the print call:
apply_twice(add_five,10))
add_five(add_five(10)) # add_five(10) = 15
add_five(15) # add_five(15) = 20
Which gives you the result: 20.
When apply_twice is called, you are passing in a function object and a value. As you can see in the apply_twice definition, where you see func that is substituted with the function object passed to it (in this case, add_five). Then, starting with the inner func(arg) call, evaluate the result, which is then passed to add_five again, in the outer return func( ... ) call.
What you need to understand here is that
apply_twice(func,arg)
is a higher function which accepts two arguments (another function named func and an argument arg). The way it works is that it first evaluate the value of the other function, then use the value as an argument inside the higher function.
remember we have a function add_five(x) which add 5 to the argument supply in it...
then this function add_five(x) is then passed as an argument to another function called
apply_twice_(func,arg) which return func(func(arg)).
now splitting func(func(arg)) we have
func(arg) #lets called it a
then func(func(arg))==func(a) since a = func(agr)
and (a) is our add_five(x) function, after it add 5, then the value we got is re-used as another fresh argument to add another 5 to it, that is why we have 20 as our result.
Another example is:
def test(func, arg):
return func(func(arg))
def mult(x):
return x * x
print(test(mult, 2))
which give 16 as result.