I'm refreshing my memory about some python features that I didn't get yet, I'm learning from this python tutorial and there's an example that I don't fully understand. It's about a decorator counting calls to a function, here's the code:
def call_counter(func):
def helper(x):
helper.calls += 1
return func(x)
helper.calls = 0
return helper
#call_counter
def succ(x):
return x + 1
if __name__ == '__main__':
print(succ.calls)
for i in range(10):
print(succ(i))
print(succ.calls)
What I don't get here is why do we increment the calls of the function wrapper (helper.calls += 1) instead of the function calls itself, and why does it actually working?
The important thing to remember about decorators is that a decorator is a function that takes a function as an argument, and returns yet another function. The returned value - yet another function - is what will be called when the name of the original function is invoked.
This model can be very simple:
def my_decorator(fn):
print("Decorator was called")
return fn
In this case, the returned function is the same as the passed-in function. But that's usually not what you do. Usually, you return either a completely different function, or you return a function that somehow chains or wraps the original function.
In your example, which is a very common model, you have an inner function that is returned:
def helper(x):
helper.calls += 1
return func(x)
This inner function calls the original function (return func(x)) but it also increments the calls counter.
This inner function is being inserted as a "replacement" for whatever function is being decorated. So when your module foo.succ() function is looked up, the result is a reference to the inner helper function returned by the decorator. That function increments the call counter and then calls the originally-defined succ function.
When you decorate a function you "substitute" you're function with the wrapper.
In this example, after the decoration, when you call succ you are actually calling helper. So if you are counting calls you have to increase the helper calls.
You can check that once you decorate a function the name is binded tho the wrapper by checking the attribute _name_ of the decorated function:
def call_counter(func):
def helper(*args, **kwargs):
helper.calls += 1
print(helper.calls)
return func(*args, **kwargs)
helper.calls = 0
return helper
#call_counter
def succ(x):
return x + 1
succ(0)
>>> 1
succ(1)
>>> 2
print(succ.__name__)
>>> 'helper'
print(succ.calls)
>>> 2
Example with Class Decorator
When you decorate a function with the Class Decorator, every function has its own call_count. This is simplicity of OOP. Every time CallCountDecorator object is called, it will increase its own call_count attribute and print it.
class CallCountDecorator:
"""
A decorator that will count and print how many times the decorated function was called
"""
def __init__(self, inline_func):
self.call_count = 0
self.inline_func = inline_func
def __call__(self, *args, **kwargs):
self.call_count += 1
self._print_call_count()
return self.inline_func(*args, **kwargs)
def _print_call_count(self):
print(f"The {self.inline_func.__name__} called {self.call_count} times")
#CallCountDecorator
def function():
pass
#CallCountDecorator
def function2(a, b):
pass
if __name__ == "__main__":
function()
function2(1, b=2)
function()
function2(a=2, b=3)
function2(0, 1)
# OUTPUT
# --------------
# The function called 1 times
# The function2 called 1 times
# The function called 2 times
# The function2 called 2 times
# The function2 called 3 times
What I don't get here is why do we increment the calls of the function wrapper (helper.calls += 1) instead of the function calls itself, and why does it actually working?
I think to make it a generically useful decorator. You could do this
def succ(x):
succ.calls += 1
return x + 1
if __name__ == '__main__':
succ.calls = 0
print(succ.calls)
for i in range(10):
print(succ(i))
print(succ.calls)
which works just fine, but you would need to put the .calls +=1 in every function you wanted to apply this too, and initialise to 0 before you ran any of them. If you had a whole bunch of functions you wanted to count this is definitely nicer. Plus it initialises them to 0 at definition, which is nice.
As i understand it it works because it replaces the function succ with the helper function from within the decorator (which is redefined every time it decorates a function) so succ = helper and succ.calls = helper.calls. (although of course the name helper is only definied within the namespace of the decorator)
Does that make sense?
As I understand this (correct me if I'm wrong) the order you program executes is:
Register call_function.
Register succ.
While registering succ function interpreter finds a decorator so it executes call_function.
Your function returns an object which is a function (helper). And adds to this object field calls.
Now your function succ has been assigned to helper. So when you call your function, you're actually calling helper function, wrapped within a decorator. So every field you add to your helper function is accessible outside by addressing succ because those 2 variables refer to same thing.
So when you call succ() it's basically the same if you would do helper(*args, **argv)
Check this out:
def helper(x):
helper.calls += 1
return 2
helper.calls = 0
def call_counter(func):
return helper
#call_counter
def succ(x):
return x + 1
if __name__ == '__main__':
print(succ == helper) # prints true.
Related
I'm learning python and I'm stuck at decorators
the way I understand it is that decorators add functionality to a function
I made a simple function that checks if a number is even
and then a decorator that adds taking the absolute value to it
def decorate(func):
def is_even_new(*args,**kwargs):
num = abs(*args,**kwargs)
func(num)
return is_even_new()
#decorate
def is_even(x):
if x%2 == 0:
return True
else:
return False
is_even(8)
but it I keep getting a TypeError: abs() takes exactly one argument (0 given)
is there is something wrong in the code or is my understanding of decorators is false?
When you decorate the function, you should return the decorated function itself:
def decorate(func):
def is_even_new(*args,**kwargs):
num = abs(*args,**kwargs)
func(num)
return is_even_new
rather than calling the decorated function and returning that:
def decorate(func):
def is_even_new(*args,**kwargs):
num = abs(*args,**kwargs)
func(num)
return is_even_new()
(notice the extra parentheses). In this last example, when you decorate a function with #decorate, the decorator defines this inner function is_even_new and then, instead of returning the latter, it tries to call it with no arguments: is_even_new(). This is also why you're getting the TypeError: abs expects (exactly) one argument, but you've given it none.
Remember that functions in Python are objects like anything else ("first-class citizens") and so you can reference them directly by their name.
Also, as a recommendation, if you know that the functions decorated with #decorate will only ever take exactly one argument (like is_even), don't use variable arguments and keyword arguments, just define the decorate function to take exactly one parameter as well:
import functools
def decorate(func):
#functools.wraps(func)
def decorated(x):
func(abs(x))
return decorated
This will make the error messages more helpful (instead of raising at the abs call,
Or, if you only want to apply abs to, say, the first argument:
def decorate(func):
#functools.wraps(func)
def decorated(x, *args, **kwargs):
func(abs(x), *args, **kwargs)
return decorated
You'll also have noticed the #functools.wraps. This is a very useful standard library utility for defining wrapper functions, such as those you return from a decorator. It sets special attributes like __name__, __module__, etc. on the wrapper function so that it essentially looks like the wrapped function.
in your code you need to return the value from the decorator in correct way and also change the input data to the decorator by keeping the limitation of other inside function in mind ie (abs take 1 positonal argument not many) and you cant directly call the return function from outside as full without provideing value as return is_new_even(), you need to pass the argument there also if want to do that way
def decorate(func):
def is_even_new(val):
num = abs(val)
return func(num)
return is_even_new
#decorate
def is_even(x):
if x%2 == 0:
return True
else:
return False
is_even(8)
I am not sure what your decorated function is trying to do as modulo is working on negative integers as well, but you basically have two problems:
like #Anakhand said you should return a function and not the function return value.
your is_even_new does not return nothing
so I guess this is the correct implemntation:
def decorate(func):
def is_even_new(*args,**kwargs):
num = abs(*args,**kwargs)
return func(num)
return is_even_new
#decorate
def is_even(x):
if x%2 == 0:
return True
else:
return False
print(is_even(-8)) # --> True
I am reading the following tutorial about Python decorators tutorial. Everything is more or less clear except the following code:
def call_counter(func):
def helper(x):
helper.calls += 1
return func(x)
helper.calls = 0
return helper
#call_counter
def succ(x):
return x + 1
print(succ.calls)
for i in range(10):
succ(i)
print(succ.calls)
I cannot fully understand helper.calls notation. Is is just variable with no relations to helper function? Also how does succ function gets access to calls?
In Python functions are objects, that means you can set variables too.
def func():
pass
func.count = 0
print(func.count) # 0
func.count += 1
print(func.count) # 1
We can rewrite the decoration to this:
def succ(x):
return x + 1
succ = call_counter(succ)
So now you have a decorated succ. As you can see in call_counter, it actually returns a function called helper. And this helper function has one attribute named calls which is used to count calls. So now when you call succ(i), you are actually calling that helper function.
And yes that calls is just a normal variable.
a function (funcA) that takes a function (funcB) and returns a function that can be evaluated to produce the same value that funcB would produce (if given the same arguments) plus 1
returned_func(x) = funcB(x) + 1
What could be the possible way of doing this? I am confused with the second part of the question as to how can a function return a value and function at the same time. Any code example would be much appreciated! Thanks!
The concept you're looking for is, essentially, a function decorator. In Python, functions are first-class objects, just like class instances are in other languages. That means that you can pass them around, just like any other object. For example:
def foo(x):
return x + 1
print(foo) # <function foo at 0x0000024202F43EA0>
bar = foo
print(bar) # <function foo at 0x0000024202F43EA0>
Here we created a reference to the function object itself. We can, using the same idea, make a function which returns a function:
def foo(x):
def inner(y):
return x + y
return inner
func = foo(5)
print(func(3)) # 8
func is assigned to the return value of foo, which is itself a function that we can evaluate. You want to return a function A that adds 1 to the result of another function, B. So, pass B to a function make_A:
def make_A(b):
def inner(x):
return b(x) + 1
return inner
def B(x):
return x
A = make_A(B)
print(A(1)) # 2
Python has excellent syntactic sugar for this type of function. See this primer for more information.
def func1(x):
#operations go here:
return(x + 1)
def func2():
#operations go here:
return(x - 1)
if func1(2) == func2(4):
print("Hooray")
Does this help? It is a demonstration of how parameters and return values can interact.
A segment like func(1) implies the return value with the parameter as 1.
Example:
Fibonaci recursive function with memoize decorator. When calling function helper there is no argument. If function helper is defined to take an argument x therefore i expect calling the function with one argument. I would like to understand why is syntax like it is?
def memoize(f):
memo = {}
def helper(x):
if x not in memo:
memo[x] = f(x)
return memo[x]
return helper
#memoize
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
print(fib(40))
You do call the helper with the argument. Decorators are syntactic sugar for this
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
fib = memoize(fib)
So your fib function is no longer the original fib-function. It is in fact the helper closure, because that's what memoize returns – a closure. So, when you call fib(40) you kind of call helper(40). The memoize decorator creates a function object, it doesn't call it, just returns it.
I believe the syntax is like this so that it looks as streamlined as possible. You put a decorator object after the # (a function that takes one argument, and returns a function), and python calls it with the function you are defining. This
#memoize
def fib(n):
...
Is exactly equivalent to the following, which doesn't use the magic decorator syntax:
def fib(n):
...
fib = memoize(fib)
If you want your head to spin a little, consider that # can actually be followed by a function call -- but this function call must return a decorator that works as above! Here's a silly example that counts how many times the decorated function is called, but lets you set the start value. (It's just an example: it's not very useful since only one function can be decorated, etc.)
def countcalls(start):
global _calls
_calls = start
def decorator(f):
def wrapper(x):
global _calls
_calls += 1
return f(x)
return wrapper
return decorator
#countcalls(3)
def say(s):
print(s)
say("hello")
# _calls is now 4
Here, countcalls(4) defines and returns (without calling it) the function decorator, which will wrap the decorated function and return the wrapper in place of the function I wrote.
i have homework and we need to do something like iterator, the func work great but the techer told he run the func with (t=Make_iterator()) like this, what i do wrong? tnx!
global x
x=-1
def Make_iterator(fn):
global x
x+=1
return fn(x)
fn=lambda y:y*2
t=Make_iterator(fn)
print(t())
I think you want a closure, which is a function defined within the local namespace of anther function, so that it can access the outer function's variables:
def make_iterator(func):
x = -1
def helper():
nonlocal x
x += 1
return func(x)
return helper
The nonlocal statement allows the inner function to modify the variable declared in the outer function (otherwise you'd either get an error, or you'd bind your own local variable without changing the outer one). It was only added in Python 3, so if you're still using Python 2, you'll need to wrap the x value in a mutable data structure, like a list.
Another approach to the same idea is to write class, rather than a function. An instance of a class can be callable (just like a function) if the class defines a __call__ method:
class MyIterator(object):
def __init__(self, func):
self.index = -1
self.func = func
def __call__(self):
self.index += 1
return self.func(self.index)
This can be useful if the state you need to keep track of is more complicated (or should change in more complicated ways) than the simple integer index used in this example. It also works in Python 2 without annoying workarounds.
I think he wants your Make_iterator function to return a function that acts as an iterator. So you could wrap the contents of your current Make_iterator function within an inner function f and return that:
def Make_iterator(fn):
def f():
global x
x+=1
return fn(x)
return f
Now if you do t = Make_iterator(fn), every time you call t() it will return the next value of the iterator, in your case 0, 2, 4, 6, 8, etc...