I'm having trouble understanding the concept of decorators, so basically if I understood correctly, decorators are used to extend the behavior of a function, without modifying the functions code . The basic example:
I have the decorator function, which take as parameter another function and then changes the functionality of the function given as argument:
def decorator(f):
def wrapper(*args):
return "Hello " + str(f(*args))
return wrapper
And here I have the function which I want to decorate :
#decorator
def text (txt):
'''function that returns the txt argument'''
return txt
So if I understand correctly , what actually happens "behind" is:
d=decorator(text)
d('some argument')
My question is , what happens in this case when we have three nested function in the decorator:
def my_function(argument):
def decorator(f):
def wrapper(*args):
return "Hello " +str(argument)+ str(f(*args))
return wrapper
return decorator
#my_function("Name ")
def text(txt):
return txt
Because the functionality is awesome, I can pass argument in the decorator, I do not understand what actually happens behind this call :
#my_function("Name ")
Thank you,
It is just another level of indirection, basically the code is equivalent to:
decorator = my_function("Name ")
decorated = decorator(text)
text = decorated
Without arguments, you already have the decorator, so
decorated = my_function(text)
text = decorated
my_function is used to create a closure here. The argument is local to my_function. But due to closure, when you return the decorator, the decorator function has a reference to it all the time. So when you apply decorator to text, the decorator adds extra functionality, as to be expected. It also can embed argument in its extra functionality since it has access to the environment in which it was defined.
Related
DESCRIPTION
Implement a Python decorator that should take whatever the decorated function returns, and write it to a file in a new line. For the sake of this problem, let us assume that the decorated functions always return a string. The decorator should be named log_message and should write to the file /tmp/decorator_logs.txt.
Implement the following design
#log_message
def a_function_that_returns_a_string():
return "A string"
#log_message
def a_function_that_returns_a_string_with_newline(s):
return "{}\n".format(s)
#log_message
def a_function_that_returns_another_string(string=""):
return "Another string"
Here is the decorator:
def log_message(func):
def wrap(*args,**kwargs):
res = func(*args,**kwargs)
with open('/tmp/decorator_logs.txt','wt') as f:
f.write(res)
return res
return wrap
But from your description of what you did in your comments, it seems that you don't fully understand the decorator concept.
You can think of decorator as a special function that takes another function as input and return a decorated function that do something slightly different.
In the code above, log_message take what ever function it decorates, and define a new function called wrap. This wrap function take whatever inputs, pass those input arguments to func, write the returned result of func to the tmp/decorator_logs.txt file, then return the same result.
Another thing you need to understand is that
#log_message
def decorated_function():
....
is the same as:
decorated_function = log_message(decorated_function)
Hope this help with your understanding of decorator.
I am new to the more advanced features of Python like decorators.
I am unable to understand how the Python interpreter actually understands where to put the original function object in a decorator.
Lets look at an example: Examples taken from here.
Simple decorator with no arguments:
def call_counter(func):
def helper(*args, **kwargs):
helper.calls += 1
return func(*args, **kwargs)
helper.calls = 0
return helper
#call_counter
def succ(x):
return x + 1
This makes perfect sense if we can assume that the first/only argument to the decorator call_counter(func) is the function object that needs to wrapped ie. in this case succ() function.
But things become inconsistent when you are talking about "decorators with parameters". Look at the example below:
Decorator with one argument:
def greeting(expr): # Shouldn't expr be the function here ? Or at least isn't there suppose to be another parameter.
def greeting_decorator(func): # How does Python know to pass the function down here ?
def function_wrapper(x):
print(expr + ", " + func.__name__ + " returns:")
func(x)
return function_wrapper
return greeting_decorator
#greeting("Hello")
def foo(x):
print(42)
foo("Hi")
Now we know Python has no concept of data-types, so function parameters give no information about what type of object they will contain.
Am I correct ?
Having said that lets look at the line from the above example:
def greeting(expr):
If for decorators the first argument is the function to be wrapped then by that logic expr should point to foo() right ? Otherwise there should be at least two parameters in greeting(), like:
def greeting(func, expr):
But instead Python can "magically" understand that the inner function needs to be passed the function reference:
def greeting(expr):
def greeting_decorator(func): # How is it correctly put one level down ?
The code has no datatypes or type information specified, so how is it that for decorators without arguments the function is passed as the first argument and for decorators with arguments the function is passed to the inner function ?
How can the interpreter detect that ?
What is going on here ?
This seems like "magic" to me.
What happens if I have 5 or 6 levels of nested functions ?
I am pretty sure I am missing something pretty basic here.
Thanks.
Python evaluates the expression after the # and uses the result as the decorator function.
Python calls the __call__ method of the object that is the decorator with the function as argument.
using
#call_counter
def succ(x):
return x + 1
callcounter is the object looked for __call__ to give the argument func
If you use
#greeting("Hello")
def foo(x):
print(42)
greeting("Hello") is evaluated and its result is an object that Python uses the __call__ method with the func argument.
I can't figure out how to do this, and frankly, I don't know if it's possible.
I want to write a decorator that changes the way a function is called. It's easiest to see with example code:
def my_print(*args, **kwargs):
print(args[0].upper())
#reroute_decorator('print', my_print)
def my_func():
print('normally this print function is just a print function...')
print('but since my_func is decorated with a special reroute_decorator...')
print('it is replaced with a different function, and its args sent there.')
my_func()
# NORMALLY THIS PRINT FUNCTION IS JUST A PRINT FUNCTION...
# BUT SINCE MY_FUNC IS DECORATED WITH A SPECIAL REROUTE_DECORATOR...
# IT IS REPLACED WITH A DIFFERENT FUNCTION, AND ITS ARGS SENT THERE.
Is a decorator with this kind of functionality even possible in python?
Now, I don't really need this if it's too complex, I just can't figure out how to do it in a simple way.
Is this kind of a problem trivial? Or is it really complex?
You can create a new function with an updated globals dictionary so that to that function it appears that the global was bound to the desired value.
Note that this is weaker than actual dynamic scope as any functions called by the function will see the original bindings and not the modified one.
See namespaced_function referenced in How does Python's types.FunctionType create dynamic Functions?
To elaborate on #Dan D.'s answer, you would create a new function object to replace the original, something like this:
from types import FunctionType
def reroute_decorator(**kwargs):
def actual_decorator(func):
globals = func.__globals__.copy()
globals.update(kwargs)
new_func = FunctionType(
func.__code__, globals, name=func.__name__,
argdefs=func.__defaults__, closure=func.__closure__)
new_func.__dict__.update(func.__dict__)
return new_func
return actual_decorator
The only catch here is that the updated function object is the only one that will see whatever kwargs you passed in, since they will be spoofed into globals. Additionally, any modifications you make to the module after calling the decorator function will not be visible to the decorated function, but that should not be an issue. You can go a layer deeper and create a proxy dictionary that would allow you to interact normally with the original, except for keys you explicitly defined, like print, but that's a bit out of scope here.
I've updated your print implementation to be a bit more general, and made the input to the decorator function more pythonic (less MATLABy):
def my_print(*args, **kwargs):
print(*(str(x).upper() for x in args), **kwargs)
#reroute_decorator(print=my_print)
def my_func():
print('normally this print function is just a print function...')
print('but since my_func is decorated with a special reroute_decorator...')
print('it is replaced with a different function, and its args sent there.')
Which results in:
>>> my_func()
NORMALLY THIS PRINT FUNCTION IS JUST A PRINT FUNCTION...
BUT SINCE MY_FUNC IS DECORATED WITH A SPECIAL REROUTE_DECORATOR...
IT IS REPLACED WITH A DIFFERENT FUNCTION, AND ITS ARGS SENT THERE.
I was working through a decorator design pattern tutorial
(credit to Jungwoo Ryoo)
I'm curious as to why I can swap the lines: return decorator
and print(hello_world()) with return decorator() and print(hello_world)
from functools import wraps
def make_blink(function):
"""Defines the decorator"""
#wraps(function)
# Define the inner function
def decorator():
# Grab the return value of the function being decorated
ret = function()
# Add new functionality to the function being decorated
return "<blink>"+ ret + "<b/link>"
return decorator #return decorator()#<THIS LINE HERE SWAPPED
# Apply the decorator here!
#make_blink
def hello_world():
"""Original function! """
return "Hello, World!"
# Check the result of decorating
print(hello_world()) #print(hello_world) #<THIS LINE HERE SWAPPED
Wouldn't the interpreter be doing something different each time? I'm just looking for some insight to have a better understanding of what's going on
Decorators are just functions really, and functions are just objects.
The lines
#make_blink
def hello_world():
# ...
are essentially the same as
def hello_world():
# ...
hello_world = make_blink(hello_world)
except the function object is never assigned to hello_world first (it's on the stack for to be passed to the decorator).
So whatever you return from make_blink() is assigned back to hello_world. That can be a function object, but it can also be something entirely different.
So when you use return decorator, you tell Python to set hello_world to the nested function object. When you use return decorator(), you tell Python to use the result of the decorator() function. Here, that's a string value. It's as if you did this:
def hello_world():
"""Original function! """
return "Hello, World!"
hello_world = "<blink>" + hello_world() + "</blink>"
And that is fine for this specific example, because body of the hello_world() function only ever returns the same string each time.
But what if you changed the original hello_world() function body to return something different each time you called it? What if you had:
import random
#make_blink
def random_greeting():
return 'Hello ' + random.choice('DonAr', 'Martijn Pieters', 'Guido van Rossum') + '!'
Now it makes a big difference what you return from the make_blink() call! For the top-level of a module, decorators are executed only once, when importing. If you used return decorator() you'd run random.choice() just once, and you have fixed the value of random_greeting to a single, static string result.
Generally speaking, decorators are expected to return a callable object again. That can be the original function (where the decorator just updates some kind of registration), a wrapper function (which does extra things before or after calling the original), or even something different entirely. But that's not set in stone anywhere, and the interpreter doesn't care either way.
Decorators are just reusable things to use in your program, a tool. If you have a specific use for a decorator that returns the result of the original function, then you are free to do so.
So I'm still kind of new to Python decorators - I've used them before, but I've never made my own. I'm reading this tutorial (that particular paragraph) and I don't seem to understand why do we need three levels of functions? Why can't we do something like this:
def decorator(func, *args, **kwargs):
return func(*args,**kwargs)
Thanks :)
Well, what would happen if you called that decorator on a function?
#decorator
def foo(): pass
This code would immediately call foo, which we don't want. Decorators are called and their return value replaces the function. It's the same as saying
def foo(): pass
foo = decorator(foo)
So if we have a decorator that calls foo, we probably want to have a function that returns a function that calls foo -- that function that it returns will replace foo.
def decorator(f):
def g(*args, **kwargs):
return f(*args, **kwargs)
return g
Now, if we want to pass options to the decorator, we can't exactly pass them ins ide-by-side with the function like in your example. There's no syntax for it. So we define a function that returns a parameterized decorator. The decorator it returns will be a closure.
def argument_decorator(will_I_call_f):
def decorator(f):
def g(*args, **kwargs):
if will_I_call_f: return f(*args, **kwargs)
return g
return decorator
so we can do
decorator = argument_decorator(True)
#decorator
def foo(): pass
And Python offers the convenience syntax where you inline the function call:
#argument_decorator(True)
def foo(): pass
And all this is syntax sugar for the non-decorator syntax of
def foo(): pass
foo = argument_decorator(True)(foo)
A decorator modifies a function by adding a wrapper to it. At the time you decorate the function, it isn't being called yet, so you don't have any arguments (or keyword arguments) to look at. All you can do for now is create a new function that will handle those arguments when it finally gets them.