Say for example I have a function
def A(): pass
I call this function from two different places
def B():
#some code
A()
#some code
def C():
#some code
A()
#some code
Decorators as we know, in layman language is something like a hook which lets you perform a set of operation before and after the function call. We apply decorators on function definition using #.
My requirement here is like I don't want the decorator to be applied every time the function is called. Like in the above example if I apply decorator on function A. It would be applied in both invokes. Can we use # at the invoke of the function A?
Something like,
def A(): pass
def B():
#some code
A()
#some code
def C():
#some code
#decorator
A()
#some code
This way I don't have to worry about merging my code after every release of particular opensource project.
A decorator is just syntactic sugar, where the outcome of the #expression is called and the return value is used to replace the decorated object.
If you only need to have a decorated function in one location you can do decorator(A)() to have it return a decorated version just to be used in C and call that decorated object on the spot.
Better yet, store that decorated version for C to use so you don't keep decorating it each time:
A_decorated = decorator(A)
def C():
# some code
A_decorated()
# some code
Related
My question starts with how decorators work in python. Let's look at the following code:
def decorator(F):
def wrapper():
print("start")
F()
print("end")
return wrapper
def f1():
print("f1")
decorated_f1 = decorator(f1)
decorated_f1()
It prints,
start
f1
end
First of all, as much as I know, python uses lazy evaluation. Therefore, it does not evaluate F(), until it is required (when actually decorated_f1() is called). By then, the scope of argument is F is over (end of the decorator function). I would like to know, what python stores in memory when a function is created and overall the memory management that happens for decorators.
The second part of my question is about the results that I get after running the following codes,
def decorator(F):
def wrapper():
print("start")
F()
print("end")
return wrapper
def f1():
print("f1")
decorated_f1 = decorator(f1)
def f1():
print("new f1")
decorated_new_f1 = decorator(f1)
decorated_f1()
decorated_new_f1()
it results in,
start
f1
end
start
new f1
end
However, the following code
def f1():
print("f1")
F = f1
def wrapper():
print("start")
F()
print("end")
wrapped_f1 = wrapper
def f1():
print("new f1")
F = f1
def wrapper():
print("start")
F()
print("end")
wrapped_new_f1 = wrapper
wrapped_f1()
wrapped_new_f1()
produces,
start
new f1
end
start
new f1
end
This makes me confused, because I thought these two codes should be very similar in output. That is why I need help for clarifying what and how things are stored in memory when functions or decorators are declared in python.
You are correct that the second and third example are quite close in how they work. However there is one key difference. The scope the functions are evaluated in. The scope in this case can be seen as all the variables some place in the code has access too, and where those come from. You might have heard of the global scope, this is the scope all code is evaluated in and all variables that are global are accessible by any part of the code. There are also function scopes. These are the variables defined inside of the function, and other functions won't have access to these variables.
To further understand the difference there is one more think you should know. Functions are a reference type in Python, this means that when you declare one all later uses reference the value rather then directly accessing the value. This is the same as for lists and dictionaries.
Why does this all matter? Well because a decorator stores the function scope to correctly evaluate later. This includes the current value of F in this function
def decorator(F):
def wrapper():
print("start")
F()
print("end")
return wrapper
However, if you don't pass F as an argument then the decorator will store the function as a reference to the global function. Thus when you chance F in the third example the F() in wrapped_f1 also changes. This explains the output you see.
Lastly, as a tip. There is a specific syntax for using decorators on functions, and it might make using them easier. Usually a decorator would look like this.
def decorator(F):
def wrapper():
print("start decorating")
F()
print("End decorating")
return wrapper
def undecorated_func():
print("Hellor world!")
#decorator
def decorated_func():
print("Hello World!")
undecorated_func()
decorated_func()
Then the output is
Hello world!
Start decorating
Hello World!
End decorating
I have a library proving the following code:
class LibClass:
def decorator_method(self, *args, **kwargs):
# some code...
pass
which is intended to be used as follows:
k = LibClass()
method_args = (,) # something I create
#k.decorator_method(method_args)
def my_func():
# Code that I want to run
Specifically, the library implements some callback-like functionality via these decorators. Naturally, I have no control over the library, nor is there any alternative functionality.
Importantly, the instance of LibClass is part of the decorator, so my_func is added to the callbacks of that very instance, k.
However, I have implemented my_func elsewhere in my code and I want to separate the function from the decoration. One way would be to create a wrapper like so:
k = LibClass()
#k.decorator_method(method_args)
def my_func_wrapper():
# Actual call
my_func()
This approach works the way I want, but it has the disadvantage of having to define and call the wrapper function which is not strictly necessary. However, I cannot apparently apply the decorator directly, since
def my_func():
# ...
#k.decorator_method(method_args)
my_func
is not valid python code. Similarly, I might think about decorating a closure in this way, like
def my_func_producer(more_args):
def my_func():
# Do something with more_args
return my_func
where
more_args = (,)
#k.decorator_method(method_args)
my_func_producer(more_args)
is also invalid. Is there any way to apply the decorator without having to define an additional wrapper function?
Instead of this:
def my_func():
# ...
#k.decorator_method(method_args)
my_func
do this:
def my_func():
# ...
my_func = k.decorator_method(method_args)(my_func)
This is doing exactly what the decorator does, but for a minor detail. The "#" syntax for decorators was introduced back in version 2.3 as a syntax sugar to just that: the decorator callable (placed after the #) is called with the function defined starting on the next line as its sole parameter.
So, if we just call the same decorator with the usual calling syntax, passing the function itself as parameter, the effect is the same. In this case we have to rebind the decorated object to the function name, hence the my_func = ... before the call. (the #syntax does an impplicit name binding and that is the "minor detail" I mentioned on the first paragraph)
Essentially, I need to keep track of the methods that I wrap with this decorator to use later by editing the original object. The code works if I call the method, but if I don't, the code in the wrapper never executes. The wrapper is the only place where I receive the object where I can modify it. So, I need some other way to modify the object without calling a method that I'm decorating in it.
I've been trying so many different ways but I just can't get this to work.
import functools
def decorator3(**kwargs):
print(1)
def decorator(function):
print(2)
#functools.wraps(function)
def wrapper(self, *args):
print(3)
self.__test__ = "test worked"
function(self, *args)
return wrapper
return decorator
class Test:
def __init__(self):
self.a = "test"
#decorator3()
def test(self):
print(self.a)
t = Test()
#t.test()
print(t.__test__)
Your code parts execute in different times.
The outer decorator function, where you print "1", is called where you read #decorator3(). The result of calling this function, decorator, is then immediately applied to the decorated function (resp. method), where you read the "2".
In your case, the inner decorator replaces the method with a different function which calls the original method. Only if you call this, you reach "3".
So:
at 1, you don't know anything about your function.
at 2, you know your function, but not in which class it lives in.
at 3, you know your object, but only if your function is called.
You have to decide what exactly you want to do: according to your description, you want to flag the functions. That's what you perfectly can do in "2".
You say
I need to keep track of the methods that I decorate later on in the code. Basically, I decorate it to "flag" the method, and I have multiple classes with different methods that I need to flag. I will be calling decorator3 in other classes, so I don't see how putting the decorator in init will help. By editing the original class, I can later on put the method in a dictionary which I was hoping the decorator would do.
Maybe the following works:
import functools
def method_flagger(function):
function._flag = "flag"
return function
list_of_methods = []
def flag_reader(function):
#functools.wraps(function)
def wrapper(self, *args):
for i in dir(self.__class__):
#for i, j in self.__class__.__dict__.items():
method_wrapper = getattr(self, i)
if hasattr(getattr(self, i), "_flag"):
list_of_methods.append((self, i))
return function(self, *args)
return wrapper
class Test:
#flag_reader
def __init__(self):
self.a = "test"
#method_flagger
def test(self):
print(self.a)
#method_flagger
def test2(self):
print(self.a)
t1 = Test()
t2 = Test()
#t.test()
#print(t.__test__)
print(list_of_methods)
It gives me the output
[(<__main__.Test object at 0x000001D56A435860>, 'test'), (<__main__.Test object at 0x000001D56A435860>, 'test2'), (<__main__.Test object at 0x000001D56A435940>, 'test'), (<__main__.Test object at 0x000001D56A435940>, 'test2')]
so for each affected object instance and each decorated function, we get a tuple denoting these both.
In Python, is there a nice way to apply a decorator (or something similar) not to a whole function, but to a subsegment of a function body?
What I want might look something like this:
def deco_double(f):
def wrapper():
f()
f()
return wrapper
def foo():
#deco_double:
print("hello")
print("stack")
#deco_double:
print("overflow")
foo()
So the execution result be like:
hello
stack
hello
stack
overflow
overflow
It's okay if the solution is not exactly a decorator, what I need is a method to wrap function subsegments, but in a well abstracted way. Thank you in advance :)
edit:
Just separating into multiple functions is not an option for me. I'm developing a framework for programmers and I want them to write functions in a compact way (for writing foo() in the example. Also, what I'm actually doing is far more complicated than just repetitions)
edit2:
By now it seems I have no choice but to expect framework users to somehow declare named functions...
Function bodies are compiled into a single "code" object that is run as a whole - changing the way this code object is run, inserting things in it, and such are things that can be as complicated as the language code itself (i.e. the code that actually "executes" Python bytecode).
So, any changes in the flow of execution are far easier done by using the statements in the language that already do that.
If you want the equivalent of a generic decorator in parts inside a function body, the easiest thing to do is to subdivide that function itself into inner functions - and then you can apply your transforms, or execute each part more than once, by simply calling those functions (and even decorate them directly).
However, in the case you bring in your question, you could, and probably should, just use a plain old for loop:
def foo():
for _ in (0, 1):
print("hello")
print("stack")
for _ in (0, 1):
print("overflow")
For arbitrary "decorator like behavior", as I wrote above, just use nested functions:
def foo():
#deco_double
def part1()
print("hello")
print("stack")
part1()
#deco_double:
def part2():
print("overflow")
part2()
You will have to extract that partial functionality of foo() to separate functions and then apply the decorator as a function and call it expilictly.
def deco_double(f):
def wrapper():
f()
f()
return wrapper
def my_f1():
print("hello")
print("stack")
def my_f2():
print("overflow")
def foo():
deco_double(my_f1)()
deco_double(my_f2)()
foo()
Simply declare two functions with any name with decorator #deco_double and call them in any other function, in your case foo() and then simply call foo().
def deco_double(f):
def wrapper():
f()
f()
return wrapper
#deco_double
def func1():
print("hello")
print("stack")
#deco_double
def func2():
print("overflow")
def foo():
func1()
func2()
foo()
Output of this code.
hello
stack
hello
stack
overflow
overflow
You'll likely need to pass functions using lambda if you want to achieve this:
def doubler(f):
f()
f()
def foo():
doubler(lambda: print("hello"))
doubler(lambda: print("world"))
foo()
"hello"
"hello"
"world"
"world"
Wrap the code you want to be run twice in the lambda, and pass the function to doubler
My problem is the following,
1 - I created a functools.wraps decorator function to log some runtime information.
I choose to use functools.wraps because it was supposed to update a wrapper function to look like the wrapped function.
So, my decorator look like this:
def log_execution(logger):
def _log_execution(method):
#functools.wraps(method)
def _logged_execution(*args, **kwargs):
logger.log('log some info')
return method(*args, **kwargs)
return _logged_execution
return _log_execution
2 - Then, considering I decorated some of my functions like that:
# Create logger
current_logger = create_logger('logger test')
# Function func_a and sub functions
def func_a():
func_a1()
func_a2()
#log_execution(current_logger)
def func_a1():
...
#log_execution(current_logger)
def func_a2():
...
# Function func_b and sub functions
def func_b():
func_b1()
func_b2()
#log_execution(current_logger)
def func_b1():
...
#log_execution(current_logger)
def func_b2():
...
3 - Now if I profile my code and if I take a look at the generated reports, it is really hard to understand which function is called by another, when passing by this decorator.
For example: if I take a look at func_a call graph, I will have the following:
What I would like is:
or:
or:
So my question is:
How can I change the decorator function name dynamically to be able to better understand my profiling reports?
functools.wraps does not seems to do the job here!