I am writing a bunch of code that has a possibility of mutable outputs, like an arithmetic function where I could have the output be a float or an int. Basically my problem is that if I were to create a decorator for each object type I need (probably seven or eight), I would go insane with the constant repetition of:
def Int(fn):
def wrapper():
return int(fn())
return wrapper
What I want to have is a class like below that would create a decorator based on the name it's instantiated with and it would be a copy of the function above but with the appropriate type modifications.
class Decorator(object):
def __init__(self):
...
...
Int = Decorator()
# Then I can use #Int
Any help would be really appreciated. Thanks.
You cannot have Decorator know what name it will be assigned to. Assignment occurs after instantiation, so the object will have already been created by the time it is assigned a name.
You could however make a decorator that creates decorators dynamically:
from functools import wraps
def set_return_type(typeobj):
def decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
return typeobj(func(*args, **kwargs))
return wrapper
return decorator
You would then use this decorator by giving a type object argument for the type you want:
#set_return_type(int) # Causes decorated function to return ints
#set_return_type(float) # Causes decorated function to return floats
Below is a demonstration:
>>> from functools import wraps
>>> def set_return_type(typeobj):
... def decorator(func):
... #wraps(func)
... def wrapper(*args, **kwargs):
... return typeobj(func(*args, **kwargs))
... return wrapper
... return decorator
...
>>> #set_return_type(float)
... def test():
... return 1
...
>>> test()
1.0
>>>
Related
I know that the decorator is a function that takes another function and extends its behavior.
In the example below, I previously assume that test() now is effectively equivalent to decorator(test)().
def decorator(func):
def wrapper(*args, **kwargs):
...
res = func(*args, **kwargs)
...
return res
return wrapper
#decorator
def test():
pass
However, after adding a function attribute in the decorator and run both test() and decorator(test)(), the results are different. It seems that in the case of decorator(test)(), the decorator function is indeed ran so that num is reset; when using #decorator instead, the decorator function is not ran as I expected?
def decorator(func):
decorator.num = 0
def wrapper(*args, **kwargs):
...
res = func(*args, **kwargs)
...
return res
return wrapper
#decorator
def test():
pass
def test2():
pass
decorator.num = 5
test()
print(decorator.num)
decorator.num = 5
decorator(test2)()
print(decorator.num)
---------------------
5
0
Your confusion stems from when the decorator runs. The syntax
#decorator
def foo(): ...
is equivalent to
def foo(): ...
foo = decorator(foo)
That is, immediately after the function is defined, the decorator is called on it, and the result of calling the decorator is assigned back to the original function name. It's called only once, at definition time, not once per function call.
The same is true of classes. The syntax
#decorator
class Foo: ...
is equivalent to
class Foo: ...
Foo = decorator(Foo)
I am reading a book whose name is python cookbook. In chapter 9.9, I don't understand the below code, especially in __init__(). It uses wraps(func)(self) in __init__(), but it doesn't assign wraps(func)(self) to self. Why self.__wrapped__ can get the original function in __call__()?
import types
from functools import wraps
class Profiled:
def __init__(self, func):
wraps(func)(self)
self.ncalls = 0
def __call__(self, *args, **kwargs):
self.ncalls += 1
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, cls):
if instance is None:
return self
else:
return types.MethodType(self, instance)
#Profiled
def add(x, y):
return x + y
print(add(1, 2))
print(add(1, 3))
print(add.ncalls)
print(add)
I changed it to self = wraps(func)(self), found that it also can work? Who can explain that? I also couldn't understand self in self = wraps(func)(self). What's "self"?
wraps is usually used as a decorator:
#wraps(some_other_func)
def func(...):
...
So that the decorated function has some of its metadata altered to make it look like it was defined with
def some_other_func(...)
Decorator syntax is just equivalent to
def func(...):
...
func = wraps(some_other_func)(func)
but when the value assigned to func is the same object that func originally referred to, the assignment isn't strictly necessary. That's not true for decorators in general, so the assignment is always made. But since you are calling the function returned by wraps(func) explicitly, you can omit the unnecessary assignment back to self.
I am developing an API through which I am passing to the user list of functionalities of a module with the documentations of each function. In order to access the documentation I used to do:
def foo(*args, **kwargs):
"""
Foo documentation is here!
"""
return None
print(foo.__doc__)
# Foo documentation is here!
Now that I added a decorator for some of those functions, the __doc__ returns None since the decorator function doesn't have any documentation.
def decor_func(func):
def wrap(*args, **kwargs):
return func(*args, **kwargs)
return wrap
#decor_func
def foo(*args, **kwargs):
"""
Foo documentation is here!
"""
return None
print(foo.__doc__)
# None
Is there any way that I can have access to decorated function's documentation?
You can update the __doc__ attribute of the wrap function:
def decor_func(func):
def wrap(*args, **kwargs):
return func(*args, **kwargs)
# Set the decorated function `__doc__` attribute
wrap.__doc__ = func.__doc__
return wrap
#decor_func
def foo(*args, **kwargs):
"""
Foo documentation is here!
"""
return None
print(foo.__doc__)
# Foo documentation is here!
However, the best approach is to use functools.wraps, as allows you to also copy additional attributes such as the original name, module and annotations:
import functools
def decor_func(func):
#functools.wraps(func)
def wrap(*args, **kwargs):
return func(*args, **kwargs)
return wrap
#decor_func
def foo(*args, **kwargs):
"""
Foo documentation is here!
"""
return None
print(foo.__doc__)
# Foo documentation is here!
Note, as others have pointed out, you should use functools.wraps so that your wrapper "looks" like the function it is wrapping, and adds the wrapped fucntion to a __wrapped__ attribute. However, note, you can always introspect the wrapper's closure to retrieve a reference to the original function, since it is a free variable in the wrapper and thus will be stored in the closure:
>>> def decor_func(func):
... def wrap(*args, **kwargs):
... return func(*args, **kwargs)
... return wrap
...
>>> #decor_func
... def foo(*args, **kwargs):
... """
... Foo documentation is here!
... """
... return None
...
>>> foo.__closure__
(<cell at 0x10e69da90: function object at 0x10e83a700>,)
So,
>>> foo.__closure__[0].cell_contents.__doc__
'\n Foo documentation is here!\n '
But again, you should use functools.wraps to begin with. The above might help if you have no control over the decorator though.
How to test the following decorator which calls 3rd party library?
import third_party_lib
import functools
class MyDecorator:
def __init__(self, ....):
self.third_party_lib = ThirdPartyLib(....) # will create a 3rd party instance
def __call__(self, ...):
def decorator(f):
#functools.wraps(f)
def wrap(*arg, **kwargs):
result = f(*arg, **kwargs)
# ....
a = self.third_party_lib.send(value=result).get()
# ....
return result
return wrap
return decorator
I need to create an unit test to assert third_party_lib.send() is called if a function is decorated by the decorator. And ideally, also assure the result of the test function is passed to the function.
decorator = MyDecorator(....)
#decorator(....)
def test_func():
ret = ...
return ret # ret should be passed to `third_party_lib.send()`
If you want to verify that the thirdparty function is called correctly, you can mock it and check that the mock is called with the correct parameters. As the ThirdPartyLib intialization shall also be mocked, as mentioned in the comments, you hav to make sure that the docorator is constructed after the mock has been set, for example by constructing it inside the test:
from unittest import mock
#mock.patch('third_party_lib.ThirdPartyLib')
def test_my_decorator(mocked_lib):
decorator = MyDecorator()
#decorator()
def example_func():
return 42
example_func()
mocked_lib.return_value.send.assert_called_once_with(value=42)
If you need the decorated function in more tests, you can wrap it in a function instead:
def decorated_func():
decorator = MyDecorator()
#decorator()
def example_func():
return 42
return example_func
#mock.patch('third_party_lib.ThirdPartyLib')
def test_my_decorator(mocked_lib):
decorated_func()()
mocked_lib.return_value.send.assert_called_once_with(value=42)
The following minimal example of a decorator on a member function:
def wrap_function(func):
def wrapper(*args, **kwargs):
print(args)
print(kwargs)
return wrapper
class Foo:
#wrap_function
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
outputs:
(<__main__.Foo object at 0x7fb294939898>, 'hi')
{}
So self is one of the args.
However when using a wrapper class:
class WrappedFunction:
def __init__(self, func):
self._func = func
def __call__(self, *args, **kwargs):
print(args)
print(kwargs)
def wrap_function(func):
return WrappedFunction(func)
class Foo:
#wrap_function
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
the output is:
('hi',)
{}
So the self, that references the Foo object, is not accessible in the body of __call__ of the WrappedFunction object.
How can I make it accessible there?
You're losing the reference to your bounded instance by wrapping the function logic (but not the instance) and redirecting it to a class instance - at that point, the class instance's own self applies instead of the wrapped instance method as it gets lost in the intermediary decorator (wrap_function()).
You either have to wrap the call to the wrapped function and pass *args/**kwargs to it, or just make a proper wrapper class instead of adding an intermediary wrapper:
class WrappedFunction(object):
def __call__(self, func):
def wrapper(*args, **kwargs):
print(args)
print(kwargs)
# NOTE: `WrappedFunction` instance is available in `self`
return wrapper
class Foo:
#WrappedFunction() # wrap directly, without an intermediary
def mem_fun(self, msg):
pass
foo = Foo()
foo.mem_fun('hi')
# (<__main__.Foo object at 0x000001A2216CDBA8>, 'hi')
# {}
Sadly, but this might be the only solution as you need it in the __call__ function.
Would suggest checking this out: What is the difference between __init__ and __call__ in Python?
def wrap_function(func):
def wrapper(*args, **kwargs):
x = WrappedFunction(func)
x(*args, **kwargs)
return wrapper