I am new to decorators but ideally I wan to use them to simply define a bunch of class functions within class OptionClass, each representing some particular option with a name and description and if it's required. I don't want to modify the operation of the class function at all if that makes sense, I only want to use the decorator to define name, description, and if it's required.
Problem 1: I construct an OptionClass() and I want to call it's option_1. When I do this I receive a TypeError as the call decorator is not receiving an instance of OptionClass. Why is this? When I call option_1 passing the instance of OptionClass() it works. How do I call option_1 without needing to always pass the instance as self.
The error when received is:
Traceback (most recent call last):
File "D:/OneDrive_P/OneDrive/projects/python/examples/dec_ex.py", line 110, in <module>
print(a.option_1("test")) # TypeError: option1() missing 1 required positional argument: 'test_text'
File "D:/OneDrive_P/OneDrive/projects/python/examples/dec_ex.py", line 80, in __call__
return self.function_ptr(*args, **kwargs)
TypeError: option_1() missing 1 required positional argument: 'test_text'
Problem 2: How would I run or call methods on the decorator to set_name, set_description, set_required?
Problem 3: Although this is a sample I intend to code an option class using async functions and decorate them. Do I need to make the decorator call be async def __call__() or is it fine since it's just returning the function?
class option_decorator(object):
def __init__(self, function_pt):
self.function_ptr = function_pt
self.__required = True
self.__name = ""
self.__description = ""
def set_name(self, text):
self.__name = text
def set_description(self, text):
self.__description = text
def set_required(self,flag:bool):
self.__required = flag
def __bool__(self):
"""returns if required"""
return self.__required
def __call__(self, *args, **kwargs):
return self.function_ptr(*args, **kwargs)
def __str__(self):
"""prints a description and name of the option """
return "{} - {}".format(self.__name, self.__description)
class OptionClass(object):
"""defines a bunch of options"""
#option_decorator
def option_1(self,test_text):
return("option {}".format(test_text))
#option_decorator
def option_2(self):
print("option 2")
def get_all_required(self):
"""would return a list of option functions within the class that have their decorator required flag set to true"""
pass
def get_all_available(self):
"""would return all options regardless of required flag set"""
pass
def print_all_functions(self):
"""would call str(option_1) and print {} - {} for example"""
pass
a = OptionClass()
print(a.option_1("test")) # TypeError: option1() missing 1 required positional argument: 'test_text'
print(a.option_1(a,"test")) #Prints: option test
Problem 1
You implemented the method wrapper as a custom callable instead of as a normal function object. This means that you must implement the __get__() descriptor that transforms a function into a method yourself. (If you had used a function this would already be present.)
from types import MethodType
class Dec:
def __init__(self, f):
self.f = f
def __call__(self, *a, **kw):
return self.f(*a, **kw)
def __get__(self, obj, objtype=None):
return self if obj is None else MethodType(self, obj)
class Foo:
#Dec
def opt1(self, text):
return 'foo' + text
>>> Foo().opt1('two')
'footwo'
See the Descriptor HowTo Guide
Problem 2
The callable option_decorator instance replaces the function in the OptionClass dict. That means that mutating the callable instance affects all instances of OptionClass that use that callable object. Make sure that's what you want to do, because if you want to customize the methods per-instance, you'll have to build this differently.
You could access it in class definition like
class OptionClass(object):
"""defines a bunch of options"""
#option_decorator
def option_1(self,test_text):
return("option {}".format(test_text))
option_1.set_name('foo')
Problem 3
The __call__ method in your example isn't returning a function. It's returning the result of the function_ptr invocation. But that will be a coroutine object if you define your options using async def, which you would have to do anyway if you're using the async/await syntax in the function body. This is similar to the way that yield transforms a function into a function that returns a generator object.
Related
I have a python module
helpers.py
def str_to_num(s: str):
'''Converts to int if possible else converts to float if possible
Returns back the string if not possible to convert to a number.
'''
# NOTE: These are not really funcs, but classes.
funcs = [int, float]
for func in funcs:
try:
res = func(s)
break
except ValueError:
continue
else:
res = s
return(res)
I have another module string_number.py
from helpers import str_to_num
class StringNumber:
def __init__(self, s):
self.s = s
str_to_num = str_to_num
#property
def value(self):
return(self.str_to_num(self.s))
def __repr__(self):
return(f'{self.__class__.__name__}({repr(self.s)})')
>>> from string_number import StringNumber
>>> sn = StringNumber(1)
>>> sn.value
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "string_number.py", line 19, in value
return(self.str_to_num(self.s))
TypeError: str_to_num() takes 1 positional argument but 2 were given
However this works when accessing the function from the class:
>>> StringNumber.str_to_num(1)
1
Q.1: Why does the str_to_num attribute require two arguments when accessing it from the instance? Is self being passed to it? If so, why?
Now, I know I can add modify the __init__ method to make it an attribute of the instance
def __init__(self, s):
self.s = s
self.str_to_num = str_to_num
Further, I can resolve this by making a class of Helper functions and then inheriting from it.
from helpers import str_to_num
class Helper:
#staticmethod
def str_to_num(s):
return(str_to_num(s))
class StringNumber(Helper):
def __init__(self, s):
self.s = s
#property
def value(self):
return(self.str_to_num(self.s))
def __repr__(self):
return(f'{self.__class__.__name__}({repr(self.s)})')
Q: 2 Is there a way to make module functions, staticmethods of a class, without using inheritance? Or is this a really bad practice?
Q: 3 Assuming I had a helpers.py module, with a large amount of module functions. To incorporate them as staticmethods into my class, what would be the best way, without making a separate Helper class?
Q.1: Why does the str_to_num attribute require two arguments when accessing it from the instance? Is self being passed to it? If so, why?
You wrote "However this works when accessing the function from the class: StringNumber.str_to_num(1)". It works because you declared your method as a static method by defining it under your class definition.
As contrary to static method, instance method does pass the instance as a first argument when it's called. So when you called instance.str_to_num(1) your str_to_num(s: str) - no matter your type hinted it as a string - received instance as s argument and complained that value 1 hasn't got variable to hold it.
I am trying to write a decorator for a method that will call a second method. When I run the code I receive the error:
AttributeError: 'Backoff' object has no attribute 'formatter'
Simplified, the code is:
class Backoff:
def __init__(self, f):
self.f = f
def __call__(self, *args, **kwargs):
n = 1
while n < 11:
try:
return self.f(self, *args, **kwargs)
except FooError as e:
<handle error>
time.sleep((2 ** n) + (random.randint(0, 1000) / 1000))
n = n + 1
class SomeClass:
def __init__(self):
pass
#Backoff
def first_method(self, foo, bar):
return self.formatter(foo, bar)
def formatter(self, x, y):
return some_function_to_format(x, y)
How can I pass the second method to the first method in a way that the decorator can recognise it?
Any help would be amazing!
You are passing a method to Backoff, assigning it as an instance variable, and then calling it. This not bound to an instance until it is being called in Backoff.__call__, where it is then bound to an instance of Backoff which does not have a formatter property.
There might be a straightforward solution to this depending on what your instance method needs access to (i.e. if it only needs to reference a class or static method, it can just call the fully qualified name directly). However, if you need the instance method to reference an instance property, I would suggest not using a class decorator at all. Using a function decorator will not run into these issues, you can create a closure and return a function with the same call signature.
I'm trying to add a decorator that adds callable attributes to functions that return slightly different objects than the return value of the function, but will execute the function at some point.
The problem I'm running into is that when the function object is passed into the decorator, it is unbound and doesn't contain the implicit self argument. When I call the created attribute function (ie. string()), I don't have access to self and can't pass it into the original function.
def deco(func):
"""
Add an attribute to the function takes the same arguments as the
function but modifies the output.
"""
def string(*args, **kwargs):
return str(func(*args, **kwargs))
func.string = string
return func
class Test(object):
def __init__(self, value):
self._value = 1
#deco
def plus(self, n):
return self._value + n
When I go to execute the attribute created by the decorator, this is the error I get, because args doesn't contain the self reference.
>>> t = Test(100)
>>> t.plus(1) # Gets passed self implicitly
101
>>> t.plus.string(1) # Does not get passed self implicitly
...
TypeError: plus() takes exactly 2 arguments (1 given)
Is there a way to create a decorator like this that can get a reference to self? Or is there a way to bind the added attribute function (string()) so that it also gets called with the implicit self argument?
You can use descriptors here:
class deco(object):
def __init__(self, func):
self.func = func
self.parent_obj = None
def __get__(self, obj, type=None):
self.parent_obj = obj
return self
def __call__(self, *args, **kwargs):
return self.func(self.parent_obj, *args, **kwargs)
def string(self, *args, **kwargs):
return str(self(*args, **kwargs))
class Test(object):
def __init__(self, value):
self._value = value
#deco
def plus(self, n):
return self._value + n
so that:
>>> test = Test(3)
>>> test.plus(1)
4
>>> test.plus.string(1)
'4'
This warrants an explanation. deco is a decorator, but it is also a descriptor. A descriptor is an object that defines alternative behavior that is to be invoked when the object is looked up as an attribute of its parent. Interestingly, bounds methods are themselves implemented using the descriptor protocol
That's a mouthful. Let's look at what happens when we run the example code. First, when we define the plus method, we apply the deco decorator. Now normally we see functions as decorators, and the return value of the function is the decorated result. Here we are using a class as a decorator. As a result, Test.plus isn't a function, but rather an instance of the deco type. This instance contains a reference to the plus function that we wish to wrap.
The deco class has a __call__ method that allows instances of it to act like functions. This implementation simply passes the arguments given to the plus function it has a reference to. Note that the first argument will be the reference to the Test instance.
The tricky part comes in implementing test.plus.string(1). To do this, we need a reference to the test instance of which the plus instance is an attribute. To accomplish this, we use the descriptor protocol. That is, we define a __get__ method which will be invoked whenever the deco instance is accessed as an attribute of some parent class instance. When this happens, it stores the parent object inside itself. Then we can simply implement plus.string as a method on the deco class, and use the reference to the parent object stored within the deco instance to get at the test instance to which plus belongs.
This is a lot of magic, so here's a disclaimer: Though this looks cool, it's probably not a great idea to implement something like this.
You need to decorate your function at instantiation time (before creating the instance method). You can do this by overriding the __new__ method:
class Test(object):
def __new__(cls, *args_, **kwargs_):
def deco(func):
def string(*args, **kwargs):
return "my_str is :" + str(func(*args, **kwargs))
# *1
func.__func__.string = string
return func
obj = object.__new__(cls, *args_, **kwargs_)
setattr(obj, 'plus', deco(getattr(obj, 'plus')))
return obj
def __init__(self, value):
self._value = 1
def plus(self, n):
return self._value + n
Demo:
>>> t = Test(100)
>>> t.plus(1)
>>> t.plus.string(5)
>>> 'my_str is :6'
1. Since python doesn't let you access the real instance attribute at setting time you can use __func__ method in order to access the real function object of the instance method.
Suppose I have this decorator:
def decorator(f):
def f_wrap(*args):
for item in args:
print(args)
return f(*args)
return f_wrap
When used as "permanent" decorators with the # syntax, args retrieves the arguments of the wrapped function. For example, when used with the class below, I receive the instance of MyObject.
Class MyObject(object):
def __init__(self):
pass
#decorator
def function(self):
return
How can I achieve the same result using a "fluid" decorator. Or a decorator that is not permanently bound to the function it is decorating? For example:
def decorator(f):
def f_wrap(*args):
if (not args):
print("Nothing in args")
return f(*args)
return f_wrap
class MyClass(object):
def __init__(self):
pass
def function(self):
return
if __name__ == "__main__":
myobj = MyClass()
myobj.function = decorator(myobj.function)
myobj.function()
In this case, the args tuple always returns empty (I always get "Nothing in args"), even though I anticipated that it would return the instance variable myobj.
EDIT:
In case it was not clear from #AChampion's post the solution is to simply call the fluid-decoratored method as an "unbound" method. E.g.,
from types import MethodType
def decorator(f):
def f_wrap(*args):
# I replaced this with an iteration through
# args. It's a bit more demonstrative.
for item in args:
print(item)
return f(*args)
return f_wrap
class MyClass(object):
def __init__(self):
pass
def function(self):
return
if __name__ == "__main__":
myobj = MyClass()
myobj.function = MethodType(decorator(MyClass.function), myobj)
myobj.function()
The reason for the difference is that you are wrapping different things, a unbound method vs a bound method:
class MyObject(object):
#decorator
def function(self):
pass
Is equivalent to:
import types
class MyClass(object):
def function(self):
pass
m = MyClass(object)
m.function = types.MethodType(decorator(MyClass.function), m)
Not:
m.function = decorator(m.function)
The first being an unbound method, the second being a bound method.
You aren't using all properly. all returns a bool on whether all conditions are met inside what you are checking for in all. In your case, you aren't really doing anything. You will always evaluate to True with how you are using all.
I believe what you are looking for is simply this:
if not args:
Now, ultimately what this checks is if the method you are executing has *args. For the case of the function you have, you aren't passing any arguments, therefore, with the if not args check, you will actually get:
"Nothing in args"
However, if you add an argument to your method as such:
def function(self, x):
return
Then call: myobj.function(1)
You will not get "Nothing in args".
To answer your last question about not getting your instance. If you print out f using this method of calling your decorator:
myobj.function = decorator(myobj.function)
myobj.function()
You will get a bound method:
<bound method MyClass.function of <__main__.MyClass object at 0x102002390>>
Now, set up your decorator as such:
#decorator
def function(self):
return
You will see you get a function attached to your class object:
<function MyClass.function at 0x102001620>
Hence showing that they aren't doing the exact same thing you would expect. Hope this helps clarify a bit.
I want to create a class that doesn't gives an Attribute Error on call of any method that may or may not exists:
My class:
class magic_class:
...
# How to over-ride method calls
...
Expected Output:
ob = magic_class()
ob.unknown_method()
# Prints 'unknown_method' was called
ob.unknown_method2()
# Prints 'unknown_method2' was called
Now, unknown_method and unknown_method2 doesn't actually exists in the class, but how can we intercept the method call in python ?
Overwrite the __getattr__() magic method:
class MagicClass(object):
def __getattr__(self, name):
def wrapper(*args, **kwargs):
print "'%s' was called" % name
return wrapper
ob = MagicClass()
ob.unknown_method()
ob.unknown_method2()
prints
'unknown_method' was called
'unknown_method2' was called
Just in case someone is trying to delegate the unknown method to an object, here's the code:
class MagicClass():
def __init__(self, obj):
self.an_obj = obj
def __getattr__(self, method_name):
def method(*args, **kwargs):
print("Handling unknown method: '{}'".format(method_name))
if kwargs:
print("It had the following key word arguments: " + str(kwargs))
if args:
print("It had the following positional arguments: " + str(args))
return getattr(self.an_obj, method_name)(*args, **kwargs)
return method
This is super useful when you need to apply the Proxy pattern.
Moreover, considering both args and kwargs, allows you to generate an interface totally user friendly, as the ones that use MagicClass treat it as it was the real object.
Override __getattr__; see http://docs.python.org/reference/datamodel.html