I implemented a metaclass that tears down the class attributes for classes created with it and builds methods from the data from those arguments, then attaches those dynamically created methods directly to the class object (the class in question allows for easy definition of web form objects for use in a web testing framework). It has been working just fine, but now I have a need to add a more complex type of method, which, to try to keep things clean, I implemented as a callable class. Unfortunately, when I try to call the callable class on an instance, it is treated as a class attribute instead of an instance method, and when called, only receives its own self. I can see why this happens, but I was hoping someone might have a better solution than the ones I've come up with. Simplified illustration of the problem:
class Foo(object):
def __init__(self, name, val):
self.name = name
self.val = val
self.__name__ = name + '_foo'
self.name = name
# This doesn't work as I'd wish
def __call__(self, instance):
return self.name + str(self.val + instance.val)
def get_methods(name, foo_val):
foo = Foo(name, foo_val)
def bar(self):
return name + str(self.val + 2)
bar.__name__ = name + '_bar'
return foo, bar
class Baz(object):
def __init__(self, val):
self.val = val
for method in get_methods('biff', 1):
setattr(Baz, method.__name__, method)
baz = Baz(10)
# baz.val == 10
# baz.biff_foo() == 'biff11'
# baz.biff_bar() == 'biff12'
I've thought of:
Using a descriptor, but that seems way more complex than is necessary here
Using a closure inside of a factory for foo, but nested closures are ugly and messy replacements for objects most of the time, imo
Wrapping the Foo instance in a method that passes its self down to the Foo instance as instance, basically a decorator, that is what I actually add to Baz, but that seems superfluous and basically just a more complicated way of doing the same thing as (2)
Is there a better way then any of these to try to accomplish what I want, or should I just bite the bullet and use some closure factory type pattern?
One way to do this is to attach the callable objects to the class as unbound methods. The method constructor will work with arbitrary callables (i.e. instances of classes with a __call__() method)—not just functions.
from types import MethodType
class Foo(object):
def __init__(self, name, val):
self.name = name
self.val = val
self.__name__ = name + '_foo'
self.name = name
def __call__(self, instance):
return self.name + str(self.val + instance.val)
class Baz(object):
def __init__(self, val):
self.val = val
Baz.biff = MethodType(Foo("biff", 42), None, Baz)
b = Baz(13)
print b.biff()
>>> biff55
In Python 3, there's no such thing as an unbound instance method (classes just have regular functions attached) so you might instead make your Foo class a descriptor that returns a bound instance method by giving it a __get__() method. (Actually, that approach will work in Python 2.x as well, but the above will perform a little better.)
from types import MethodType
class Foo(object):
def __init__(self, name, val):
self.name = name
self.val = val
self.__name__ = name + '_foo'
self.name = name
def __call__(self, instance):
return self.name + str(self.val + instance.val)
def __get__(self, instance, owner):
return MethodType(self, instance) if instance else self
# Python 2: MethodType(self, instance, owner)
class Baz(object):
def __init__(self, val):
self.val = val
Baz.biff = Foo("biff", 42)
b = Baz(13)
print b.biff()
>>> biff55
The trouble you're running into is that your object is not being bound as a method of the Baz class you're putting it in. This is because it is not a descriptor, which regular functions are!
You can fix this by adding a simple __get__ method to your Foo class that makes it into a method when it's accessed as a descriptor:
import types
class Foo(object):
# your other stuff here
def __get__(self, obj, objtype=None):
if obj is None:
return self # unbound
else:
return types.MethodType(self, obj) # bound to obj
Related
I am trying to understand descriptors better.
I don't understand why in the foo method the descriptors __get__ method doesn't get called.
As far as I understand descriptors the __get__ method always get called when I access the objects attribute via dot operator or when I use __getattribute__().
According to the Python documentation:
class RevealAccess(object):
def __init__(self, initval=None, name='var'):
self.val = initval
self.name = name
def __get__(self, obj, objtype):
print('Retrieving', self.name)
return self.val
def __set__(self, obj, val):
print('Updating', self.name)
self.val = val
class MyClass(object):
x = RevealAccess(10, 'var "x"')
y = 5
def foo(self):
self.z = RevealAccess(13, 'var "z"')
self.__getattribute__('z')
print(self.z)
m = MyClass()
m.foo()
m.z # no print
m.x # prints var x
z is an attribute on the instance, not on the class. The descriptor protocol only applies to attributes retrieved from a class.
From the Descriptor HOWTO:
For objects, the machinery is in object.__getattribute__() which transforms b.x into type(b).__dict__['x'].__get__(b, type(b)).
and in the Implementing Descriptors section of the Python Data Model:
The following methods only apply when an instance of the class containing the method (a so-called descriptor class) appears in an owner class (the descriptor must be in either the owner’s class dictionary or in the class dictionary for one of its parents).
Your m.z cannot be found in the class dict; type(m).__dict__['z'] does not exist; it is found in m.__dict__['z'] instead. Here m is the instance and the owner class is MyClass, and z does not appear in the owner class dictionary.
I am creating a descriptor, and I want to create a list inside it that holds references to all objects implementing it, it is supposed to be some kind of a shortcut where I can call the method on the next instance in line from the instances.
The only daft solution I could find is just on __init__ of each objects trigger the setter on descriptor that adds the item to the list, even though that solution does work indeed, I can sense that something is wrong with it.
Does anyone have a better way of adding the class instance to a descriptor list other than setting arbitrary value on __init__, just to trigger the setter?
class GetResult(object):
def __init__(self, value):
self.instances = []
def __get__(self, instance, owner):
return self
def __set__(self, instance, value):
self.instances.append(instance)
def getInstances(self):
return self.instances
class A(object):
result = GetResult(0)
def __init__(self):
self.result = 0
def getAll(self):
print self.result.getInstances()
a1 = A()
a2 = A()
a3 = A()
print a2.result.getInstances()
>> [<__main__.A object at 0x02302DF0>, <__main__.A object at 0x02302E10>, <__main__.Aobject at 0x02302E30>]
If that's all your descriptor do, it's a bit of an abuse of the descriptor protocol. Just overriding your class __new__ or __init__ would be simpler:
class Foo(object):
_instances = []
def __new__(cls, *args, **kw):
instance = object.__new__(cls)
cls._instances.append(instance)
return instance
#classmethod
def get_instances(cls):
return self._instances
I need a delegated class to delegate a #classmethod. Here's what I've tried:
class Foo(object):
def __init__(self, a):
self.a = a
#classmethod
def from_a(cls, a):
return cls(a)
class Bar(object):
def __init__(self, foo):
elf._foo = foo
def __getattribute__(self, name):
return getattr(self._foo, name)
But, of course this doesn't define how to look up attributes of Foo (not of an instance of Foo), so Bar.from_a(5) will raise an AttributeError. While it is of course possible to do this explicitly by defining a from_a method on Bar or to do this at instantiation by calling Bar(Foo.from_a(5)), I would rather do this implicitly. Ideas?
I started working on what I thought would be a simple approach for this using a metaclass, but it is actually fairly complex. What you should probably be doing here is having Bar inherit from Foo, but I'll show you what I came up with all the same:
import types
import functools
def make_delegating_type(delegatee):
class DelegatingType(type):
def __getattr__(self, name):
obj = getattr(delegatee, name)
if isinstance(obj, (types.FunctionType, types.MethodType)):
#functools.wraps(obj)
def wrapper(*args, **kwargs):
result = obj(*args, **kwargs)
if isinstance(result, delegatee):
return self(result)
return result
return wrapper
return obj
return DelegatingType
class Foo(object):
def __init__(self, a): self.a = a
#classmethod
def from_a(cls, a): return cls(a)
class Bar(object):
__metaclass__ = make_delegating_type(Foo)
def __init__(self, foo): self._foo = foo
def __getattr__(self, name): return getattr(self._foo, name)
Note that in 3.x you would use class Bar(object, metaclass=make_delegating_type(Foo) instead of the __metaclass__ = make_delegating_type(Foo) line at the top of the Bar class body.
Here is how this works. Your current version currently delegates attribute lookups on instances of Bar to an instance of Foo, this uses a metaclass so that attributes lookups on the class Bar are delegated to the class Foo as well. Unfortunately it is not as simple as just using a __getattr__ definition that returns getattr(delegatee, name), because if the attribute your a looking up is a factory function as in your example you need a version of that factory function that will return an instance of your delegating type. So for example Bar.from_a(5) should be the same as Bar(Foo.from_a(5)), and with the naive approach you would just get Foo.from_a(5). That is why there is all the logic detecting if the attribute is a function or method, and creating a wrapper that checks the return type of that function/method.
To reiterate, I do not recommend that you use this code! It is much more complicated then just defining from_a on Bar or having Bar inherit from Foo. But hopefully it will be a learning experience for you, as it was for me.
I'm just getting to grips with decorators in Python and using them to add callbacks to some instance variables using the following simple pattern:
class A(object):
def __init__(self):
self._var = 0
self.var_callbacks = []
#property
def var(self):
return self._var
#var.setter
def var(self, x):
self._var = x
for f in self.var_callbacks:
f(x)
The property decorator is a neat way of allowing me to introduce callbacks where necessary without changing the class interface. However, after the third or fourth variable it's making the code a bit repetitive.
Is there a way to refactor this pattern into something along the following:
class A(object):
def __init__(self):
self.var = 0
enable_callback(self, 'var', 'var_callbacks')
You'll need to set the property on the class (since it is a descriptor), so using a enable_callback call in the initializer is not going to work.
You could use a class decorator to set the properties from a pattern:
def callback_properties(callbacks_attribute, *names):
def create_callback_property(name):
def getter(self):
return getattr(self, '_' + name)
def setter(self, value):
setattr(self, '_' + name, value)
for f in getattr(self, callbacks_attribute):
f(value)
return property(getter, setter)
def add_callback_properties(cls):
for name in names:
setattr(cls, name, create_callback_property(name)
return cls
return add_callback_properties
Then use that as:
#add_callback_properties('var_callbacks', 'var1', 'var2')
class A(object):
# everything else
Have a look at the Python descriptor protocol. In essence, you can define a class that handles the getting, setting and deleting of a property. So you could define a descriptor that runs your callbacks on setting the attribute.
Descriptors are regular classes, and can be parameterized. So you could implement a descriptor that takes the destination variable its constructor. Something like the following:
class A(object):
var = CallbackDescriptor('var')
foo = CallbackDescriptor('foo')
class A(object):
#classmethod
def print(cls):
print 'A'
def __print(self):
print 'B'
def __init__(self):
self.print = self.__print
a = A()
a.print()
A.print()
I think it's too ugly, is there any other method to implement the same features? do not say combinemethod, because it creates an object every time.
The simplest solution is to create a descriptor decorator like classmethod but that also passes the instance to the method:
from functools import partial
class descriptormethod(object):
def __init__(self, fn):
self.fn = fn
def __get__(self, instance, owner):
return partial(self.fn, instance, owner)
class A(object):
#descriptormethod
def print_(self, cls):
print 'A' if self is None else 'B'
Don't worry about the overhead of the descriptor or partial objects; it's no different from what happens when you call an instance or class method normally.