I am trying to understand descriptors better.
I don't understand why in the foo method the descriptors __get__ method doesn't get called.
As far as I understand descriptors the __get__ method always get called when I access the objects attribute via dot operator or when I use __getattribute__().
According to the Python documentation:
class RevealAccess(object):
def __init__(self, initval=None, name='var'):
self.val = initval
self.name = name
def __get__(self, obj, objtype):
print('Retrieving', self.name)
return self.val
def __set__(self, obj, val):
print('Updating', self.name)
self.val = val
class MyClass(object):
x = RevealAccess(10, 'var "x"')
y = 5
def foo(self):
self.z = RevealAccess(13, 'var "z"')
self.__getattribute__('z')
print(self.z)
m = MyClass()
m.foo()
m.z # no print
m.x # prints var x
z is an attribute on the instance, not on the class. The descriptor protocol only applies to attributes retrieved from a class.
From the Descriptor HOWTO:
For objects, the machinery is in object.__getattribute__() which transforms b.x into type(b).__dict__['x'].__get__(b, type(b)).
and in the Implementing Descriptors section of the Python Data Model:
The following methods only apply when an instance of the class containing the method (a so-called descriptor class) appears in an owner class (the descriptor must be in either the owner’s class dictionary or in the class dictionary for one of its parents).
Your m.z cannot be found in the class dict; type(m).__dict__['z'] does not exist; it is found in m.__dict__['z'] instead. Here m is the instance and the owner class is MyClass, and z does not appear in the owner class dictionary.
Related
From the descriptor docs:
A descriptor can be called directly by its method name. For example, d.__get__(obj).
What would be an example of this with the following class?
class Descriptor:
def __init__(self, color="red"):
self.color = color
For example, what is d and what is obj? How would "d.__get__(obj)" be called with the above class/instance?
Working Example
To make your example a descriptor, it needs to have a __get__() method:
class Descriptor:
def __init__(self, color="red"):
self.color = color
def __get__(self, obj, objtype=None):
return obj.size + ' ' + self.color
Use that descriptor in another class:
class A:
pair = Descriptor('green')
def __init__(self, size):
self.size = size
Invoke the descriptor like this:
>>> a = A('big')
>>> a.pair
'big green'
Hope this working example helps :-)
Key points
1) A class is a descriptor if defines any one of __get__(), __set__(), or __delete__().
2) Put it to work by making an instance of the descriptor and storing it as a class variable in another class.
3) Invoke the descriptor with normal attribute lookup using the dot operator.
That's really all there is to it :-)
Descriptors are a way to make code look like data and also to achieve polymorphism.
There's a good example on the page that you linked to, under "Descriptor Example"
I'll copy it here:
class RevealAccess(object):
"""A data descriptor that sets and returns values
normally and prints a message logging their access.
"""
def __init__(self, initval=None, name='var'):
self.val = initval
self.name = name
def __get__(self, obj, objtype):
print('Retrieving', self.name)
return self.val
def __set__(self, obj, val):
print('Updating', self.name)
self.val = val
>>> class MyClass(object):
... x = RevealAccess(10, 'var "x"')
... y = 5
...
>>> m = MyClass()
>>> m.x
Retrieving var "x"
10
>>> m.x = 20
Updating var "x"
>>> m.x
Retrieving var "x"
20
>>> m.y
5
Do you see how the class MyClass assigns the RevealAccess descriptor to it's x attribute?
In your case, you would need to also need to add a class that uses the descriptor you made.
I have a class Foo that uses lazy loading for Foo.bar.
class Foo(object):
#property
def bar(self):
if not hasattr(self, '_bar'):
self._initBar()
return self._bar
def _initBar(self):
self._bar = 'bar'
foo = Foo()
print(foo.bar) # prints "bar"
However when i try to convert Foo to use class methods only, Foo.bar is not giving me bar, but instead giving:
<property object at 0x000001CA1C344728>
Why is it not giving me bar?
class Foo(object):
#property
#classmethod
def bar(cls):
if not hasattr(cls, '_bar'):
cls._initBar()
return cls._bar
#classmethod
def _initBar(cls):
cls._bar = 'bar'
print(Foo.bar) # prints "<property object at 0x000001CA1C344728>"
The property built-in is a handy tool in Python that presents an easy use case of a more powerful mechanism, which is the "descriptor protocol".
Basically, any object when retrieved from an instance or from a class is first checked if it has one of __get__, __set__ or __del__ methods. property wraps getter functions to be called by __get__ when the attribute is retrieved from an instance, but returns the property object itself when it is retrieved from the class instead. (That is even a common use case for other descriptors)
Thus, if you want a property like behavior for class attributes, you have to create your own descriptor class, sporting the __get__ method - or, simply create your class with a metaclass, and use property as is, on the metaclass. The drawbacks of the later are many: you willneed one custom metaclass for each class where you want the managed class attributes being just the first of them. On the other hand, creating your own descriptor is quite easy:
class MyProperty:
def __init__(self, initializer):
self.initializer = initializer
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if not hasattr(owner, "_" + self.name):
initializer = getattr(owner, self.initializer)
initializer()
return getattr(owner, "_" + self.name)
class Foo:
bar = MyProperty("_initBar")
#classmethod
def _initBar(cls):
cls._bar = 'bar'
Please note that __set_name__ is only implemented from Python 3.6 on. On older Python's, including Python 2.x, you should use:
class MyProperty(object):
def __init__(self, initializer, name):
self.initializer = initializer
self.name = name
def __get__(self, instance, owner):
if not hasattr(owner, "_" + self.name):
initializer = getattr(owner, self.initializer)
initializer(owner)
return getattr(owner, "_" + self.name)
class Foo(object):
bar = MyProperty("_initBar", name='bar')
#classmethod
def _initBar(cls):
cls._bar = 'bar'
You could use a metaclass, because property objects are meant to be accessed through the instance and if your intended instance is the class itself, then you need to put the property on the class of the class, i.e. the metaclass:
In [37]: class MetaFoo(type):
...: #property
...: def bar(cls):
...: if not hasattr(cls, '_bar'):
...: print("hard at work!")
...: cls._init_bar()
...: return cls._bar
...:
In [38]: class Foo(metaclass=MetaFoo):
...: #classmethod
...: def _init_bar(cls):
...: cls._bar = 'bar'
...:
In [39]: Foo.bar
hard at work!
Out[39]: 'bar'
In [40]: Foo.bar
Out[40]: 'bar'
Of course, while this may be possible, I'm agnostic about whether or not it is not advisable.
Edit
As #jsbueno demonstrates, it is much more sane to simply define your own descriptor, which can give you a much more flexible behavior.
I am creating a descriptor, and I want to create a list inside it that holds references to all objects implementing it, it is supposed to be some kind of a shortcut where I can call the method on the next instance in line from the instances.
The only daft solution I could find is just on __init__ of each objects trigger the setter on descriptor that adds the item to the list, even though that solution does work indeed, I can sense that something is wrong with it.
Does anyone have a better way of adding the class instance to a descriptor list other than setting arbitrary value on __init__, just to trigger the setter?
class GetResult(object):
def __init__(self, value):
self.instances = []
def __get__(self, instance, owner):
return self
def __set__(self, instance, value):
self.instances.append(instance)
def getInstances(self):
return self.instances
class A(object):
result = GetResult(0)
def __init__(self):
self.result = 0
def getAll(self):
print self.result.getInstances()
a1 = A()
a2 = A()
a3 = A()
print a2.result.getInstances()
>> [<__main__.A object at 0x02302DF0>, <__main__.A object at 0x02302E10>, <__main__.Aobject at 0x02302E30>]
If that's all your descriptor do, it's a bit of an abuse of the descriptor protocol. Just overriding your class __new__ or __init__ would be simpler:
class Foo(object):
_instances = []
def __new__(cls, *args, **kw):
instance = object.__new__(cls)
cls._instances.append(instance)
return instance
#classmethod
def get_instances(cls):
return self._instances
I implemented a metaclass that tears down the class attributes for classes created with it and builds methods from the data from those arguments, then attaches those dynamically created methods directly to the class object (the class in question allows for easy definition of web form objects for use in a web testing framework). It has been working just fine, but now I have a need to add a more complex type of method, which, to try to keep things clean, I implemented as a callable class. Unfortunately, when I try to call the callable class on an instance, it is treated as a class attribute instead of an instance method, and when called, only receives its own self. I can see why this happens, but I was hoping someone might have a better solution than the ones I've come up with. Simplified illustration of the problem:
class Foo(object):
def __init__(self, name, val):
self.name = name
self.val = val
self.__name__ = name + '_foo'
self.name = name
# This doesn't work as I'd wish
def __call__(self, instance):
return self.name + str(self.val + instance.val)
def get_methods(name, foo_val):
foo = Foo(name, foo_val)
def bar(self):
return name + str(self.val + 2)
bar.__name__ = name + '_bar'
return foo, bar
class Baz(object):
def __init__(self, val):
self.val = val
for method in get_methods('biff', 1):
setattr(Baz, method.__name__, method)
baz = Baz(10)
# baz.val == 10
# baz.biff_foo() == 'biff11'
# baz.biff_bar() == 'biff12'
I've thought of:
Using a descriptor, but that seems way more complex than is necessary here
Using a closure inside of a factory for foo, but nested closures are ugly and messy replacements for objects most of the time, imo
Wrapping the Foo instance in a method that passes its self down to the Foo instance as instance, basically a decorator, that is what I actually add to Baz, but that seems superfluous and basically just a more complicated way of doing the same thing as (2)
Is there a better way then any of these to try to accomplish what I want, or should I just bite the bullet and use some closure factory type pattern?
One way to do this is to attach the callable objects to the class as unbound methods. The method constructor will work with arbitrary callables (i.e. instances of classes with a __call__() method)—not just functions.
from types import MethodType
class Foo(object):
def __init__(self, name, val):
self.name = name
self.val = val
self.__name__ = name + '_foo'
self.name = name
def __call__(self, instance):
return self.name + str(self.val + instance.val)
class Baz(object):
def __init__(self, val):
self.val = val
Baz.biff = MethodType(Foo("biff", 42), None, Baz)
b = Baz(13)
print b.biff()
>>> biff55
In Python 3, there's no such thing as an unbound instance method (classes just have regular functions attached) so you might instead make your Foo class a descriptor that returns a bound instance method by giving it a __get__() method. (Actually, that approach will work in Python 2.x as well, but the above will perform a little better.)
from types import MethodType
class Foo(object):
def __init__(self, name, val):
self.name = name
self.val = val
self.__name__ = name + '_foo'
self.name = name
def __call__(self, instance):
return self.name + str(self.val + instance.val)
def __get__(self, instance, owner):
return MethodType(self, instance) if instance else self
# Python 2: MethodType(self, instance, owner)
class Baz(object):
def __init__(self, val):
self.val = val
Baz.biff = Foo("biff", 42)
b = Baz(13)
print b.biff()
>>> biff55
The trouble you're running into is that your object is not being bound as a method of the Baz class you're putting it in. This is because it is not a descriptor, which regular functions are!
You can fix this by adding a simple __get__ method to your Foo class that makes it into a method when it's accessed as a descriptor:
import types
class Foo(object):
# your other stuff here
def __get__(self, obj, objtype=None):
if obj is None:
return self # unbound
else:
return types.MethodType(self, obj) # bound to obj
Or does the attribute have to be defined outside of any class methods?
So my descriptor object is this. The IDN object already has some information about the UserNameField, so I want to use it.
class UserNameElement(basePageElement):
_testMethodName="UserNameElement Test method"
def __init__(self, IDN, PTF):
print "creating UserNameElement"
self.locator = IDN.UserNameField()
And here is my calling class. Where I want to instantiate the UserNameElement object
class LoginPageObject(basePageObject):
_testMethodName="LoginPageObject Test method"
print "creating LoginPageObject"
def __init__(self, BaseURL):
super(LoginPageObject, self).__init__()
self.username=UserNameElement(IDN=self.IDN, PTF=self.PTF)
It seems that the standard process would put the username= in in the general class definition, like this:
class LoginPageObject(basePageObject):
_testMethodName="LoginPageObject Test method"
username=UserNameElement()
print "creating LoginPageObject"
def __init__(self, BaseURL):
super(LoginPageObject, self).__init__()
But then I don't have the PTF and IDN that I define in the basePageObject class.
What can I do to make those available when the username attribute is created?
Thanks
I am afraid that will not be possible, as your attribute username will be resolved via normal attribute access see http://docs.python.org/howto/descriptor.html#invoking-descriptors
May be you can get away by overriding __getattribute__ and simulating what type.__getattribute__() does
class MyD(object):
def __init__(self, val):
self.val = val
def __get__(self, obj, objtype):
return self.val
def __set__(self, obj, val):
self.val = val
class C(object):
a = MyD(42)
def __init__(self):
self.d = MyD(42)
def __getattribute__(self, name):
attr = super(C, self).__getattribute__(name)
if hasattr(attr, '__get__'):
return attr.__get__(self, C)
return attr
c = C()
print c.d
print c.a
Output:
42
42
Since you probably won't need the username until the object has been instantiated, it's probably best to just make it a property and write a getter for it.