Let's say I have the following descriptor:
class MyDescriptor(object):
def __init__(self, name, type_):
self.name = name
self.type_ = type_
def __set__(self, obj, value):
assert isinstance(value, self.type_)
obj.__dict__[self.name] = value
Is there a way to access type_ from an object employing MyDescriptor?
i.e.
class MyObject(object):
x = MyDescriptor('x', int)
my_object = MyObject()
my_object.x = 5
print my_object.x.type_
As far as I'm aware, this will raise AttributeError as my_object.x is an int. But, I'm wondering if there's a good way to associate metadata with descriptors.
EDIT: adjusted wording to indicate that there's one instance of a descriptor per class.
Is there a way to access type_ from the object instance which owns the MyDescriptor instance?
There is no object instance which owns the MyDescriptor instance. There is one instance of MyDescriptor which is stored on the class of which the descriptor is an attribute (MyObject in your example). That's how descriptors work. You can access this descriptor instance via the class as described in user2357112's answer, but be aware that you're accessing class-level data. If you want to store instance-level data with the descriptor, you need to store it on the instance itself (i.e., on the object passed as obj to your __set__/__get__) rather than on the descriptor.
You need to access the actual descriptor object. For your descriptor, that can be done with
type(my_object).x.type_
or
MyObject.x.type_
For descriptors where MyObject.x is not the actual descriptor object, such as functions on Python 2, you may need to find the descriptor by looking in the class __dict__, or looking through the dicts of all classes in the MRO if you want a generic way to find inherited descriptors. (For the specific case I just mentioned, you can also use the __func__ attribute of the unbound method object, but that won't work for other descriptors.)
Related
'Every thing in python is an object'
So, should all objects have to have attributes and methods ?
I read below statemts in tutorial site, could you give example of pre-defined object in python that has neither attributes nor methods ?
Some objects have neither attributes nor methods
Everything in Python is indeed an object, this is true. Even classes themselves are considered to be objects and they're indeed the product of the builtin class typewhich is not surprisingly an object too. But objects will almost certainly inherit attributes including methods or data in most circumstances.
So, should all objects have to have attributes and methods ?
Not necessarily all objects have their own attributes. For instance, an object can inherit the attribute from its class and its class's superclasses, but that attribute or method doesn't necessarily live within the instance's namespace dictionary. In fact, instances' namespaces could be empty like the following:
class Foo:
pass
a = A()
print(a.__dict__)
a here doesn't have any attributes aside from those inherited from its class so if you check its namespace through the builtin attribute __dict__ you'll find the namespace to be an empty dictionary. But you might wonder isn't a.__dict__ an attribute of a? Make a distinction between class-level attributes--attributes inherited from the class or its superclasses and instance-attributes--attributes that belong to the instance and usually live in its namespace __dict__.
Could you give example of pre-defined object in python that has neither attributes nor methods ?
If you meant by predefined object, a builtin object, I couldn't imagine such scenario. Again, even if there are no attributes at the object itself, there would be attributes inherited from its class or the class's superclasses if there's any superclass in most cases. Probably and I'm guessing here, the tutorial is asking you to create class that assigns no attributes to its objects, just like the code I included above.
And this already answers your question better: Is everything an object in python like ruby?
There's a hackish way to emulate a Python object with no attributes.
class NoAttr(object):
def __getattribute__(self, attr):
raise AttributeError("no attribute: %s" % attr)
def __setattr__(self, attr, value):
raise AttributeError("can't set attribute: %s" % attr)
def __delattr__(self, attr):
raise AttributeError("no attribute: %s" % attr)
a = NoAttr()
This instance a, for all intents and purposes, in pure Python, behaves like an object with no attributes (you can try hasattr on it).
There may be a low-level way to do this in a C extension by implementing a type in C that pathologically stops Python's object implementation from working. Anyway the margin here is too small for writing one.
A pre-defined object with no attributes would defeat the purpose of pre-defining it.
According to Python 2.7.12 documentation:
If __setattr__() wants to assign to an instance attribute, it should
not simply execute self.name = value — this would cause a recursive
call to itself. Instead, it should insert the value in the dictionary
of instance attributes, e.g., self.__dict__[name] = value. For
new-style classes, rather than accessing the instance dictionary, it
should call the base class method with the same name, for example,
object.__setattr__(self, name, value).
However, the following code works as one would expect:
class Class(object):
def __setattr__(self, name, val):
self.__dict__[name] = val;
c = Class()
c.val = 42
print c.val
I know super(Class, obj).__setattr__(name, value) can ensure the __setattr__ methods of all base classes to be called, but classic class can also inherit from bases classes. So why is it only recommended for new style classes?
Or, on the other hand, why is doing so not recommended for classic classes?
New-style classes could be using slots, at which point there is no __dict__ to assign to. New-style classes also support other data descriptors, objects defined on the class that handle attribute setting or deletion for certain names.
From the documentation on slots:
By default, instances of both old and new-style classes have a dictionary for attribute storage. This wastes space for objects having very few instance variables. The space consumption can become acute when creating large numbers of instances.
The default can be overridden by defining __slots__ in a new-style class definition. The __slots__ declaration takes a sequence of instance variables and reserves just enough space in each instance to hold a value for each variable. Space is saved because __dict__ is not created for each instance.
Access to slots is instead implemented by adding data descriptors on the class; an object with __set__ and / or __del__ methods for each such attribute.
Another example of data descriptors are property() objects that have a setter or deleter function attached. Setting a key with the same name as such a descriptor object in the __dict__ would be ignored as data descriptors cause attribute lookup to bypass the __dict__ altogether.
object.__setattr__() knows how to handle data descriptors, which is why you should just call that.
I just read about descriptors and it felt very unintentional that the behavior of a class can depend on who uses it. The two methods
__get__(self, instance, owner)
__set__(self, instance, value)
do exactly that. They get in the instance of the class that uses them. What is the reason for this design decision? How is it used?
Update: I think of descriptors as normal types. The class that uses them as a member type can be easily manipulated by side effects of the descriptor. Here is an example of what I mean. Why does Python supprt that?
class Age(object):
def __init__(value):
self.value = value
def __get__(self, instance, owener):
instance.name = 'You got manipulated'
return self.value
class Person(object):
age = Age(42)
name = 'Peter'
peter = Person()
print(peter.name, 'is', peter.age)
__get__ and __set__ receive no information about who's calling them. The 3 arguments are the descriptor object itself, the object whose attribute is being accessed, and the type of the object.
I think the best way to clear this up is with an example. So, here's one:
class Class:
def descriptor(self):
return
foo_instance = Foo()
method_object = foo_instance.descriptor
Functions are descriptors. When you access an object's method, the method object is created by finding the function that implements the method and calling __get__. Here,
method_object = foo_instance.descriptor
calls descriptor.__get__(foo_instance, Foo) to create the method_object. The __get__ method receives no information about who's calling it, only the information needed to perform its task of attribute access.
Descriptors are used to implement binding behaviour; a descriptor requires a context, the object on which they act.
That object is the instance object passed in.
Note that without a descriptor, attribute access on an object acts directly on the object attributes (the instance __dict__ when setting or deleting, otherwise the class and base classes attributes are searched as well).
A descriptor lets you delegate that access to a separate object entirely, encapsulating getting, setting and deleting. But to be able to do so, that object needs access to the context, the instance. Because getting an attribute also normally searches the class and its bases, the __get__ descriptor method is also passed the class (owner) of the instance.
Take functions, for example. A function is a descriptor too, and binding them to an instance produces a method. A class can have any number of instances, but it makes little sense to store bound methods on all those instances when you create the instance, that would be wasteful.
Instead, functions are bound dynamically; you look up the function name on the instance, the function is found on the class instead, and with a call to __get__ the function is bound to the instance, returning a method object. This method object can then pass in the instance to the function when called, producing the self argument.
An example of the descriptor protocol in action is bound methods. When you access an instance method o.foo you can either call it immediately or save it into a variable: a = o.foo. Now, when you call a(x, y, z) the instance o is passed to foo as the first self parameter:
class C(object):
def foo(self, x, y, z):
print(self, x, y, z)
o = C()
a = o.foo
a(1, 2, 3) # prints <C instance at 0x...> 1 2 3
This works because functions implement the descriptor protocol; when you __get__ a function on an object instance it returns a bound method, with the instance bound to the function.
There would be no way for the above to work without the descriptor protocol giving access to the object instance.
class Cls():
def __init__(self, start):
self.value = start
class Desc():
def __get__(self, instance ,owner):
print("In Descriptor's __get__method")
return self.value
def __set__(self, instance, start):
print("In Descriptor's __set__ method")
self.value = start
value = Desc()
X = Cls('Hello')
X.value = "Hi"
Above implementation of descriptor is obscure for me. X.value and Cls.value are refering to same object and is of class str. but Cls.__dict__['value'] is descriptor object. There are two types assigned to a name 'value'.
Can somebody explain this?. What is the logic behind this particular implementation. Why Cls.value or X.value is not descriptor object. I am using python 3.3
You are confusing things by using the name value for two things: one is an attribute of Cls, and its value is a descriptor object. The other is an attribute of that descriptor object, and that is the one whose value is the string.
The key thing to remember is that there is only one descriptor object, shared across all instances of the class. When you do self.value = start in your __set__ method, self refers to the descriptor object, so you are setting a "value" attribute on the descriptor object itself, not on the Cls instance. (If you change it to instance.value = start instead, you will get a recursion error, since that will try to call __set__ again.)
You will see what is going on if you create multiple instances of your class:
>>> x = Cls("oops")
In Descriptor's __set__ method
>>> y = Cls("dang")
In Descriptor's __set__ method
>>> x.value
In Descriptor's __get__method
'dang'
>>> Cls.__dict__['value'].value
'dang'
Notice that creating y changed x.value. This is because there is only one descriptor object, and it only has one "value" attribute, so that value is shared across all instances of Cls.
It's not clear what you're trying to achieve here, so it's hard to say how to "fix" this. Some general principles are:
Don't use self in __get__ and __set__ unless you want to store class-level information. To store instance-specific data, you need to use instance.
Even if you do the above, don't use the same name for the descriptor attribute itself and the hidden attribute where it stores its data, or you will step on your own toes.
Unless you want to do something really fancy, you probably can just use property and not write your own descriptor at all. If you "fixed" the descriptor you wrote above, it would still be useless, because it wouldn't do anything that property doesn't already do.
Why does the __get__ method in a python descriptor accept the owner class as it's third argument? Can you give an example of it's use?
The first argument (self) is self evident, the second (instances) makes sense in the context of the typically shown descriptor pattern (ex to follow), but I've never really seen the third (owner) used. Can someone explain what the use case is for it?
Just by way of reference and facilitating answers this is the typical use of descriptors I've seen:
class Container(object):
class ExampleDescriptor(object):
def __get__(self, instance, owner):
return instance._name
def __set__(self, instance, value):
instance._name = value
managed_attr = ExampleDescriptor()
Given that instance.__class__ is available all I can think of is that explicitly passing the class has something to do with directly accessing the descriptor from the class instead of an instances (ex Container.managed_attr). Even so I'm not clear on what one would do in __get__ in this situation.
owner is used when the attribute is accessed from the class instead of an instance of the class, in which case instance will be None.
In your example attempting something like print(Container.managed_attr) would fail because instance is None so instance._name would raise an AttributeError.
You could improve this behavior by checking to see if instance is None, and it may be useful for logging or raising a more helpful exception to know which class the descriptor belongs to, hence the owner attribute. For example:
def __get__(self, instance, owner):
if instance is None:
# special handling for Customer.managed_attr
else:
return instance._name
When the descriptor is accessed from the class, instance will be None. If you have not accounted for that situation (as your example code does not) then an error will occur at that point.
What should you do in that case? Whatever is sensible. ;) If nothing else makes sense you could follow property's example and return the descriptor itself when accessed from the class.
Yes, it's used so that the descriptor can see Container when Container.managed_attr is accessed. You could return some object appropriate to the use case, like an unbound method when descriptors are used to implement methods.
I think the most famous application of the owner parameter of the __get__ method in Python is the classmethod decorator. Here is a pure Python version:
import types
class ClassMethod:
"Emulate PyClassMethod_Type() in Objects/funcobject.c."
def __init__(self, f):
self.f = f
def __get__(self, instance, owner=None):
if instance is None and owner is None:
raise TypeError("__get__(None, None) is invalid")
if owner is None:
owner = type(instance)
if hasattr(self.f, "__get__"):
return self.f.__get__(owner)
return types.MethodType(self.f, owner)
Thanks to the owner parameter, classmethod works for attribute lookup not only from an instance but also from a class:
class A:
#ClassMethod
def name(cls):
return cls.__name__
A().name() # returns 'A' so attribute lookup from an instance works
A.name() # returns 'A' so attribute lookup from a class works too