Say I have a class which defines __slots__:
class Foo(object):
__slots__ = ['x']
def __init__(self, x=1):
self.x = x
# will the following work?
def __setattr__(self, key, value):
if key == 'x':
object.__setattr__(self, name, -value) # Haha - let's set to minus x
Can I define __setattr__() for it?
Since Foo has no __dict__, what will it update?
All your code does, apart from negate the value, is call the parent class __setattr__, which is exactly what would happen without your __setattr__ method. So the short answer is: Sure you can define a __setattr__.
What you cannot do is redefine __setattr__ to use self.__dict__, because instances of a class with slots do not have a __dict__ attribute. But such instances do have a self.x attribute, it's contents are just not stored in a dictionary on the instance.
Instead, slot values are stored in the same location a __dict__ instance dictionary would otherwise be stored; on the object heap. Space is reserved for len(__slots__) references, and descriptors on the class access these references on your behalf.
So, in a __setattr__ hook, you can just call those descriptors directly instead:
def __setattr__(self, key, value):
if key == 'x':
Foo.__dict__[key].__set__(self, -value)
Interesting detour: yes, on classes without a __slots__ attribute, there is a descriptor that would give you access to the __dict__ object of instances:
>>> class Bar(object): pass
...
>>> Bar.__dict__['__dict__']
<attribute '__dict__' of 'Bar' objects>
>>> Bar.__dict__['__dict__'].__get__(Bar(), Bar)
{}
which is how normal instances can look up self.__dict__. Which makes you wonder where the Bar.__dict__ object is found. In Python, it is turtles all the way down, you'd look that object up on the type object of course:
>>> type.__dict__['__dict__']
<attribute '__dict__' of 'type' objects>
>>> type.__dict__['__dict__'].__get__(Bar, type)
dict_proxy({'__dict__': <attribute '__dict__' of 'Bar' objects>, '__module__': '__main__', '__weakref__': <attribute '__weakref__' of 'Bar' objects>, '__doc__': None})
Related
Class creation seems to never re-define the __dict__ and __weakref__ class attributes (i.e. if they already exist in the dictionary of a superclass, they are not added to the dictionaries of its subclasses), but to always re-define the __doc__ and __module__ class attributes. Why?
>>> class A: pass
...
>>> class B(A): pass
...
>>> class C(B): __slots__ = ()
...
>>> vars(A)
mappingproxy({'__module__': '__main__',
'__dict__': <attribute '__dict__' of 'A' objects>,
'__weakref__': <attribute '__weakref__' of 'A' objects>,
'__doc__': None})
>>> vars(B)
mappingproxy({'__module__': '__main__', '__doc__': None})
>>> vars(C)
mappingproxy({'__module__': '__main__', '__slots__': (), '__doc__': None})
>>> class A: __slots__ = ()
...
>>> class B(A): pass
...
>>> class C(B): pass
...
>>> vars(A)
mappingproxy({'__module__': '__main__', '__slots__': (), '__doc__': None})
>>> vars(B)
mappingproxy({'__module__': '__main__',
'__dict__': <attribute '__dict__' of 'B' objects>,
'__weakref__': <attribute '__weakref__' of 'B' objects>,
'__doc__': None})
>>> vars(C)
mappingproxy({'__module__': '__main__', '__doc__': None})
The '__dict__' and '__weakref__' entries in a class's __dict__ (when present) are descriptors used for retrieving an instance's dict pointer and weakref pointer from the instance memory layout. They're not the actual class's __dict__ and __weakref__ attributes - those are managed by descriptors on the metaclass.
There's no point adding those descriptors if a class's ancestors already provide one. However, a class does need its own __module__ and __doc__, regardless of whether its parents already have one - it doesn't make sense for a class to inherit its parent's module name or docstring.
You can see the implementation in type_new, the (very long) C implementation of type.__new__. Look for the add_weak and add_dict variables - those are the variables that determine whether type.__new__ should add space for __dict__ and __weakref__ in the class's instance memory layout. If type.__new__ decides it should add space for one of those attributes to the instance memory layout, it also adds getset descriptors to the class (through tp_getset) to retrieve the attributes:
if (add_dict) {
if (base->tp_itemsize)
type->tp_dictoffset = -(long)sizeof(PyObject *);
else
type->tp_dictoffset = slotoffset;
slotoffset += sizeof(PyObject *);
}
if (add_weak) {
assert(!base->tp_itemsize);
type->tp_weaklistoffset = slotoffset;
slotoffset += sizeof(PyObject *);
}
type->tp_basicsize = slotoffset;
type->tp_itemsize = base->tp_itemsize;
type->tp_members = PyHeapType_GET_MEMBERS(et);
if (type->tp_weaklistoffset && type->tp_dictoffset)
type->tp_getset = subtype_getsets_full;
else if (type->tp_weaklistoffset && !type->tp_dictoffset)
type->tp_getset = subtype_getsets_weakref_only;
else if (!type->tp_weaklistoffset && type->tp_dictoffset)
type->tp_getset = subtype_getsets_dict_only;
else
type->tp_getset = NULL;
If add_dict or add_weak are false, no space is reserved and no descriptor is added. One condition for add_dict or add_weak to be false is if one of the parents already reserved space:
add_dict = 0;
add_weak = 0;
may_add_dict = base->tp_dictoffset == 0;
may_add_weak = base->tp_weaklistoffset == 0 && base->tp_itemsize == 0;
This check doesn't actually care about any ancestor descriptors, just whether an ancestor reserved space for an instance dict pointer or weakref pointer, so if a C ancestor reserved space without providing a descriptor, the child won't reserve space or provide a descriptor. For example, set has a nonzero tp_weaklistoffset, but no __weakref__ descriptor, so descendants of set won't provide a __weakref__ descriptor either, even though instances of set (including subclass instances) support weak references.
You'll also see an && base->tp_itemsize == 0 in the initialization for may_add_weak - you can't add weakref support to a subclass of a class with variable-length instances.
I am trying to print a list of the methods that have not been inherited from other classes (e.g.., not inheritted from object or another base class). As an example say I have the following class:
class Point:
def __init__(self, x, y):
self.__x=x
self.__y=y
calling this method should print:
[__init__] without __str__ (inheritted from object).
I've tried:
dir(Point)
but the problem is that it includes already inheritted methods.
To print the non-inherited attributes of an object, such as a class object, use vars which checks the __dict__ of that object:
In [1]: class Point:
...: def __init__(self, x, y):
...: self.__x=x
...: self.__y=y
...:
In [2]: vars(Point)
Out[2]:
mappingproxy({'__dict__': <attribute '__dict__' of 'Point' objects>,
'__doc__': None,
'__init__': <function __main__.Point.__init__>,
'__module__': '__main__',
'__weakref__': <attribute '__weakref__' of 'Point' objects>})
Since a method is merely a callable object in the class, you can check for it using something to the effect of:
In [3]: for k, v in vars(Point).items():
...: if callable(v):
...: print(k)
...:
__init__
You can look into the __dict__ of the class itself:
import types
def list_methods(t):
for name, item in t.__dict__.items():
if isinstance(item, types.FunctionType):
print(name)
t is a class object here, not an instance of a class. If you want to operate on instances, replace t.__dict__.items() with type(t).__dict__.items() in the loop.
Is it possible to obtain a list of all #property decorated methods in a class? If so how?
Example:
class MyClass(object):
#property
def foo(self):
pass
#property
def bar(self):
pass
How would I obtain ['foo', 'bar'] from this class?
Anything decorated with property leaves a dedicated object in your class namespace. Look at the __dict__ of the class, or use the vars() function to obtain the same, and any value that is an instance of the property type is a match:
[name for name, value in vars(MyClass).items() if isinstance(value, property)]
Demo:
>>> class MyClass(object):
... #property
... def foo(self):
... pass
... #property
... def bar(self):
... pass
...
>>> vars(MyClass)
dict_proxy({'__module__': '__main__', 'bar': <property object at 0x1006620a8>, '__dict__': <attribute '__dict__' of 'MyClass' objects>, 'foo': <property object at 0x100662050>, '__weakref__': <attribute '__weakref__' of 'MyClass' objects>, '__doc__': None})
>>> [name for name, value in vars(MyClass).items() if isinstance(value, property)]
['bar', 'foo']
Note that this will include anything that used property() directly (which is what a decorator does, really), and that the order of the names is arbitrary (as dictionaries have no set order).
Is it possible to obtain a list of all #property decorated methods in a class? If so how?
Example:
class MyClass(object):
#property
def foo(self):
pass
#property
def bar(self):
pass
How would I obtain ['foo', 'bar'] from this class?
Anything decorated with property leaves a dedicated object in your class namespace. Look at the __dict__ of the class, or use the vars() function to obtain the same, and any value that is an instance of the property type is a match:
[name for name, value in vars(MyClass).items() if isinstance(value, property)]
Demo:
>>> class MyClass(object):
... #property
... def foo(self):
... pass
... #property
... def bar(self):
... pass
...
>>> vars(MyClass)
dict_proxy({'__module__': '__main__', 'bar': <property object at 0x1006620a8>, '__dict__': <attribute '__dict__' of 'MyClass' objects>, 'foo': <property object at 0x100662050>, '__weakref__': <attribute '__weakref__' of 'MyClass' objects>, '__doc__': None})
>>> [name for name, value in vars(MyClass).items() if isinstance(value, property)]
['bar', 'foo']
Note that this will include anything that used property() directly (which is what a decorator does, really), and that the order of the names is arbitrary (as dictionaries have no set order).
There is this code:
class A:
def __init__(self, x):
self.x = x
def __get__(self, obj, type=None):
print("__get__")
return self.x
def __set__(self, obj, value):
pass
class B:
a_oc = A(44)
def __init__(self, y):
self.a_ob = A(y)
b = B(3)
print(b.a_oc) # class attribute called __get__
print(b.a_ob) # __get__ not called
For class attribute __get__ is called, for instance attribute it is not. Why?
The attribute lookup rule for the new type class(class in 3.x and class inherits from object in 2.x) is, take obj.attr:
if the value is generated by Python, such as __hash__, return it
lookup in obj.__class__.__dict__, if it exists and there exists __get__, return the result of attr.__get__(obj, obj.__class__), if not, lookup in the parent class recursively.
lookup in obj.__dict__. If obj is an instance and the attr exists, return it, or next step. Else if the obj is a class, lookup in itself's, its parents' __dict__, if it is a descriptor, return attr.__get__(None, obj.__class__) or the attr itself.
lookup in obj.__class__.__dict__. If attr is a non-data descriptor, return the result of it. Else return the attr itself if it exists.
raise AttributeError
See you class:
>>> b.__class__
<class 'des.B'>
>>> b.__class__.__dict__
mappingproxy({'__init__': <function B.__init__ at 0x7f2dacb4e290>, '__doc__': None, '__weakref__': <attribute '__weakref__' of 'B' objects>, '__dict__': <attribute '__dict__' of 'B' objects>, 'a_oc': <des.A object at 0x7f2dacb5de50>, '__module__': 'des', '__qualname__': 'B'})
>>>
>>> b.__dict__
{'a_ob': <des.A object at 0x7f2dacb5df10>}
>>>
b.a_oc fits step 2 and b.a_ob fits step3. I put your code in module des.