I have the following classes:
class A(object):
x = 1
class B(A):
pass
class C(A):
pass
When I print the value of x from each class I get:
>>>A.x, B.x, C.x
(1,1,1)
Then I assign 2 to B.x
B.x = 2
A.x, B.x, C.x
>>>(1,2,1)
Everything was normal but when I assigned 3 to A.x I got this :
A.x=3
A.x, B.x, C.x
>>>(3,2,3)
I thought it would return (3,2,1).
This is fundamentally how inheritance works in Python: for class-level variables, it first checks' the classes namespace, then the namespace of every class in the method resolution order. So, both B and C inherit x from A:
In [1]: class A(object):
...: x = 1
...: class B(A):
...: pass
...: class C(A):
...: pass
...:
In [2]: vars(A)
Out[2]:
mappingproxy({'__module__': '__main__',
'x': 1,
'__dict__': <attribute '__dict__' of 'A' objects>,
'__weakref__': <attribute '__weakref__' of 'A' objects>,
'__doc__': None})
In [3]: vars(B)
Out[3]: mappingproxy({'__module__': '__main__', '__doc__': None})
In [4]: vars(C)
Out[4]: mappingproxy({'__module__': '__main__', '__doc__': None})
When you ask for B.x or C.x, it looks into that class namespace, doesn't find any "x", then tries A's namespace, finds it, and returns it.
Now, when you assign a variable to B.x = 2, that adds it to B's class namespace directly:
In [5]: B.x = 2
...:
In [6]: vars(B)
Out[6]: mappingproxy({'__module__': '__main__', '__doc__': None, 'x': 2})
And similarly, when you assign it to A.x=3, it overwrites the old value:
In [7]: A.x=3
...:
In [8]: vars(A)
Out[8]:
mappingproxy({'__module__': '__main__',
'x': 3,
'__dict__': <attribute '__dict__' of 'A' objects>,
'__weakref__': <attribute '__weakref__' of 'A' objects>,
'__doc__': None})
In [9]: vars(B)
Out[9]: mappingproxy({'__module__': '__main__', '__doc__': None, 'x': 2})
In [10]: vars(C)
Out[10]: mappingproxy({'__module__': '__main__', '__doc__': None})
So now, same as before, when you look for C.x, it doesn't find it's own, then it looks for x inside A, and finds it.
Note, inheritance works like this with instances too, just it checks the instance namespace first, then the instances class's namespace, then all the namespace of the classes in it's method resolution order.
I think it's because of this fact that you did not set "a" field for instance of the "C" class.
Thus it gets its default value from the superclass ("Parent class").
If you set the value of "a" in the c instance, You will get "(3,2,1)".
Related
Class creation seems to never re-define the __dict__ and __weakref__ class attributes (i.e. if they already exist in the dictionary of a superclass, they are not added to the dictionaries of its subclasses), but to always re-define the __doc__ and __module__ class attributes. Why?
>>> class A: pass
...
>>> class B(A): pass
...
>>> class C(B): __slots__ = ()
...
>>> vars(A)
mappingproxy({'__module__': '__main__',
'__dict__': <attribute '__dict__' of 'A' objects>,
'__weakref__': <attribute '__weakref__' of 'A' objects>,
'__doc__': None})
>>> vars(B)
mappingproxy({'__module__': '__main__', '__doc__': None})
>>> vars(C)
mappingproxy({'__module__': '__main__', '__slots__': (), '__doc__': None})
>>> class A: __slots__ = ()
...
>>> class B(A): pass
...
>>> class C(B): pass
...
>>> vars(A)
mappingproxy({'__module__': '__main__', '__slots__': (), '__doc__': None})
>>> vars(B)
mappingproxy({'__module__': '__main__',
'__dict__': <attribute '__dict__' of 'B' objects>,
'__weakref__': <attribute '__weakref__' of 'B' objects>,
'__doc__': None})
>>> vars(C)
mappingproxy({'__module__': '__main__', '__doc__': None})
The '__dict__' and '__weakref__' entries in a class's __dict__ (when present) are descriptors used for retrieving an instance's dict pointer and weakref pointer from the instance memory layout. They're not the actual class's __dict__ and __weakref__ attributes - those are managed by descriptors on the metaclass.
There's no point adding those descriptors if a class's ancestors already provide one. However, a class does need its own __module__ and __doc__, regardless of whether its parents already have one - it doesn't make sense for a class to inherit its parent's module name or docstring.
You can see the implementation in type_new, the (very long) C implementation of type.__new__. Look for the add_weak and add_dict variables - those are the variables that determine whether type.__new__ should add space for __dict__ and __weakref__ in the class's instance memory layout. If type.__new__ decides it should add space for one of those attributes to the instance memory layout, it also adds getset descriptors to the class (through tp_getset) to retrieve the attributes:
if (add_dict) {
if (base->tp_itemsize)
type->tp_dictoffset = -(long)sizeof(PyObject *);
else
type->tp_dictoffset = slotoffset;
slotoffset += sizeof(PyObject *);
}
if (add_weak) {
assert(!base->tp_itemsize);
type->tp_weaklistoffset = slotoffset;
slotoffset += sizeof(PyObject *);
}
type->tp_basicsize = slotoffset;
type->tp_itemsize = base->tp_itemsize;
type->tp_members = PyHeapType_GET_MEMBERS(et);
if (type->tp_weaklistoffset && type->tp_dictoffset)
type->tp_getset = subtype_getsets_full;
else if (type->tp_weaklistoffset && !type->tp_dictoffset)
type->tp_getset = subtype_getsets_weakref_only;
else if (!type->tp_weaklistoffset && type->tp_dictoffset)
type->tp_getset = subtype_getsets_dict_only;
else
type->tp_getset = NULL;
If add_dict or add_weak are false, no space is reserved and no descriptor is added. One condition for add_dict or add_weak to be false is if one of the parents already reserved space:
add_dict = 0;
add_weak = 0;
may_add_dict = base->tp_dictoffset == 0;
may_add_weak = base->tp_weaklistoffset == 0 && base->tp_itemsize == 0;
This check doesn't actually care about any ancestor descriptors, just whether an ancestor reserved space for an instance dict pointer or weakref pointer, so if a C ancestor reserved space without providing a descriptor, the child won't reserve space or provide a descriptor. For example, set has a nonzero tp_weaklistoffset, but no __weakref__ descriptor, so descendants of set won't provide a __weakref__ descriptor either, even though instances of set (including subclass instances) support weak references.
You'll also see an && base->tp_itemsize == 0 in the initialization for may_add_weak - you can't add weakref support to a subclass of a class with variable-length instances.
I discovered a weird behaviour (at least weird for me) on python class variables.
class Base(object):
_var = 0
#classmethod
def inc_class(cls):
cls._var += 1
#staticmethod
def inc_static():
Base._var += 1
class A(Base):
pass
class B(Base):
pass
a = A()
b = B()
a.inc_class()
b.inc_class()
a.inc_static()
b.inc_static()
print(a._var)
print(b._var)
print(Base._var)
The output is 1 1 2.
This is surprising me (I was expecting 4 4 4) and I'm wondering why?
When decorated with #classmethod the first argument cls to inc_class(cls) is, well, the class. <class '__main__.A'> and <class '__main__.B'> respectively for A and B. So cls._var refers to A's _var, and similarly for B. In inc_static, decorated with #staticmethod there is no argument, you're explicitly referring to <class '__main__.Base'>, a different _var.
Note the '_var': 0 attribute in Base's and A's __dict__. #classmethod is doing what you'd expect it to do, binding members to classes, in this case A and B.
>>> Base.__dict__
mappingproxy({'__module__': '__main__', '_var': 0, 'inc_class': <classmethod
object at 0x7f23037a8b38>, 'inc_static': <staticmethod object at
0x7f23037a8c18>, '__dict__': <attribute '__dict__' of 'Base' objects>,
'__weakref__': <attribute '__weakref__' of 'Base' objects>, '__doc__': None})
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None})`
After calling Base.inc_static():
>>> Base.__dict__
mappingproxy({'__module__': '__main__', '_var': 1, 'inc_class':
<classmethod object at 0x7f23037a8b38>, 'inc_static': <staticmethod
object at 0x7f23037a8c18>, '__dict__': <attribute '__dict__' of 'Base'
objects>, '__weakref__': <attribute '__weakref__' of 'Base' objects>,
'__doc__': None})
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None})
After calling A.inc_class():
>>> Base.__dict__
mappingproxy({'__module__': '__main__', '_var': 1, 'inc_class':
<classmethod object at 0x7f23037a8b38>, 'inc_static': <staticmethod
object at 0x7f23037a8c18>, '__dict__': <attribute '__dict__' of 'Base'
objects>, '__weakref__': <attribute '__weakref__' of 'Base' objects>,
'__doc__': None})
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None, '_var': 1})
What's interesting is how A's _var is initialised. Note that you do cls._var += 1 before cls._var has been defined. As explained here, cls._var += 1 is equivalent to cls._var = cls._var; cls._var += 1. Because of the way python does lookup the first read of cls._var will fail in A and continue to find it in Base. At the assignment _var is added to A's __dict__ with the value of Base._var, and then all is fine.
>>> class Base(object):
... _var = 10
... #classmethod
... def inc_class(cls):
... cls._var += 1
...
>>> class A(Base):
... pass
...
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None})
>>> A.inc_class()
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None, '_var': 11})
Even though the two classes inherit from the Base class, they are completely different objects. Through the instantiation of a and b, you have two objects that belong to two separate classes. When you call
a.inc_class()
b.inc_class()
you increment the _var attribute of class A once, and then you do the same for class B. Even though they share the same name, they are different objects. If you had a second instance of class A, say a2, and you would call the function again, then both calls would manipulate the same variable. This explains how you get your first two outputs.
The third output refers to the Base class object. Again, even though it is the same name, it is a different object. You increment the 3rd object twice, therefore you get 2 as the answer.
I have a class Foo with a class variable remote. Can I access the class variable remote using self.remote?
class Foo:
remote = False
def __init__(self):
self.remote = True
#classmethod
def print_remote(cls):
print(cls.remote) #prints False but why?
Assigning remote to self in __init__ means that instance.remote is found first when you access it through self (granted no descriptors are around). To get both options, access either from self or from type(self), that is, either from the instance or the class:
def print_remote(self):
print(type(self).remote) # class remote
print(self.remote) # instance remote
type(self).remote is essentially equivalent to self.__class__.remote but, in general, you should avoid grabbing dunder names (__*__) when there's a built in that does it for you (type in this case)
These live in different dictionaries and are different variables. self.remote lives in the instance dict while class.remote in the class dict.
>>> Foo().__dict__['remote']
True
>>> Foo.__dict__['remote']
False
When you access through cls with a classmethod (or type(self) in a normal method) you'll get the class one, when you access through self you get the instance one.
In [1]: class Foo:
...: x = 0
...:
In [2]: f = Foo()
In [4]: f.__dict__ # empty
Out[4]: {}
In [5]: Foo.__dict__ # have the variable x = 0
Out[5]:
mappingproxy({'__dict__': <attribute '__dict__' of 'Foo' objects>,
'__doc__': None,
'__module__': '__main__',
'__weakref__': <attribute '__weakref__' of 'Foo' objects>,
'x': 0})
When you try access a variable in a object, Python will look first in the object, if it is not there then it looks in class dict.
In [6]: Foo.x = 10 # changing class variable
In [7]: f.__dict__ # still empty.
Out[7]: {}
In [8]: f.x # gives you Foo's x as object doesn't have that item.
Out[8]: 10
In [9]: f.x = 20 # this line creates a new variable in x. Now both class and object has their own variable x
In [10]: f.__dict__ # f has its own x.
Out[10]: {'x': 20}
In [11]: Foo.__dict__ # Foo has its own x.
Out[11]:
mappingproxy({'__dict__': <attribute '__dict__' of 'Foo' objects>,
...
'x': 10})
In [12]: f.x # always you will get the value from f.__dict__
Out[12]: 20
In [16]: f.x = 50 # changing the value of object's variable
In [17]: Foo.x # above statement didn't affect class's variable.
Out[17]: 10
In [13]: del f.x # delete object's x
In [14]: f.x # now f doesn't have x, you get the value from class Foo.
Out[14]: 10
Yes, you can access the class variable with self. But, in case you have an instance variable, you will be accessing the instance variable when you use self as it is shadowing the class variable.
Please consider the following python example:
In [3]: class test(object):
...: attribute='3'
...: def __init__(self):
...: self.other='4'
...:
In [4]: b=test()
In [5]: b.attribute
Out[5]: '3'
In [6]: b.__dict__
Out[6]: {'other': '4'}
Why is it that __dict__ only shows the "other" attribute and not "atribute"?
And how do I get a dictionary with all the classe's attributes and values? That is, how do I get this?
{'other': '4', 'attribute': '3'}
And I mean by using __dict__ or by some other simple means.
PS: related to this question, but couldn't quite get a dict from there.
PS2: I'm not look for test.__dict__ or b.__class__.__dict__, I'm looking for something that can be used as
In [3]: class test(object):
...: attribute='3'
...: def __init__(self):
...: self.other='4'
...: def _print_atr(self):
...: # This should print exactly {'other': '4', 'attribute': '3'}
...: print(self.__all_atr__)
In [4]: b=test()
In [5]: b.attribute
Out[5]: '3'
In [6]: b.__dict__
Out[6]: {'other': '4'}
Cheers
attribute is not an instance attribute but a class attribute (can be seen in the mappingproxy test.__dict__).
You can get attribute in the instance __dict__ if you update the value of attribute from the instance:
>>> b = test()
>>> b.__dict__
{'other': '4'}
>>> b.attribute
'3'
>>> b.attribute = 5
>>> b.__dict__
{'attribute': 5, 'other': '4'}
Or keep the original value with
>>> b.attribute = b.__class__.attribute # may not be necessary
Or you could change the definition of the class and move attribute into one of the class methods and bind it to the instance via self.
b.__dict__ is only a mapping of attributes on b, not on b's class (notice that __init__ isn't there either). The attributes on b's class are on the class's __dict__.
>>> class test(object):
... attribute = 1
... def __init__(self):
... self.other = 2
...
>>> b = test()
>>> b.__dict__
{'other': 2}
>>> test.__dict__
dict_proxy({'__module__': '__main__', 'attribute': 1, '__dict__': <attribute '__dict__' of 'test' objects>, '__weakref__': <attribute '__weakref__' of 'test' objects>, '__doc__': None, '__init__': <function __init__ at 0x1030f72a8>})
If you want both, you can do something like:
d = dict(vars(type(b)))
d.update(vars(b))
(note that some prefer vars(b) to b.__dict__) Of course, this doesn't get subclasses ...
If you want subclasses, you'll need to walk the method resolution order...
d = {}
for cls in type(b).__mro__:
d.update(vars(cls))
d.update(vars(b))
Try typing:
test.__dict__
And it shows a key with 'attribute'. This happens exactly because attribute is a class variable not an instance variable.
Code goes first,
#Python 2.7
>>>class A(object):
pass
>>>a1 = A()
>>>a2 = A()
>>>A.__dict__
dict_proxy({'__dict__': <attribute '__dict__' of 'A' objects>, '__module__': '__main__', '__weakref__': <attribute '__weakref__' of 'A' objects>, '__doc__': None})
Question
1.what is dict_proxy and why use it?
2.A.__dict__ contains an attr -- '__dict': <attribute '__dict__' of 'A' objects>. What is this? Is it for a1 and a2? But objects of A have their own __dict__, don't they?
For your fist question I quote from Fredrik Lundh: http://www.velocityreviews.com/forums/t359039-dictproxy-what-is-this.html:
a CPython implementation detail, used to protect an internal data structure used
by new-style objects from unexpected modifications.
For your second question:
>>> class A(object):
pass
>>> a1 = A()
>>> a2 = A()
>>> a1.foo="spam"
>>> a1.__dict__
{'foo': 'spam'}
>>> A.bacon = 'delicious'
>>> a1.bacon
'delicious'
>>> a2.bacon
'delicious'
>>> a2.foo
Traceback (most recent call last):
File "<pyshell#314>", line 1, in <module>
a2.foo
AttributeError: 'A' object has no attribute 'foo'
>>> a1.__dict__
{'foo': 'spam'}
>>> A.__dict__
dict_proxy({'__dict__': <attribute '__dict__' of 'A' objects>, 'bacon': 'delicious', '__module__': '__main__', '__weakref__': <attribute '__weakref__' of 'A' objects>, '__doc__': None})
Does this answer your question?
If not, dive deeper: https://stackoverflow.com/a/4877655/1324545
dict_proxy prevents you from creating new attributes on a class object by assigning them to the __dict__. If you want to do that use setattr(A, attribute_name, value).
a1 and a2 are instances of A and not class objects. They don't have the protection A has and you can assign using a1.__dict__['abc'] = 'xyz'