Possibilities when trying to access an instance attribute? [duplicate] - python

Python provides us many possibilities on instance/class attribute, for example:
class A(object):
def __init__(self):
self.foo = "hello"
a = A()
There are many ways to access/change the value of self.foo:
direct access a.foo
inner dict a.__dict__['foo']
get and set a.__get__ and a.__set__,of course there two are pre-defined methods.
getattribute a.__getattribute__
__getattr__ and __setattr__
maybe more.
While reading source code, I always get lost of what's their ultimate access order? When I use a.foo, how do I know which method/attribute will get called actually?

I found out this great post that has a detailed explanation on object/class attribute lookup.
For object attribute lookup:
Assuming Class is the class and instance is an instance of Class, evaluating instance.foobar roughly equates to this:
Call the type slot for Class.__getattribute__ (tp_getattro). The default does this:
Does Class.__dict__ have a foobar item that is a data descriptor ?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, Class).
Does instance.__dict__ have a 'foobar' item in it?
If yes, return instance.__dict__['foobar'].
Does Class.__dict__ have a foobar item that is not a data descriptor [9]?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, klass). [6]
If the attribute still wasn't found, and there's a Class.__getattr__, call Class.__getattr__('foobar').
There is an illustrated image for this:
Please do check out the original blog if interested which gives a outstanding explanation on python class, attribute lookup, and metaclass.

bar = a.foo...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__get__() if defined on A.
The returned value would then be assigned to bar.
a.foo = bar...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__set__(bar) if defined on A.

Related

What is the use of class attribute which is a method/function

In Python when we define class all its members including variables and methods also becomes attributes of that class. In following example MyClass1.a and MyClass1.mydef1 are attributes of class MyClass1.
class MyClass1:
a = 10
def mydef1(self):
return 0
ins1 = MyClass1() # create instance
print(MyClass1.a) # access class attribute which is class variable
print(MyClass1.mydef1) # No idea what to do with it so just printing
print(ins1.mydef1) # No idea what to do with it so just printing
Output
10
<function MyClass1.mydef1 at 0x0000000002122EA0>
<bound method MyClass1.mydef1 of <__main__.MyClass1 object at 0x000000000212D0F0>>
Here attribute a is a variable and it can be used like any other variable.
But mydef1 is a method, if it is not invoked and just used like MyClass1.mydef1 or ins1.mydef1, it returns object for that method(correct me if I am wrong).
So my question is, what can we do with the Class/instance methods without invoking it? Are there any use cases for it or is it just good to know thing?
An attribute of a class that happens to be a function becomes a method for instances or that class:
inst.foo(params, ...)
is internally translated into:
cls.foo(inst, params, ...)
That means that what is actually invoked is the attribute from the class of the instance, and the instance itself is prepended to the argument list. It is just Python syntax to invoke methods on objects.
In your example the correct uses would be:
print(MyClass1.mydef1(ins1)) # prints 0
print(ins1.mydef1()) # also prints 0
Well instance methods can be called with the appropriate parameters of course:
print(ins1.mydef1()) # no parameters, so empty parenthesis, this call should print "0" in your example instead of the method description
If you use it without the parenthesis, you are playing with reference to the function, I don't think you can have any use of it, except checking the list of methods available in a class or something like that.

What is the python attribute get and set order?

Python provides us many possibilities on instance/class attribute, for example:
class A(object):
def __init__(self):
self.foo = "hello"
a = A()
There are many ways to access/change the value of self.foo:
direct access a.foo
inner dict a.__dict__['foo']
get and set a.__get__ and a.__set__,of course there two are pre-defined methods.
getattribute a.__getattribute__
__getattr__ and __setattr__
maybe more.
While reading source code, I always get lost of what's their ultimate access order? When I use a.foo, how do I know which method/attribute will get called actually?
I found out this great post that has a detailed explanation on object/class attribute lookup.
For object attribute lookup:
Assuming Class is the class and instance is an instance of Class, evaluating instance.foobar roughly equates to this:
Call the type slot for Class.__getattribute__ (tp_getattro). The default does this:
Does Class.__dict__ have a foobar item that is a data descriptor ?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, Class).
Does instance.__dict__ have a 'foobar' item in it?
If yes, return instance.__dict__['foobar'].
Does Class.__dict__ have a foobar item that is not a data descriptor [9]?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, klass). [6]
If the attribute still wasn't found, and there's a Class.__getattr__, call Class.__getattr__('foobar').
There is an illustrated image for this:
Please do check out the original blog if interested which gives a outstanding explanation on python class, attribute lookup, and metaclass.
bar = a.foo...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__get__() if defined on A.
The returned value would then be assigned to bar.
a.foo = bar...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__set__(bar) if defined on A.

Class variables in python not found in __dict__

There is a code:
class C():
a=1
def f(self):
print "f func"
a=C()
print a.a
a.f()
>>> 1
>>> f func
And when i trying to get a.__dict__ or vars(a), it shows me just {}.
But
a.b=123
print a.__dict__
>>> {'b': 123}
I don't understand, why it is.
Looking at a.__dict__ or vars(a) gives you attributes of a, which is an instance. That instance initially has no attributes of its own. The attribute a that you created in your class definition is an attribute of the class itself, not of its instances. Later when you do a.b = 123, you create an attribute just on the instance, so you can see it in the instance __dict__. You will see the attribute a if you look at C.__dict__.
When you do print a.a, Python dynamically finds the attribute a on the class C. It sees that the instance doesn't have an attribute a, so it looks on the class and finds one there. That is the value that is printed. The class attribute is not "copied" to the instance when the instance is created; rather, every individual time you try to read the instance attribute, Python checks to see if it exists on the instance, and if not it looks it up on the class.
>>> C.a
1
>>> vars(C)['a']
1
(The whole vars dictionary for a class is rather long.)
Like your title says, it's a class variable. It belongs to the class, not the object. Python is doing some special logic behind the scenes to look on the type object for you when you call a.a. I'm not an expert, but I suspect it's finding a the same way it would find a method. In languages such as Java, I believe this sort of usage is discouraged. I don't know if it's discouraged in Python.

How does attribute resolution work in Python?

Consider the following code:
class A(object):
def do(self):
print self.z
class B(A):
def __init__(self, y):
self.z = y
b = B(3)
b.do()
Why does this work? When executing b = B(3), attribute z is set. When b.do() is called, Python's MRO finds the do function in class A. But why is it able to access an attribute defined in a subclass?
Is there a use case for this functionality? I would love an example.
It works in a pretty simple way: when a statement is executed that sets an attribute, it is set. When a statement is executed that reads an attribute, it is read. When you write code that reads an attribute, Python does not try to guess whether the attribute will exist when that code is executed; it just waits until the code actually is executed, and if at that time the attribute doesn't exist, then you'll get an exception.
By default, you can always set any attribute on an instance of a user-defined class; classes don't normally define lists of "allowed" attributes that could be set (although you can make that happen too), they just actually set attributes. Of course, you can only read attributes that exist, but again, what matters is whether they exist when you actually try to read them. So it doesn't matter if an attribute exists when you define a function that tries to read it; it only matters when (or if) you actually call that function.
In your example, it doesn't matter that there are two classes, because there is only one instance. Since you only create one instance and call methods on one instance, the self in both methods is the same object. First __init__ is run and it sets the attribute on self. Then do is run and it reads the attribute from the same self. That's all there is to it. It doesn't matter where the attribute is set; once it is set on the instance, it can be accessed from anywhere: code in a superclass, subclass, other class, or not in any class.
Since new attributes can be added to any object at any time, attribute resolution happens at execution time, not compile time. Consider this example which may be a bit more instructive, derived from yours:
class A(object):
def do(self):
print(self.z) # references an attribute which we have't "declared" in an __init__()
#make a new A
aa = A()
# this next line will error, as you would expect, because aa doesn't have a self.z
aa.do()
# but we can make it work now by simply doing
aa.z = -42
aa.do()
The first one will squack at you, but the second will print -42 as expected.
Python objects are just dictionaries. :)
When retrieving an attribute from an object (print self.attrname) Python follows these steps:
If attrname is a special (i.e. Python-provided) attribute for objectname, return it.
Check objectname.__class__.__dict__ for attrname. If it exists and is a data-descriptor, return the descriptor result. Search all bases of objectname.__class__ for the same case.
Check objectname.__dict__ for attrname, and return if found. If objectname is a class, search its bases too. If it is a class and a descriptor exists in it or its bases, return the descriptor result.
Check objectname.__class__.__dict__ for attrname. If it exists and is a non-data descriptor, return the descriptor result. If it exists, and is not a descriptor, just return it. If it exists and is a data descriptor, we shouldn't be here because we would have returned at point 2. Search all bases of objectname.__class__ for same case.
Raise AttributeError
Source
Understanding get and set and Python descriptors
Since you instanciated a B object, B.__init__ was invoked and added an attribute z. This attribute is now present in the object. It's not some weird overloaded magical shared local variable of B methods that somehow becomes inaccessible to code written elsewhere. There's no such thing. Neither does self become a different object when it's passed to a superclass' method (how's polymorphism supposed to work if that happens?).
There's also no such thing as a declaration that A objects have no such object (try o = A(); a.z = whatever), and neither is self in do required to be an instance of A1. In fact, there are no declarations at all. It's all "go ahead and try it"; that's kind of the definition of a dynamic language (not just dynamic typing).
That object's z attribute present "everywhere", all the time2, regardless of the "context" from which it is accessed. It never matters where code is defined for the resolution process, or for several other behaviors3. For the same reason, you can access a list's methods despite not writing C code in listobject.c ;-) And no, methods aren't special. They are just objects too (instances of the type function, as it happens) and are involved in exactly the same lookup sequence.
1 This is a slight lie; in Python 2, A.do would be "bound method" object which in fact throws an error if the first argument doesn't satisfy isinstance(A, <first arg>).
2 Until it's removed with del or one of its function equivalents (delattr and friends).
3 Well, there's name mangling, and in theory, code could inspect the stack, and thereby the caller code object, and thereby the location of its source code.

Python Suite, Package, Module, TestCase and TestSuite differences

Best Guess:
method - def(self, maybeSomeVariables); lines of code which achieve some purpose
Function - same as method but returns something
Class - group of methods/functions
Module - a script, OR one or more classes. Basically a .py file.
Package - a folder which has modules in, and also a __init__.py file in there.
Suite - Just a word that gets thrown around a lot, by convention
TestCase - unittest's equivalent of a function
TestSuite - unittest's equivalent of a Class (or Module?)
My question is: Is this completely correct, and did I miss any hierarchical building blocks from that list?
I feel that you're putting in differences that don't actually exist. There isn't really a hierarchy as such. In python everything is an object. This isn't some abstract notion, but quite fundamental to how you should think about constructs you create when using python. An object is just a bunch of other objects. There is a slight subtlety in whether you're using new-style classes or not, but in the absence of a good reason otherwise, just use and assume new-style classes. Everything below is assuming new-style classes.
If an object is callable, you can call it using the calling syntax of a pair of braces, with the arguments inside them: my_callable(arg1, arg2). To be callable, an object needs to implement the __call__ method (or else have the correct field set in its C level type definition).
In python an object has a type associated with it. The type describes how the object was constructed. So, for example, a list object is of type list and a function object is of type function. The types themselves are of type type. You can find the type by using the built-in function type(). A list of all the built-in types can be found in the python documentation. Types are actually callable objects, and are used to create instances of a given type.
Right, now that's established, the nature of a given object is defined by it's type. This describes the objects of which it comprises. Coming back to your questions then:
Firstly, the bunch of objects that make up some object are called the attributes of that object. These attributes can be anything, but they typically consist of methods and some way of storing state (which might be types such as int or list).
A function is an object of type function. Crucially, that means it has the __call__ method as an attribute which makes it a callable (the __call__ method is also an object that itself has the __call__ method. It's __call__ all the way down ;)
A class, in the python world, can be considered as a type, but typically is used to refer to types that are not built-in. These objects are used to create other objects. You can define your own classes with the class keyword, and to create a class which is new-style you must inherit from object (or some other new-style class). When you inherit, you create a type that acquires all the characteristics of the parent type, and then you can overwrite the bits you want to (and you can overwrite any bits you want!). When you instantiate a class (or more generally, a type) by calling it, another object is returned which is created by that class (how the returned object is created can be changed in weird and crazy ways by modifying the class object).
A method is a special type of function that is called using the attribute notation. That is, when it is created, 2 extra attributes are added to the method (remember it's an object!) called im_self and im_func. im_self I will describe in a few sentences. im_func is a function that implements the method. When the method is called, like, for example, foo.my_method(10), this is equivalent to calling foo.my_method.im_func(im_self, 10). This is why, when you define a method, you define it with the extra first argument which you apparently don't seem to use (as self).
When you write a bunch of methods when defining a class, these become unbound methods. When you create an instance of that class, those methods become bound. When you call an bound method, the im_self argument is added for you as the object in which the bound method resides. You can still call the unbound method of the class, but you need to explicitly add the class instance as the first argument:
class Foo(object):
def bar(self):
print self
print self.bar
print self.bar.im_self # prints the same as self
We can show what happens when we call the various manifestations of the bar method:
>>> a = Foo()
>>> a.bar()
<__main__.Foo object at 0x179b610>
<bound method Foo.bar of <__main__.Foo object at 0x179b610>>
<__main__.Foo object at 0x179b610>
>>> Foo.bar()
TypeError: unbound method bar() must be called with Foo instance as first argument (got nothing instead)
>>> Foo.bar(a)
<__main__.Foo object at 0x179b610>
<bound method Foo.bar of <__main__.Foo object at 0x179b610>>
<__main__.Foo object at 0x179b610>
Bringing all the above together, we can define a class as follows:
class MyFoo(object):
a = 10
def bar(self):
print self.a
This generates a class with 2 attributes: a (which is an integer of value 10) and bar, which is an unbound method. We can see that MyFoo.a is just 10.
We can create extra attributes at run time, both within the class methods, and outside. Consider the following:
class MyFoo(object):
a = 10
def __init__(self):
self.b = 20
def bar(self):
print self.a
print self.b
def eep(self):
print self.c
__init__ is just the method that is called immediately after an object has been created from a class.
>>> foo = Foo()
>>> foo.bar()
10
20
>>> foo.eep()
AttributeError: 'MyFoo' object has no attribute 'c'
>>> foo.c = 30
>>> foo.eep()
30
This example shows 2 ways of adding an attribute to a class instance at run time (that is, after the object has been created from it's class).
I hope you can see then, that TestCase and TestSuite are just classes that are used to create test objects. There's nothing special about them except that they happen to have some useful features for writing tests. You can subclass and overwrite them to your heart's content!
Regarding your specific point, both methods and functions can return anything they want.
Your description of module, package and suite seems pretty sound. Note that modules are also objects!

Categories