Python provides us many possibilities on instance/class attribute, for example:
class A(object):
def __init__(self):
self.foo = "hello"
a = A()
There are many ways to access/change the value of self.foo:
direct access a.foo
inner dict a.__dict__['foo']
get and set a.__get__ and a.__set__,of course there two are pre-defined methods.
getattribute a.__getattribute__
__getattr__ and __setattr__
maybe more.
While reading source code, I always get lost of what's their ultimate access order? When I use a.foo, how do I know which method/attribute will get called actually?
I found out this great post that has a detailed explanation on object/class attribute lookup.
For object attribute lookup:
Assuming Class is the class and instance is an instance of Class, evaluating instance.foobar roughly equates to this:
Call the type slot for Class.__getattribute__ (tp_getattro). The default does this:
Does Class.__dict__ have a foobar item that is a data descriptor ?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, Class).
Does instance.__dict__ have a 'foobar' item in it?
If yes, return instance.__dict__['foobar'].
Does Class.__dict__ have a foobar item that is not a data descriptor [9]?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, klass). [6]
If the attribute still wasn't found, and there's a Class.__getattr__, call Class.__getattr__('foobar').
There is an illustrated image for this:
Please do check out the original blog if interested which gives a outstanding explanation on python class, attribute lookup, and metaclass.
bar = a.foo...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__get__() if defined on A.
The returned value would then be assigned to bar.
a.foo = bar...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__set__(bar) if defined on A.
Related
Python noob here, trying to work more with classes and have broken my code down into a more simple example of what I am working with:
Lets say I have a class, with an instance and an attribute associated with that instance
class Foo(object):
def __init__(self, attribute=None):
self.attribute = attribute
def dosomething(self):
print('I did a thing!')
a = Foo()
a.attribute='bar'
Now, running the method from the instance 'a' works as expected
a.dosomething()
Out:
I did a thing!
However, trying to call the method from an attribute of 'a' results in error
a.attribute='bar'
a.attribute.dosomething()
AttributeError: 'str' object has no attribute 'dosomething'
This results in an attribute error, which is expected. 'a.attribute' is no longer a class type of 'Foo' but rather a class type of 'str'. My question is, can I do something inside the class to allow the attribute to use the class methods?
Furthermore, since I have not found any results for this during my interwebs searches, Im assuming that it likely is not recommended and probably not pythonic? (just a guess). I am open to suggestions on how to make this better, but ideally, I would like to keep the attribute the way it is while still being able to call the class methods. The dot syntax of attributes for classes really helps me keep things organized, but it is useless to me if it cannot call the methods in the class from which it originates.
Thanks in advance!
No, this doesn't work. And it really doesn't make sense if you understand how classes work.
That a.attribute is just a name for the string 'bar'. That string doesn't know anything about a, and shouldn't.1 There could be plenty of other names for the same string. For example:
>>> bar = 'bar'
>>> a.attribute = bar
>>> bar.dosomething()
AttributeError: 'str' object has no attribute 'dosomething'
That's exactly what you'd expect, right? But…
>>> a.attribute is bar
True
Since bar is the same object as a.attribute, it clearly has to do the same thing:
>>> a.attribute.dosomething()
AttributeError: 'str' object has no attribute 'dosomething'
1. There are some special cases where something like this would be useful, but they're solved by having the attribute not just be a string, but instead be some kind of smart object that works kind of like a string but also knows its class, instance, or parent. See the enum module in the stdlib for an example—and click on the source code link to see how it's implemented.
In this example the attribute is is a string when you execute a.attribute='bar' so if you were to do type(a.attribute) it would return string. So Python is looking for the attribute dosomething() in the str class and not finding it.
What you seem to be trying to do doesn't really make sense. At a guess, it seems like you've mixed up some concepts related to inheritance with your understanding of python's dot syntax for member reference, when these have nothing to do with each other.
You wrote
'a.attribute' is no longer a class type of 'Foo' but rather a class type of 'str'.
and this indicates a fundamental misunderstanding of what is going on. No state has changed – a is still the instance a, of type Foo. It has a member attribute, which is a string. The expression a.attribute is merely pointing to that string. If you need to call a method on a, you can still do so anywhere you were trying to call it on a.attribute:
>>> a.attribute = 'bar'
>>> a.doSomething()
I did a thing!
The common scenario in which an object of one type invokes a method that is not defined in its class is inheritance:
class Foo(object):
def superclass_method(self):
print "Hi"
class Bar(Foo):
pass
b = Bar()
b.superclass_method() # prints "Hi"
In this case, we say that b, an instance of Bar, is a Foo as well. Therefore any method that Foo defines, a Bar instance has access to.
If you tell us a bit more about what you're trying to accomplish, I can be more specific about whether and how you should be using inheritance. But the dot notation is just a way to talk about values attached to objects. It's literally just looking in the left object's dictionary for a value associated with the name on the right – foo.bar is equivalent to getattr(foo, 'bar').
Python provides us many possibilities on instance/class attribute, for example:
class A(object):
def __init__(self):
self.foo = "hello"
a = A()
There are many ways to access/change the value of self.foo:
direct access a.foo
inner dict a.__dict__['foo']
get and set a.__get__ and a.__set__,of course there two are pre-defined methods.
getattribute a.__getattribute__
__getattr__ and __setattr__
maybe more.
While reading source code, I always get lost of what's their ultimate access order? When I use a.foo, how do I know which method/attribute will get called actually?
I found out this great post that has a detailed explanation on object/class attribute lookup.
For object attribute lookup:
Assuming Class is the class and instance is an instance of Class, evaluating instance.foobar roughly equates to this:
Call the type slot for Class.__getattribute__ (tp_getattro). The default does this:
Does Class.__dict__ have a foobar item that is a data descriptor ?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, Class).
Does instance.__dict__ have a 'foobar' item in it?
If yes, return instance.__dict__['foobar'].
Does Class.__dict__ have a foobar item that is not a data descriptor [9]?
If yes, return the result of Class.__dict__['foobar'].__get__(instance, klass). [6]
If the attribute still wasn't found, and there's a Class.__getattr__, call Class.__getattr__('foobar').
There is an illustrated image for this:
Please do check out the original blog if interested which gives a outstanding explanation on python class, attribute lookup, and metaclass.
bar = a.foo...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__get__() if defined on A.
The returned value would then be assigned to bar.
a.foo = bar...
invokes a.__getattribute__('foo')
which in turn by default looks up a.__dict__['foo']
or invokes foo's .__set__(bar) if defined on A.
There is a code:
class C():
a=1
def f(self):
print "f func"
a=C()
print a.a
a.f()
>>> 1
>>> f func
And when i trying to get a.__dict__ or vars(a), it shows me just {}.
But
a.b=123
print a.__dict__
>>> {'b': 123}
I don't understand, why it is.
Looking at a.__dict__ or vars(a) gives you attributes of a, which is an instance. That instance initially has no attributes of its own. The attribute a that you created in your class definition is an attribute of the class itself, not of its instances. Later when you do a.b = 123, you create an attribute just on the instance, so you can see it in the instance __dict__. You will see the attribute a if you look at C.__dict__.
When you do print a.a, Python dynamically finds the attribute a on the class C. It sees that the instance doesn't have an attribute a, so it looks on the class and finds one there. That is the value that is printed. The class attribute is not "copied" to the instance when the instance is created; rather, every individual time you try to read the instance attribute, Python checks to see if it exists on the instance, and if not it looks it up on the class.
>>> C.a
1
>>> vars(C)['a']
1
(The whole vars dictionary for a class is rather long.)
Like your title says, it's a class variable. It belongs to the class, not the object. Python is doing some special logic behind the scenes to look on the type object for you when you call a.a. I'm not an expert, but I suspect it's finding a the same way it would find a method. In languages such as Java, I believe this sort of usage is discouraged. I don't know if it's discouraged in Python.
Consider the following code:
class A(object):
def do(self):
print self.z
class B(A):
def __init__(self, y):
self.z = y
b = B(3)
b.do()
Why does this work? When executing b = B(3), attribute z is set. When b.do() is called, Python's MRO finds the do function in class A. But why is it able to access an attribute defined in a subclass?
Is there a use case for this functionality? I would love an example.
It works in a pretty simple way: when a statement is executed that sets an attribute, it is set. When a statement is executed that reads an attribute, it is read. When you write code that reads an attribute, Python does not try to guess whether the attribute will exist when that code is executed; it just waits until the code actually is executed, and if at that time the attribute doesn't exist, then you'll get an exception.
By default, you can always set any attribute on an instance of a user-defined class; classes don't normally define lists of "allowed" attributes that could be set (although you can make that happen too), they just actually set attributes. Of course, you can only read attributes that exist, but again, what matters is whether they exist when you actually try to read them. So it doesn't matter if an attribute exists when you define a function that tries to read it; it only matters when (or if) you actually call that function.
In your example, it doesn't matter that there are two classes, because there is only one instance. Since you only create one instance and call methods on one instance, the self in both methods is the same object. First __init__ is run and it sets the attribute on self. Then do is run and it reads the attribute from the same self. That's all there is to it. It doesn't matter where the attribute is set; once it is set on the instance, it can be accessed from anywhere: code in a superclass, subclass, other class, or not in any class.
Since new attributes can be added to any object at any time, attribute resolution happens at execution time, not compile time. Consider this example which may be a bit more instructive, derived from yours:
class A(object):
def do(self):
print(self.z) # references an attribute which we have't "declared" in an __init__()
#make a new A
aa = A()
# this next line will error, as you would expect, because aa doesn't have a self.z
aa.do()
# but we can make it work now by simply doing
aa.z = -42
aa.do()
The first one will squack at you, but the second will print -42 as expected.
Python objects are just dictionaries. :)
When retrieving an attribute from an object (print self.attrname) Python follows these steps:
If attrname is a special (i.e. Python-provided) attribute for objectname, return it.
Check objectname.__class__.__dict__ for attrname. If it exists and is a data-descriptor, return the descriptor result. Search all bases of objectname.__class__ for the same case.
Check objectname.__dict__ for attrname, and return if found. If objectname is a class, search its bases too. If it is a class and a descriptor exists in it or its bases, return the descriptor result.
Check objectname.__class__.__dict__ for attrname. If it exists and is a non-data descriptor, return the descriptor result. If it exists, and is not a descriptor, just return it. If it exists and is a data descriptor, we shouldn't be here because we would have returned at point 2. Search all bases of objectname.__class__ for same case.
Raise AttributeError
Source
Understanding get and set and Python descriptors
Since you instanciated a B object, B.__init__ was invoked and added an attribute z. This attribute is now present in the object. It's not some weird overloaded magical shared local variable of B methods that somehow becomes inaccessible to code written elsewhere. There's no such thing. Neither does self become a different object when it's passed to a superclass' method (how's polymorphism supposed to work if that happens?).
There's also no such thing as a declaration that A objects have no such object (try o = A(); a.z = whatever), and neither is self in do required to be an instance of A1. In fact, there are no declarations at all. It's all "go ahead and try it"; that's kind of the definition of a dynamic language (not just dynamic typing).
That object's z attribute present "everywhere", all the time2, regardless of the "context" from which it is accessed. It never matters where code is defined for the resolution process, or for several other behaviors3. For the same reason, you can access a list's methods despite not writing C code in listobject.c ;-) And no, methods aren't special. They are just objects too (instances of the type function, as it happens) and are involved in exactly the same lookup sequence.
1 This is a slight lie; in Python 2, A.do would be "bound method" object which in fact throws an error if the first argument doesn't satisfy isinstance(A, <first arg>).
2 Until it's removed with del or one of its function equivalents (delattr and friends).
3 Well, there's name mangling, and in theory, code could inspect the stack, and thereby the caller code object, and thereby the location of its source code.
Best Guess:
method - def(self, maybeSomeVariables); lines of code which achieve some purpose
Function - same as method but returns something
Class - group of methods/functions
Module - a script, OR one or more classes. Basically a .py file.
Package - a folder which has modules in, and also a __init__.py file in there.
Suite - Just a word that gets thrown around a lot, by convention
TestCase - unittest's equivalent of a function
TestSuite - unittest's equivalent of a Class (or Module?)
My question is: Is this completely correct, and did I miss any hierarchical building blocks from that list?
I feel that you're putting in differences that don't actually exist. There isn't really a hierarchy as such. In python everything is an object. This isn't some abstract notion, but quite fundamental to how you should think about constructs you create when using python. An object is just a bunch of other objects. There is a slight subtlety in whether you're using new-style classes or not, but in the absence of a good reason otherwise, just use and assume new-style classes. Everything below is assuming new-style classes.
If an object is callable, you can call it using the calling syntax of a pair of braces, with the arguments inside them: my_callable(arg1, arg2). To be callable, an object needs to implement the __call__ method (or else have the correct field set in its C level type definition).
In python an object has a type associated with it. The type describes how the object was constructed. So, for example, a list object is of type list and a function object is of type function. The types themselves are of type type. You can find the type by using the built-in function type(). A list of all the built-in types can be found in the python documentation. Types are actually callable objects, and are used to create instances of a given type.
Right, now that's established, the nature of a given object is defined by it's type. This describes the objects of which it comprises. Coming back to your questions then:
Firstly, the bunch of objects that make up some object are called the attributes of that object. These attributes can be anything, but they typically consist of methods and some way of storing state (which might be types such as int or list).
A function is an object of type function. Crucially, that means it has the __call__ method as an attribute which makes it a callable (the __call__ method is also an object that itself has the __call__ method. It's __call__ all the way down ;)
A class, in the python world, can be considered as a type, but typically is used to refer to types that are not built-in. These objects are used to create other objects. You can define your own classes with the class keyword, and to create a class which is new-style you must inherit from object (or some other new-style class). When you inherit, you create a type that acquires all the characteristics of the parent type, and then you can overwrite the bits you want to (and you can overwrite any bits you want!). When you instantiate a class (or more generally, a type) by calling it, another object is returned which is created by that class (how the returned object is created can be changed in weird and crazy ways by modifying the class object).
A method is a special type of function that is called using the attribute notation. That is, when it is created, 2 extra attributes are added to the method (remember it's an object!) called im_self and im_func. im_self I will describe in a few sentences. im_func is a function that implements the method. When the method is called, like, for example, foo.my_method(10), this is equivalent to calling foo.my_method.im_func(im_self, 10). This is why, when you define a method, you define it with the extra first argument which you apparently don't seem to use (as self).
When you write a bunch of methods when defining a class, these become unbound methods. When you create an instance of that class, those methods become bound. When you call an bound method, the im_self argument is added for you as the object in which the bound method resides. You can still call the unbound method of the class, but you need to explicitly add the class instance as the first argument:
class Foo(object):
def bar(self):
print self
print self.bar
print self.bar.im_self # prints the same as self
We can show what happens when we call the various manifestations of the bar method:
>>> a = Foo()
>>> a.bar()
<__main__.Foo object at 0x179b610>
<bound method Foo.bar of <__main__.Foo object at 0x179b610>>
<__main__.Foo object at 0x179b610>
>>> Foo.bar()
TypeError: unbound method bar() must be called with Foo instance as first argument (got nothing instead)
>>> Foo.bar(a)
<__main__.Foo object at 0x179b610>
<bound method Foo.bar of <__main__.Foo object at 0x179b610>>
<__main__.Foo object at 0x179b610>
Bringing all the above together, we can define a class as follows:
class MyFoo(object):
a = 10
def bar(self):
print self.a
This generates a class with 2 attributes: a (which is an integer of value 10) and bar, which is an unbound method. We can see that MyFoo.a is just 10.
We can create extra attributes at run time, both within the class methods, and outside. Consider the following:
class MyFoo(object):
a = 10
def __init__(self):
self.b = 20
def bar(self):
print self.a
print self.b
def eep(self):
print self.c
__init__ is just the method that is called immediately after an object has been created from a class.
>>> foo = Foo()
>>> foo.bar()
10
20
>>> foo.eep()
AttributeError: 'MyFoo' object has no attribute 'c'
>>> foo.c = 30
>>> foo.eep()
30
This example shows 2 ways of adding an attribute to a class instance at run time (that is, after the object has been created from it's class).
I hope you can see then, that TestCase and TestSuite are just classes that are used to create test objects. There's nothing special about them except that they happen to have some useful features for writing tests. You can subclass and overwrite them to your heart's content!
Regarding your specific point, both methods and functions can return anything they want.
Your description of module, package and suite seems pretty sound. Note that modules are also objects!