We Keep Coding

Why does super()() not do the same as super().__init__()?

I have been wondering this for a while now, and I hope this isn't a stupid question with an obvious answer I'm not realizing: Why can't I just call the __init__ method of super() like super()()? I have to call the method like this instead: super().__init__()
Here is an example that gets a TypeError: 'super' object is not callable error when I run it (specifically on line 6 that comes from line 3 in __init__):
class My_Int(int):
def __init__(self,value,extr_data=None):
super()(value)
self.extr_data=extr_data
x=My_Int(3)
Doesn't super() get the inherited class int making super()(value) the same as int(value)?
Furthermore, why can't I use the len function with super() when inheriting from the class list? Doesn't len() do the same as __len__()?
class My_List(list):
def some_method1(self):
print(len(super()))
def some_method2(self):
print(super().__len__())
x=My_List((1,2,3,4))
x.some_method2()
x.some_method1()
This example prints 4 and then an error as expected. Here is the output exactly:
4
Traceback (most recent call last):
File "/home/user/test.py", line 11, in <module>
x.some_method1()
File "/home/user/test.py", line 3, in some_method1
print(len(super()))
TypeError: object of type 'super' has no len()
Notice I called some_method2 before calling some_method1 (sorry for the confusion).
Am I missing something obvious here?
P.S. Thanks for all the help!

super() objects can't intercept most special method calls, because they bypass the instance and look up the method on the type directly, and they don't want to implement all the special methods when many of them won't apply for any given usage. This case gets weirder, super()() would try to lookup a __call__ method on the super type itself, and pass it the super instance.
They don't do this because it's ambiguous, and not particularly explicit. Does super()() mean invoke the super class's __init__? Its __call__? What if we're in a __new__ method, do you invoke __new__, __init__ or both? Does this mean all super uses must implicitly know which method they're called in (even more magical than knowing the class they were defined in and the self passed when constructed with zero arguments)?
Rather than deal with all this, and to avoid implementing all the special methods on super just so it can delegate them if they exist on the instance in question, they required you to explicitly specify the special method you intend to call.

Delegation in Python [duplicate]

I am trying to understand when to define __getattr__ or __getattribute__. The python documentation mentions __getattribute__ applies to new-style classes. What are new-style classes?

A key difference between __getattr__ and __getattribute__ is that __getattr__ is only invoked if the attribute wasn't found the usual ways. It's good for implementing a fallback for missing attributes, and is probably the one of two you want.
__getattribute__ is invoked before looking at the actual attributes on the object, and so can be tricky to implement correctly. You can end up in infinite recursions very easily.
New-style classes derive from object, old-style classes are those in Python 2.x with no explicit base class. But the distinction between old-style and new-style classes is not the important one when choosing between __getattr__ and __getattribute__.
You almost certainly want __getattr__.

Lets see some simple examples of both __getattr__ and __getattribute__ magic methods.
__getattr__
Python will call __getattr__ method whenever you request an attribute that hasn't already been defined. In the following example my class Count has no __getattr__ method. Now in main when I try to access both obj1.mymin and obj1.mymax attributes everything works fine. But when I try to access obj1.mycurrent attribute -- Python gives me AttributeError: 'Count' object has no attribute 'mycurrent'
class Count():
def __init__(self,mymin,mymax):
self.mymin=mymin
self.mymax=mymax
obj1 = Count(1,10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.mycurrent) --> AttributeError: 'Count' object has no attribute 'mycurrent'
Now my class Count has __getattr__ method. Now when I try to access obj1.mycurrent attribute -- python returns me whatever I have implemented in my __getattr__ method. In my example whenever I try to call an attribute which doesn't exist, python creates that attribute and sets it to integer value 0.
class Count:
def __init__(self,mymin,mymax):
self.mymin=mymin
self.mymax=mymax
def __getattr__(self, item):
self.__dict__[item]=0
return 0
obj1 = Count(1,10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.mycurrent1)
__getattribute__
Now lets see the __getattribute__ method. If you have __getattribute__ method in your class, python invokes this method for every attribute regardless whether it exists or not. So why do we need __getattribute__ method? One good reason is that you can prevent access to attributes and make them more secure as shown in the following example.
Whenever someone try to access my attributes that starts with substring 'cur' python raises AttributeError exception. Otherwise it returns that attribute.
class Count:
def __init__(self,mymin,mymax):
self.mymin=mymin
self.mymax=mymax
self.current=None
def __getattribute__(self, item):
if item.startswith('cur'):
raise AttributeError
return object.__getattribute__(self,item)
# or you can use ---return super().__getattribute__(item)
obj1 = Count(1,10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.current)
Important: In order to avoid infinite recursion in __getattribute__ method, its implementation should always call the base class method with the same name to access any attributes it needs. For example: object.__getattribute__(self, name) or super().__getattribute__(item) and not self.__dict__[item]
IMPORTANT
If your class contain both getattr and getattribute magic methods then __getattribute__ is called first. But if __getattribute__ raises
AttributeError exception then the exception will be ignored and __getattr__ method will be invoked. See the following example:
class Count(object):
def __init__(self,mymin,mymax):
self.mymin=mymin
self.mymax=mymax
self.current=None
def __getattr__(self, item):
self.__dict__[item]=0
return 0
def __getattribute__(self, item):
if item.startswith('cur'):
raise AttributeError
return object.__getattribute__(self,item)
# or you can use ---return super().__getattribute__(item)
# note this class subclass object
obj1 = Count(1,10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.current)

This is just an example based on Ned Batchelder's explanation.
__getattr__ example:
class Foo(object):
def __getattr__(self, attr):
print "looking up", attr
value = 42
self.__dict__[attr] = value
return value
f = Foo()
print f.x
#output >>> looking up x 42
f.x = 3
print f.x
#output >>> 3
print ('__getattr__ sets a default value if undefeined OR __getattr__ to define how to handle attributes that are not found')
And if same example is used with __getattribute__ You would get >>> RuntimeError: maximum recursion depth exceeded while calling a Python object

New-style classes inherit from object, or from another new style class:
class SomeObject(object):
pass
class SubObject(SomeObject):
pass
Old-style classes don't:
class SomeObject:
pass
This only applies to Python 2 - in Python 3 all the above will create new-style classes.
See 9. Classes (Python tutorial), NewClassVsClassicClass and What is the difference between old style and new style classes in Python? for details.

New-style classes are ones that subclass "object" (directly or indirectly). They have a __new__ class method in addition to __init__ and have somewhat more rational low-level behavior.
Usually, you'll want to override __getattr__ (if you're overriding either), otherwise you'll have a hard time supporting "self.foo" syntax within your methods.
Extra info: http://www.devx.com/opensource/Article/31482/0/page/4

getattribute: Is used to retrieve an attribute from an instance. It captures every attempt to access an instance attribute by using dot notation or getattr() built-in function.
getattr: Is executed as the last resource when attribute is not found in an object. You can choose to return a default value or to raise AttributeError.
Going back to the __getattribute__ function; if the default implementation was not overridden; the following checks are done when executing the method:
Check if there is a descriptor with the same name (attribute name) defined in any class in the MRO chain (method object resolution)
Then looks into the instance’s namespace
Then looks into the class namespace
Then into each base’s namespace and so on.
Finally, if not found, the default implementation calls the fallback getattr() method of the instance and it raises an AttributeError exception as default implementation.
This is the actual implementation of the object.__getattribute__ method:
.. c:function:: PyObject* PyObject_GenericGetAttr(PyObject *o,
PyObject *name) Generic attribute getter function that is meant to
be put into a type object's tp_getattro slot. It looks for a
descriptor in the dictionary of classes in the object's MRO as well
as an attribute in the object's :attr:~object.dict (if
present). As outlined in :ref:descriptors, data descriptors take
preference over instance attributes, while non-data descriptors
don't. Otherwise, an :exc:AttributeError is raised.

I find that no one mentions this difference:
__getattribute__ has a default implementation, but __getattr__ does not.
class A:
pass
a = A()
a.__getattr__ # error
a.__getattribute__ # return a method-wrapper
This has a clear meaning: since __getattribute__ has a default implementation, while __getattr__ not, clearly python encourages users to implement __getattr__.

In reading through Beazley & Jones PCB, I have stumbled on an explicit and practical use-case for __getattr__ that helps answer the "when" part of the OP's question. From the book:
"The __getattr__() method is kind of like a catch-all for attribute lookup. It's a method that gets called if code tries to access an attribute that doesn't exist." We know this from the above answers, but in PCB recipe 8.15, this functionality is used to implement the delegation design pattern. If Object A has an attribute Object B that implements many methods that Object A wants to delegate to, rather than redefining all of Object B's methods in Object A just to call Object B's methods, define a __getattr__() method as follows:
def __getattr__(self, name):
return getattr(self._b, name)
where _b is the name of Object A's attribute that is an Object B. When a method defined on Object B is called on Object A, the __getattr__ method will be invoked at the end of the lookup chain. This would make code cleaner as well, since you do not have a list of methods defined just for delegating to another object.

Getting memory location from getattr()

from my research, I understand that using getattr() allows one to get a method on a certain object such as:
getattr(obj, method)
is the equivalent to
obj.method()
However, I tried to use it in my code and the statement returns a memory location:
<bound method _Class.method of <Class instance at 0x7f412df0c320>>
I don't understand what I am doing wrong. Any help is appreciated.

Methods are just attributes, and getattr() retrieves attributes. As such
getattr(obj, 'method')
is the same as
obj.method
so without the () call primary, so no call to the method object made. Just add a call expression, to call the result of the getattr() method:
getattr(obj, 'method')()
__getattr__ is a hook method to allow a class to customise how missing attributes are handled. You should generally not need to call it directly.

Python 3 __getattribute__ vs dot access behaviour

I read a bit on python's object attribute lookup (here: https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/#object-attribute-lookup).
Seems pretty straight forward, so I tried it out (python3):
class A:
def __getattr__(self, attr):
return (1,2,3)
a = A()
a.foobar #returns (1,2,3) as expected
a.__getattribute__('foobar') # raises AttributeError
My question is, aren't the two supposed to be identical?
Why does the second one raise an attribute error?
So apparently the answer is that the logic for a.foobar IS different from the logic for a.__getattribute("foobar"). According to the data model: a.foobar calls a.__getattribute("foobar") and if it raises an AttributeError, it calls a.-__getattr__('foobar')
So it seems the article has a mistake in their diagram. Is this correct?
And another question: Where does the real logic for a.foobar sit? I thought it was in __getattribute__ but apparently not entirely.
Edit:
Not a duplicate of
Difference between __getattr__ vs __getattribute__.
I am asking here what is the different between object.foo and object.__getattribute__("foo"). This is different from __getattr__ vs __getatribute__ which is trivial...

It's easy to get the impression that __getattribute__ is responsible for more than it really is. thing.attr doesn't directly translate to thing.__getattribute__('attr'), and __getattribute__ is not responsible for calling __getattr__.
The fallback to __getattr__ happens in the part of the attribute access machinery that lies outside __getattribute__. The attribute lookup process works like this:
Find the __getattribute__ method through a direct search of the object's type's MRO, bypassing the regular attribute lookup process.
Try __getattribute__.
If __getattribute__ returned something, the attribute lookup process is complete, and that's the attribute value.
If __getattribute__ raised a non-AttributeError, the attribute lookup process is complete, and the exception propagates out of the lookup.
Otherwise, __getattribute__ raised an AttributeError. The lookup continues.
Find the __getattr__ method the same way we found __getattribute__.
If there is no __getattr__, the attribute lookup process is complete, and the AttributeError from __getattribute__ propagates.
Try __getattr__, and return or raise whatever __getattr__ returns or raises.
At least, in terms of the language semantics, it works like that. In terms of the low-level implementation, some of these steps may be optimized out in cases where they're unnecessary, and there are C hooks like tp_getattro that I haven't described. You don't need to worry about that kind of thing unless you want to dive into the CPython interpreter source code.

When does __getattribute__ not get involved in attribute lookup?

Consider the following:
class A(object):
def __init__(self):
print 'Hello!'
def foo(self):
print 'Foo!'
def __getattribute__(self, att):
raise AttributeError()
a = A() # Works, prints "Hello!"
a.foo() # throws AttributeError as expected
The implementation of __getattribute__ obviously fails all lookups. My questions:
Why is it still possible to instantiate an object? I would have expected the lookup of the __init__ method itself to fail as well.
What's the list of attributes that are not subject to __getattribute__?

The implementation of __getattribute__ obviously fails all lookups
Let's say it fails for all vanilla lookups.
So how did __getattribute__ itself get called in the first place since it is also an attribute of the class?
An attribute would refer to any name following a dot. So to get an attribute of a class instance, __getattribute__ is summoned unconditionally when you try to access that attribute (through dot reference).
However magic methods like __init__ are part of the language construct and so are not directly invoked (via dot reference) since they are implemented as part of the language.
Why is it still possible to instantiate an object?
When you do:
a = A()
The __init__ method gets called behind the scenes, but not via a vanilla lookup. The language handles this. Same applies to other methods like __setattr__, __delattr__, __getattribute__ also and others.
But if you directly called __init__:
a.__init__()
It would raise an error. Eh, this does not make any sense since the class is already initialized.
More subtly, if you tried to access __getattribute__ from your class instance via a dot reference:
a.__getattribute__
it would also raise an AttributeError; the language invocation of the same method attempted to lookup on the attribute __getattribute__, but failed with error.
What's the list of attributes that are not subject to
__getattribute__?
Summarily, __getattribute__ comes play when you try to access any attribute via dot reference. As long as you don't try to explicitly call a magic method, __getattribute__ will not be called.