calling an overridden method from base class? - python

Dive into Python -
Guido, the original author of Python, explains method overriding this way: "Derived classes may override methods of their base classes. Because methods have no special privileges when calling other methods of the same object, a method of a base class that calls another method defined in the same base class, may in fact end up calling a method of a derived class that overrides it. (For C++ programmers: all methods in Python are effectively virtual.)" If that doesn't make sense to you (it confuses the hell out of me), feel free to ignore it. I just thought I'd pass it along.
I am trying to figure out an example for: a method of a base class that calls another method defined in the same base class, may in fact end up calling a method of a derived class that overrides it
class A:
def foo(self): print 'A.foo'
def bar(self): self.foo()
class B(A):
def foo(self): print 'B.foo'
if __name__ == '__main__':
a = A()
a.bar() # echoes A.foo
b = B()
b.bar() # echoes B.foo
... but both of these seem kind of obvious.
am I missing something that was hinted out in the quote?
UPDATE
edited typo of calling a.foo() (instead of a.bar())and b.foo() (instead of b.bar()) in the original code

Yes, you're missing this:
b.bar() # echoes B.foo
B has no bar method of its own, just the one inherited from A. A's bar calls self.foo, but in an instance of B ends up calling B's foo, and not A's foo.
Let's look at your quote again:
a method of a base class that calls
another method defined in the same
base class, may in fact end up calling
a method of a derived class that
overrides it
To translate:
bar (method of A, the base class)
calls self.foo, but may in fact end up
calling a method of the derived class
that overrides it (B.foo that
overrides A.foo)

Note, this won't work for private methods in Python 3.6:
class A:
def __foo(self): print 'A.foo'
def bar(self): self.__foo()
class B(A):
def __foo(self): print 'B.foo'
if __name__ == '__main__':
a = A()
a.bar() # echoes A.foo
b = B()
b.bar() # echoes A.foo, not B.foo
I spent an hour to find out the reason for this

Related

Calling super().method() vs. BaseClass.method(self)

There are two main ways for a derived class to call a base class's methods.
Base.method(self):
class Derived(Base):
def method(self):
Base.method(self)
...
or super().method():
class Derived(Base):
def method(self):
super().method()
...
Suppose I now do this:
obj = Derived()
obj.method()
As far as I know, both Base.method(self) and super().method() do the same thing. Both will call Base.method with a reference to obj. In particular, super() doesn't do the legwork to instantiate an object of type Base. Instead, it creates a new object of type super and grafts the instance attributes from obj onto it, then it dynamically looks up the right attribute from Base when you try to get it from the super object.
The super() method has the advantage of minimizing the work you need to do when you change the base for a derived class. On the other hand, Base.method uses less magic and may be simpler and clearer when a class inherits from multiple base classes.
Most of the discussions I've seen recommend calling super(), but is this an established standard among Python coders? Or are both of these methods widely used in practice? For example, answers to this stackoverflow question go both ways, but generally use the super() method. On the other hand, the Python textbook I am teaching from this semester only shows the Base.method approach.
Using super() implies the idea that whatever follows should be delegated to the base class, no matter what it is. It's about the semantics of the statement. Referring explicitly to Base on the other hand conveys the idea that Base was chosen explicitly for some reason (perhaps unknown to the reader), which might have its applications too.
Apart from that however there is a very practical reason for using super(), namely cooperative multiple inheritance. Suppose you've designed the following class hierarchy:
class Base:
def test(self):
print('Base.test')
class Foo(Base):
def test(self):
print('Foo.test')
Base.test(self)
class Bar(Base):
def test(self):
print('Bar.test')
Base.test(self)
Now you can use both Foo and Bar and everything works as expected. However these two classes won't work together in a multiple inheritance schema:
class Test(Foo, Bar):
pass
Test().test()
# Output:
# Foo.test
# Base.test
That last call to test skips over Bar's implementation since Foo didn't specify that it wants to delegate to the next class in method resolution order but instead explicitly specified Base. Using super() resolves this issue:
class Base:
def test(self):
print('Base.test')
class Foo(Base):
def test(self):
print('Foo.test')
super().test()
class Bar(Base):
def test(self):
print('Bar.test')
super().test()
class Test(Foo, Bar):
pass
Test().test()
# Output:
# Foo.test
# Bar.test
# Base.test

How to pass an instance method, that uses instance variables, as argument to another function?

I want to pass a method foo of an instance of a class A to another function run_function. The method foo will use instance variables of its class A.
Here is a minimal example, in which the instance variable of A is simply self.a_var, and foo just prints that variable.
class A:
def __init__(self,a_var):
self.a_var=a_var
def foo(self):
print(self.a_var)
class B:
def __init__(self, A):
self.A=A
# EDIT (comment to compile)
self.A.do_something()
def run_function(self):
self.A.foo()
def run_function_2(self, bar):
bar()
myA = A(42)
myA.foo()
# Current implementation
myB=B(myA)
myB.run_function()
# Better(?) implementation
myB.run_function_2(myA.foo)
At the moment I pass the instance myA of class A to the instance of B and explicitly call self.A.foo(). This forces the name of the function of Ato be foo. Which is stupid.
The better (?) implementation passes the function of the instance to run_function2. This works, but I am not sure if this is "safe".
Question:
Are there any loopholes that I don't see at the moment?
The important part is, that the method foo, that is passed, needs to access instance variables of the (its) class instance. So, will foo that is called inside run_function_2 always have access to all instance variables of myA?
Is there a better way to implement this?
EDIT: I forgot to add, that class B will always have an instance of A, since it has to do_something with that instance. Maybe that will change something(?). Sorry!
For your second implementation, have you considered the following:
>>> myAa = A(42)
>>> myAb = A(43)
>>> myB = B(myAb)
>>> myB.run_function_2(myAa.foo)
42
This might not be what you want. How about using getattr() and just passing in the desired method name:
>>> class C:
... def __init__(self, A):
... self.A = A
... def run_fct(self, bar):
... fct = getattr(self.A, bar)
... fct()
...
>>> myC = C(myAa)
>>> myC.run_fct('foo')
42
To answer your questions:
Any function executed in the context of an object instance will have access to the instance variables.
There may be a better way to implement this, you could try defining an interface for class A and other classes that might be like it. The you know that the function will always be called foo(). If not, I'd question why it is you need to have some object call an arbitrary method on another object. If you can give more concrete examples about what you're trying to do it would help.
The main difference between run_function and run_function_2 is that the former calls foo on the object that was given to the B() constructor. run_function_2 is independent of what object is saved as self.A; it just calls the function/method you give it. For example
class A:
def __init__(self,a_var):
self.a_var=a_var
def foo(self):
print(self.a_var)
class B:
def __init__(self, A):
self.A=A
def run_function(self):
self.A.foo()
def run_function_2(self, bar):
bar()
myA = A(42)
myB = B(myA)
myA2 = A(3.14)
myB.run_function()
myB.run_function_2(myA.foo)
myB.run_function_2(myA2.foo)
Output
42
42
3.14
Are there any loopholes that I don't see at the moment?
These two ways of calling methods are fine. Though I agree that function_run_2 is more convenient since it doesn't fix the method name, it makes you ask... what's the purpose of giving an A object to the B constructor in the first place if it's never used?
The important part is, that the method foo, that is passed, needs to access instance variables of the (its) class instance. So, will foo that is called inside run_function_2 always have access to all instance variables of myA?
Yes. run_function_2 arguments requires a function. In this case, you pass myA.foo, an object myA's method defined in class A. When you call foo inside run_function_2, you are only dealing with attributes variables of the instance myA; this is the idea of encapsulation in classes.
Is there a better way to implement this?
Answering also your question on safety, it's perfectly safe. Functions and methods are objects in Python, and they can be passed around like values. You're basically leaning on the idea of function currying or partial functions. See How do I pass a method as a parameter in Python. These two ways are fine.

What class is super(classname, instance) actually calling?

I've seen a bunch of the python method resolution order questions on Stack Overflow, many of which are excellently answered. I have one that does not quite fit.
When requesting super(MyClassName, self).method_name, I get a type that is not returned by the (single) parent class. Putting debug into the parent class shows that it isn't hit.
I would add some code snippets, but the codebase is massive. I have been into every class listed from MyClassName.__mro__ (which tells us what the method resolution order is) and NONE of them return the type I'm getting. So the question is...
What tool or attribute in Python can I use to find out what code is actually being called so that if this happens again I can easily find out what is actually being called? I ended up finding the solution, but I'd rather know how to tackle it in a less labour intensive manner.
You can use e.g. inspect.getmodule to, per its documentation:
Try to guess which module an object was defined in.
A simple example, with a.py:
class Parent(object):
def method(self):
return True
and b.py:
import inspect
from a import Parent
class Child(Parent):
def method(self):
parent_method = super(Child, self).method # get the parent method
print "inherited method defined in {}".format(
inspect.getmodule(parent_method), # and find out where it came from
)
return parent_method()
if __name__ == '__main__':
Child().method()
Running b.py gives the result:
Parent defined in <module 'a' from 'C:/Python27\a.py'>
I think you might be getting confused between what a bound method is, and what the method resolution order is.
The returned method still counts as a bound method of the class of the actual object, even if function the method derives from is found on the parent class. This is because the method has been bound to an instance of the child class as opposed to the parent class.
eg.
class A:
def f(self):
return "A"
class B(A):
def g(self):
return super().f
class C(B):
def f(self):
return "C"
c = C()
method = c.g()
print(method) # prints <bound method C.f of <__main__.C object at 0x02D4FA10>>
print(method()) # prints A
In this instance, c.g() returns the function A.f bound to an instance of C.
To find the actual function that the bound method will call just examine the __func__ attribute:
assert method.__func__ is A.f

Introspect calling object

How do I introspect A's instance from within b.func() (i.e. A's instance's self):
class A():
def go(self):
b=B()
b.func()
class B():
def func(self):
# Introspect to find the calling A instance here
In general we don't want that func to have access back to the calling instance of A because this breaks encapsulation. Inside of b.func you should have access to any args and kwargs passed, the state/attributes of the instance b (via self here), and any globals hanging around.
If you want to know about a calling object, the valid ways are:
Pass the calling object in as an argument to the function
Explicitly add a handle to the caller onto b instance sometime before using func, and then access that handle through self.
However, with that disclaimer out of the way, it's still worth knowing that Python's introspection capabilities are powerful enough to access the caller module in some cases. In the CPython implementation, here is how you could access the calling A instance without changing your existing function signatures:
class A:
def go(self):
b=B()
b.func()
class B:
def func(self):
import inspect
print inspect.currentframe().f_back.f_locals["self"]
if __name__ == "__main__":
a = A()
a.go()
Output:
<__main__.A instance at 0x15bd9e0>
This might be a useful trick to know about for debugging purposes. A similar technique is even used in stdlib logging, here, so that loggers are able to discover the source code/file name/line number/function name without needing to be explicitly passed that context. However, in normal use cases, it would not usually be a sensible design decision to access stack frames in the case that B.func actually needed to use A, because it's cleaner and easier to pass along the information that you need rather than to try and "reach back" to a caller.
You pass it to b.func() as an argument.
Do this by refactoring your code to work like
class A():
def go(self):
b = B(self)
b.func()
class B():
def __init__(self, a):
self.a = a
def func(self):
# Use self.a
or
class A():
def go(self):
b = B()
b.func(self)
class B():
def func(self, a):
# a
I agree with Benjamin - pass it to b.func() as an argument and don't introspect it!!!!
If your life really depends on it, then I think you can deduce the answer from this answer.

Python multiple inheritance: Whats wrong doing it dynamically?

Based on this answer, of how __new__ and __init__ are supposed to work in Python,
I wrote this code to dynamically define and create a new class and object.
class A(object):
def __new__(cls):
class C(cls, B):
pass
self = C()
return self
def foo(self):
print 'foo'
class B(object):
def bar(self):
print 'bar'
a = A()
a.foo()
a.bar()
Basically, because the __new__ of A returns a dynamically created C that inherits A and B, it should have an attribute bar.
Why does C not have a bar attribute?
Resolve the infinite recursion:
class A(object):
def __new__(cls):
class C(cls, B):
pass
self = object.__new__(C)
return self
(Thanks to balpha for pointing out the actual question.)
Since there is no actual question in the question, I am going to take it literally:
Whats wrong doing it dynamically?
Well, it is practically unreadable, extremely opaque and non-obvious to the user of your code (that includes you in a month :P).
From my experience (quite limited, I must admit, unfortunately I don't have 20 years of programming under the belt), a need for such solutions indicates, that the class structure is not well defined, - means, there's almost always a better, more readable and less arcane way to do such things.
For example, if you really want to define base classes on the fly, you are better off using a factory function, that will return appropriate classes according to your needs.
Another take on the question:
Whats wrong doing it dynamically?
In your current implementation, it gives me a "maximum recursion depth exceeded" error. That happens, because A.__new__ calls itself from within itself indefinitely (since it inherits from itself and from B).
10: Inside A.__new__, "cls" is set to <class '.A'>. Inside the constructor you define a class C, which inherits from cls (which is actually A) and another class B. Upon instantiating C, its __new__ is called. Since it doesn't define its own __new__, its base class' __new__ is called. The base class just happens to be A.
20: GOTO 10
If your question is "How can I accomplish this" – this works:
class A(object):
#classmethod
def get_with_B(cls):
class C(B, cls):
pass
return C()
def foo(self):
print 'foo'
class B(object):
def bar(self):
print 'bar'
a = A.get_with_B()
a.foo()
a.bar()
If your question is "Why doesn't it work" – that's because you run into an infinite recursion when you call C(), which leads to A.__new__ being called, which again calls C() etc.

Categories