We Keep Coding

Refer to a superclass from the class body

I've got some code where I need to refer to a superclass when defining stuff in a derived class:
class Base:
def foo(self):
print('foo')
def bar(self):
print('bar')
class Derived_A(Base):
meth = Base.foo
class Derived_B(Base):
meth = Base.bar
Derived_A().meth()
Derived_B().meth()
This works, but I don't like verbatim references to Base in derived classes. Is there a way to use super or alike for this?

You can't do that.
class keyword in Python is used to create classes which are instances of type type. In it's simplified version, it does the following:
Python creates a namespace and executes the body of the class in that namespace so that it will be populated with all methods and attributes and so on...
Then calls the three-arguments form of type(). The result of this call is your class which is then assign to a symbol which is the name of your class.
The point is when the body of the class is being executed. It doesn't know about the "bases". Those bases are passed to the type() after that.
I also explained the reasons why you can't use super() here.

Does this work for you?
class Base:
def foo(self):
print('foo')
def bar(self):
print('bar')
class Derived_A(Base):
def __init__(self):
self.meth = super().foo
class Derived_B(Base):
def __init__(self):
self.meth = super().bar
a = Derived_A().meth()
b = Derived_B().meth()

You'll need to lookup the method on the base class after the new type is created. In the body of the class definition, the type and base classes are not accessible.
Something like:
class Derived_A(Base):
def meth(self):
return super().foo()
Now, it is possible to do some magic behind the scenes to expose Base to the scope of the class definition as its being executed, but that's much dirtier, and would mean that you'd need to supply a metaclass in your class definition.

Since you want "magic", there is still one sane option we can take before diving into metaclasses. Requires Python 3.9+
def alias(name):
def inner(cls):
return getattr(cls, name).__get__(cls)
return classmethod(property(inner))
class Base:
def foo(self):
...
class Derived_A(Base):
meth = alias("foo")
Derived_A().meth() # works
Derived_A.meth() # also works
Yes, this does require passing the method name as a string, which destroys your IDE and typechecker's ability to reason about it. But there isn't a good way to get what you are wanting without some compromises like that.
Really, a bit of redundancy for readability is probably worth it here.

When and how to check Python subclasses meet specification

I have a base class that looks something like this:
class myBaseClass:
def __init__(self):
self.name = None # All subclasses must define this
def foo(self): # All subclasses must define this
raise NotImplementedError
def bar(self): # Optional -- not all subclasses will define this
raise NotImplementedError
My API specification stipulates that anyone creating a subclass of myBaseClass must provide a meaningful value for .name, and for the function .foo(). However, .bar() is optional and calling code should be able to handle the case where that results in a NotImplementedError.
When and how should I check that subclasses contributed by third parties meet these requirements?
The options seem to be:
Build subclasses exclusively via metaclasses. However, this approach will be unfamiliar and potentially confusing to most of the contributors to my project, who tend not to be expert developers.
Add an __init_subclass__ method to the base class and use this to infer whether the subclass has overridden everything it is supposed to override. Seems to work, but feels a bit 'kludgy'.
Write build-time tests to instantiate each subclass, call each 'required' method, and verify that they do not raise a NotImplementedError. Seems like an excessive computational effort to answer such a simple question (calling .foo() may be expensive).
Ignore the issue. Deal with it if and when it causes something else to break.
I'm sure I'm not the only person who needs to deal with this issue - is there a 'correct' approach here?

Here's how I would structure it.
First off, what you're looking for here is an abstract base class. Using the built-in modules you can easily define it as such and have methods be forced to have an implementation, otherwise the class will raise an exception when instantiated.
If the name attribute needs to be set always, then you should make it part of the constructor arguments.
Because bar is not always required I wouldn't define it as a method in the base class you have. Instead I would make a child class that is also abstract and define it there as required. When checking to see if the method is available you can use isinstance.
This is what my final code would look like:
from abc import ABC, abstractmethod
class FooBaseClass(ABC):
def __init__(self, name):
self.name = name
#abstractmethod
def foo(self):
"""Some useful docs for foo"""
class FooBarBaseClass(FooBaseClass, ABC):
#abstractmethod
def bar(self):
"""Some useful docs for bar"""
When creating instances you can pick the base class you want and will be forced to define the methods.
class FooClass(FooBaseClass):
def __init__(self):
super().__init__("foo")
def foo(self):
print("Calling foo from FooClass")
class FooBarClass(FooBarBaseClass):
def __init__(self):
super().__init__("foobar")
def foo(self):
print("Calling foo from FooBarClass")
def bar(self):
print("Calling bar from FooBarClass")
Example checking if bar is callable:
def do_operation(obj: FooBaseClass):
obj.foo()
if isinstance(obj, FooBarBaseClass):
obj.bar()
Example:
do_operation(FooClass())
do_operation(FooBarClass())
Calling foo from FooClass
Calling foo from FooBarClass
Calling bar from FooBarClass
An example of invalid code
class InvalidClass(FooBaseClass):
def __init__(self):
super().__init__("foo")
InvalidClass()
Traceback (most recent call last):
File "C:\workspace\so\test.py", line 52, in <module>
InvalidClass()
TypeError: Can't instantiate abstract class InvalidClass with abstract method foo

Calling super().method() vs. BaseClass.method(self)

There are two main ways for a derived class to call a base class's methods.
Base.method(self):
class Derived(Base):
def method(self):
Base.method(self)
...
or super().method():
class Derived(Base):
def method(self):
super().method()
...
Suppose I now do this:
obj = Derived()
obj.method()
As far as I know, both Base.method(self) and super().method() do the same thing. Both will call Base.method with a reference to obj. In particular, super() doesn't do the legwork to instantiate an object of type Base. Instead, it creates a new object of type super and grafts the instance attributes from obj onto it, then it dynamically looks up the right attribute from Base when you try to get it from the super object.
The super() method has the advantage of minimizing the work you need to do when you change the base for a derived class. On the other hand, Base.method uses less magic and may be simpler and clearer when a class inherits from multiple base classes.
Most of the discussions I've seen recommend calling super(), but is this an established standard among Python coders? Or are both of these methods widely used in practice? For example, answers to this stackoverflow question go both ways, but generally use the super() method. On the other hand, the Python textbook I am teaching from this semester only shows the Base.method approach.

Using super() implies the idea that whatever follows should be delegated to the base class, no matter what it is. It's about the semantics of the statement. Referring explicitly to Base on the other hand conveys the idea that Base was chosen explicitly for some reason (perhaps unknown to the reader), which might have its applications too.
Apart from that however there is a very practical reason for using super(), namely cooperative multiple inheritance. Suppose you've designed the following class hierarchy:
class Base:
def test(self):
print('Base.test')
class Foo(Base):
def test(self):
print('Foo.test')
Base.test(self)
class Bar(Base):
def test(self):
print('Bar.test')
Base.test(self)
Now you can use both Foo and Bar and everything works as expected. However these two classes won't work together in a multiple inheritance schema:
class Test(Foo, Bar):
pass
Test().test()
# Output:
# Foo.test
# Base.test
That last call to test skips over Bar's implementation since Foo didn't specify that it wants to delegate to the next class in method resolution order but instead explicitly specified Base. Using super() resolves this issue:
class Base:
def test(self):
print('Base.test')
class Foo(Base):
def test(self):
print('Foo.test')
super().test()
class Bar(Base):
def test(self):
print('Bar.test')
super().test()
class Test(Foo, Bar):
pass
Test().test()
# Output:
# Foo.test
# Bar.test
# Base.test

Method can be overridden, but doesn't have to be

I understand that #abstractmethods use in an ABC is to dictate that a method must be implemented in the concrete implementation of the ABC.
What if I have a class with methods that can be overridden, but don't have to be? What's the best way to let users know that the method must be overridden to provide functionality?
Situation where base method is overridden:
import warnings
class BaseClass(object):
def foo(self):
"""This method can do things, but doesn't."""
warnings.warn('This method must be overridden to do anything.')
class ConcreteClass(BaseClass):
def foo(self):
"""This method definitely does things."""
# very complex operation
bar = 5
return bar
Usage:
>>> a = ConcreteClass()
>>> a.foo()
5
Situation where base method is not overridden:
import warnings
class BaseClass(object):
def foo(self):
"""This method can do things, but doesn't."""
warnings.warn('This method must be overridden to do anything.')
class ConcreteClass(BaseClass):
def other_method(self):
"""Completely different method."""
# code here
def yet_another_method(self):
"""One more different method."""
# code here
Usage:
>>> a = ConcreteClass()
>>> a.foo()
__main__:1: UserWarning: This method must be overridden to do anything.
The reason I want to have the base method do anything at all is mainly due to user-friendliness. Colleagues within my group that have less experience using software could benefit from a kick in the rear reminding them that the script they wrote with my package isn't broken, they just forgot to add something.

A method in python already can be overridden but doesn't have to be.
So for the rest of the question:
What's the best way to let users know that the method must be
overridden to provide functionality?
You can raise a NotImplementedError:
class BaseClass(object):
def foo(self):
raise NotImplementedError
class ConcreteClass(BaseClass):
def foo(self):
pass
And regarding
a kick in the rear reminding them that the script they wrote with my
package isn't broken, they just forgot to add something.
An exception is much more explicit and helpful than a warning (that could easily be missed when thousands of log records had printed)

How to implement virtual methods in Python?

I know virtual methods from PHP or Java.
How can they be implemented in Python?
Or have I to define an empty method in an abstract class and override it?

Sure, and you don't even have to define a method in the base class. In Python methods are better than virtual - they're completely dynamic, as the typing in Python is duck typing.
class Dog:
def say(self):
print "hau"
class Cat:
def say(self):
print "meow"
pet = Dog()
pet.say() # prints "hau"
another_pet = Cat()
another_pet.say() # prints "meow"
my_pets = [pet, another_pet]
for a_pet in my_pets:
a_pet.say()
Cat and Dog in Python don't even have to derive from a common base class to allow this behavior - you gain it for free. That said, some programmers prefer to define their class hierarchies in a more rigid way to document it better and impose some strictness of typing. This is also possible - see for example the abc standard module.

raise NotImplementedError() (dynamic type checking)
This is the recommended exception to raise on "pure virtual methods" of "abstract" base classes that don't implement a method.
https://docs.python.org/3.5/library/exceptions.html#NotImplementedError says:
This exception is derived from RuntimeError. In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method.
As others said, this is mostly a documentation convention and is not required, but this way you get a more meaningful exception than a missing attribute error.
dynamic.py
class Base(object):
def virtualMethod(self):
raise NotImplementedError()
def usesVirtualMethod(self):
return self.virtualMethod() + 1
class Derived(Base):
def virtualMethod(self):
return 1
print Derived().usesVirtualMethod()
Base().usesVirtualMethod()
gives:
2
Traceback (most recent call last):
File "./dynamic.py", line 13, in <module>
Base().usesVirtualMethod()
File "./dynamic.py", line 6, in usesVirtualMethod
return self.virtualMethod() + 1
File "./dynamic.py", line 4, in virtualMethod
raise NotImplementedError()
NotImplementedError
typing.Protocol (static type checking, Python 3.8)
Python 3.8 added typing.Protocol which now allows us to also statically type check that a virtual method is implemented on a subclass.
protocol.py
from typing import Protocol
class CanFly(Protocol):
def fly(self) -> str:
pass
def fly_fast(self) -> str:
return 'CanFly.fly_fast'
class Bird(CanFly):
def fly(self):
return 'Bird.fly'
def fly_fast(self):
return 'Bird.fly_fast'
class FakeBird(CanFly):
pass
assert Bird().fly() == 'Bird.fly'
assert Bird().fly_fast() == 'Bird.fly_fast'
# mypy error
assert FakeBird().fly() is None
# mypy error
assert FakeBird().fly_fast() == 'CanFly.fly_fast'
If we run this file, the asserts pass, as we didn't add any dynamic typechecking:
python protocol.py
but if we typecheck if mypy:
python -m pip install --user mypy
mypy protocol.py
we get an error as expected:
protocol.py:22: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
protocol.py:24: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
It is a bit unfortunate however that the error checking only picks up the error on instantiation, and not at class definition.
typing.Protocol counts methods as abstract when their body is "empty"
I'm not sure what they count as empty, but both all of the following count as empty:
pass
... ellipsis object
raise NotImplementedError()
So the best possibility is likely:
protocol_empty.py
from typing import Protocol
class CanFly(Protocol):
def fly(self) -> None:
raise NotImplementedError()
class Bird(CanFly):
def fly(self):
return None
class FakeBird(CanFly):
pass
Bird().fly()
FakeBird().fly()
which fails as desired:
protocol_empty.py:15: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
protocol_empty.py:15: note: The following method was marked implicitly abstract because it has an empty function body: "fly". If it is not meant to be abstract, explicitly return None.
but if e.g. we replace the:
raise NotImplementedError()
with some random "non-empty" statement such as:
x = 1
then mypy does not count them as virtual and gives no errors.
#abc.abstractmethod: metaclass syntax changed in Python 3
In Python 3 metaclasses are declared as:
class C(metaclass=abc.ABCMeta):
instead of the Python 2:
class C:
__metaclass__=abc.ABCMeta
so now to use #abc.abstractmethod which was previously mentioned at https://stackoverflow.com/a/19316077/895245 you need:
abc_cheat.py
class C(metaclass=abc.ABCMeta):
#abc.abstractmethod
def m(self, i):
pass
try:
c = C()
except TypeError:
pass
else:
assert False
But TODO: what is the advantage of ABCMeta over just raise NotImplementedError? It has a disadvantage that you are forced to define a metaclass, so more work, but I don't see an advantage. https://peps.python.org/pep-0544 does mention both approaches in passing.
Outro
Bibiography:
https://peps.python.org/pep-0544 the typing.Protocol PEP
Is it possible to make abstract classes?
What to use in replacement of an interface/protocol in python
Tested on Python 3.10.7, mypy 0.982, Ubuntu 21.10.

Python methods are always virtual.

Actually, in version 2.6 python provides something called abstract base classes and you can explicitly set virtual methods like this:
from abc import ABCMeta
from abc import abstractmethod
...
class C:
__metaclass__ = ABCMeta
#abstractmethod
def my_abstract_method(self, ...):
It works very well, provided the class does not inherit from classes that already use metaclasses.
source: http://docs.python.org/2/library/abc.html

Python methods are always virtual
like Ignacio said yet
Somehow class inheritance may be a better approach to implement what you want.
class Animal:
def __init__(self,name,legs):
self.name = name
self.legs = legs
def getLegs(self):
return "{0} has {1} legs".format(self.name, self.legs)
def says(self):
return "I am an unknown animal"
class Dog(Animal): # <Dog inherits from Animal here (all methods as well)
def says(self): # <Called instead of Animal says method
return "I am a dog named {0}".format(self.name)
def somethingOnlyADogCanDo(self):
return "be loyal"
formless = Animal("Animal", 0)
rover = Dog("Rover", 4) #<calls initialization method from animal
print(formless.says()) # <calls animal say method
print(rover.says()) #<calls Dog says method
print(rover.getLegs()) #<calls getLegs method from animal class
Results should be:
I am an unknown animal
I am a dog named Rover
Rover has 4 legs

Something like a virtual method in C++ (calling method implementation of a derived class through a reference or pointer to the base class) doesn't make sense in Python, as Python doesn't have typing. (I don't know how virtual methods work in Java and PHP though.)
But if by "virtual" you mean calling the bottom-most implementation in the inheritance hierarchy, then that's what you always get in Python, as several answers point out.
Well, almost always...
As dplamp pointed out, not all methods in Python behave like that. Dunder method don't. And I think that's a not so well known feature.
Consider this artificial example
class A:
def prop_a(self):
return 1
def prop_b(self):
return 10 * self.prop_a()
class B(A):
def prop_a(self):
return 2
Now
>>> B().prop_b()
20
>>> A().prob_b()
10
However, consider this one
class A:
def __prop_a(self):
return 1
def prop_b(self):
return 10 * self.__prop_a()
class B(A):
def __prop_a(self):
return 2
Now
>>> B().prop_b()
10
>>> A().prob_b()
10
The only thing we've changes was making prop_a() a dunder method.
A problem with the first behavior can be that you can't change the behavior of prop_a() in the derived class without impacting the behavior of prop_b(). This very nice talk by Raymond Hettinger gives an example for a use case where this is inconvenient.

Python 3.6 introduced __init_subclass__ and this let you simply do this:
class A:
def method(self):
'''method needs to be overwritten'''
return NotImplemented
def __init_subclass__(cls):
if cls.method is A.method:
raise NotImplementedError(
'Subclass has not overwritten method {method}!')
The benefit of this solution is that you avoid the abc metaclass and give the user a direct imperative how to do it right. In addition to another answer here that raises NotImplementedError when calling the method. This solution is checked on runtime and not only IF the user calls the method.