I have a base class that looks something like this:
class myBaseClass:
def __init__(self):
self.name = None # All subclasses must define this
def foo(self): # All subclasses must define this
raise NotImplementedError
def bar(self): # Optional -- not all subclasses will define this
raise NotImplementedError
My API specification stipulates that anyone creating a subclass of myBaseClass must provide a meaningful value for .name, and for the function .foo(). However, .bar() is optional and calling code should be able to handle the case where that results in a NotImplementedError.
When and how should I check that subclasses contributed by third parties meet these requirements?
The options seem to be:
Build subclasses exclusively via metaclasses. However, this approach will be unfamiliar and potentially confusing to most of the contributors to my project, who tend not to be expert developers.
Add an __init_subclass__ method to the base class and use this to infer whether the subclass has overridden everything it is supposed to override. Seems to work, but feels a bit 'kludgy'.
Write build-time tests to instantiate each subclass, call each 'required' method, and verify that they do not raise a NotImplementedError. Seems like an excessive computational effort to answer such a simple question (calling .foo() may be expensive).
Ignore the issue. Deal with it if and when it causes something else to break.
I'm sure I'm not the only person who needs to deal with this issue - is there a 'correct' approach here?
Here's how I would structure it.
First off, what you're looking for here is an abstract base class. Using the built-in modules you can easily define it as such and have methods be forced to have an implementation, otherwise the class will raise an exception when instantiated.
If the name attribute needs to be set always, then you should make it part of the constructor arguments.
Because bar is not always required I wouldn't define it as a method in the base class you have. Instead I would make a child class that is also abstract and define it there as required. When checking to see if the method is available you can use isinstance.
This is what my final code would look like:
from abc import ABC, abstractmethod
class FooBaseClass(ABC):
def __init__(self, name):
self.name = name
#abstractmethod
def foo(self):
"""Some useful docs for foo"""
class FooBarBaseClass(FooBaseClass, ABC):
#abstractmethod
def bar(self):
"""Some useful docs for bar"""
When creating instances you can pick the base class you want and will be forced to define the methods.
class FooClass(FooBaseClass):
def __init__(self):
super().__init__("foo")
def foo(self):
print("Calling foo from FooClass")
class FooBarClass(FooBarBaseClass):
def __init__(self):
super().__init__("foobar")
def foo(self):
print("Calling foo from FooBarClass")
def bar(self):
print("Calling bar from FooBarClass")
Example checking if bar is callable:
def do_operation(obj: FooBaseClass):
obj.foo()
if isinstance(obj, FooBarBaseClass):
obj.bar()
Example:
do_operation(FooClass())
do_operation(FooBarClass())
Calling foo from FooClass
Calling foo from FooBarClass
Calling bar from FooBarClass
An example of invalid code
class InvalidClass(FooBaseClass):
def __init__(self):
super().__init__("foo")
InvalidClass()
Traceback (most recent call last):
File "C:\workspace\so\test.py", line 52, in <module>
InvalidClass()
TypeError: Can't instantiate abstract class InvalidClass with abstract method foo
Related
I've got some code where I need to refer to a superclass when defining stuff in a derived class:
class Base:
def foo(self):
print('foo')
def bar(self):
print('bar')
class Derived_A(Base):
meth = Base.foo
class Derived_B(Base):
meth = Base.bar
Derived_A().meth()
Derived_B().meth()
This works, but I don't like verbatim references to Base in derived classes. Is there a way to use super or alike for this?
You can't do that.
class keyword in Python is used to create classes which are instances of type type. In it's simplified version, it does the following:
Python creates a namespace and executes the body of the class in that namespace so that it will be populated with all methods and attributes and so on...
Then calls the three-arguments form of type(). The result of this call is your class which is then assign to a symbol which is the name of your class.
The point is when the body of the class is being executed. It doesn't know about the "bases". Those bases are passed to the type() after that.
I also explained the reasons why you can't use super() here.
Does this work for you?
class Base:
def foo(self):
print('foo')
def bar(self):
print('bar')
class Derived_A(Base):
def __init__(self):
self.meth = super().foo
class Derived_B(Base):
def __init__(self):
self.meth = super().bar
a = Derived_A().meth()
b = Derived_B().meth()
You'll need to lookup the method on the base class after the new type is created. In the body of the class definition, the type and base classes are not accessible.
Something like:
class Derived_A(Base):
def meth(self):
return super().foo()
Now, it is possible to do some magic behind the scenes to expose Base to the scope of the class definition as its being executed, but that's much dirtier, and would mean that you'd need to supply a metaclass in your class definition.
Since you want "magic", there is still one sane option we can take before diving into metaclasses. Requires Python 3.9+
def alias(name):
def inner(cls):
return getattr(cls, name).__get__(cls)
return classmethod(property(inner))
class Base:
def foo(self):
...
class Derived_A(Base):
meth = alias("foo")
Derived_A().meth() # works
Derived_A.meth() # also works
Yes, this does require passing the method name as a string, which destroys your IDE and typechecker's ability to reason about it. But there isn't a good way to get what you are wanting without some compromises like that.
Really, a bit of redundancy for readability is probably worth it here.
I am given a designated factory of A-type objects. I would like to make a new version of A-type objects that also have the methods in a Mixin class. For reasons that are too long to explain here, I can't use class A(Mixin), I have to use the A_factory. Below I try to give a bare bones example.
I thought naively that it would be sufficient to inherit from Mixin to endow A-type objects with the mixin methods, but the attempts below don't work:
class A: pass
class A_factory:
def __new__(self):
return A()
class Mixin:
def method(self):
print('aha!')
class A_v2(Mixin): # attempt 1
def __new__(cls):
return A_factory()
class A_v3(Mixin): # attempt 2
def __new__(cls):
self = A_factory()
super().__init__(self)
return self
In fact A_v2().method() and A_v3().method() raises AttributeError: 'A' object has no attribute 'method'.
What is the correct way of using A_factory within class A_vn(Mixin) so that A-type objects created by the factory inherit the mixin methods?
There's no obvious reason why you should need __new__ for what you're showing here. There's a nice discussion here on the subject: Why is __init__() always called after __new__()?
If you try the below it should work:
class Mixin:
def method(self):
print('aha!')
class A(Mixin):
def __init__(self):
super().__init__()
test = A()
test.method()
If you need to use a factory method, it should be a function rather than a class. There's a very good discussion of how to use factory methods here: https://realpython.com/factory-method-python/
I understand that #abstractmethods use in an ABC is to dictate that a method must be implemented in the concrete implementation of the ABC.
What if I have a class with methods that can be overridden, but don't have to be? What's the best way to let users know that the method must be overridden to provide functionality?
Situation where base method is overridden:
import warnings
class BaseClass(object):
def foo(self):
"""This method can do things, but doesn't."""
warnings.warn('This method must be overridden to do anything.')
class ConcreteClass(BaseClass):
def foo(self):
"""This method definitely does things."""
# very complex operation
bar = 5
return bar
Usage:
>>> a = ConcreteClass()
>>> a.foo()
5
Situation where base method is not overridden:
import warnings
class BaseClass(object):
def foo(self):
"""This method can do things, but doesn't."""
warnings.warn('This method must be overridden to do anything.')
class ConcreteClass(BaseClass):
def other_method(self):
"""Completely different method."""
# code here
def yet_another_method(self):
"""One more different method."""
# code here
Usage:
>>> a = ConcreteClass()
>>> a.foo()
__main__:1: UserWarning: This method must be overridden to do anything.
The reason I want to have the base method do anything at all is mainly due to user-friendliness. Colleagues within my group that have less experience using software could benefit from a kick in the rear reminding them that the script they wrote with my package isn't broken, they just forgot to add something.
A method in python already can be overridden but doesn't have to be.
So for the rest of the question:
What's the best way to let users know that the method must be
overridden to provide functionality?
You can raise a NotImplementedError:
class BaseClass(object):
def foo(self):
raise NotImplementedError
class ConcreteClass(BaseClass):
def foo(self):
pass
And regarding
a kick in the rear reminding them that the script they wrote with my
package isn't broken, they just forgot to add something.
An exception is much more explicit and helpful than a warning (that could easily be missed when thousands of log records had printed)
I'm trying to figure out how to ensure that a method of a class inheriting from an ABC is created using the appropriate decorator. I understand (hopefully) how ABCs work in general.
from abc import ABCMeta, abstractmethod
class MyABC(metaclass=ABCMeta):
#abstractmethod
def my_abstract_method(self):
pass
class MyClass(MyABC):
pass
MyClass()
This gives "TypeError: Can't instantiate abstract class MyClass with abstract methods my_abstract_method". Great, makes sense. Just create a method with that name.
class MyClass(MyABC):
def my_abstract_method(self):
pass
MyClass()
Boom. You're done. But what about this case?
from abc import ABCMeta, abstractmethod
class MyABC(metaclass=ABCMeta):
#property
#abstractmethod
def my_attribute(self):
pass
class MyClass(MyABC):
def my_attribute(self):
pass
MyClass()
The MyClass() call works even though my_attribute is not a property. I guess in the end all ABCs do is ensure that a method with a given name exists. Thats it. If you want more from it, you have to look at MyABC's source code and read the documentation. The decorators and comments there will inform you of how you need to construct your sub-class.
Do I have it right or am I missing something here?
You're correct that ABCs do not enforce that. There isn't a way to enforce something like "has a particular decorator". Decorators are just functions that return objects (e.g., property returns a property object). ABCMeta doesn't do anything to ensure that the defined attributes on the class are anything in particular; it just makes sure they are there. This "works" without errors:
class MyABC(metaclass=ABCMeta):
#abstractmethod
def my_abstract_method(self):
pass
class MyClass(MyABC):
my_abstract_method = 2
MyClass()
That is, ABCMeta doesn't even ensure that the abstract method as provided on the subclass is a method at all. There just has to be an attribute of some kind with that name,
You could certainly write your own metaclass that does more sophisticated checking to ensure that certain attributes have certain kinds of values, but that's beyond the scope of ABCMeta.
I'm trying to understand the benefits of using abstract base classes. Consider these two pieces of code:
Abstract base class:
from abc import ABCMeta, abstractmethod, abstractproperty
class CanFly:
__metaclass__ = ABCMeta
#abstractmethod
def fly(self):
pass
#abstractproperty
def speed(self):
pass
class Bird(CanFly):
def __init__(self):
self.name = 'flappy'
#property
def speed(self):
return 1
def fly(self):
print('fly')
b = Bird()
print(isinstance(b, CanFly)) # True
print(issubclass(Bird, CanFly)) # True
Plain inheritance:
class CanFly(object):
def fly(self):
raise NotImplementedError
#property
def speed(self):
raise NotImplementedError()
class Bird(CanFly):
#property
def speed(self):
return 1
def fly(self):
print('fly')
b = Bird()
print(isinstance(b, CanFly)) # True
print(issubclass(Bird, CanFly)) # True
As you see, both methods support inflection using isinstance and issubclass.
Now, one difference I know is that, if you try to instantiate a subclass of an abstract base class without overriding all abstract methods/properties, your program will fail loudly. However, if you use plain inheritance with NotImplementedError, your code won't fail until you actually invoke the method/property in question.
Other than that, what makes using abstract base class different?
The most notable answer in terms of concrete specifics, besides what you mentioned in your question, is that the presence of the #abstractmethod or #abstractproperty1 decorators, along with inheriting from ABC (or having the ABCMeta metaclass) prevents you from instantiating the object at all.
from abc import ABC, abstractmethod
class AbsParent(ABC):
#abstractmethod
def foo(self):
pass
AbsParent()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class AbsParent with abstract methods foo
However, there's more at play here. Abstract Base Classes were introduced to Python in PEP 3119. I'd recommend reading through the "Rationale" section for Guido's take on why they were introduced in the first place. My sophomoric summary would be that they're less about their concrete features and more about their philosophy. Their purpose is to signal to external inspectors that the object is inheriting from the ABC, and because it's inheriting from an ABC it will follow a good-faith agreement. This "good-faith agreement" is that the child object will follow the intention of the parent. The actual implementation of this agreement is left up to you, which is why it's a good-faith agreement, and not an explicit contract.
This primarily shows up through the lens of the register() method. Any class that has ABCMeta as its metaclass (or simply inherits from ABC) will have a register() method on it. By registering a class with an ABC you are signaling that it inherits from the ABC, even though it technically doesn't. This is where the good-faith agreement comes in.
from abc import ABC, abstractmethod
class MyABC(ABC):
#abstractmethod
def foo(self):
"""should return string 'foo'"""
pass
class MyConcreteClass(object):
def foo(self):
return 'foo'
assert not isinstance(MyConcreteClass(), MyABC)
assert not issubclass(MyConcreteClass, MyABC)
While MyConcreteClass, at this point is unrelated to MyABC, it does implement the API of MyABC according to the requirements laid out in the comments. Now, if we register MyConcreteClass with MyABC, it will pass isinstance and issubclass checks.
MyABC.register(MyConcreteClass)
assert isinstance(MyConcreteClass(), MyABC)
assert issubclass(MyConcreteClass, MyABC)
Again, this is where the "good-faith agreement" comes into play. You do not have to follow the API laid out in MyABC. By registering the concrete class with the ABC we are telling any external inspectors that we, the programmers, are adhering to the API we're supposed to.
1 note that #abstractproperty is no longer preferred. Instead you should use:
#property
#abstractmethod
def foo(self):
pass