I'm trying to figure out how to provide a base class to plugin writers so that they provide definitions for several static methods.
A plugin class is a collection of static methods which will never be instantiated.
I know how to use ABC to prevent instantiation of a class missing method implementations, which will not provide the safety I would like. Is there a pattern to prevent definition?
You can do it by writing your own metaclass similar to ABCMeta, which checks for abstract methods at class-definition time and raises an error if it finds any. Here's an example:
class ClassABC(type):
def __init__(cls, name, bases, attrs):
abstracts = set()
for base in bases:
abstracts.update(getattr(base, '__abstractclassmethods__', set()))
for abstract in abstracts:
if getattr(getattr(cls, abstract), '__isabstractmethod__', False):
raise TypeError("Your class doesn't define {0}".format(abstract))
for attr in attrs:
if getattr(attrs[attr], '__isabstractmethod__', False):
abstracts.add(attr)
cls.__abstractclassmethods__ = abstracts
class BaseClass(object):
__metaclass__ = ClassABC
#abc.abstractmethod
def foo(self):
print("I am base foo")
Then:
>>> class Derived(BaseClass):
... pass
Traceback (most recent call last):
File "<pyshell#10>", line 1, in <module>
class Derived(BaseClass):
File "<pyshell#8>", line 8, in __init__
raise TypeError("Your class doesn't define {0}".format(abstract))
TypeError: Your class doesn't define foo
My example is fairly quick and dirty and only scantily tested, and you might want to refine it to check various sorts of edge cases. What I did is I raised the error if there is an abstract method that wasn't defined in the currently-being-defined class. (Otherwise an error would be raised because the abstract base class doesn't define concrete implementations of its own abstract methods.)
Related
Consider the following code example
import abc
class ABCtest(abc.ABC):
#abc.abstractmethod
def foo(self):
raise RuntimeError("Abstract method was called, this should be impossible")
class ABCtest_B(ABCtest):
pass
test = ABCtest_B()
This correctly raises the error:
Traceback (most recent call last):
File "/.../test.py", line 10, in <module>
test = ABCtest_B()
TypeError: Can't instantiate abstract class ABCtest_B with abstract methods foo
However when the subclass of ABCtest also inherits from a built in type like str or list there is no error and test.foo() calls the abstract method:
class ABCtest_C(ABCtest, str):
pass
>>> test = ABCtest_C()
>>> test.foo()
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
test.foo()
File "/.../test.py", line 5, in foo
raise RuntimeError("Abstract method was called, this should be impossible")
RuntimeError: Abstract method was called, this should be impossible
This seems to happen when inheriting from any class defined in C including itertools.chain and numpy.ndarray but still correctly raises errors with classes defined in python. Why would implementing one of a built in types break the functionality of abstract classes?
Surprisingly, the test that prevents instantiating abstract classes happens in object.__new__, rather than anything defined by the abc module itself:
static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
...
if (type->tp_flags & Py_TPFLAGS_IS_ABSTRACT) {
...
PyErr_Format(PyExc_TypeError,
"Can't instantiate abstract class %s "
"with abstract methods %U",
type->tp_name,
joined);
(Almost?) all built-in types that aren't object supply a different __new__ that overrides object.__new__ and does not call object.__new__. When you multiple-inherit from a non-object built-in type, you inherit its __new__ method, bypassing the abstract method check.
I don't see anything about __new__ or multiple inheritance from built-in types in the abc documentation. The documentation could use enhancement here.
It seems kind of strange that they'd use a metaclass for the ABC implementation, making it a mess to use other metaclasses with abstract classes, and then put the crucial check in core language code that has nothing to do with abc and runs for both abstract and non-abstract classes.
There's a report for this issue on the issue tracker that's been languishing since 2009.
I asked a similar question and based on user2357112 supports Monicas linked bug report, I came up with this workaround (based on the suggestion from Xiang Zhang):
from abc import ABC, abstractmethod
class Base(ABC):
#abstractmethod
def foo(self):
pass
#abstractmethod
def bar(self):
pass
def __new__(cls, *args, **kwargs):
abstractmethods = getattr(cls, '__abstractmethods__', None)
if abstractmethods:
msg = "Can't instantiate abstract class {name} with abstract method{suffix} {methods}"
suffix = 's' if len(abstractmethods) > 1 else ''
raise TypeError(msg.format(name=cls.__name__, suffix=suffix, methods=', '.join(abstractmethods)))
return super().__new__(cls, *args, **kwargs)
class Derived(Base, tuple):
pass
Derived()
This raises TypeError: Can't instantiate abstract class Derived with abstract methods bar, foo, which is the original behaviour.
So, here is a problem:
I want to define an abstract class, let's say AbstractA, which does not require subclasses to implement any of its methods, but rather to extend its functionality. In terms of Java that would be interface class.
Moreover, I want to be able to create an abstract subclass, let's say AbstractB, of the AbstractA with the same properties, but some methods redefining or extending base class methods.
I don't want though to make class (AbstractA) abstract e.g. through the check of class name in __init__ or __new__, because that would require abstract subclass (AbstractB) to redefine that method along with it's main functionality, i.e. construction or initialization of a new instance. Or to call super().__init__(...) which I'd prefer to avoid as well (maybe I'm wrong here).
So, I want something like that:
class AbstractA:
def __init__(self):
# do initialization stuff
def very_common_method(self, ...):
# do very common stuff
class AbstractB(AbstractA):
# do not duplicate initialization stuff here, inherit instead
def less_common_method(self, ...):
# do less common stuff
class AX(AbstractA):
def specific_method_1(self, ...):
class BX(AbstractB):
def specific_method_2(self, ...):
# Instantiating AbstractA or AbstractB should result in error.
# Instantiating AX or BX should not.
Below I have a possible solution.
Is there any disadvantages I overlook? Better solution?
Thanks!
Here's possible solution:
class AbstractA:
_is_abstract = True
def __init__(self):
if self._is_abstract:
raise RuntimeError("Abstract class instantiation.")
# do initialization stuff
def __init_subclass__(self): # is called every time class is subclassed
self._is_abstract = False # thus makes sure abstract check fails on a subclass
class AbstractMixin:
def __init_subclass__(self):
self._is_abstract = True
class AbstractB(AbstractMixin, AbstractA): # AbstractMixin takes precendence on MRO,
# inherit __init__ # so the class abstract check returns True.
def __init_subclass__(self):
self._is_abstract = False
class A(AbstractA):
pass
class B(AbstractB):
pass
AbstractA()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in __init__
RuntimeError: Abstract class instantiation.
AbstractB()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in __init__
RuntimeError: Abstract class instantiation.
A()
<__main__.A object at 0x7f0bba5112e8>
B()
<__main__.B object at 0x7f0bba511438>
class A(object):
def __init__(self):
if self.__class__ == A:
raise RuntimeError("Abstract class instantiation.")
print(self.__class__.__name__)
class B(A):
pass
>>> A()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in __init__
RuntimeError: Abstract class instantiation.
>>> B()
B
<__main__.B object at 0x7f8c816a58d0>
>>>
In Python, you would probably implement the "abstract" base classes as mix-ins. There's nothing special you need to do; by convention, you would add Mixin to the name to indicate that it isn't meant to be instantiated directly, but simply used as a base class for other classes.
class AMixin:
def __init__(self):
# do initialization stuff
def very_common_method(self, ...):
# do very common stuff
class BMixin(AMixin):
# do not duplicate initialization stuff here, inherit instead
def less_common_method(self, ...):
# do less common stuff
class AX(AMixin):
def specific_method_1(self, ...):
class BX(BMixin):
def specific_method_2(self, ...):
class Foo:
...
class Bar(Foo, BMixin):
...
I have written the follow code to demonstrate abstract methods that must be implemented by its subclasses. I read that when a method in a parent class is decorated as abstract, its subclass must implement it otherwise it cannot be instantiated. However, in the following code, the 'Slug' subclass does not implement the abstract method, yet it can still instantiate without errors being thrown. I thought in this case Python will complain?
Have I misunderstood something?
Thanks
import abc
class Animal(object):
__metaclass__ = abc.ABCMeta
def __init__(self, species):
self._species=species
def get_species(self):
return self._species
#abc.abstractmethod
def eat(self):
pass
class Slug(Animal):
def __init(self, species):
super(species)
def run(self):
print("running")
# def eat(self):
# pass
sl = Slug("slug")
print(sl.get_species())
no, you understood perfectly well! it is just that the python 3 syntax for abc.ABCMeta is:
class Animal(metaclass=abc.ABCMeta):
...
instead of __metaclass__ = abc.ABCMeta which was used in python 2. alternatively you can just inherit from ABC: class Animal(abc.ABC):. see doc.
and then:
class Slug(Animal):
def __init(self, species):
super.__init__(species)
...
this will result in
Traceback (most recent call last):
File "/home/.../...py", line 33, in <module>
sl = Slug("slug")
TypeError: Can't instantiate abstract class Slug with abstract methods eat
class F(object):
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def f(self, a, b): print("F")
class FF(F):
def f(self): print("FF")
f = FF()
f.f()
Here, I define an abstract method f with two arguments. I want to restrict subclass so that it has same arguments like superclass.
How to do that?
There isn't any trivial way to check that the method signatures match.
However, assuming:
Your subclass only the derives from the concerned abstract class or the abstract class is the first in its mro
And that the implementation of the abstract class is empty, to eliminate any possibility of unwanted results such as binding new attributes to the subclass,
You could make a super call to the abstract method passing the paramters from the subclass method. The call only serves to enforce a signature match.
class F(object):
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def f(self, a, b): pass
class FF(F):
def f(self, *args, **kwargs):
super(FF, self).f(*args, **kwargs)
...
f = FF()
f.f()
Traceback (most recent call last):
File "python", line 17, in <module>
File "python", line 12, in f
TypeError: f() takes exactly 3 arguments (1 given)
Python abstract methods (unlike others) can have implementations and can be called via super; so maybe this is one use case after all.
You can use abcmeta library: https://github.com/mortymacs/abcmeta
. It gives you more restrictions on derived classes.
I created the empty abstract class AbstractStorage and inherited the Storage class from it:
import abc
import pymongo as mongo
host = mongo.MongoClient()
print(host.alive()) # True
class AbstractStorage(metaclass=abc.ABCMeta):
pass
class Storage(AbstractStorage):
dbh = host
def __init__(self):
print('__init__')
Storage()
I expected the output to be
True
__init__
however, the one I'm getting is
True
Traceback (most recent call last):
File "/home/vaultah/run.py", line 16, in <module>
Storage()
TypeError: Can't instantiate abstract class Storage with abstract methods dbh
The problem (apparently) goes away if I remove metaclass=abc.ABCMeta (so that AbstractStorage becomes an ordinary class) and/or if I set dbh to some other value.
What's going on here?
This isn't really a problem with ABCs, it's a problem with PyMongo. There is an issue about it here. It seems that pymongo overrides __getattr__ to return some sort of database class. This means that host.__isabstractmethod__ returns a Database object, which is true in a boolean context. This cause ABCMeta to believe that host is an abstract method:
>>> bool(host.__isabstractmethod__)
True
The workaround described in the issue report is to manually set host.__isabstractmethod__ = False on your object. The last comment on the issue suggests a fix has been put in for pymongo 3.0.
Short Version
mongo.MongoClient returns an object that appears to be (is?) an abstract method, which you then assign to the dbh field in Storage. This makes Storage an abstract class, so instantiating it raises a TypeError.
Note that I don't have pymongo, so I can't tell you anything more about MongoClient than how it gets treated by ABCMeta.
Long Version
The ABCMeta.__new__ method looks inside each field of the new class it's creating. Any field that itself has a True (or "true-like") __isabstractmethod__ field is considered an abstract method. If a class has any non-overridden abstract methods, the whole class is considered abstract, so any attempt to instantiate it is an error.
From an earlier version of the standard library's abc.py:
def __new__(mcls, name, bases, namespace):
cls = super().__new__(mcls, name, bases, namespace)
# Compute set of abstract method names
abstracts = {name
for name, value in namespace.items()
if getattr(value, "__isabstractmethod__", False)}
# ...
cls.__abstractmethods__ = frozenset(abstracts)
# ...
This is not mentioned in the abc.ABCMeta class docs, but a bit lower, under the #abc.abstractmethod decorator:
In order to correctly interoperate with the abstract base class machinery, the descriptor must identify itself as abstract using __isabstractmethod__. In general, this attribute should be True if any of the methods used to compose the descriptor are abstract.
Example
I created a bogus "abstract-looking" class with an __isabstractmethod__ attribute, and two supposedly-concrete subclasses of AbstractStorage. You'll see that one produces the exact error you're getting:
#!/usr/bin/env python3
import abc
# I don't have pymongo, so I have to fake it. See CounterfeitAbstractMethod.
#import pymongo as mongo
class CounterfeitAbstractMethod():
"""
This class appears to be an abstract method to the abc.ABCMeta.__new__
method.
Normally, finding an abstract method in a class's namespace means
that class is also abstract, so instantiating that class is an
error.
If a class derived from abc.ABCMeta has an instance of
CounterfeitAbstractMethod as a value anywhere in its namespace
dictionary, any attempt to instantiate that class will raise a
TypeError: Can't instantiate abstract class <classname> with
abstract method <fieldname>.
"""
__isabstractmethod__ = True
class AbstractStorage(metaclass=abc.ABCMeta):
def __init__(self):
"""
Do-nothing initializer that prints the name of the (sub)class
being initialized.
"""
print(self.__class__.__name__ + ".__init__ executing.")
return
class ConcreteStorage(AbstractStorage):
"""
A concrete class that also _appears_ concrete to abc.ABCMeta. This
class can be instantiated normally.
"""
whatever = "Anything that doesn't appear to be an abstract method will do."
class BogusStorage(AbstractStorage):
"""
This is (supposedly) a concrete class, but its whatever field appears
to be an abstract method, making this whole class abstract ---
abc.ABCMeta will refuse to construct any this class.
"""
#whatever = mongo.MongoClient('localhost', 27017)
whatever = CounterfeitAbstractMethod()
def main():
"""
Print details of the ConcreteStorage and BogusStorage classes.
"""
for cls in ConcreteStorage, BogusStorage:
print(cls.__name__ + ":")
print(" whatever field: " + str(cls.whatever))
print(" abstract methods: " + str(cls.__abstractmethods__))
print(" Instantiating...")
print(" ", end="")
# KABOOM! Instantiating BogusStorage will raise a TypeError,
# because it appears to be an _abstract_ class.
instance = cls()
print(" instance: " + str(instance))
print()
return
if "__main__" == __name__:
main()
Running this produces:
$ ./storage.py
ConcreteStorage:
whatever field: Anything that doesn't appear to be an abstract method will do.
abstract methods: frozenset()
Instantiating...
ConcreteStorage.__init__ executing.
instance: <__main__.ConcreteStorage object at 0x253afd0>
BogusStorage:
whatever field: <__main__.CounterfeitAbstractMethod object at 0x253ad50>
abstract methods: frozenset({'whatever'})
Instantiating...
Traceback (most recent call last):
File "./storage.py", line 75, in <module>
main()
File "./storage.py", line 68, in main
instance = cls()
TypeError: Can't instantiate abstract class BogusStorage with abstract methods whatever