I always thought one should inherit from abc.ABC when one does not want the class to be instantiated. But I've just realized that if a class has an #abstractmethod then one can also not instanciate it.
Is there any other reason to inherit from ABC?
Unless you use abc.ABCMeta as the metaclass for your class (either explicitly or by inheriting from abc.ABC), using abstractmethod doesn't really do anything.
>>> from abc import abstractmethod, ABC
>>> class Foo:
... #abstractmethod
... def bar(self):
... pass
...
>>> f = Foo()
>>>
Likewise, using ABCMeta doesn't mean much unless you mark at least one method as abstract:
>>> class Bar(ABC):
... pass
...
>>> b = Bar()
>>>
It's the combination of the two that allows a class to be (nominally) uninstantiable:
>>> class Baz(ABC):
... #abstractmethod
... def m(self):
... pass
...
>>> b = Baz()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Baz with abstract methods m
>>>
(Even then, note that all #abstractmethod does is add the decorated method to a set which the metaclass machinery consults when trying to instantiate the class. It is trivial to defeat that machinery:
>>> Baz.__abstractmethods__
frozenset({'m'})
>>> Baz.__abstractmethods__ = set()
>>> b = Baz()
>>>
)
Note that ABC itself is a trivial class that uses ABCMeta as its metaclass, which makes any of its descendants use it as well.
# Docstring omitted; see
# https://github.com/python/cpython/blob/3.7/Lib/abc.py#L166
# for the original
class ABC(metaclass=ABCMeta):
__slots__ = ()
What chepner said, and also readability. Inheriting from ABC makes it clear to your readers what you're up to.
>>> from abc import ABC, abstractmethod
>>>
>>> class Foo:
... #abstractmethod
... def f(self):
... pass
...
>>> class Bar(Foo):
... pass
...
>>> Bar().f()
>>>
>>> class Baz(ABC):
... #abstractmethod
... def f(self):
... pass
...
>>> class Quux(Baz):
... pass
...
>>> Quux().f()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Quux with abstract methods f
Related
By definition, we cannot instantiate an abstract class:
>>> import abc
>>> class A(abc.ABC):
... #abc.abstractmethod
... def f(self): raise NotImplementedError
...
>>> A()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class A with abstract method f
So isn’t it a contradiction that an instance of a concrete subclass is an instance of the abstract class?
>>> class B(A):
... def f(self): return 'foo'
...
>>> isinstance(B(), A)
True
There's a difference between an object being an instance of a class and the act of instantiating a class. Inheritance means that if B is a subclass of A, then isinstance(B(), A) is true, even though B, not A, is the class being instantiated.
If you could never concretize an abstract class, there would be no point in defining the abstract class in the first place. The purpose of an abstract class is to provide an incomplete template for other classes; you can't simply use the abstract class as-is without doing making some additional definitions.
Put another way, given b = B(), b is an instance of A and B, but only B is the type of b. Is-type-of and is-instance-of are two different relations.
This is the setup I want:
A should be an abstract base class with a static & abstract method f(). B should inherit from A. Requirements:
1. You should not be able to instantiate A
2. You should not be able to instantiate B, unless it implements a static f()
Taking inspiration from this question, I've tried a couple of approaches. With these definitions:
class abstractstatic(staticmethod):
__slots__ = ()
def __init__(self, function):
super(abstractstatic, self).__init__(function)
function.__isabstractmethod__ = True
__isabstractmethod__ = True
class A:
__metaclass__ = abc.ABCMeta
#abstractstatic
def f():
pass
class B(A):
def f(self):
print 'f'
class A2:
__metaclass__ = abc.ABCMeta
#staticmethod
#abc.abstractmethod
def f():
pass
class B2(A2):
def f(self):
print 'f'
Here A2 and B2 are defined using usual Python conventions and A & B are defined using the way suggested in this answer. Following are some operations I tried and the results that were undesired.
With classes A/B:
>>> B().f()
f
#This should have thrown, since B doesn't implement a static f()
With classes A2/B2:
>>> A2()
<__main__.A2 object at 0x105beea90>
#This should have thrown since A2 should be an uninstantiable abstract class
>>> B2().f()
f
#This should have thrown, since B2 doesn't implement a static f()
Since neither of these approaches give me the output I want, how do I achieve what I want?
You can't do what you want with just ABCMeta. ABC enforcement doesn't do any type checking, only the presence of an attribute with the correct name is enforced.
Take for example:
>>> from abc import ABCMeta, abstractmethod, abstractproperty
>>> class Abstract(object):
... __metaclass__ = ABCMeta
... #abstractmethod
... def foo(self): pass
... #abstractproperty
... def bar(self): pass
...
>>> class Concrete(Abstract):
... foo = 'bar'
... bar = 'baz'
...
>>> Concrete()
<__main__.Concrete object at 0x104b4df90>
I was able to construct Concrete() even though both foo and bar are simple attributes.
The ABCMeta metaclass only tracks how many objects are left with the __isabstractmethod__ attribute being true; when creating a class from the metaclass (ABCMeta.__new__ is called) the cls.__abstractmethods__ attribute is then set to a frozenset object with all the names that are still abstract.
type.__new__ then tests for that frozenset and throws a TypeError if you try to create an instance.
You'd have to produce your own __new__ method here; subclass ABCMeta and add type checking in a new __new__ method. That method should look for __abstractmethods__ sets on the base classes, find the corresponding objects with the __isabstractmethod__ attribute in the MRO, then does typechecking on the current class attributes.
This'd mean that you'd throw the exception when defining the class, not an instance, however. For that to work you'd add a __call__ method to your ABCMeta subclass and have that throw the exception based on information gathered by your own __new__ method about what types were wrong; a similar two-stage process as what ABCMeta and type.__new__ do at the moment. Alternatively, update the __abstractmethods__ set on the class to add any names that were implemented but with the wrong type and leave it to type.__new__ to throw the exception.
The following implementation takes that last tack; add names back to __abstractmethods__ if the implemented type doesn't match (using a mapping):
from types import FunctionType
class ABCMetaTypeCheck(ABCMeta):
_typemap = { # map abstract type to expected implementation type
abstractproperty: property,
abstractstatic: staticmethod,
# abstractmethods return function objects
FunctionType: FunctionType,
}
def __new__(mcls, name, bases, namespace):
cls = super(ABCMetaTypeCheck, mcls).__new__(mcls, name, bases, namespace)
wrong_type = set()
seen = set()
abstractmethods = cls.__abstractmethods__
for base in bases:
for name in getattr(base, "__abstractmethods__", set()):
if name in seen or name in abstractmethods:
continue # still abstract or later overridden
value = base.__dict__.get(name) # bypass descriptors
if getattr(value, "__isabstractmethod__", False):
seen.add(name)
expected = mcls._typemap[type(value)]
if not isinstance(namespace[name], expected):
wrong_type.add(name)
if wrong_type:
cls.__abstractmethods__ = abstractmethods | frozenset(wrong_type)
return cls
With this metaclass you get your expected output:
>>> class Abstract(object):
... __metaclass__ = ABCMetaTypeCheck
... #abstractmethod
... def foo(self): pass
... #abstractproperty
... def bar(self): pass
... #abstractstatic
... def baz(): pass
...
>>> class ConcreteWrong(Abstract):
... foo = 'bar'
... bar = 'baz'
... baz = 'spam'
...
>>> ConcreteWrong()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class ConcreteWrong with abstract methods bar, baz, foo
>>>
>>> class ConcreteCorrect(Abstract):
... def foo(self): return 'bar'
... #property
... def bar(self): return 'baz'
... #staticmethod
... def baz(): return 'spam'
...
>>> ConcreteCorrect()
<__main__.ConcreteCorrect object at 0x104ce1d10>
Following on from this question, I attempted to patch class A() with Mock() so that when B() was initialised, the Mock was used as a base e.g.:
class A(object): ...
class B(A): ...
def setUp(self):
with patch('A', new_callable=Mock) as MockObject:
self.b = B()
self.b.__class__.__base__ = MockOjbect
Which doesn't work because base is read only. What's the correct way to go about doing this?
update:
>>> from mock import Mock
>>> class A(object):
... pass
...
>>> class B(A):
... pass
...
>>> b.__class__.__bases__ = (Mock, )
>>> b.__class__.__bases__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/envs/myenv/local/lib/python2.7/site-packages/mock.py", line 656, in __getattr__
elif self._mock_methods is not None:
File "/opt/envs/myenv/local/lib/python2.7/site-packages/mock.py", line 655, in __getattr__
raise AttributeError(name)
AttributeError: _mock_methods
To be clear, I'm not convinced this is the best way to achieve what I want to do, I'm half hoping someone else will come up with another way.
It's __bases__ which is a tuple.
Corrected version:
class A(object): ...
class B(A): ...
def setUp(self):
with patch('A', new_callable=Mock) as MockObject:
self.b = B()
self.b.__class__.__bases__ = (MockOjbect,)
See:
>>> class Foo(object):
... pass
...
>>> Foo.__class__.__bases__
(<type 'object'>,)
tuple's are immutable but the __bases__ attribute is most certainly not read-only.
I am looking for ways / best practices on testing methods defined in an abstract base class. One thing I can think of directly is performing the test on all concrete subclasses of the base class, but that seems excessive at some times.
Consider this example:
import abc
class Abstract(object):
__metaclass__ = abc.ABCMeta
#abc.abstractproperty
def id(self):
return
#abc.abstractmethod
def foo(self):
print "foo"
def bar(self):
print "bar"
Is it possible to test bar without doing any subclassing?
In newer versions of Python you can use unittest.mock.patch()
class MyAbcClassTest(unittest.TestCase):
#patch.multiple(MyAbcClass, __abstractmethods__=set())
def test(self):
self.instance = MyAbcClass() # Ha!
Here is what I have found: If you set __abstractmethods__ attribute to be an empty set you'll be able to instantiate abstract class. This behaviour is specified in PEP 3119:
If the resulting __abstractmethods__ set is non-empty, the class is considered abstract, and attempts to instantiate it will raise TypeError.
So you just need to clear this attribute for the duration of tests.
>>> import abc
>>> class A(metaclass = abc.ABCMeta):
... #abc.abstractmethod
... def foo(self): pass
You cant instantiate A:
>>> A()
Traceback (most recent call last):
TypeError: Can't instantiate abstract class A with abstract methods foo
If you override __abstractmethods__ you can:
>>> A.__abstractmethods__=set()
>>> A() #doctest: +ELLIPSIS
<....A object at 0x...>
It works both ways:
>>> class B(object): pass
>>> B() #doctest: +ELLIPSIS
<....B object at 0x...>
>>> B.__abstractmethods__={"foo"}
>>> B()
Traceback (most recent call last):
TypeError: Can't instantiate abstract class B with abstract methods foo
You can also use unittest.mock (from 3.3) to override temporarily ABC behaviour.
>>> class A(metaclass = abc.ABCMeta):
... #abc.abstractmethod
... def foo(self): pass
>>> from unittest.mock import patch
>>> p = patch.multiple(A, __abstractmethods__=set())
>>> p.start()
{}
>>> A() #doctest: +ELLIPSIS
<....A object at 0x...>
>>> p.stop()
>>> A()
Traceback (most recent call last):
TypeError: Can't instantiate abstract class A with abstract methods foo
As properly put by lunaryon, it is not possible. The very purpose of ABCs containing abstract methods is that they are not instantiatable as declared.
However, it is possible to create a utility function that introspects an ABC, and creates a dummy, non abstract class on the fly. This function could be called directly inside your test method/function and spare you of having to wite boiler plate code on the test file just for testing a few methods.
def concreter(abclass):
"""
>>> import abc
>>> class Abstract(metaclass=abc.ABCMeta):
... #abc.abstractmethod
... def bar(self):
... return None
>>> c = concreter(Abstract)
>>> c.__name__
'dummy_concrete_Abstract'
>>> c().bar() # doctest: +ELLIPSIS
(<abc_utils.Abstract object at 0x...>, (), {})
"""
if not "__abstractmethods__" in abclass.__dict__:
return abclass
new_dict = abclass.__dict__.copy()
for abstractmethod in abclass.__abstractmethods__:
#replace each abc method or property with an identity function:
new_dict[abstractmethod] = lambda x, *args, **kw: (x, args, kw)
#creates a new class, with the overriden ABCs:
return type("dummy_concrete_%s" % abclass.__name__, (abclass,), new_dict)
You can use multiple inheritance practice to have access to the implemented methods of the abstract class. Obviously following such design decision depends on the structure of the abstract class since you need to implement abstract methods (at least bring the signature) in your test case.
Here is the example for your case:
class Abstract(object):
__metaclass__ = abc.ABCMeta
#abc.abstractproperty
def id(self):
return
#abc.abstractmethod
def foo(self):
print("foo")
def bar(self):
print("bar")
class AbstractTest(unittest.TestCase, Abstract):
def foo(self):
pass
def test_bar(self):
self.bar()
self.assertTrue(1==1)
No, it's not. The very purpose of abc is to create classes that cannot be instantiated unless all abstract attributes are overridden with concrete implementations. Hence you need to derive from the abstract base class and override all abstract methods and properties.
Perhaps a more compact version of the concreter proposed by #jsbueno could be:
def concreter(abclass):
class concreteCls(abclass):
pass
concreteCls.__abstractmethods__ = frozenset()
return type('DummyConcrete' + abclass.__name__, (concreteCls,), {})
The resulting class still has all original abstract methods (which can be now called, even if this is not likely to be useful...) and can be mocked as needed.
How can you make a class method static after the class is defined? In other words, why does the third case fail?
>>> class b:
... #staticmethod
... def foo():
... pass
...
>>> b.foo()
>>> class c:
... def foo():
... pass
... foo = staticmethod( foo )
...
>>> c.foo()
>>> class d:
... def foo():
... pass
...
>>> d.foo = staticmethod( d.foo )
>>> d.foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method foo() must be called with d instance as first argument (got nothing instead)
After the class is defined, foo is an unbound method object (a wrapper that ties the function to the class), not a function. The wrapper's going to try to pass an instance of the class (self) to the function as the first argument and so it complains that you haven't given it one.
staticmethod() should really be used only with functions. While the class is still being defined (as in your b and c) foo() is still a function, as the method wrapper is applied when the class definition is complete.
It's possible to extract the function from the unbound method object, apply staticmethod(), and assign it back to the class.
class d(object):
def foo():
pass
d.foo = staticmethod(d.foo.im_func)
In Python 3, there are no unbound instance methods (methods stored on classes are plain functions). So your original code would actually work fine on Python 3.
A way that works on both Python 2 and Python 3 is:
d.foo = staticmethod(d.__dict__['foo'])
(It also avoids creating and discarding the method wrapper.)
You must pass the function, not the unbound method.
>>> class d:
... def foo():
... pass
...
>>> d.foo = staticmethod(d.foo.im_func)
>>> d.foo()