When I was reading the code online, I have encountered the following cases of using abstract classes:
from abc import abstractmethod,ABCMeta
class Generator(object,metaclass=ABCMeta):
#abstractmethod
def generate(self):
raise NotImplementedError("method not implemented")
generator=Generator()
generator.generate()
The following error is returned, as expected:
TypeError: Can't instantiate abstract class Generator with abstract methods generate
But if I write it like this (the only difference is in the second line)
from abc import abstractmethod,ABCMeta
class Generator(object):
#abstractmethod
def generate(self):
raise NotImplementedError("method not implemented")
generator=Generator()
generator.generate()
Although there are changes in the error message,
NotImplementedError: method not implemented
When I implemented the generate method, both of the above ways of Generator were executed correctly,
class GeneticAlgorithm(Generator):
def generate(self):
print("ABC")
ga=GeneticAlgorithm()
ga.generate()
>>> ABC
So why do we need the statement metaclass=ABCMeta?
I know something from GeeksforGeeks that
ABCMeta metaclass provides a method called register method that can be invoked by its instance. By using this register method, any abstract base class can become an ancestor of any arbitrary concrete class.
But this still doesn't make me understand the necessity of declaring metaclass=ABCMeta, it feels like #abstractmethod modifying the method is enough.
You "need" the metaclass=ABCMeta to enforce the rules at instantiation time.
generator=Generator() # Errors immediately when using ABCMeta
generator.generate() # Only errors if and when you call generate otherwise
Imagine if the class had several abstract methods, only some of which were implemented in a child. It might work for quite a while, and only error when you got around to calling an unimplemented method. Failing eagerly before you rely on the ABC is generally a good thing, in the same way it's usually better for a function to raise an exception rather than just returning None to indicate failure; you want to know as soon as things are wrong, not get a weird error later without knowing the ultimate cause of the error.
Side-note: There's a much more succinct way to be an ABC than explicitly using the metaclass=ABCMeta syntax:
from abc import abstractmethod, ABC
class Generator(ABC):
Python almost always makes empty base classes that use the metaclass to simplify use (especially during the 2 to 3 transition period, where there was no compatible metaclass syntax that worked in both, and direct inheritance was the only thing that worked).
The second example,
from abc import abstractmethod,ABCMeta
class Generator(object):
#abstractmethod
def generate(self):
raise NotImplementedError("method not implemented")
generator=Generator()
generator.generate() # HERE it raises
does not use the #abstractclass decorator in any way. It only raises the NotImplemetedError exception when the generate() function is called, whereas in the first example an error is raised on the instantiation (generator=Generator()).
In order to use #abstractmethod, the class has to have metaclass ABCMeta (or it has to inherit from such class, e.g. ABC).
By the way, there are ways to check whether a method is implemented even sooner (on class definition) using, for example __init_subclass__ or by defining a custom metaclass and modifying its __new__ method.
Related
Consider the following class and mixin:
class Target(ClassThatUsesAMetaclass):
def foo(self):
pass
class Mixin:
def __init__(self):
self.foo() # type error: type checker doesn't know Mixin will have
# access to foo once in use.
class Combined(Mixin, Target):
def __init__(self):
Target.__init__(self)
Mixin.__init__(self)
I'm trying to avoid the type checker error in the above scenario. One option is this:
from typing import Protocol
class Fooable(Protocol):
def foo(self): ...
class Mixin(Fooable):
def __init__(self):
self.foo()
Would've worked great, except that Target inherits from a class that uses a metaclass, so Combined can't inherit from both Target and Mixin.
So now I'm trying an alternative, annotating self in Mixin:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .this import Mixin, Target
Mixin_T = type('Mixin_T', (Mixin, Target), {})
class Mixin:
def __init__(self: Mixin_T):
self.foo() # No longer an error
class Combined(Mixin, Target):
def __init__(self):
Target.__init__(self)
Mixin.__init__(self) # Now this is an error: "Type[Mixin]" is not
# assignable to parameter "self"
# "Mixin" is incompatible with "Mixin_T"
So how am I supposed to win this aside from using # type: ignore?
I found a very simple solution:
if TYPE_CHECKING:
from .this import Target
Mixin_T = Target
else:
Mixin_T = object
class Mixin(Mixin_T):
...
Now all of Target's methods are recognized within Mixin by the type checker, and there's no need to override the type of self into something imcompatible with Mixin. This might be a little awkward if the mixin is destined to all kinds of Target classes, but for my uses this is perfectly acceptable, since my case is a group of mixins extending a very specific target class.
Other than that, there is to little code and some msconceptions above that make this question not answrable at all, apart from providing some clarifications.
To start, are you sure you are "inheriting from a metaclass"?? It does not make sense to inherit a metaclass unless to create another metaclass. Your snippets show you inhriting froma supposed metaclass (with no code given), to create Target and them attempting to use Target as a parent to a normal class (a non-meta class). That makes no sense.
You might just have confused the terms and the hidden InheritFromMetaclass class actually just uses the metaclass, and do not "inherit" from it. Then your problem does not have to do with metaclasses at all.
So, the real visible problem in the snippet is that the static checkr does not "see" a self.foo method in the Mixin class - and guess what? There is no self.foo method in Mixin - the checker is just throwing a cold truth in your face: while Python does allow one to reference methods and attributes that are not available in a class, knowing that it will be used along other classes that do have those attributes, that is no good design and error prone. The kind of bad design static type checking exists to weed-off.
So, what you need is to have a base of Mixin that is an abstract class and have Foo as an abstract method. (Or have Mixin itself be that abstract class).
If - due to usage of other metaclass you can't have Mixin inheit from abc.ABC due to metaclass conflict, you have to either: create a combined metaclass from the metaclass acutually used by InheritsFromMetaclass with ABCMeta , nd use that as the metaclass for Mixin - or just create a stub foo method in Mixin as is (which could raise a NotImplementedError - thus having the same behavior of an abstract method, but without really having to inherit from it.
The important part to have in and is that an methods and attributes you access in code inside a class body have to exist in that class, without depending on attributes that will exist in a subclass of it.
If that does not solve your problem, you need to provide more data - including a reproducible complete example involving your actual metaclass. (and it mgt be solved just by combining the metaclasses as mentioned above)
I'm trying to create mocks from scratch that can pass the test issubclass(class_mock, base_class) where the base class is an abstract class derived from abc.ABC. Before you ask the question, I will answer why I'm trying to do it.
I have an internal package containing a base class and a collection of sub-classes that properly implement the abstract interface. Besides, I have a factory class that can instantiate the sub-classes. The factory is built is such a way that it can inspect its own package and have access to the existing sub-classes. The factory is meant to be always in the same package as the derived and base class (constraint). I think you guessed that I'm actually testing the factory... However, since the sub-classes can change in number, their name or their package name, etc., I cannot implement a correct unit test that directly refers to the actual cub-classes (because it introduces a coupling) and I need mocks.
The problem is that I didn't succeed to create a mock that satisfies the above conditions for a class derived from an abstract class. What I was able to achieve is for a class derived from another non-abstract class.
Here is the code that illustrates the problem more concretely.
import unittest.mock
import inspect
import abc
class A:
pass
class B(A):
pass
class TestSubClass(unittest.TestCase):
def test_sub_class(self):
b_class_mock = self._create_class_mock("B", A)
print(isinstance(b_class_mock, type))
print(inspect.isclass(b_class_mock))
print(issubclass(b_class_mock, A))
#staticmethod
def _create_class_mock(mock_name, base_class):
class_mock = unittest.mock.MagicMock(spec=type(base_class), name=mock_name)
class_mock.__bases__ = (base_class,)
return class_mock
So, for this code, everything is ok. It prints 3 True as wanted.
But as long as the class A is defined as abstract (class A(abc.ABC)), the last test is failing with an error saying that the mock is not a class even if the 2 previous tests are saying the opposite.
I dived a bit into the implementation of abc.ABCMeta and found out that __subclasscheck__ is overridden. I tried to know the process behind it but when I reached the C code and everything became a way more complicated, I tried to rather track when the error message is generated. Unfortunately, I didn't succeed to understand why it is actually not working.
I'm trying to create a custom base class for exceptions raised within my application. I want to make it an abstract class (using the abc module) that cannot be directly instantiated, to force myself to define more specific concrete subclasses for different types of error situations.
I can define an abstract class that inherits from a custom concrete class. However, to my surprise, if I make the abstract class inherit (directly or indirectly) from Exception, it can again be instantiated directly, defeating the purpose of making it abstract in the first place.
I want to understand what's happening here so that I can make my custom exception class abstract for real and not directly instantiable.
I've tried different variations of declaring my custom exception class as abstract, including using the metaclass=abc.ABCMeta syntax as well as inheriting from abc.ABC. Regardless of the way it is declared as abstract, it ceases to behave like an abstract class as soon as I make it inherit from Exception.
Relevant Python versions for me are 3.5, 3.6 and 3.7. I've tested the below code on Python 3.5.2 (Ubuntu 16.04) and 3.6.8 (Ubuntu 18.04).
The following seems to work as expected: instantiating AppException fails because of the abstract method (TypeError: Can't instantiate abstract class AppException with abstract methods abs_method).
Note that although the classes are called *Exception, they are not (yet) inheriting from the real built-in Exception class.
import abc
class BaseException():
pass
class AppException(BaseException, abc.ABC):
#abc.abstractmethod
def abs_method(self):
pass
class ConcreteException(AppException):
def abs_method(self):
return "concrete method"
# BaseException can be instantiated just fine
a = BaseException()
# ConcreteException can be instantiated just fine
c = ConcreteException()
# It shouldn't be possible to instantiate AppException directly,
# so this line should raise a TypeError
b = AppException()
When I change the definition of BaseException to inherit from the actual Exception class:
class BaseException(Exception):
pass
then the TypeError goes away, so the instantiation of AppException did work this time. AppException is no longer behaving as an abstract class, even though in my understanding, it should.
Here is the actual code, currently stuck as a draft PR until I can figure out what's going on.
This was covered earlier in this SO discussion.
There is no obvious solution, but one possible workaround (from the above SO discussion) is to add an __init__ method into the "abstract" extension class that prevents it from being instantiated directly.
I'm trying to figure out how to ensure that a method of a class inheriting from an ABC is created using the appropriate decorator. I understand (hopefully) how ABCs work in general.
from abc import ABCMeta, abstractmethod
class MyABC(metaclass=ABCMeta):
#abstractmethod
def my_abstract_method(self):
pass
class MyClass(MyABC):
pass
MyClass()
This gives "TypeError: Can't instantiate abstract class MyClass with abstract methods my_abstract_method". Great, makes sense. Just create a method with that name.
class MyClass(MyABC):
def my_abstract_method(self):
pass
MyClass()
Boom. You're done. But what about this case?
from abc import ABCMeta, abstractmethod
class MyABC(metaclass=ABCMeta):
#property
#abstractmethod
def my_attribute(self):
pass
class MyClass(MyABC):
def my_attribute(self):
pass
MyClass()
The MyClass() call works even though my_attribute is not a property. I guess in the end all ABCs do is ensure that a method with a given name exists. Thats it. If you want more from it, you have to look at MyABC's source code and read the documentation. The decorators and comments there will inform you of how you need to construct your sub-class.
Do I have it right or am I missing something here?
You're correct that ABCs do not enforce that. There isn't a way to enforce something like "has a particular decorator". Decorators are just functions that return objects (e.g., property returns a property object). ABCMeta doesn't do anything to ensure that the defined attributes on the class are anything in particular; it just makes sure they are there. This "works" without errors:
class MyABC(metaclass=ABCMeta):
#abstractmethod
def my_abstract_method(self):
pass
class MyClass(MyABC):
my_abstract_method = 2
MyClass()
That is, ABCMeta doesn't even ensure that the abstract method as provided on the subclass is a method at all. There just has to be an attribute of some kind with that name,
You could certainly write your own metaclass that does more sophisticated checking to ensure that certain attributes have certain kinds of values, but that's beyond the scope of ABCMeta.
I need something like an abstract protected method in Python (3.2):
class Abstract:
def use_concrete_implementation(self):
print(self._concrete_method())
def _concrete_method(self):
raise NotImplementedError()
class Concrete(Abstract):
def _concrete_method(self):
return 2 * 3
Is it actually useful to define an "abstract" method only to raise a NotImplementedError?
Is it good style to use an underscore for abstract methods, that would be protected in other languages?
Would an abstract base class (abc) improve anything?
In Python, you usually avoid having such abstract methods alltogether. You define an interface by the documentation, and simply assume the objects that are passed in fulfil that interface ("duck typing").
If you really want to define an abstract base class with abstract methods, this can be done using the abc module:
from abc import ABCMeta, abstractmethod
class Abstract(metaclass=ABCMeta):
def use_concrete_implementation(self):
print(self._concrete_method())
#abstractmethod
def _concrete_method(self):
pass
class Concrete(Abstract):
def _concrete_method(self):
return 2 * 3
Again, that is not the usual Python way to do things. One of the main objectives of the abc module was to introduce a mechanism to overload isinstance(), but isinstance() checks are normally avoided in favour of duck typing. Use it if you need it, but not as a general pattern for defining interfaces.
When in doubt, do as Guido does.
No underscore. Just define the "abstract method" as a one-liner which raises NotImplementedError:
class Abstract():
def ConcreteMethod(self):
raise NotImplementedError("error message")
Basically, an empty method in the base class is not necessary here. Just do it like this:
class Abstract:
def use_concrete_implementation(self):
print(self._concrete_method())
class Concrete(Abstract):
def _concrete_method(self):
return 2 * 3
In fact, you usually don't even need the base class in Python. Since all calls are resolved dynamically, if the method is present, it will be invoked, if not, an AttributeError will be raised.
Attention: It is import to mention in the documentation that _concrete_method needs to be implemented in subclasses.