I'm trying to create a custom base class for exceptions raised within my application. I want to make it an abstract class (using the abc module) that cannot be directly instantiated, to force myself to define more specific concrete subclasses for different types of error situations.
I can define an abstract class that inherits from a custom concrete class. However, to my surprise, if I make the abstract class inherit (directly or indirectly) from Exception, it can again be instantiated directly, defeating the purpose of making it abstract in the first place.
I want to understand what's happening here so that I can make my custom exception class abstract for real and not directly instantiable.
I've tried different variations of declaring my custom exception class as abstract, including using the metaclass=abc.ABCMeta syntax as well as inheriting from abc.ABC. Regardless of the way it is declared as abstract, it ceases to behave like an abstract class as soon as I make it inherit from Exception.
Relevant Python versions for me are 3.5, 3.6 and 3.7. I've tested the below code on Python 3.5.2 (Ubuntu 16.04) and 3.6.8 (Ubuntu 18.04).
The following seems to work as expected: instantiating AppException fails because of the abstract method (TypeError: Can't instantiate abstract class AppException with abstract methods abs_method).
Note that although the classes are called *Exception, they are not (yet) inheriting from the real built-in Exception class.
import abc
class BaseException():
pass
class AppException(BaseException, abc.ABC):
#abc.abstractmethod
def abs_method(self):
pass
class ConcreteException(AppException):
def abs_method(self):
return "concrete method"
# BaseException can be instantiated just fine
a = BaseException()
# ConcreteException can be instantiated just fine
c = ConcreteException()
# It shouldn't be possible to instantiate AppException directly,
# so this line should raise a TypeError
b = AppException()
When I change the definition of BaseException to inherit from the actual Exception class:
class BaseException(Exception):
pass
then the TypeError goes away, so the instantiation of AppException did work this time. AppException is no longer behaving as an abstract class, even though in my understanding, it should.
Here is the actual code, currently stuck as a draft PR until I can figure out what's going on.
This was covered earlier in this SO discussion.
There is no obvious solution, but one possible workaround (from the above SO discussion) is to add an __init__ method into the "abstract" extension class that prevents it from being instantiated directly.
Related
This is my code snippet of trying to create a custom exception class which takes a input parameter about the exception
class Error(Exception):
pass
mssg = "None"
class timepasserror(mssg ,Error):
print(f"the error is {mssg}")
and I am planning to use it as mentioned below
if 1==1:#some conditon
raise timepasserror('something')
I am getting this error :
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
I am not sure if its possible to do so . If yes can anyone guide me
Thanks
You are having an error because you are trying something that does not make sense, trying to inherit from a string instance (mssg = "None") - this will never work.
Otherwise one will see this error (when trying to do things correctly) when trying to inherit from two classes which have differing metaclasses, and this question have been asked around plenty of times: the solution would be creating a base metaclass that inherits from both conflicting metaclasses - Abstract class inheriting from ABC and QMainWindow
In your case, please, take a step back and find out how to implement the behavior you want in your custom exception: it won't take multiple inheritance.
When I was reading the code online, I have encountered the following cases of using abstract classes:
from abc import abstractmethod,ABCMeta
class Generator(object,metaclass=ABCMeta):
#abstractmethod
def generate(self):
raise NotImplementedError("method not implemented")
generator=Generator()
generator.generate()
The following error is returned, as expected:
TypeError: Can't instantiate abstract class Generator with abstract methods generate
But if I write it like this (the only difference is in the second line)
from abc import abstractmethod,ABCMeta
class Generator(object):
#abstractmethod
def generate(self):
raise NotImplementedError("method not implemented")
generator=Generator()
generator.generate()
Although there are changes in the error message,
NotImplementedError: method not implemented
When I implemented the generate method, both of the above ways of Generator were executed correctly,
class GeneticAlgorithm(Generator):
def generate(self):
print("ABC")
ga=GeneticAlgorithm()
ga.generate()
>>> ABC
So why do we need the statement metaclass=ABCMeta?
I know something from GeeksforGeeks that
ABCMeta metaclass provides a method called register method that can be invoked by its instance. By using this register method, any abstract base class can become an ancestor of any arbitrary concrete class.
But this still doesn't make me understand the necessity of declaring metaclass=ABCMeta, it feels like #abstractmethod modifying the method is enough.
You "need" the metaclass=ABCMeta to enforce the rules at instantiation time.
generator=Generator() # Errors immediately when using ABCMeta
generator.generate() # Only errors if and when you call generate otherwise
Imagine if the class had several abstract methods, only some of which were implemented in a child. It might work for quite a while, and only error when you got around to calling an unimplemented method. Failing eagerly before you rely on the ABC is generally a good thing, in the same way it's usually better for a function to raise an exception rather than just returning None to indicate failure; you want to know as soon as things are wrong, not get a weird error later without knowing the ultimate cause of the error.
Side-note: There's a much more succinct way to be an ABC than explicitly using the metaclass=ABCMeta syntax:
from abc import abstractmethod, ABC
class Generator(ABC):
Python almost always makes empty base classes that use the metaclass to simplify use (especially during the 2 to 3 transition period, where there was no compatible metaclass syntax that worked in both, and direct inheritance was the only thing that worked).
The second example,
from abc import abstractmethod,ABCMeta
class Generator(object):
#abstractmethod
def generate(self):
raise NotImplementedError("method not implemented")
generator=Generator()
generator.generate() # HERE it raises
does not use the #abstractclass decorator in any way. It only raises the NotImplemetedError exception when the generate() function is called, whereas in the first example an error is raised on the instantiation (generator=Generator()).
In order to use #abstractmethod, the class has to have metaclass ABCMeta (or it has to inherit from such class, e.g. ABC).
By the way, there are ways to check whether a method is implemented even sooner (on class definition) using, for example __init_subclass__ or by defining a custom metaclass and modifying its __new__ method.
Consider the following class and mixin:
class Target(ClassThatUsesAMetaclass):
def foo(self):
pass
class Mixin:
def __init__(self):
self.foo() # type error: type checker doesn't know Mixin will have
# access to foo once in use.
class Combined(Mixin, Target):
def __init__(self):
Target.__init__(self)
Mixin.__init__(self)
I'm trying to avoid the type checker error in the above scenario. One option is this:
from typing import Protocol
class Fooable(Protocol):
def foo(self): ...
class Mixin(Fooable):
def __init__(self):
self.foo()
Would've worked great, except that Target inherits from a class that uses a metaclass, so Combined can't inherit from both Target and Mixin.
So now I'm trying an alternative, annotating self in Mixin:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .this import Mixin, Target
Mixin_T = type('Mixin_T', (Mixin, Target), {})
class Mixin:
def __init__(self: Mixin_T):
self.foo() # No longer an error
class Combined(Mixin, Target):
def __init__(self):
Target.__init__(self)
Mixin.__init__(self) # Now this is an error: "Type[Mixin]" is not
# assignable to parameter "self"
# "Mixin" is incompatible with "Mixin_T"
So how am I supposed to win this aside from using # type: ignore?
I found a very simple solution:
if TYPE_CHECKING:
from .this import Target
Mixin_T = Target
else:
Mixin_T = object
class Mixin(Mixin_T):
...
Now all of Target's methods are recognized within Mixin by the type checker, and there's no need to override the type of self into something imcompatible with Mixin. This might be a little awkward if the mixin is destined to all kinds of Target classes, but for my uses this is perfectly acceptable, since my case is a group of mixins extending a very specific target class.
Other than that, there is to little code and some msconceptions above that make this question not answrable at all, apart from providing some clarifications.
To start, are you sure you are "inheriting from a metaclass"?? It does not make sense to inherit a metaclass unless to create another metaclass. Your snippets show you inhriting froma supposed metaclass (with no code given), to create Target and them attempting to use Target as a parent to a normal class (a non-meta class). That makes no sense.
You might just have confused the terms and the hidden InheritFromMetaclass class actually just uses the metaclass, and do not "inherit" from it. Then your problem does not have to do with metaclasses at all.
So, the real visible problem in the snippet is that the static checkr does not "see" a self.foo method in the Mixin class - and guess what? There is no self.foo method in Mixin - the checker is just throwing a cold truth in your face: while Python does allow one to reference methods and attributes that are not available in a class, knowing that it will be used along other classes that do have those attributes, that is no good design and error prone. The kind of bad design static type checking exists to weed-off.
So, what you need is to have a base of Mixin that is an abstract class and have Foo as an abstract method. (Or have Mixin itself be that abstract class).
If - due to usage of other metaclass you can't have Mixin inheit from abc.ABC due to metaclass conflict, you have to either: create a combined metaclass from the metaclass acutually used by InheritsFromMetaclass with ABCMeta , nd use that as the metaclass for Mixin - or just create a stub foo method in Mixin as is (which could raise a NotImplementedError - thus having the same behavior of an abstract method, but without really having to inherit from it.
The important part to have in and is that an methods and attributes you access in code inside a class body have to exist in that class, without depending on attributes that will exist in a subclass of it.
If that does not solve your problem, you need to provide more data - including a reproducible complete example involving your actual metaclass. (and it mgt be solved just by combining the metaclasses as mentioned above)
I'm trying to create mocks from scratch that can pass the test issubclass(class_mock, base_class) where the base class is an abstract class derived from abc.ABC. Before you ask the question, I will answer why I'm trying to do it.
I have an internal package containing a base class and a collection of sub-classes that properly implement the abstract interface. Besides, I have a factory class that can instantiate the sub-classes. The factory is built is such a way that it can inspect its own package and have access to the existing sub-classes. The factory is meant to be always in the same package as the derived and base class (constraint). I think you guessed that I'm actually testing the factory... However, since the sub-classes can change in number, their name or their package name, etc., I cannot implement a correct unit test that directly refers to the actual cub-classes (because it introduces a coupling) and I need mocks.
The problem is that I didn't succeed to create a mock that satisfies the above conditions for a class derived from an abstract class. What I was able to achieve is for a class derived from another non-abstract class.
Here is the code that illustrates the problem more concretely.
import unittest.mock
import inspect
import abc
class A:
pass
class B(A):
pass
class TestSubClass(unittest.TestCase):
def test_sub_class(self):
b_class_mock = self._create_class_mock("B", A)
print(isinstance(b_class_mock, type))
print(inspect.isclass(b_class_mock))
print(issubclass(b_class_mock, A))
#staticmethod
def _create_class_mock(mock_name, base_class):
class_mock = unittest.mock.MagicMock(spec=type(base_class), name=mock_name)
class_mock.__bases__ = (base_class,)
return class_mock
So, for this code, everything is ok. It prints 3 True as wanted.
But as long as the class A is defined as abstract (class A(abc.ABC)), the last test is failing with an error saying that the mock is not a class even if the 2 previous tests are saying the opposite.
I dived a bit into the implementation of abc.ABCMeta and found out that __subclasscheck__ is overridden. I tried to know the process behind it but when I reached the C code and everything became a way more complicated, I tried to rather track when the error message is generated. Unfortunately, I didn't succeed to understand why it is actually not working.
Why does the following class declaration inherit from object?
class MyClass(object):
...
Is there any reason for a class declaration to inherit from object?
In Python 3, apart from compatibility between Python 2 and 3, no reason. In Python 2, many reasons.
Python 2.x story:
In Python 2.x (from 2.2 onwards) there's two styles of classes depending on the presence or absence of object as a base-class:
"classic" style classes: they don't have object as a base class:
>>> class ClassicSpam: # no base class
... pass
>>> ClassicSpam.__bases__
()
"new" style classes: they have, directly or indirectly (e.g inherit from a built-in type), object as a base class:
>>> class NewSpam(object): # directly inherit from object
... pass
>>> NewSpam.__bases__
(<type 'object'>,)
>>> class IntSpam(int): # indirectly inherit from object...
... pass
>>> IntSpam.__bases__
(<type 'int'>,)
>>> IntSpam.__bases__[0].__bases__ # ... because int inherits from object
(<type 'object'>,)
Without a doubt, when writing a class you'll always want to go for new-style classes. The perks of doing so are numerous, to list some of them:
Support for descriptors. Specifically, the following constructs are made possible with descriptors:
classmethod: A method that receives the class as an implicit argument instead of the instance.
staticmethod: A method that does not receive the implicit argument self as a first argument.
properties with property: Create functions for managing the getting, setting and deleting of an attribute.
__slots__: Saves memory consumptions of a class and also results in faster attribute access. Of course, it does impose limitations.
The __new__ static method: lets you customize how new class instances are created.
Method resolution order (MRO): in what order the base classes of a class will be searched when trying to resolve which method to call.
Related to MRO, super calls. Also see, super() considered super.
If you don't inherit from object, forget these. A more exhaustive description of the previous bullet points along with other perks of "new" style classes can be found here.
One of the downsides of new-style classes is that the class itself is more memory demanding. Unless you're creating many class objects, though, I doubt this would be an issue and it's a negative sinking in a sea of positives.
Python 3.x story:
In Python 3, things are simplified. Only new-style classes exist (referred to plainly as classes) so, the only difference in adding object is requiring you to type in 8 more characters. This:
class ClassicSpam:
pass
is completely equivalent (apart from their name :-) to this:
class NewSpam(object):
pass
and to this:
class Spam():
pass
All have object in their __bases__.
>>> [object in cls.__bases__ for cls in {Spam, NewSpam, ClassicSpam}]
[True, True, True]
So, what should you do?
In Python 2: always inherit from object explicitly. Get the perks.
In Python 3: inherit from object if you are writing code that tries to be Python agnostic, that is, it needs to work both in Python 2 and in Python 3. Otherwise don't, it really makes no difference since Python inserts it for you behind the scenes.
Python 3
class MyClass(object): = New-style class
class MyClass: = New-style class (implicitly inherits from object)
Python 2
class MyClass(object): = New-style class
class MyClass: = OLD-STYLE CLASS
Explanation:
When defining base classes in Python 3.x, you’re allowed to drop the object from the definition. However, this can open the door for a seriously hard to track problem…
Python introduced new-style classes back in Python 2.2, and by now old-style classes are really quite old. Discussion of old-style classes is buried in the 2.x docs, and non-existent in the 3.x docs.
The problem is, the syntax for old-style classes in Python 2.x is the same as the alternative syntax for new-style classes in Python 3.x. Python 2.x is still very widely used (e.g. GAE, Web2Py), and any code (or coder) unwittingly bringing 3.x-style class definitions into 2.x code is going to end up with some seriously outdated base objects. And because old-style classes aren’t on anyone’s radar, they likely won’t know what hit them.
So just spell it out the long way and save some 2.x developer the tears.
Yes, this is a 'new style' object. It was a feature introduced in python2.2.
New style objects have a different object model to classic objects, and some things won't work properly with old style objects, for instance, super(), #property and descriptors. See this article for a good description of what a new style class is.
SO link for a description of the differences: What is the difference between old style and new style classes in Python?
History from Learn Python the Hard Way:
Python's original rendition of a class was broken in many serious
ways. By the time this fault was recognized it was already too late,
and they had to support it. In order to fix the problem, they needed
some "new class" style so that the "old classes" would keep working
but you can use the new more correct version.
They decided that they would use a word "object", lowercased, to be
the "class" that you inherit from to make a class. It is confusing,
but a class inherits from the class named "object" to make a class but
it's not an object really its a class, but don't forget to inherit
from object.
Also just to let you know what the difference between new-style classes and old-style classes is, it's that new-style classes always inherit from object class or from another class that inherited from object:
class NewStyle(object):
pass
Another example is:
class AnotherExampleOfNewStyle(NewStyle):
pass
While an old-style base class looks like this:
class OldStyle():
pass
And an old-style child class looks like this:
class OldStyleSubclass(OldStyle):
pass
You can see that an Old Style base class doesn't inherit from any other class, however, Old Style classes can, of course, inherit from one another. Inheriting from object guarantees that certain functionality is available in every Python class. New style classes were introduced in Python 2.2
Yes, it's historical. Without it, it creates an old-style class.
If you use type() on an old-style object, you just get "instance". On a new-style object you get its class.
The syntax of the class creation statement:
class <ClassName>(superclass):
#code follows
In the absence of any other superclasses that you specifically want to inherit from, the superclass should always be object, which is the root of all classes in Python.
object is technically the root of "new-style" classes in Python. But the new-style classes today are as good as being the only style of classes.
But, if you don't explicitly use the word object when creating classes, then as others mentioned, Python 3.x implicitly inherits from the object superclass. But I guess explicit is always better than implicit (hell)
Reference