Is it a good practice to have non-abstract methods on abstract base classes? I mean, methods that can, but don't have to, be present on subclasses of particular ABC?
Technically it is possible, as seen in the below example (ConcreteDataProvider does not implement disconnect; it only has to implement connect):
from abc import ABC, abstractmethod
class AbstractDataProvider(ABC):
#abstractmethod
def connect(self):
pass
def disconnect(self):
pass
class ConcreteDataProvider(AbstractDataProvider):
def connect(self):
pass
data_provider = ConcreteDataProvider()
It is fine to have ABC with concrete methods to provide default implementations. The standard library has several such cases:
The example of using abc.ABCMeta provides a non-abstract method.
The get_iterator() method is also part of the MyIterable abstract base class, but it does not have to be overridden in non-abstract derived classes.
The ABCs of the collections.abc module provide default methods (termed "Mixin Methods").
ABC
Inherits from
Abstract Methods
Mixin Methods
...
...
...
...
Iterator
Iterable
__next__
__iter__
Generator
Iterator
send, throw
close, __iter__, __next__
...
...
...
...
When "do nothing" is a sensible default, there is no problem with a concrete method having a default implementation that is just pass. This can especially be useful when you expect that many implementations will need this method: by providing a default, it is safe for client code to always call this method.
Note: When the pattern is specifically connect/disconnect, open/close, or similar pairs of methods to be called before/after usage, the __enter__/__exit__ pair of a context manager is the appropriate interface.
I'm using the abc module to define an interface that subclasses must support. There're also some common methods that are present in all subclasses. Is it ok to put them in the abstract class or should that only contain abstract methods (i.e. decorated with #abc.abstractmethod) ?
TL; DR; Yes, it is OK for an abstract class to have non-abstract methods.
Typically what we call an abstract class is just a class that cannot be instantiated.
On the other hand what we call an interface is a class which has only method declarations but no implementations. In particular its an abstract class because it doesn't have a constructor.
Of course in Python there are no real interfaces: every method has to have a body. But we can somewhat emulate interfaces via raise NotImplementedError().
Anyway interfaces form a subset of abstract classes. This obviously suggests that there are abstract classes that are not interfaces. This is exactly the case you are describing. Yes, abstract class may contain implemented, non-abstract methods. And it is not a bad practice. This is especially useful when a given method does not depend on concrete implementation.
For example consider an interface for a generic parser (I'm thinking about json.load and json.loads):
class ILoader(ABC):
#abstractmethod
def load(self, stream):
raise NotImplementedError()
It's completely OK to give loads method which accepts a string instead of stream with a default implementation:
class AbstractLoader(ABC):
#abstractmethod
def load(self, stream):
raise NotImplementedError()
def loads(self, text):
stream = io.StringIO(text)
return self.load(stream)
although I would use Abstract prefix instead of I. ;)
In python to implement an abstract class, we have to import the following:
from abc import ABCMeta, abstractmethod
In the above import statement what is the purpose of abstractmethod?
If it is a decorator, what's the exact operation its performing?
From the documentation:
#abc.abstractmethod
A decorator indicating abstract methods.
Using this decorator requires that the class’s metaclass is ABCMeta or is derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods and properties are overridden. The abstract methods can be called using any of the normal ‘super’ call mechanisms. abstractmethod() may be used to declare abstract methods for properties and descriptors.
I need something like an abstract protected method in Python (3.2):
class Abstract:
def use_concrete_implementation(self):
print(self._concrete_method())
def _concrete_method(self):
raise NotImplementedError()
class Concrete(Abstract):
def _concrete_method(self):
return 2 * 3
Is it actually useful to define an "abstract" method only to raise a NotImplementedError?
Is it good style to use an underscore for abstract methods, that would be protected in other languages?
Would an abstract base class (abc) improve anything?
In Python, you usually avoid having such abstract methods alltogether. You define an interface by the documentation, and simply assume the objects that are passed in fulfil that interface ("duck typing").
If you really want to define an abstract base class with abstract methods, this can be done using the abc module:
from abc import ABCMeta, abstractmethod
class Abstract(metaclass=ABCMeta):
def use_concrete_implementation(self):
print(self._concrete_method())
#abstractmethod
def _concrete_method(self):
pass
class Concrete(Abstract):
def _concrete_method(self):
return 2 * 3
Again, that is not the usual Python way to do things. One of the main objectives of the abc module was to introduce a mechanism to overload isinstance(), but isinstance() checks are normally avoided in favour of duck typing. Use it if you need it, but not as a general pattern for defining interfaces.
When in doubt, do as Guido does.
No underscore. Just define the "abstract method" as a one-liner which raises NotImplementedError:
class Abstract():
def ConcreteMethod(self):
raise NotImplementedError("error message")
Basically, an empty method in the base class is not necessary here. Just do it like this:
class Abstract:
def use_concrete_implementation(self):
print(self._concrete_method())
class Concrete(Abstract):
def _concrete_method(self):
return 2 * 3
In fact, you usually don't even need the base class in Python. Since all calls are resolved dynamically, if the method is present, it will be invoked, if not, an AttributeError will be raised.
Attention: It is import to mention in the documentation that _concrete_method needs to be implemented in subclasses.
What is the difference between abstract class and interface in Python?
What you'll see sometimes is the following:
class Abstract1:
"""Some description that tells you it's abstract,
often listing the methods you're expected to supply."""
def aMethod(self):
raise NotImplementedError("Should have implemented this")
Because Python doesn't have (and doesn't need) a formal Interface contract, the Java-style distinction between abstraction and interface doesn't exist. If someone goes through the effort to define a formal interface, it will also be an abstract class. The only differences would be in the stated intent in the docstring.
And the difference between abstract and interface is a hairsplitting thing when you have duck typing.
Java uses interfaces because it doesn't have multiple inheritance.
Because Python has multiple inheritance, you may also see something like this
class SomeAbstraction:
pass # lots of stuff - but missing something
class Mixin1:
def something(self):
pass # one implementation
class Mixin2:
def something(self):
pass # another
class Concrete1(SomeAbstraction, Mixin1):
pass
class Concrete2(SomeAbstraction, Mixin2):
pass
This uses a kind of abstract superclass with mixins to create concrete subclasses that are disjoint.
What is the difference between abstract class and interface in Python?
An interface, for an object, is a set of methods and attributes on that object.
In Python, we can use an abstract base class to define and enforce an interface.
Using an Abstract Base Class
For example, say we want to use one of the abstract base classes from the collections module:
import collections
class MySet(collections.Set):
pass
If we try to use it, we get an TypeError because the class we created does not support the expected behavior of sets:
>>> MySet()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class MySet with abstract methods
__contains__, __iter__, __len__
So we are required to implement at least __contains__, __iter__, and __len__. Let's use this implementation example from the documentation:
class ListBasedSet(collections.Set):
"""Alternate set implementation favoring space over speed
and not requiring the set elements to be hashable.
"""
def __init__(self, iterable):
self.elements = lst = []
for value in iterable:
if value not in lst:
lst.append(value)
def __iter__(self):
return iter(self.elements)
def __contains__(self, value):
return value in self.elements
def __len__(self):
return len(self.elements)
s1 = ListBasedSet('abcdef')
s2 = ListBasedSet('defghi')
overlap = s1 & s2
Implementation: Creating an Abstract Base Class
We can create our own Abstract Base Class by setting the metaclass to abc.ABCMeta and using the abc.abstractmethod decorator on relevant methods. The metaclass will be add the decorated functions to the __abstractmethods__ attribute, preventing instantiation until those are defined.
import abc
For example, "effable" is defined as something that can be expressed in words. Say we wanted to define an abstract base class that is effable, in Python 2:
class Effable(object):
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def __str__(self):
raise NotImplementedError('users must define __str__ to use this base class')
Or in Python 3, with the slight change in metaclass declaration:
class Effable(object, metaclass=abc.ABCMeta):
#abc.abstractmethod
def __str__(self):
raise NotImplementedError('users must define __str__ to use this base class')
Now if we try to create an effable object without implementing the interface:
class MyEffable(Effable):
pass
and attempt to instantiate it:
>>> MyEffable()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class MyEffable with abstract methods __str__
We are told that we haven't finished the job.
Now if we comply by providing the expected interface:
class MyEffable(Effable):
def __str__(self):
return 'expressable!'
we are then able to use the concrete version of the class derived from the abstract one:
>>> me = MyEffable()
>>> print(me)
expressable!
There are other things we could do with this, like register virtual subclasses that already implement these interfaces, but I think that is beyond the scope of this question. The other methods demonstrated here would have to adapt this method using the abc module to do so, however.
Conclusion
We have demonstrated that the creation of an Abstract Base Class defines interfaces for custom objects in Python.
Python >= 2.6 has Abstract Base Classes.
Abstract Base Classes (abbreviated
ABCs) complement duck-typing by
providing a way to define interfaces
when other techniques like hasattr()
would be clumsy. Python comes with
many builtin ABCs for data structures
(in the collections module), numbers
(in the numbers module), and streams
(in the io module). You can create
your own ABC with the abc module.
There is also the Zope Interface module, which is used by projects outside of zope, like twisted. I'm not really familiar with it, but there's a wiki page here that might help.
In general, you don't need the concept of abstract classes, or interfaces in python (edited - see S.Lott's answer for details).
In a more basic way to explain:
An interface is sort of like an empty muffin pan.
It's a class file with a set of method definitions that have no code.
An abstract class is the same thing, but not all functions need to be empty. Some can have code. It's not strictly empty.
Why differentiate:
There's not much practical difference in Python, but on the planning level for a large project, it could be more common to talk about interfaces, since there's no code. Especially if you're working with Java programmers who are accustomed to the term.
Python doesn't really have either concept.
It uses duck typing, which removed the need for interfaces (at least for the computer :-))
Python <= 2.5:
Base classes obviously exist, but there is no explicit way to mark a method as 'pure virtual', so the class isn't really abstract.
Python >= 2.6:
Abstract base classes do exist (http://docs.python.org/library/abc.html). And allow you to specify methods that must be implemented in subclasses. I don't much like the syntax, but the feature is there. Most of the time it's probably better to use duck typing from the 'using' client side.
In general, interfaces are used only in languages that use the single-inheritance class model. In these single-inheritance languages, interfaces are typically used if any class could use a particular method or set of methods. Also in these single-inheritance languages, abstract classes are used to either have defined class variables in addition to none or more methods, or to exploit the single-inheritance model to limit the range of classes that could use a set of methods.
Languages that support the multiple-inheritance model tend to use only classes or abstract base classes and not interfaces. Since Python supports multiple inheritance, it does not use interfaces and you would want to use base classes or abstract base classes.
http://docs.python.org/library/abc.html
Abstract classes are classes that contain one or more abstract methods. Along with abstract methods, Abstract classes can have static, class and instance methods.
But in case of interface, it will only have abstract methods not other. Hence it is not compulsory to inherit abstract class but it is compulsory to inherit interface.
For completeness, we should mention PEP3119
where ABC was introduced and compared with interfaces,
and original Talin's comment.
The abstract class is not perfect interface:
belongs to the inheritance hierarchy
is mutable
But if you consider writing it your own way:
def some_function(self):
raise NotImplementedError()
interface = type(
'your_interface', (object,),
{'extra_func': some_function,
'__slots__': ['extra_func', ...]
...
'__instancecheck__': your_instance_checker,
'__subclasscheck__': your_subclass_checker
...
}
)
ok, rather as a class
or as a metaclass
and fighting with python to achieve the immutable object
and doing refactoring
...
you'll quite fast realize that you're inventing the wheel
to eventually achieve
abc.ABCMeta
abc.ABCMeta was proposed as a useful addition of the missing interface functionality,
and that's fair enough in a language like python.
Certainly, it was able to be enhanced better whilst writing version 3, and adding new syntax and immutable interface concept ...
Conclusion:
The abc.ABCMeta IS "pythonic" interface in python