Consider the following class and mixin:
class Target(ClassThatUsesAMetaclass):
def foo(self):
pass
class Mixin:
def __init__(self):
self.foo() # type error: type checker doesn't know Mixin will have
# access to foo once in use.
class Combined(Mixin, Target):
def __init__(self):
Target.__init__(self)
Mixin.__init__(self)
I'm trying to avoid the type checker error in the above scenario. One option is this:
from typing import Protocol
class Fooable(Protocol):
def foo(self): ...
class Mixin(Fooable):
def __init__(self):
self.foo()
Would've worked great, except that Target inherits from a class that uses a metaclass, so Combined can't inherit from both Target and Mixin.
So now I'm trying an alternative, annotating self in Mixin:
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .this import Mixin, Target
Mixin_T = type('Mixin_T', (Mixin, Target), {})
class Mixin:
def __init__(self: Mixin_T):
self.foo() # No longer an error
class Combined(Mixin, Target):
def __init__(self):
Target.__init__(self)
Mixin.__init__(self) # Now this is an error: "Type[Mixin]" is not
# assignable to parameter "self"
# "Mixin" is incompatible with "Mixin_T"
So how am I supposed to win this aside from using # type: ignore?
I found a very simple solution:
if TYPE_CHECKING:
from .this import Target
Mixin_T = Target
else:
Mixin_T = object
class Mixin(Mixin_T):
...
Now all of Target's methods are recognized within Mixin by the type checker, and there's no need to override the type of self into something imcompatible with Mixin. This might be a little awkward if the mixin is destined to all kinds of Target classes, but for my uses this is perfectly acceptable, since my case is a group of mixins extending a very specific target class.
Other than that, there is to little code and some msconceptions above that make this question not answrable at all, apart from providing some clarifications.
To start, are you sure you are "inheriting from a metaclass"?? It does not make sense to inherit a metaclass unless to create another metaclass. Your snippets show you inhriting froma supposed metaclass (with no code given), to create Target and them attempting to use Target as a parent to a normal class (a non-meta class). That makes no sense.
You might just have confused the terms and the hidden InheritFromMetaclass class actually just uses the metaclass, and do not "inherit" from it. Then your problem does not have to do with metaclasses at all.
So, the real visible problem in the snippet is that the static checkr does not "see" a self.foo method in the Mixin class - and guess what? There is no self.foo method in Mixin - the checker is just throwing a cold truth in your face: while Python does allow one to reference methods and attributes that are not available in a class, knowing that it will be used along other classes that do have those attributes, that is no good design and error prone. The kind of bad design static type checking exists to weed-off.
So, what you need is to have a base of Mixin that is an abstract class and have Foo as an abstract method. (Or have Mixin itself be that abstract class).
If - due to usage of other metaclass you can't have Mixin inheit from abc.ABC due to metaclass conflict, you have to either: create a combined metaclass from the metaclass acutually used by InheritsFromMetaclass with ABCMeta , nd use that as the metaclass for Mixin - or just create a stub foo method in Mixin as is (which could raise a NotImplementedError - thus having the same behavior of an abstract method, but without really having to inherit from it.
The important part to have in and is that an methods and attributes you access in code inside a class body have to exist in that class, without depending on attributes that will exist in a subclass of it.
If that does not solve your problem, you need to provide more data - including a reproducible complete example involving your actual metaclass. (and it mgt be solved just by combining the metaclasses as mentioned above)
I'm using the abc module to define an interface that subclasses must support. There're also some common methods that are present in all subclasses. Is it ok to put them in the abstract class or should that only contain abstract methods (i.e. decorated with #abc.abstractmethod) ?
TL; DR; Yes, it is OK for an abstract class to have non-abstract methods.
Typically what we call an abstract class is just a class that cannot be instantiated.
On the other hand what we call an interface is a class which has only method declarations but no implementations. In particular its an abstract class because it doesn't have a constructor.
Of course in Python there are no real interfaces: every method has to have a body. But we can somewhat emulate interfaces via raise NotImplementedError().
Anyway interfaces form a subset of abstract classes. This obviously suggests that there are abstract classes that are not interfaces. This is exactly the case you are describing. Yes, abstract class may contain implemented, non-abstract methods. And it is not a bad practice. This is especially useful when a given method does not depend on concrete implementation.
For example consider an interface for a generic parser (I'm thinking about json.load and json.loads):
class ILoader(ABC):
#abstractmethod
def load(self, stream):
raise NotImplementedError()
It's completely OK to give loads method which accepts a string instead of stream with a default implementation:
class AbstractLoader(ABC):
#abstractmethod
def load(self, stream):
raise NotImplementedError()
def loads(self, text):
stream = io.StringIO(text)
return self.load(stream)
although I would use Abstract prefix instead of I. ;)
Especially in unittests we use this "design pattern" I call "get class from class level"
framworktest.py:
class FrameWorkHttpClient(object):
....
class FrameWorkTestCase(unittest.TestCase):
# Subclass can control the class which gets used in get_response()
HttpClient=FrameWorkHttpClient
def get_response(self, url):
client=self.HttpClient()
return client.get(url)
mytest.py:
class MyHttpClient(FrameWorkHttpClient):
....
class MyTestCase(FrameWorkTestCase):
HttpClient=MyHttpClient
def test_something(self):
response=self.get_response()
...
The method get_response() gets the class from self not by importing it. This way a subclass can modify the class and use a different HttpClient.
What's the name of this (get class from class level) "design pattern"?
Is this a way of "inversion of control" or "dependency injection"?
Your code is very similar to Factory method pattern. The only difference is that your variant uses factory class variable instead of factory method.
I believe this has the same purpose as just simple polymorphism implemented using Python-specific syntax. Instead of having a virtual method returning a new instances, you have the instance type stored as "an overridable variable" in a class/subclass.
This can be rewritten as a virtual method (sorry I am not fluent in Python so this is just pseudocode)
virtual HttpClient GetClient()
return new FrameworkHttpClient()
then in the subclass, you change the implementation of the method to return a different type:
override HttpClient GetClient()
return new MyHttpClient()
If you want to call this a pattern, I would say it is similar to Strategy GoF pattern. In your particular case, the algorithm being abstracted away is the creation of the particular HttpClient implementation.
And after second thought - as you stated, indeed this can be looked at as an IoC example.
I'm not exactly a design pattern 'Guru', but to me it looks a bit like the Template Method Pattern. You are defining the 'skeleton' of the get_response method in your base class, and leaving one step (defining which class to use) to the subclasses.
If this can be considered the template pattern, it is an example of inversion of control.
You want to let the sub classes decide which class to instantiate.
This is what the factory method pattern already offers:
Define an interface for creating an object, but let the classes that implement the interface decide which class to instantiate. The Factory method lets a class defer instantiation to subclasses. (GoF)
Your solving the same problem by replacing a variable of the parent class.
It works but your solution has at least two drawbacks (compared to the classic pattern):
you introduce a temporal coupling (design smell). Client must call the instructions in the right order. (first initialize the HttpClient then invoke get_response)
your test case is not immutable. Immutable class are simplest than the mutable ones. And in my opinion test should always be simple.
In the following class:
class SomeClass(object):
def __init__(self, somedependency):
self._somedependency = somedependency
def do_something(self):
self._field = _somedependency.get_something()
Is it possible to "know" that somedependency has a function called get_something() at runtime just as you would know the public methods and properties of a constructor parameter in C# at compile time?
EDIT: I'd like to get the somedependency's members and create a new type with those members, so what I need to perform an action such as "I'm calling methods and properties of this constructor parameter in the calling class. Give me all those methods and parameters as these are seen in the calling class". As I don't know the type of the constructor parameter I cannot determine this in a way you can do in static languages.
If you just want to verify that a known method exists, use func = getattr(_somedependency, 'get_something'); if func: func().
If you want more inspection like what methods a class or instance has, check out the built in inspect module.
What is the difference between abstract class and interface in Python?
What you'll see sometimes is the following:
class Abstract1:
"""Some description that tells you it's abstract,
often listing the methods you're expected to supply."""
def aMethod(self):
raise NotImplementedError("Should have implemented this")
Because Python doesn't have (and doesn't need) a formal Interface contract, the Java-style distinction between abstraction and interface doesn't exist. If someone goes through the effort to define a formal interface, it will also be an abstract class. The only differences would be in the stated intent in the docstring.
And the difference between abstract and interface is a hairsplitting thing when you have duck typing.
Java uses interfaces because it doesn't have multiple inheritance.
Because Python has multiple inheritance, you may also see something like this
class SomeAbstraction:
pass # lots of stuff - but missing something
class Mixin1:
def something(self):
pass # one implementation
class Mixin2:
def something(self):
pass # another
class Concrete1(SomeAbstraction, Mixin1):
pass
class Concrete2(SomeAbstraction, Mixin2):
pass
This uses a kind of abstract superclass with mixins to create concrete subclasses that are disjoint.
What is the difference between abstract class and interface in Python?
An interface, for an object, is a set of methods and attributes on that object.
In Python, we can use an abstract base class to define and enforce an interface.
Using an Abstract Base Class
For example, say we want to use one of the abstract base classes from the collections module:
import collections
class MySet(collections.Set):
pass
If we try to use it, we get an TypeError because the class we created does not support the expected behavior of sets:
>>> MySet()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class MySet with abstract methods
__contains__, __iter__, __len__
So we are required to implement at least __contains__, __iter__, and __len__. Let's use this implementation example from the documentation:
class ListBasedSet(collections.Set):
"""Alternate set implementation favoring space over speed
and not requiring the set elements to be hashable.
"""
def __init__(self, iterable):
self.elements = lst = []
for value in iterable:
if value not in lst:
lst.append(value)
def __iter__(self):
return iter(self.elements)
def __contains__(self, value):
return value in self.elements
def __len__(self):
return len(self.elements)
s1 = ListBasedSet('abcdef')
s2 = ListBasedSet('defghi')
overlap = s1 & s2
Implementation: Creating an Abstract Base Class
We can create our own Abstract Base Class by setting the metaclass to abc.ABCMeta and using the abc.abstractmethod decorator on relevant methods. The metaclass will be add the decorated functions to the __abstractmethods__ attribute, preventing instantiation until those are defined.
import abc
For example, "effable" is defined as something that can be expressed in words. Say we wanted to define an abstract base class that is effable, in Python 2:
class Effable(object):
__metaclass__ = abc.ABCMeta
#abc.abstractmethod
def __str__(self):
raise NotImplementedError('users must define __str__ to use this base class')
Or in Python 3, with the slight change in metaclass declaration:
class Effable(object, metaclass=abc.ABCMeta):
#abc.abstractmethod
def __str__(self):
raise NotImplementedError('users must define __str__ to use this base class')
Now if we try to create an effable object without implementing the interface:
class MyEffable(Effable):
pass
and attempt to instantiate it:
>>> MyEffable()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class MyEffable with abstract methods __str__
We are told that we haven't finished the job.
Now if we comply by providing the expected interface:
class MyEffable(Effable):
def __str__(self):
return 'expressable!'
we are then able to use the concrete version of the class derived from the abstract one:
>>> me = MyEffable()
>>> print(me)
expressable!
There are other things we could do with this, like register virtual subclasses that already implement these interfaces, but I think that is beyond the scope of this question. The other methods demonstrated here would have to adapt this method using the abc module to do so, however.
Conclusion
We have demonstrated that the creation of an Abstract Base Class defines interfaces for custom objects in Python.
Python >= 2.6 has Abstract Base Classes.
Abstract Base Classes (abbreviated
ABCs) complement duck-typing by
providing a way to define interfaces
when other techniques like hasattr()
would be clumsy. Python comes with
many builtin ABCs for data structures
(in the collections module), numbers
(in the numbers module), and streams
(in the io module). You can create
your own ABC with the abc module.
There is also the Zope Interface module, which is used by projects outside of zope, like twisted. I'm not really familiar with it, but there's a wiki page here that might help.
In general, you don't need the concept of abstract classes, or interfaces in python (edited - see S.Lott's answer for details).
In a more basic way to explain:
An interface is sort of like an empty muffin pan.
It's a class file with a set of method definitions that have no code.
An abstract class is the same thing, but not all functions need to be empty. Some can have code. It's not strictly empty.
Why differentiate:
There's not much practical difference in Python, but on the planning level for a large project, it could be more common to talk about interfaces, since there's no code. Especially if you're working with Java programmers who are accustomed to the term.
Python doesn't really have either concept.
It uses duck typing, which removed the need for interfaces (at least for the computer :-))
Python <= 2.5:
Base classes obviously exist, but there is no explicit way to mark a method as 'pure virtual', so the class isn't really abstract.
Python >= 2.6:
Abstract base classes do exist (http://docs.python.org/library/abc.html). And allow you to specify methods that must be implemented in subclasses. I don't much like the syntax, but the feature is there. Most of the time it's probably better to use duck typing from the 'using' client side.
In general, interfaces are used only in languages that use the single-inheritance class model. In these single-inheritance languages, interfaces are typically used if any class could use a particular method or set of methods. Also in these single-inheritance languages, abstract classes are used to either have defined class variables in addition to none or more methods, or to exploit the single-inheritance model to limit the range of classes that could use a set of methods.
Languages that support the multiple-inheritance model tend to use only classes or abstract base classes and not interfaces. Since Python supports multiple inheritance, it does not use interfaces and you would want to use base classes or abstract base classes.
http://docs.python.org/library/abc.html
Abstract classes are classes that contain one or more abstract methods. Along with abstract methods, Abstract classes can have static, class and instance methods.
But in case of interface, it will only have abstract methods not other. Hence it is not compulsory to inherit abstract class but it is compulsory to inherit interface.
For completeness, we should mention PEP3119
where ABC was introduced and compared with interfaces,
and original Talin's comment.
The abstract class is not perfect interface:
belongs to the inheritance hierarchy
is mutable
But if you consider writing it your own way:
def some_function(self):
raise NotImplementedError()
interface = type(
'your_interface', (object,),
{'extra_func': some_function,
'__slots__': ['extra_func', ...]
...
'__instancecheck__': your_instance_checker,
'__subclasscheck__': your_subclass_checker
...
}
)
ok, rather as a class
or as a metaclass
and fighting with python to achieve the immutable object
and doing refactoring
...
you'll quite fast realize that you're inventing the wheel
to eventually achieve
abc.ABCMeta
abc.ABCMeta was proposed as a useful addition of the missing interface functionality,
and that's fair enough in a language like python.
Certainly, it was able to be enhanced better whilst writing version 3, and adding new syntax and immutable interface concept ...
Conclusion:
The abc.ABCMeta IS "pythonic" interface in python