I am using Django 1.3.1 and I have the following piece of models:
class masterData(models.Model):
uid = models.CharField(max_length=20,primary_key=True)
class Meta:
abstract = True;
class Type1(masterData):
pass;
class Type2(masterData):
pass;
Now, I am trying to get a list of all child classes of masterData. I
have tried:
masterData.__subclasses__()
The very interesting thing that I found about the above is that it
works flawlessly in python manage.py shell and does not work at all
when running the webserver!
So how do I get a list of models derived from an Abstract Base Class model?
Thanks :)
Metaclass for defining Abstract Base Classes (ABCs).
Use this metaclass to create an ABC. An ABC can be subclassed directly, and then acts as a mix-in
class. You can also register unrelated concrete classes (even built-in classes) and unrelated ABCs
as 'virtual subclasses' -- these and their descendants will be considered subclasses of the
registering ABC by the built-in issubclass() function, but the registering ABC won't show up in
their MRO (Method Resolution Order) nor will method implementations defined by the registering
ABC be callable (not even via super()).
I haven't used ABCMeta much (have a bit today actually..). You need to use the 'issubclass()' function
since the ABC won't show up in they're mro.
If you used inheritance, subclasses() would work.
>>> class foo(object):
... pass
...
>>> class bar(foo):
... pass
...
>>> a = bar()
>>> a.__class__.__mro__
(<class '__main__.bar'>, <class '__main__.foo'>, <type 'object'>)
>>> foo.__subclasses__()
[<class '__main__.bar'>]
Related
I checked the SocketServer.py package.
Find the one function of class BaseServer--- '_handle_request_noblock()'
It has :request, client_address = self.get_request()
But the function get_request() is built from BaseServer's subclass TCPServer and UDPServer.
How can I understand this?
On top of Dynamic Dispatch as referred to by #JeremyFisher, such behavior is also related to the OOP concepts of Inheritance and Polymorphism (or this) which in turn is related to Liskov's Substitution Principle. In Python, this behavior is visible in MRO.
Let's say we have this hierarchy of classes
>>> class BaseServer:
... def handle_request_noblock(self):
... print("BaseServer:handle_request_noblock")
... self.get_request()
...
>>> class TCPServer(BaseServer):
... def get_request(self):
... print("TCPServer:get_request")
...
>>>
>>> class UDPServer(BaseServer):
... def get_request(self):
... print("UDPServer:get_request")
...
>>>
Let's see the attributes of the child class objects
>>> # base_server = BaseServer() # This should not be done because "abstract" base classes are just blueprints, thus they are incomplete as in our example here where it has no definition of <get_request>. A derived "concrete" subclass is needed.
>>>
>>> tcp_server = TCPServer()
>>> udp_server = UDPServer()
>>>
>>> print(dir(tcp_server))
[...<trimmed for better viewing>..., 'get_request', 'handle_request_noblock']
>>> print(dir(udp_server))
[...<trimmed for better viewing>..., 'get_request', 'handle_request_noblock']
>>>
As you can see, both subclasses has the handle_request_noblock attribute even if they don't explicitly define it, they just inherited it from the base class.
an object created through inheritance, a "child object", acquires all the properties and behaviors of the "parent object"
Then, they have the get_request which they implemented on their own.
Let's call now the entrypoint function
>>> tcp_server.handle_request_noblock()
BaseServer:handle_request_noblock
TCPServer:get_request
>>> udp_server.handle_request_noblock()
BaseServer:handle_request_noblock
UDPServer:get_request
>>>
As you can see, BaseServer:handle_request_noblock was able to call e.g. TCPServer:get_request. Why? Remember that the object we used is a TCPServer instance and based on the available methods we displayed on the earlier step, we saw that both handle_request_noblock and get_request are available to it, so we know that both can be called. Same with UDPServer.
In Python, the technicalities of such dispatch from base to derived class is through MRO.
Python supports classes inheriting from other classes. The class being inherited is called the Parent or Superclass, while the class that inherits is called the Child or Subclass. In python, method resolution order defines the order in which the base classes are searched when executing a method. First, the method or attribute is searched within a class and then it follows the order we specified while inheriting.
>>> print(TCPServer.mro())
[<class '__main__.TCPServer'>, <class '__main__.BaseServer'>, <class 'object'>]
>>> print(UDPServer.mro())
[<class '__main__.UDPServer'>, <class '__main__.BaseServer'>, <class 'object'>]
>>>
So for TCPServer
This means that when we called handle_request_noblock, it checked first if the implementation is present in class TCPServer, since it isn't, it checks to the next which is class BaseServer, since it is there, then it uses that implementation, so if you override it within TCPServer, it wouldn't have to call the implementation in the BaseServer anymore.
Then when we called get_request, it checked first if the implementation is present in class TCPServer, since it is there, then it uses that implementation, no more need to check if it is present in BaseServer.
Same idea with UDPServer.
I've created an abstract class property for class Parent using metaclasses:
from abc import abstractmethod, ABCMeta
class ParentMeta(ABCMeta):
#property
#abstractmethod
def CONSTANT(cls):
raise NotImplementedError()
class Parent(metaclass=ParentMeta):
pass
I can set a value for it as follows:
class ChildMeta(ParentMeta):
CONSTANT = 4
class Child(Parent, metaclass=ChildMeta):
pass
print(Child.CONSTANT) // 4
Is it also possible to give it a value without going through an extra metaclass? For example, as follows?
class OtherChild(Parent):
CONSTANT = 5
OtherChild.CONSTANT // NotImplementedError
The declaration of CONSTANT with the abstract method modifier should be on the base class (Parent), not on the metaclass.
You don't have to meddle with metaclasses for this at all, just use abc.ABC as your base class:
In [14]: import abc
In [15]: class Parent(abc.ABC):
...: #property
...: #abc.abstractmethod
...: def CONSTANT(self): pass
...:
In [16]: class Child1(Parent):
...: CONSTANT = 5
...:
In [17]: Child1()
Out[17]: <__main__.Child1 at 0x7fc55246b670>
In [18]: class Child2(Parent):
...: pass
...:
...:
In [19]: Child2()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-19-59958dc9047d> in <module>
----> 1 Child2()
TypeError: Can't instantiate abstract class Child2 with abstract method CONSTANT
As for "abstractproperties" declaring things of the ABC module in the metaclass themselves: that is not the intended us, and if you got anything close to your intent, that was sheer luck.
The idea is that abc.ABCMeta + some mechanisms in the language core provide the mechanism for abstract attributes and methods to be checked in the classes themselves, not in the metaclasses.
An attribute defined in a class is already a class attribute.
On a completly unrelated way (unrelated to abstract classes) property will work as a "class property" if created on the metaclass due to the extreme consistency of the object model in Python: classes in this case behave as instances of the metaclass, and them the property on the metaclass is used. However, setting properties and attributes on a metaclass to be reflected and viewed on the class is something extremely rare in a normal design. Reading your question, it just looks like you need a normal class attribute as above.
If you want something at class level to behave like an actual property (with code to be run when the attribute is accessed, so it is dynamically generated), it is possible by creating a descriptor class, akin to property, that would also work for classes - or, just use property on the metaclass as you have half done. If you just want to check if the attribute is declared in each child class, again, the plain use of abc is what you need.
Otherwise, if you are relying on real properties (not just a way to declare "abstractattribute"), and using the property-on-metaclass mechanism, of course you have to create an intermediary metaclass in order to override it: a property on the class would work for instances, not for the class itself.
There are mechanisms that could be used by actually having some code on the metaclass __new__ method- for example, it would be possible to have a marker decorator that could make a property declared on the class to be "transplanted" to the metaclass on class creation, and function as a class property, and just let the plain use of abc.ABC to handle the abstractmethod part. As it does not seem to be what you need, I won't implement it in full: it'd take sometime to be done correctly.
Suppose you have the following class:
class Base(object):
def abstract_method(self):
raise NotImplementedError
Can you then implement a inheriting class, which does not implement the abstract method? For example, when it does not need that specific method. Will that give problems or is it just bad practice?
If you're implementing abstract methods the way you show, there's nothing enforcing the abstractness of the class as a whole, only of the methods that don't have a concrete definition. So you can create an instance of Base, not only of its subclasses.
b = Base() # this works following your code, only b.abstract_method() raises
def Derived(Base):
... # no concrete implementation of abstract_method, so this class works the same
However, if you use the abc module from the standard library to designate abstract methods, it will not allow you to instantiate an instance of any class that does not have a concrete implementation of any abstract methods it has inherited. You can leave inherited abstract methods unimplemented in an intermediate abstract base class (e.g. a subclass of the original base, that is itself intended to still be abstract), but you can't make any instances.
Here's what using abc looks like:
from abc import ABCMeta, abstractmethod
class ABCBase(metaclass=ABCMeta):
#abstractmethod
def abstract_method(self, arg):
...
class ABCDerived(ABCBase):
... # we don't implement abstract_method here, so we're also an abstract class
d = ABCDerived() # raises an error
I want to ask what the with_metaclass() call means in the definition of a class.
E.g.:
class Foo(with_metaclass(Cls1, Cls2)):
Is it a special case where a class inherits from a metaclass?
Is the new class a metaclass, too?
with_metaclass() is a utility class factory function provided by the six library to make it easier to develop code for both Python 2 and 3.
It uses a little sleight of hand (see below) with a temporary metaclass, to attach a metaclass to a regular class in a way that's cross-compatible with both Python 2 and Python 3.
Quoting from the documentation:
Create a new class with base class base and metaclass metaclass. This is designed to be used in class declarations like this:
from six import with_metaclass
class Meta(type):
pass
class Base(object):
pass
class MyClass(with_metaclass(Meta, Base)):
pass
This is needed because the syntax to attach a metaclass changed between Python 2 and 3:
Python 2:
class MyClass(object):
__metaclass__ = Meta
Python 3:
class MyClass(metaclass=Meta):
pass
The with_metaclass() function makes use of the fact that metaclasses are a) inherited by subclasses, and b) a metaclass can be used to generate new classes and c) when you subclass from a base class with a metaclass, creating the actual subclass object is delegated to the metaclass. It effectively creates a new, temporary base class with a temporary metaclass metaclass that, when used to create the subclass swaps out the temporary base class and metaclass combo with the metaclass of your choice:
def with_metaclass(meta, *bases):
"""Create a base class with a metaclass."""
# This requires a bit of explanation: the basic idea is to make a dummy
# metaclass for one level of class instantiation that replaces itself with
# the actual metaclass.
class metaclass(type):
def __new__(cls, name, this_bases, d):
return meta(name, bases, d)
#classmethod
def __prepare__(cls, name, this_bases):
return meta.__prepare__(name, bases)
return type.__new__(metaclass, 'temporary_class', (), {})
Breaking the above down:
type.__new__(metaclass, 'temporary_class', (), {}) uses the metaclass metaclass to create a new class object named temporary_class that is entirely empty otherwise. type.__new__(metaclass, ...) is used instead of metaclass(...) to avoid using the special metaclass.__new__() implementation that is needed for the slight of hand in a next step to work.
In Python 3 only, when temporary_class is used as a base class, Python first calls metaclass.__prepare__() (passing in the derived class name, (temporary_class,) as the this_bases argument. The intended metaclass meta is then used to call meta.__prepare__(), ignoring this_bases and passing in the bases argument.
next, after using the return value of metaclass.__prepare__() as the base namespace for the class attributes (or just using a plain dictionary when on Python 2), Python calls metaclass.__new__() to create the actual class. This is again passed (temporary_class,) as the this_bases tuple, but the code above ignores this and uses bases instead, calling on meta(name, bases, d) to create the new derived class.
As a result, using with_metaclass() gives you a new class object with no additional base classes:
>>> class FooMeta(type): pass
...
>>> with_metaclass(FooMeta) # returns a temporary_class object
<class '__main__.temporary_class'>
>>> type(with_metaclass(FooMeta)) # which has a custom metaclass
<class '__main__.metaclass'>
>>> class Foo(with_metaclass(FooMeta)): pass
...
>>> Foo.__mro__ # no extra base classes
(<class '__main__.Foo'>, <type 'object'>)
>>> type(Foo) # correct metaclass
<class '__main__.FooMeta'>
UPDATE: the six.with_metaclass() function has since been patched with a decorator variant, i.e. #six.add_metaclass(). This update fixes some mro issues related to the base objects. The new decorator would be applied as follows:
import six
#six.add_metaclass(Meta)
class MyClass(Base):
pass
Here are the patch notes and here is a similar, detailed example and explanation for using a decorator alternative.
I'm not seeing what I expect when I use ABCMeta and abstractmethod.
This works fine in python3:
from abc import ABCMeta, abstractmethod
class Super(metaclass=ABCMeta):
#abstractmethod
def method(self):
pass
a = Super()
TypeError: Can't instantiate abstract class Super ...
And in 2.6:
class Super():
__metaclass__ = ABCMeta
#abstractmethod
def method(self):
pass
a = Super()
TypeError: Can't instantiate abstract class Super ...
They both also work fine (I get the expected exception) if I derive Super from object, in addition to ABCMeta.
They both "fail" (no exception raised) if I derive Super from list.
I want an abstract base class to be a list but abstract, and concrete in sub classes.
Am I doing it wrong, or should I not want this in python?
With Super build as in your working snippets, what you're calling when you do Super() is:
>>> Super.__init__
<slot wrapper '__init__' of 'object' objects>
If Super inherits from list, call it Superlist:
>>> Superlist.__init__
<slot wrapper '__init__' of 'list' objects>
Now, abstract base classes are meant to be usable as mixin classes, to be multiply inherited from (to gain the "Template Method" design pattern features that an ABC may offer) together with a concrete class, without making the resulting descendant abstract. So consider:
>>> class Listsuper(Super, list): pass
...
>>> Listsuper.__init__
<slot wrapper '__init__' of 'list' objects>
See the problem? By the rules of multiple inheritance calling Listsuper() (which is not allowed to fail just because there's a dangling abstract method) runs the same code as calling Superlist() (which you'd like to fail). That code, in practice (list.__init__), does not object to dangling abstract methods -- only object.__init__ does. And fixing that would probably break code that relies on the current behavior.
The suggested workaround is: if you want an abstract base class, all its bases must be abstract. So, instead of having concrete list among your bases, use as a base collections.MutableSequence, add an __init__ that makes a ._list attribute, and implement MutableSequence's abstract methods by direct delegation to self._list. Not perfect, but not all that painful either.
Actually, the issue is with __new__, rather than with __init__. Example:
from abc import ABCMeta, abstractmethod
from collections import OrderedDict
class Foo(metaclass=ABCMeta):
#abstractmethod
def foo(self):
return 42
class Empty:
def __init__(self):
pass
class C1(Empty, Foo): pass
class C2(OrderedDict, Foo): pass
C1() fails with a TypeError as expected, while C2.foo() returns 42.
>>> C1.__init__
<function Empty.__init__ at 0x7fa9a6c01400>
As you can see, it's not using object.__init__ nor is it even invoking its superclass (object) __init__
You can verify it by calling __new__ yourself:
C2.__new__(C2) works just fine, while you'll get the usual TypeError with C1.__new__(C1)
So, imho it's not as clear cut as
if you want an abstract base class, all its bases must be abstract.
While that's a good suggestion, the converse it's not necessarily true: neither OrderedDict nor Empty are abstract, and yet the former's subclass is "concrete", while the latter is "abstract"
If you're wondering, I used OrderedDict in the example instead of list because the latter is a "built-in" type, and thus you cannot do:
OrderedDict.bar = lambda self: 42
And I wanted to make it explicit that the issue is not related to it.