I was hoping to make a list of all subclasses of a given class by having each subclass register itself in a list that a parent class holds, ie something like this:
class Monster(object):
monsters = list()
class Lochness(Monster):
Monster.monsters.append(Lochness)
class Yeti(Monster):
Monster.monsters.append(Yeti)
This obviously doesn't work because the classes haven't been created yet when I want to add them to the list. And, it'd be much nicer if it were done automatically (like __subclass__)
I'm aware that __subclass__ has this functionality, but I was wondering (for my own edification) how you'd implement it yourself.
It seems like you'd want to create some sort of subclass of the metaclass which is creating everything to register it with Monster? Or is that completely off base
Classes already register what subclasses are defined; call the class.__subclasses__() method to get a list:
>>> class Monster(object):
... pass
...
>>> class Lochness(Monster):
... pass
...
>>> class Yeti(Monster):
... pass
...
>>> Monster.__subclasses__()
[<class '__main__.Lochness'>, <class '__main__.Yeti'>]
.__subclasses__() returns a list of currently still alive subclasses. If you ever would clear all references to Yeti (del Yeti in the module, delete all instances, subclasses, imports, etc.) then it'd no longer be listed when you call .__subclasses__(). Note that in essence, .__subclasses__() is a CPython implementation detail, but the method is present in all Python versions that support new-style classes (2.2 and up, all the way to 3.x).
Otherwise, the canonical way to hook into class creation is to define a metaclass:
class MonstersMeta(type):
def __new__(metaclass, name, bases, namespace):
cls = super(MonstersMeta, metaclass).__new__(metaclass, name, bases, namespace)
if issubclass(cls, Monster) and not cls is Monster:
Monster.monsters.append(cls)
return cls
class Monster(object):
__metaclass__ = MonstersMeta
monsters = []
class Lochness(Monster):
pass
class Yeti(Monster):
pass
Demo:
>>> class Monster(object):
... __metaclass__ = MonstersMeta
... monsters = []
...
>>> class Lochness(Monster):
... pass
...
>>> class Yeti(Monster):
... pass
...
>>> Monster.monsters
[<class '__main__.Lochness'>, <class '__main__.Yeti'>]
or you can use a class decorator:
def registered_monster(cls):
Monster.monsters.append(cls)
return cls
class Monster(object):
monsters = []
#registered_monster
class Lochness(Monster):
pass
#registered_monster
class Yeti(Monster):
pass
Demo:
>>> class Monster(object):
... monsters = []
...
>>> #registered_monster
... class Lochness(Monster):
... pass
...
>>> #registered_monster
... class Yeti(Monster):
... pass
...
>>> Monster.monsters
[<class '__main__.Lochness'>, <class '__main__.Yeti'>]
The difference being where you put the responsibility of registering monsters; with the base MonstersMeta type, or with explicit decorators.
Either way, the metaclass or the class decorator registers a permanent reference. You can use the weakref module if you really, really want to emulate the .__subclasses__() behaviour.
Except from the obvious solution for this case to use type.__subclasses__(), you could use a decorator for similar problems:
class Monster(object):
monsters = list()
def isMonster(cls):
Monster.monsters.append(cls)
return cls
#isMonster
class Lochness(Monster):
pass
#isMonster
class Yeti(Monster):
pass
print(Monster.monsters) # [<class '__main__.Lochness'>, <class '__main__.Yeti'>]
Just to keep your code, add classes outside their definition:
class Monster(object):
monsters = list()
class Lochness(Monster):
pass
Monster.monsters.append(Lochness)
class Yeti(Monster):
pass
Monster.monsters.append(Yeti)
But, as said: if this is a common feature, create a Metaclass
Related
Take a common vehicle inheritance example where Vehicle and Car are ABCs and the latter inherits from the former. We then have e.g. FordMustang which is not abstract and should inherit from Car.
So, we have
class Vehicle(ABC):
#abstractmethod
def abstract_vehicle_method(self):
pass
Which is presumably all fine and dandy. However, for the Car class, I am not sure whether it should inherit only from Vehicle or from Vehicle and ABC, and if so, in what order? Moreover, I would like to force abstract_vehicle_method() to be defined in FordMustang and all such non-abstract classes, so should I repeat the abstract_vehicle_method definition in Car or will the inheritance sort this out?
To enumerate the options:
Should it be
class Car(Vehicle):
[...]
or
class Car(ABC, Vehicle):
[...]
or
class Car(Vehicle, ABC):
[...]
and (assuming the first is correct for simplicity) should it be
class Vehicle(ABC):
#abstractmethod
def abstract_vehicle_method(self):
pass
class Car(Vehicle):
#abstractmethod
def abstract_vehicle_method(self):
pass
# Some car specific defs here
class FordMustang(Car):
def abstract_vehicle_method(self):
# Concrete def here
or just
class Vehicle(ABC):
#abstractmethod
def abstract_vehicle_method(self):
pass
class Car(Vehicle):
# Some car specific defs here
class FordMustang(Car):
def abstract_vehicle_method(self):
# Concrete def here
You can directly inherit from Vehicle; the metaclass that makes Vehicle 'abstract' is inherited along with it. You do not need to mix in ABC for those subclasses.
You also don't need to re-define the method. It is inherited too, along with it's 'abstractness'.
Under the hood, the metaclass tracks what attributes are 'abstract'; as long as there are any, the class can't be used to create instances. You can add more abstract methods, or by providing a different attribute for the same name, remove abstract methods from the set.
Demo for your specific example:
>>> from abc import ABC, abstractmethod
>>> class Vehicle(ABC):
... #abstractmethod
... def abstract_vehicle_method(self):
... pass
...
>>> type(Vehicle) # it's an abstract class
<class 'abc.ABCMeta'>
>>> Vehicle.__abstractmethods__ # this set determines what is still abstract
frozenset({'abstract_vehicle_method'})
>>> class Car(Vehicle):
... pass
...
>>> type(Car) # still an abstract class
<class 'abc.ABCMeta'>
>>> Car.__abstractmethods__ # still has abstract methods, incl. inherited methods
frozenset({'abstract_vehicle_method'})
>>> class FordMustang(Car):
... def abstract_vehicle_method(self):
... pass
...
>>> type(FordMustang) # still an abstract class
<class 'abc.ABCMeta'>
>>> FordMustang.__abstractmethods__ # but with no abstract methods left
frozenset()
>>> FordMustang() # so we can create an instance
<__main__.FordMustang object at 0x106054f98>
This is the setup I want:
A should be an abstract base class with a static & abstract method f(). B should inherit from A. Requirements:
1. You should not be able to instantiate A
2. You should not be able to instantiate B, unless it implements a static f()
Taking inspiration from this question, I've tried a couple of approaches. With these definitions:
class abstractstatic(staticmethod):
__slots__ = ()
def __init__(self, function):
super(abstractstatic, self).__init__(function)
function.__isabstractmethod__ = True
__isabstractmethod__ = True
class A:
__metaclass__ = abc.ABCMeta
#abstractstatic
def f():
pass
class B(A):
def f(self):
print 'f'
class A2:
__metaclass__ = abc.ABCMeta
#staticmethod
#abc.abstractmethod
def f():
pass
class B2(A2):
def f(self):
print 'f'
Here A2 and B2 are defined using usual Python conventions and A & B are defined using the way suggested in this answer. Following are some operations I tried and the results that were undesired.
With classes A/B:
>>> B().f()
f
#This should have thrown, since B doesn't implement a static f()
With classes A2/B2:
>>> A2()
<__main__.A2 object at 0x105beea90>
#This should have thrown since A2 should be an uninstantiable abstract class
>>> B2().f()
f
#This should have thrown, since B2 doesn't implement a static f()
Since neither of these approaches give me the output I want, how do I achieve what I want?
You can't do what you want with just ABCMeta. ABC enforcement doesn't do any type checking, only the presence of an attribute with the correct name is enforced.
Take for example:
>>> from abc import ABCMeta, abstractmethod, abstractproperty
>>> class Abstract(object):
... __metaclass__ = ABCMeta
... #abstractmethod
... def foo(self): pass
... #abstractproperty
... def bar(self): pass
...
>>> class Concrete(Abstract):
... foo = 'bar'
... bar = 'baz'
...
>>> Concrete()
<__main__.Concrete object at 0x104b4df90>
I was able to construct Concrete() even though both foo and bar are simple attributes.
The ABCMeta metaclass only tracks how many objects are left with the __isabstractmethod__ attribute being true; when creating a class from the metaclass (ABCMeta.__new__ is called) the cls.__abstractmethods__ attribute is then set to a frozenset object with all the names that are still abstract.
type.__new__ then tests for that frozenset and throws a TypeError if you try to create an instance.
You'd have to produce your own __new__ method here; subclass ABCMeta and add type checking in a new __new__ method. That method should look for __abstractmethods__ sets on the base classes, find the corresponding objects with the __isabstractmethod__ attribute in the MRO, then does typechecking on the current class attributes.
This'd mean that you'd throw the exception when defining the class, not an instance, however. For that to work you'd add a __call__ method to your ABCMeta subclass and have that throw the exception based on information gathered by your own __new__ method about what types were wrong; a similar two-stage process as what ABCMeta and type.__new__ do at the moment. Alternatively, update the __abstractmethods__ set on the class to add any names that were implemented but with the wrong type and leave it to type.__new__ to throw the exception.
The following implementation takes that last tack; add names back to __abstractmethods__ if the implemented type doesn't match (using a mapping):
from types import FunctionType
class ABCMetaTypeCheck(ABCMeta):
_typemap = { # map abstract type to expected implementation type
abstractproperty: property,
abstractstatic: staticmethod,
# abstractmethods return function objects
FunctionType: FunctionType,
}
def __new__(mcls, name, bases, namespace):
cls = super(ABCMetaTypeCheck, mcls).__new__(mcls, name, bases, namespace)
wrong_type = set()
seen = set()
abstractmethods = cls.__abstractmethods__
for base in bases:
for name in getattr(base, "__abstractmethods__", set()):
if name in seen or name in abstractmethods:
continue # still abstract or later overridden
value = base.__dict__.get(name) # bypass descriptors
if getattr(value, "__isabstractmethod__", False):
seen.add(name)
expected = mcls._typemap[type(value)]
if not isinstance(namespace[name], expected):
wrong_type.add(name)
if wrong_type:
cls.__abstractmethods__ = abstractmethods | frozenset(wrong_type)
return cls
With this metaclass you get your expected output:
>>> class Abstract(object):
... __metaclass__ = ABCMetaTypeCheck
... #abstractmethod
... def foo(self): pass
... #abstractproperty
... def bar(self): pass
... #abstractstatic
... def baz(): pass
...
>>> class ConcreteWrong(Abstract):
... foo = 'bar'
... bar = 'baz'
... baz = 'spam'
...
>>> ConcreteWrong()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class ConcreteWrong with abstract methods bar, baz, foo
>>>
>>> class ConcreteCorrect(Abstract):
... def foo(self): return 'bar'
... #property
... def bar(self): return 'baz'
... #staticmethod
... def baz(): return 'spam'
...
>>> ConcreteCorrect()
<__main__.ConcreteCorrect object at 0x104ce1d10>
I need to make a mixin that knows the name of the class who is using it. Sort of like:
class FooMixin(...):
bar = self.__class__
Except that self is not defined at attribute definition time. Is there a clean way to achieve this so it's transparent for the class who inherits the mixin?
At the time of definition of your mixin, nobody knows in which classes your mixin is used. You can only get the name dynamically in class methods by using self.__class__.__name__:
class FooMixin(object):
def some_method(self):
print "I'm in class %s" % self.__class__.__name__
class Main(FooMixin):
pass
instance = Main()
instance.some_method() # "I'm in class Main"
Daniel's answer gives the reason why this is not possible in the declarative way you seem to like it - nobody knows at Mixin's definition-time where and when it will be used.
However, if you don't care about the time, but want the syntax, meaning you want to access bar defined as property in Mixin, and return self.class, this should work:
class classproperty(object):
def __get__(self, instance, clazz):
return clazz
class Mixin(object):
bar = classproperty()
class Foo(Mixin):
pass
print Foo().bar
First off, no special action is needed to know the name of a class:
class MyMixin(object):
def frob(self):
print "frobbing a", self.__class__.__name__
class Foo(MyMixin): pass
class Bar(MyMixin): pass
>>> Foo().frob()
frobbing a Foo
>>> Bar().frob()
frobbing a Bar
similarly, no special action is needed to discover subclasses:
>>> MyMixin.__subclasses__()
[__main__.Foo, __main__.Bar]
If these aren't what you need, because you want to take action when your base class is subclassed, you need a metaclass!:
class MyMixinMeta(type):
def __init__(cls, name, bases, attrs):
if bases != (object,):
print name, cls, "is a subclass of", bases
class MyMixin(object):
__metaclass__ = MyMixinMeta
def frob(self):
print "frobbing a", self.__class__.__name__
>>> class Foo(MyMixin): pass
Foo <class '__main__.Foo'> is a subclass of (<class '__main__.MyMixin'>,)
>>> class Bar(MyMixin): pass
Bar <class '__main__.Bar'> is a subclass of (<class '__main__.MyMixin'>,)
I am looking for ways / best practices on testing methods defined in an abstract base class. One thing I can think of directly is performing the test on all concrete subclasses of the base class, but that seems excessive at some times.
Consider this example:
import abc
class Abstract(object):
__metaclass__ = abc.ABCMeta
#abc.abstractproperty
def id(self):
return
#abc.abstractmethod
def foo(self):
print "foo"
def bar(self):
print "bar"
Is it possible to test bar without doing any subclassing?
In newer versions of Python you can use unittest.mock.patch()
class MyAbcClassTest(unittest.TestCase):
#patch.multiple(MyAbcClass, __abstractmethods__=set())
def test(self):
self.instance = MyAbcClass() # Ha!
Here is what I have found: If you set __abstractmethods__ attribute to be an empty set you'll be able to instantiate abstract class. This behaviour is specified in PEP 3119:
If the resulting __abstractmethods__ set is non-empty, the class is considered abstract, and attempts to instantiate it will raise TypeError.
So you just need to clear this attribute for the duration of tests.
>>> import abc
>>> class A(metaclass = abc.ABCMeta):
... #abc.abstractmethod
... def foo(self): pass
You cant instantiate A:
>>> A()
Traceback (most recent call last):
TypeError: Can't instantiate abstract class A with abstract methods foo
If you override __abstractmethods__ you can:
>>> A.__abstractmethods__=set()
>>> A() #doctest: +ELLIPSIS
<....A object at 0x...>
It works both ways:
>>> class B(object): pass
>>> B() #doctest: +ELLIPSIS
<....B object at 0x...>
>>> B.__abstractmethods__={"foo"}
>>> B()
Traceback (most recent call last):
TypeError: Can't instantiate abstract class B with abstract methods foo
You can also use unittest.mock (from 3.3) to override temporarily ABC behaviour.
>>> class A(metaclass = abc.ABCMeta):
... #abc.abstractmethod
... def foo(self): pass
>>> from unittest.mock import patch
>>> p = patch.multiple(A, __abstractmethods__=set())
>>> p.start()
{}
>>> A() #doctest: +ELLIPSIS
<....A object at 0x...>
>>> p.stop()
>>> A()
Traceback (most recent call last):
TypeError: Can't instantiate abstract class A with abstract methods foo
As properly put by lunaryon, it is not possible. The very purpose of ABCs containing abstract methods is that they are not instantiatable as declared.
However, it is possible to create a utility function that introspects an ABC, and creates a dummy, non abstract class on the fly. This function could be called directly inside your test method/function and spare you of having to wite boiler plate code on the test file just for testing a few methods.
def concreter(abclass):
"""
>>> import abc
>>> class Abstract(metaclass=abc.ABCMeta):
... #abc.abstractmethod
... def bar(self):
... return None
>>> c = concreter(Abstract)
>>> c.__name__
'dummy_concrete_Abstract'
>>> c().bar() # doctest: +ELLIPSIS
(<abc_utils.Abstract object at 0x...>, (), {})
"""
if not "__abstractmethods__" in abclass.__dict__:
return abclass
new_dict = abclass.__dict__.copy()
for abstractmethod in abclass.__abstractmethods__:
#replace each abc method or property with an identity function:
new_dict[abstractmethod] = lambda x, *args, **kw: (x, args, kw)
#creates a new class, with the overriden ABCs:
return type("dummy_concrete_%s" % abclass.__name__, (abclass,), new_dict)
You can use multiple inheritance practice to have access to the implemented methods of the abstract class. Obviously following such design decision depends on the structure of the abstract class since you need to implement abstract methods (at least bring the signature) in your test case.
Here is the example for your case:
class Abstract(object):
__metaclass__ = abc.ABCMeta
#abc.abstractproperty
def id(self):
return
#abc.abstractmethod
def foo(self):
print("foo")
def bar(self):
print("bar")
class AbstractTest(unittest.TestCase, Abstract):
def foo(self):
pass
def test_bar(self):
self.bar()
self.assertTrue(1==1)
No, it's not. The very purpose of abc is to create classes that cannot be instantiated unless all abstract attributes are overridden with concrete implementations. Hence you need to derive from the abstract base class and override all abstract methods and properties.
Perhaps a more compact version of the concreter proposed by #jsbueno could be:
def concreter(abclass):
class concreteCls(abclass):
pass
concreteCls.__abstractmethods__ = frozenset()
return type('DummyConcrete' + abclass.__name__, (concreteCls,), {})
The resulting class still has all original abstract methods (which can be now called, even if this is not likely to be useful...) and can be mocked as needed.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Can I get a reference to the 'owner' class during the init method of a descriptor?
Code is worth a thousand words:
>>> class ShortRib(object):
>>> def __init__(self, owner):
>>> self.owner = owner
>>>
>>> ... some more methods and stuff ...
>>>
>>>
>>> class Cow(object):
>>> shortRib = ShortRib(self)
>>>
>>>
>>> class BrownCow(Cow):
>>> pass
>>>
>>> BrownCow.shortRib.owner
<class '__main__.BrownCow'>
This doesn't work, though i wish it would. Basically, I want each class to have some static/class variables (i'm not sure which it is in this case?) but need each of those guys to know who (which class) it belongs to. Unfortunately, I can't "get" at the class in the body of the class declaration. Of course, I could always do this using a decorator:
>>> def vars(**kwargs):
>>> def wrap(cls):
>>> for k, w in kwargs.items():
>>> setattr(cls, k, w(cls))
>>> return cls
>>> return wrap
>>>
>>> #vars(shortRib=lambda cls: ShortRib(cls)
>>> class BrownCow(Cow):
>>> ...
>>>
>>> BrownCow.shortRib.owner
which would work. Another way would to have a class decorator that goes through all the shortRibs and similar static variables and sets their owner after the class declaration is complete. However, this seems like an incredibly roundabout and unintuitive way of doing what should be a pretty simple operation: having the static/class members of a class know who they belong to.
Is there a "proper" way of doing this?
Clarification:
I want these members to belong to the class, not to the instances. I'm trying to go for a almost-purely-functional style, using classes only for inheritance of shared behavior, and not creating instances of them at all. Instances would tend to give my functions access to arbitrary instance data shared across all functions, which would break the pure-functioness I am trying for. I could just use empty instances which I don't touch, but I think using pure classes would be cleaner.
You can easily do this in __new__:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class Cow(object):
shortRib = None
def __new__(cls, *args, **kwargs):
if cls.shortRib == None:
cls.shortRib = ShortRib(cls)
return super(Cow, cls).__new__(cls, *args, **kwargs)
Cow()
Cow.shortRib.owner
Or even __init__, if you don't mind referencing self.__class___.
You can also do it with a metaclass:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class MetaCow(type):
def __new__(cls, name, base, attrs):
attrs['shortRib'] = ShortRib(cls)
return super(MetaCow, cls).__new__(cls, name, base, attrs)
class Cow(object):
__metaclass__ = MetaCow
Cow.shortRib.owner
Why not let the instances of the Cow class have shortRibs, instead of the class itself?:
class ShortRib(object):
def __init__(self,owner):
self.owner=owner
class Cow(object):
def __init__(self):
self.shortRib=ShortRib(self)
class BrownCow(Cow):
pass
print(BrownCow().shortRib.owner)
# <__main__.BrownCow object at 0xb76a8d6c>
(Otherwise, you'll need a class decorator or metaclass -- as you've already mentioned. But simple is better than complex, so why not choose simple?)
By the way, if you really do want to use classes instead of instances:
class ShortRib(object):
def __init__(self, owner):
self.owner = owner
class MetaCow(type):
def __init__(cls, name, base, attrs):
super(MetaCow, cls).__init__(name, base, attrs)
cls.shortRib = ShortRib(cls)
class Cow(object):
__metaclass__ = MetaCow
class BrownCow(Cow):
pass
print(Cow.shortRib.owner)
# <class '__main__.Cow'>
print(BrownCow.shortRib.owner)
# <class '__main__.BrownCow'>
Using
class MetaCow(type):
def __new__(cls, name, base, attrs):
is incorrect. The signature for type.__new__ is
class MetaCow(type):
def __new__(meta, name, base, attrs):
Since you want to modify the attributes of cls, not meta, use the MetaCow.__init__ not MetaCow__new__.
Two methods to to do what you want:
You can override the __getattr__ method in any class to return anything you desire when you ask for the value of an attribute.
You can use a property, which has a getter that returns the object you want it to return.
Both __getattr__ methods and properties are inherited.