This question is specific for python 3. Suppose I have a class hierarchy like this
class Base():
def calculate():
return 0
class Derived1(Base):
def calculate():
# some calculation
class Derived2(Base):
def calculate():
# some calculation
Now, what I want to do is make a class that defines a generic way to inherit from the Derived classes, and then overrides calculate. In other words, something in the spirit of C++ templates, to avoid copying over the subclasses code, but specify a generic way of subclassing, and then be able to define the subclasses as one liners, like shown below:
# pseudocode
class GenericDerived5(someGenericBase):
def calculate():
return super().calculate() + 5
class GenericDerived6(someGenericBase):
def calculate():
return super().calculate() + 5
class Derived5_1 = GenericDerived5(Derived1)
class Derived6_1 = GenericDerived6(Derived2)
(the calculation is not literally like this, just illustrating the combinatorial nature of the inheritance structure)
How would this code look like, and what are the relevant tools from python3 that I need? I've heard of metaclasses, but not very familiar.
class definition inside a factory-function body
The most straightforward way to go there is really straightforward - but can feel a bit awkward:
def derived_5_factory(Base):
class GenericDerived5(Base):
def calculate(self):
return super().calculate() + 5
return GenericDerived5
def derived_6_factory(Base):
class GenericDerived6(Base):
def calculate(self):
return super().calculate() + 6
return GenericDerived6
Derived5_1 = derived_5_factory(Derived1)
Derived6_2 = derived_6_factory(Derived2)
The inconvenient part is that your classes that need generic bases
have to be defined inside function bodies. That way, Python re-executes
the class statement itself, with a different Base, taking advantage
that in Python classes are first class objects.
This code have the inconveniences that (1) the class bodies must be inside functions, and (2) it can be the wrong approach at all:
Multiple inheritance
If you can have an extra inheritance level - that is the only difference for your example, this is the "correct" way to go. Actually, apart from having the former "GenericDerived" classes explicitly in their inheritance chain, they will behave exactly as intended:
class Base():
def calculate():
return 0
class Derived1(Base):
def calculate(self):
return 1
class Derived2(Base):
def calculate(self):
return 2
# mix-in bases:
class MixinDerived5(Base):
def calculate(self):
return super().calculate() + 5
class MixinDerived6(Base):
def calculate(self):
return super().calculate() + 6
Derived5_1 = type("Derived5_1", (MixinDerived5, Derived1), {})
Derived6_2 = type("Derived6_2", (MixinDerived6, Derived2), {})
Here, instead of using the class statement, a dynamic class is created with the type call, using both the class that needs a dybamic base and that dynamic base as its bases parameter. That is it - Derived5_1 is a fully working Python class with both Bases in its inheritance chain
Note that Python's super() will do exactly what common sense would expect it to do, "rerouting" itself through the extra intermediary "derived" classes before reaching "Base". So, this is what I get on the interactive console after pasting the code above:
In [6]: Derived5_1().calculate()
Out[6]: 6
In [7]: Derived6_2().calculate()
Out[7]: 8
A mix-in class, roughly speaking, is a class that isn't intended to be instantiated directly or act as a standalone base class (other than for other, more specialized mix-in classes), but to provide a small subset of functionality that another class can inherit.
In this case, your GenericDerived classes are perfect examples of mix-ins: you aren't creating instances of GenericDerived, but you can inherit from them to add a calculate method to your own class.
class Calculator:
def calculate(self):
return 9
class Calculator1(Calculator):
def calculate(self):
return super().calculate() + 5
class Calculator2(Calculator):
def calculate(self):
return super().calculate() + 10
class Base(Calculator):
...
Note that the Base and Calculator hierarchies are independent of each other. Base provides, in addition to whatever else it does, basic calculate functionality. A subclass of Base can use calculate that it inherits from Base (via Calculator), or it can inherit from a subclass of Calculator as well.
class Derived1(Base):
...
class Derived2(Base, Calculator1):
...
class Derived3(Base, Calculator2):
...
Related
I'm currently working on redesigning a class to be under an abstract base class. The current class has a method func that does some logic for two things, say A and B.
(note that all the code below is very simplified. There's a lot more functionality than what is shown)
class current_class:
def func(self):
# does stuff for A
# does stuff for B
During logic A, it loads a large dataset into a dictionary, say, dataset and later dataset.keys() is used for logic B, but other than that, A and B are independent of each other.
I will create an alternate class, say, another_class that is similar to current_class, but this class doesn't need B and only needs A. So something like
class another_class:
def func(self):
# does stuff for A
And then both will be under an abstract base class base. Since both inherited classes involves A, I plan on just creating a method in base class that does A, say, func_A. But I'm having trouble with figuring out the best way to approach this so that the function signatures conform and without having to reload dataset for B.
If another_class also needed the logic for B, I think we can just return dataset.keys() from func_A and use it in func_B, but another_class doesn't.
So I don't know if there's a good way to conform this without having different signatures for the methods.
So in code, I have the following two ideas:
1)
class base:
#abstractmethod
def func(self):
pass
def func_A(self):
# does stuff for A and gets the dataset
return dataset.keys()
class current_class:
def func_B(self, keys):
# does stuff for B
def func(self):
keys = self.func_A
self.func_B(keys)
class current_class:
def func(self):
_ = self.func_A() # the return is unused...
class base:
#abstractmethod
def func(self):
pass
class current_class:
def func_A(self):
# does stuff for A and gets the dataset
return dataset.keys()
def func_B(self, keys):
# does stuff for B
def func(self):
keys = self.func_A()
self.func_B(keys)
class current_class:
def func_A(self):
# does same stuff as func_A for current_class, and doesn't return anything
def func(self):
self.func_A()
I don't like the first design because func_A only needs to return something for one of the subclasses and not for all of them. I also don't like the second design because we have to separately implement func_A in each inherited class even though they're identical methods, except one needs to return something and the other doesn't.
It's not a big deal to ignore the return value of a function that is primarily called for its side effects. Just define func_A once in the base class and let both child classes use it as appropriate to their needs.
class Base:
#abstractmethod
def func(self):
pass
def func_A(self):
# does stuff for A and gets the dataset
return dataset.keys()
class Child1:
def func_B(self, keys):
# does stuff for B
def func(self):
keys = self.func_A
self.func_B(keys)
class Child2:
def func(self):
self.func_A()
If there is more in func_A that isn't necessary for Child2, then it should of course be split up to avoid doing unnecessary work in Child2.func. But simply returning a value is not in anyway time- or space-intensive, and should not be a concern.
I have a Python library which will be used by other people:
class BaseClassA:
class BaseClassB:
def func0(self):
this.class_a_obj = BaseClassA()
BaseClassB creates a BaseClassA object and stores a pointer. This is an issue because I want to allow the user to extend my library classes:
class ExtendClassA(BaseClassA):
And my library should choose the extended class (ExtendClassA) instead of the base class (BaseClassA) in the func0 method.
Above is a very simple example my problem statement. In reality I have 10ish classes where extending/creation happens. I want to avoid the user having to rewrite func0 in an extended BaseClassB to support the new ExtendClassA class they created.
I'm reaching out to the stack overflow community to see what solutions other people have implemented for issues like this. My initial thought is to have a global dict which 'registers' class types/constructors and classes would get the class constructors from the global dict. When a user wants to extend a class they would replace the class in the dict with the new class.
Library code:
global lib_class_dict
lib_class_dict['ClassA'] = BaseClassA()
lib_class_dict['ClassB'] = BaseClassB()
class BaseClassA:
class BaseClassB:
def func0(self):
this.class_a_obj = lib_class_dict['ClassB']
User code:
lib_class_dict['ClassA'] = ExtendClassA():
class ExtendClassA:
EDIT: Adding more details regarding the complexities I'm dealing with.
I have scenarios where method calls are buried deep within the library, which makes it hard to pass a class from the user entry point -> function:
(user would call BaseClassB.func0() in below example)
class BaseClassA:
class BaseClassB:
def func0(self):
this.class_c_obj = BaseClassC()
class BaseClassC:
def __init__(self):
this.class_d_obj = BaseClassD()
class BaseClassD:
def __init__(self):
this.class_a_obj = BaseClassA()
Multiple classes can create one type of object:
class BaseClassA:
class BaseClassB:
def func0(self):
this.class_a_obj = BaseClassA()
class BaseClassC:
def __init__(self):
this.class_a_obj = BaseClassA()
class BaseClassD:
def __init__(self):
this.class_a_obj = BaseClassA()
For these reasons I'm hoping to have a global or central location all classes can grab the correct class.
Allow them to specify the class to use as an optional parameter to func0
def BaseClassB:
def func0(self, objclass=BaseClassA):
self.class_a_obj = objclass()
obj1 = BlassClassB()
obj1.func0()
obj2 = BassClassB()
obj2.func0(objclass = ExtendClassA)
So, I've tried a PoC that, if I understand correctly, might do the trick. Give it a look.
By the way, whether it does work or not, I have a strong feeling this is actually a bad practice in almost all scenarios, as it changes class behavior in a obscure, unexpected way that would be very difficult to debug.
For example, in the below PoC if you inherit the same BaseClassA multiple times - only the latter inheritance shall be written in the class library, which would be a huge pain for the programmer trying to understand what on earth is happening with his code and why.
But of course, there are some use cases when shooting ourselves in a leg is less painful than designing & using a proper architecture :)
So, the first example where we have inheritance (I specified multiple inherited classes, just to show that only the last inherited one would be saved in a library):
#################################
# 1. We define all base classes
class BaseClassA:
def whoami(self):
print(type(self))
def __init_subclass__(cls):
omfg_that_feels_like_a_reeeeally_bad_practise['ClassA'] = cls
print('Class Dict Updated:')
print('New Class A: ' + str(cls))
#################################
# 2. We define a class library
global omfg_that_feels_like_a_reeeeally_bad_practise
omfg_that_feels_like_a_reeeeally_bad_practise = {}
omfg_that_feels_like_a_reeeeally_bad_practise['ClassA'] = BaseClassA
#################################
# 3. We define a first class that refer our base class (before inheriting from base class)
class UserClassA:
def __init__(self):
self.class_a_obj = omfg_that_feels_like_a_reeeeally_bad_practise['ClassA']()
#################################
# 4. We inherit from the base class several times
class FirstExtendedClassA(BaseClassA):
pass
class SecondExtendedClassA(BaseClassA):
pass
class SuperExtendedClassA(FirstExtendedClassA):
pass
#################################
# 5. We define a second class that refer our base class (after inheriting from base class)
class UserClassB:
def __init__(self):
self.class_a_obj = omfg_that_feels_like_a_reeeeally_bad_practise['ClassA']()
#################################
## 6. Now we try to refer both user classes
insane_class_test = UserClassA()
print(str(insane_class_test.class_a_obj))
### LOOK - A LAST INHERITED CHILD CLASS OBJECT IS USED!
# <__main__.SuperExtendedClassA object at 0x00000DEADBEEF>
insane_class_test = UserClassB()
print(str(insane_class_test.class_a_obj))
### LOOK - A LAST INHERITED CHILD CLASS OBJECT IS USED!
# <__main__.SuperExtendedClassA object at 0x00000DEADBEEF>
And if we remove inheritance, the base class will be used:
#################################
# 1. We define all base classes
class BaseClassA:
def whoami(self):
print(type(self))
def __init_subclass__(cls):
omfg_that_feels_like_a_reeeeally_bad_practise['ClassA'] = cls
print('Class Dict Updated:')
print('New Class A: ' + str(cls))
#################################
# 2. We define a class library
global omfg_that_feels_like_a_reeeeally_bad_practise
omfg_that_feels_like_a_reeeeally_bad_practise = {}
omfg_that_feels_like_a_reeeeally_bad_practise['ClassA'] = BaseClassA
#################################
# 3. We define a first class that refer our base class
class UserClassA:
def __init__(self):
self.class_a_obj = omfg_that_feels_like_a_reeeeally_bad_practise['ClassA']()
#################################
# 5. We define a second class that refer our base class
class UserClassB:
def __init__(self):
self.class_a_obj = omfg_that_feels_like_a_reeeeally_bad_practise['ClassA']()
#################################
## 6. Now we try to refer both user classes
insane_class_test = UserClassA()
print(str(insane_class_test.class_a_obj))
### LOOK - A DEFAULT CLASS OBJECT IS USED!
# <__main__.BaseClassA object at 0x00000DEADBEEF>
insane_class_test = UserClassB()
print(str(insane_class_test.class_a_obj))
### LOOK - A DEFAULT CLASS OBJECT IS USED!
# <__main__.BaseClassA object at 0x00000DEADBEEF>
I know the title is probably a bit confusing, so let me give you an example. Suppose you have a base class Base which is intended to be subclassed to create more complex objects. But you also have optional functionality that you don't need for every subclass, so you put it in a secondary class OptionalStuffA that is always intended to be subclassed together with the base class. Should you also make that secondary class a subclass of Base?
This is of course only relevant if you have more than one OptionalStuff class and you want to combine them in different ways, because otherwise you don't need to subclass both Base and OptionalStuffA (and just have OptionalStuffA be a subclass of Base so you only need to subclass OptionalStuffA). I understand that it shouldn't make a difference for the MRO if Base is inherited from more than once, but I'm not sure if there are any drawbacks to making all the secondary classes inherit from Base.
Below is an example scenario. I've also thrown in the QObject class as a 'third party' token class whose functionality is necessary for one of the secondary classes to work. Where do I subclass it? The example below shows how I've done it so far, but I doubt this is the way to go.
from PyQt5.QtCore import QObject
class Base:
def __init__(self):
self._basic_stuff = None
def reset(self):
self._basic_stuff = None
class OptionalStuffA:
def __init__(self):
super().__init__()
self._optional_stuff_a = None
def reset(self):
if hasattr(super(), 'reset'):
super().reset()
self._optional_stuff_a = None
def do_stuff_that_only_works_if_my_children_also_inherited_from_Base(self):
self._basic_stuff = not None
class OptionalStuffB:
def __init__(self):
super().__init__()
self._optional_stuff_b = None
def reset(self):
if hasattr(super(), 'reset'):
super().reset()
self._optional_stuff_b = None
def do_stuff_that_only_works_if_my_children_also_inherited_from_QObject(self):
print(self.objectName())
class ClassThatIsActuallyUsed(Base, OptionalStuffA, OptionalStuffB, QObject):
def __init__(self):
super().__init__()
self._unique_stuff = None
def reset(self):
if hasattr(super(), 'reset'):
super().reset()
self._unique_stuff = None
What I can get from your problem is that you want to have different functions and properties based on different condition, that sounds like good reason to use MetaClass.
It all depends how complex your each class is, and what are you building, if it is for some library or API then MetaClass can do magic if used rightly.
MetaClass is perfect to add functions and property to the class based on some sort of condition, you just have to add all your subclass function into one meta class and add that MetaClass to your main class
From Where to start
you can read about MetaClass here, or you can watch it here.
After you have better understanding about MetaClass see the source code of Django ModelForm from here and here, but before that take a brief look on how the Django Form works from outside this will give You an idea on how to implement it.
This is how I would implement it.
#You can also inherit it from other MetaClass but type has to be top of inheritance
class meta_class(type):
# create class based on condition
"""
msc: meta_class, behaves much like self (not exactly sure).
name: name of the new class (ClassThatIsActuallyUsed).
base: base of the new class (Base).
attrs: attrs of the new class (Meta,...).
"""
def __new__(mcs, name, bases, attrs):
meta = attrs.get('Meta')
if(meta.optionA){
attrs['reset'] = resetA
}if(meta.optionB){
attrs['reset'] = resetB
}if(meta.optionC){
attrs['reset'] = resetC
}
if("QObject" in bases){
attrs['do_stuff_that_only_works_if_my_children_also_inherited_from_QObject'] = functionA
}
return type(name, bases, attrs)
class Base(metaclass=meta_class): #you can also pass kwargs to metaclass here
#define some common functions here
class Meta:
# Set default values here for the class
optionA = False
optionB = False
optionC = False
class ClassThatIsActuallyUsed(Base):
class Meta:
optionA = True
# optionB is False by default
optionC = True
EDIT: Elaborated on how to implement MetaClass.
Let me start with another alternative. In the example below the Base.foo method is a plain identity function, but options can override that.
class Base:
def foo(self, x):
return x
class OptionDouble:
def foo(self, x):
x *= 2 # preprocess example
return super().foo(x)
class OptionHex:
def foo(self, x):
result = super().foo(x)
return hex(result) # postprocess example
class Combined(OptionDouble, OptionHex, Base):
pass
b = Base()
print(b.foo(10)) # 10
c = Combined()
print(c.foo(10)) # 2x10 = 20, as hex string: "0x14"
The key is that in the definition of the Combined's bases are Options specified before the Base:
class Combined(OptionDouble, OptionHex, Base):
Read the class names left-to right and in this simple case
this is the order in which foo() implementations are ordered.
It is called the method resolution order (MRO).
It also defines what exactly super() means in particular classes and that is important, because Options are written as wrappers around the super() implementation
If you do it the other way around, it won't work:
class Combined(Base, OptionDouble, OptionHex):
pass
c = Combined()
print(Combined.__mro__)
print(c.foo(10)) # 10, options not effective!
In this case the Base implementation is called first and it directly returns the result.
You could take care of the correct base order manually or you could write a function that checks it. It walks through the MRO list and once it sees the Base it will not allow an Option after it.
class Base:
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
base_seen = False
for mr in cls.__mro__:
if base_seen:
if issubclass(mr, Option):
raise TypeError( f"The order of {cls.__name__} base classes is incorrect")
elif mr is Base:
base_seen = True
def foo(self, x):
return x
class Option:
pass
class OptionDouble(Option):
...
class OptionHex(Option):
...
Now to answer your comment. I wrote that #wettler's approach could be simplified. I meant something like this:
class Base:
def __init_subclass__(cls, *args, **kwargs):
super().__init_subclass__(*args, **kwargs)
print("options for the class", cls.__name__)
print('A', cls.optionA)
print('B', cls.optionB)
print('C', cls.optionC)
# ... modify the class according to the options ...
bases = cls.__bases__
# ... check if QObject is present in bases ...
# defaults
optionA = False
optionB = False
optionC = False
class ClassThatIsActuallyUsed(Base):
optionA = True
optionC = True
This demo will print:
options for the class ClassThatIsActuallyUsed
A True
B False
C True
I have a class with a static method which is called multiple times by other methods. For example:
class A:
def __init__(self):
return
#staticmethod
def one():
return 1
def two(self):
return 2 * A.one()
def three(self):
return 3 * A.one()
Method one is a utility function that belongs inside the class but isn't logically an attribute of the class or the class instance.
If the name of the class were to be changed from A to B, do I have to explicitly change every call to method one from A.one() to B.one()? Is there a better way of doing this?
I pondered this question once upon a time and, while I agree that using a refactoring utility is probably the best way to go, as far as I can tell it is technically possible to achieve this behaviour in two ways:
Declare the method a classmethod.
Use the __class__ attribute. Leads to rather messy code, and may well be deemed unsafe or inefficient for reasons I am not aware of(?).
class A:
def __init__(self):
return
#staticmethod
def one():
return 1
#classmethod
def two(cls):
return 2 * cls.one()
def three(self):
return 3 * self.__class__.one()
a = A()
print(a.two())
print(a.three())
I have a number of Python types that describe a hierarchy in the sense that they are increasingly specific in terms of their properties. Instead of trying to describe it in words, here is an example:
class A:
#property
def prop1(self):
return self._prop1
class B:
#property
def prop1(self):
return self._prop1
#property
def prop2(self):
return self._prop2
class C:
#property
def prop1(self):
return self._prop1
#property
def prop2(self):
return self._prop2
#property
def prop3(self):
return self._prop3
So, as you go down the class list, B has all of the properties of A and then some extra ones. C has all of the properties of B and then some extra ones, and so on. I would like to minimize the duplication of the above definitions if possible.
One obvious solution would be to use inheritance, making B a subclass of A and so on. However, the semantics of these types do not follow an is-a relationship; I do not want isinstance(bObject, A) to be True. Is there an alternative way in Python to straightforwardly allow this sharing of attributes without using subclasses?
You can use a decorator:
def has_prop1(cls):
#property
def prop1(self):
return self._prop1
cls.prop1 = prop1
return cls
#has_prop1
class A(object):
pass
Compositing would go like this:
#has_prop1
#has_prop2
class B(object):
pass
Or even like this:
def has_many_properties(cls):
return has_prop1(has_prop2(has_prop3(cls)))
#has_many_properties
class C(object):
pass
You say "I don't want to lie and say that C is an A when it really isn't. The characteristics of the types don't fit the semantics of subclassing." With your obscured example, it's not clear which way to go.
If you're saying this because there's additional functionality in class A that doesn't apply to class C, then I would hoist out the property part of the class compositionally, so you have A, B, C containing AProps, BProps, and CProps respectively, and give the property classes a proper inheritance relationship.
If there's no such additional functionality, and it's not just coincidental that the properties are built up the way you show, then mentally rename class A to class ProvidesProp1, note that class C is-a class that provides prop1, and accept that class C therefore is-a class A.