A senior dev would like me to implement Object Oriented Programming in Python where we instantiate all object creation using the Base class. It does not sit well with me because there are abstract methods in the Base class that the Derived class has to implement. His reasoning to use the Base class only as a way to instantiate our objects is so that when we iterate through a list of our objects, we can access its variables and methods the same way. Since each derived object of the base class has more attributes instantiated than the Base class, he suggests the init function to take in *args and **kwargs as part of the arguments.
Is this a good way to go about doing it? If not, can you help suggest a better alternative?
Here's a simple example of the implementation.
import abc
class Base(metaclass = abc.ABCMeta):
def __init__(self, reqarg1, reqarg2, **kwargs):
self.reqarg1 = reqarg1
self.reqarg2 = reqarg2
self.optarg1 = kwargs.get("argFromDerivedA", 0.123)
self.optarg2 = kwargs.get("argFromDerivedB", False)
self.dict = self.create_dict()
#abstractmethod
def create_dict(self):
pass
def get_subset_list(self, id):
return [item for item in self.dict.values() if item.id == id]
def __iter__(self):
for item in self.dict.values():
yield item
raise StopIteration()
class Derived_A(Base):
def __init__(self, regarg1, regarg2, optarg1):
super().__init__(regarg1, regarg2, optarg1)
def create_dict(self):
# some implementation
return dict
class Derived_B(Base):
def __init__(self, regarg1, regarg2, optarg2):
super().__init__(regarg1, regarg2, optarg2)
def create_dict(self):
# some implementation
return dict
EDIT: Just to make it clear, I don't quite know how to handle the abstractmethod in the base class properly as the senior dev would like to use it as follows:
def main():
b = Base(100, 200)
for i in get_subset_list(30):
print(i)
But dict in the Base class is not defined because it is defined in the derived classes and therefore will output the following error:
NameError: name 'abstractmethod' is not defined
My suggestion is that you use a factory class method in the Base class. You would only have to be able to determine the Derived class that you would need to return depending on the supplied input. I'll copy an implementation that assumes that you wanted a Derived_A if you supply the keyword optarg1, and Derived_B if you supply the keyword optarg2. Of course, this is completely artificial and you should change it to suit your needs.
import abc
class Base(metaclass = abc.ABCMeta):
#classmethod
def factory(cls,reqarg1,reqarg2,**kwargs):
if 'optarg1' in kwargs.keys():
return Derived_A(reqarg1=reqarg1,reqarg2=reqarg2,optarg1=kwargs['optarg1'])
elif 'optarg2' in kwargs.keys():
return Derived_B(reqarg1=reqarg1,reqarg2=reqarg2,optarg2=kwargs['optarg2'])
else:
raise ValueError('Could not determine Derived class from input')
def __init__(self, reqarg1, reqarg2, optarg1=0.123, optarg2=False):
self.reqarg1 = reqarg1
self.reqarg2 = reqarg2
self.optarg1 = optarg1
self.optarg2 = optarg2
self.dict = self.create_dict()
#abc.abstractmethod
def create_dict(self):
pass
def get_subset_list(self, id):
return [item for item in self.dict.values() if item.id == id]
def __iter__(self):
for item in self.dict.values():
yield item
class Derived_A(Base):
def __init__(self, reqarg1, reqarg2, optarg1):
super().__init__(reqarg1, reqarg2, optarg1=optarg1)
def create_dict(self):
# some implementation
dict = {'instanceOf':'Derived_A'}
return dict
class Derived_B(Base):
def __init__(self, reqarg1, reqarg2, optarg2):
super().__init__(reqarg1, reqarg2, optarg2=optarg2)
def create_dict(self):
# some implementation
dict = {'instanceOf':'Derived_B'}
return dict
This will allow you to always create a Derived_X class instance that will have the create_dict non-abstract method defined for when you __init__ it.
In [2]: b = Base.factory(100, 200)
ValueError: Could not determine Derived class from input
In [3]: b = Base.factory(100, 200, optarg1=1213.12)
In [4]: print(b.dict)
{'instanceOf': 'Derived_A'}
In [5]: b = Base.factory(100, 200, optarg2=True)
In [6]: print(b.dict)
{'instanceOf': 'Derived_B'}
Moreover, you can have more than one factory method. Look here for a short tutorial.
You don't have to use keyword arguments at all; just define the variables with their default value in the parameters section of the function, and send only the parameters you want to send from the derived classes.
Note that parameters with a default value doesn't have to be supplied - that way you can have a function with a ranging number of arguments (where the arguments are unique, and can not be treated as a list).
Here is a partial example (taken from your code):
import abc
class Base(metaclass = abc.ABCMeta):
def __init__(self, reqarg1, reqarg2, optarg1 = 0.123, optarg2 = False):
self.reqarg1, self.reqarg2 = reqarg1, reqarg2
self.optarg1, self.optarg2 = optarg1, optarg2
...
class Derived_A(Base):
def __init__(self, regarg1, regarg2, optarg1):
super().__init__(regarg1, regarg2, optarg1=optarg1)
...
class Derived_B(Base):
def __init__(self, regarg1, regarg2, optarg2):
super().__init__(regarg1, regarg2, optarg2=optarg2)
...
EDIT: As the question update, I would give just a small note - abstract method is there to make sure that a mixed list of some derived Base objects can call the same method. Base object itself can not call this method - it is abstract to the base class, and is just there so we can make sure every derived instance will have to implement this method.
Related
In python, how can I setup a parent class to track methods with a specific decorator for each child seperatly? A quick code snippet of what I am trying to do:
class Parent:
decorated_func_dict = {} #dictionary that stores name->func for decorated functions
def get_func_by_decorator_name(self, name):
#stuff
pass
class Child1(Parent):
#func_name("Bob")
def bob_func(self, *args):
pass
#func_name("Tom")
def func2(self, *args):
pass
class Child2(Parent):
#func_name("Bob")
def func_bob2(self, *args):
pass
foo = Child1()
bar = Child2()
foo.get_func_by_decorator_name("Bob")
#Returns foo.bob_func
bar.get_func_by_decorator_name("Bob")
#Returns bar.func_bob2
Using Python 3.9.
A decorator is not something that makes a function look pretty. It is a callable that ingests an object (not only functions), does some arbitrary operations, and returns a replacement object.
In this case, your decorator should be storing references to function objects in a dictionary somewhere. The problem is that you won't be able to reference the class in which the functions are defined until it is created, which happens well after the decorator is run. You can avoid this by storing the name of the class as well as the name of the function.
The final step here is to properly bind the function objects to methods on the right object. That is something that get_func_by_decorated_name can do for you.
In sum, you can write something like this:
decorated_func_dict = {}
def func_name(cls_name, func_name):
def decorator(func):
decorated_func_dict.setdefault(cls_name, {})[func_name] = func
return func
return decorator
class Parent:
def get_func_by_decorator_name(self, name):
return decorated_func_dict[type(self).__name__][name].__get__(self)
class Child1(Parent):
#func_name("Child1", "Bob")
def bob_func(self, *args):
pass
#func_name("Child1", "Tom")
def func2(self, *args):
pass
class Child2(Parent):
#func_name("Child2", "Bob")
def func_bob2(self, *args):
pass
And indeed you get:
>>> foo.get_func_by_decorator_name("Bob")
<bound method Child1.bob_func of <__main__.Child1 object at 0x000001D58181E070>>
>>> bar.get_func_by_decorator_name("Bob")
<bound method Child2.func_bob2 of <__main__.Child2 object at 0x000001D582041F10>>
Another way to do this is to give your functions a name attribute, which you can then aggregate into a mapping in __init_subclass__ in Parent. This allows you to make an interface a bit closer to what you originally intended:
def func_name(func_name):
def decorator(func):
func.special_name = func_name
return func
return decorator
class Parent:
def __init_subclass__(cls):
cls.decorated_func_dict = {}
for item in cls.__dict__.values():
if hasattr(item, 'special_name'):
cls.decorated_func_dict[item.special_name] = item
del item.special_name # optional
def get_func_by_decorator_name(self, name):
return self.decorated_func_dict[name].__get__(self)
class Child1(Parent):
#func_name("Bob")
def bob_func(self, *args):
pass
#func_name("Tom")
def func2(self, *args):
pass
class Child2(Parent):
#func_name("Bob")
def func_bob2(self, *args):
pass
The results are identical to the first example.
The easiest way would of course be to get access to the child's namespace before the class is created, e.g. with a metaclass.
I'm trying to store specific actions that are defined within a class.
To reduce code duplication, I would like to make use of a mixin class that stores all the actions based on a decorator.
The idea is that it should be straightforward for other people to extend the classes with new actions. I especially want to avoid that these actions are explicitly listed in the source code (this should be handled by the decorator).
This is what I came up with. Unfortunately, in all .actions lists, all the actions from all the classes are listed.
However, I would like to have a solution that only the actions of the specific class are listed.
class ActionMixin:
actions = []
#staticmethod
def action(fun):
ActionMixin.actions.append(fun)
return fun
class Human(ActionMixin):
#ActionMixin.action
def talk(self):
pass
class Dog(ActionMixin):
#ActionMixin.action
def wuff(self):
pass
class Cat(ActionMixin):
#ActionMixin.action
def miau(self):
pass
if __name__ == "__main__":
party = [Human(), Dog()]
possible_actions = [action for memer in party for action in member.actions]
# I would like that possible_actions is now only Human.talk() and Dog.wuff()
# instead it is 2 times all actions
print(len(possible_actions)) # == 6
I would just write my own descriptor here. So:
class Registry:
def __init__(self):
self._registered = []
def __call__(self, func):
self._registered.append(func)
return func
def __get__(self, obj, objtype=None):
return self._registered
class Human:
actions = Registry()
#actions
def talk(self):
pass
class Dog:
actions = Registry()
#actions
def wuff(self):
pass
class Cat:
actions = Registry()
#actions
def miau(self):
pass
So, instead of inheriting from a mixin, just initialize the descriptor object. Then that object itself can be used as the decorator (the __call__ method!).
Note, the decorator would be whatever name you assigned it, and it would be the name of the attribute where the actions are stored.
In the REPL:
In [11]: party = [Human(), Dog()]
In [12]: [action for member in party for action in member.actions]
Out[12]: [<function __main__.Human.talk(self)>, <function __main__.Dog.wuff(self)>]
EDIT:
You would have to change the implementation if you want this to live in a base class. Basically, use a dict to keep track of the registries, unfortunately, we have to rely on the brittle __qualname__ to get the class in __call__:
class ActionsRegistry:
def __init__(self):
self._registry = {}
def __call__(self, func):
klass_name, func_name = func.__qualname__.rsplit('.', 1)
if klass_name not in self._registry:
self._registry[klass_name] = []
self._registry[klass_name].append(func)
return func
def __get__(self, obj, objtype=None):
if obj is None:
return self
return self._registry[objtype.__qualname__]
class Base:
actions = ActionsRegistry()
class Human(Base):
#Base.actions
def talk(self):
pass
class Dog(Base):
#Base.actions
def wuff(self):
pass
class Cat(Base):
#Base.actions
def miau(self):
pass
Something I don't like about #absctractmethod is that it only produces error when the instance is created. For example, this will not fail:
from abc import abstractmethod, ABC
class AbstractClass(ABC):
#abstractmethod
def func(self):
pass
class RealClass(AbstractClass):
pass
it will only fail if I create an instance:
r = RealClass()
I want to reimplent this mechanism but so that it fails on class definition, not instantiation. For this, I created a metaclass:
class ABCEarlyFailMeta(type):
direct_inheritors = {}
def __new__(cls, clsname, bases, clsdict):
klass = super().__new__(cls, clsname, bases, clsdict)
class_path = clsdict['__module__'] + '.' + clsdict['__qualname__']
if bases == ():
# we get here when we create base abstract class.
# The registry will later be filled with abstract methods
cls.direct_inheritors[class_path] = {}
for name, value in clsdict.items():
# adding abstract methods on the proper base abstract class
if getattr(value, '__isabstractmethod__', None) is True:
cls.direct_inheritors[class_path][name] = signature(value)
else:
# we get here when create inheritors.
# Here, we need to extract list of abstractmethods
base_class = bases[0].__module__ + '.' + bases[0].__qualname__
abstract_method_names = cls.direct_inheritors[base_class]
# here, we compare current list of methods
# with list of abstractmethods and fail if some methods are missing
cls_dictkeys = set(clsdict.keys())
missing_methods = set(abstract_method_names) - cls_dictkeys
if missing_methods:
raise Exception(
f'{clsname} must implement methods: {missing_methods}'
)
return klass
this will fail when class is created, not instantiated:
class ABCEarlyFail(metaclass=ABCEarlyFailMeta):
#abstractmethod
def func(self):
pass
class Child(ABCEarlyFail):
pass
>>> Exception: Child must implement methods: {'func'}
My question is, how do search for proper base class in bases? In this example, I look for bases[0], but it will fail if the inheritor class has a mixin:
class Child(SomeMixin, ABCEarlyFail):
pass
so, what is a nicer way?
Or maybe I am reinventing the wheel?
Consider for example that we have a class 'Agent' as below:
class Agent:
def __init__(self, number):
self.position = []
self.number = number
for i in range(number):
self.position.append([0, 0])
I can make an instance of the class by:
agent = Agent(10)
and then access the i'th agent's position by:
agent.position[i]
However, this does not seem elegant enough and to me it's a bit counter-intuitive. Instead I want to index the class instance itself. For example:
pos_i = agent[i].position
which should return the same answer as the one-line code above. Is there a way to accomplish this?
If you want to do that, you just need a class-level container, with all instances.
Since your positions, given your example, are created in an arbitrary order, I'd suggest using a dictionary.
You can just fill the class-level "position" dictionary. You could then just implement the __getitem__ method to retrieve elements from this dictionary:
class Agent:
position = {}
def __new__(cls, pos):
if pos in cls.position:
return cls.position[pos]
instance = super().__new__(cls)
cls.position[pos] = instance
return instance
def __getitem__(self, item):
return self.position[pos]
This, however, will only allow you to retrieve an instance given the position from an instance - i.e.:
agent_5 = Agent(5)
agent_10 = agent_5[10]
would work, but not:
agent_10 = Agent[10]
If you want that, you have to use a custom metaclass, and put the __getitem__ method there:
class MAgent(type):
def __getitem__(cls, item):
return cls.position[pos]
class Agent(metaclass=MAgent):
position = {}
def __new__(cls, pos):
if pos in cls.position:
return cls.position[pos]
instance = super().__new__(cls)
cls.position[pos] = instance
return instance
If you want to overload the indexing operator just overload the __getitem__ method in the class.
class Agent:
def __getitem__(self, key):
return self.position[key]
>>> myobj = MyClass()
>>> myobj[3]
I have a big class which has a lot of functions and attributes.
the instances are created from data in a remote database.
the process of creating each instance is very long and heavy.
In performance sake ive created a bunch class from this heavy class.
so accessing the attributed is easy and works great .
the problem is how to use the methods from that class.
ex :
class clsA():
def __init__(self,obj):
self.attrA=obj.attrA
def someFunc(self):
print self
class bunchClsA(bunch):
def __getattr__(self, attr):
# this is the problem:
try:
#try and return a func
func = clsA.attr
return func
except:
# return simple attribute
return self.attr
Clearly this dosent work , Is there a way i could access the instance function staticly and override the "self" var ?
Found out a nice solution to the problem :
from bunch import Bunch
import types
#Original class:
class A():
y=6
def __init__(self,num):
self.x=num
def funcA(self):
print self.x
#class that wraps A using Bunch(thats what i needed .. u can use another):
class B(Bunch):
def __init__(self, data, cls):
self._cls = cls # notice, not an instance just the class it self
super(B, self).__init__(data)
def __getattr__(self, attr):
# Handles normal Bunch, dict attributes
if attr in self.keys():
return self[attr]
else:
res = getattr(self._cls, attr)
if isinstance(res, types.MethodType):
# returns the class func with self overriden
return types.MethodType(res.im_func, self, type(self))
else:
# returns class attributes like y
return res
data = {'x': 3}
ins_b = B(data, A)
print ins_b.funcA() # returns 3
print ins_b.y # returns 6
And this solves my issue, its a hack and if you have the privileges, redesign the code.