I'm in scenario where I want to refactor several classes which have identical and/or similar methods. The number of class are around ~20 and the number of similar methods are around ~15. All sorts of combinations exist within this space, which is why I'm a bit reluctant to using inheritance to solve this issue (rightfully?).
The code is part of a wrapper around another application that is controlled by a com api. The wrapper in turn is part of a package that is distributed internally at the company where I work. Therefore the interfaces of the classes have to remain the same (for backwards compatibility).
This example illustrates some very simplified versions of the classes:
class FirstCollectionLike:
def __init__(self):
self._collection = list()
def add(self, arg):
self._collection.append(arg)
def remove(self, index):
del self._collection[index]
class SecondCollectionLike:
def __init__(self):
self._collection = list()
self._resource = some_module.get_resource()
def start(self):
some_module.start(self.resource)
def add(self, arg):
self._collection.append(arg)
def remove(self, value):
self._collection.remove(value)
class SomeOtherClass:
def __init__(self):
self._some_attribute = 0
self._resource = some_module.get_resource()
def add(self, value):
self._some_attribute += value
def start(self):
some_module.start(self._resource)
Are there any design patterns I could look into that would help me solve this issue?
My initial thought was to create method classes like Add, RemoveByIndex and RemoveByName that implements __call__ like so:
class Add:
def __init__(self, owner):
self.owner = owner
def __call__(self, item):
self._collection.append(item)
class AddAndInstantiate:
def __init__(self, owner, type_to_instantiate):
self.owner = owner
self.type_to_instantiate = type_to_instantiate
def __call__(self, name):
self._collection.append(type_to_instantiate(name))
and then assign instances of those classes as instance attributes to their respective owner objects:
class RefactoredClassOne:
def __init__(self):
self.add = Add(self)
self.remove = RemoveByIndex(self)
class RefactoredClassTwo:
def __init__(self):
self.add = AddAndInstantiate(self, SomeClass)
self.remove = RemoveByName(self)
This way I could quite easily add any method I want to a class and provide some arguments to the method class if needed (like the type of the class to instantiate in the example above). The downside is that it is a bit harder to follow what is happening, and the automatic documentation generation we use (sphinx) does not work if the methods are implemented in this way.
Does this seem like a bad approach? What are the alternatives?
First, if your classes are as simple as you example suggest, I'm not sure OOP is the right tool. What your classes are doing is just renaming a couple of basic calls. This is useless abstraction and IMO a bad practice (why force me to look to into the SecondClassCollectionLike.py file to discover that .add() is 1) in fact a wrongly named append and 2) that my collection is in fact a listwith a fancy name?)
In that case I'd say that a functional approach might be better, and a workflow such as:
a = SecondClassCollectionLike()
a.add("x")
a.add("y")
a.remove(0)
a.start()
would be a lot clearer if it looked like
a = list()
a.append("x")
a.append(y)
del a[0]
somemodule.start()
If your classes are in fact more complex and you really want to keep the OOP approach, I think that this solution is probably close to your solution and what you're looking for.
The idea is to have modules which hold the logic. For example a _collection_behaviour.py module, which holds the add(), remove(), increment() or whatever. And a _runtime.py module, which holds that start(), stop(), etc. logic.
This way you could have classes which exibit behaviour from these modules:
calss MyClass():
def __init__(self):
self._collection = list()
from ._collection_behaviour import add
from ._collection_behaviour import remove
from ._runtime import start
But I do not see the point in making these functions classes which implement __call__ if that's all they do.
Related
A bit of an odd question, but I'm wondering how to import an object from one class to another. I imagine adding more class methods and attributes as I expand my program, and I still want to be able to use old data. I am thinking something as follows:
class old_obj:
def __init__(self, text):
self.name = text
def set_amount(self, num):
self.amount = num
def introduce_yourself(self):
print("I am {} and I am {} many".format(self.name, self.amount))
oldest = old_obj("myself")
oldest.set_amount(15)
also_old = old_obj("Bach")
class new_obj:
def __init__(self):
#some code
#more code
I want to be able to write something like:
renewed = new_obj(oldest)
also_new = new_obj(also_old)
Here, I want to retain the 15 from oldest.amount, but not complain that also_old.amount is None. In particular, I want to retain any attributes that oldest has, while not requiring that it have all possible attributes. Is there a way for me to copy over instances of a class to a new class?
Edit: edited for clarity
You could copy the object instance dict to the new class.
from copy import deepcopy
class old_obj:
def __init__(self, text):
self.name = text
def set_amount(self, num):
self.amount = num
def introduce_yourself(self):
print("I am {} and I am {} many".format(self.name, self.amount))
oldest = old_obj("myself")
class new_obj:
def __init__(self, my_old_obj):
for var, val in my_old_obj.__dict__.items():
setattr(self, var, deepcopy(val))
#some code
#more code
newest = new_obj(oldest)
I did a deepcopy of the value assuming you want unique values in the new object. But that can also be problematic because not everything can be copied (file objects for instance). There can be other oddities when duplicating attributes such as what you want to do with a generator. And if this is something like a GUI widget, it could get stranger still.
But for a lot of object types, this would work.
Slightly different take:
Your new class has a set of concerns that are probably similar to your old class. This should guide the way you update it and build out the behavior in question. With this in mind...
Provide a class method in your new class to allow construction of the new object from the old object. Don’t make this behavior a part of __init__. Your __init__ should have a more limited responsibility. For the class method, updating the new object’s __dict__ using the old object’s __dict__ would do the job.
Don’t use inheritance to make new versions of classes. Use inheritance to move from general to specific or abstract to concrete. Otherwise, you end up with code that is hard to understand and update. (Imagine several generations down of just sub-classing in order to add some new methods.)
If the number of methods and attributes is growing, you might want to consider whether or not you’re encapsulating data/behaviors that should be split into multiple classes. The guiding principle is that you should encapsulate the data/behaviors that are likely to change together. That is, when you change the way you’re implementing your program, things that don’t need to change should probably be encapsulated separate from things that need changing. If you find that a lot of your static data is bound up with an object class that you’re frequently updating (but wanting to just import the old data unchanged), then you’ve probably got two different sets of concerns, at least.
You can simply initialize the new object by passing it the old one.
class old_obj:
def __init__(self, text):
self.text = text
oldest = old_obj("myself")
class new_obj:
def __init__(self, old_inst):
self.text = old_inst.text
renewed = new_obj(oldest)
print(renewed.text)
First, make your new_obj class inherit from old_obj, so that new_obj has all the methods old_obj had:
class new_obj(olb_obj):
Then, in __init__ method of the new class you can check what is passed as the argument - a string or an object:
def __init__(self, arg):
if isinstance(arg, str):
self.text = arg
elif isinstance(arg, old_obj):
self.text = arg.text
else:
raise TypeError
I have a pretty big class that i want to break down in smaller classes that each handle a single part of the whole. So each child takes care of only one aspect of the whole.
Each of these child classes still need to communicate with one another.
For example Data Access creates a dictionary that Plotting Controller needs to have access to.
And then plotting Controller needs to update stuff on Main GUI Controller. But these children have various more inter-communication functions.
How do I achieve this?
I've read Metaclasses, Cooperative Multiple Inheritence and Wonders of Cooperative Multiple Inheritence, but i cannot figure out how to do this.
The closest I've come is the following code:
class A:
def __init__(self):
self.myself = 'ClassA'
def method_ONE_from_class_A(self, caller):
print(f"I am method ONE from {self.myself} called by {caller}")
self.method_ONE_from_class_B(self.myself)
def method_TWO_from_class_A(self, caller):
print(f"I am method TWO from {self.myself} called by {caller}")
self.method_TWO_from_class_B(self.myself)
class B:
def __init__(self):
self.me = 'ClassB'
def method_ONE_from_class_B(self, caller):
print(f"I am method ONE from {self.me} called by {caller}")
self.method_TWO_from_class_A(self.me)
def method_TWO_from_class_B(self, caller):
print(f"I am method TWO from {self.me} called by {caller}")
class C(A, B):
def __init__(self):
A.__init__(self)
B.__init__(self)
def children_start_talking(self):
self.method_ONE_from_class_A('Big Poppa')
poppa = C()
poppa.children_start_talking()
which results correctly in:
I am method ONE from ClassA called by Big Poppa
I am method ONE from ClassB called by ClassA
I am method TWO from ClassA called by ClassB
I am method TWO from ClassB called by ClassA
But... even though Class B and Class A correctly call the other children's functions, they don't actually find their declaration. Nor do i "see" them when i'm typing the code, which is both frustrating and worrisome that i might be doing something wrong.
Is there a good way to achieve this? Or is it an actually bad idea?
EDIT: Python 3.7 if it makes any difference.
Inheritance
When breaking a class hierarchy like this, the individual "partial" classes, we call "mixins", will "see" only what is declared directly on them, and on their base-classes. In your example, when writing class A, it does not know anything about class B - you as the author, can know that methods from class B will be present, because methods from class A will only be called from class C, that inherits both.
Your programming tools, the IDE including, can't know that. (That you should know better than your programming aid, is a side track). It would work, if run, but this is a poor design.
If all methods are to be present directly on a single instance of your final class, all of them have to be "present" in a super-class for them all - you can even write independent subclasses in different files, and then a single subclass that will inherit all of them:
from abc import abstractmethod, ABC
class Base(ABC):
#abstractmethod
def method_A_1(self):
pass
#abstractmethod
def method_A_2(self):
pass
#abstractmethod
def method_B_1(self):
pass
class A(Base):
def __init__(self, *args, **kwargs):
# pop consumed named parameters from "kwargs"
...
super().__init__(*args, **kwargs)
# This call ensures all __init__ in bases are called
# because Python linearize the base classes on multiple inheritance
def method_A_1(self):
...
def method_A_2(self):
...
class B(Base):
def __init__(self, *args, **kwargs):
# pop consumed named parameters from "kwargs"
...
super().__init__(*args, **kwargs)
# This call ensures all __init__ in bases are called
# because Python linearize the base classes on multiple inheritance
def method_B_1(self):
...
...
class C(A, B):
pass
(The "ABC" and "abstractmethod" are a bit of sugar - they will work, but this design would work without any of that - thought their presence help whoever is looking at your code to figure out what is going on, and will raise an earlier runtime error if you per mistake create an instance of one of the incomplete base classes)
Composite
This works, but if your methods are actually for wildly different domains, instead
of multiple inheritance, you should try using the "composite design pattern".
No need for multiple inheritance if it does not arise naturally.
In this case, you instantiate objects of the classes that drive the different domains on the __init__ of the shell class, and pass its own instance to those child, which will keep a reference to it (in a self.parent attribute, for example). Chances are your IDE still won't know what you are talking about, but you will have a saner design.
class Parent:
def __init__(self):
self.a_domain = A(self)
self.b_domain = B(self)
class A:
def __init__(self, parent):
self.parent = parent
# no need to call any "super...init", this is called
# as part of the initialization of the parent class
def method_A_1(self):
...
def method_A_2(self):
...
class B:
def __init__(self, parent):
self.parent = parent
def method_B_1(self):
# need result from 'A' domain:
a_value = self.parent.a_domain.method_A_1()
...
This example uses the basic of the language features, but if you decide
to go for it in a complex application, you can sophisticate it - there are
interface patterns, that could allow you to swap the classes used
for different domains, in specialized subclasses, and so on. But typically
the pattern above is what you would need.
My apologies for what may be a basic question. I'm a C++ programmer who is relatively new to python.
I have a python class whose behavior depends significantly on one of its constructor arguments:
class MyClass():
def __init__(self, some_arg):
self.some_arg = some_arg
...
def abcd(self):
if self.some_arg == 1:
...
else:
...
def efgh(self):
if self.some_arg == 1:
...
else:
...
I would like to refactor this into two classes with different values of some_arg. Of course, the most straightforward thing would be to have two classes (perhaps with a common base class) and then have a factory function pick which one to instantiate. Something along the lines of:
def MyClassSomeArg1():
def __init__(self):
...
def abcd(self):
...
def efgh(self):
...
def MyClassSomeArgNot1():
def __init__(self):
...
def abcd(self):
...
def efgh(self):
...
def buildMyClass(some_arg):
if some_arg == 1:
return MyClassSomeArg1()
else:
return MyClassSomeArgNot1()
I'm sure that would work fine. The problem is that I don't want to change client code. Clients expect to instantiate an object of class "MyClass" with a constructor argument of some_arg. Is there a decent way to refactor this under the hood without changing client code?
I have tried using an implementation hierarchy: MyClassImpl as a base class with subclasses MyClassImplSomeArg1 and MyClassImplSomeArgNot1. MyClass itself then becomes mostly empty:
class MyClass():
def __init__(self, some_arg):
if some_arg == 1:
self._impl = MyClassImplSomeArg1()
else:
self._impl = MyClassImplSomeArgNone1()
def __getattr__(self, a):
# For performance, I could store this in self so it doesn't need to be looked up each time
return getattr(self._impl, a)
This basically works, but it doesn't seem to be the most straightforward thing. For one thing, magic methods like __str__ and __eq__ don't seem to get delegated through the __getattr__ mechanism, and I don't know why. It's not difficult to write delegation methods myself, though. Also, this confuses pydoc (it has no way of seeing the delegated attributes), and I'm not sure how to fix that.
Is there some sugar to make this delegation scheme work nicely? Or is delegation even the best way to handle an issue like this?
Thanks,
You might override the __new__ function and return a subclass based on some_arg. It's a common pattern for implementing factory
class MyClassImplSomeArg1:
pass
class MyClassImplSomeArgNone1:
pass
class MyClass:
def __new__(cls, some_arg):
if some_arg:
return MyClassImplSomeArg1()
else:
return MyClassImplSomeArgNone1()
assert isinstance(MyClass(some_arg=True), (MyClassImplSomeArg1,))
assert isinstance(MyClass(some_arg=False), (MyClassImplSomeArgNone1,))
First off, let me say that yes I have researched this extensively for a few days now with no luck. I have looked at numerous examples and similar situations such as this one, but so far nothing has been able to resolve me issue.
My problem is I have a Python project that has a primary class, with two nested classes (yea yea I know), one of those classes is a subclass of the first. I can not figure out why I keep getting NameError: global name 'InnerSubClass' is not defined.
I understand scoping (both classes in question are in the same scope) but nothing I try seems to resolve the issue (I want to keep the two classes nested at a minimum) despite this problem working for other people.
Here is a simple example of what I am trying to do:
class SomeClass(object):
def __init__(self):
"""lots of other working stuff"""
class MainClass(object):
def __init__(self):
self.stuff = []
self.moreStuffs = []
class InnerClass(object):
def __init__(self, thing, otherThing):
self.thing = thing
self.otherThing = otherThing
self.otherStuff = []
class InnerSubClass(InnerClass):
def __init__(self, thing, otherThing, newThing):
super(InnerSubClass).__init__(thing, otherThing)
self.newThing = newThing
"""other code that worked before the addition of 'InnerSubClass'"""
def doSomething(self):
innerclass = self.InnerSubClass('thisthing', 'thatthing', 'thingthing')
print("just more thing words %s" % innerclass.newThing)
myThing = MainClass()
myThing.doSomething()
I have tried changing super(InnerSubClass).__init__(thing, otherThing)
to
super(InnerClass.InnerSubClass).__init__(thing, otherThing)
and even
super(MainClass.InnerClass.InnerSubClass).__init__(thing, otherThing) with no success. I made "InnerSubClass" inherit straight from object InnerSubClass(object): etc, and it still doesn't work.
Granted I am far from a seasoned python developer and come from mostly other compiled OO languages, and can't seem to wrap my head around why this isn't working. If I get rid of the "InnerSubClass", everything works just fine.
It doesn't seem like python offers "private" classes and functions like other languages, which is fine but I would like to utilize the nesting to at least keep objects "lumped" together. In this case, nothing should be instantiating "InnerClass" or "InnerSubClass" except functions in "MainClass".
Please provide helpful advice and explain why it doesn't work as expected with background information on how this should be done properly. If this was as simple as it seems, it would have been figured out by now.
edit: for clarification, this is only for v2
There is no "class scope" in lookup order
When creating a new class, the code in the body is executed and the resulting names are passed to type for creation. Python lookups go from inner to outer, but you don't have a "class level", only the names you define to become attributes/methods of your new class. In fact, if you want to access class variables inside a method, you use MyClass.attr instead of simple attr.
The inheritance works because InnerSubClass(InnerClass) occurs inside the class creation. To access InnerClass after MainClass has been created, do the same as you would for class attributes: MainClass.InnerClass
Just to include an example:
class Outer:
out = 1
class Inner:
inside = 2
try:
print(out) # this is confusing
except NameError:
print("can't find out")
def f(self):
try:
print(inside) # this is clear
except NameError:
print("can't find inside")
try:
print(Inner.inside) # this is less clear
except NameError:
print("can't find Inner.inside")
Outer.Inner().f()
# can't find anything
Edit:
The above is a general view, to apply it directly to your situation, look at your inner classes the way you look at regular class attributes. You'd access these as MyClass.attr, where MyClass is defined globally. If you replace attr with InnerSubClass, you get the class (attribute lookup doesn't care about inheritance, but about where the attributes are).
A stripped-down example with nested inheriting classes:
class MainClass(object):
class Inner(object):
pass
class InnerSub(Inner):
def __init__(self):
print(super(MainClass.InnerSub)) # note you use MainClass, known globally
def f(self):
return self.InnerSub()
MainClass().f() # prints "<super ...>" and returns a MainCLass.InnerSub object
Here they do it like this
super(MainClass.InnerSubClass, self).__init__(thing, otherThing)
So that you can test it here is the full working example
class SomeClass(object):
def __init__(self):
"""lots of other working stuff"""
class MainClass(object):
def __init__(self):
self.stuff = []
self.moreStuffs = []
class InnerClass(object):
def __init__(self, thing, otherThing):
self.thing = thing
self.otherThing = otherThing
self.otherStuff = []
class InnerSubClass(InnerClass):
def __init__(self, thing, otherThing, newThing):
super(MainClass.InnerSubClass, self).__init__(thing, otherThing)
self.newThing = newThing
"""other code that worked before the addition of 'InnerSubClass'"""
def doSomething(self):
innerclass = self.InnerSubClass('thisthing', 'thatthing', 'thingthing')
print("just more thing words %s" % innerclass.newThing)
print("and I also inherit from InnerClass %s" % innerclass.otherThing)
myThing = MainClass()
myThing.doSomething()
The output is
just more thing words thingthing
and I also inherit from InnerClass thatthing
If you have reasons for not using MainClass.InnerSubClass, you can also use type(self) or self.__class__ (OK, but which one) inside __init__ to get the containing class. This works well lots of layers deep (which shouldn't happen anyway), and requires the argument passed to super to be the type of the instance (which it should be anyway) but breaks if you subclass, as seen here. The concept might be clearer to you than scoping rules:
class MainClass:
class Inner:
pass
class InnerSub(Inner):
def __init__(self):
print(super(self.__class__))
print(super(type(self)))
MainClass().InnerSub()
Is there no magic python way of accessing the instance of the class that has a reference to the current self inside it?
ie:
class A(object):
def __init__(self):
self.B = B()
def say_hi(self):
print "Hi"
class B(object)
def __init__(self):
__get_owner_ref__.say_hi()
A()
get_owner_ref being the magic bullet that does not exist.
Is there a feature in python for this behaviour?
Yes I know I could pass a reference in to the constructor, but I'm looking for a more elegant solution.
No, You'd have to do something like this
class A(object):
def __init__(self):
self.B = B(parent=self)
def say_hi(self):
print "Hi"
class B(object)
def __init__(self, parent):
self.parent = parent # you don't need to do this, but it might be a good idea
parent.say_hi()
A()
On the second thought, what you're looking for pretty closely resembles descriptors. Consider:
class Agent(object):
def __get__(self, obj, objtype):
print 'Agent %s called from %s ' % (id(self), obj.name)
class X(object):
agent = Agent()
def __init__(self, name):
self.name = name
a = X('Foo')
a.agent
b = X('Bar')
b.agent
Here the agent is attached to two different instances and "knows" each time which instance wants to talk to him.
No, there is no nice way of doing this. Pass a reference in to the initializer.
To preclude questions, it's probably possible in most cases to find the owner heuristically by inspecting the stack, something like in this question. But it will be fragile, buggy and difficult to understand. And it goes against the "explicit > implicit" philosophy.
As far as I know such a feature does not exist. Also, passing it in as reference to the constructor and calling self.parent.say_hi() is much more explicit and (indeed) elegant. And explicit is better than implicit or using magic language features.
Technically, you can use sys._getframe:
class B(object):
def __init__(self):
import sys
a = sys._getframe(1).f_locals['self']
a.say_hi()
But you should not do that. It's bound to lead to confusion, will break on new Python implementations, will complicate debugging, and is prone to break.There's a reason why sys._getframe is listed in 5 Years of Bad Ideas.
Instead, pass a reference, either to the parent object, or to the say_hi method.