I have a tornado application that previously had the following structure:
class handler(tornado.websocket.WebSocketHandler):
#tornado.gen.coroutine
def do_something(self,m):
# do something
def on_message(self,msg):
tornado.ioloop.IOLoop.current().spawn_callback(self.do_something,m)
Unfortunately this approach is becoming too cumbersome as my application has increased in size. I need to split out the helper methods E.g. do_something() from the handler class. The solution I have thought about is the creating another class:
class SessionMsgHandlers(object):
#staticmethod
#tornado.gen.coroutine
def do_something(ws,m):
# do stuff
then calling this function like:
tornado.ioloop.IOLoop.current().spawn_callback(SessionMsgHandlers.do_something,self,m)
This will allow me to group my functions outside of the WebSocketHandler class however it will mean nearly all my functions will be static. The only real benefit of having them associated to a class is that i can group different handler functions together. From a design perspective would it be better to just have a standard module with functions rather than associate them to a specific class (that will never have an instance) that has static methods? Also is there much performance penalty in having these as static methods rather than normal functions?
If you need to extract the methods in order to group them by common functionality, perhaps it might be the best to make your classes into mixins:
class SessionMsgMixin(object):
#tornado.gen.coroutine
def do_something(ws,m):
# do stuff
class MyHandler(tornado.websocket.WebSocketHandler, SessionMsgMixin):
def on_message(self,msg):
tornado.ioloop.IOLoop.current().spawn_callback(self.do_something,m)
Related
Here's my problem:
I have a class. And I have two objects of that class: ObjectOne and ObjectTwo
I'd like my class to have certain methods for ObjectOne and different methods for ObjectTwo.
I'd also like to choose those methods from a variety depending on some condition.
and of course, I need to call the methods I have 'from the outside code'
As I see the solution on my own (just logic, no code):
I make a default class. And I make a list of functions defined somewhere.
IF 'some condition' is True I construct a child class that takes one of those functions and adds it into class as class method. Otherwise I add some default set of methods. Then I make ObjectOne of this child class.
The question is: can I do that at all? And how do I do that? And how do I call such a method once it is added? They all would surely be named differently...
I do not ask for a piece of working code here. If you could give me a hint on where to look or maybe a certain topic to learn, this would do just fine!
PS: In case you wonder, the context is this: I am making a simple game prototype, and my objects represent two game units (characters) that fight each other automatically. Something like an auto-chess. Each unit may have unique abilities and therefore should act (make decisions on the battlefield) depending on the abilities it has. At first I tried to make a unified decision-making routine that would include all possible abilities at once (such as: if hasDoubleStrike else if... etc). But it turned out to be a very complex task, because there are tens of abilities overall, each unit may have any two, so the number of combinations is... vast. So, now I am trying to distribute this logic over separate units: each one would 'know' only of its own two abilities.
I mean I believe this is what would generally be referred to as a bad idea, but... you could have an argument passed into the class's constructor and then define the behavior/existence of a function depending on that condition. Like So:
class foo():
def __init__(self, condition):
if condition:
self.func = lambda : print('baz')
else:
self.func = lambda : print('bar')
if __name__ == '__main__':
obj1 = foo(True)
obj2 = foo(False)
obj1.func()
obj2.func()
Outputs:
baz
bar
You'd likely be better off just having different classes or setting up some sort of class hierarchy.
So in the end the best solution was the classical factory method and factory class. Like this:
import abc
import Actions # a module that works as a library of standard actions
def make_creature(some_params):
creature_factory = CreatureFactory()
tempCreature = creature_factory.make_creature(some_params)
return tempCreature
class CreatureFactory:
def make_creature(some_params):
...
if "foo" in some_params:
return fooChildCreature()
class ParentCreature(metaclass=abc.ABCMeta):
someStaticParams = 'abc'
#abc.abstractmethod
def decisionMaking():
pass
class fooChildCreature(ParentCreature):
def decisionMaking():
Actions.foo_action()
Actions.bar_action()
# some creature-specific decision making here that calls same static functions from 'Actions'
NewCreature = make_creature(some_params)
This is not ideal, this still requires much manual work to define decision making for various kinds of creatures, but it is still WAY better than anything else. Thank you very much for this advice.
I want to use metaclass to implement a factory which make processors for data coming in from different sources. Following is the skeleton code:
class ProcessorFactory
def __call__(self, classname, supers, classdict):
...
def __New__(self, classname, supers, classdict):
...
def __int__(self):
...
class MQ_AddOn(object):
# MQ-specific code
class File_AddOn(object):
# Filesystem-specific code
class Web_AddOn(object):
# Web-specific code
class MQ_Processor(MQ_AddOn, metaclass=ProcessorFactory()):
# code common to all channels (MQ, Filesystem, Web)
class File_Processor(File_AddOn, metaclass=ProcessorFactory()):
# code common to all channels (MQ, Filesystem, Web)
class Web_Processor(Web_AddOn, metaclass=ProcessorFactory()):
# code common to all channels (MQ, Filesystem, Web)
My question is whether there is a way, similar to macro expansion in assembly, to factor out the code common to all channels (MQ, Filesystem, Web) so that it doesn't have to be copied for each of those class?
Sorry - I think you have to expand somewhat more on your pseudocode for a signficative answer.
the way for not copying code around is just use the normal inheritance mechanisms in Python - If you have code common to all those classes, just put that code in a common baseclass, which might be a mixin (i.e. no need for it to be the "only" baseclass), and your are set.
If the common code have to call methods for data aquisition or processing that is specific for each of the subclasses, just write fine grained methods to perform that, and place calls for those in the common method.
Also, there is no need in this process to have ither a "metaclass" or for an "inline macro expansion" - just use plain methods and finer-grained methods.
class Base:
def process(self,...):
# preparing code
...
# data gather
data = self.fetch_data()
# common data pre-processing code:
...
# specific data processing code:
post_data = self.refine_data(data)
# more common code
...
# specific output:
self.output(post_data)
def fetch_data(self, ...):
pass
def refine_data(self, data):
pass
def output(self, data):
pass
And then, on the subclasses you implement those addressing the specific channel peculiarities. There is no big secret there.
Even if you need a lot more things to be done, like steps that are to be called from more than one "leaf" class in each step, you are better of having your class feature a "pipeline" data member, were the steps are annotated and called in order - still, no need to (and no sense in) involving metaclasses.
In your example, the "Base" class could fit all the "common code" you mention, and be used with multiple inheritance:
class MQProcessor(MQAddOn, Base):
...
...
(when you write __call__ and __new__ methods in what would be a metaclass, one might try to think 'ok -these are metaclass-related things that possibly make sense' - but __int__ in a metaclass makes no sense at all: it would provide a way to map a class (not an instance, not some data content) to an integer - even if you are putting these classes in a list, and they need indexes to be located in the list, and you want to cross-reference those, you should just add a custom ".index" attribute to the class, not write __int__ on the metaclass)
I have a main class that has a ton of different functions in it. It's getting hard to manage. I'd like to be able to separate those functions into a separate file, but I'm finding it hard to come up with a good way to do so.
Here's what I've done so far:
File main.py
import separate
class MainClass(object):
self.global_var_1 = ...
self.global_var_2 = ...
def func_1(self, x, y):
...
def func_2(self, z):
...
# tons of similar functions, and then the ones I moved out:
def long_func_1(self, a, b):
return separate.long_func_1(self, a, b)
File separate.py
def long_func_1(obj, a, b):
if obj.global_var_1:
...
obj.func_2(z)
...
return ...
# Lots of other similar functions that use info from MainClass
I do this because if I do:
obj_1 = MainClass()
I want to be able to do:
obj_1.long_func_1(a, b)
instead of:
separate.long_func_1(obj_1, a, b)
I know this seems kind of nit-picky, but I want just about all of the code to start with obj_1., so there isn't confusion.
Is there a better solution that what I'm currently doing? The only issues that I have with my current setup are:
I have to change arguments for both instances of the function
It seems needlessly repetitive
I know this has been asked a couple of times, but I couldn't quite understand the previous answers and/or I don't think the solution quite represents what I'm shooting for. I'm still pretty new to Python, so I'm having a tough time figuring this out.
Here is how I do it:
Class (or group of) is actually a full module. You don't have to do it this way, but if you're splitting a class on multiple files I think this is 'cleanest' (opinion).
The definition is in __init__.py, methods are split into files by a meaningful grouping.
A method file is just a regular Python file with functions, except you can't forget 'self' as a first argument. You can have auxiliary methods here, both taking self and not.
Methods are imported directly into the class definition.
Suppose my class is some fitting GUI (this is actually what I did this for first time). So my file hierarchy may look something like
mymodule/
__init__.py
_plotstuff.py
_fitstuff.py
_datastuff.py
So plot stuff will have plotting methods, fit stuff contains fitting methods, and data stuff contains methods for loading and handling of data - you get the point. By convention I mark the files with a _ to indicate these really aren't meant to be imported directly anywhere outside the module. So _plotsuff.py for example may look like:
def plot(self,x,y):
#body
def clear(self):
#body
etc. Now the important thing is file __init__.py:
class Fitter(object):
def __init__(self,whatever):
self.field1 = 0
self.field2 = whatever
# Imported methods
from ._plotstuff import plot, clear
from ._fitstuff import fit
from ._datastuff import load
# static methods need to be set
from ._static_example import something
something = staticmethod(something)
# Some more small functions
def printHi(self):
print("Hello world")
Tom Sawyer mentions PEP-8 recommends putting all imports at the top, so you may wish to put them before __init__, but I prefer it this way. I have to say, my Flake8 checker does not complain, so likely this is PEP-8 compliant.
Note the from ... import ... is particularly useful to hide some 'helper' functions to your methods you don't want accessible through objects of the class. I usually also place the custom exceptions for the class in the different files, but import them directly so they can be accessed as Fitter.myexception.
If this module is in your path then you can access your class with
from mymodule import Fitter
f = Fitter()
f.load('somefile') # Imported method
f.plot() # Imported method
It is not completely intuitive, but not too difficult either. The short version for your specific problem was you were close - just move the import into the class, and use
from separate import long_func_1
and don't forget your self!
How to use super addendum
super() is a useful nifty function allowing parent method access in a simple and readable manner from the child object. These kind of classes are big to begin with, so inheritance not always make sense, but if it does come up:
For methods defined in the class itself, within __init__.py, you can use super() normally, as is.
If you define you method in another module (which is kind of the point here), you can't use super as is since the function is not defined in the context of your cell, and will fail. The way to handle this is to use the self argument, and add the context yourself:
def print_super(self):
print('Super is:', super(type(self), self))
Note you cannot omit the second argument, since out of context super does not bind the object method (which you usually want for calls like super(...).__init__()).
If this is something you want to do in many methods in different modules, you may want to provide a super method in the __init__.py file for use:
def MySuper(self):
return super()
usable by self in all methods.
I use the approach I found here. It shows many different approaches, but if you scroll down to the end, the preferred method is to basically go the opposite direction of #Martin Pieter's suggestion which is have a base class that inherits other classes with your methods in those classes.
So the folder structure is something like:
_DataStore/
__init__.py
DataStore.py
_DataStore.py
So your base class would be:
File DataStore.py
import _DataStore
class DataStore(_DataStore.Mixin): # Could inherit many more mixins
def __init__(self):
self._a = 1
self._b = 2
self._c = 3
def small_method(self):
return self._a
Then your Mixin class:
File _DataStore.py
class Mixin:
def big_method(self):
return self._b
def huge_method(self):
return self._c
Your separate methods would be located in other appropriately named files, and in this example it is just _DataStore.
I am interested to hear what others think about this approach. I showed it to someone at work and they were scared by it, but it seemed to be a clean and easy way to separate a class into multiple files.
Here is an implementation of Martijn Pieters's comment to use subclasses:
File main.py
from separate import BaseClass
class MainClass(BaseClass):
def long_func_1(self, a, b):
if self.global_var_1:
...
self.func_2(z)
...
return ...
# Lots of other similar functions that use info from BaseClass
File separate.py
class BaseClass(object):
# You almost always want to initialize instance variables in the `__init__` method.
def __init__(self):
self.global_var_1 = ...
self.global_var_2 = ...
def func_1(self, x, y):
...
def func_2(self, z):
...
# tons of similar functions, and then the ones I moved out:
#
# Why are there "tons" of _similar_ functions?
# Remember that functions can be defined to take a
# variable number of/optional arguments, lists/tuples
# as arguments, dicts as arguments, etc.
from main import MainClass
m = MainClass()
m.func_1(1, 2)
....
Python 3.6
I just found myself programming this type of inheritance structure (below). Where a sub class is calling methods and attributes of an object a parent has.
In my use case I'm placing code in class A that would otherwise be ugly in class B.
Almost like a reverse inheritance call or something, which doesn't seem like a good idea... (Pycharm doesn't seem to like it)
Can someone please explain what is best practice in this scenario?
Thanks!
class A(object):
def call_class_c_method(self):
self.class_c.do_something(self)
class B(A):
def __init__(self, class_c):
self.class_c = class_c
self.begin_task()
def begin_task(self):
self.call_class_c_method()
class C(object):
def do_something(self):
print("I'm doing something super() useful")
a = A
c = C
b = B(c)
outputs:
I'm doing something super() useful
There is nothing wrong with implementing a small feature in class A and use it as a base class for B. This pattern is known as mixin in Python. It makes a lot of sense if you want to re-use A or want to compose B from many such optional features.
But make sure your mixin is complete in itself!
The original implementation of class A depends on the derived class to set a member variable. This is a particularly ugly approach. Better define class_c as a member of A where it is used:
class A(object):
def __init__(self, class_c):
self.class_c = class_c
def call_class_c_method(self):
self.class_c.do_something()
class B(A):
def __init__(self, class_c):
super().__init__(class_c)
self.begin_task()
def begin_task(self):
self.call_class_c_method()
class C(object):
def do_something(self):
print("I'm doing something super() useful")
c = C()
b = B(c)
I find that reducing things to abstract letters in cases like this makes it harder for me to reason about whether the interaction makes sense.
In effect, you're asking whether it is reasonable for a class(A) to depend on a member that conforms to a given interface (C). The answer is that there are cases where it clearly does.
As an example, consider the model-view-controller pattern in web application design.
You might well have something like
class Controller:
def get(self, request)
return self.view.render(self, request)
or similar. Then elsewhere you'd have some code that found the view and populated self.view in the controller. Typical examples of doing that include some routing lookups or include having a specific view associated with a controller. While not Python, the Rails web framework does a lot of this.
When we have specific examples, it's a lot easier to reason about whether the abstractions make sense.
In the above example, the controller interface depends on having access to some instance of the view interface to do its work. The controller instance encapsulates an instance that implements that view interface.
Here are some things to consider when evaluating such designs:
Can you clearly articulate the boundaries of each interface/class? That is, can you explain what the controller's job is and what the view's job is?
Does your decision to encapsulate an instance agree with those scopes?
Do the interface and class scopes seem reasonable when you think about future extensibility and about minimizing the scope of code changes?
I really don't know how to word this problem, so I'll try to explain it with an example.
Let's say I have three GUI classes:
Base Surface class
Detailed Surface Class
Sprite Class
All of them are independent of each other, no inheritance among them.
Now I have a function "drag()" that makes a surface/sprite dragable, and I want to implement this function as a method for all three of them.
Since it's the exact same code for all implementations I find it annoying, cumbersome and bad practice to rewrite the code.
The only thing I came up with so far was to make a saperate class for it and inherit this class. But that also doesn't seem to be the way to go.
I'd be very thankfull for some advice.
EDIT
Another example with a slightly different setup - I have the following classes:
BaseSurface
Dragable
Resizable
EventHandler
Only the first one is independent, the others depend on the first (must be inherited).
The end user should, without any effort, be able to choose between a simple BaseSurface, one with that implements dragable, one with resizable, one with eventHandler, and any combination. By "without any effort" I mean the end user should not have to make e custom Class and inherit the desired classes plus call the appropriate methods (init, update, ...) that some classes share.
So what I could do is make a class for every possible combination, eg.
"BaseSurfaceDrag", "BaseSurfaceDragResize", ...
which will get messy really quickly. Whats a different and better approach to this?
This is exactly the kind of case that you should use a parent class for. In both cases it looks like your parent class (logically) should be something like:
class Drawable(object):
def drag(self, *args, **kwargs):
"""Drag and drop behavior"""
# Your code goes here
Then each of your other classes inherits from that
class BaseSurface(Drawable):
# stuff
class DetailedSurface(Drawable):
# stuff
class Sprite(Drawable):
# stuff
In the second case what you have are interfaces, so you could logically do something like:
class DragInterface(object):
"""Implements a `drag` method"""
def drag(self):
"""Drag and drop behavior"""
# Your code goes here
class ResizeInterface(object):
"""Implements a `resize` method"""
def resize(self):
"""Drag and drop resize"""
# Code
class EventHandlerInterface(object):
"""Handles events"""
def handle(self, evt):
# Code
class MyNewSurface(BaseSurface, DragInterface, ResizeInterface):
"""Draggable, resizeable surface"""
# Implement here