Redefinition of class method in python - python

Context
I'm trying to have some "plugins" (I'm not sure this is the correct definition for this) to my code. By "plugin", I mean a module which defines a model (this is a scientific code) in such a way that its existence is enough to use it anywhere else in the code.
Of course these plugins must follow a template which uses some modules/function/classes defined in my code. Here is a small snippet for the relevant part of my code:
# [In the code]
class AllModels():
def __init__(self):
"""
Init.
"""
self.count = 0
def register(self, name, model):
"""
Adds a model to the code
"""
setattr(self, name, model)
self.count += 1
return
class Model():
def __init__(self, **kwargs):
"""
Some constants that defines a model
"""
self.a = kwargs.get("a", None)
self.b = kwargs.get("b", None)
# and so on...
def function1(self, *args, **kwargs):
"""
A function that all models will have, but which needs:
- to have a default behavior (when the instance is created)
- to be redefinable by the "plugin" (ie. the model)
"""
# default code for the default behavior
return
instance = AllModels()
and here is the relevant part of the "plugin":
# [in the plugin file]
from code import Model, instance
newmodel = Model(a="a name", b="some other stuff")
def function1(*args, **kwargs):
"""
Work to do by this model
"""
# some specific model-dependent work
return
instance.register(newmodel)
Additional information and requirements
function1 has exactly the same signature for any model plugin, but
is usually doing a different job for each.
I'd like a default behavior for the function1 so that if it is not
defined by the plugin, I'll still be able to do something (try
different possibilities, and/or raise a warning/error).
In the plugin, function1 may use some other functions that are only defined in this plugin. I'm stating this because the code is running with the multiprocessing module, and I need the instance instance of AllModels to be able to call function1 in child processes. instance is defined in the parent process, as well as the model plugins, but will be used in different child processes (no modification is done to it though).
it would be awesome that function1, when "redefined" by the plugin, be able to access the attributes of the Model instance (ie. self).
Problem
I've read many different sources of python documentation and some SO question. I only see two/three possible solutions to this problem:
1) not declaring function1 method in Model class, but just set it as an attribute when the plugin creates a new instance of it.
# [in the plugin file]
def function1(*args, **kwargs):
# ....
return
newmodel.function1 = function1
and then call it whenever needed. In that case the attribute function1 in the object Model would be initiate to None probably. One caveat of that is that there is no "default behaviour" for function1 (it has to be dealt in the code, eg. testing if instance.function1 is None: ...), and an even bigger one is that I can't access self this way...
2) using somehow the python decorators. I've never used this, and the documentation I've read is not that simple (I mean not straight forward due to the huge number of possibilities on its usage). But it seems to be a good solution. However I'm worried about its performance impact (I've read that it could slow down the execution of the decorated function/method). If this solution is the best option, then I'd like to know how to use it (a quick snippet maybe), and if it is possible to use attributes of the class Model:
# [in the plugin file]
#mydecorator
def function1(self, *args, **kwargs):
"""
I'm not sure I can use *self*, but it would be great since some attributes of self are used for some other function similar to *function1*...
"""
# some stuff using *self*, eg.:
x = self.var **2 + 3.4
# where self.var has been defined before, eg.: newmodel.var = 100.
3) using the module types and its MethodType... I'm not sure that is relevant in my case... but I may be wrong.
As you can probably see after this long question, I'm not very familiar with such python features, and my understanding of decorators is really poor now. While keeping reading some documentation, I thought that might be worth to ask the question here since I'm not sure of the direction to take in order to treat my problem.
Solution
The beauty of the answer of Senderle is that it is really simple and obvious... And having missed it is a shame. Sorry for polluting SO with that question.

Well, unless I'm mistaken, you want to subclass Model. This is sort of like creating an instance of Model and replacing its function1 attribute with a function defined in the plugin module (i.e. your option 1); but it's much cleaner, and takes care of all the details for you:
# [in the plugin file]
from code import Model, instance
class MyModel(Model):
def function1(*args, **kwargs):
"""
Work to do by this model
"""
# some specific model-dependent work
return
newmodel = MyModel(a="a name", b="some other stuff")
instance.register(newmodel)
This way, all the other methods (functions "attached" to a Model instance) are inherited from Model; they will behave in just the same way, but function1 will be overridden, and will follow your customized function1 definition.

Could you write a dummy function1() function in the Model class and raise a NotImplementedError? That way, if anyone tries to inherit from Model without implementing function1(), they'll get an exception when they try to run the code. If you're running the code for them, you can catch that error and return a helpful error message to the user.
For example:
class Model:
#Your code
def function1():
raise NotImplementedError("You need to implement function1
when you inherit from Model")
Then, you can do the following when you run the code:
try:
modelObj.function1()
except NotImplementedError as e:
#Perform error handling here
EDIT: The official Python documentation for NotImplementedError states: "In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method." That does seem to fit the requirements here.

What you are trying to do canbe done in pretty straightforward ways - just using Objected Oriented techniques and taking advantage that in Python functions are also normal objects -
One simple thing to do is just having your "model" class to accept the "function1" as a parameter, and store it as an object member.
Some code like this, with minimal changes to your code - though much more interesting things are certainly possible:
# [In the code]
class AllModels():
def __init__(self):
"""
Init.
"""
self.count = 0
def register(self, name, **kwargs):
"""
Adds a model to the code
"""
model = Model(**kwargs)
setattr(self, name, model)
self.count += 1
return
class Model():
def __init__(self, **kwargs):
"""
Some constants that defines a model
"""
self.a = kwargs.get("a", None)
self.b = kwargs.get("b", None)
if "function1" in kwargs:
self.real_function1 = kwargs["function1"]
self.function1.__doc__ = kwargs["function1"].__doc__
# and so on...
def function1(self, *args, **kwargs):
"""
A function that all models will have, but which needs:
- to have a default behavior (when the instance is created)
- to be redefinable by the "plugin" (ie. the model)
"""
if self.real_function1:
return self.real_function1(self, *args, **kwargs)
# default code for the default behavior
return
instance = AllModels()
and
# [in the plugin file]
from code import Model, instance
newmodel = Model(a="a name", b="some other stuff")
def function1(self, *args, **kwargs):
"""
Work to do by this model
"""
# some specific model-dependent work
return
instance.register("name", function1 = function1, a="a name", b="some other stuff")

Related

How to write a wrapper class that contains multiple workers in Python?

I have a couple of classes that have the same methods but do things slightly different (WorkerOne, WorkerTwo). Those classes inherit from an abstract base class using the abc module and #abstractmethod annotation for the methods that should be implemented in WorkerOne and WorkerTwo.
Note: The actual question comes at the end.
Here's the shortened code:
class AbstractWorker(metaclass=ABCMeta):
#abstractmethod
def log_value(self, value):
pass
class WorkerOne(AbstractWorker):
def log_value(self, value):
# do something differently
class WorkerTwo(AbstractWorker):
def log_value(self, value):
# do something differently
This works fine and I can create objects for both worker classes and execute the functions accordingly.
E.g.
worker_one = WorkerOne()
worker_two = WorkerTwo()
worker_one.log_value(1)
worker_two.log_value('text')
Please note that this is simplified. Each worker uses a different package to track experiments in the ML field and not just differentiates between int and str.
I've been trying to find a way to not call both objects every single time I want to log something. I want to unify these methods in some sort of wrapper class that takes those two objects, and executes the method called on that wrapper, on each object. I call that wrapper a hive as it contains it workers.
Currently, I see two solutions to this but both are lacking quality. The first would be the easier one but results in duplication of code. It is simple and it works but it doesn't follow the DRY principle.
Solution #1:
class HiveSimple(AbstractWorker):
def __init__(self, workers: List[AbstractWorker]):
self.workers = workers
def log_value(self, value):
for worker in self.workers:
worker.log_value(value)
...
The idea is to have the wrapper/hive class to inherit from the abstract class as well, so we are forced to implement the functions. The workers are passed as a list for creating the object. For the log_value function we would iterate through the list of workers and execute their own implementation of that method. The problem, as shortly mentioned, is 1) duplicated code and 2) the hive class also grows or needs to be altered when a new method is added to the abstract base class.
The second solution is a bit more advanced and avoids duplicated code but has also a disadvantage.
Solution #2:
class Hive:
def __init__(self, trackers: List[AbstractWorker]):
self.workers = workers
self._ls_functions = []
def __getattr__(self, name):
for worker in self.workers:
self._ls_functions.append(getattr(worker, name))
return self.fn_executor
def fn_executor(self, *args, **kwargs):
for fn in self._ls_functions:
fn(*args, **kwargs)
self._ls_functions = []
In this solution I make use of the __getattr__ function. If I call the log_value() function on the hive object (hive.log_value()) it looks first if it has the log_value attribute/function. If the attribute does not exist, it enters the __getattr__ function and executes the code. There, I iterate through the list of workers and collect the functions with the same name. I then return the function fn_executor, because otherwise I wouldn't be able to hand over the parameters with which the log_value() function was called on the hive object. Although this works fine, the issue is that you need to know the parameters and the types beforehand. Since we don't use inheritance we don't have the advantage of IntelliSense, because the functions are no members of the hive class. Makes sense.
So I wanted to mitigate that by adding functions as attributes during the __init__, which works.
Solution #2.1:
class Hive:
def __init__(self, trackers: List[AbstractWorker]):
self.workers = workers
self._ls_functions = []
for fn_name in dir(AbstractWorker):
if not (fn_name.startswith('__') or fn_name.startswith('_')):
setattr(self, fn_name, self.fn_wrapper(str(fn_name)))
def fn_wrapper(self, name):
def fn(*params, **kwargs):
return self.__getattr__(name)(*params, **kwargs)
return fn
def __getattr__(self, name):
for worker in self.workers:
self._ls_functions.append(getattr(worker, name))
return self.fn_executor
def fn_executor(self, *args, **kwargs):
for fn in self._ls_functions:
fn(*args, **kwargs)
self._ls_functions = []
In solution #2.1 I try to fetch all functions from the abstract base class with dir(AbstractWorker), removing dunder functions and "private" ones with the if and set the name of the functions as an attribute. Additionally, I assign a wrapper function (similar to partial or a decorator) that contains the __getattr__ function. During runtime the members are correctly set, but since IntelliSense relies on static code analysis, it is difficult to handle dynamic attribute assignment and as a result IntelliSense refuses to bring them up.
Now to the question:
What would be the best approach to create a wrapper/hive class that knows about the signature of the functions from the abstract base class but gets rid of the duplication of code shown in solution #1?

Patching in Python

I have a python file say
python_file_a.py
def load_content():
dir = "/down/model/"
model = Model(model_dir=dir)
return model
model = load_content()
def invoke(req):
return model.execute(req)
test_python_file_a.py
#patch("module.python_file_a.load_content")
#patch("module.python_file_a.model", Mock(spec=Model))
def test_invoke():
from module.python_file_a import model, invoke
model.execute = Mock(return_value="Some response")
invoke("some request")
This is still trying to load the actual model from the path "/down/model/" in the test. What is the correct way of patching so that the load_content function is mocked in the test?
Without knowing more about what your code does or how it's used it's hard to say exactly, but in this case the correct approach--and in many cases--is to not hard-code values as local variables in functions. Change your load_content() function to take an argument like:
def load_content(dirname):
...
or even give it a default value like
def load_content(dirname="/default/path"):
pass
For the test don't use the model instance instantiated at module level (arguably you should not be doing this in the first place, but again it depends on what you're trying to do).
Update: Upon closer inspect the problem really seems to stem from you instantiating a module-global instance at import time. Maybe try to avoid doing that and use lazy instantiation instead, like:
model = None
then if you really must write a function that accesses the global variable:
def invoke():
global model
if model is None:
model = load_content()
Alternatively you can use a PEP 562 module-level __getattr__ function.
Or write a class instead of putting everything at module-level.
class ModelInvoker:
def __init__(self, dirname='/path/to/content'):
self.dirname = dirname
#functools.cached_property
def model(self):
return load_content(self.dirname)
def invoke(self, req):
return model.execute(req)
Many other approaches to this depending on your use case. But finding some form of encapsulation is what you need if you want to be able to easily mock and replace parts of some code, and not execute code unnecessarily at import time.

Register a post_load hook after Schema defintion?

I'm trying to implement a Schema that deserializes into an object, where the object class is not known at Schema-definition time. I would have assumed that I can register a post_load function at runtime, but it appears that post_load works only for class methods.
It seems like I could make it work by either:
Updating Schema._hooks manually or by
Somehow creating a bound method at runtime and registering that.
Since both of these options are somewhat hack-ish, is there an official way to achieve the same result?
I don't think you need a metaclass.
Define a base schema with a post-load method that just needs the class.
class CustomSchema(Schema):
#post_load
def make_obj(self, data):
return self.OBJ_CLS(**data)
If the class is known at import time (not your use case), this allows you to factorize the instantiation by just providing the class. Nice already.
class PetSchema(CustomSchema):
OBJ_CLS = Pet
If the class is not known at import time, then it can be provided afterwards.
class PetSchema(CustomSchema):
pass
PetSchema.OBJ_CLS = Pet
If you need some more processing before instantiating, then you can override make_obj in any class, as you show in your answer.
class PetSchema(CustomSchema):
def make_obj(self, data):
data = my_func(data)
return Pet(**data)
More generally, this mechanism allows you to define hooks in a base schema. This is a good way to overcome a current limitation in marshmallow: the fact that multiple post_load methods can be executed in any order. Define a single post_load method in a base class with a hook for each processing step. (This contrived example doesn't really illustrate the point.)
class CustomSchema(Schema):
#post_load
def post_load_steps(self, data):
data = self.post_load_step_1(data)
data = self.post_load_step_2(data)
data = self.post_load_step_3(data)
return data
def post_load_step_1(self, data):
return data
def post_load_step_2(self, data):
return data
def post_load_step_3(self, data):
return data
Just if anyone else need this, I've solved it with a custom metaclass that pre-registers a post_load function whose actual implementation can be supplied at runtime:
from types import MethodType
from marshmallow import Schema, post_load
from marshmallow.schema import SchemaMeta
class MyCustomSchemaMeta(SchemaMeta):
def __init__(cls, *args, **kwargs):
super().__init__(*args, **kwargs)
def make_obj(*args, **kwargs):
raise NotImplementedError
# This post_load call registers the method with the Schema._hooks dict
cls.make_obj = post_load(make_obj)
class MyCustomSchema(Schema, metaclass=MyCustomSchemaMeta):
"""This is the base class that my schemas inherit."""
# The actual implementation of make_obj (and hence the class to deserialize to)
# can now be provided at runtime. The post_load call does not affect the schema
# anymore, but sets some parameters on the method.
MyCustomSchema.make_obj = MethodType(
post_load(lambda self, data: MyClass(**data)), MyCustomSchema
)

how to pass in dynamic data to decorators

I am trying to write a base crud controller class that does the
following:
class BaseCrudController:
model = ""
field_validation = {}
template_dir = ""
#expose(self.template_dir)
def new(self, *args, **kwargs)
....
#validate(self.field_validation, error_handler=new)
#expose()
def post(self, *args, **kwargs):
...
My intent is to have my controllers extend this base class, set the
model, field_validation, and template locations, and am ready to go.
Unfortunately, decorators (to my understanding), are interpreted when
the function is defined. Hence it won't have access to instance's
value. Is there a way to pass in dynamic data or values from the sub
class?
For example:
class AddressController(BaseCrudController):
model = Address
template_dir = "addressbook.templates.addresses"
When I try to load AddressController, it says "self is not defined". I am assuming that the base class is evaluating the decorator before the sub class is initialized.
Thanks,
Steve
Perhaps using a factory to create the class would be better than subclassing:
def CrudControllerFactory(model, field_validation, template_dir):
class BaseCrudController:
#expose(template_dir)
def new(self, *args, **kwargs)
....
#validate(field_validation, error_handler=new)
#expose()
def post(self, *args, **kwargs):
....
return BaseCrudController
Unfortunately, decorators (to my
understanding), are interpreted when
the function is defined. Hence it
won't have access to instance's value.
Is there a way to pass in dynamic data
or values from the sub class?
The template needs to be called with the name of the relevant attribute; the wrapper can then get that attribute's value dynamically. For example:
import functools
def expose(attname=None):
if attname:
def makewrapper(f):
#functools.wraps(f)
def wrapper(self, *a, **k):
attvalue = getattr(self, attname, None)
...use attvalue as needed...
return wrapper
return makewrapper
else:
...same but without the getattr...
Note that the complication is only because, judging from the code snippets in your Q, you want to allow the expose decorator to be used both with and without an argument (you could move the if attname guard to live within wrapper, but then you'd uselessly repeat the check at each call -- the code within wrapper may also need to be pretty different in the two cases, I imagine -- so, shoehorning two different control flows into one wrapper may be even more complicated). BTW, this is a dubious design decision, IMHO. But, it's quite separate from your actual Q about "dynamic data".
The point is, by using the attribute name as the argument, you empower your decorator to fetch the value dynamically "just in time" when it's needed. Think of it as "an extra level of indirection", that well-known panacea for all difficulties in programming!-)

What is the purpose of class methods?

I'm teaching myself Python and my most recent lesson was that Python is not Java, and so I've just spent a while turning all my Class methods into functions.
I now realise that I don't need to use Class methods for what I would done with static methods in Java, but now I'm not sure when I would use them. All the advice I can find about Python Class methods is along the lines of newbies like me should steer clear of them, and the standard documentation is at its most opaque when discussing them.
Does anyone have a good example of using a Class method in Python or at least can someone tell me when Class methods can be sensibly used?
Class methods are for when you need to have methods that aren't specific to any particular instance, but still involve the class in some way. The most interesting thing about them is that they can be overridden by subclasses, something that's simply not possible in Java's static methods or Python's module-level functions.
If you have a class MyClass, and a module-level function that operates on MyClass (factory, dependency injection stub, etc), make it a classmethod. Then it'll be available to subclasses.
Factory methods (alternative constructors) are indeed a classic example of class methods.
Basically, class methods are suitable anytime you would like to have a method which naturally fits into the namespace of the class, but is not associated with a particular instance of the class.
As an example, in the excellent unipath module:
Current directory
Path.cwd()
Return the actual current directory; e.g., Path("/tmp/my_temp_dir"). This is a class method.
.chdir()
Make self the current directory.
As the current directory is process wide, the cwd method has no particular instance with which it should be associated. However, changing the cwd to the directory of a given Path instance should indeed be an instance method.
Hmmm... as Path.cwd() does indeed return a Path instance, I guess it could be considered to be a factory method...
Think about it this way: normal methods are useful to hide the details of dispatch: you can type myobj.foo() without worrying about whether the foo() method is implemented by the myobj object's class or one of its parent classes. Class methods are exactly analogous to this, but with the class object instead: they let you call MyClass.foo() without having to worry about whether foo() is implemented specially by MyClass because it needed its own specialized version, or whether it is letting its parent class handle the call.
Class methods are essential when you are doing set-up or computation that precedes the creation of an actual instance, because until the instance exists you obviously cannot use the instance as the dispatch point for your method calls. A good example can be viewed in the SQLAlchemy source code; take a look at the dbapi() class method at the following link:
https://github.com/zzzeek/sqlalchemy/blob/ab6946769742602e40fb9ed9dde5f642885d1906/lib/sqlalchemy/dialects/mssql/pymssql.py#L47
You can see that the dbapi() method, which a database backend uses to import the vendor-specific database library it needs on-demand, is a class method because it needs to run before instances of a particular database connection start getting created — but that it cannot be a simple function or static function, because they want it to be able to call other, supporting methods that might similarly need to be written more specifically in subclasses than in their parent class. And if you dispatch to a function or static class, then you "forget" and lose the knowledge about which class is doing the initializing.
I recently wanted a very light-weight logging class that would output varying amounts of output depending on the logging level that could be programmatically set. But I didn't want to instantiate the class every time I wanted to output a debugging message or error or warning. But I also wanted to encapsulate the functioning of this logging facility and make it reusable without the declaration of any globals.
So I used class variables and the #classmethod decorator to achieve this.
With my simple Logging class, I could do the following:
Logger._level = Logger.DEBUG
Then, in my code, if I wanted to spit out a bunch of debugging information, I simply had to code
Logger.debug( "this is some annoying message I only want to see while debugging" )
Errors could be out put with
Logger.error( "Wow, something really awful happened." )
In the "production" environment, I can specify
Logger._level = Logger.ERROR
and now, only the error message will be output. The debug message will not be printed.
Here's my class:
class Logger :
''' Handles logging of debugging and error messages. '''
DEBUG = 5
INFO = 4
WARN = 3
ERROR = 2
FATAL = 1
_level = DEBUG
def __init__( self ) :
Logger._level = Logger.DEBUG
#classmethod
def isLevel( cls, level ) :
return cls._level >= level
#classmethod
def debug( cls, message ) :
if cls.isLevel( Logger.DEBUG ) :
print "DEBUG: " + message
#classmethod
def info( cls, message ) :
if cls.isLevel( Logger.INFO ) :
print "INFO : " + message
#classmethod
def warn( cls, message ) :
if cls.isLevel( Logger.WARN ) :
print "WARN : " + message
#classmethod
def error( cls, message ) :
if cls.isLevel( Logger.ERROR ) :
print "ERROR: " + message
#classmethod
def fatal( cls, message ) :
if cls.isLevel( Logger.FATAL ) :
print "FATAL: " + message
And some code that tests it just a bit:
def logAll() :
Logger.debug( "This is a Debug message." )
Logger.info ( "This is a Info message." )
Logger.warn ( "This is a Warn message." )
Logger.error( "This is a Error message." )
Logger.fatal( "This is a Fatal message." )
if __name__ == '__main__' :
print "Should see all DEBUG and higher"
Logger._level = Logger.DEBUG
logAll()
print "Should see all ERROR and higher"
Logger._level = Logger.ERROR
logAll()
Alternative constructors are the classic example.
It allows you to write generic class methods that you can use with any compatible class.
For example:
#classmethod
def get_name(cls):
print cls.name
class C:
name = "tester"
C.get_name = get_name
#call it:
C.get_name()
If you don't use #classmethod you can do it with self keyword but it needs an instance of Class:
def get_name(self):
print self.name
class C:
name = "tester"
C.get_name = get_name
#call it:
C().get_name() #<-note the its an instance of class C
When a user logs in on my website, a User() object is instantiated from the username and password.
If I need a user object without the user being there to log in (e.g. an admin user might want to delete another users account, so i need to instantiate that user and call its delete method):
I have class methods to grab the user object.
class User():
#lots of code
#...
# more code
#classmethod
def get_by_username(cls, username):
return cls.query(cls.username == username).get()
#classmethod
def get_by_auth_id(cls, auth_id):
return cls.query(cls.auth_id == auth_id).get()
I think the most clear answer is AmanKow's one. It boils down to how u want to organize your code. You can write everything as module level functions which are wrapped in the namespace of the module i.e
module.py (file 1)
---------
def f1() : pass
def f2() : pass
def f3() : pass
usage.py (file 2)
--------
from module import *
f1()
f2()
f3()
def f4():pass
def f5():pass
usage1.py (file 3)
-------------------
from usage import f4,f5
f4()
f5()
The above procedural code is not well organized, as you can see after only 3 modules it gets confusing, what is each method do ? You can use long descriptive names for functions(like in java) but still your code gets unmanageable very quick.
The object oriented way is to break down your code into manageable blocks i.e Classes & objects and functions can be associated with objects instances or with classes.
With class functions you gain another level of division in your code compared with module level functions.
So you can group related functions within a class to make them more specific to a task that you assigned to that class. For example you can create a file utility class :
class FileUtil ():
def copy(source,dest):pass
def move(source,dest):pass
def copyDir(source,dest):pass
def moveDir(source,dest):pass
//usage
FileUtil.copy("1.txt","2.txt")
FileUtil.moveDir("dir1","dir2")
This way is more flexible and more maintainable, you group functions together and its more obvious to what each function do. Also you prevent name conflicts, for example the function copy may exist in another imported module(for example network copy) that you use in your code, so when you use the full name FileUtil.copy() you remove the problem and both copy functions can be used side by side.
Honestly? I've never found a use for staticmethod or classmethod. I've yet to see an operation that can't be done using a global function or an instance method.
It would be different if python used private and protected members more like Java does. In Java, I need a static method to be able to access an instance's private members to do stuff. In Python, that's rarely necessary.
Usually, I see people using staticmethods and classmethods when all they really need to do is use python's module-level namespaces better.
I used to work with PHP and recently I was asking myself, whats going on with this classmethod? Python manual is very technical and very short in words so it wont help with understanding that feature. I was googling and googling and I found answer -> http://code.anjanesh.net/2007/12/python-classmethods.html.
If you are lazy to click it. My explanation is shorter and below. :)
in PHP (maybe not all of you know PHP, but this language is so straight forward that everybody should understand what I'm talking about) we have static variables like this:
class A
{
static protected $inner_var = null;
static public function echoInnerVar()
{
echo self::$inner_var."\n";
}
static public function setInnerVar($v)
{
self::$inner_var = $v;
}
}
class B extends A
{
}
A::setInnerVar(10);
B::setInnerVar(20);
A::echoInnerVar();
B::echoInnerVar();
The output will be in both cases 20.
However in python we can add #classmethod decorator and thus it is possible to have output 10 and 20 respectively. Example:
class A(object):
inner_var = 0
#classmethod
def setInnerVar(cls, value):
cls.inner_var = value
#classmethod
def echoInnerVar(cls):
print cls.inner_var
class B(A):
pass
A.setInnerVar(10)
B.setInnerVar(20)
A.echoInnerVar()
B.echoInnerVar()
Smart, ain't?
Class methods provide a "semantic sugar" (don't know if this term is widely used) - or "semantic convenience".
Example: you got a set of classes representing objects. You might want to have the class method all() or find() to write User.all() or User.find(firstname='Guido'). That could be done using module level functions of course...
if you are not a "programmer by training", this should help:
I think I have understood the technical explanations above and elsewhere on the net, but I was always left with a question "Nice, but why do I need it? What is a practical, use case?". and now life gave me a good example that clarified all:
I am using it to control the global-shared variable that is shared among instances of a class instantiated by multi-threading module. in humane language, I am running multiple agents that create examples for deep learning IN PARALLEL. (imagine multiple players playing ATARI game at the same time and each saving the results of their game to one common repository (the SHARED VARIABLE))
I instantiate the players/agents with the following code (in Main/Execution Code):
a3c_workers = [A3C_Worker(self.master_model, self.optimizer, i, self.env_name, self.model_dir) for i in range(multiprocessing.cpu_count())]
it creates as many players as there are processor cores on my comp
A3C_Worker - is a class that defines the agent
a3c_workers - is a list of the instances of that class (i.e. each instance is one player/agent)
now i want to know how many games have been played across all players/agents thus within the A3C_Worker definition I define the variable to be shared across all instances:
class A3C_Worker(threading.Thread):
global_shared_total_episodes_across_all_workers = 0
now as the workers finish their games they increase that count by 1 each for each game finished
at the end of my example generation i was closing the instances but the shared variable had assigned the total number of games played. so when I was re-running it again my initial total number of episodes was that of the previous total. but i needed that count to represent that value for each run individually
to fix that i specified :
class A3C_Worker(threading.Thread):
#classmethod
def reset(cls):
A3C_Worker.global_shared_total_episodes_across_all_workers = 0
than in the execution code i just call:
A3C_Worker.reset()
note that it is a call to the CLASS overall not any INSTANCE of it individually. thus it will set my counter to 0 for every new agent I initiate from now on.
using the usual method definition def play(self):, would require us to reset that counter for each instance individually, which would be more computationally demanding and difficult to track.
What just hit me, coming from Ruby, is that a so-called class method and a so-called instance method is just a function with semantic meaning applied to its first parameter, which is silently passed when the function is called as a method of an object (i.e. obj.meth()).
Normally that object must be an instance but the #classmethod method decorator changes the rules to pass a class. You can call a class method on an instance (it's just a function) - the first argument will be its class.
Because it's just a function, it can only be declared once in any given scope (i.e. class definition). If follows therefore, as a surprise to a Rubyist, that you can't have a class method and an instance method with the same name.
Consider this:
class Foo():
def foo(x):
print(x)
You can call foo on an instance
Foo().foo()
<__main__.Foo instance at 0x7f4dd3e3bc20>
But not on a class:
Foo.foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method foo() must be called with Foo instance as first argument (got nothing instead)
Now add #classmethod:
class Foo():
#classmethod
def foo(x):
print(x)
Calling on an instance now passes its class:
Foo().foo()
__main__.Foo
as does calling on a class:
Foo.foo()
__main__.Foo
It's only convention that dictates that we use self for that first argument on an instance method and cls on a class method. I used neither here to illustrate that it's just an argument. In Ruby, self is a keyword.
Contrast with Ruby:
class Foo
def foo()
puts "instance method #{self}"
end
def self.foo()
puts "class method #{self}"
end
end
Foo.foo()
class method Foo
Foo.new.foo()
instance method #<Foo:0x000000020fe018>
The Python class method is just a decorated function and you can use the same techniques to create your own decorators. A decorated method wraps the real method (in the case of #classmethod it passes the additional class argument). The underlying method is still there, hidden but still accessible.
footnote: I wrote this after a name clash between a class and instance method piqued my curiosity. I am far from a Python expert and would like comments if any of this is wrong.
This is an interesting topic. My take on it is that python classmethod operates like a singleton rather than a factory (which returns a produced an instance of a class). The reason it is a singleton is that there is a common object that is produced (the dictionary) but only once for the class but shared by all instances.
To illustrate this here is an example. Note that all instances have a reference to the single dictionary. This is not Factory pattern as I understand it. This is probably very unique to python.
class M():
#classmethod
def m(cls, arg):
print "arg was", getattr(cls, "arg" , None),
cls.arg = arg
print "arg is" , cls.arg
M.m(1) # prints arg was None arg is 1
M.m(2) # prints arg was 1 arg is 2
m1 = M()
m2 = M()
m1.m(3) # prints arg was 2 arg is 3
m2.m(4) # prints arg was 3 arg is 4 << this breaks the factory pattern theory.
M.m(5) # prints arg was 4 arg is 5
I was asking myself the same question few times. And even though the guys here tried hard to explain it, IMHO the best answer (and simplest) answer I have found is the description of the Class method in the Python Documentation.
There is also reference to the Static method. And in case someone already know instance methods (which I assume), this answer might be the final piece to put it all together...
Further and deeper elaboration on this topic can be found also in the documentation:
The standard type hierarchy (scroll down to Instance methods section)
#classmethod can be useful for easily instantiating objects of that class from outside resources. Consider the following:
import settings
class SomeClass:
#classmethod
def from_settings(cls):
return cls(settings=settings)
def __init__(self, settings=None):
if settings is not None:
self.x = settings['x']
self.y = settings['y']
Then in another file:
from some_package import SomeClass
inst = SomeClass.from_settings()
Accessing inst.x will give the same value as settings['x'].
A class defines a set of instances, of course. And the methods of a class work on the individual instances. The class methods (and variables) a place to hang other information that is related to the set of instances over all.
For example if your class defines a the set of students you might want class variables or methods which define things like the set of grade the students can be members of.
You can also use class methods to define tools for working on the entire set. For example Student.all_of_em() might return all the known students. Obviously if your set of instances have more structure than just a set you can provide class methods to know about that structure. Students.all_of_em(grade='juniors')
Techniques like this tend to lead to storing members of the set of instances into data structures that are rooted in class variables. You need to take care to avoid frustrating the garbage collection then.
Classes and Objects concepts are very useful in organizing things. It's true that all the operations that can be done by a method can also be done using a static function.
Just think of a scenario, to build a Students Databases System to maintain student details.
You need to have details about students, teachers and staff. You need to build functions to calculate fees, salary, marks, etc. Fees and marks are only applicable for students, salary is only applicable for staff and teachers. So if you create separate classes for every type of people, the code will be organized.

Categories