Related
I have a main class that has a ton of different functions in it. It's getting hard to manage. I'd like to be able to separate those functions into a separate file, but I'm finding it hard to come up with a good way to do so.
Here's what I've done so far:
File main.py
import separate
class MainClass(object):
self.global_var_1 = ...
self.global_var_2 = ...
def func_1(self, x, y):
...
def func_2(self, z):
...
# tons of similar functions, and then the ones I moved out:
def long_func_1(self, a, b):
return separate.long_func_1(self, a, b)
File separate.py
def long_func_1(obj, a, b):
if obj.global_var_1:
...
obj.func_2(z)
...
return ...
# Lots of other similar functions that use info from MainClass
I do this because if I do:
obj_1 = MainClass()
I want to be able to do:
obj_1.long_func_1(a, b)
instead of:
separate.long_func_1(obj_1, a, b)
I know this seems kind of nit-picky, but I want just about all of the code to start with obj_1., so there isn't confusion.
Is there a better solution that what I'm currently doing? The only issues that I have with my current setup are:
I have to change arguments for both instances of the function
It seems needlessly repetitive
I know this has been asked a couple of times, but I couldn't quite understand the previous answers and/or I don't think the solution quite represents what I'm shooting for. I'm still pretty new to Python, so I'm having a tough time figuring this out.
Here is how I do it:
Class (or group of) is actually a full module. You don't have to do it this way, but if you're splitting a class on multiple files I think this is 'cleanest' (opinion).
The definition is in __init__.py, methods are split into files by a meaningful grouping.
A method file is just a regular Python file with functions, except you can't forget 'self' as a first argument. You can have auxiliary methods here, both taking self and not.
Methods are imported directly into the class definition.
Suppose my class is some fitting GUI (this is actually what I did this for first time). So my file hierarchy may look something like
mymodule/
__init__.py
_plotstuff.py
_fitstuff.py
_datastuff.py
So plot stuff will have plotting methods, fit stuff contains fitting methods, and data stuff contains methods for loading and handling of data - you get the point. By convention I mark the files with a _ to indicate these really aren't meant to be imported directly anywhere outside the module. So _plotsuff.py for example may look like:
def plot(self,x,y):
#body
def clear(self):
#body
etc. Now the important thing is file __init__.py:
class Fitter(object):
def __init__(self,whatever):
self.field1 = 0
self.field2 = whatever
# Imported methods
from ._plotstuff import plot, clear
from ._fitstuff import fit
from ._datastuff import load
# static methods need to be set
from ._static_example import something
something = staticmethod(something)
# Some more small functions
def printHi(self):
print("Hello world")
Tom Sawyer mentions PEP-8 recommends putting all imports at the top, so you may wish to put them before __init__, but I prefer it this way. I have to say, my Flake8 checker does not complain, so likely this is PEP-8 compliant.
Note the from ... import ... is particularly useful to hide some 'helper' functions to your methods you don't want accessible through objects of the class. I usually also place the custom exceptions for the class in the different files, but import them directly so they can be accessed as Fitter.myexception.
If this module is in your path then you can access your class with
from mymodule import Fitter
f = Fitter()
f.load('somefile') # Imported method
f.plot() # Imported method
It is not completely intuitive, but not too difficult either. The short version for your specific problem was you were close - just move the import into the class, and use
from separate import long_func_1
and don't forget your self!
How to use super addendum
super() is a useful nifty function allowing parent method access in a simple and readable manner from the child object. These kind of classes are big to begin with, so inheritance not always make sense, but if it does come up:
For methods defined in the class itself, within __init__.py, you can use super() normally, as is.
If you define you method in another module (which is kind of the point here), you can't use super as is since the function is not defined in the context of your cell, and will fail. The way to handle this is to use the self argument, and add the context yourself:
def print_super(self):
print('Super is:', super(type(self), self))
Note you cannot omit the second argument, since out of context super does not bind the object method (which you usually want for calls like super(...).__init__()).
If this is something you want to do in many methods in different modules, you may want to provide a super method in the __init__.py file for use:
def MySuper(self):
return super()
usable by self in all methods.
I use the approach I found here. It shows many different approaches, but if you scroll down to the end, the preferred method is to basically go the opposite direction of #Martin Pieter's suggestion which is have a base class that inherits other classes with your methods in those classes.
So the folder structure is something like:
_DataStore/
__init__.py
DataStore.py
_DataStore.py
So your base class would be:
File DataStore.py
import _DataStore
class DataStore(_DataStore.Mixin): # Could inherit many more mixins
def __init__(self):
self._a = 1
self._b = 2
self._c = 3
def small_method(self):
return self._a
Then your Mixin class:
File _DataStore.py
class Mixin:
def big_method(self):
return self._b
def huge_method(self):
return self._c
Your separate methods would be located in other appropriately named files, and in this example it is just _DataStore.
I am interested to hear what others think about this approach. I showed it to someone at work and they were scared by it, but it seemed to be a clean and easy way to separate a class into multiple files.
Here is an implementation of Martijn Pieters's comment to use subclasses:
File main.py
from separate import BaseClass
class MainClass(BaseClass):
def long_func_1(self, a, b):
if self.global_var_1:
...
self.func_2(z)
...
return ...
# Lots of other similar functions that use info from BaseClass
File separate.py
class BaseClass(object):
# You almost always want to initialize instance variables in the `__init__` method.
def __init__(self):
self.global_var_1 = ...
self.global_var_2 = ...
def func_1(self, x, y):
...
def func_2(self, z):
...
# tons of similar functions, and then the ones I moved out:
#
# Why are there "tons" of _similar_ functions?
# Remember that functions can be defined to take a
# variable number of/optional arguments, lists/tuples
# as arguments, dicts as arguments, etc.
from main import MainClass
m = MainClass()
m.func_1(1, 2)
....
Using "new" style classes (I'm in python 3.2) is there a way to split a class over multiple files? I've got a large class (which really should be a single class from an object-oriented design perspective, considering coupling, etc, but it'd be nice to split over a few files just for ease of editing the class.
If your problem really is just working with a large class in an editor, the first solution I'd actually look for is a better way to break down the problem. The second solution would be a better editor, preferably one with code folding.
That said, there are a couple of ways you might break up a class into multiple files. Python lets you use a folder as a module by putting an __init__.py in it, which can then import things from other files. We'll use this capability in each solution. Make a folder called, say, bigclass first.
In the folder put the various .py files that will eventually comprise your class. Each should contain functions and variable definitions for the eventual class, not classes. In __init__.py in the same folder write the following to join them all together.
class Bigclass(object):
from classdef1 import foo, bar, baz, quux
from classdef2 import thing1, thing2
from classdef3 import magic, moremagic
# unfortunately, "from classdefn import *" is an error or warning
num = 42 # add more members here if you like
This has the advantage that you end up with a single class derived directly from object, which will look nice in your inheritance graphs.
You could use multiple inheritance to combine the various parts of your class. In your individual modules you would write a class definition for Bigclass with parts of the class. Then in your __init__.py write:
import classdef1, classdef2, classdef3
class Bigclass(classdef1.Bigclass, classdef2.Bigclass, classdef3.Bigclass):
num = 42 # add more members if desired
If the multiple inheritance becomes an issue, you can use single inheritance: just have each class inherit from another one in chain fashion. Assuming you don't define anything in more than one class, the order doesn't matter. For example, classdef2.py would be like:
import classdef1
class Bigclass(classdef1.Bigclass):
# more member defs here
classdef3 would import Bigclass from classdef2 and add to it, and so on. Your __init__.py would just import the last one:
from classdef42 import Bigclass
I'd generally prefer #1 because it's more explicit about what members you're importing from which files but any of these solutions could work for you.
To use the class in any of these scenarios you can just import it, using the folder name as the module name: from bigclass import Bigclass
You can do this with decorators like so:
class Car(object):
def start(self):
print 'Car has started'
def extends(klass):
def decorator(func):
setattr(klass, func.__name__, func)
return func
return decorator
#this can go in a different module/file
#extends(Car)
def do_start(self):
self.start()
#so can this
car = Car()
car.do_start()
#=> Car has started
Class definitions containing hundreds of lines do occur "in the wild" (I have seen some in popular open-source Python-based frameworks), but I believe that if you ponder what the methods are doing, it will be possible to reduce the length of most classes to a manageable point. Some examples:
Look for places where mostly the same code occurs more than once. Break that code out into its own method and call it from each place with arguments.
"Private" methods that do not use any of the object state can be brought out of the class as stand-alone functions.
Methods that should be called only under certain conditions may indicate a need to place those methods in a subclass.
To directly address your question, it is possible to split up the definition of a class. One way is to "monkey-patch" the class by defining it and then adding outside functions to it as methods. Another is to use the built-in type function to create the class "by hand", supplying its name, any base classes, and its methods and attributes in a dictionary. But I do not recommend doing this just because the definition would be long otherwise. That sort of cure is worse than the disease in my opinion.
I've previously toyed around with something similar. My usecase was a class hierarchy of nodes in an abstract syntax tree, and then I wanted to put all e.g. prettyprinting functions in a separate prettyprint.py file but still have them as methods in the classes.
One thing I tried was to use a decorator that puts the decorated function as an attribute on a specified class. In my case this would mean that prettyprint.py contains lots of def prettyprint(self) all decorated with different #inclass(...)
A problem with this is that one must make sure that the sub files are always imported, and that they depend on the main class, which makes for a circular dependency, which may be messy.
def inclass(kls):
"""
Decorator that adds the decorated function
as a method in specified class
"""
def _(func):
setattr(kls,func.__name__, func)
return func
return _
## exampe usage
class C:
def __init__(self, d):
self.d = d
# this would be in a separate file.
#inclass(C)
def meth(self, a):
"""Some method"""
print "attribute: %s - argument: %s" % (self.d, a)
i = C(10)
print i.meth.__doc__
i.meth(20)
I've not used it, but this package called partial claims to add support for partial classes.
It seems like there's a few other ways you could implement this yourself as well.
You could implement separate parts of the class as mixins in seperate files, then import them all somewhere and subclass them.
Alternatively, you could implement each of the methods of your class somewhere then in a central file import them and assign them as attributes on a class, to create the whole object. Like so:
a.py:
def AFunc( self, something ):
# Do something
pass
b.py:
def BFunc( self, something ):
# Do something else
pass
c.py:
import a, b
class C:
AFunc = a.AFunc
BFunc = b.BFunc
You could even go so far as to automate this process if you really wanted - loop through all the functions provided by modules a and b and then add them as attributes on C. Though that might be total overkill.
There might be other (possibly better) ways to go about it, but those are the 2 that popped into mind.
I would like to add that the pythonic way of doing this is through multiple inheritance, not necessarily using mixins. Instance attributes can be added using super().__init__(*args, **kwargs) in __init__ calls to pass arguments to baseclasses (see ‘super considered super’ presentation by Raymond Hettinger 1). This also enables dependency injection and kind of forces you to think about organization of base classes (it works best if only one baseclass sets an attribute in __init__ and all classes using the attribute inherit from it).
This does usually require you having control over the base classes (or they being written for this pattern).
Another option is using descriptors returning functions through __get__ to add functionality to classes in a decoupled way.
You could also look at __init_subclass__ to add e.g. methods to classes during class generation (i think added in python 3.6, but check)
First I'd like to say that something this complicated it probably not a good idea just to make finding your place in the class easier - it would be best to add comments, highlight sections etc. However, I see two ways you could do this:
Write the class in several files, then read them in as text, concatenate them and exec the resulting string.
Create a separate class in each file, then inherit them all into a master class as mixins. However, if you're subclassing another class already this could lead to MRO problems. You could get around this by creating a metaclass for your master class which manually resolves the MRO, but this could get messy.
The easiest would be the first option.
First off, I don't see how splitting the class into multiple files makes editing any easier. A decent IDE should be able to find any method easily whether in one file or multiple; if you're not using a decent IDE, splitting the class means the maintainer has to guess which file a given method is in, which sounds harder rather than easier.
More fundamentally, this class - so large that you want a special language feature just to support its weight - sounds fundamentally broken. How many lines of code are we talking about? Almost certainly, it would be a better idea to do one of:
Refactor duplicated code into fewer, more general primitives
Define a base class and extend it with subclasses as Karoly Horvath suggests in comments (this is the closest thing to the 'partial classes' that you're asking for that I would endorse)
Define a few separate classes to encapsulate different parts of this
class's functionality, and compose this class of instances of those
smaller ones.
I met the same situation - I want to slipt my class to 2 files.
the reason is that - I want part 1 for GUI layout, only layout
and another file keeps all function.
like c#'s Partial class. one for XAML and another one for functions.
Using "new" style classes (I'm in python 3.2) is there a way to split a class over multiple files? I've got a large class (which really should be a single class from an object-oriented design perspective, considering coupling, etc, but it'd be nice to split over a few files just for ease of editing the class.
If your problem really is just working with a large class in an editor, the first solution I'd actually look for is a better way to break down the problem. The second solution would be a better editor, preferably one with code folding.
That said, there are a couple of ways you might break up a class into multiple files. Python lets you use a folder as a module by putting an __init__.py in it, which can then import things from other files. We'll use this capability in each solution. Make a folder called, say, bigclass first.
In the folder put the various .py files that will eventually comprise your class. Each should contain functions and variable definitions for the eventual class, not classes. In __init__.py in the same folder write the following to join them all together.
class Bigclass(object):
from classdef1 import foo, bar, baz, quux
from classdef2 import thing1, thing2
from classdef3 import magic, moremagic
# unfortunately, "from classdefn import *" is an error or warning
num = 42 # add more members here if you like
This has the advantage that you end up with a single class derived directly from object, which will look nice in your inheritance graphs.
You could use multiple inheritance to combine the various parts of your class. In your individual modules you would write a class definition for Bigclass with parts of the class. Then in your __init__.py write:
import classdef1, classdef2, classdef3
class Bigclass(classdef1.Bigclass, classdef2.Bigclass, classdef3.Bigclass):
num = 42 # add more members if desired
If the multiple inheritance becomes an issue, you can use single inheritance: just have each class inherit from another one in chain fashion. Assuming you don't define anything in more than one class, the order doesn't matter. For example, classdef2.py would be like:
import classdef1
class Bigclass(classdef1.Bigclass):
# more member defs here
classdef3 would import Bigclass from classdef2 and add to it, and so on. Your __init__.py would just import the last one:
from classdef42 import Bigclass
I'd generally prefer #1 because it's more explicit about what members you're importing from which files but any of these solutions could work for you.
To use the class in any of these scenarios you can just import it, using the folder name as the module name: from bigclass import Bigclass
You can do this with decorators like so:
class Car(object):
def start(self):
print 'Car has started'
def extends(klass):
def decorator(func):
setattr(klass, func.__name__, func)
return func
return decorator
#this can go in a different module/file
#extends(Car)
def do_start(self):
self.start()
#so can this
car = Car()
car.do_start()
#=> Car has started
Class definitions containing hundreds of lines do occur "in the wild" (I have seen some in popular open-source Python-based frameworks), but I believe that if you ponder what the methods are doing, it will be possible to reduce the length of most classes to a manageable point. Some examples:
Look for places where mostly the same code occurs more than once. Break that code out into its own method and call it from each place with arguments.
"Private" methods that do not use any of the object state can be brought out of the class as stand-alone functions.
Methods that should be called only under certain conditions may indicate a need to place those methods in a subclass.
To directly address your question, it is possible to split up the definition of a class. One way is to "monkey-patch" the class by defining it and then adding outside functions to it as methods. Another is to use the built-in type function to create the class "by hand", supplying its name, any base classes, and its methods and attributes in a dictionary. But I do not recommend doing this just because the definition would be long otherwise. That sort of cure is worse than the disease in my opinion.
I've previously toyed around with something similar. My usecase was a class hierarchy of nodes in an abstract syntax tree, and then I wanted to put all e.g. prettyprinting functions in a separate prettyprint.py file but still have them as methods in the classes.
One thing I tried was to use a decorator that puts the decorated function as an attribute on a specified class. In my case this would mean that prettyprint.py contains lots of def prettyprint(self) all decorated with different #inclass(...)
A problem with this is that one must make sure that the sub files are always imported, and that they depend on the main class, which makes for a circular dependency, which may be messy.
def inclass(kls):
"""
Decorator that adds the decorated function
as a method in specified class
"""
def _(func):
setattr(kls,func.__name__, func)
return func
return _
## exampe usage
class C:
def __init__(self, d):
self.d = d
# this would be in a separate file.
#inclass(C)
def meth(self, a):
"""Some method"""
print "attribute: %s - argument: %s" % (self.d, a)
i = C(10)
print i.meth.__doc__
i.meth(20)
I've not used it, but this package called partial claims to add support for partial classes.
It seems like there's a few other ways you could implement this yourself as well.
You could implement separate parts of the class as mixins in seperate files, then import them all somewhere and subclass them.
Alternatively, you could implement each of the methods of your class somewhere then in a central file import them and assign them as attributes on a class, to create the whole object. Like so:
a.py:
def AFunc( self, something ):
# Do something
pass
b.py:
def BFunc( self, something ):
# Do something else
pass
c.py:
import a, b
class C:
AFunc = a.AFunc
BFunc = b.BFunc
You could even go so far as to automate this process if you really wanted - loop through all the functions provided by modules a and b and then add them as attributes on C. Though that might be total overkill.
There might be other (possibly better) ways to go about it, but those are the 2 that popped into mind.
I would like to add that the pythonic way of doing this is through multiple inheritance, not necessarily using mixins. Instance attributes can be added using super().__init__(*args, **kwargs) in __init__ calls to pass arguments to baseclasses (see ‘super considered super’ presentation by Raymond Hettinger 1). This also enables dependency injection and kind of forces you to think about organization of base classes (it works best if only one baseclass sets an attribute in __init__ and all classes using the attribute inherit from it).
This does usually require you having control over the base classes (or they being written for this pattern).
Another option is using descriptors returning functions through __get__ to add functionality to classes in a decoupled way.
You could also look at __init_subclass__ to add e.g. methods to classes during class generation (i think added in python 3.6, but check)
First I'd like to say that something this complicated it probably not a good idea just to make finding your place in the class easier - it would be best to add comments, highlight sections etc. However, I see two ways you could do this:
Write the class in several files, then read them in as text, concatenate them and exec the resulting string.
Create a separate class in each file, then inherit them all into a master class as mixins. However, if you're subclassing another class already this could lead to MRO problems. You could get around this by creating a metaclass for your master class which manually resolves the MRO, but this could get messy.
The easiest would be the first option.
First off, I don't see how splitting the class into multiple files makes editing any easier. A decent IDE should be able to find any method easily whether in one file or multiple; if you're not using a decent IDE, splitting the class means the maintainer has to guess which file a given method is in, which sounds harder rather than easier.
More fundamentally, this class - so large that you want a special language feature just to support its weight - sounds fundamentally broken. How many lines of code are we talking about? Almost certainly, it would be a better idea to do one of:
Refactor duplicated code into fewer, more general primitives
Define a base class and extend it with subclasses as Karoly Horvath suggests in comments (this is the closest thing to the 'partial classes' that you're asking for that I would endorse)
Define a few separate classes to encapsulate different parts of this
class's functionality, and compose this class of instances of those
smaller ones.
I met the same situation - I want to slipt my class to 2 files.
the reason is that - I want part 1 for GUI layout, only layout
and another file keeps all function.
like c#'s Partial class. one for XAML and another one for functions.
Not sure if this is a dupe or not. Here it goes.
I need to write some Python code that looks like:
class TestClass:
def test_case(self):
def get_categories(self):
return [“abc”,”bcd”]
# do the test here
and then have a test engine class that scans all these test classes, loads all the test_case functions and for each invokes get_categories to find out if the test belongs t the group of interest for the specific run.
The problem is that get_categories is not seen as an attribute of test_case, and even if I manually assign it
class TestClass:
def test_case(self):
def get_categories(self):
return [“abc”,”bcd”]
# do the test here
test_case.get_categories = get_categories
this is only going to happen when test_case first runs, too late for me.
The reason why this function can’t go on the class (or at least why I want it to be also available at the per-function level) is that a TestClass can have multiple test cases.
Since this is an already existing testing infrastructure, and the categories mechanism works (other than the categories-on-function scenario, which is of lesser importance), a rewrite is not in the plans.
Language tricks dearly appreciated.
Nested functions don't become attributes any more than any other assignment.
I suspect your test infrastructure is doing some severely weird things if this isn't supported (and uses old-style classes!), but you could just do this:
class TestClass:
def test_case(self):
# ...
def _get_categories(self):
return [...]
test_case.get_categories = _get_categories
del _get_categories
Class bodies are executable code like any other block.
What you need is nested classes. Functions aren't made to do what you are trying to do, so you have to move up a notch. Function attributes are mainly used as markup, whereas classes can have anything you want.
class TestClass(object):
class TestCase(object):
#classmethod
def get_categories(cls):
return ['abc', 'efg']
Note that I used #classmethod so that you could use it without instantiating TestCase(); modify if you want to do test_case = TestCase().
I'm teaching myself Python and my most recent lesson was that Python is not Java, and so I've just spent a while turning all my Class methods into functions.
I now realise that I don't need to use Class methods for what I would done with static methods in Java, but now I'm not sure when I would use them. All the advice I can find about Python Class methods is along the lines of newbies like me should steer clear of them, and the standard documentation is at its most opaque when discussing them.
Does anyone have a good example of using a Class method in Python or at least can someone tell me when Class methods can be sensibly used?
Class methods are for when you need to have methods that aren't specific to any particular instance, but still involve the class in some way. The most interesting thing about them is that they can be overridden by subclasses, something that's simply not possible in Java's static methods or Python's module-level functions.
If you have a class MyClass, and a module-level function that operates on MyClass (factory, dependency injection stub, etc), make it a classmethod. Then it'll be available to subclasses.
Factory methods (alternative constructors) are indeed a classic example of class methods.
Basically, class methods are suitable anytime you would like to have a method which naturally fits into the namespace of the class, but is not associated with a particular instance of the class.
As an example, in the excellent unipath module:
Current directory
Path.cwd()
Return the actual current directory; e.g., Path("/tmp/my_temp_dir"). This is a class method.
.chdir()
Make self the current directory.
As the current directory is process wide, the cwd method has no particular instance with which it should be associated. However, changing the cwd to the directory of a given Path instance should indeed be an instance method.
Hmmm... as Path.cwd() does indeed return a Path instance, I guess it could be considered to be a factory method...
Think about it this way: normal methods are useful to hide the details of dispatch: you can type myobj.foo() without worrying about whether the foo() method is implemented by the myobj object's class or one of its parent classes. Class methods are exactly analogous to this, but with the class object instead: they let you call MyClass.foo() without having to worry about whether foo() is implemented specially by MyClass because it needed its own specialized version, or whether it is letting its parent class handle the call.
Class methods are essential when you are doing set-up or computation that precedes the creation of an actual instance, because until the instance exists you obviously cannot use the instance as the dispatch point for your method calls. A good example can be viewed in the SQLAlchemy source code; take a look at the dbapi() class method at the following link:
https://github.com/zzzeek/sqlalchemy/blob/ab6946769742602e40fb9ed9dde5f642885d1906/lib/sqlalchemy/dialects/mssql/pymssql.py#L47
You can see that the dbapi() method, which a database backend uses to import the vendor-specific database library it needs on-demand, is a class method because it needs to run before instances of a particular database connection start getting created — but that it cannot be a simple function or static function, because they want it to be able to call other, supporting methods that might similarly need to be written more specifically in subclasses than in their parent class. And if you dispatch to a function or static class, then you "forget" and lose the knowledge about which class is doing the initializing.
I recently wanted a very light-weight logging class that would output varying amounts of output depending on the logging level that could be programmatically set. But I didn't want to instantiate the class every time I wanted to output a debugging message or error or warning. But I also wanted to encapsulate the functioning of this logging facility and make it reusable without the declaration of any globals.
So I used class variables and the #classmethod decorator to achieve this.
With my simple Logging class, I could do the following:
Logger._level = Logger.DEBUG
Then, in my code, if I wanted to spit out a bunch of debugging information, I simply had to code
Logger.debug( "this is some annoying message I only want to see while debugging" )
Errors could be out put with
Logger.error( "Wow, something really awful happened." )
In the "production" environment, I can specify
Logger._level = Logger.ERROR
and now, only the error message will be output. The debug message will not be printed.
Here's my class:
class Logger :
''' Handles logging of debugging and error messages. '''
DEBUG = 5
INFO = 4
WARN = 3
ERROR = 2
FATAL = 1
_level = DEBUG
def __init__( self ) :
Logger._level = Logger.DEBUG
#classmethod
def isLevel( cls, level ) :
return cls._level >= level
#classmethod
def debug( cls, message ) :
if cls.isLevel( Logger.DEBUG ) :
print "DEBUG: " + message
#classmethod
def info( cls, message ) :
if cls.isLevel( Logger.INFO ) :
print "INFO : " + message
#classmethod
def warn( cls, message ) :
if cls.isLevel( Logger.WARN ) :
print "WARN : " + message
#classmethod
def error( cls, message ) :
if cls.isLevel( Logger.ERROR ) :
print "ERROR: " + message
#classmethod
def fatal( cls, message ) :
if cls.isLevel( Logger.FATAL ) :
print "FATAL: " + message
And some code that tests it just a bit:
def logAll() :
Logger.debug( "This is a Debug message." )
Logger.info ( "This is a Info message." )
Logger.warn ( "This is a Warn message." )
Logger.error( "This is a Error message." )
Logger.fatal( "This is a Fatal message." )
if __name__ == '__main__' :
print "Should see all DEBUG and higher"
Logger._level = Logger.DEBUG
logAll()
print "Should see all ERROR and higher"
Logger._level = Logger.ERROR
logAll()
Alternative constructors are the classic example.
It allows you to write generic class methods that you can use with any compatible class.
For example:
#classmethod
def get_name(cls):
print cls.name
class C:
name = "tester"
C.get_name = get_name
#call it:
C.get_name()
If you don't use #classmethod you can do it with self keyword but it needs an instance of Class:
def get_name(self):
print self.name
class C:
name = "tester"
C.get_name = get_name
#call it:
C().get_name() #<-note the its an instance of class C
When a user logs in on my website, a User() object is instantiated from the username and password.
If I need a user object without the user being there to log in (e.g. an admin user might want to delete another users account, so i need to instantiate that user and call its delete method):
I have class methods to grab the user object.
class User():
#lots of code
#...
# more code
#classmethod
def get_by_username(cls, username):
return cls.query(cls.username == username).get()
#classmethod
def get_by_auth_id(cls, auth_id):
return cls.query(cls.auth_id == auth_id).get()
I think the most clear answer is AmanKow's one. It boils down to how u want to organize your code. You can write everything as module level functions which are wrapped in the namespace of the module i.e
module.py (file 1)
---------
def f1() : pass
def f2() : pass
def f3() : pass
usage.py (file 2)
--------
from module import *
f1()
f2()
f3()
def f4():pass
def f5():pass
usage1.py (file 3)
-------------------
from usage import f4,f5
f4()
f5()
The above procedural code is not well organized, as you can see after only 3 modules it gets confusing, what is each method do ? You can use long descriptive names for functions(like in java) but still your code gets unmanageable very quick.
The object oriented way is to break down your code into manageable blocks i.e Classes & objects and functions can be associated with objects instances or with classes.
With class functions you gain another level of division in your code compared with module level functions.
So you can group related functions within a class to make them more specific to a task that you assigned to that class. For example you can create a file utility class :
class FileUtil ():
def copy(source,dest):pass
def move(source,dest):pass
def copyDir(source,dest):pass
def moveDir(source,dest):pass
//usage
FileUtil.copy("1.txt","2.txt")
FileUtil.moveDir("dir1","dir2")
This way is more flexible and more maintainable, you group functions together and its more obvious to what each function do. Also you prevent name conflicts, for example the function copy may exist in another imported module(for example network copy) that you use in your code, so when you use the full name FileUtil.copy() you remove the problem and both copy functions can be used side by side.
Honestly? I've never found a use for staticmethod or classmethod. I've yet to see an operation that can't be done using a global function or an instance method.
It would be different if python used private and protected members more like Java does. In Java, I need a static method to be able to access an instance's private members to do stuff. In Python, that's rarely necessary.
Usually, I see people using staticmethods and classmethods when all they really need to do is use python's module-level namespaces better.
I used to work with PHP and recently I was asking myself, whats going on with this classmethod? Python manual is very technical and very short in words so it wont help with understanding that feature. I was googling and googling and I found answer -> http://code.anjanesh.net/2007/12/python-classmethods.html.
If you are lazy to click it. My explanation is shorter and below. :)
in PHP (maybe not all of you know PHP, but this language is so straight forward that everybody should understand what I'm talking about) we have static variables like this:
class A
{
static protected $inner_var = null;
static public function echoInnerVar()
{
echo self::$inner_var."\n";
}
static public function setInnerVar($v)
{
self::$inner_var = $v;
}
}
class B extends A
{
}
A::setInnerVar(10);
B::setInnerVar(20);
A::echoInnerVar();
B::echoInnerVar();
The output will be in both cases 20.
However in python we can add #classmethod decorator and thus it is possible to have output 10 and 20 respectively. Example:
class A(object):
inner_var = 0
#classmethod
def setInnerVar(cls, value):
cls.inner_var = value
#classmethod
def echoInnerVar(cls):
print cls.inner_var
class B(A):
pass
A.setInnerVar(10)
B.setInnerVar(20)
A.echoInnerVar()
B.echoInnerVar()
Smart, ain't?
Class methods provide a "semantic sugar" (don't know if this term is widely used) - or "semantic convenience".
Example: you got a set of classes representing objects. You might want to have the class method all() or find() to write User.all() or User.find(firstname='Guido'). That could be done using module level functions of course...
if you are not a "programmer by training", this should help:
I think I have understood the technical explanations above and elsewhere on the net, but I was always left with a question "Nice, but why do I need it? What is a practical, use case?". and now life gave me a good example that clarified all:
I am using it to control the global-shared variable that is shared among instances of a class instantiated by multi-threading module. in humane language, I am running multiple agents that create examples for deep learning IN PARALLEL. (imagine multiple players playing ATARI game at the same time and each saving the results of their game to one common repository (the SHARED VARIABLE))
I instantiate the players/agents with the following code (in Main/Execution Code):
a3c_workers = [A3C_Worker(self.master_model, self.optimizer, i, self.env_name, self.model_dir) for i in range(multiprocessing.cpu_count())]
it creates as many players as there are processor cores on my comp
A3C_Worker - is a class that defines the agent
a3c_workers - is a list of the instances of that class (i.e. each instance is one player/agent)
now i want to know how many games have been played across all players/agents thus within the A3C_Worker definition I define the variable to be shared across all instances:
class A3C_Worker(threading.Thread):
global_shared_total_episodes_across_all_workers = 0
now as the workers finish their games they increase that count by 1 each for each game finished
at the end of my example generation i was closing the instances but the shared variable had assigned the total number of games played. so when I was re-running it again my initial total number of episodes was that of the previous total. but i needed that count to represent that value for each run individually
to fix that i specified :
class A3C_Worker(threading.Thread):
#classmethod
def reset(cls):
A3C_Worker.global_shared_total_episodes_across_all_workers = 0
than in the execution code i just call:
A3C_Worker.reset()
note that it is a call to the CLASS overall not any INSTANCE of it individually. thus it will set my counter to 0 for every new agent I initiate from now on.
using the usual method definition def play(self):, would require us to reset that counter for each instance individually, which would be more computationally demanding and difficult to track.
What just hit me, coming from Ruby, is that a so-called class method and a so-called instance method is just a function with semantic meaning applied to its first parameter, which is silently passed when the function is called as a method of an object (i.e. obj.meth()).
Normally that object must be an instance but the #classmethod method decorator changes the rules to pass a class. You can call a class method on an instance (it's just a function) - the first argument will be its class.
Because it's just a function, it can only be declared once in any given scope (i.e. class definition). If follows therefore, as a surprise to a Rubyist, that you can't have a class method and an instance method with the same name.
Consider this:
class Foo():
def foo(x):
print(x)
You can call foo on an instance
Foo().foo()
<__main__.Foo instance at 0x7f4dd3e3bc20>
But not on a class:
Foo.foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method foo() must be called with Foo instance as first argument (got nothing instead)
Now add #classmethod:
class Foo():
#classmethod
def foo(x):
print(x)
Calling on an instance now passes its class:
Foo().foo()
__main__.Foo
as does calling on a class:
Foo.foo()
__main__.Foo
It's only convention that dictates that we use self for that first argument on an instance method and cls on a class method. I used neither here to illustrate that it's just an argument. In Ruby, self is a keyword.
Contrast with Ruby:
class Foo
def foo()
puts "instance method #{self}"
end
def self.foo()
puts "class method #{self}"
end
end
Foo.foo()
class method Foo
Foo.new.foo()
instance method #<Foo:0x000000020fe018>
The Python class method is just a decorated function and you can use the same techniques to create your own decorators. A decorated method wraps the real method (in the case of #classmethod it passes the additional class argument). The underlying method is still there, hidden but still accessible.
footnote: I wrote this after a name clash between a class and instance method piqued my curiosity. I am far from a Python expert and would like comments if any of this is wrong.
This is an interesting topic. My take on it is that python classmethod operates like a singleton rather than a factory (which returns a produced an instance of a class). The reason it is a singleton is that there is a common object that is produced (the dictionary) but only once for the class but shared by all instances.
To illustrate this here is an example. Note that all instances have a reference to the single dictionary. This is not Factory pattern as I understand it. This is probably very unique to python.
class M():
#classmethod
def m(cls, arg):
print "arg was", getattr(cls, "arg" , None),
cls.arg = arg
print "arg is" , cls.arg
M.m(1) # prints arg was None arg is 1
M.m(2) # prints arg was 1 arg is 2
m1 = M()
m2 = M()
m1.m(3) # prints arg was 2 arg is 3
m2.m(4) # prints arg was 3 arg is 4 << this breaks the factory pattern theory.
M.m(5) # prints arg was 4 arg is 5
I was asking myself the same question few times. And even though the guys here tried hard to explain it, IMHO the best answer (and simplest) answer I have found is the description of the Class method in the Python Documentation.
There is also reference to the Static method. And in case someone already know instance methods (which I assume), this answer might be the final piece to put it all together...
Further and deeper elaboration on this topic can be found also in the documentation:
The standard type hierarchy (scroll down to Instance methods section)
#classmethod can be useful for easily instantiating objects of that class from outside resources. Consider the following:
import settings
class SomeClass:
#classmethod
def from_settings(cls):
return cls(settings=settings)
def __init__(self, settings=None):
if settings is not None:
self.x = settings['x']
self.y = settings['y']
Then in another file:
from some_package import SomeClass
inst = SomeClass.from_settings()
Accessing inst.x will give the same value as settings['x'].
A class defines a set of instances, of course. And the methods of a class work on the individual instances. The class methods (and variables) a place to hang other information that is related to the set of instances over all.
For example if your class defines a the set of students you might want class variables or methods which define things like the set of grade the students can be members of.
You can also use class methods to define tools for working on the entire set. For example Student.all_of_em() might return all the known students. Obviously if your set of instances have more structure than just a set you can provide class methods to know about that structure. Students.all_of_em(grade='juniors')
Techniques like this tend to lead to storing members of the set of instances into data structures that are rooted in class variables. You need to take care to avoid frustrating the garbage collection then.
Classes and Objects concepts are very useful in organizing things. It's true that all the operations that can be done by a method can also be done using a static function.
Just think of a scenario, to build a Students Databases System to maintain student details.
You need to have details about students, teachers and staff. You need to build functions to calculate fees, salary, marks, etc. Fees and marks are only applicable for students, salary is only applicable for staff and teachers. So if you create separate classes for every type of people, the code will be organized.